2024-04-26T18:08:25.041 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T18:08:25.041 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917 --name leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi --verbose --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} --owner scheduled_leonidus@teuthology -- /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917/orig.config.yaml 2024-04-26T18:08:25.108 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T18:08:25.224 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917 branch: wip-lusov-quiescer-fixes description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: null first_in_suite: false job_id: '7674917' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer-fixes ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer-fixes sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 9496 sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 sleep_before_teardown: 0 subset: 200/111111 suite: fs suite_branch: wip-lusov-quiescer-fixes suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi084.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAk6YE0WOWx27lNtOjSzOSA8fEtqcuBRxcVtS1ubIrxqNuvH7X6Xe/TZ/8J22Jv+aS2LTWhU+d1nyAo57BIj2+E= smithi103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLIXSwe8Zz7Dc6p3X2fwbNESx22Vvi0mPDHU4O1WxbgWc8ZbLIT96jp6RCrs55weXJeYsbpJEb1/3oFT9TtSAY4= smithi174.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCwX/fcLLQmVf3Acy60cSR78kI0WeUTc+bNI2YI6kKzOz5JMhXJcdRmYXHc3qdkKIeU9U19rLSaAk/umLustLZ0= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_17:24:23 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-26T18:08:25.224 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa; will attempt to use it 2024-04-26T18:08:25.225 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/qa/tasks 2024-04-26T18:08:25.225 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T18:08:25.228 INFO:teuthology.task.internal:Checking packages... 2024-04-26T18:08:25.270 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1' 2024-04-26T18:08:25.270 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T18:08:25.271 INFO:teuthology.packaging:ref: None 2024-04-26T18:08:25.271 INFO:teuthology.packaging:tag: None 2024-04-26T18:08:25.271 INFO:teuthology.packaging:branch: wip-lusov-quiescer-fixes 2024-04-26T18:08:25.271 INFO:teuthology.packaging:sha1: bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:08:25.271 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer-fixes 2024-04-26T18:08:25.492 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3267.gbc94a2a9 2024-04-26T18:08:25.493 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T18:08:25.580 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T18:08:25.580 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T18:08:25.642 INFO:teuthology.task.internal:Saving configuration 2024-04-26T18:08:25.663 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T18:08:25.669 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T18:08:25.695 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi084.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 18:03:06.810158', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAk6YE0WOWx27lNtOjSzOSA8fEtqcuBRxcVtS1ubIrxqNuvH7X6Xe/TZ/8J22Jv+aS2LTWhU+d1nyAo57BIj2+E='} 2024-04-26T18:08:25.717 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi103.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 18:03:06.808722', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLIXSwe8Zz7Dc6p3X2fwbNESx22Vvi0mPDHU4O1WxbgWc8ZbLIT96jp6RCrs55weXJeYsbpJEb1/3oFT9TtSAY4='} 2024-04-26T18:08:25.740 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi174.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 18:03:06.803875', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCwX/fcLLQmVf3Acy60cSR78kI0WeUTc+bNI2YI6kKzOz5JMhXJcdRmYXHc3qdkKIeU9U19rLSaAk/umLustLZ0='} 2024-04-26T18:08:25.740 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T18:08:25.746 INFO:teuthology.task.internal:roles: ubuntu@smithi084.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-26T18:08:25.746 INFO:teuthology.task.internal:roles: ubuntu@smithi103.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-26T18:08:25.746 INFO:teuthology.task.internal:roles: ubuntu@smithi174.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-26T18:08:25.746 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T18:08:25.849 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7652c270d0>, signals=[15]) 2024-04-26T18:08:25.849 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T18:08:25.855 INFO:teuthology.task.internal:Opening connections... 2024-04-26T18:08:25.856 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi084.front.sepia.ceph.com 2024-04-26T18:08:25.858 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T18:08:25.949 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T18:08:25.949 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T18:08:26.025 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi174.front.sepia.ceph.com 2024-04-26T18:08:26.026 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T18:08:26.105 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T18:08:26.112 DEBUG:teuthology.orchestra.run.smithi084:> uname -m 2024-04-26T18:08:26.130 INFO:teuthology.orchestra.run.smithi084.stdout:x86_64 2024-04-26T18:08:26.131 DEBUG:teuthology.orchestra.run.smithi084:> cat /etc/os-release 2024-04-26T18:08:26.187 INFO:teuthology.orchestra.run.smithi084.stdout:NAME="CentOS Stream" 2024-04-26T18:08:26.187 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION="9" 2024-04-26T18:08:26.187 INFO:teuthology.orchestra.run.smithi084.stdout:ID="centos" 2024-04-26T18:08:26.188 INFO:teuthology.orchestra.run.smithi084.stdout:ID_LIKE="rhel fedora" 2024-04-26T18:08:26.188 INFO:teuthology.orchestra.run.smithi084.stdout:VERSION_ID="9" 2024-04-26T18:08:26.188 INFO:teuthology.orchestra.run.smithi084.stdout:PLATFORM_ID="platform:el9" 2024-04-26T18:08:26.188 INFO:teuthology.orchestra.run.smithi084.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T18:08:26.188 INFO:teuthology.orchestra.run.smithi084.stdout:ANSI_COLOR="0;31" 2024-04-26T18:08:26.188 INFO:teuthology.orchestra.run.smithi084.stdout:LOGO="fedora-logo-icon" 2024-04-26T18:08:26.188 INFO:teuthology.orchestra.run.smithi084.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T18:08:26.188 INFO:teuthology.orchestra.run.smithi084.stdout:HOME_URL="https://centos.org/" 2024-04-26T18:08:26.188 INFO:teuthology.orchestra.run.smithi084.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T18:08:26.188 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T18:08:26.188 INFO:teuthology.orchestra.run.smithi084.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T18:08:26.189 INFO:teuthology.lock.ops:Updating smithi084.front.sepia.ceph.com on lock server 2024-04-26T18:08:26.213 DEBUG:teuthology.orchestra.run.smithi103:> uname -m 2024-04-26T18:08:26.231 INFO:teuthology.orchestra.run.smithi103.stdout:x86_64 2024-04-26T18:08:26.232 DEBUG:teuthology.orchestra.run.smithi103:> cat /etc/os-release 2024-04-26T18:08:26.289 INFO:teuthology.orchestra.run.smithi103.stdout:NAME="CentOS Stream" 2024-04-26T18:08:26.289 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION="9" 2024-04-26T18:08:26.289 INFO:teuthology.orchestra.run.smithi103.stdout:ID="centos" 2024-04-26T18:08:26.290 INFO:teuthology.orchestra.run.smithi103.stdout:ID_LIKE="rhel fedora" 2024-04-26T18:08:26.290 INFO:teuthology.orchestra.run.smithi103.stdout:VERSION_ID="9" 2024-04-26T18:08:26.290 INFO:teuthology.orchestra.run.smithi103.stdout:PLATFORM_ID="platform:el9" 2024-04-26T18:08:26.290 INFO:teuthology.orchestra.run.smithi103.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T18:08:26.290 INFO:teuthology.orchestra.run.smithi103.stdout:ANSI_COLOR="0;31" 2024-04-26T18:08:26.290 INFO:teuthology.orchestra.run.smithi103.stdout:LOGO="fedora-logo-icon" 2024-04-26T18:08:26.290 INFO:teuthology.orchestra.run.smithi103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T18:08:26.290 INFO:teuthology.orchestra.run.smithi103.stdout:HOME_URL="https://centos.org/" 2024-04-26T18:08:26.290 INFO:teuthology.orchestra.run.smithi103.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T18:08:26.290 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T18:08:26.290 INFO:teuthology.orchestra.run.smithi103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T18:08:26.291 INFO:teuthology.lock.ops:Updating smithi103.front.sepia.ceph.com on lock server 2024-04-26T18:08:26.312 DEBUG:teuthology.orchestra.run.smithi174:> uname -m 2024-04-26T18:08:26.330 INFO:teuthology.orchestra.run.smithi174.stdout:x86_64 2024-04-26T18:08:26.331 DEBUG:teuthology.orchestra.run.smithi174:> cat /etc/os-release 2024-04-26T18:08:26.387 INFO:teuthology.orchestra.run.smithi174.stdout:NAME="CentOS Stream" 2024-04-26T18:08:26.388 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION="9" 2024-04-26T18:08:26.388 INFO:teuthology.orchestra.run.smithi174.stdout:ID="centos" 2024-04-26T18:08:26.388 INFO:teuthology.orchestra.run.smithi174.stdout:ID_LIKE="rhel fedora" 2024-04-26T18:08:26.388 INFO:teuthology.orchestra.run.smithi174.stdout:VERSION_ID="9" 2024-04-26T18:08:26.388 INFO:teuthology.orchestra.run.smithi174.stdout:PLATFORM_ID="platform:el9" 2024-04-26T18:08:26.388 INFO:teuthology.orchestra.run.smithi174.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T18:08:26.388 INFO:teuthology.orchestra.run.smithi174.stdout:ANSI_COLOR="0;31" 2024-04-26T18:08:26.388 INFO:teuthology.orchestra.run.smithi174.stdout:LOGO="fedora-logo-icon" 2024-04-26T18:08:26.388 INFO:teuthology.orchestra.run.smithi174.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T18:08:26.388 INFO:teuthology.orchestra.run.smithi174.stdout:HOME_URL="https://centos.org/" 2024-04-26T18:08:26.388 INFO:teuthology.orchestra.run.smithi174.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T18:08:26.388 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T18:08:26.388 INFO:teuthology.orchestra.run.smithi174.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T18:08:26.389 INFO:teuthology.lock.ops:Updating smithi174.front.sepia.ceph.com on lock server 2024-04-26T18:08:26.412 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T18:08:26.423 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T18:08:26.429 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T18:08:26.429 DEBUG:teuthology.orchestra.run.smithi084:> test '!' -e /home/ubuntu/cephtest 2024-04-26T18:08:26.432 DEBUG:teuthology.orchestra.run.smithi103:> test '!' -e /home/ubuntu/cephtest 2024-04-26T18:08:26.434 DEBUG:teuthology.orchestra.run.smithi174:> test '!' -e /home/ubuntu/cephtest 2024-04-26T18:08:26.450 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T18:08:26.457 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T18:08:26.457 DEBUG:teuthology.orchestra.run.smithi084:> test -z $(ls -A /var/lib/ceph) 2024-04-26T18:08:26.489 DEBUG:teuthology.orchestra.run.smithi103:> test -z $(ls -A /var/lib/ceph) 2024-04-26T18:08:26.492 DEBUG:teuthology.orchestra.run.smithi174:> test -z $(ls -A /var/lib/ceph) 2024-04-26T18:08:26.570 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T18:08:26.648 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T18:08:26.663 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-26T18:08:26.663 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-26T18:08:26.663 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-26T18:08:26.664 DEBUG:teuthology.orchestra.run.smithi084:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T18:08:26.680 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:08:26.681 DEBUG:teuthology.orchestra.run.smithi084:> uname -r 2024-04-26T18:08:26.739 INFO:teuthology.orchestra.run.smithi084.stdout:5.14.0-437.el9.x86_64 2024-04-26T18:08:26.739 INFO:teuthology.task.kernel:Running kernel on smithi084: 5.14.0-437.el9.x86_64 2024-04-26T18:08:26.739 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum install -y kernel 2024-04-26T18:08:30.016 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.0 MB 00:01 2024-04-26T18:08:34.058 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 6.4 MB/s | 19 MB 00:02 2024-04-26T18:08:39.216 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 6.2 MB 00:01 2024-04-26T18:08:40.834 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 36 kB/s | 16 kB 00:00 2024-04-26T18:08:42.618 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-26T18:08:47.718 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-26T18:08:49.590 INFO:teuthology.orchestra.run.smithi084.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T18:08:49.590 INFO:teuthology.orchestra.run.smithi084.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T18:08:49.678 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-04-26T18:08:49.679 INFO:teuthology.orchestra.run.smithi084.stdout:Nothing to do. 2024-04-26T18:08:49.679 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-04-26T18:08:49.791 DEBUG:teuthology.orchestra.run.smithi084:> echo no | sudo yum reinstall kernel || true 2024-04-26T18:08:50.292 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 06:08:47 PM UTC. 2024-04-26T18:08:50.411 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-04-26T18:08:50.421 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-04-26T18:08:50.421 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-04-26T18:08:50.421 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-04-26T18:08:50.421 INFO:teuthology.orchestra.run.smithi084.stdout:Reinstalling: 2024-04-26T18:08:50.421 INFO:teuthology.orchestra.run.smithi084.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T18:08:50.421 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:08:50.421 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-04-26T18:08:50.421 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-04-26T18:08:50.421 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:08:50.422 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 255 k 2024-04-26T18:08:50.422 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 0 2024-04-26T18:08:50.422 INFO:teuthology.orchestra.run.smithi084.stderr:Operation aborted. 2024-04-26T18:08:50.490 INFO:teuthology.orchestra.run.smithi084.stdout:Is this ok [y/N]: 2024-04-26T18:08:50.491 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum reinstall -y kernel || true 2024-04-26T18:08:50.982 INFO:teuthology.orchestra.run.smithi084.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 06:08:47 PM UTC. 2024-04-26T18:08:51.097 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-04-26T18:08:51.097 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-04-26T18:08:51.098 INFO:teuthology.orchestra.run.smithi084.stdout: Package Architecture Version Repository Size 2024-04-26T18:08:51.098 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-04-26T18:08:51.098 INFO:teuthology.orchestra.run.smithi084.stdout:Reinstalling: 2024-04-26T18:08:51.098 INFO:teuthology.orchestra.run.smithi084.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T18:08:51.098 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:08:51.098 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-04-26T18:08:51.098 INFO:teuthology.orchestra.run.smithi084.stdout:================================================================================ 2024-04-26T18:08:51.098 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:08:51.098 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 255 k 2024-04-26T18:08:51.098 INFO:teuthology.orchestra.run.smithi084.stdout:Installed size: 0 2024-04-26T18:08:51.098 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-04-26T18:08:51.486 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.4 MB/s | 255 kB 00:00 2024-04-26T18:08:51.486 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-04-26T18:08:51.486 INFO:teuthology.orchestra.run.smithi084.stdout:Total 657 kB/s | 255 kB 00:00 2024-04-26T18:08:51.506 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-04-26T18:08:51.698 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-04-26T18:08:51.698 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-04-26T18:08:51.703 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-04-26T18:08:51.703 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-04-26T18:08:51.864 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-04-26T18:08:51.946 INFO:teuthology.orchestra.run.smithi084.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T18:08:51.946 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T18:08:52.601 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T18:08:52.601 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T18:08:53.278 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T18:08:53.279 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:08:53.279 INFO:teuthology.orchestra.run.smithi084.stdout:Reinstalled: 2024-04-26T18:08:53.279 INFO:teuthology.orchestra.run.smithi084.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T18:08:53.279 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:08:53.279 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-04-26T18:08:53.466 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T18:08:53.513 INFO:teuthology.orchestra.run.smithi084.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T18:08:53.513 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T18:08:53.513 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T18:08:53.514 DEBUG:teuthology.parallel:result is None 2024-04-26T18:08:53.514 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T18:08:53.523 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T18:08:53.523 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T18:08:53.525 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T18:08:53.528 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T18:08:53.545 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T18:08:53.552 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T18:08:53.558 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T18:08:53.558 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T18:08:53.580 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T18:08:53.586 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T18:08:53.752 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T18:08:53.759 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T18:08:53.760 DEBUG:teuthology.orchestra.run.smithi103:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T18:08:53.776 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:08:53.777 DEBUG:teuthology.orchestra.run.smithi174:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T18:08:53.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:08:53.794 DEBUG:teuthology.orchestra.run.smithi084:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T18:08:53.796 DEBUG:teuthology.orchestra.run.smithi103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T18:08:53.819 DEBUG:teuthology.orchestra.run.smithi174:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T18:08:53.839 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T18:08:53.853 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T18:08:53.853 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T18:08:53.887 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T18:08:53.889 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T18:08:53.900 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T18:08:53.902 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T18:08:53.909 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T18:08:53.909 DEBUG:teuthology.orchestra.run.smithi084:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T18:08:53.917 DEBUG:teuthology.orchestra.run.smithi103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T18:08:53.934 DEBUG:teuthology.orchestra.run.smithi174:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T18:08:53.962 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T18:08:53.971 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T18:08:53.972 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T18:08:53.976 DEBUG:teuthology.orchestra.run.smithi103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T18:08:54.000 DEBUG:teuthology.orchestra.run.smithi174:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T18:08:54.020 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T18:08:54.067 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T18:08:54.112 DEBUG:teuthology.orchestra.run.smithi084:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T18:08:54.190 DEBUG:teuthology.orchestra.run.smithi084:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T18:08:54.231 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:08:54.231 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T18:08:54.294 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T18:08:54.334 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T18:08:54.384 DEBUG:teuthology.orchestra.run.smithi103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T18:08:54.465 DEBUG:teuthology.orchestra.run.smithi103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T18:08:54.510 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T18:08:54.510 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T18:08:54.575 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T18:08:54.616 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T18:08:54.677 DEBUG:teuthology.orchestra.run.smithi174:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T18:08:54.759 DEBUG:teuthology.orchestra.run.smithi174:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T18:08:54.802 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T18:08:54.803 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T18:08:54.869 DEBUG:teuthology.orchestra.run.smithi084:> sudo service rsyslog restart 2024-04-26T18:08:54.871 DEBUG:teuthology.orchestra.run.smithi103:> sudo service rsyslog restart 2024-04-26T18:08:54.874 DEBUG:teuthology.orchestra.run.smithi174:> sudo service rsyslog restart 2024-04-26T18:08:54.902 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T18:08:54.911 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T18:08:54.960 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T18:08:55.413 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T18:08:55.421 INFO:teuthology.task.internal:Starting timer... 2024-04-26T18:08:55.421 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T18:08:55.431 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T18:08:55.439 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T18:08:55.440 DEBUG:teuthology.orchestra.run.smithi084:> sudo service auditd rotate 2024-04-26T18:08:55.517 INFO:teuthology.orchestra.run.smithi084.stdout:Rotating logs: 2024-04-26T18:08:55.519 DEBUG:teuthology.orchestra.run.smithi103:> sudo service auditd rotate 2024-04-26T18:08:55.574 INFO:teuthology.orchestra.run.smithi103.stdout:Rotating logs: 2024-04-26T18:08:55.576 DEBUG:teuthology.orchestra.run.smithi174:> sudo service auditd rotate 2024-04-26T18:08:55.668 INFO:teuthology.orchestra.run.smithi174.stdout:Rotating logs: 2024-04-26T18:08:55.670 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T18:08:55.670 DEBUG:teuthology.orchestra.run.smithi084:> /usr/sbin/getenforce 2024-04-26T18:08:55.697 INFO:teuthology.orchestra.run.smithi084.stdout:Permissive 2024-04-26T18:08:55.697 DEBUG:teuthology.orchestra.run.smithi103:> /usr/sbin/getenforce 2024-04-26T18:08:55.725 INFO:teuthology.orchestra.run.smithi103.stdout:Permissive 2024-04-26T18:08:55.725 DEBUG:teuthology.orchestra.run.smithi174:> /usr/sbin/getenforce 2024-04-26T18:08:55.765 INFO:teuthology.orchestra.run.smithi174.stdout:Permissive 2024-04-26T18:08:55.765 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi084.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi103.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi174.front.sepia.ceph.com': 'permissive'} 2024-04-26T18:08:55.765 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T18:08:55.792 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:08:55.793 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T18:08:55.819 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:08:55.820 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T18:08:55.847 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:08:55.848 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T18:08:55.848 DEBUG:teuthology.orchestra.run.smithi084:> sudo /usr/sbin/setenforce permissive 2024-04-26T18:08:55.873 DEBUG:teuthology.orchestra.run.smithi103:> sudo /usr/sbin/setenforce permissive 2024-04-26T18:08:55.899 DEBUG:teuthology.orchestra.run.smithi174:> sudo /usr/sbin/setenforce permissive 2024-04-26T18:08:55.933 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T18:08:55.943 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-26T18:08:55.951 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-26T18:08:56.068 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T18:08:56.086 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T18:08:56.088 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi084.front.sepia.ceph.com,smithi103.front.sepia.ceph.com,smithi174.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T18:13:02.191 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi084.front.sepia.ceph.com'), Remote(name='ubuntu@smithi103.front.sepia.ceph.com'), Remote(name='ubuntu@smithi174.front.sepia.ceph.com')] 2024-04-26T18:13:02.193 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-04-26T18:13:02.194 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi084.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T18:13:02.270 DEBUG:teuthology.orchestra.run.smithi084:> true 2024-04-26T18:13:02.303 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi084.front.sepia.ceph.com' 2024-04-26T18:13:02.303 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-26T18:13:02.304 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T18:13:02.377 DEBUG:teuthology.orchestra.run.smithi103:> true 2024-04-26T18:13:02.453 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi103.front.sepia.ceph.com' 2024-04-26T18:13:02.453 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-04-26T18:13:02.454 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi174.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T18:13:02.527 DEBUG:teuthology.orchestra.run.smithi174:> true 2024-04-26T18:13:02.609 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi174.front.sepia.ceph.com' 2024-04-26T18:13:02.610 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T18:13:02.622 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T18:13:02.623 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T18:13:02.623 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T18:13:02.626 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T18:13:02.626 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T18:13:02.628 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T18:13:02.628 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T18:13:02.653 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T18:13:02.654 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T18:13:02.666 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T18:13:02.668 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T18:13:02.678 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T18:13:02.689 INFO:teuthology.orchestra.run.smithi103.stderr:sudo: ntpd: command not found 2024-04-26T18:13:02.691 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T18:13:02.691 INFO:teuthology.orchestra.run.smithi084.stderr:sudo: ntpd: command not found 2024-04-26T18:13:02.700 INFO:teuthology.orchestra.run.smithi103.stdout:506 Cannot talk to daemon 2024-04-26T18:13:02.702 INFO:teuthology.orchestra.run.smithi084.stdout:506 Cannot talk to daemon 2024-04-26T18:13:02.713 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T18:13:02.714 INFO:teuthology.orchestra.run.smithi174.stderr:sudo: ntpd: command not found 2024-04-26T18:13:02.715 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T18:13:02.724 INFO:teuthology.orchestra.run.smithi174.stdout:506 Cannot talk to daemon 2024-04-26T18:13:02.726 INFO:teuthology.orchestra.run.smithi103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T18:13:02.728 INFO:teuthology.orchestra.run.smithi084.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T18:13:02.737 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T18:13:02.750 INFO:teuthology.orchestra.run.smithi174.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T18:13:02.781 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-04-26T18:13:02.784 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T18:13:02.785 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-04-26T18:13:02.785 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:13:02.785 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:13:02.785 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:13:02.785 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:13:02.786 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-04-26T18:13:02.788 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T18:13:02.788 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-04-26T18:13:02.788 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:13:02.788 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:13:02.788 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:13:02.788 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:13:02.805 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-04-26T18:13:02.809 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T18:13:02.809 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-04-26T18:13:02.809 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:13:02.809 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:13:02.809 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:13:02.809 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:13:02.810 INFO:teuthology.run_tasks:Running task install... 2024-04-26T18:13:02.818 DEBUG:teuthology.task.install:project ceph 2024-04-26T18:13:02.819 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1'}} 2024-04-26T18:13:02.819 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1'} 2024-04-26T18:13:02.819 INFO:teuthology.task.install:Using flavor: default 2024-04-26T18:13:02.825 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T18:13:02.825 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-26T18:13:02.825 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'tag': None, 'wait_for_package': False} 2024-04-26T18:13:02.826 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:13:02.828 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'tag': None, 'wait_for_package': False} 2024-04-26T18:13:02.828 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:13:02.829 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'tag': None, 'wait_for_package': False} 2024-04-26T18:13:02.829 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:13:02.975 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/ 2024-04-26T18:13:02.975 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3267.gbc94a2a9 2024-04-26T18:13:03.045 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/ 2024-04-26T18:13:03.045 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3267.gbc94a2a9 2024-04-26T18:13:03.049 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/ 2024-04-26T18:13:03.049 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3267.gbc94a2a9 2024-04-26T18:13:03.147 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T18:13:03.147 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T18:13:03.147 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T18:13:03.171 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T18:13:03.172 DEBUG:teuthology.orchestra.run.smithi103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T18:13:03.212 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T18:13:03.212 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:13:03.212 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T18:13:03.219 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T18:13:03.219 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T18:13:03.219 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T18:13:03.236 DEBUG:teuthology.orchestra.run.smithi103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T18:13:03.237 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T18:13:03.237 DEBUG:teuthology.orchestra.run.smithi084:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T18:13:03.244 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-26T18:13:03.244 DEBUG:teuthology.orchestra.run.smithi174:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T18:13:03.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T18:13:03.309 DEBUG:teuthology.orchestra.run.smithi103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T18:13:03.309 DEBUG:teuthology.orchestra.run.smithi174:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T18:13:03.371 INFO:teuthology.orchestra.run.smithi103.stdout:check_obsoletes = 1 2024-04-26T18:13:03.373 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum clean all 2024-04-26T18:13:03.376 DEBUG:teuthology.orchestra.run.smithi084:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T18:13:03.383 DEBUG:teuthology.orchestra.run.smithi174:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T18:13:03.439 INFO:teuthology.orchestra.run.smithi084.stdout:check_obsoletes = 1 2024-04-26T18:13:03.441 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum clean all 2024-04-26T18:13:03.445 INFO:teuthology.orchestra.run.smithi174.stdout:check_obsoletes = 1 2024-04-26T18:13:03.446 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum clean all 2024-04-26T18:13:03.598 INFO:teuthology.orchestra.run.smithi103.stdout:48 files removed 2024-04-26T18:13:03.625 DEBUG:teuthology.orchestra.run.smithi103:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T18:13:03.665 INFO:teuthology.orchestra.run.smithi084.stdout:48 files removed 2024-04-26T18:13:03.671 INFO:teuthology.orchestra.run.smithi174.stdout:48 files removed 2024-04-26T18:13:03.692 DEBUG:teuthology.orchestra.run.smithi084:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T18:13:03.699 DEBUG:teuthology.orchestra.run.smithi174:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-26T18:13:04.302 INFO:teuthology.orchestra.run.smithi103.stdout:ceph packages for x86_64 201 kB/s | 84 kB 00:00 2024-04-26T18:13:04.317 INFO:teuthology.orchestra.run.smithi174.stdout:ceph packages for x86_64 237 kB/s | 84 kB 00:00 2024-04-26T18:13:04.334 INFO:teuthology.orchestra.run.smithi084.stdout:ceph packages for x86_64 220 kB/s | 84 kB 00:00 2024-04-26T18:13:04.668 INFO:teuthology.orchestra.run.smithi084.stdout:ceph noarch packages 40 kB/s | 12 kB 00:00 2024-04-26T18:13:04.676 INFO:teuthology.orchestra.run.smithi174.stdout:ceph noarch packages 37 kB/s | 12 kB 00:00 2024-04-26T18:13:04.728 INFO:teuthology.orchestra.run.smithi103.stdout:ceph noarch packages 31 kB/s | 12 kB 00:00 2024-04-26T18:13:05.060 INFO:teuthology.orchestra.run.smithi084.stdout:ceph source packages 5.3 kB/s | 1.9 kB 00:00 2024-04-26T18:13:05.170 INFO:teuthology.orchestra.run.smithi103.stdout:ceph source packages 4.7 kB/s | 1.9 kB 00:00 2024-04-26T18:13:05.185 INFO:teuthology.orchestra.run.smithi174.stdout:ceph source packages 4.0 kB/s | 1.9 kB 00:00 2024-04-26T18:13:05.619 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-26T18:13:05.718 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-26T18:13:05.772 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.0 MB 00:00 2024-04-26T18:13:07.936 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-26T18:13:07.940 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-26T18:13:10.112 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 19 MB 00:03 2024-04-26T18:13:12.195 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 6.2 MB 00:00 2024-04-26T18:13:13.383 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.2 MB 00:01 2024-04-26T18:13:13.621 INFO:teuthology.orchestra.run.smithi174.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-04-26T18:13:14.545 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - CRB 7.1 MB/s | 6.2 MB 00:00 2024-04-26T18:13:15.100 INFO:teuthology.orchestra.run.smithi174.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-26T18:13:15.177 INFO:teuthology.orchestra.run.smithi103.stdout:CentOS Stream 9 - Extras packages 26 kB/s | 16 kB 00:00 2024-04-26T18:13:16.160 INFO:teuthology.orchestra.run.smithi103.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-26T18:13:16.254 INFO:teuthology.orchestra.run.smithi084.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-04-26T18:13:17.264 INFO:teuthology.orchestra.run.smithi084.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-04-26T18:13:20.247 INFO:teuthology.orchestra.run.smithi174.stdout:lab-extras 45 kB/s | 1.7 kB 00:00 2024-04-26T18:13:21.277 INFO:teuthology.orchestra.run.smithi103.stdout:lab-extras 8.9 kB/s | 1.7 kB 00:00 2024-04-26T18:13:22.271 INFO:teuthology.orchestra.run.smithi084.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-26T18:13:22.408 INFO:teuthology.orchestra.run.smithi174.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T18:13:22.409 INFO:teuthology.orchestra.run.smithi174.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T18:13:22.420 INFO:teuthology.orchestra.run.smithi174.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T18:13:22.421 INFO:teuthology.orchestra.run.smithi174.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T18:13:22.422 INFO:teuthology.orchestra.run.smithi174.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T18:13:22.423 INFO:teuthology.orchestra.run.smithi174.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T18:13:22.424 INFO:teuthology.orchestra.run.smithi174.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T18:13:22.427 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T18:13:22.427 INFO:teuthology.orchestra.run.smithi174.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T18:13:22.489 INFO:teuthology.orchestra.run.smithi174.stdout:Dependencies resolved. 2024-04-26T18:13:22.515 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================= 2024-04-26T18:13:22.515 INFO:teuthology.orchestra.run.smithi174.stdout: Package Arch Version Repository Size 2024-04-26T18:13:22.515 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================= 2024-04-26T18:13:22.515 INFO:teuthology.orchestra.run.smithi174.stdout:Installing: 2024-04-26T18:13:22.516 INFO:teuthology.orchestra.run.smithi174.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T18:13:22.516 INFO:teuthology.orchestra.run.smithi174.stdout: ceph x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 6.6 k 2024-04-26T18:13:22.516 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T18:13:22.516 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 890 k 2024-04-26T18:13:22.516 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 145 k 2024-04-26T18:13:22.516 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 1.5 M 2024-04-26T18:13:22.516 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 142 k 2024-04-26T18:13:22.516 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 3.5 M 2024-04-26T18:13:22.516 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 7.4 M 2024-04-26T18:13:22.516 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 49 k 2024-04-26T18:13:22.516 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 11 M 2024-04-26T18:13:22.516 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 49 M 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 273 k 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 762 k 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 224 k 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 27 k 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 18 k 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 32 k 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 746 k 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 127 k 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T18:13:22.517 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T18:13:22.518 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T18:13:22.518 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T18:13:22.518 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T18:13:22.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 162 k 2024-04-26T18:13:22.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 320 k 2024-04-26T18:13:22.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 298 k 2024-04-26T18:13:22.518 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 99 k 2024-04-26T18:13:22.518 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 85 k 2024-04-26T18:13:22.518 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T18:13:22.518 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 170 k 2024-04-26T18:13:22.518 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T18:13:22.518 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrading: 2024-04-26T18:13:22.518 INFO:teuthology.orchestra.run.smithi174.stdout: librados2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.4 M 2024-04-26T18:13:22.519 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T18:13:22.519 INFO:teuthology.orchestra.run.smithi174.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T18:13:22.519 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T18:13:22.519 INFO:teuthology.orchestra.run.smithi174.stdout:Installing dependencies: 2024-04-26T18:13:22.519 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T18:13:22.519 INFO:teuthology.orchestra.run.smithi174.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T18:13:22.519 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T18:13:22.519 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 21 M 2024-04-26T18:13:22.519 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 34 k 2024-04-26T18:13:22.519 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 2.4 M 2024-04-26T18:13:22.519 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 249 k 2024-04-26T18:13:22.519 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 4.7 M 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 17 M 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 16 k 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 25 k 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 163 k 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T18:13:22.520 INFO:teuthology.orchestra.run.smithi174.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T18:13:22.521 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T18:13:22.521 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T18:13:22.521 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T18:13:22.521 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 501 k 2024-04-26T18:13:22.521 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T18:13:22.521 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T18:13:22.521 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T18:13:22.521 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T18:13:22.521 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T18:13:22.521 INFO:teuthology.orchestra.run.smithi174.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T18:13:22.521 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T18:13:22.521 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T18:13:22.521 INFO:teuthology.orchestra.run.smithi174.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T18:13:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T18:13:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T18:13:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T18:13:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T18:13:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T18:13:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T18:13:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T18:13:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T18:13:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T18:13:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T18:13:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 45 k 2024-04-26T18:13:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 135 k 2024-04-26T18:13:22.522 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T18:13:22.523 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T18:13:22.524 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T18:13:22.525 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T18:13:22.526 INFO:teuthology.orchestra.run.smithi174.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T18:13:22.527 INFO:teuthology.orchestra.run.smithi174.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T18:13:22.527 INFO:teuthology.orchestra.run.smithi174.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T18:13:22.527 INFO:teuthology.orchestra.run.smithi174.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T18:13:22.527 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T18:13:22.527 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T18:13:22.527 INFO:teuthology.orchestra.run.smithi174.stdout:Installing weak dependencies: 2024-04-26T18:13:22.527 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T18:13:22.527 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:13:22.527 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction Summary 2024-04-26T18:13:22.527 INFO:teuthology.orchestra.run.smithi174.stdout:======================================================================================= 2024-04-26T18:13:22.527 INFO:teuthology.orchestra.run.smithi174.stdout:Install 148 Packages 2024-04-26T18:13:22.527 INFO:teuthology.orchestra.run.smithi174.stdout:Upgrade 4 Packages 2024-04-26T18:13:22.527 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:13:22.528 INFO:teuthology.orchestra.run.smithi174.stdout:Total download size: 222 M 2024-04-26T18:13:22.528 INFO:teuthology.orchestra.run.smithi174.stdout:Downloading Packages: 2024-04-26T18:13:23.395 INFO:teuthology.orchestra.run.smithi103.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T18:13:23.396 INFO:teuthology.orchestra.run.smithi103.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T18:13:23.407 INFO:teuthology.orchestra.run.smithi103.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T18:13:23.407 INFO:teuthology.orchestra.run.smithi103.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T18:13:23.408 INFO:teuthology.orchestra.run.smithi103.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T18:13:23.409 INFO:teuthology.orchestra.run.smithi103.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T18:13:23.411 INFO:teuthology.orchestra.run.smithi103.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T18:13:23.413 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T18:13:23.414 INFO:teuthology.orchestra.run.smithi103.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T18:13:23.475 INFO:teuthology.orchestra.run.smithi103.stdout:Dependencies resolved. 2024-04-26T18:13:23.485 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-04-26T18:13:23.485 INFO:teuthology.orchestra.run.smithi103.stdout: Package Arch Version Repository Size 2024-04-26T18:13:23.485 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-04-26T18:13:23.485 INFO:teuthology.orchestra.run.smithi103.stdout:Installing: 2024-04-26T18:13:23.485 INFO:teuthology.orchestra.run.smithi103.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T18:13:23.485 INFO:teuthology.orchestra.run.smithi103.stdout: ceph x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 6.6 k 2024-04-26T18:13:23.485 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T18:13:23.485 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 890 k 2024-04-26T18:13:23.485 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 145 k 2024-04-26T18:13:23.485 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 1.5 M 2024-04-26T18:13:23.485 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 142 k 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 3.5 M 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 7.4 M 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 49 k 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 11 M 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 49 M 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 273 k 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 762 k 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 224 k 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 27 k 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 18 k 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T18:13:23.486 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 32 k 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 746 k 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 127 k 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 162 k 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 320 k 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 298 k 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 99 k 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 85 k 2024-04-26T18:13:23.487 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 170 k 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrading: 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout: librados2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.4 M 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout:Installing dependencies: 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 21 M 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 34 k 2024-04-26T18:13:23.488 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 2.4 M 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 249 k 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 4.7 M 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 17 M 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 16 k 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 25 k 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 163 k 2024-04-26T18:13:23.489 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T18:13:23.490 INFO:teuthology.orchestra.run.smithi103.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T18:13:23.490 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T18:13:23.490 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T18:13:23.490 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T18:13:23.490 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 501 k 2024-04-26T18:13:23.490 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T18:13:23.490 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T18:13:23.490 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T18:13:23.490 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T18:13:23.490 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T18:13:23.490 INFO:teuthology.orchestra.run.smithi103.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T18:13:23.490 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T18:13:23.490 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 45 k 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 135 k 2024-04-26T18:13:23.491 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T18:13:23.492 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T18:13:23.493 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T18:13:23.494 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T18:13:23.495 INFO:teuthology.orchestra.run.smithi103.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T18:13:23.496 INFO:teuthology.orchestra.run.smithi103.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T18:13:23.496 INFO:teuthology.orchestra.run.smithi103.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T18:13:23.496 INFO:teuthology.orchestra.run.smithi103.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T18:13:23.496 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T18:13:23.496 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T18:13:23.496 INFO:teuthology.orchestra.run.smithi103.stdout:Installing weak dependencies: 2024-04-26T18:13:23.496 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T18:13:23.496 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:13:23.496 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction Summary 2024-04-26T18:13:23.496 INFO:teuthology.orchestra.run.smithi103.stdout:======================================================================================= 2024-04-26T18:13:23.496 INFO:teuthology.orchestra.run.smithi103.stdout:Install 148 Packages 2024-04-26T18:13:23.496 INFO:teuthology.orchestra.run.smithi103.stdout:Upgrade 4 Packages 2024-04-26T18:13:23.496 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:13:23.497 INFO:teuthology.orchestra.run.smithi103.stdout:Total download size: 222 M 2024-04-26T18:13:23.497 INFO:teuthology.orchestra.run.smithi103.stdout:Downloading Packages: 2024-04-26T18:13:23.942 INFO:teuthology.orchestra.run.smithi174.stdout:(1/152): ceph-19.0.0-3267.gbc94a2a9.el9.x86_64. 14 kB/s | 6.6 kB 00:00 2024-04-26T18:13:24.159 INFO:teuthology.orchestra.run.smithi174.stdout:(2/152): ceph-fuse-19.0.0-3267.gbc94a2a9.el9.x8 4.0 MB/s | 890 kB 00:00 2024-04-26T18:13:24.209 INFO:teuthology.orchestra.run.smithi174.stdout:(3/152): ceph-immutable-object-cache-19.0.0-326 2.8 MB/s | 145 kB 00:00 2024-04-26T18:13:24.422 INFO:teuthology.orchestra.run.smithi084.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T18:13:24.422 INFO:teuthology.orchestra.run.smithi084.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-26T18:13:24.433 INFO:teuthology.orchestra.run.smithi084.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-26T18:13:24.433 INFO:teuthology.orchestra.run.smithi084.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-26T18:13:24.434 INFO:teuthology.orchestra.run.smithi084.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-26T18:13:24.435 INFO:teuthology.orchestra.run.smithi084.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-26T18:13:24.436 INFO:teuthology.orchestra.run.smithi084.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-26T18:13:24.439 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-26T18:13:24.439 INFO:teuthology.orchestra.run.smithi084.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-26T18:13:24.500 INFO:teuthology.orchestra.run.smithi084.stdout:Dependencies resolved. 2024-04-26T18:13:24.509 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-04-26T18:13:24.509 INFO:teuthology.orchestra.run.smithi084.stdout: Package Arch Version Repository Size 2024-04-26T18:13:24.510 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-04-26T18:13:24.510 INFO:teuthology.orchestra.run.smithi084.stdout:Installing: 2024-04-26T18:13:24.510 INFO:teuthology.orchestra.run.smithi084.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-26T18:13:24.510 INFO:teuthology.orchestra.run.smithi084.stdout: ceph x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 6.6 k 2024-04-26T18:13:24.510 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T18:13:24.510 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 890 k 2024-04-26T18:13:24.510 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 145 k 2024-04-26T18:13:24.510 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 1.5 M 2024-04-26T18:13:24.510 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 142 k 2024-04-26T18:13:24.510 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 3.5 M 2024-04-26T18:13:24.510 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 7.4 M 2024-04-26T18:13:24.510 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 49 k 2024-04-26T18:13:24.510 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 11 M 2024-04-26T18:13:24.511 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 49 M 2024-04-26T18:13:24.511 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 273 k 2024-04-26T18:13:24.511 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 762 k 2024-04-26T18:13:24.511 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 224 k 2024-04-26T18:13:24.511 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-shell x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 27 k 2024-04-26T18:13:24.511 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 18 k 2024-04-26T18:13:24.511 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-26T18:13:24.511 INFO:teuthology.orchestra.run.smithi084.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-26T18:13:24.511 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-26T18:13:24.511 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 32 k 2024-04-26T18:13:24.511 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 746 k 2024-04-26T18:13:24.511 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 127 k 2024-04-26T18:13:24.511 INFO:teuthology.orchestra.run.smithi084.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-26T18:13:24.512 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-26T18:13:24.512 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-26T18:13:24.512 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-26T18:13:24.512 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-26T18:13:24.512 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-26T18:13:24.512 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 162 k 2024-04-26T18:13:24.512 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 320 k 2024-04-26T18:13:24.512 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 298 k 2024-04-26T18:13:24.512 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 99 k 2024-04-26T18:13:24.512 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 85 k 2024-04-26T18:13:24.512 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T18:13:24.512 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 170 k 2024-04-26T18:13:24.512 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrading: 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout: librados2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.4 M 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 3.1 M 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout:Installing dependencies: 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 21 M 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 34 k 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 2.4 M 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 249 k 2024-04-26T18:13:24.513 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 4.7 M 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 17 M 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3267.gbc94a2a9.el9 ceph-noarch 16 k 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 25 k 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 163 k 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T18:13:24.514 INFO:teuthology.orchestra.run.smithi084.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 501 k 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2 x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 5.4 M 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T18:13:24.515 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T18:13:24.516 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T18:13:24.516 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-26T18:13:24.516 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-26T18:13:24.516 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-26T18:13:24.516 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T18:13:24.516 INFO:teuthology.orchestra.run.smithi084.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-26T18:13:24.516 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T18:13:24.516 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T18:13:24.516 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T18:13:24.516 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 45 k 2024-04-26T18:13:24.516 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common x86_64 2:19.0.0-3267.gbc94a2a9.el9 ceph 135 k 2024-04-26T18:13:24.516 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T18:13:24.516 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T18:13:24.517 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-26T18:13:24.518 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T18:13:24.519 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-26T18:13:24.520 INFO:teuthology.orchestra.run.smithi084.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T18:13:24.521 INFO:teuthology.orchestra.run.smithi084.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T18:13:24.521 INFO:teuthology.orchestra.run.smithi084.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T18:13:24.521 INFO:teuthology.orchestra.run.smithi084.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-26T18:13:24.521 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T18:13:24.521 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-26T18:13:24.521 INFO:teuthology.orchestra.run.smithi084.stdout:Installing weak dependencies: 2024-04-26T18:13:24.521 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-26T18:13:24.521 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:13:24.521 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction Summary 2024-04-26T18:13:24.521 INFO:teuthology.orchestra.run.smithi084.stdout:======================================================================================= 2024-04-26T18:13:24.521 INFO:teuthology.orchestra.run.smithi084.stdout:Install 148 Packages 2024-04-26T18:13:24.521 INFO:teuthology.orchestra.run.smithi084.stdout:Upgrade 4 Packages 2024-04-26T18:13:24.521 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:13:24.522 INFO:teuthology.orchestra.run.smithi084.stdout:Total download size: 222 M 2024-04-26T18:13:24.522 INFO:teuthology.orchestra.run.smithi084.stdout:Downloading Packages: 2024-04-26T18:13:24.534 INFO:teuthology.orchestra.run.smithi174.stdout:(4/152): ceph-base-19.0.0-3267.gbc94a2a9.el9.x8 5.1 MB/s | 5.4 MB 00:01 2024-04-26T18:13:24.576 INFO:teuthology.orchestra.run.smithi174.stdout:(5/152): ceph-mds-19.0.0-3267.gbc94a2a9.el9.x86 6.4 MB/s | 2.4 MB 00:00 2024-04-26T18:13:24.710 INFO:teuthology.orchestra.run.smithi174.stdout:(6/152): ceph-mgr-19.0.0-3267.gbc94a2a9.el9.x86 8.7 MB/s | 1.5 MB 00:00 2024-04-26T18:13:24.887 INFO:teuthology.orchestra.run.smithi103.stdout:(1/152): ceph-19.0.0-3267.gbc94a2a9.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-04-26T18:13:25.152 INFO:teuthology.orchestra.run.smithi174.stdout:(7/152): ceph-mon-19.0.0-3267.gbc94a2a9.el9.x86 8.2 MB/s | 4.7 MB 00:00 2024-04-26T18:13:25.662 INFO:teuthology.orchestra.run.smithi103.stdout:(2/152): ceph-fuse-19.0.0-3267.gbc94a2a9.el9.x8 1.1 MB/s | 890 kB 00:00 2024-04-26T18:13:25.781 INFO:teuthology.orchestra.run.smithi084.stdout:(1/152): ceph-19.0.0-3267.gbc94a2a9.el9.x86_64. 11 kB/s | 6.6 kB 00:00 2024-04-26T18:13:25.813 INFO:teuthology.orchestra.run.smithi103.stdout:(3/152): ceph-immutable-object-cache-19.0.0-326 964 kB/s | 145 kB 00:00 2024-04-26T18:13:26.602 INFO:teuthology.orchestra.run.smithi174.stdout:(8/152): ceph-osd-19.0.0-3267.gbc94a2a9.el9.x86 8.9 MB/s | 17 MB 00:01 2024-04-26T18:13:26.639 INFO:teuthology.orchestra.run.smithi084.stdout:(2/152): ceph-fuse-19.0.0-3267.gbc94a2a9.el9.x8 1.0 MB/s | 890 kB 00:00 2024-04-26T18:13:26.680 INFO:teuthology.orchestra.run.smithi103.stdout:(4/152): ceph-mds-19.0.0-3267.gbc94a2a9.el9.x86 2.7 MB/s | 2.4 MB 00:00 2024-04-26T18:13:26.727 INFO:teuthology.orchestra.run.smithi174.stdout:(9/152): ceph-radosgw-19.0.0-3267.gbc94a2a9.el9 6.7 MB/s | 11 MB 00:01 2024-04-26T18:13:26.731 INFO:teuthology.orchestra.run.smithi084.stdout:(3/152): ceph-immutable-object-cache-19.0.0-326 1.5 MB/s | 145 kB 00:00 2024-04-26T18:13:26.752 INFO:teuthology.orchestra.run.smithi174.stdout:(10/152): ceph-selinux-19.0.0-3267.gbc94a2a9.el 168 kB/s | 25 kB 00:00 2024-04-26T18:13:26.861 INFO:teuthology.orchestra.run.smithi174.stdout:(11/152): cephfs-mirror-19.0.0-3267.gbc94a2a9.e 2.0 MB/s | 224 kB 00:00 2024-04-26T18:13:26.953 INFO:teuthology.orchestra.run.smithi174.stdout:(12/152): cephfs-shell-19.0.0-3267.gbc94a2a9.el 294 kB/s | 27 kB 00:00 2024-04-26T18:13:27.023 INFO:teuthology.orchestra.run.smithi084.stdout:(4/152): ceph-base-19.0.0-3267.gbc94a2a9.el9.x8 2.9 MB/s | 5.4 MB 00:01 2024-04-26T18:13:27.036 INFO:teuthology.orchestra.run.smithi174.stdout:(13/152): libcephfs-devel-19.0.0-3267.gbc94a2a9 381 kB/s | 32 kB 00:00 2024-04-26T18:13:27.072 INFO:teuthology.orchestra.run.smithi103.stdout:(5/152): ceph-mgr-19.0.0-3267.gbc94a2a9.el9.x86 3.9 MB/s | 1.5 MB 00:00 2024-04-26T18:13:27.353 INFO:teuthology.orchestra.run.smithi174.stdout:(14/152): libcephfs2-19.0.0-3267.gbc94a2a9.el9. 2.3 MB/s | 746 kB 00:00 2024-04-26T18:13:27.432 INFO:teuthology.orchestra.run.smithi084.stdout:(5/152): ceph-mgr-19.0.0-3267.gbc94a2a9.el9.x86 3.7 MB/s | 1.5 MB 00:00 2024-04-26T18:13:27.436 INFO:teuthology.orchestra.run.smithi174.stdout:(15/152): libcephsqlite-19.0.0-3267.gbc94a2a9.e 1.9 MB/s | 163 kB 00:00 2024-04-26T18:13:27.529 INFO:teuthology.orchestra.run.smithi174.stdout:(16/152): librados-devel-19.0.0-3267.gbc94a2a9. 1.3 MB/s | 127 kB 00:00 2024-04-26T18:13:27.771 INFO:teuthology.orchestra.run.smithi174.stdout:(17/152): libradosstriper1-19.0.0-3267.gbc94a2a 2.0 MB/s | 501 kB 00:00 2024-04-26T18:13:27.822 INFO:teuthology.orchestra.run.smithi103.stdout:(6/152): ceph-base-19.0.0-3267.gbc94a2a9.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-04-26T18:13:28.155 INFO:teuthology.orchestra.run.smithi103.stdout:(7/152): ceph-mon-19.0.0-3267.gbc94a2a9.el9.x86 4.3 MB/s | 4.7 MB 00:01 2024-04-26T18:13:28.798 INFO:teuthology.orchestra.run.smithi084.stdout:(6/152): ceph-mds-19.0.0-3267.gbc94a2a9.el9.x86 1.1 MB/s | 2.4 MB 00:02 2024-04-26T18:13:28.857 INFO:teuthology.orchestra.run.smithi084.stdout:(7/152): ceph-mon-19.0.0-3267.gbc94a2a9.el9.x86 3.3 MB/s | 4.7 MB 00:01 2024-04-26T18:13:29.729 INFO:teuthology.orchestra.run.smithi174.stdout:(18/152): librgw2-19.0.0-3267.gbc94a2a9.el9.x86 2.7 MB/s | 5.4 MB 00:01 2024-04-26T18:13:29.829 INFO:teuthology.orchestra.run.smithi174.stdout:(19/152): python3-ceph-argparse-19.0.0-3267.gbc 452 kB/s | 45 kB 00:00 2024-04-26T18:13:29.921 INFO:teuthology.orchestra.run.smithi174.stdout:(20/152): python3-ceph-common-19.0.0-3267.gbc94 1.4 MB/s | 135 kB 00:00 2024-04-26T18:13:30.013 INFO:teuthology.orchestra.run.smithi174.stdout:(21/152): python3-cephfs-19.0.0-3267.gbc94a2a9. 1.7 MB/s | 162 kB 00:00 2024-04-26T18:13:30.180 INFO:teuthology.orchestra.run.smithi174.stdout:(22/152): python3-rados-19.0.0-3267.gbc94a2a9.e 1.9 MB/s | 320 kB 00:00 2024-04-26T18:13:30.347 INFO:teuthology.orchestra.run.smithi174.stdout:(23/152): python3-rbd-19.0.0-3267.gbc94a2a9.el9 1.7 MB/s | 298 kB 00:00 2024-04-26T18:13:30.447 INFO:teuthology.orchestra.run.smithi174.stdout:(24/152): python3-rgw-19.0.0-3267.gbc94a2a9.el9 996 kB/s | 99 kB 00:00 2024-04-26T18:13:30.532 INFO:teuthology.orchestra.run.smithi174.stdout:(25/152): rbd-fuse-19.0.0-3267.gbc94a2a9.el9.x8 1.0 MB/s | 85 kB 00:00 2024-04-26T18:13:30.682 INFO:teuthology.orchestra.run.smithi084.stdout:(8/152): ceph-common-19.0.0-3267.gbc94a2a9.el9. 3.8 MB/s | 21 MB 00:05 2024-04-26T18:13:30.714 INFO:teuthology.orchestra.run.smithi103.stdout:(8/152): ceph-radosgw-19.0.0-3267.gbc94a2a9.el9 4.1 MB/s | 11 MB 00:02 2024-04-26T18:13:30.749 INFO:teuthology.orchestra.run.smithi084.stdout:(9/152): ceph-selinux-19.0.0-3267.gbc94a2a9.el9 379 kB/s | 25 kB 00:00 2024-04-26T18:13:30.781 INFO:teuthology.orchestra.run.smithi103.stdout:(9/152): ceph-selinux-19.0.0-3267.gbc94a2a9.el9 378 kB/s | 25 kB 00:00 2024-04-26T18:13:30.781 INFO:teuthology.orchestra.run.smithi174.stdout:(26/152): ceph-common-19.0.0-3267.gbc94a2a9.el9 2.9 MB/s | 21 MB 00:07 2024-04-26T18:13:30.940 INFO:teuthology.orchestra.run.smithi174.stdout:(27/152): rbd-nbd-19.0.0-3267.gbc94a2a9.el9.x86 1.0 MB/s | 170 kB 00:00 2024-04-26T18:13:31.014 INFO:teuthology.orchestra.run.smithi103.stdout:(10/152): ceph-osd-19.0.0-3267.gbc94a2a9.el9.x8 5.3 MB/s | 17 MB 00:03 2024-04-26T18:13:31.065 INFO:teuthology.orchestra.run.smithi174.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3267.g 274 kB/s | 34 kB 00:00 2024-04-26T18:13:31.123 INFO:teuthology.orchestra.run.smithi103.stdout:(11/152): cephfs-mirror-19.0.0-3267.gbc94a2a9.e 2.0 MB/s | 224 kB 00:00 2024-04-26T18:13:31.223 INFO:teuthology.orchestra.run.smithi103.stdout:(12/152): cephfs-shell-19.0.0-3267.gbc94a2a9.el 271 kB/s | 27 kB 00:00 2024-04-26T18:13:31.241 INFO:teuthology.orchestra.run.smithi174.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3267.gbc94a2a 813 kB/s | 142 kB 00:00 2024-04-26T18:13:31.315 INFO:teuthology.orchestra.run.smithi103.stdout:(13/152): libcephfs-devel-19.0.0-3267.gbc94a2a9 345 kB/s | 32 kB 00:00 2024-04-26T18:13:31.532 INFO:teuthology.orchestra.run.smithi103.stdout:(14/152): libcephfs2-19.0.0-3267.gbc94a2a9.el9. 3.4 MB/s | 746 kB 00:00 2024-04-26T18:13:31.549 INFO:teuthology.orchestra.run.smithi174.stdout:(30/152): rbd-mirror-19.0.0-3267.gbc94a2a9.el9. 3.0 MB/s | 3.1 MB 00:01 2024-04-26T18:13:31.616 INFO:teuthology.orchestra.run.smithi103.stdout:(15/152): libcephsqlite-19.0.0-3267.gbc94a2a9.e 1.9 MB/s | 163 kB 00:00 2024-04-26T18:13:31.716 INFO:teuthology.orchestra.run.smithi103.stdout:(16/152): librados-devel-19.0.0-3267.gbc94a2a9. 1.2 MB/s | 127 kB 00:00 2024-04-26T18:13:31.874 INFO:teuthology.orchestra.run.smithi103.stdout:(17/152): libradosstriper1-19.0.0-3267.gbc94a2a 3.1 MB/s | 501 kB 00:00 2024-04-26T18:13:32.324 INFO:teuthology.orchestra.run.smithi084.stdout:(10/152): ceph-radosgw-19.0.0-3267.gbc94a2a9.el 3.1 MB/s | 11 MB 00:03 2024-04-26T18:13:32.524 INFO:teuthology.orchestra.run.smithi084.stdout:(11/152): cephfs-mirror-19.0.0-3267.gbc94a2a9.e 1.1 MB/s | 224 kB 00:00 2024-04-26T18:13:32.616 INFO:teuthology.orchestra.run.smithi084.stdout:(12/152): cephfs-shell-19.0.0-3267.gbc94a2a9.el 294 kB/s | 27 kB 00:00 2024-04-26T18:13:32.708 INFO:teuthology.orchestra.run.smithi084.stdout:(13/152): libcephfs-devel-19.0.0-3267.gbc94a2a9 346 kB/s | 32 kB 00:00 2024-04-26T18:13:32.833 INFO:teuthology.orchestra.run.smithi103.stdout:(18/152): librgw2-19.0.0-3267.gbc94a2a9.el9.x86 5.6 MB/s | 5.4 MB 00:00 2024-04-26T18:13:32.933 INFO:teuthology.orchestra.run.smithi103.stdout:(19/152): python3-ceph-argparse-19.0.0-3267.gbc 452 kB/s | 45 kB 00:00 2024-04-26T18:13:32.975 INFO:teuthology.orchestra.run.smithi084.stdout:(14/152): libcephfs2-19.0.0-3267.gbc94a2a9.el9. 2.7 MB/s | 746 kB 00:00 2024-04-26T18:13:33.033 INFO:teuthology.orchestra.run.smithi103.stdout:(20/152): python3-ceph-common-19.0.0-3267.gbc94 1.3 MB/s | 135 kB 00:00 2024-04-26T18:13:33.075 INFO:teuthology.orchestra.run.smithi084.stdout:(15/152): libcephsqlite-19.0.0-3267.gbc94a2a9.e 1.6 MB/s | 163 kB 00:00 2024-04-26T18:13:33.134 INFO:teuthology.orchestra.run.smithi103.stdout:(21/152): python3-cephfs-19.0.0-3267.gbc94a2a9. 1.6 MB/s | 162 kB 00:00 2024-04-26T18:13:33.175 INFO:teuthology.orchestra.run.smithi084.stdout:(16/152): librados-devel-19.0.0-3267.gbc94a2a9. 1.2 MB/s | 127 kB 00:00 2024-04-26T18:13:33.234 INFO:teuthology.orchestra.run.smithi103.stdout:(22/152): python3-rados-19.0.0-3267.gbc94a2a9.e 3.1 MB/s | 320 kB 00:00 2024-04-26T18:13:33.385 INFO:teuthology.orchestra.run.smithi084.stdout:(17/152): libradosstriper1-19.0.0-3267.gbc94a2a 2.3 MB/s | 501 kB 00:00 2024-04-26T18:13:33.476 INFO:teuthology.orchestra.run.smithi103.stdout:(23/152): ceph-common-19.0.0-3267.gbc94a2a9.el9 2.4 MB/s | 21 MB 00:08 2024-04-26T18:13:33.502 INFO:teuthology.orchestra.run.smithi103.stdout:(24/152): python3-rbd-19.0.0-3267.gbc94a2a9.el9 1.1 MB/s | 298 kB 00:00 2024-04-26T18:13:33.558 INFO:teuthology.orchestra.run.smithi174.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3267.gbc94a 1.5 MB/s | 3.5 MB 00:02 2024-04-26T18:13:33.569 INFO:teuthology.orchestra.run.smithi103.stdout:(25/152): python3-rgw-19.0.0-3267.gbc94a2a9.el9 1.0 MB/s | 99 kB 00:00 2024-04-26T18:13:33.594 INFO:teuthology.orchestra.run.smithi103.stdout:(26/152): rbd-fuse-19.0.0-3267.gbc94a2a9.el9.x8 924 kB/s | 85 kB 00:00 2024-04-26T18:13:33.703 INFO:teuthology.orchestra.run.smithi103.stdout:(27/152): rbd-nbd-19.0.0-3267.gbc94a2a9.el9.x86 1.5 MB/s | 170 kB 00:00 2024-04-26T18:13:33.742 INFO:teuthology.orchestra.run.smithi174.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3267.gbc 1.3 MB/s | 249 kB 00:00 2024-04-26T18:13:33.795 INFO:teuthology.orchestra.run.smithi103.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3267.g 373 kB/s | 34 kB 00:00 2024-04-26T18:13:33.842 INFO:teuthology.orchestra.run.smithi174.stdout:(33/152): ceph-mgr-rook-19.0.0-3267.gbc94a2a9.e 494 kB/s | 49 kB 00:00 2024-04-26T18:13:33.904 INFO:teuthology.orchestra.run.smithi103.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3267.gbc94a2a 1.3 MB/s | 142 kB 00:00 2024-04-26T18:13:33.959 INFO:teuthology.orchestra.run.smithi174.stdout:(34/152): ceph-mgr-diskprediction-local-19.0.0- 3.1 MB/s | 7.4 MB 00:02 2024-04-26T18:13:33.975 INFO:teuthology.orchestra.run.smithi174.stdout:(35/152): ceph-prometheus-alerts-19.0.0-3267.gb 123 kB/s | 16 kB 00:00 2024-04-26T18:13:34.117 INFO:teuthology.orchestra.run.smithi174.stdout:(36/152): ceph-volume-19.0.0-3267.gbc94a2a9.el9 1.7 MB/s | 273 kB 00:00 2024-04-26T18:13:34.201 INFO:teuthology.orchestra.run.smithi174.stdout:(37/152): cephfs-top-19.0.0-3267.gbc94a2a9.el9. 219 kB/s | 18 kB 00:00 2024-04-26T18:13:34.351 INFO:teuthology.orchestra.run.smithi174.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 537 kB/s | 80 kB 00:00 2024-04-26T18:13:34.485 INFO:teuthology.orchestra.run.smithi174.stdout:(39/152): cephadm-19.0.0-3267.gbc94a2a9.el9.noa 1.5 MB/s | 762 kB 00:00 2024-04-26T18:13:34.585 INFO:teuthology.orchestra.run.smithi174.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-04-26T18:13:34.587 INFO:teuthology.orchestra.run.smithi103.stdout:(30/152): rbd-mirror-19.0.0-3267.gbc94a2a9.el9. 3.0 MB/s | 3.1 MB 00:01 2024-04-26T18:13:34.646 INFO:teuthology.orchestra.run.smithi103.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3267.gbc94a 4.8 MB/s | 3.5 MB 00:00 2024-04-26T18:13:34.669 INFO:teuthology.orchestra.run.smithi174.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-26T18:13:34.710 INFO:teuthology.orchestra.run.smithi174.stdout:(42/152): libquadmath-11.4.1-3.el9.x86_64.rpm 867 kB/s | 195 kB 00:00 2024-04-26T18:13:34.746 INFO:teuthology.orchestra.run.smithi103.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3267.gbc 2.4 MB/s | 249 kB 00:00 2024-04-26T18:13:34.769 INFO:teuthology.orchestra.run.smithi174.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-26T18:13:34.828 INFO:teuthology.orchestra.run.smithi174.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-26T18:13:34.838 INFO:teuthology.orchestra.run.smithi103.stdout:(33/152): ceph-mgr-rook-19.0.0-3267.gbc94a2a9.e 537 kB/s | 49 kB 00:00 2024-04-26T18:13:34.886 INFO:teuthology.orchestra.run.smithi174.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-04-26T18:13:34.920 INFO:teuthology.orchestra.run.smithi174.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 4.9 MB/s | 1.2 MB 00:00 2024-04-26T18:13:34.922 INFO:teuthology.orchestra.run.smithi103.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3267.gb 197 kB/s | 16 kB 00:00 2024-04-26T18:13:34.970 INFO:teuthology.orchestra.run.smithi174.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-26T18:13:35.014 INFO:teuthology.orchestra.run.smithi103.stdout:(35/152): ceph-volume-19.0.0-3267.gbc94a2a9.el9 2.9 MB/s | 273 kB 00:00 2024-04-26T18:13:35.029 INFO:teuthology.orchestra.run.smithi174.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 6.2 MB/s | 689 kB 00:00 2024-04-26T18:13:35.106 INFO:teuthology.orchestra.run.smithi174.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 6.8 MB/s | 943 kB 00:00 2024-04-26T18:13:35.109 INFO:teuthology.orchestra.run.smithi084.stdout:(18/152): librgw2-19.0.0-3267.gbc94a2a9.el9.x86 3.1 MB/s | 5.4 MB 00:01 2024-04-26T18:13:35.138 INFO:teuthology.orchestra.run.smithi174.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 986 kB/s | 107 kB 00:00 2024-04-26T18:13:35.164 INFO:teuthology.orchestra.run.smithi103.stdout:(36/152): cephadm-19.0.0-3267.gbc94a2a9.el9.noa 5.0 MB/s | 762 kB 00:00 2024-04-26T18:13:35.180 INFO:teuthology.orchestra.run.smithi174.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 292 kB/s | 21 kB 00:00 2024-04-26T18:13:35.192 INFO:teuthology.orchestra.run.smithi084.stdout:(19/152): python3-ceph-argparse-19.0.0-3267.gbc 544 kB/s | 45 kB 00:00 2024-04-26T18:13:35.230 INFO:teuthology.orchestra.run.smithi174.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 3.3 MB/s | 313 kB 00:00 2024-04-26T18:13:35.239 INFO:teuthology.orchestra.run.smithi103.stdout:(37/152): cephfs-top-19.0.0-3267.gbc94a2a9.el9. 243 kB/s | 18 kB 00:00 2024-04-26T18:13:35.255 INFO:teuthology.orchestra.run.smithi174.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 436 kB/s | 33 kB 00:00 2024-04-26T18:13:35.268 INFO:teuthology.orchestra.run.smithi084.stdout:(20/152): python3-ceph-common-19.0.0-3267.gbc94 1.8 MB/s | 135 kB 00:00 2024-04-26T18:13:35.305 INFO:teuthology.orchestra.run.smithi174.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 366 kB/s | 18 kB 00:00 2024-04-26T18:13:35.343 INFO:teuthology.orchestra.run.smithi084.stdout:(21/152): python3-cephfs-19.0.0-3267.gbc94a2a9. 2.1 MB/s | 162 kB 00:00 2024-04-26T18:13:35.347 INFO:teuthology.orchestra.run.smithi174.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-04-26T18:13:35.390 INFO:teuthology.orchestra.run.smithi103.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 536 kB/s | 80 kB 00:00 2024-04-26T18:13:35.460 INFO:teuthology.orchestra.run.smithi084.stdout:(22/152): python3-rados-19.0.0-3267.gbc94a2a9.e 2.7 MB/s | 320 kB 00:00 2024-04-26T18:13:35.489 INFO:teuthology.orchestra.run.smithi174.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2024-04-26T18:13:35.515 INFO:teuthology.orchestra.run.smithi174.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 959 kB/s | 160 kB 00:00 2024-04-26T18:13:35.574 INFO:teuthology.orchestra.run.smithi174.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 540 kB/s | 45 kB 00:00 2024-04-26T18:13:35.586 INFO:teuthology.orchestra.run.smithi084.stdout:(23/152): python3-rbd-19.0.0-3267.gbc94a2a9.el9 2.3 MB/s | 298 kB 00:00 2024-04-26T18:13:35.598 INFO:teuthology.orchestra.run.smithi103.stdout:(39/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.8 MB/s | 803 kB 00:00 2024-04-26T18:13:35.641 INFO:teuthology.orchestra.run.smithi174.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-04-26T18:13:35.653 INFO:teuthology.orchestra.run.smithi084.stdout:(24/152): python3-rgw-19.0.0-3267.gbc94a2a9.el9 1.5 MB/s | 99 kB 00:00 2024-04-26T18:13:35.682 INFO:teuthology.orchestra.run.smithi103.stdout:(40/152): libquadmath-11.4.1-3.el9.x86_64.rpm 2.3 MB/s | 195 kB 00:00 2024-04-26T18:13:35.691 INFO:teuthology.orchestra.run.smithi174.stdout:(60/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 512 kB/s | 25 kB 00:00 2024-04-26T18:13:35.716 INFO:teuthology.orchestra.run.smithi174.stdout:(61/152): libtool-2.4.6-45.el9.x86_64.rpm 4.0 MB/s | 582 kB 00:00 2024-04-26T18:13:35.719 INFO:teuthology.orchestra.run.smithi084.stdout:(25/152): rbd-fuse-19.0.0-3267.gbc94a2a9.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-26T18:13:35.766 INFO:teuthology.orchestra.run.smithi103.stdout:(41/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-04-26T18:13:35.766 INFO:teuthology.orchestra.run.smithi174.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 631 kB/s | 47 kB 00:00 2024-04-26T18:13:35.816 INFO:teuthology.orchestra.run.smithi174.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2024-04-26T18:13:35.858 INFO:teuthology.orchestra.run.smithi174.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-26T18:13:35.884 INFO:teuthology.orchestra.run.smithi174.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 527 kB/s | 35 kB 00:00 2024-04-26T18:13:36.007 INFO:teuthology.orchestra.run.smithi103.stdout:(42/152): python3-cryptography-36.0.1-4.el9.x86 5.1 MB/s | 1.2 MB 00:00 2024-04-26T18:13:36.058 INFO:teuthology.orchestra.run.smithi103.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-26T18:13:36.109 INFO:teuthology.orchestra.run.smithi174.stdout:(66/152): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-04-26T18:13:36.116 INFO:teuthology.orchestra.run.smithi103.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-26T18:13:36.158 INFO:teuthology.orchestra.run.smithi103.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-26T18:13:36.184 INFO:teuthology.orchestra.run.smithi174.stdout:(67/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-04-26T18:13:36.195 INFO:teuthology.orchestra.run.smithi084.stdout:(26/152): ceph-osd-19.0.0-3267.gbc94a2a9.el9.x8 2.3 MB/s | 17 MB 00:07 2024-04-26T18:13:36.259 INFO:teuthology.orchestra.run.smithi103.stdout:(46/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.1 MB/s | 215 kB 00:00 2024-04-26T18:13:36.270 INFO:teuthology.orchestra.run.smithi084.stdout:(27/152): rbd-nbd-19.0.0-3267.gbc94a2a9.el9.x86 2.2 MB/s | 170 kB 00:00 2024-04-26T18:13:36.284 INFO:teuthology.orchestra.run.smithi174.stdout:(68/152): postgresql-13.14-1.el9.x86_64.rpm 9.1 MB/s | 1.6 MB 00:00 2024-04-26T18:13:36.335 INFO:teuthology.orchestra.run.smithi174.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 5.9 MB/s | 911 kB 00:00 2024-04-26T18:13:36.337 INFO:teuthology.orchestra.run.smithi084.stdout:(28/152): ceph-grafana-dashboards-19.0.0-3267.g 512 kB/s | 34 kB 00:00 2024-04-26T18:13:36.377 INFO:teuthology.orchestra.run.smithi174.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-04-26T18:13:36.456 INFO:teuthology.orchestra.run.smithi084.stdout:(29/152): ceph-mgr-cephadm-19.0.0-3267.gbc94a2a 1.2 MB/s | 142 kB 00:00 2024-04-26T18:13:36.459 INFO:teuthology.orchestra.run.smithi103.stdout:(47/152): ceph-mgr-diskprediction-local-19.0.0- 4.0 MB/s | 7.4 MB 00:01 2024-04-26T18:13:36.477 INFO:teuthology.orchestra.run.smithi174.stdout:(71/152): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-26T18:13:36.484 INFO:teuthology.orchestra.run.smithi103.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 3.0 MB/s | 689 kB 00:00 2024-04-26T18:13:36.535 INFO:teuthology.orchestra.run.smithi103.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-04-26T18:13:36.544 INFO:teuthology.orchestra.run.smithi174.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-04-26T18:13:36.585 INFO:teuthology.orchestra.run.smithi103.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 432 kB/s | 21 kB 00:00 2024-04-26T18:13:36.606 INFO:teuthology.orchestra.run.smithi084.stdout:(30/152): rbd-mirror-19.0.0-3267.gbc94a2a9.el9. 3.5 MB/s | 3.1 MB 00:00 2024-04-26T18:13:36.636 INFO:teuthology.orchestra.run.smithi174.stdout:(73/152): postgresql-server-13.14-1.el9.x86_64. 19 MB/s | 5.8 MB 00:00 2024-04-26T18:13:36.669 INFO:teuthology.orchestra.run.smithi103.stdout:(51/152): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-04-26T18:13:36.711 INFO:teuthology.orchestra.run.smithi174.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-26T18:13:36.719 INFO:teuthology.orchestra.run.smithi103.stdout:(52/152): flexiblas-3.0.4-8.el9.x86_64.rpm 653 kB/s | 33 kB 00:00 2024-04-26T18:13:36.753 INFO:teuthology.orchestra.run.smithi103.stdout:(53/152): bison-3.7.4-5.el9.x86_64.rpm 3.1 MB/s | 943 kB 00:00 2024-04-26T18:13:36.794 INFO:teuthology.orchestra.run.smithi103.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-04-26T18:13:36.811 INFO:teuthology.orchestra.run.smithi174.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-04-26T18:13:36.845 INFO:teuthology.orchestra.run.smithi103.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-26T18:13:36.895 INFO:teuthology.orchestra.run.smithi174.stdout:(76/152): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-04-26T18:13:36.903 INFO:teuthology.orchestra.run.smithi103.stdout:(56/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-04-26T18:13:36.920 INFO:teuthology.orchestra.run.smithi174.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 236 kB/s | 26 kB 00:00 2024-04-26T18:13:36.962 INFO:teuthology.orchestra.run.smithi174.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 287 kB/s | 19 kB 00:00 2024-04-26T18:13:36.962 INFO:teuthology.orchestra.run.smithi103.stdout:(57/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 776 kB/s | 45 kB 00:00 2024-04-26T18:13:37.004 INFO:teuthology.orchestra.run.smithi174.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-26T18:13:37.029 INFO:teuthology.orchestra.run.smithi174.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 519 kB/s | 35 kB 00:00 2024-04-26T18:13:37.105 INFO:teuthology.orchestra.run.smithi174.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-04-26T18:13:37.112 INFO:teuthology.orchestra.run.smithi103.stdout:(58/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-04-26T18:13:37.229 INFO:teuthology.orchestra.run.smithi103.stdout:(59/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.9 MB/s | 3.0 MB 00:00 2024-04-26T18:13:37.257 INFO:teuthology.orchestra.run.smithi084.stdout:(31/152): ceph-mgr-dashboard-19.0.0-3267.gbc94a 4.4 MB/s | 3.5 MB 00:00 2024-04-26T18:13:37.263 INFO:teuthology.orchestra.run.smithi103.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-04-26T18:13:37.288 INFO:teuthology.orchestra.run.smithi103.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-26T18:13:37.314 INFO:teuthology.orchestra.run.smithi103.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-04-26T18:13:37.372 INFO:teuthology.orchestra.run.smithi103.stdout:(63/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-04-26T18:13:37.390 INFO:teuthology.orchestra.run.smithi084.stdout:(32/152): ceph-mgr-modules-core-19.0.0-3267.gbc 1.8 MB/s | 249 kB 00:00 2024-04-26T18:13:37.398 INFO:teuthology.orchestra.run.smithi103.stdout:(64/152): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-04-26T18:13:37.423 INFO:teuthology.orchestra.run.smithi103.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 700 kB/s | 35 kB 00:00 2024-04-26T18:13:37.440 INFO:teuthology.orchestra.run.smithi084.stdout:(33/152): ceph-mgr-rook-19.0.0-3267.gbc94a2a9.e 985 kB/s | 49 kB 00:00 2024-04-26T18:13:37.499 INFO:teuthology.orchestra.run.smithi084.stdout:(34/152): ceph-prometheus-alerts-19.0.0-3267.gb 283 kB/s | 16 kB 00:00 2024-04-26T18:13:37.505 INFO:teuthology.orchestra.run.smithi174.stdout:(82/152): ceph-test-19.0.0-3267.gbc94a2a9.el9.x 4.6 MB/s | 49 MB 00:10 2024-04-26T18:13:37.563 INFO:teuthology.orchestra.run.smithi174.stdout:(83/152): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-26T18:13:37.583 INFO:teuthology.orchestra.run.smithi084.stdout:(35/152): ceph-volume-19.0.0-3267.gbc94a2a9.el9 3.2 MB/s | 273 kB 00:00 2024-04-26T18:13:37.589 INFO:teuthology.orchestra.run.smithi174.stdout:(84/152): python3-oauthlib-3.1.1-5.el9.noarch.r 459 kB/s | 222 kB 00:00 2024-04-26T18:13:37.655 INFO:teuthology.orchestra.run.smithi174.stdout:(85/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.8 MB/s | 5.1 MB 00:00 2024-04-26T18:13:37.681 INFO:teuthology.orchestra.run.smithi174.stdout:(86/152): python3-prettytable-0.7.2-27.el9.noar 361 kB/s | 42 kB 00:00 2024-04-26T18:13:37.731 INFO:teuthology.orchestra.run.smithi174.stdout:(87/152): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-04-26T18:13:37.750 INFO:teuthology.orchestra.run.smithi084.stdout:(36/152): cephadm-19.0.0-3267.gbc94a2a9.el9.noa 4.5 MB/s | 762 kB 00:00 2024-04-26T18:13:37.773 INFO:teuthology.orchestra.run.smithi174.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-26T18:13:37.807 INFO:teuthology.orchestra.run.smithi174.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-04-26T18:13:37.808 INFO:teuthology.orchestra.run.smithi084.stdout:(37/152): cephfs-top-19.0.0-3267.gbc94a2a9.el9. 314 kB/s | 18 kB 00:00 2024-04-26T18:13:37.832 INFO:teuthology.orchestra.run.smithi174.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 507 kB/s | 51 kB 00:00 2024-04-26T18:13:37.857 INFO:teuthology.orchestra.run.smithi174.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 642 kB/s | 54 kB 00:00 2024-04-26T18:13:37.908 INFO:teuthology.orchestra.run.smithi174.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-04-26T18:13:37.933 INFO:teuthology.orchestra.run.smithi174.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 588 kB/s | 44 kB 00:00 2024-04-26T18:13:37.950 INFO:teuthology.orchestra.run.smithi084.stdout:(38/152): fuse-2.9.9-15.el9.x86_64.rpm 568 kB/s | 80 kB 00:00 2024-04-26T18:13:37.983 INFO:teuthology.orchestra.run.smithi174.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-04-26T18:13:38.033 INFO:teuthology.orchestra.run.smithi174.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 1.7 MB/s | 221 kB 00:00 2024-04-26T18:13:38.034 INFO:teuthology.orchestra.run.smithi084.stdout:(39/152): ceph-mgr-diskprediction-local-19.0.0- 5.2 MB/s | 7.4 MB 00:01 2024-04-26T18:13:38.057 INFO:teuthology.orchestra.run.smithi103.stdout:(66/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.0 MB/s | 4.4 MB 00:00 2024-04-26T18:13:38.058 INFO:teuthology.orchestra.run.smithi174.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 632 kB/s | 47 kB 00:00 2024-04-26T18:13:38.126 INFO:teuthology.orchestra.run.smithi174.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 867 kB/s | 58 kB 00:00 2024-04-26T18:13:38.140 INFO:teuthology.orchestra.run.smithi103.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.4 MB/s | 4.7 MB 00:00 2024-04-26T18:13:38.151 INFO:teuthology.orchestra.run.smithi174.stdout:(98/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.5 MB/s | 305 kB 00:00 2024-04-26T18:13:38.167 INFO:teuthology.orchestra.run.smithi084.stdout:(40/152): libgfortran-11.4.1-3.el9.x86_64.rpm 3.6 MB/s | 803 kB 00:00 2024-04-26T18:13:38.201 INFO:teuthology.orchestra.run.smithi084.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-04-26T18:13:38.217 INFO:teuthology.orchestra.run.smithi174.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.0 MB/s | 94 kB 00:00 2024-04-26T18:13:38.243 INFO:teuthology.orchestra.run.smithi174.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 692 kB/s | 64 kB 00:00 2024-04-26T18:13:38.259 INFO:teuthology.orchestra.run.smithi084.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.7 MB/s | 253 kB 00:00 2024-04-26T18:13:38.293 INFO:teuthology.orchestra.run.smithi174.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 595 kB/s | 45 kB 00:00 2024-04-26T18:13:38.318 INFO:teuthology.orchestra.run.smithi084.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-26T18:13:38.319 INFO:teuthology.orchestra.run.smithi174.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 296 kB/s | 22 kB 00:00 2024-04-26T18:13:38.377 INFO:teuthology.orchestra.run.smithi084.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-26T18:13:38.427 INFO:teuthology.orchestra.run.smithi084.stdout:(45/152): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-26T18:13:38.477 INFO:teuthology.orchestra.run.smithi174.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-04-26T18:13:38.485 INFO:teuthology.orchestra.run.smithi084.stdout:(46/152): python3-cryptography-36.0.1-4.el9.x86 4.3 MB/s | 1.2 MB 00:00 2024-04-26T18:13:38.511 INFO:teuthology.orchestra.run.smithi084.stdout:(47/152): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-04-26T18:13:38.624 INFO:teuthology.orchestra.run.smithi103.stdout:(68/152): ceph-test-19.0.0-3267.gbc94a2a9.el9.x 6.3 MB/s | 49 MB 00:07 2024-04-26T18:13:38.628 INFO:teuthology.orchestra.run.smithi084.stdout:(48/152): automake-1.16.2-8.el9.noarch.rpm 4.7 MB/s | 689 kB 00:00 2024-04-26T18:13:38.657 INFO:teuthology.orchestra.run.smithi103.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 1.7 MB/s | 911 kB 00:00 2024-04-26T18:13:38.669 INFO:teuthology.orchestra.run.smithi174.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 2.2 MB/s | 863 kB 00:00 2024-04-26T18:13:38.694 INFO:teuthology.orchestra.run.smithi103.stdout:(70/152): postgresql-13.14-1.el9.x86_64.rpm 2.5 MB/s | 1.6 MB 00:00 2024-04-26T18:13:38.720 INFO:teuthology.orchestra.run.smithi084.stdout:(49/152): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-04-26T18:13:38.745 INFO:teuthology.orchestra.run.smithi084.stdout:(50/152): boost-program-options-1.75.0-8.el9.x8 915 kB/s | 107 kB 00:00 2024-04-26T18:13:38.812 INFO:teuthology.orchestra.run.smithi084.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 234 kB/s | 21 kB 00:00 2024-04-26T18:13:38.846 INFO:teuthology.orchestra.run.smithi084.stdout:(52/152): flex-2.6.4-9.el9.x86_64.rpm 3.0 MB/s | 313 kB 00:00 2024-04-26T18:13:38.879 INFO:teuthology.orchestra.run.smithi084.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 488 kB/s | 33 kB 00:00 2024-04-26T18:13:38.916 INFO:teuthology.orchestra.run.smithi103.stdout:(71/152): postgresql-private-libs-13.14-1.el9.x 456 kB/s | 133 kB 00:00 2024-04-26T18:13:38.928 INFO:teuthology.orchestra.run.smithi174.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 75 kB/s | 19 kB 00:00 2024-04-26T18:13:38.929 INFO:teuthology.orchestra.run.smithi084.stdout:(54/152): flexiblas-openblas-openmp-3.0.4-8.el9 365 kB/s | 18 kB 00:00 2024-04-26T18:13:38.950 INFO:teuthology.orchestra.run.smithi103.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-04-26T18:13:38.963 INFO:teuthology.orchestra.run.smithi084.stdout:(55/152): libaio-devel-0.3.111-13.el9.x86_64.rp 336 kB/s | 11 kB 00:00 2024-04-26T18:13:38.978 INFO:teuthology.orchestra.run.smithi174.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 832 kB/s | 548 kB 00:00 2024-04-26T18:13:38.992 INFO:teuthology.orchestra.run.smithi103.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.1 MB/s | 87 kB 00:00 2024-04-26T18:13:39.003 INFO:teuthology.orchestra.run.smithi174.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 588 kB/s | 309 kB 00:00 2024-04-26T18:13:39.038 INFO:teuthology.orchestra.run.smithi084.stdout:(56/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-04-26T18:13:39.081 INFO:teuthology.orchestra.run.smithi084.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.3 MB/s | 160 kB 00:00 2024-04-26T18:13:39.106 INFO:teuthology.orchestra.run.smithi084.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 674 kB/s | 45 kB 00:00 2024-04-26T18:13:39.128 INFO:teuthology.orchestra.run.smithi174.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 173 kB/s | 26 kB 00:00 2024-04-26T18:13:39.150 INFO:teuthology.orchestra.run.smithi103.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2024-04-26T18:13:39.153 INFO:teuthology.orchestra.run.smithi174.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 323 kB/s | 49 kB 00:00 2024-04-26T18:13:39.189 INFO:teuthology.orchestra.run.smithi084.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 6.0 MB/s | 662 kB 00:00 2024-04-26T18:13:39.214 INFO:teuthology.orchestra.run.smithi084.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-04-26T18:13:39.220 INFO:teuthology.orchestra.run.smithi174.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 734 kB/s | 67 kB 00:00 2024-04-26T18:13:39.242 INFO:teuthology.orchestra.run.smithi103.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.6 MB/s | 248 kB 00:00 2024-04-26T18:13:39.317 INFO:teuthology.orchestra.run.smithi103.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 342 kB/s | 26 kB 00:00 2024-04-26T18:13:39.329 INFO:teuthology.orchestra.run.smithi174.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 862 kB/s | 151 kB 00:00 2024-04-26T18:13:39.453 INFO:teuthology.orchestra.run.smithi103.stdout:(77/152): python3-jsonpointer-2.0-4.el9.noarch. 142 kB/s | 19 kB 00:00 2024-04-26T18:13:39.527 INFO:teuthology.orchestra.run.smithi103.stdout:(78/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.3 MB/s | 172 kB 00:00 2024-04-26T18:13:39.610 INFO:teuthology.orchestra.run.smithi103.stdout:(79/152): python3-markupsafe-1.1.1-12.el9.x86_6 416 kB/s | 35 kB 00:00 2024-04-26T18:13:39.621 INFO:teuthology.orchestra.run.smithi174.stdout:(112/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.0 MB/s | 839 kB 00:00 2024-04-26T18:13:39.646 INFO:teuthology.orchestra.run.smithi174.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 1.7 MB/s | 548 kB 00:00 2024-04-26T18:13:39.665 INFO:teuthology.orchestra.run.smithi084.stdout:(61/152): ceph-test-19.0.0-3267.gbc94a2a9.el9.x 5.5 MB/s | 49 MB 00:08 2024-04-26T18:13:39.690 INFO:teuthology.orchestra.run.smithi084.stdout:(62/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 51 kB/s | 25 kB 00:00 2024-04-26T18:13:39.703 INFO:teuthology.orchestra.run.smithi103.stdout:(80/152): python3-babel-2.9.1-2.el9.noarch.rpm 7.9 MB/s | 6.0 MB 00:00 2024-04-26T18:13:39.715 INFO:teuthology.orchestra.run.smithi084.stdout:(63/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 95 kB/s | 47 kB 00:00 2024-04-26T18:13:39.730 INFO:teuthology.orchestra.run.smithi174.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 400 kB/s | 43 kB 00:00 2024-04-26T18:13:39.755 INFO:teuthology.orchestra.run.smithi174.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 296 kB/s | 32 kB 00:00 2024-04-26T18:13:39.766 INFO:teuthology.orchestra.run.smithi084.stdout:(64/152): openblas-0.3.21-2.el9.x86_64.rpm 707 kB/s | 35 kB 00:00 2024-04-26T18:13:39.786 INFO:teuthology.orchestra.run.smithi103.stdout:(81/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.9 MB/s | 250 kB 00:00 2024-04-26T18:13:39.791 INFO:teuthology.orchestra.run.smithi084.stdout:(65/152): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-04-26T18:13:39.839 INFO:teuthology.orchestra.run.smithi174.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 138 kB/s | 15 kB 00:00 2024-04-26T18:13:39.855 INFO:teuthology.orchestra.run.smithi174.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-26T18:13:39.913 INFO:teuthology.orchestra.run.smithi103.stdout:(82/152): postgresql-server-13.14-1.el9.x86_64. 4.6 MB/s | 5.8 MB 00:01 2024-04-26T18:13:39.937 INFO:teuthology.orchestra.run.smithi103.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-04-26T18:13:39.958 INFO:teuthology.orchestra.run.smithi084.stdout:(66/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.0 MB/s | 292 kB 00:00 2024-04-26T18:13:39.995 INFO:teuthology.orchestra.run.smithi103.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 939 kB/s | 77 kB 00:00 2024-04-26T18:13:40.012 INFO:teuthology.orchestra.run.smithi103.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 562 kB/s | 42 kB 00:00 2024-04-26T18:13:40.016 INFO:teuthology.orchestra.run.smithi084.stdout:(67/152): openblas-openmp-0.3.21-2.el9.x86_64.r 19 MB/s | 4.7 MB 00:00 2024-04-26T18:13:40.031 INFO:teuthology.orchestra.run.smithi174.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 1.8 MB/s | 358 kB 00:00 2024-04-26T18:13:40.056 INFO:teuthology.orchestra.run.smithi174.stdout:(119/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.5 MB/s | 303 kB 00:00 2024-04-26T18:13:40.075 INFO:teuthology.orchestra.run.smithi084.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-04-26T18:13:40.079 INFO:teuthology.orchestra.run.smithi103.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.2 MB/s | 269 kB 00:00 2024-04-26T18:13:40.105 INFO:teuthology.orchestra.run.smithi103.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-04-26T18:13:40.142 INFO:teuthology.orchestra.run.smithi084.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-26T18:13:40.155 INFO:teuthology.orchestra.run.smithi103.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-04-26T18:13:40.157 INFO:teuthology.orchestra.run.smithi174.stdout:(120/152): python3-colorama-0.4.4-12.el9.noarch 282 kB/s | 35 kB 00:00 2024-04-26T18:13:40.180 INFO:teuthology.orchestra.run.smithi103.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-04-26T18:13:40.184 INFO:teuthology.orchestra.run.smithi084.stdout:(70/152): postgresql-contrib-13.14-1.el9.x86_64 5.3 MB/s | 911 kB 00:00 2024-04-26T18:13:40.222 INFO:teuthology.orchestra.run.smithi103.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 803 kB/s | 54 kB 00:00 2024-04-26T18:13:40.223 INFO:teuthology.orchestra.run.smithi174.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 1.2 MB/s | 206 kB 00:00 2024-04-26T18:13:40.259 INFO:teuthology.orchestra.run.smithi084.stdout:(71/152): postgresql-13.14-1.el9.x86_64.rpm 5.3 MB/s | 1.6 MB 00:00 2024-04-26T18:13:40.289 INFO:teuthology.orchestra.run.smithi103.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 627 kB/s | 42 kB 00:00 2024-04-26T18:13:40.318 INFO:teuthology.orchestra.run.smithi084.stdout:(72/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-26T18:13:40.332 INFO:teuthology.orchestra.run.smithi174.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-26T18:13:40.351 INFO:teuthology.orchestra.run.smithi084.stdout:(73/152): protobuf-3.14.0-13.el9.x86_64.rpm 6.0 MB/s | 1.0 MB 00:00 2024-04-26T18:13:40.356 INFO:teuthology.orchestra.run.smithi103.stdout:(92/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 663 kB/s | 44 kB 00:00 2024-04-26T18:13:40.431 INFO:teuthology.orchestra.run.smithi103.stdout:(93/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-26T18:13:40.457 INFO:teuthology.orchestra.run.smithi174.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 85 kB/s | 11 kB 00:00 2024-04-26T18:13:40.498 INFO:teuthology.orchestra.run.smithi103.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 538 kB/s | 36 kB 00:00 2024-04-26T18:13:40.567 INFO:teuthology.orchestra.run.smithi174.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 163 kB/s | 18 kB 00:00 2024-04-26T18:13:40.569 INFO:teuthology.orchestra.run.smithi084.stdout:(74/152): postgresql-server-13.14-1.el9.x86_64. 14 MB/s | 5.8 MB 00:00 2024-04-26T18:13:40.582 INFO:teuthology.orchestra.run.smithi103.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 569 kB/s | 47 kB 00:00 2024-04-26T18:13:40.593 INFO:teuthology.orchestra.run.smithi084.stdout:(75/152): python3-devel-3.9.18-3.el9.x86_64.rpm 1.0 MB/s | 244 kB 00:00 2024-04-26T18:13:40.649 INFO:teuthology.orchestra.run.smithi103.stdout:(96/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-26T18:13:40.652 INFO:teuthology.orchestra.run.smithi084.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 441 kB/s | 26 kB 00:00 2024-04-26T18:13:40.658 INFO:teuthology.orchestra.run.smithi174.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 253 kB/s | 23 kB 00:00 2024-04-26T18:13:40.686 INFO:teuthology.orchestra.run.smithi084.stdout:(77/152): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-26T18:13:40.707 INFO:teuthology.orchestra.run.smithi103.stdout:(97/152): uuid-1.6.2-55.el9.x86_64.rpm 995 kB/s | 58 kB 00:00 2024-04-26T18:13:40.728 INFO:teuthology.orchestra.run.smithi084.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 254 kB/s | 19 kB 00:00 2024-04-26T18:13:40.742 INFO:teuthology.orchestra.run.smithi174.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-04-26T18:13:40.766 INFO:teuthology.orchestra.run.smithi103.stdout:(98/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-26T18:13:40.769 INFO:teuthology.orchestra.run.smithi084.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.0 MB/s | 172 kB 00:00 2024-04-26T18:13:40.795 INFO:teuthology.orchestra.run.smithi084.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 518 kB/s | 35 kB 00:00 2024-04-26T18:13:40.825 INFO:teuthology.orchestra.run.smithi103.stdout:(99/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-26T18:13:40.850 INFO:teuthology.orchestra.run.smithi174.stdout:(127/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.9 MB/s | 2.0 MB 00:00 2024-04-26T18:13:40.876 INFO:teuthology.orchestra.run.smithi174.stdout:(128/152): python3-jaraco-text-3.2.0-6.el9.noar 147 kB/s | 20 kB 00:00 2024-04-26T18:13:40.895 INFO:teuthology.orchestra.run.smithi084.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-26T18:13:40.917 INFO:teuthology.orchestra.run.smithi103.stdout:(100/152): python3-numpy-1.20.1-5.el9.x86_64.rp 3.9 MB/s | 5.1 MB 00:01 2024-04-26T18:13:40.942 INFO:teuthology.orchestra.run.smithi103.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 384 kB/s | 45 kB 00:00 2024-04-26T18:13:40.945 INFO:teuthology.orchestra.run.smithi084.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-26T18:13:40.959 INFO:teuthology.orchestra.run.smithi174.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 558 kB/s | 46 kB 00:00 2024-04-26T18:13:40.967 INFO:teuthology.orchestra.run.smithi103.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 444 kB/s | 22 kB 00:00 2024-04-26T18:13:40.996 INFO:teuthology.orchestra.run.smithi084.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-04-26T18:13:41.021 INFO:teuthology.orchestra.run.smithi084.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-04-26T18:13:41.042 INFO:teuthology.orchestra.run.smithi103.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 8.4 MB/s | 863 kB 00:00 2024-04-26T18:13:41.071 INFO:teuthology.orchestra.run.smithi084.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 562 kB/s | 42 kB 00:00 2024-04-26T18:13:41.135 INFO:teuthology.orchestra.run.smithi174.stdout:(130/152): python3-more-itertools-8.12.0-2.el9. 448 kB/s | 79 kB 00:00 2024-04-26T18:13:41.138 INFO:teuthology.orchestra.run.smithi084.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.2 MB/s | 269 kB 00:00 2024-04-26T18:13:41.180 INFO:teuthology.orchestra.run.smithi084.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.4 MB/s | 159 kB 00:00 2024-04-26T18:13:41.239 INFO:teuthology.orchestra.run.smithi084.stdout:(88/152): python3-pytz-2021.1-5.el9.noarch.rpm 874 kB/s | 51 kB 00:00 2024-04-26T18:13:41.243 INFO:teuthology.orchestra.run.smithi103.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-04-26T18:13:41.264 INFO:teuthology.orchestra.run.smithi084.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.2 MB/s | 279 kB 00:00 2024-04-26T18:13:41.285 INFO:teuthology.orchestra.run.smithi174.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 2.5 MB/s | 1.0 MB 00:00 2024-04-26T18:13:41.302 INFO:teuthology.orchestra.run.smithi174.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 347 kB/s | 58 kB 00:00 2024-04-26T18:13:41.331 INFO:teuthology.orchestra.run.smithi084.stdout:(90/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.1 MB/s | 5.1 MB 00:00 2024-04-26T18:13:41.356 INFO:teuthology.orchestra.run.smithi084.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 457 kB/s | 54 kB 00:00 2024-04-26T18:13:41.368 INFO:teuthology.orchestra.run.smithi174.stdout:(133/152): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-04-26T18:13:41.385 INFO:teuthology.orchestra.run.smithi174.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 196 kB/s | 16 kB 00:00 2024-04-26T18:13:41.407 INFO:teuthology.orchestra.run.smithi084.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 554 kB/s | 42 kB 00:00 2024-04-26T18:13:41.432 INFO:teuthology.orchestra.run.smithi084.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-04-26T18:13:41.452 INFO:teuthology.orchestra.run.smithi174.stdout:(135/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-04-26T18:13:41.478 INFO:teuthology.orchestra.run.smithi174.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 307 kB/s | 28 kB 00:00 2024-04-26T18:13:41.482 INFO:teuthology.orchestra.run.smithi084.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 720 kB/s | 36 kB 00:00 2024-04-26T18:13:41.508 INFO:teuthology.orchestra.run.smithi084.stdout:(95/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-04-26T18:13:41.526 INFO:teuthology.orchestra.run.smithi103.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 69 kB/s | 19 kB 00:00 2024-04-26T18:13:41.536 INFO:teuthology.orchestra.run.smithi174.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 368 kB/s | 31 kB 00:00 2024-04-26T18:13:41.561 INFO:teuthology.orchestra.run.smithi174.stdout:(138/152): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-26T18:13:41.566 INFO:teuthology.orchestra.run.smithi084.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-04-26T18:13:41.620 INFO:teuthology.orchestra.run.smithi174.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 708 kB/s | 59 kB 00:00 2024-04-26T18:13:41.633 INFO:teuthology.orchestra.run.smithi084.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 2.4 MB/s | 305 kB 00:00 2024-04-26T18:13:41.635 INFO:teuthology.orchestra.run.smithi103.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 821 kB/s | 548 kB 00:00 2024-04-26T18:13:41.646 INFO:teuthology.orchestra.run.smithi174.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 428 kB/s | 36 kB 00:00 2024-04-26T18:13:41.666 INFO:teuthology.orchestra.run.smithi084.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 576 kB/s | 58 kB 00:00 2024-04-26T18:13:41.668 INFO:teuthology.orchestra.run.smithi103.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 493 kB/s | 309 kB 00:00 2024-04-26T18:13:41.704 INFO:teuthology.orchestra.run.smithi174.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 611 kB/s | 51 kB 00:00 2024-04-26T18:13:41.750 INFO:teuthology.orchestra.run.smithi084.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 801 kB/s | 94 kB 00:00 2024-04-26T18:13:41.775 INFO:teuthology.orchestra.run.smithi084.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 586 kB/s | 64 kB 00:00 2024-04-26T18:13:41.796 INFO:teuthology.orchestra.run.smithi174.stdout:(142/152): python3-websocket-client-1.2.3-2.el9 981 kB/s | 90 kB 00:00 2024-04-26T18:13:41.821 INFO:teuthology.orchestra.run.smithi174.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-04-26T18:13:41.826 INFO:teuthology.orchestra.run.smithi084.stdout:(101/152): lua-devel-5.4.4-4.el9.x86_64.rpm 445 kB/s | 22 kB 00:00 2024-04-26T18:13:41.859 INFO:teuthology.orchestra.run.smithi084.stdout:(102/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 411 kB/s | 45 kB 00:00 2024-04-26T18:13:41.902 INFO:teuthology.orchestra.run.smithi103.stdout:(108/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 97 kB/s | 26 kB 00:00 2024-04-26T18:13:41.963 INFO:teuthology.orchestra.run.smithi174.stdout:(144/152): libarrow-9.0.0-11.el9.x86_64.rpm 1.5 MB/s | 4.4 MB 00:03 2024-04-26T18:13:41.989 INFO:teuthology.orchestra.run.smithi174.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 120 kB/s | 20 kB 00:00 2024-04-26T18:13:42.014 INFO:teuthology.orchestra.run.smithi174.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.9 MB/s | 427 kB 00:00 2024-04-26T18:13:42.028 INFO:teuthology.orchestra.run.smithi103.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 135 kB/s | 49 kB 00:00 2024-04-26T18:13:42.035 INFO:teuthology.orchestra.run.smithi084.stdout:(103/152): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-04-26T18:13:42.055 INFO:teuthology.orchestra.run.smithi174.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-26T18:13:42.185 INFO:teuthology.orchestra.run.smithi084.stdout:(104/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-26T18:13:42.253 INFO:teuthology.orchestra.run.smithi103.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 192 kB/s | 67 kB 00:00 2024-04-26T18:13:42.272 INFO:teuthology.orchestra.run.smithi174.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-04-26T18:13:42.453 INFO:teuthology.orchestra.run.smithi103.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-04-26T18:13:42.556 INFO:teuthology.orchestra.run.smithi174.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 4.8 MB/s | 1.3 MB 00:00 2024-04-26T18:13:42.668 INFO:teuthology.orchestra.run.smithi084.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 42 kB/s | 19 kB 00:00 2024-04-26T18:13:42.865 INFO:teuthology.orchestra.run.smithi174.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.8 MB/s | 2.4 MB 00:00 2024-04-26T18:13:42.912 INFO:teuthology.orchestra.run.smithi103.stdout:(112/152): libarrow-9.0.0-11.el9.x86_64.rpm 3.2 MB/s | 4.4 MB 00:01 2024-04-26T18:13:42.945 INFO:teuthology.orchestra.run.smithi103.stdout:(113/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 1.2 MB/s | 839 kB 00:00 2024-04-26T18:13:42.985 INFO:teuthology.orchestra.run.smithi084.stdout:(106/152): gperftools-libs-2.9.1-2.el9.x86_64.r 325 kB/s | 309 kB 00:00 2024-04-26T18:13:43.079 INFO:teuthology.orchestra.run.smithi103.stdout:(114/152): python3-asyncssh-2.13.2-5.el9.noarch 876 kB/s | 548 kB 00:00 2024-04-26T18:13:43.094 INFO:teuthology.orchestra.run.smithi084.stdout:(107/152): libarrow-doc-9.0.0-11.el9.noarch.rpm 240 kB/s | 26 kB 00:00 2024-04-26T18:13:43.177 INFO:teuthology.orchestra.run.smithi084.stdout:(108/152): abseil-cpp-20211102.0-3.el9.x86_64.r 416 kB/s | 548 kB 00:01 2024-04-26T18:13:43.203 INFO:teuthology.orchestra.run.smithi084.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 448 kB/s | 49 kB 00:00 2024-04-26T18:13:43.221 INFO:teuthology.orchestra.run.smithi103.stdout:(115/152): python3-cachetools-4.2.4-1.el9.noarc 117 kB/s | 32 kB 00:00 2024-04-26T18:13:43.263 INFO:teuthology.orchestra.run.smithi103.stdout:(116/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 124 kB/s | 43 kB 00:00 2024-04-26T18:13:43.346 INFO:teuthology.orchestra.run.smithi103.stdout:(117/152): python3-certifi-2021.10.8-2.el9.noar 56 kB/s | 15 kB 00:00 2024-04-26T18:13:43.361 INFO:teuthology.orchestra.run.smithi084.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 367 kB/s | 67 kB 00:00 2024-04-26T18:13:43.470 INFO:teuthology.orchestra.run.smithi084.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 566 kB/s | 151 kB 00:00 2024-04-26T18:13:43.647 INFO:teuthology.orchestra.run.smithi103.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 404 kB/s | 172 kB 00:00 2024-04-26T18:13:43.772 INFO:teuthology.orchestra.run.smithi103.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 705 kB/s | 358 kB 00:00 2024-04-26T18:13:43.847 INFO:teuthology.orchestra.run.smithi103.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 605 kB/s | 303 kB 00:00 2024-04-26T18:13:43.914 INFO:teuthology.orchestra.run.smithi103.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-04-26T18:13:44.057 INFO:teuthology.orchestra.run.smithi174.stdout:(151/152): librbd1-19.0.0-3267.gbc94a2a9.el9.x8 1.6 MB/s | 3.1 MB 00:02 2024-04-26T18:13:44.212 INFO:teuthology.orchestra.run.smithi084.stdout:(112/152): python3-asyncssh-2.13.2-5.el9.noarch 739 kB/s | 548 kB 00:00 2024-04-26T18:13:44.281 INFO:teuthology.orchestra.run.smithi103.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 404 kB/s | 206 kB 00:00 2024-04-26T18:13:44.312 INFO:teuthology.orchestra.run.smithi084.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 435 kB/s | 43 kB 00:00 2024-04-26T18:13:44.315 INFO:teuthology.orchestra.run.smithi174.stdout:(152/152): librados2-19.0.0-3267.gbc94a2a9.el9. 1.5 MB/s | 3.4 MB 00:02 2024-04-26T18:13:44.318 INFO:teuthology.orchestra.run.smithi174.stdout:-------------------------------------------------------------------------------- 2024-04-26T18:13:44.318 INFO:teuthology.orchestra.run.smithi174.stdout:Total 10 MB/s | 222 MB 00:21 2024-04-26T18:13:44.331 INFO:teuthology.orchestra.run.smithi103.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 346 kB/s | 144 kB 00:00 2024-04-26T18:13:44.412 INFO:teuthology.orchestra.run.smithi084.stdout:(114/152): python3-cachetools-4.2.4-1.el9.noarc 322 kB/s | 32 kB 00:00 2024-04-26T18:13:44.446 INFO:teuthology.orchestra.run.smithi084.stdout:(115/152): parquet-libs-9.0.0-11.el9.x86_64.rpm 774 kB/s | 839 kB 00:01 2024-04-26T18:13:44.465 INFO:teuthology.orchestra.run.smithi103.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-04-26T18:13:44.505 INFO:teuthology.orchestra.run.smithi084.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-26T18:13:44.650 INFO:teuthology.orchestra.run.smithi103.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 56 kB/s | 18 kB 00:00 2024-04-26T18:13:44.713 INFO:teuthology.orchestra.run.smithi084.stdout:(117/152): python3-cheroot-8.6.0-4.el9.noarch.r 643 kB/s | 172 kB 00:00 2024-04-26T18:13:44.758 INFO:teuthology.orchestra.run.smithi103.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 2.2 MB/s | 2.0 MB 00:00 2024-04-26T18:13:44.774 INFO:teuthology.orchestra.run.smithi103.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 75 kB/s | 23 kB 00:00 2024-04-26T18:13:44.933 INFO:teuthology.orchestra.run.smithi103.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 68 kB/s | 19 kB 00:00 2024-04-26T18:13:45.005 INFO:teuthology.orchestra.run.smithi084.stdout:(118/152): python3-cherrypy-18.6.1-2.el9.noarch 716 kB/s | 358 kB 00:00 2024-04-26T18:13:45.025 INFO:teuthology.orchestra.run.smithi103.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 74 kB/s | 20 kB 00:00 2024-04-26T18:13:45.106 INFO:teuthology.orchestra.run.smithi084.stdout:(119/152): python3-colorama-0.4.4-12.el9.noarch 353 kB/s | 35 kB 00:00 2024-04-26T18:13:45.139 INFO:teuthology.orchestra.run.smithi084.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 712 kB/s | 303 kB 00:00 2024-04-26T18:13:45.283 INFO:teuthology.orchestra.run.smithi103.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 132 kB/s | 46 kB 00:00 2024-04-26T18:13:45.391 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction check 2024-04-26T18:13:45.451 INFO:teuthology.orchestra.run.smithi103.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 185 kB/s | 79 kB 00:00 2024-04-26T18:13:45.456 INFO:teuthology.orchestra.run.smithi084.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 587 kB/s | 206 kB 00:00 2024-04-26T18:13:45.532 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction check succeeded. 2024-04-26T18:13:45.533 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction test 2024-04-26T18:13:45.626 INFO:teuthology.orchestra.run.smithi103.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 169 kB/s | 58 kB 00:00 2024-04-26T18:13:45.740 INFO:teuthology.orchestra.run.smithi084.stdout:(122/152): python3-grpcio-tools-1.46.7-10.el9.x 510 kB/s | 144 kB 00:00 2024-04-26T18:13:45.857 INFO:teuthology.orchestra.run.smithi084.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 92 kB/s | 11 kB 00:00 2024-04-26T18:13:45.901 INFO:teuthology.orchestra.run.smithi103.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 928 kB/s | 1.0 MB 00:01 2024-04-26T18:13:45.926 INFO:teuthology.orchestra.run.smithi103.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 55 kB/s | 16 kB 00:00 2024-04-26T18:13:45.957 INFO:teuthology.orchestra.run.smithi084.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 177 kB/s | 18 kB 00:00 2024-04-26T18:13:45.960 INFO:teuthology.orchestra.run.smithi103.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 535 kB/s | 272 kB 00:00 2024-04-26T18:13:46.116 INFO:teuthology.orchestra.run.smithi084.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 146 kB/s | 23 kB 00:00 2024-04-26T18:13:46.202 INFO:teuthology.orchestra.run.smithi103.stdout:(136/152): python3-pyperclip-1.8.2-2.el9.noarch 102 kB/s | 28 kB 00:00 2024-04-26T18:13:46.216 INFO:teuthology.orchestra.run.smithi084.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-04-26T18:13:46.227 INFO:teuthology.orchestra.run.smithi103.stdout:(137/152): python3-repoze-lru-0.7-16.el9.noarch 115 kB/s | 31 kB 00:00 2024-04-26T18:13:46.316 INFO:teuthology.orchestra.run.smithi084.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2024-04-26T18:13:46.377 INFO:teuthology.orchestra.run.smithi103.stdout:(138/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 189 kB/s | 90 kB 00:00 2024-04-26T18:13:46.569 INFO:teuthology.orchestra.run.smithi103.stdout:(139/152): python3-rsa-4.9-2.el9.noarch.rpm 173 kB/s | 59 kB 00:00 2024-04-26T18:13:46.628 INFO:teuthology.orchestra.run.smithi103.stdout:(140/152): python3-routes-2.5.1-5.el9.noarch.rp 443 kB/s | 188 kB 00:00 2024-04-26T18:13:46.653 INFO:teuthology.orchestra.run.smithi103.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 130 kB/s | 36 kB 00:00 2024-04-26T18:13:46.920 INFO:teuthology.orchestra.run.smithi103.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 145 kB/s | 51 kB 00:00 2024-04-26T18:13:47.079 INFO:teuthology.orchestra.run.smithi103.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 211 kB/s | 90 kB 00:00 2024-04-26T18:13:47.179 INFO:teuthology.orchestra.run.smithi103.stdout:(144/152): python3-webob-1.8.7-6.el9.noarch.rpm 417 kB/s | 230 kB 00:00 2024-04-26T18:13:47.349 INFO:teuthology.orchestra.run.smithi174.stdout:Transaction test succeeded. 2024-04-26T18:13:47.350 INFO:teuthology.orchestra.run.smithi174.stdout:Running transaction 2024-04-26T18:13:47.413 INFO:teuthology.orchestra.run.smithi103.stdout:(145/152): python3-zc-lockfile-2.0-10.el9.noarc 60 kB/s | 20 kB 00:00 2024-04-26T18:13:47.521 INFO:teuthology.orchestra.run.smithi103.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 711 kB/s | 427 kB 00:00 2024-04-26T18:13:47.616 INFO:teuthology.orchestra.run.smithi084.stdout:(128/152): python3-kubernetes-26.1.0-2.el9.noar 804 kB/s | 1.0 MB 00:01 2024-04-26T18:13:47.658 INFO:teuthology.orchestra.run.smithi084.stdout:(129/152): python3-grpcio-1.46.7-10.el9.x86_64. 828 kB/s | 2.0 MB 00:02 2024-04-26T18:13:47.688 INFO:teuthology.orchestra.run.smithi103.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 376 kB/s | 192 kB 00:00 2024-04-26T18:13:47.725 INFO:teuthology.orchestra.run.smithi084.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 426 kB/s | 46 kB 00:00 2024-04-26T18:13:47.850 INFO:teuthology.orchestra.run.smithi084.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 411 kB/s | 79 kB 00:00 2024-04-26T18:13:47.901 INFO:teuthology.orchestra.run.smithi084.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 329 kB/s | 58 kB 00:00 2024-04-26T18:13:48.001 INFO:teuthology.orchestra.run.smithi084.stdout:(133/152): python3-portend-3.1.0-2.el9.noarch.r 164 kB/s | 16 kB 00:00 2024-04-26T18:13:48.243 INFO:teuthology.orchestra.run.smithi084.stdout:(134/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 372 kB/s | 90 kB 00:00 2024-04-26T18:13:48.288 INFO:teuthology.orchestra.run.smithi103.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 1.8 MB/s | 1.6 MB 00:00 2024-04-26T18:13:48.360 INFO:teuthology.orchestra.run.smithi084.stdout:(135/152): python3-pecan-1.4.2-3.el9.noarch.rpm 535 kB/s | 272 kB 00:00 2024-04-26T18:13:48.435 INFO:teuthology.orchestra.run.smithi084.stdout:(136/152): libarrow-9.0.0-11.el9.x86_64.rpm 784 kB/s | 4.4 MB 00:05 2024-04-26T18:13:48.460 INFO:teuthology.orchestra.run.smithi084.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 130 kB/s | 28 kB 00:00 2024-04-26T18:13:48.485 INFO:teuthology.orchestra.run.smithi084.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 245 kB/s | 31 kB 00:00 2024-04-26T18:13:48.497 INFO:teuthology.orchestra.run.smithi103.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 1.3 MB 00:00 2024-04-26T18:13:48.610 INFO:teuthology.orchestra.run.smithi084.stdout:(139/152): python3-tempora-5.0.0-2.el9.noarch.r 286 kB/s | 36 kB 00:00 2024-04-26T18:13:48.689 INFO:teuthology.orchestra.run.smithi103.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 13 MB/s | 2.4 MB 00:00 2024-04-26T18:13:48.694 INFO:teuthology.orchestra.run.smithi084.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 253 kB/s | 59 kB 00:00 2024-04-26T18:13:48.728 INFO:teuthology.orchestra.run.smithi084.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 436 kB/s | 51 kB 00:00 2024-04-26T18:13:48.881 INFO:teuthology.orchestra.run.smithi103.stdout:(151/152): librados2-19.0.0-3267.gbc94a2a9.el9. 2.5 MB/s | 3.4 MB 00:01 2024-04-26T18:13:48.903 INFO:teuthology.orchestra.run.smithi084.stdout:(142/152): python3-routes-2.5.1-5.el9.noarch.rp 426 kB/s | 188 kB 00:00 2024-04-26T18:13:48.939 INFO:teuthology.orchestra.run.smithi103.stdout:(152/152): librbd1-19.0.0-3267.gbc94a2a9.el9.x8 2.5 MB/s | 3.1 MB 00:01 2024-04-26T18:13:48.942 INFO:teuthology.orchestra.run.smithi103.stdout:-------------------------------------------------------------------------------- 2024-04-26T18:13:48.942 INFO:teuthology.orchestra.run.smithi103.stdout:Total 8.7 MB/s | 222 MB 00:25 2024-04-26T18:13:48.961 INFO:teuthology.orchestra.run.smithi084.stdout:(143/152): python3-websocket-client-1.2.3-2.el9 384 kB/s | 90 kB 00:00 2024-04-26T18:13:49.078 INFO:teuthology.orchestra.run.smithi084.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 171 kB/s | 20 kB 00:00 2024-04-26T18:13:49.137 INFO:teuthology.orchestra.run.smithi084.stdout:(145/152): python3-webob-1.8.7-6.el9.noarch.rpm 519 kB/s | 230 kB 00:00 2024-04-26T18:13:49.487 INFO:teuthology.orchestra.run.smithi084.stdout:(146/152): re2-20211101-3.el9.x86_64.rpm 469 kB/s | 192 kB 00:00 2024-04-26T18:13:49.604 INFO:teuthology.orchestra.run.smithi084.stdout:(147/152): python3-werkzeug-2.0.3-3.el9.1.noarc 609 kB/s | 427 kB 00:00 2024-04-26T18:13:50.017 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction check 2024-04-26T18:13:50.163 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction check succeeded. 2024-04-26T18:13:50.164 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction test 2024-04-26T18:13:50.355 INFO:teuthology.orchestra.run.smithi084.stdout:(148/152): librados2-19.0.0-3267.gbc94a2a9.el9. 3.9 MB/s | 3.4 MB 00:00 2024-04-26T18:13:50.621 INFO:teuthology.orchestra.run.smithi084.stdout:(149/152): librbd1-19.0.0-3267.gbc94a2a9.el9.x8 3.1 MB/s | 3.1 MB 00:01 2024-04-26T18:13:50.672 INFO:teuthology.orchestra.run.smithi084.stdout:(150/152): openssl-3.2.1-1.el9.x86_64.rpm 4.3 MB/s | 1.3 MB 00:00 2024-04-26T18:13:50.982 INFO:teuthology.orchestra.run.smithi084.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.7 MB/s | 2.4 MB 00:00 2024-04-26T18:13:51.355 INFO:teuthology.orchestra.run.smithi084.stdout:(152/152): thrift-0.15.0-2.el9.x86_64.rpm 730 kB/s | 1.6 MB 00:02 2024-04-26T18:13:51.362 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-04-26T18:13:51.362 INFO:teuthology.orchestra.run.smithi084.stdout:Total 8.3 MB/s | 222 MB 00:26 2024-04-26T18:13:52.089 INFO:teuthology.orchestra.run.smithi103.stdout:Transaction test succeeded. 2024-04-26T18:13:52.089 INFO:teuthology.orchestra.run.smithi103.stdout:Running transaction 2024-04-26T18:13:52.470 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction check 2024-04-26T18:13:52.611 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction check succeeded. 2024-04-26T18:13:52.611 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction test 2024-04-26T18:13:53.251 INFO:teuthology.orchestra.run.smithi174.stdout: Preparing : 1/1 2024-04-26T18:13:53.335 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T18:13:53.432 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T18:13:53.603 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T18:13:53.653 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T18:13:53.695 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T18:13:53.738 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T18:13:54.067 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T18:13:54.171 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:13:54.301 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:13:54.330 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:13:54.378 INFO:teuthology.orchestra.run.smithi084.stdout:Transaction test succeeded. 2024-04-26T18:13:54.378 INFO:teuthology.orchestra.run.smithi084.stdout:Running transaction 2024-04-26T18:13:54.430 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:13:54.468 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T18:13:54.613 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T18:13:54.666 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T18:13:54.684 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:13:54.787 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:13:54.810 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T18:13:54.856 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T18:13:54.878 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T18:13:54.902 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T18:13:55.102 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T18:13:55.136 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T18:13:55.234 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T18:13:55.338 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T18:13:55.376 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T18:13:55.410 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T18:13:55.474 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T18:13:55.509 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T18:13:55.553 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T18:13:55.586 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T18:13:55.614 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T18:13:55.682 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T18:13:55.735 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T18:13:55.766 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T18:13:55.801 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T18:13:55.876 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T18:13:55.912 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 31/156 2024-04-26T18:13:55.956 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 32/156 2024-04-26T18:13:55.984 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 33/156 2024-04-26T18:13:56.054 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T18:13:56.099 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T18:13:56.173 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T18:13:56.303 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T18:13:56.359 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T18:13:56.875 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T18:13:56.955 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T18:13:56.984 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T18:13:57.018 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T18:13:57.057 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 43/156 2024-04-26T18:13:57.099 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T18:13:57.248 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T18:13:57.309 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T18:13:57.364 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T18:13:57.444 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T18:13:57.476 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T18:13:57.513 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T18:13:57.559 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T18:13:57.595 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T18:13:57.639 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T18:13:57.680 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T18:13:57.814 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T18:13:57.861 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T18:13:57.891 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T18:13:57.932 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T18:13:58.027 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T18:13:58.058 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T18:13:58.097 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T18:13:58.109 INFO:teuthology.orchestra.run.smithi103.stdout: Preparing : 1/1 2024-04-26T18:13:58.131 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T18:13:58.156 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T18:13:58.201 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T18:13:58.229 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T18:13:58.239 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T18:13:58.301 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T18:13:58.336 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T18:13:58.362 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T18:13:58.364 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T18:13:58.397 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T18:13:58.404 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T18:13:58.424 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T18:13:58.459 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T18:13:58.498 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T18:13:58.529 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T18:13:58.559 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T18:13:58.593 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T18:13:58.627 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T18:13:58.660 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T18:13:58.695 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T18:13:58.732 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T18:13:58.781 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T18:13:58.825 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T18:13:58.837 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:13:58.856 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T18:13:58.904 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T18:13:58.959 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:13:58.980 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:13:59.000 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T18:13:59.057 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T18:13:59.071 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:13:59.101 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T18:13:59.248 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T18:13:59.307 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T18:13:59.343 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:13:59.447 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:13:59.469 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T18:13:59.531 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T18:13:59.553 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T18:13:59.573 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T18:13:59.745 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T18:13:59.770 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T18:13:59.778 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T18:13:59.813 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T18:13:59.847 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T18:13:59.890 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T18:13:59.945 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T18:13:59.986 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T18:13:59.996 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T18:14:00.011 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T18:14:00.046 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T18:14:00.084 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T18:14:00.139 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T18:14:00.140 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T18:14:00.178 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T18:14:00.223 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T18:14:00.255 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T18:14:00.282 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T18:14:00.317 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T18:14:00.346 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T18:14:00.350 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T18:14:00.405 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T18:14:00.423 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T18:14:00.444 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T18:14:00.479 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T18:14:00.525 INFO:teuthology.orchestra.run.smithi084.stdout: Preparing : 1/1 2024-04-26T18:14:00.554 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T18:14:00.590 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 31/156 2024-04-26T18:14:00.600 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-26T18:14:00.633 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 32/156 2024-04-26T18:14:00.638 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-26T18:14:00.662 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 33/156 2024-04-26T18:14:00.700 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-26T18:14:00.731 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T18:14:00.734 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-26T18:14:00.759 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-26T18:14:00.777 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T18:14:00.802 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-26T18:14:00.850 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T18:14:00.993 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T18:14:01.046 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T18:14:01.057 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T18:14:01.122 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-26T18:14:01.193 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T18:14:01.236 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:14:01.357 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:14:01.378 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:14:01.478 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:14:01.508 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T18:14:01.557 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T18:14:01.650 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T18:14:01.655 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-26T18:14:01.679 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T18:14:01.704 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-26T18:14:01.722 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T18:14:01.732 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:14:01.760 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 43/156 2024-04-26T18:14:01.802 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T18:14:01.820 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:14:01.850 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T18:14:01.904 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 14/156 2024-04-26T18:14:01.926 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T18:14:01.946 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-26T18:14:01.968 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T18:14:02.022 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T18:14:02.076 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T18:14:02.166 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T18:14:02.186 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T18:14:02.206 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T18:14:02.218 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-26T18:14:02.242 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T18:14:02.289 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T18:14:02.325 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T18:14:02.327 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-26T18:14:02.368 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T18:14:02.402 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T18:14:02.411 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T18:14:02.422 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 18/156 2024-04-26T18:14:02.450 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-26T18:14:02.485 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-26T18:14:02.499 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T18:14:02.527 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T18:14:02.546 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T18:14:02.550 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-26T18:14:02.560 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T18:14:02.584 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-26T18:14:02.591 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T18:14:02.621 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T18:14:02.628 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-26T18:14:02.661 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-26T18:14:02.663 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T18:14:02.688 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-26T18:14:02.758 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-26T18:14:02.758 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T18:14:02.788 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T18:14:02.810 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-26T18:14:02.828 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T18:14:02.841 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-26T18:14:02.854 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T18:14:02.873 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T18:14:02.876 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-26T18:14:02.879 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T18:14:02.906 INFO:teuthology.orchestra.run.smithi174.stdout: Upgrading : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T18:14:02.952 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T18:14:02.953 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-26T18:14:02.997 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T18:14:03.028 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 100/156 2024-04-26T18:14:03.067 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T18:14:03.080 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T18:14:03.112 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T18:14:03.140 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T18:14:03.162 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 31/156 2024-04-26T18:14:03.174 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T18:14:03.206 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 32/156 2024-04-26T18:14:03.213 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T18:14:03.234 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 33/156 2024-04-26T18:14:03.245 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T18:14:03.275 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T18:14:03.304 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-26T18:14:03.309 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T18:14:03.343 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T18:14:03.349 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-26T18:14:03.384 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T18:14:03.419 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T18:14:03.423 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-26T18:14:03.505 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T18:14:03.534 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T18:14:03.549 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T18:14:03.553 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-26T18:14:03.567 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T18:14:03.589 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T18:14:03.609 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-26T18:14:03.635 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T18:14:03.638 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T18:14:03.661 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 104/156 2024-04-26T18:14:03.734 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T18:14:03.782 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T18:14:04.135 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-26T18:14:04.222 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-26T18:14:04.251 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-26T18:14:04.294 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-26T18:14:04.331 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 43/156 2024-04-26T18:14:04.373 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-26T18:14:04.501 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T18:14:04.525 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-26T18:14:04.554 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T18:14:04.576 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-26T18:14:04.588 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T18:14:04.648 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-26T18:14:04.687 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T18:14:04.720 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-26T18:14:04.738 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T18:14:04.752 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-26T18:14:04.789 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-26T18:14:04.825 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T18:14:04.836 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-26T18:14:04.871 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-26T18:14:04.874 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T18:14:04.914 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-26T18:14:04.956 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-26T18:14:05.053 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T18:14:05.080 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T18:14:05.091 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-26T18:14:05.137 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-26T18:14:05.164 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T18:14:05.167 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-26T18:14:05.209 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-26T18:14:05.301 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-26T18:14:05.334 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-26T18:14:05.374 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-26T18:14:05.408 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-26T18:14:05.433 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 63/156 2024-04-26T18:14:05.498 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-26T18:14:05.625 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-26T18:14:05.665 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-26T18:14:05.693 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-26T18:14:05.727 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-26T18:14:05.766 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-26T18:14:05.798 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-26T18:14:05.799 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T18:14:05.828 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-26T18:14:05.862 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-26T18:14:05.895 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-26T18:14:05.920 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T18:14:05.937 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-26T18:14:05.944 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:14:05.972 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-26T18:14:05.974 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:14:06.059 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-26T18:14:06.110 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-26T18:14:06.142 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-26T18:14:06.189 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-26T18:14:06.286 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-26T18:14:06.335 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-26T18:14:06.513 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:14:06.552 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T18:14:06.643 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T18:14:06.643 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T18:14:06.643 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T18:14:06.644 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:14:06.670 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T18:14:07.054 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-26T18:14:07.116 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-26T18:14:07.132 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T18:14:07.149 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-26T18:14:07.202 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T18:14:07.230 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T18:14:07.247 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-26T18:14:07.266 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T18:14:07.307 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-26T18:14:07.393 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-26T18:14:07.426 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-26T18:14:07.574 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T18:14:07.602 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-26T18:14:07.618 INFO:teuthology.orchestra.run.smithi103.stdout: Upgrading : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T18:14:07.632 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-26T18:14:07.709 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T18:14:07.710 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-26T18:14:07.740 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 100/156 2024-04-26T18:14:07.776 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T18:14:08.255 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T18:14:08.280 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T18:14:08.350 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T18:14:08.351 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-26T18:14:08.373 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 104/156 2024-04-26T18:14:08.462 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-26T18:14:09.682 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-26T18:14:09.761 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-26T18:14:09.789 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-26T18:14:09.822 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-26T18:14:10.133 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-26T18:14:10.177 INFO:teuthology.orchestra.run.smithi084.stdout: Upgrading : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T18:14:10.266 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 99/156 2024-04-26T18:14:10.298 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 100/156 2024-04-26T18:14:10.334 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-26T18:14:10.701 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:14:10.745 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:14:10.814 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 102/156 2024-04-26T18:14:10.846 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T18:14:10.917 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 103/156 2024-04-26T18:14:10.939 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 104/156 2024-04-26T18:14:11.289 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:14:11.340 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T18:14:11.436 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T18:14:11.437 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T18:14:11.437 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T18:14:11.437 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:14:11.467 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T18:14:13.238 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:14:13.269 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:14:13.859 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 105/156 2024-04-26T18:14:14.089 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T18:14:14.182 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 106/156 2024-04-26T18:14:14.183 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T18:14:14.183 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T18:14:14.183 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:14:14.215 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T18:14:18.144 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T18:14:18.144 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /sys 2024-04-26T18:14:18.144 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /proc 2024-04-26T18:14:18.144 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /mnt 2024-04-26T18:14:18.144 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /var/tmp 2024-04-26T18:14:18.145 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /home 2024-04-26T18:14:18.145 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /root 2024-04-26T18:14:18.145 INFO:teuthology.orchestra.run.smithi174.stdout:skipping the directory /tmp 2024-04-26T18:14:18.145 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:14:18.314 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T18:14:18.329 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T18:14:18.330 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:18.330 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T18:14:18.330 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:14:19.817 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T18:14:19.842 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T18:14:19.842 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:19.842 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T18:14:19.843 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T18:14:19.843 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T18:14:19.843 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:14:20.068 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T18:14:20.090 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T18:14:20.090 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:20.090 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T18:14:20.090 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T18:14:20.090 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T18:14:20.091 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:14:20.526 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T18:14:20.549 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T18:14:20.549 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:20.549 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T18:14:20.549 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T18:14:20.549 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T18:14:20.550 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:14:20.648 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T18:14:20.696 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T18:14:20.751 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T18:14:20.937 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T18:14:20.985 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T18:14:21.066 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T18:14:21.151 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T18:14:21.290 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T18:14:21.335 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T18:14:21.367 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T18:14:21.395 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T18:14:21.804 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T18:14:21.835 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T18:14:21.864 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 124/156 2024-04-26T18:14:21.928 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 125/156 2024-04-26T18:14:21.958 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T18:14:22.908 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T18:14:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /sys 2024-04-26T18:14:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /proc 2024-04-26T18:14:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /mnt 2024-04-26T18:14:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /var/tmp 2024-04-26T18:14:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /home 2024-04-26T18:14:22.908 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /root 2024-04-26T18:14:22.909 INFO:teuthology.orchestra.run.smithi103.stdout:skipping the directory /tmp 2024-04-26T18:14:22.909 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:14:22.995 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T18:14:23.043 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T18:14:23.098 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T18:14:23.113 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T18:14:23.113 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:23.113 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T18:14:23.113 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:14:24.006 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T18:14:24.043 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T18:14:24.145 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T18:14:24.321 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 129/156 2024-04-26T18:14:24.354 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T18:14:24.376 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T18:14:24.376 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:24.376 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T18:14:24.376 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T18:14:24.377 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T18:14:24.377 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:14:24.421 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T18:14:24.432 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T18:14:24.529 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 132/156 2024-04-26T18:14:24.577 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 133/156 2024-04-26T18:14:24.593 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T18:14:24.618 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T18:14:24.618 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:24.619 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T18:14:24.619 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T18:14:24.619 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T18:14:24.619 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:14:24.844 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T18:14:24.867 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T18:14:24.867 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:24.867 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T18:14:24.867 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T18:14:24.867 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T18:14:24.867 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:14:25.312 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T18:14:25.334 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T18:14:25.334 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:25.334 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T18:14:25.334 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T18:14:25.334 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T18:14:25.334 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:14:25.433 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T18:14:25.490 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T18:14:25.498 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T18:14:25.523 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T18:14:25.545 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T18:14:25.546 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T18:14:25.546 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:25.546 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T18:14:25.546 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T18:14:25.546 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T18:14:25.546 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:14:25.576 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 107/156 2024-04-26T18:14:25.577 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /sys 2024-04-26T18:14:25.577 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /proc 2024-04-26T18:14:25.577 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /mnt 2024-04-26T18:14:25.577 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /var/tmp 2024-04-26T18:14:25.577 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /home 2024-04-26T18:14:25.577 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /root 2024-04-26T18:14:25.577 INFO:teuthology.orchestra.run.smithi084.stdout:skipping the directory /tmp 2024-04-26T18:14:25.577 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:14:25.590 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T18:14:25.612 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T18:14:25.613 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:25.613 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T18:14:25.613 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:14:25.666 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T18:14:25.688 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T18:14:25.689 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:25.689 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T18:14:25.689 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T18:14:25.689 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T18:14:25.689 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:14:25.722 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T18:14:25.752 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T18:14:25.768 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 108/156 2024-04-26T18:14:25.768 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:25.768 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-26T18:14:25.768 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:14:25.771 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T18:14:25.849 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T18:14:25.936 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T18:14:25.992 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T18:14:26.014 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T18:14:26.014 INFO:teuthology.orchestra.run.smithi174.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:26.014 INFO:teuthology.orchestra.run.smithi174.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T18:14:26.014 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T18:14:26.015 INFO:teuthology.orchestra.run.smithi174.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T18:14:26.015 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:14:26.084 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T18:14:26.121 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T18:14:26.153 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T18:14:26.181 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T18:14:26.575 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T18:14:26.605 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T18:14:26.637 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 124/156 2024-04-26T18:14:26.731 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 125/156 2024-04-26T18:14:26.761 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T18:14:27.246 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T18:14:27.272 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 109/156 2024-04-26T18:14:27.272 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:27.272 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T18:14:27.272 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T18:14:27.272 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T18:14:27.272 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:14:27.488 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T18:14:27.511 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 110/156 2024-04-26T18:14:27.511 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:27.511 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T18:14:27.511 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T18:14:27.511 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T18:14:27.511 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:14:27.801 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T18:14:27.838 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T18:14:27.947 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T18:14:27.970 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 111/156 2024-04-26T18:14:27.970 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:27.970 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T18:14:27.970 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T18:14:27.970 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T18:14:27.970 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:14:28.069 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-26T18:14:28.118 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-26T18:14:28.173 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-26T18:14:28.358 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-26T18:14:28.406 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-26T18:14:28.476 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-26T18:14:28.564 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T18:14:28.704 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T18:14:28.748 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-26T18:14:28.780 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-26T18:14:28.804 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T18:14:28.807 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-26T18:14:28.830 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T18:14:28.931 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T18:14:29.092 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 129/156 2024-04-26T18:14:29.132 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T18:14:29.155 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T18:14:29.155 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:29.155 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T18:14:29.156 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T18:14:29.156 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T18:14:29.156 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:14:29.199 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T18:14:29.210 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T18:14:29.259 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T18:14:29.281 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 123/156 2024-04-26T18:14:29.308 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 132/156 2024-04-26T18:14:29.313 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 124/156 2024-04-26T18:14:29.357 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 133/156 2024-04-26T18:14:29.391 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 125/156 2024-04-26T18:14:29.421 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T18:14:30.281 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T18:14:30.319 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T18:14:30.341 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T18:14:30.341 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:30.341 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T18:14:30.342 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T18:14:30.342 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T18:14:30.342 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:14:30.386 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T18:14:30.408 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T18:14:30.408 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:30.408 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T18:14:30.408 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:14:30.450 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 126/156 2024-04-26T18:14:30.453 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T18:14:30.476 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T18:14:30.476 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:30.476 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T18:14:30.476 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T18:14:30.477 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T18:14:30.477 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:14:30.489 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T18:14:30.628 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 139/156 2024-04-26T18:14:30.670 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 140/156 2024-04-26T18:14:30.711 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 141/156 2024-04-26T18:14:30.743 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T18:14:30.770 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T18:14:30.779 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T18:14:30.802 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T18:14:30.802 INFO:teuthology.orchestra.run.smithi103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:30.802 INFO:teuthology.orchestra.run.smithi103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T18:14:30.802 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T18:14:30.802 INFO:teuthology.orchestra.run.smithi103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T18:14:30.802 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:14:30.953 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 144/156 2024-04-26T18:14:30.999 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T18:14:31.029 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 146/156 2024-04-26T18:14:31.452 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 127/156 2024-04-26T18:14:31.481 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T18:14:31.581 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 128/156 2024-04-26T18:14:31.591 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:14:31.635 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:14:31.654 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:14:31.742 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 129/156 2024-04-26T18:14:31.775 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T18:14:31.798 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 130/156 2024-04-26T18:14:31.798 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:31.798 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T18:14:31.798 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T18:14:31.798 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T18:14:31.798 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:14:31.842 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T18:14:31.853 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 131/156 2024-04-26T18:14:31.950 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 132/156 2024-04-26T18:14:31.998 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 133/156 2024-04-26T18:14:32.484 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 148/156 2024-04-26T18:14:32.572 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T18:14:32.612 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T18:14:32.693 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T18:14:32.726 INFO:teuthology.orchestra.run.smithi174.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T18:14:32.726 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T18:14:32.810 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T18:14:32.852 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T18:14:32.852 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T18:14:32.920 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-26T18:14:32.943 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T18:14:32.953 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T18:14:32.976 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 135/156 2024-04-26T18:14:32.976 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:32.976 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T18:14:32.976 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T18:14:32.976 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T18:14:32.976 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:14:33.007 INFO:teuthology.orchestra.run.smithi174.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:14:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T18:14:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 136/156 2024-04-26T18:14:33.042 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:33.042 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T18:14:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:14:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T18:14:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 137/156 2024-04-26T18:14:33.110 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:33.110 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-26T18:14:33.110 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T18:14:33.110 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-26T18:14:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:14:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T18:14:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 138/156 2024-04-26T18:14:33.436 INFO:teuthology.orchestra.run.smithi084.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T18:14:33.436 INFO:teuthology.orchestra.run.smithi084.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T18:14:33.436 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T18:14:33.436 INFO:teuthology.orchestra.run.smithi084.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T18:14:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:14:35.392 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 139/156 2024-04-26T18:14:35.433 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 140/156 2024-04-26T18:14:35.474 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 141/156 2024-04-26T18:14:35.506 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T18:14:35.533 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T18:14:35.717 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 144/156 2024-04-26T18:14:35.763 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T18:14:35.793 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 146/156 2024-04-26T18:14:36.397 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:14:36.433 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:14:36.453 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:14:37.296 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 148/156 2024-04-26T18:14:37.398 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T18:14:37.464 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T18:14:37.545 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T18:14:37.585 INFO:teuthology.orchestra.run.smithi103.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T18:14:37.586 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T18:14:37.669 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T18:14:37.694 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T18:14:37.694 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T18:14:37.784 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T18:14:37.850 INFO:teuthology.orchestra.run.smithi103.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:14:38.025 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 139/156 2024-04-26T18:14:38.067 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 140/156 2024-04-26T18:14:38.107 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 141/156 2024-04-26T18:14:38.139 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-26T18:14:38.167 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-26T18:14:38.358 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 144/156 2024-04-26T18:14:38.404 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-26T18:14:38.435 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 146/156 2024-04-26T18:14:38.753 INFO:teuthology.orchestra.run.smithi174.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:14:38.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 1/156 2024-04-26T18:14:38.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2/156 2024-04-26T18:14:38.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 3/156 2024-04-26T18:14:38.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 4/156 2024-04-26T18:14:38.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 5/156 2024-04-26T18:14:38.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 6/156 2024-04-26T18:14:38.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 7/156 2024-04-26T18:14:38.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:14:38.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:14:38.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T18:14:38.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 11/156 2024-04-26T18:14:38.754 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 12/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 14/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 15/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 16/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 17/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 18/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 19/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 20/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 21/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 22/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 23/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 24/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 25/156 2024-04-26T18:14:38.755 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 26/156 2024-04-26T18:14:38.756 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 27/156 2024-04-26T18:14:38.756 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 28/156 2024-04-26T18:14:38.756 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 29/156 2024-04-26T18:14:38.767 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 30/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 31/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 32/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 33/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 34/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 35/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 36/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 37/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 38/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 39/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T18:14:38.768 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T18:14:38.769 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T18:14:38.769 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T18:14:38.769 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T18:14:38.769 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T18:14:38.769 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T18:14:38.769 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T18:14:38.769 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T18:14:38.769 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T18:14:38.769 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T18:14:38.769 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T18:14:38.769 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T18:14:38.769 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T18:14:38.769 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T18:14:38.770 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T18:14:38.770 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T18:14:38.770 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T18:14:38.770 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T18:14:38.770 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T18:14:38.770 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T18:14:38.770 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T18:14:38.770 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T18:14:38.770 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T18:14:38.770 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T18:14:38.770 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T18:14:38.770 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T18:14:38.770 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T18:14:38.771 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T18:14:38.771 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T18:14:38.771 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T18:14:38.771 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T18:14:38.771 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T18:14:38.771 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T18:14:38.771 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T18:14:38.771 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T18:14:38.771 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T18:14:38.771 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T18:14:38.771 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T18:14:38.771 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T18:14:38.771 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T18:14:38.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T18:14:38.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T18:14:38.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T18:14:38.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T18:14:38.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T18:14:38.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T18:14:38.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T18:14:38.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T18:14:38.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T18:14:38.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T18:14:38.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T18:14:38.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T18:14:38.772 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T18:14:38.773 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T18:14:38.774 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T18:14:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T18:14:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T18:14:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T18:14:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T18:14:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T18:14:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T18:14:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T18:14:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T18:14:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T18:14:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T18:14:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T18:14:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T18:14:38.775 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 149/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T18:14:38.776 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 151/156 2024-04-26T18:14:38.777 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T18:14:38.777 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T18:14:38.777 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T18:14:38.777 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T18:14:39.005 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:14:39.048 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:14:39.068 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-26T18:14:39.902 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 148/156 2024-04-26T18:14:39.978 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-26T18:14:40.019 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-26T18:14:40.103 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-26T18:14:40.140 INFO:teuthology.orchestra.run.smithi084.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-26T18:14:40.140 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T18:14:40.233 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-26T18:14:40.257 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T18:14:40.258 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T18:14:40.347 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-26T18:14:40.416 INFO:teuthology.orchestra.run.smithi084.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:14:40.909 INFO:teuthology.orchestra.run.smithi174.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:14:40.909 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:14:40.910 INFO:teuthology.orchestra.run.smithi174.stdout:Upgraded: 2024-04-26T18:14:40.910 INFO:teuthology.orchestra.run.smithi174.stdout: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.910 INFO:teuthology.orchestra.run.smithi174.stdout: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.910 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T18:14:40.910 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T18:14:40.910 INFO:teuthology.orchestra.run.smithi174.stdout:Installed: 2024-04-26T18:14:40.910 INFO:teuthology.orchestra.run.smithi174.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T18:14:40.910 INFO:teuthology.orchestra.run.smithi174.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T18:14:40.910 INFO:teuthology.orchestra.run.smithi174.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T18:14:40.910 INFO:teuthology.orchestra.run.smithi174.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T18:14:40.910 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.910 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.911 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.911 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.911 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:40.911 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.911 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.911 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.911 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:40.911 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:40.911 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:40.911 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:40.911 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:40.911 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.913 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.913 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:40.913 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.913 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.913 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.913 INFO:teuthology.orchestra.run.smithi174.stdout: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:40.913 INFO:teuthology.orchestra.run.smithi174.stdout: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:40.914 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.914 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.914 INFO:teuthology.orchestra.run.smithi174.stdout: cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:40.914 INFO:teuthology.orchestra.run.smithi174.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T18:14:40.914 INFO:teuthology.orchestra.run.smithi174.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T18:14:40.914 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T18:14:40.914 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T18:14:40.914 INFO:teuthology.orchestra.run.smithi174.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T18:14:40.914 INFO:teuthology.orchestra.run.smithi174.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T18:14:40.914 INFO:teuthology.orchestra.run.smithi174.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T18:14:40.914 INFO:teuthology.orchestra.run.smithi174.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T18:14:40.914 INFO:teuthology.orchestra.run.smithi174.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T18:14:40.914 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.915 INFO:teuthology.orchestra.run.smithi174.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T18:14:40.916 INFO:teuthology.orchestra.run.smithi174.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T18:14:40.916 INFO:teuthology.orchestra.run.smithi174.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T18:14:40.916 INFO:teuthology.orchestra.run.smithi174.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T18:14:40.916 INFO:teuthology.orchestra.run.smithi174.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T18:14:40.916 INFO:teuthology.orchestra.run.smithi174.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T18:14:40.916 INFO:teuthology.orchestra.run.smithi174.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T18:14:40.916 INFO:teuthology.orchestra.run.smithi174.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T18:14:40.916 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T18:14:40.916 INFO:teuthology.orchestra.run.smithi174.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T18:14:40.916 INFO:teuthology.orchestra.run.smithi174.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T18:14:40.916 INFO:teuthology.orchestra.run.smithi174.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T18:14:40.916 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T18:14:40.916 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T18:14:40.917 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T18:14:40.917 INFO:teuthology.orchestra.run.smithi174.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T18:14:40.917 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T18:14:40.917 INFO:teuthology.orchestra.run.smithi174.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T18:14:40.917 INFO:teuthology.orchestra.run.smithi174.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T18:14:40.917 INFO:teuthology.orchestra.run.smithi174.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T18:14:40.917 INFO:teuthology.orchestra.run.smithi174.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T18:14:40.917 INFO:teuthology.orchestra.run.smithi174.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T18:14:40.917 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T18:14:40.917 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.917 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.917 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.917 INFO:teuthology.orchestra.run.smithi174.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T18:14:40.918 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T18:14:40.919 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T18:14:40.919 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T18:14:40.919 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T18:14:40.919 INFO:teuthology.orchestra.run.smithi174.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T18:14:40.919 INFO:teuthology.orchestra.run.smithi174.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T18:14:40.919 INFO:teuthology.orchestra.run.smithi174.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T18:14:40.919 INFO:teuthology.orchestra.run.smithi174.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T18:14:40.919 INFO:teuthology.orchestra.run.smithi174.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T18:14:40.919 INFO:teuthology.orchestra.run.smithi174.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T18:14:40.919 INFO:teuthology.orchestra.run.smithi174.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T18:14:40.919 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T18:14:40.919 INFO:teuthology.orchestra.run.smithi174.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T18:14:40.919 INFO:teuthology.orchestra.run.smithi174.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.920 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.921 INFO:teuthology.orchestra.run.smithi174.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T18:14:40.921 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T18:14:40.921 INFO:teuthology.orchestra.run.smithi174.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T18:14:40.921 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.921 INFO:teuthology.orchestra.run.smithi174.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T18:14:40.921 INFO:teuthology.orchestra.run.smithi174.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T18:14:40.921 INFO:teuthology.orchestra.run.smithi174.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T18:14:40.921 INFO:teuthology.orchestra.run.smithi174.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T18:14:40.921 INFO:teuthology.orchestra.run.smithi174.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T18:14:40.921 INFO:teuthology.orchestra.run.smithi174.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T18:14:40.921 INFO:teuthology.orchestra.run.smithi174.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T18:14:40.921 INFO:teuthology.orchestra.run.smithi174.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T18:14:40.922 INFO:teuthology.orchestra.run.smithi174.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T18:14:40.922 INFO:teuthology.orchestra.run.smithi174.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T18:14:40.922 INFO:teuthology.orchestra.run.smithi174.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T18:14:40.922 INFO:teuthology.orchestra.run.smithi174.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T18:14:40.922 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T18:14:40.922 INFO:teuthology.orchestra.run.smithi174.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T18:14:40.922 INFO:teuthology.orchestra.run.smithi174.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T18:14:40.922 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.922 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.922 INFO:teuthology.orchestra.run.smithi174.stdout: rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:40.922 INFO:teuthology.orchestra.run.smithi174.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T18:14:40.922 INFO:teuthology.orchestra.run.smithi174.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T18:14:40.922 INFO:teuthology.orchestra.run.smithi174.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T18:14:40.923 INFO:teuthology.orchestra.run.smithi174.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T18:14:40.923 INFO:teuthology.orchestra.run.smithi174.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T18:14:40.923 INFO:teuthology.orchestra.run.smithi174.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T18:14:40.923 INFO:teuthology.orchestra.run.smithi174.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T18:14:40.923 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:14:40.923 INFO:teuthology.orchestra.run.smithi174.stdout:Complete! 2024-04-26T18:14:41.324 DEBUG:teuthology.parallel:result is None 2024-04-26T18:14:43.915 INFO:teuthology.orchestra.run.smithi103.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:14:43.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 1/156 2024-04-26T18:14:43.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2/156 2024-04-26T18:14:43.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 3/156 2024-04-26T18:14:43.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 4/156 2024-04-26T18:14:43.915 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 5/156 2024-04-26T18:14:43.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 6/156 2024-04-26T18:14:43.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 7/156 2024-04-26T18:14:43.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:14:43.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:14:43.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T18:14:43.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 11/156 2024-04-26T18:14:43.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 12/156 2024-04-26T18:14:43.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:14:43.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 14/156 2024-04-26T18:14:43.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 15/156 2024-04-26T18:14:43.916 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 16/156 2024-04-26T18:14:43.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 17/156 2024-04-26T18:14:43.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 18/156 2024-04-26T18:14:43.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 19/156 2024-04-26T18:14:43.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 20/156 2024-04-26T18:14:43.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 21/156 2024-04-26T18:14:43.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 22/156 2024-04-26T18:14:43.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 23/156 2024-04-26T18:14:43.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 24/156 2024-04-26T18:14:43.917 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 25/156 2024-04-26T18:14:43.929 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 26/156 2024-04-26T18:14:43.929 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 27/156 2024-04-26T18:14:43.930 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 28/156 2024-04-26T18:14:43.930 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 29/156 2024-04-26T18:14:43.930 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 30/156 2024-04-26T18:14:43.930 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 31/156 2024-04-26T18:14:43.930 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 32/156 2024-04-26T18:14:43.930 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 33/156 2024-04-26T18:14:43.930 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 34/156 2024-04-26T18:14:43.930 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 35/156 2024-04-26T18:14:43.930 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 36/156 2024-04-26T18:14:43.930 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 37/156 2024-04-26T18:14:43.930 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 38/156 2024-04-26T18:14:43.930 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 39/156 2024-04-26T18:14:43.930 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T18:14:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T18:14:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T18:14:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T18:14:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T18:14:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T18:14:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T18:14:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T18:14:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T18:14:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T18:14:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T18:14:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T18:14:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T18:14:43.931 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T18:14:43.932 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T18:14:43.933 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T18:14:43.933 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T18:14:43.933 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T18:14:43.933 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T18:14:43.933 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T18:14:43.933 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T18:14:43.933 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T18:14:43.933 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T18:14:43.933 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T18:14:43.933 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T18:14:43.933 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T18:14:43.933 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T18:14:43.933 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T18:14:43.934 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T18:14:43.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T18:14:43.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T18:14:43.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T18:14:43.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T18:14:43.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T18:14:43.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T18:14:43.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T18:14:43.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T18:14:43.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T18:14:43.935 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T18:14:43.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T18:14:43.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T18:14:43.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T18:14:43.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T18:14:43.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T18:14:43.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T18:14:43.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T18:14:43.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T18:14:43.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T18:14:43.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T18:14:43.936 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T18:14:43.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T18:14:43.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T18:14:43.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T18:14:43.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T18:14:43.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T18:14:43.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T18:14:43.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T18:14:43.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T18:14:43.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T18:14:43.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T18:14:43.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T18:14:43.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T18:14:43.937 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T18:14:43.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T18:14:43.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T18:14:43.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T18:14:43.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T18:14:43.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T18:14:43.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T18:14:43.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T18:14:43.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T18:14:43.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T18:14:43.938 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 149/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 151/156 2024-04-26T18:14:43.939 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T18:14:43.940 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T18:14:43.940 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T18:14:43.940 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T18:14:45.979 INFO:teuthology.orchestra.run.smithi103.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:14:45.979 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:14:45.979 INFO:teuthology.orchestra.run.smithi103.stdout:Upgraded: 2024-04-26T18:14:45.979 INFO:teuthology.orchestra.run.smithi103.stdout: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.979 INFO:teuthology.orchestra.run.smithi103.stdout: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.979 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T18:14:45.979 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T18:14:45.980 INFO:teuthology.orchestra.run.smithi103.stdout:Installed: 2024-04-26T18:14:45.980 INFO:teuthology.orchestra.run.smithi103.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T18:14:45.980 INFO:teuthology.orchestra.run.smithi103.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T18:14:45.980 INFO:teuthology.orchestra.run.smithi103.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T18:14:45.980 INFO:teuthology.orchestra.run.smithi103.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T18:14:45.980 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.980 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.980 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.980 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.980 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:45.980 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.980 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.980 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:45.981 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.982 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.982 INFO:teuthology.orchestra.run.smithi103.stdout: cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:45.982 INFO:teuthology.orchestra.run.smithi103.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T18:14:45.982 INFO:teuthology.orchestra.run.smithi103.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T18:14:45.982 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T18:14:45.982 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T18:14:45.982 INFO:teuthology.orchestra.run.smithi103.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T18:14:45.982 INFO:teuthology.orchestra.run.smithi103.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T18:14:45.982 INFO:teuthology.orchestra.run.smithi103.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T18:14:45.982 INFO:teuthology.orchestra.run.smithi103.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T18:14:45.982 INFO:teuthology.orchestra.run.smithi103.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T18:14:45.982 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T18:14:45.982 INFO:teuthology.orchestra.run.smithi103.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T18:14:45.983 INFO:teuthology.orchestra.run.smithi103.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T18:14:45.984 INFO:teuthology.orchestra.run.smithi103.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T18:14:45.984 INFO:teuthology.orchestra.run.smithi103.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T18:14:45.984 INFO:teuthology.orchestra.run.smithi103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T18:14:45.984 INFO:teuthology.orchestra.run.smithi103.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T18:14:45.984 INFO:teuthology.orchestra.run.smithi103.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T18:14:45.984 INFO:teuthology.orchestra.run.smithi103.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T18:14:45.984 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T18:14:45.984 INFO:teuthology.orchestra.run.smithi103.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T18:14:45.984 INFO:teuthology.orchestra.run.smithi103.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T18:14:45.984 INFO:teuthology.orchestra.run.smithi103.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T18:14:45.984 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T18:14:45.984 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T18:14:45.984 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T18:14:45.985 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T18:14:45.986 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T18:14:45.986 INFO:teuthology.orchestra.run.smithi103.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T18:14:45.986 INFO:teuthology.orchestra.run.smithi103.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T18:14:45.986 INFO:teuthology.orchestra.run.smithi103.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T18:14:45.986 INFO:teuthology.orchestra.run.smithi103.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T18:14:45.986 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T18:14:45.986 INFO:teuthology.orchestra.run.smithi103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T18:14:45.986 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T18:14:45.986 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T18:14:45.986 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T18:14:45.986 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T18:14:45.986 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T18:14:45.987 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T18:14:45.988 INFO:teuthology.orchestra.run.smithi103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T18:14:45.989 INFO:teuthology.orchestra.run.smithi103.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T18:14:45.990 INFO:teuthology.orchestra.run.smithi103.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T18:14:45.990 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.990 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.990 INFO:teuthology.orchestra.run.smithi103.stdout: rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:45.990 INFO:teuthology.orchestra.run.smithi103.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T18:14:45.990 INFO:teuthology.orchestra.run.smithi103.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T18:14:45.990 INFO:teuthology.orchestra.run.smithi103.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T18:14:45.990 INFO:teuthology.orchestra.run.smithi103.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T18:14:45.990 INFO:teuthology.orchestra.run.smithi103.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T18:14:45.990 INFO:teuthology.orchestra.run.smithi103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T18:14:45.991 INFO:teuthology.orchestra.run.smithi103.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T18:14:45.991 INFO:teuthology.orchestra.run.smithi103.stdout: 2024-04-26T18:14:45.991 INFO:teuthology.orchestra.run.smithi103.stdout:Complete! 2024-04-26T18:14:46.359 INFO:teuthology.orchestra.run.smithi084.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:14:46.360 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 1/156 2024-04-26T18:14:46.360 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2/156 2024-04-26T18:14:46.360 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 3/156 2024-04-26T18:14:46.360 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 4/156 2024-04-26T18:14:46.360 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3267.gbc94a 5/156 2024-04-26T18:14:46.360 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 6/156 2024-04-26T18:14:46.361 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 7/156 2024-04-26T18:14:46.361 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 8/156 2024-04-26T18:14:46.361 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 9/156 2024-04-26T18:14:46.361 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 10/156 2024-04-26T18:14:46.361 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 11/156 2024-04-26T18:14:46.362 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 12/156 2024-04-26T18:14:46.362 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 13/156 2024-04-26T18:14:46.362 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 14/156 2024-04-26T18:14:46.362 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_ 15/156 2024-04-26T18:14:46.362 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 16/156 2024-04-26T18:14:46.362 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 17/156 2024-04-26T18:14:46.363 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_6 18/156 2024-04-26T18:14:46.363 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86 19/156 2024-04-26T18:14:46.363 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 20/156 2024-04-26T18:14:46.363 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el 21/156 2024-04-26T18:14:46.364 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9. 22/156 2024-04-26T18:14:46.364 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_6 23/156 2024-04-26T18:14:46.364 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 24/156 2024-04-26T18:14:46.365 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 25/156 2024-04-26T18:14:46.365 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 26/156 2024-04-26T18:14:46.365 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 27/156 2024-04-26T18:14:46.365 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 28/156 2024-04-26T18:14:46.365 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 29/156 2024-04-26T18:14:46.365 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9. 30/156 2024-04-26T18:14:46.366 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noa 31/156 2024-04-26T18:14:46.366 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.n 32/156 2024-04-26T18:14:46.366 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc9 33/156 2024-04-26T18:14:46.366 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el 34/156 2024-04-26T18:14:46.366 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 35/156 2024-04-26T18:14:46.367 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.e 36/156 2024-04-26T18:14:46.367 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 37/156 2024-04-26T18:14:46.367 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 38/156 2024-04-26T18:14:46.367 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 39/156 2024-04-26T18:14:46.367 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-26T18:14:46.367 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-26T18:14:46.368 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-26T18:14:46.368 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-26T18:14:46.368 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-26T18:14:46.368 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-26T18:14:46.368 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-26T18:14:46.368 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-26T18:14:46.369 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-26T18:14:46.369 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-26T18:14:46.369 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-26T18:14:46.369 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-26T18:14:46.369 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-26T18:14:46.369 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-26T18:14:46.369 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-26T18:14:46.370 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-26T18:14:46.370 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-26T18:14:46.370 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-26T18:14:46.371 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-26T18:14:46.371 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-26T18:14:46.371 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-26T18:14:46.371 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-26T18:14:46.371 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-26T18:14:46.371 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-26T18:14:46.372 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-26T18:14:46.372 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-26T18:14:46.372 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-26T18:14:46.372 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-26T18:14:46.372 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-26T18:14:46.372 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-26T18:14:46.373 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-26T18:14:46.373 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-26T18:14:46.373 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-26T18:14:46.373 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-26T18:14:46.373 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-26T18:14:46.373 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-26T18:14:46.374 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-26T18:14:46.374 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-26T18:14:46.374 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-26T18:14:46.374 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-26T18:14:46.374 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-26T18:14:46.374 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-26T18:14:46.375 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-26T18:14:46.375 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-26T18:14:46.375 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-26T18:14:46.375 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-26T18:14:46.376 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-26T18:14:46.377 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-26T18:14:46.377 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-26T18:14:46.377 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-26T18:14:46.377 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-26T18:14:46.377 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-26T18:14:46.377 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-26T18:14:46.377 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-26T18:14:46.377 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-26T18:14:46.377 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/156 2024-04-26T18:14:46.377 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/156 2024-04-26T18:14:46.377 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-26T18:14:46.377 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-26T18:14:46.377 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-26T18:14:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/156 2024-04-26T18:14:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-26T18:14:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-26T18:14:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-26T18:14:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-26T18:14:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-26T18:14:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-26T18:14:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-26T18:14:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-26T18:14:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-26T18:14:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-26T18:14:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-26T18:14:46.378 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-26T18:14:46.379 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-26T18:14:46.379 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-26T18:14:46.379 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-26T18:14:46.379 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-26T18:14:46.379 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-26T18:14:46.379 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-26T18:14:46.379 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-26T18:14:46.379 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-26T18:14:46.379 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-26T18:14:46.379 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-26T18:14:46.380 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-26T18:14:46.380 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-26T18:14:46.380 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-26T18:14:46.380 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-26T18:14:46.380 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-26T18:14:46.380 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-26T18:14:46.380 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-26T18:14:46.380 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-26T18:14:46.381 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-26T18:14:46.381 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-26T18:14:46.381 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-26T18:14:46.381 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-26T18:14:46.381 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-26T18:14:46.381 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 149/156 2024-04-26T18:14:46.381 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-26T18:14:46.381 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 151/156 2024-04-26T18:14:46.381 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-26T18:14:46.381 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-26T18:14:46.381 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-26T18:14:46.381 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-26T18:14:46.384 DEBUG:teuthology.parallel:result is None 2024-04-26T18:14:48.431 INFO:teuthology.orchestra.run.smithi084.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-26T18:14:48.431 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:14:48.431 INFO:teuthology.orchestra.run.smithi084.stdout:Upgraded: 2024-04-26T18:14:48.431 INFO:teuthology.orchestra.run.smithi084.stdout: librados2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.431 INFO:teuthology.orchestra.run.smithi084.stdout: librbd1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.431 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-26T18:14:48.431 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-26T18:14:48.432 INFO:teuthology.orchestra.run.smithi084.stdout:Installed: 2024-04-26T18:14:48.432 INFO:teuthology.orchestra.run.smithi084.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-26T18:14:48.432 INFO:teuthology.orchestra.run.smithi084.stdout: automake-1.16.2-8.el9.noarch 2024-04-26T18:14:48.432 INFO:teuthology.orchestra.run.smithi084.stdout: bison-3.7.4-5.el9.x86_64 2024-04-26T18:14:48.432 INFO:teuthology.orchestra.run.smithi084.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T18:14:48.432 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.432 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-base-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.432 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.432 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.432 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-grafana-dashboards-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:48.432 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-immutable-object-cache-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.432 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mds-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.432 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.433 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:48.433 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-dashboard-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:48.433 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:48.433 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-modules-core-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:48.433 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mgr-rook-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:48.433 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-mon-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.433 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-osd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.433 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-prometheus-alerts-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:48.433 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-radosgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.433 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-selinux-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.433 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-test-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.433 INFO:teuthology.orchestra.run.smithi084.stdout: ceph-volume-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:48.434 INFO:teuthology.orchestra.run.smithi084.stdout: cephadm-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:48.434 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.434 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-shell-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.434 INFO:teuthology.orchestra.run.smithi084.stdout: cephfs-top-2:19.0.0-3267.gbc94a2a9.el9.noarch 2024-04-26T18:14:48.434 INFO:teuthology.orchestra.run.smithi084.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-26T18:14:48.434 INFO:teuthology.orchestra.run.smithi084.stdout: flex-2.6.4-9.el9.x86_64 2024-04-26T18:14:48.434 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T18:14:48.434 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T18:14:48.434 INFO:teuthology.orchestra.run.smithi084.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T18:14:48.434 INFO:teuthology.orchestra.run.smithi084.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-26T18:14:48.434 INFO:teuthology.orchestra.run.smithi084.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T18:14:48.434 INFO:teuthology.orchestra.run.smithi084.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-26T18:14:48.434 INFO:teuthology.orchestra.run.smithi084.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-26T18:14:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T18:14:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T18:14:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: libcephfs2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: libcephsqlite-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T18:14:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T18:14:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T18:14:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T18:14:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T18:14:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: librados-devel-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: libradosstriper1-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T18:14:48.436 INFO:teuthology.orchestra.run.smithi084.stdout: librgw2-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.436 INFO:teuthology.orchestra.run.smithi084.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-26T18:14:48.436 INFO:teuthology.orchestra.run.smithi084.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T18:14:48.436 INFO:teuthology.orchestra.run.smithi084.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-26T18:14:48.436 INFO:teuthology.orchestra.run.smithi084.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-26T18:14:48.436 INFO:teuthology.orchestra.run.smithi084.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T18:14:48.436 INFO:teuthology.orchestra.run.smithi084.stdout: lua-5.4.4-4.el9.x86_64 2024-04-26T18:14:48.436 INFO:teuthology.orchestra.run.smithi084.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-26T18:14:48.436 INFO:teuthology.orchestra.run.smithi084.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-26T18:14:48.436 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T18:14:48.436 INFO:teuthology.orchestra.run.smithi084.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T18:14:48.436 INFO:teuthology.orchestra.run.smithi084.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-26T18:14:48.437 INFO:teuthology.orchestra.run.smithi084.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T18:14:48.437 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-13.14-1.el9.x86_64 2024-04-26T18:14:48.437 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-26T18:14:48.437 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-26T18:14:48.437 INFO:teuthology.orchestra.run.smithi084.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-26T18:14:48.437 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-26T18:14:48.437 INFO:teuthology.orchestra.run.smithi084.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-26T18:14:48.437 INFO:teuthology.orchestra.run.smithi084.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T18:14:48.437 INFO:teuthology.orchestra.run.smithi084.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-26T18:14:48.437 INFO:teuthology.orchestra.run.smithi084.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T18:14:48.437 INFO:teuthology.orchestra.run.smithi084.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T18:14:48.437 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T18:14:48.437 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-argparse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.438 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ceph-common-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.438 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cephfs-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.438 INFO:teuthology.orchestra.run.smithi084.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T18:14:48.438 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T18:14:48.438 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T18:14:48.438 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T18:14:48.438 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-26T18:14:48.438 INFO:teuthology.orchestra.run.smithi084.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-26T18:14:48.438 INFO:teuthology.orchestra.run.smithi084.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T18:14:48.438 INFO:teuthology.orchestra.run.smithi084.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T18:14:48.439 INFO:teuthology.orchestra.run.smithi084.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T18:14:48.439 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-26T18:14:48.439 INFO:teuthology.orchestra.run.smithi084.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-26T18:14:48.439 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T18:14:48.439 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T18:14:48.439 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T18:14:48.439 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T18:14:48.439 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T18:14:48.439 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T18:14:48.439 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T18:14:48.439 INFO:teuthology.orchestra.run.smithi084.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T18:14:48.439 INFO:teuthology.orchestra.run.smithi084.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T18:14:48.440 INFO:teuthology.orchestra.run.smithi084.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-26T18:14:48.441 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T18:14:48.441 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T18:14:48.441 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T18:14:48.441 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T18:14:48.441 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-26T18:14:48.441 INFO:teuthology.orchestra.run.smithi084.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T18:14:48.441 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rados-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.441 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.441 INFO:teuthology.orchestra.run.smithi084.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T18:14:48.442 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T18:14:48.442 INFO:teuthology.orchestra.run.smithi084.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T18:14:48.442 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rgw-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.442 INFO:teuthology.orchestra.run.smithi084.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T18:14:48.442 INFO:teuthology.orchestra.run.smithi084.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T18:14:48.442 INFO:teuthology.orchestra.run.smithi084.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T18:14:48.442 INFO:teuthology.orchestra.run.smithi084.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T18:14:48.442 INFO:teuthology.orchestra.run.smithi084.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T18:14:48.442 INFO:teuthology.orchestra.run.smithi084.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T18:14:48.442 INFO:teuthology.orchestra.run.smithi084.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T18:14:48.442 INFO:teuthology.orchestra.run.smithi084.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-26T18:14:48.442 INFO:teuthology.orchestra.run.smithi084.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T18:14:48.442 INFO:teuthology.orchestra.run.smithi084.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T18:14:48.443 INFO:teuthology.orchestra.run.smithi084.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T18:14:48.443 INFO:teuthology.orchestra.run.smithi084.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T18:14:48.443 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-26T18:14:48.443 INFO:teuthology.orchestra.run.smithi084.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-26T18:14:48.443 INFO:teuthology.orchestra.run.smithi084.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-26T18:14:48.443 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-fuse-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.443 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-mirror-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.443 INFO:teuthology.orchestra.run.smithi084.stdout: rbd-nbd-2:19.0.0-3267.gbc94a2a9.el9.x86_64 2024-04-26T18:14:48.443 INFO:teuthology.orchestra.run.smithi084.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T18:14:48.443 INFO:teuthology.orchestra.run.smithi084.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-26T18:14:48.443 INFO:teuthology.orchestra.run.smithi084.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T18:14:48.443 INFO:teuthology.orchestra.run.smithi084.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-26T18:14:48.443 INFO:teuthology.orchestra.run.smithi084.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-26T18:14:48.444 INFO:teuthology.orchestra.run.smithi084.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T18:14:48.444 INFO:teuthology.orchestra.run.smithi084.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-26T18:14:48.444 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:14:48.444 INFO:teuthology.orchestra.run.smithi084.stdout:Complete! 2024-04-26T18:14:48.818 DEBUG:teuthology.parallel:result is None 2024-04-26T18:14:48.819 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:14:49.038 DEBUG:teuthology.orchestra.run.smithi084:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T18:14:49.060 INFO:teuthology.orchestra.run.smithi084.stdout:19.0.0-3267.gbc94a2a9.el9 2024-04-26T18:14:49.060 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3267.gbc94a2a9.el9 2024-04-26T18:14:49.061 INFO:teuthology.task.install:The correct ceph version 19.0.0-3267.gbc94a2a9 is installed. 2024-04-26T18:14:49.062 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:14:49.197 DEBUG:teuthology.orchestra.run.smithi103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T18:14:49.220 INFO:teuthology.orchestra.run.smithi103.stdout:19.0.0-3267.gbc94a2a9.el9 2024-04-26T18:14:49.220 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3267.gbc94a2a9.el9 2024-04-26T18:14:49.220 INFO:teuthology.task.install:The correct ceph version 19.0.0-3267.gbc94a2a9 is installed. 2024-04-26T18:14:49.222 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:14:49.356 DEBUG:teuthology.orchestra.run.smithi174:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T18:14:49.382 INFO:teuthology.orchestra.run.smithi174.stdout:19.0.0-3267.gbc94a2a9.el9 2024-04-26T18:14:49.382 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3267.gbc94a2a9.el9 2024-04-26T18:14:49.382 INFO:teuthology.task.install:The correct ceph version 19.0.0-3267.gbc94a2a9 is installed. 2024-04-26T18:14:49.384 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T18:14:49.385 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:14:49.385 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T18:14:49.410 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T18:14:49.410 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T18:14:49.438 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T18:14:49.438 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T18:14:49.468 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T18:14:49.468 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:14:49.468 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T18:14:49.491 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T18:14:49.555 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T18:14:49.555 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T18:14:49.579 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T18:14:49.643 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T18:14:49.643 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T18:14:49.671 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T18:14:49.740 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T18:14:49.740 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:14:49.741 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T18:14:49.764 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T18:14:49.827 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T18:14:49.827 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T18:14:49.852 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T18:14:49.916 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T18:14:49.916 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T18:14:49.940 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T18:14:50.006 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T18:14:50.006 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:14:50.006 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T18:14:50.030 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T18:14:50.093 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T18:14:50.093 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T18:14:50.117 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T18:14:50.193 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T18:14:50.193 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T18:14:50.217 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T18:14:50.281 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T18:14:50.399 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-26T18:14:50.400 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:14:50.401 INFO:tasks.cephadm:Cluster fsid is df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:14:50.401 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T18:14:50.401 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.84', 'mon.b': '172.21.15.103', 'mon.c': '172.21.15.174'} 2024-04-26T18:14:50.401 INFO:tasks.cephadm:First mon is mon.a on smithi084 2024-04-26T18:14:50.402 INFO:tasks.cephadm:First mgr is x 2024-04-26T18:14:50.402 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T18:14:50.402 DEBUG:teuthology.orchestra.run.smithi084:> sudo hostname $(hostname -s) 2024-04-26T18:14:50.426 DEBUG:teuthology.orchestra.run.smithi103:> sudo hostname $(hostname -s) 2024-04-26T18:14:50.452 DEBUG:teuthology.orchestra.run.smithi174:> sudo hostname $(hostname -s) 2024-04-26T18:14:50.475 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-26T18:14:50.476 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:14:50.697 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78913/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3267-gbc94a2a9', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3267.gbc94a2a9'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-26 08:22:46.917786', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer-fixes', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-04-26T18:14:50.912 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-lusov-quiescer-fixes, sha1 bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:14:50.914 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/x86_64/flavors/default/cephadm 2024-04-26T18:14:50.914 DEBUG:teuthology.orchestra.run.smithi084:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T18:14:51.488 INFO:teuthology.orchestra.run.smithi084.stdout:-rw-r--r--. 1 ubuntu ubuntu 781590 Apr 26 18:14 /home/ubuntu/cephtest/cephadm 2024-04-26T18:14:51.489 DEBUG:teuthology.orchestra.run.smithi103:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T18:14:52.086 INFO:teuthology.orchestra.run.smithi103.stdout:-rw-r--r--. 1 ubuntu ubuntu 781590 Apr 26 18:14 /home/ubuntu/cephtest/cephadm 2024-04-26T18:14:52.086 DEBUG:teuthology.orchestra.run.smithi174:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer-fixes/bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T18:14:53.032 INFO:teuthology.orchestra.run.smithi174.stdout:-rw-r--r--. 1 ubuntu ubuntu 781590 Apr 26 18:14 /home/ubuntu/cephtest/cephadm 2024-04-26T18:14:53.032 DEBUG:teuthology.orchestra.run.smithi084:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T18:14:53.051 DEBUG:teuthology.orchestra.run.smithi103:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T18:14:53.070 DEBUG:teuthology.orchestra.run.smithi174:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T18:14:53.107 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 on all hosts... 2024-04-26T18:14:53.107 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 pull 2024-04-26T18:14:53.109 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 pull 2024-04-26T18:14:53.113 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 pull 2024-04-26T18:14:54.680 INFO:teuthology.orchestra.run.smithi084.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T18:14:54.732 INFO:teuthology.orchestra.run.smithi103.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T18:14:54.819 INFO:teuthology.orchestra.run.smithi174.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T18:15:22.314 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-26T18:15:22.314 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_version": "ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev)", 2024-04-26T18:15:22.314 INFO:teuthology.orchestra.run.smithi103.stdout: "image_id": "3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555", 2024-04-26T18:15:22.314 INFO:teuthology.orchestra.run.smithi103.stdout: "repo_digests": [ 2024-04-26T18:15:22.315 INFO:teuthology.orchestra.run.smithi103.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc" 2024-04-26T18:15:22.315 INFO:teuthology.orchestra.run.smithi103.stdout: ] 2024-04-26T18:15:22.315 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-26T18:15:34.770 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-04-26T18:15:34.770 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_version": "ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev)", 2024-04-26T18:15:34.770 INFO:teuthology.orchestra.run.smithi174.stdout: "image_id": "3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555", 2024-04-26T18:15:34.770 INFO:teuthology.orchestra.run.smithi174.stdout: "repo_digests": [ 2024-04-26T18:15:34.770 INFO:teuthology.orchestra.run.smithi174.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc" 2024-04-26T18:15:34.770 INFO:teuthology.orchestra.run.smithi174.stdout: ] 2024-04-26T18:15:34.770 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-04-26T18:15:57.247 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:15:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_version": "ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev)", 2024-04-26T18:15:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: "image_id": "3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555", 2024-04-26T18:15:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: "repo_digests": [ 2024-04-26T18:15:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc" 2024-04-26T18:15:57.247 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-04-26T18:15:57.247 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:15:57.271 DEBUG:teuthology.orchestra.run.smithi084:> sudo mkdir -p /etc/ceph 2024-04-26T18:15:57.329 DEBUG:teuthology.orchestra.run.smithi103:> sudo mkdir -p /etc/ceph 2024-04-26T18:15:57.362 DEBUG:teuthology.orchestra.run.smithi174:> sudo mkdir -p /etc/ceph 2024-04-26T18:15:57.393 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 777 /etc/ceph 2024-04-26T18:15:57.416 DEBUG:teuthology.orchestra.run.smithi103:> sudo chmod 777 /etc/ceph 2024-04-26T18:15:57.444 DEBUG:teuthology.orchestra.run.smithi174:> sudo chmod 777 /etc/ceph 2024-04-26T18:15:57.472 INFO:tasks.cephadm:Writing seed config... 2024-04-26T18:15:57.473 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T18:15:57.473 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T18:15:57.473 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T18:15:57.473 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T18:15:57.473 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T18:15:57.473 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-26T18:15:57.473 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T18:15:57.473 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T18:15:57.474 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-26T18:15:57.475 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-26T18:15:57.476 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T18:15:57.476 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T18:15:57.476 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-26T18:15:57.476 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T18:15:57.476 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T18:15:57.476 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T18:15:57.476 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T18:15:57.476 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T18:15:57.476 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T18:15:57.476 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-26T18:15:57.476 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-26T18:15:57.477 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:15:57.477 DEBUG:teuthology.orchestra.run.smithi084:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T18:15:57.492 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = df3f3816-03f8-11ef-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T18:15:57.492 DEBUG:teuthology.orchestra.run.smithi084:mon.a> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.a.service 2024-04-26T18:15:57.534 DEBUG:teuthology.orchestra.run.smithi084:mgr.x> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mgr.x.service 2024-04-26T18:15:57.577 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T18:15:57.578 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 -v bootstrap --fsid df3f3816-03f8-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.84 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:15:57.765 INFO:teuthology.orchestra.run.smithi084.stdout:-------------------------------------------------------------------------------- 2024-04-26T18:15:57.765 INFO:teuthology.orchestra.run.smithi084.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1', '-v', 'bootstrap', '--fsid', 'df3f3816-03f8-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.84', '--skip-admin-label'] 2024-04-26T18:15:57.766 INFO:teuthology.orchestra.run.smithi084.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T18:15:57.766 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying podman|docker is present... 2024-04-26T18:15:57.792 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T18:15:57.792 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying lvm2 is present... 2024-04-26T18:15:57.792 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying time synchronization is in place... 2024-04-26T18:15:57.799 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T18:15:57.799 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T18:15:57.805 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T18:15:57.805 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout inactive 2024-04-26T18:15:57.811 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout enabled 2024-04-26T18:15:57.818 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout active 2024-04-26T18:15:57.818 INFO:teuthology.orchestra.run.smithi084.stdout:Unit chronyd.service is enabled and running 2024-04-26T18:15:57.818 INFO:teuthology.orchestra.run.smithi084.stdout:Repeating the final host check... 2024-04-26T18:15:57.842 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T18:15:57.842 INFO:teuthology.orchestra.run.smithi084.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T18:15:57.842 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl is present 2024-04-26T18:15:57.842 INFO:teuthology.orchestra.run.smithi084.stdout:lvcreate is present 2024-04-26T18:15:57.848 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T18:15:57.848 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T18:15:57.854 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T18:15:57.854 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout inactive 2024-04-26T18:15:57.860 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout enabled 2024-04-26T18:15:57.866 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout active 2024-04-26T18:15:57.866 INFO:teuthology.orchestra.run.smithi084.stdout:Unit chronyd.service is enabled and running 2024-04-26T18:15:57.866 INFO:teuthology.orchestra.run.smithi084.stdout:Host looks OK 2024-04-26T18:15:57.867 INFO:teuthology.orchestra.run.smithi084.stdout:Cluster fsid: df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:15:57.867 INFO:teuthology.orchestra.run.smithi084.stdout:Acquiring lock 140039319735456 on /run/cephadm/df3f3816-03f8-11ef-bc93-c7b262605968.lock 2024-04-26T18:15:57.867 INFO:teuthology.orchestra.run.smithi084.stdout:Lock 140039319735456 acquired on /run/cephadm/df3f3816-03f8-11ef-bc93-c7b262605968.lock 2024-04-26T18:15:57.867 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying IP 172.21.15.84 port 3300 ... 2024-04-26T18:15:57.867 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying IP 172.21.15.84 port 6789 ... 2024-04-26T18:15:57.867 INFO:teuthology.orchestra.run.smithi084.stdout:Base mon IP(s) is [172.21.15.84:3300, 172.21.15.84:6789], mon addrv is [v2:172.21.15.84:3300,v1:172.21.15.84:6789] 2024-04-26T18:15:57.870 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.84 metric 100 2024-04-26T18:15:57.870 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.84 metric 100 2024-04-26T18:15:57.873 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T18:15:57.873 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-26T18:15:57.873 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-26T18:15:57.875 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T18:15:57.875 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T18:15:57.875 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T18:15:57.875 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-26T18:15:57.876 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout inet6 fe80::9516:45:208e:5bbf/64 scope link noprefixroute 2024-04-26T18:15:57.876 INFO:teuthology.orchestra.run.smithi084.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T18:15:57.876 INFO:teuthology.orchestra.run.smithi084.stdout:Mon IP `172.21.15.84` is in CIDR network `172.21.0.0/20` 2024-04-26T18:15:57.876 INFO:teuthology.orchestra.run.smithi084.stdout:Mon IP `172.21.15.84` is in CIDR network `172.21.0.0/20` 2024-04-26T18:15:57.876 INFO:teuthology.orchestra.run.smithi084.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T18:15:57.877 INFO:teuthology.orchestra.run.smithi084.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T18:15:57.878 INFO:teuthology.orchestra.run.smithi084.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T18:15:58.423 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stdout 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 2024-04-26T18:15:58.424 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1... 2024-04-26T18:15:58.424 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T18:15:58.424 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stderr Copying blob sha256:449452a284afe5840255d087488a224329a4679c5f5d200bb867a639c75ddfd2 2024-04-26T18:15:58.424 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stderr Copying blob sha256:3d665f4c032644be9cae748af04bee69f2bfe96cdc3616dabe0ab17899633ab5 2024-04-26T18:15:58.424 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stderr Copying config sha256:3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 2024-04-26T18:15:58.424 INFO:teuthology.orchestra.run.smithi084.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T18:15:59.243 INFO:teuthology.orchestra.run.smithi084.stdout:ceph: stdout ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:15:59.243 INFO:teuthology.orchestra.run.smithi084.stdout:Ceph version: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev) 2024-04-26T18:15:59.243 INFO:teuthology.orchestra.run.smithi084.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T18:16:00.219 INFO:teuthology.orchestra.run.smithi084.stdout:stat: stdout 167 167 2024-04-26T18:16:00.219 INFO:teuthology.orchestra.run.smithi084.stdout:Creating initial keys... 2024-04-26T18:16:01.168 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph-authtool: stdout AQBg7ytm9Tf+IRAAfJGHkmLrhtiMqvULxKUsvA== 2024-04-26T18:16:02.091 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph-authtool: stdout AQBh7ytmYw4BHRAAf5giMLD11joHj/iLsguvoQ== 2024-04-26T18:16:02.988 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph-authtool: stdout AQBi7ytmQIvfGRAAHTAsgLBLu8RGplkFiTRqwA== 2024-04-26T18:16:02.989 INFO:teuthology.orchestra.run.smithi084.stdout:Creating initial monmap... 2024-04-26T18:16:03.872 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T18:16:03.872 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-26T18:16:03.872 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:16:03.872 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T18:16:03.872 INFO:teuthology.orchestra.run.smithi084.stdout:monmaptool for a [v2:172.21.15.84:3300,v1:172.21.15.84:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T18:16:03.873 INFO:teuthology.orchestra.run.smithi084.stdout:setting min_mon_release = quincy 2024-04-26T18:16:03.873 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/monmaptool: set fsid to df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:16:03.873 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T18:16:03.873 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:16:03.873 INFO:teuthology.orchestra.run.smithi084.stdout:Creating mon... 2024-04-26T18:16:04.812 INFO:teuthology.orchestra.run.smithi084.stdout:create mon.a on 2024-04-26T18:16:05.018 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T18:16:05.171 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T18:16:05.346 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-df3f3816-03f8-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-df3f3816-03f8-11ef-bc93-c7b262605968.target. 2024-04-26T18:16:05.347 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-df3f3816-03f8-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-df3f3816-03f8-11ef-bc93-c7b262605968.target. 2024-04-26T18:16:05.595 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.a 2024-04-26T18:16:05.596 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Failed to reset failed state of unit ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.a.service: Unit ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.a.service not loaded. 2024-04-26T18:16:05.766 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-df3f3816-03f8-11ef-bc93-c7b262605968.target.wants/ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-df3f3816-03f8-11ef-bc93-c7b262605968@.service. 2024-04-26T18:16:06.304 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T18:16:06.305 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout disabled 2024-04-26T18:16:06.313 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T18:16:06.313 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout inactive 2024-04-26T18:16:06.313 INFO:teuthology.orchestra.run.smithi084.stdout:firewalld.service is not enabled 2024-04-26T18:16:06.313 INFO:teuthology.orchestra.run.smithi084.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T18:16:06.313 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for mon to start... 2024-04-26T18:16:06.313 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for mon... 2024-04-26T18:16:07.375 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout id: df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout services: 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.568684s) 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout data: 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T18:16:07.376 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-04-26T18:16:07.377 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.758+0000 7fa4e0e4a700 1 Processor -- start 2024-04-26T18:16:07.377 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.758+0000 7fa4e0e4a700 1 -- start start 2024-04-26T18:16:07.377 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.758+0000 7fa4e0e4a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 0x7fa4dc07db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:07.377 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.758+0000 7fa4e0e4a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc07e150 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.377 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.758+0000 7fa4da59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 0x7fa4dc07db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:07.377 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.758+0000 7fa4da59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 0x7fa4dc07db80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42896/0 (socket says 172.21.15.84:42896) 2024-04-26T18:16:07.377 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.758+0000 7fa4da59c700 1 -- 172.21.15.84:0/3827363570 learned_addr learned my addr 172.21.15.84:0/3827363570 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:07.377 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.759+0000 7fa4da59c700 1 -- 172.21.15.84:0/3827363570 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4dc07e2d0 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.377 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.759+0000 7fa4da59c700 1 --2- 172.21.15.84:0/3827363570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 0x7fa4dc07db80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0098a0 tx=0x7fa4cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a184eb85b4f417f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:07.377 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.759+0000 7fa4d959a700 1 -- 172.21.15.84:0/3827363570 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4cc004160 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.377 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.759+0000 7fa4d959a700 1 -- 172.21.15.84:0/3827363570 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4cc004300 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.377 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.759+0000 7fa4d959a700 1 -- 172.21.15.84:0/3827363570 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4cc0036a0 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.760+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3827363570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 msgr2=0x7fa4dc07db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.760+0000 7fa4e0e4a700 1 --2- 172.21.15.84:0/3827363570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 0x7fa4dc07db80 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0098a0 tx=0x7fa4cc009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.760+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3827363570 shutdown_connections 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.760+0000 7fa4e0e4a700 1 --2- 172.21.15.84:0/3827363570 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 0x7fa4dc07db80 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.760+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3827363570 >> 172.21.15.84:0/3827363570 conn(0x7fa4dc104470 msgr2=0x7fa4dc1068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.760+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3827363570 shutdown_connections 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.760+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3827363570 wait complete. 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.761+0000 7fa4e0e4a700 1 Processor -- start 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.761+0000 7fa4e0e4a700 1 -- start start 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.761+0000 7fa4e0e4a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 0x7fa4dc19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.761+0000 7fa4e0e4a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa4dc07e800 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.761+0000 7fa4da59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 0x7fa4dc19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.761+0000 7fa4da59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 0x7fa4dc19e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42902/0 (socket says 172.21.15.84:42902) 2024-04-26T18:16:07.378 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.761+0000 7fa4da59c700 1 -- 172.21.15.84:0/3640721532 learned_addr learned my addr 172.21.15.84:0/3640721532 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:07.379 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.761+0000 7fa4da59c700 1 -- 172.21.15.84:0/3640721532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4dc19ecd0 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.379 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.761+0000 7fa4da59c700 1 --2- 172.21.15.84:0/3640721532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 0x7fa4dc19e790 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc000c00 tx=0x7fa4cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:07.379 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.761+0000 7fa4cb7fe700 1 -- 172.21.15.84:0/3640721532 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4cc003db0 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.379 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.761+0000 7fa4cb7fe700 1 -- 172.21.15.84:0/3640721532 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4cc011630 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.379 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.761+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3640721532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4dc19ef60 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.379 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.761+0000 7fa4cb7fe700 1 -- 172.21.15.84:0/3640721532 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4cc011950 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.379 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.762+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3640721532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4dc1a1c50 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.379 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.762+0000 7fa4cb7fe700 1 -- 172.21.15.84:0/3640721532 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa4cc011af0 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.379 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.762+0000 7fa4cb7fe700 1 -- 172.21.15.84:0/3640721532 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa4cc01c450 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.379 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.762+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3640721532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4dc064a80 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.379 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.765+0000 7fa4cb7fe700 1 -- 172.21.15.84:0/3640721532 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa4cc032020 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.379 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.834+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3640721532 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa4dc02e6a0 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.379 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.834+0000 7fa4cb7fe700 1 -- 172.21.15.84:0/3640721532 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fa4cc004260 con 0x7fa4dc07f6d0 2024-04-26T18:16:07.380 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.835+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3640721532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 msgr2=0x7fa4dc19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:07.380 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.835+0000 7fa4e0e4a700 1 --2- 172.21.15.84:0/3640721532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 0x7fa4dc19e790 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc000c00 tx=0x7fa4cc004750 comp rx=0 tx=0).stop 2024-04-26T18:16:07.380 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.835+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3640721532 shutdown_connections 2024-04-26T18:16:07.380 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.835+0000 7fa4e0e4a700 1 --2- 172.21.15.84:0/3640721532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa4dc07f6d0 0x7fa4dc19e790 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:07.380 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.835+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3640721532 >> 172.21.15.84:0/3640721532 conn(0x7fa4dc104470 msgr2=0x7fa4dc106790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:07.380 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.835+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3640721532 shutdown_connections 2024-04-26T18:16:07.380 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:06.835+0000 7fa4e0e4a700 1 -- 172.21.15.84:0/3640721532 wait complete. 2024-04-26T18:16:07.380 INFO:teuthology.orchestra.run.smithi084.stdout:mon is available 2024-04-26T18:16:07.380 INFO:teuthology.orchestra.run.smithi084.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T18:16:08.414 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-04-26T18:16:08.414 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T18:16:08.414 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout fsid = df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:16:08.414 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.84:3300,v1:172.21.15.84:6789] 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T18:16:08.415 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.806+0000 7f4f5bdb0700 1 Processor -- start 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.806+0000 7f4f5bdb0700 1 -- start start 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.806+0000 7f4f5bdb0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 0x7f4f541092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.806+0000 7f4f5bdb0700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4f54109800 con 0x7f4f54108ea0 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.807+0000 7f4f59b4c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 0x7f4f541092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.807+0000 7f4f59b4c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 0x7f4f541092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42912/0 (socket says 172.21.15.84:42912) 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.807+0000 7f4f59b4c700 1 -- 172.21.15.84:0/4145322253 learned_addr learned my addr 172.21.15.84:0/4145322253 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.807+0000 7f4f59b4c700 1 -- 172.21.15.84:0/4145322253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5410a040 con 0x7f4f54108ea0 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.807+0000 7f4f59b4c700 1 --2- 172.21.15.84:0/4145322253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 0x7f4f541092c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4f500098a0 tx=0x7f4f50009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db29cfce3f702e7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.807+0000 7f4f58b4a700 1 -- 172.21.15.84:0/4145322253 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f50004160 con 0x7f4f54108ea0 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.807+0000 7f4f58b4a700 1 -- 172.21.15.84:0/4145322253 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f50004300 con 0x7f4f54108ea0 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.807+0000 7f4f58b4a700 1 -- 172.21.15.84:0/4145322253 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f500036a0 con 0x7f4f54108ea0 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.808+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/4145322253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 msgr2=0x7f4f541092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:08.416 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.808+0000 7f4f5bdb0700 1 --2- 172.21.15.84:0/4145322253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 0x7f4f541092c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4f500098a0 tx=0x7f4f50009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.808+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/4145322253 shutdown_connections 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.808+0000 7f4f5bdb0700 1 --2- 172.21.15.84:0/4145322253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 0x7f4f541092c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.808+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/4145322253 >> 172.21.15.84:0/4145322253 conn(0x7f4f54104470 msgr2=0x7f4f541068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.808+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/4145322253 shutdown_connections 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.808+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/4145322253 wait complete. 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.808+0000 7f4f5bdb0700 1 Processor -- start 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.808+0000 7f4f5bdb0700 1 -- start start 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f5bdb0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 0x7f4f5419c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f5bdb0700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5410a510 con 0x7f4f54108ea0 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f59b4c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 0x7f4f5419c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f59b4c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 0x7f4f5419c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42924/0 (socket says 172.21.15.84:42924) 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f59b4c700 1 -- 172.21.15.84:0/3688079356 learned_addr learned my addr 172.21.15.84:0/3688079356 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f59b4c700 1 -- 172.21.15.84:0/3688079356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5419cb10 con 0x7f4f54108ea0 2024-04-26T18:16:08.417 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f59b4c700 1 --2- 172.21.15.84:0/3688079356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 0x7f4f5419c5d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4f50000c00 tx=0x7f4f50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f4affd700 1 -- 172.21.15.84:0/3688079356 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f50003db0 con 0x7f4f54108ea0 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f4affd700 1 -- 172.21.15.84:0/3688079356 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4f50011630 con 0x7f4f54108ea0 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/3688079356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f5419cda0 con 0x7f4f54108ea0 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f4affd700 1 -- 172.21.15.84:0/3688079356 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f50011950 con 0x7f4f54108ea0 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/3688079356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f5419fa40 con 0x7f4f54108ea0 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f4affd700 1 -- 172.21.15.84:0/3688079356 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4f50011af0 con 0x7f4f54108ea0 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.809+0000 7f4f4affd700 1 -- 172.21.15.84:0/3688079356 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4f5001c450 con 0x7f4f54108ea0 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.810+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/3688079356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f54064a80 con 0x7f4f54108ea0 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.812+0000 7f4f4affd700 1 -- 172.21.15.84:0/3688079356 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4f50004260 con 0x7f4f54108ea0 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.878+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/3688079356 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f4f5402e610 con 0x7f4f54108ea0 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.882+0000 7f4f4affd700 1 -- 172.21.15.84:0/3688079356 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f4f5001b450 con 0x7f4f54108ea0 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.882+0000 7f4f4affd700 1 -- 172.21.15.84:0/3688079356 <== mon.0 v2:172.21.15.84:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4f5001b630 con 0x7f4f54108ea0 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.883+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/3688079356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 msgr2=0x7f4f5419c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:08.418 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.883+0000 7f4f5bdb0700 1 --2- 172.21.15.84:0/3688079356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 0x7f4f5419c5d0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4f50000c00 tx=0x7f4f50004750 comp rx=0 tx=0).stop 2024-04-26T18:16:08.419 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.883+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/3688079356 shutdown_connections 2024-04-26T18:16:08.419 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.883+0000 7f4f5bdb0700 1 --2- 172.21.15.84:0/3688079356 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f54108ea0 0x7f4f5419c5d0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:08.419 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.883+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/3688079356 >> 172.21.15.84:0/3688079356 conn(0x7f4f54104470 msgr2=0x7f4f54193230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:08.419 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.883+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/3688079356 shutdown_connections 2024-04-26T18:16:08.419 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:07.883+0000 7f4f5bdb0700 1 -- 172.21.15.84:0/3688079356 wait complete. 2024-04-26T18:16:08.419 INFO:teuthology.orchestra.run.smithi084.stdout:Generating new minimal ceph.conf... 2024-04-26T18:16:09.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:09 smithi084 ceph-mon[28767]: from='client.? 172.21.15.84:0/3877115348' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:16:14.557 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.798+0000 7faf053e3700 1 Processor -- start 2024-04-26T18:16:14.561 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.798+0000 7faf053e3700 1 -- start start 2024-04-26T18:16:14.561 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.798+0000 7faf053e3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 0x7faf0010b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:14.561 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.798+0000 7faf053e3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faf0010ba60 con 0x7faf0010b070 2024-04-26T18:16:14.562 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.798+0000 7faefeffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 0x7faf0010b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:14.562 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.798+0000 7faefeffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 0x7faf0010b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42926/0 (socket says 172.21.15.84:42926) 2024-04-26T18:16:14.562 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.798+0000 7faefeffd700 1 -- 172.21.15.84:0/3737984302 learned_addr learned my addr 172.21.15.84:0/3737984302 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:14.562 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.799+0000 7faefeffd700 1 -- 172.21.15.84:0/3737984302 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0010c280 con 0x7faf0010b070 2024-04-26T18:16:14.562 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.799+0000 7faefeffd700 1 --2- 172.21.15.84:0/3737984302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 0x7faf0010b490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faef0009e80 tx=0x7faef000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2933fa288209b6c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:14.562 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.799+0000 7faefdffb700 1 -- 172.21.15.84:0/3737984302 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef0017070 con 0x7faf0010b070 2024-04-26T18:16:14.562 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.799+0000 7faefdffb700 1 -- 172.21.15.84:0/3737984302 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faef000b8c0 con 0x7faf0010b070 2024-04-26T18:16:14.562 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.799+0000 7faefdffb700 1 -- 172.21.15.84:0/3737984302 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef0004490 con 0x7faf0010b070 2024-04-26T18:16:14.562 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.800+0000 7faf053e3700 1 -- 172.21.15.84:0/3737984302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 msgr2=0x7faf0010b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:14.562 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.800+0000 7faf053e3700 1 --2- 172.21.15.84:0/3737984302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 0x7faf0010b490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7faef0009e80 tx=0x7faef000b330 comp rx=0 tx=0).stop 2024-04-26T18:16:14.562 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.800+0000 7faf053e3700 1 -- 172.21.15.84:0/3737984302 shutdown_connections 2024-04-26T18:16:14.562 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.800+0000 7faf053e3700 1 --2- 172.21.15.84:0/3737984302 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 0x7faf0010b490 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.800+0000 7faf053e3700 1 -- 172.21.15.84:0/3737984302 >> 172.21.15.84:0/3737984302 conn(0x7faf0007e5e0 msgr2=0x7faf0007ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.800+0000 7faf053e3700 1 -- 172.21.15.84:0/3737984302 shutdown_connections 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.800+0000 7faf053e3700 1 -- 172.21.15.84:0/3737984302 wait complete. 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.800+0000 7faf053e3700 1 Processor -- start 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.800+0000 7faf053e3700 1 -- start start 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.801+0000 7faf053e3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 0x7faf0019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.801+0000 7faf053e3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faf0010f630 con 0x7faf0010b070 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.801+0000 7faefeffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 0x7faf0019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.801+0000 7faefeffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 0x7faf0019ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42934/0 (socket says 172.21.15.84:42934) 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.801+0000 7faefeffd700 1 -- 172.21.15.84:0/3877115348 learned_addr learned my addr 172.21.15.84:0/3877115348 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.801+0000 7faefeffd700 1 -- 172.21.15.84:0/3877115348 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf0019f4c0 con 0x7faf0010b070 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.801+0000 7faefeffd700 1 --2- 172.21.15.84:0/3877115348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 0x7faf0019ef80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faef000b7b0 tx=0x7faef000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.801+0000 7faef7fff700 1 -- 172.21.15.84:0/3877115348 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef0020020 con 0x7faf0010b070 2024-04-26T18:16:14.563 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.801+0000 7faef7fff700 1 -- 172.21.15.84:0/3877115348 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7faef0012430 con 0x7faf0010b070 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.801+0000 7faf053e3700 1 -- 172.21.15.84:0/3877115348 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0019f750 con 0x7faf0010b070 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.801+0000 7faef7fff700 1 -- 172.21.15.84:0/3877115348 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faef0017040 con 0x7faf0010b070 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.801+0000 7faf053e3700 1 -- 172.21.15.84:0/3877115348 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf001a2440 con 0x7faf0010b070 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.802+0000 7faef7fff700 1 -- 172.21.15.84:0/3877115348 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7faef0023020 con 0x7faf0010b070 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.802+0000 7faef7fff700 1 -- 172.21.15.84:0/3877115348 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7faef000f050 con 0x7faf0010b070 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.803+0000 7faf053e3700 1 -- 172.21.15.84:0/3877115348 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf00064a80 con 0x7faf0010b070 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.805+0000 7faef7fff700 1 -- 172.21.15.84:0/3877115348 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7faef002a020 con 0x7faf0010b070 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.872+0000 7faf053e3700 1 -- 172.21.15.84:0/3877115348 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7faf001a2a00 con 0x7faf0010b070 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.873+0000 7faef7fff700 1 -- 172.21.15.84:0/3877115348 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7faef001cc80 con 0x7faf0010b070 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.873+0000 7faf053e3700 1 -- 172.21.15.84:0/3877115348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 msgr2=0x7faf0019ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.873+0000 7faf053e3700 1 --2- 172.21.15.84:0/3877115348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 0x7faf0019ef80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7faef000b7b0 tx=0x7faef000be20 comp rx=0 tx=0).stop 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.873+0000 7faf053e3700 1 -- 172.21.15.84:0/3877115348 shutdown_connections 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.873+0000 7faf053e3700 1 --2- 172.21.15.84:0/3877115348 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf0010b070 0x7faf0019ef80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.873+0000 7faf053e3700 1 -- 172.21.15.84:0/3877115348 >> 172.21.15.84:0/3877115348 conn(0x7faf0007e5e0 msgr2=0x7faf00108d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:14.564 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.874+0000 7faf053e3700 1 -- 172.21.15.84:0/3877115348 shutdown_connections 2024-04-26T18:16:14.565 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:08.874+0000 7faf053e3700 1 -- 172.21.15.84:0/3877115348 wait complete. 2024-04-26T18:16:14.565 INFO:teuthology.orchestra.run.smithi084.stdout:Restarting the monitor... 2024-04-26T18:16:14.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:14 smithi084 systemd[1]: Stopping Ceph mon.a for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:16:16.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:15 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a[28763]: 2024-04-26T18:16:15.613+0000 7f6743569700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:16:16.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:15 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a[28763]: 2024-04-26T18:16:15.613+0000 7f6743569700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T18:16:16.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:15 smithi084 podman[29014]: 2024-04-26 18:16:15.767572692 +0000 UTC m=+1.033595393 container died 6409a1b42e2e5a727dd043f130cd109090842c78e038a3aa889a1848b49fb91d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T18:16:16.459 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:16 smithi084 podman[29014]: 2024-04-26 18:16:16.254210584 +0000 UTC m=+1.520233291 container cleanup 6409a1b42e2e5a727dd043f130cd109090842c78e038a3aa889a1848b49fb91d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T18:16:16.459 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:16 smithi084 bash[29014]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a 2024-04-26T18:16:16.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:16 smithi084 podman[29028]: 2024-04-26 18:16:16.505368925 +0000 UTC m=+0.737394391 container remove 6409a1b42e2e5a727dd043f130cd109090842c78e038a3aa889a1848b49fb91d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:16:16.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:16 smithi084 systemd[1]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-26T18:16:16.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:16 smithi084 systemd[1]: Stopped Ceph mon.a for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:16:16.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:16 smithi084 systemd[1]: Starting Ceph mon.a for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:16:17.165 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:16 smithi084 podman[29122]: 2024-04-26 18:16:16.794980271 +0000 UTC m=+0.017619635 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:16:17.166 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:16 smithi084 podman[29122]: 2024-04-26 18:16:16.897556992 +0000 UTC m=+0.120196442 container create b25e6427f5bac1cabae0495c3c24e5ae335036bcf695e962e0ab151853ef5bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-26T18:16:17.166 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 podman[29122]: 2024-04-26 18:16:17.091368829 +0000 UTC m=+0.314008165 container init b25e6427f5bac1cabae0495c3c24e5ae335036bcf695e962e0ab151853ef5bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T18:16:17.166 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 podman[29122]: 2024-04-26 18:16:17.094284978 +0000 UTC m=+0.316924315 container start b25e6427f5bac1cabae0495c3c24e5ae335036bcf695e962e0ab151853ef5bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-26T18:16:17.166 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T18:16:17.166 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev), process ceph-mon, pid 2 2024-04-26T18:16:17.166 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: pidfile_write: ignore empty --pid-file 2024-04-26T18:16:17.166 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: load: jerasure load: lrc 2024-04-26T18:16:17.166 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: RocksDB version: 7.9.2 2024-04-26T18:16:17.166 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Git sha 0 2024-04-26T18:16:17.166 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Compile date 2024-04-26 07:20:26 2024-04-26T18:16:17.166 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: DB SUMMARY 2024-04-26T18:16:17.167 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: DB Session ID: ENQ7ALZ54NG2WMYW566F 2024-04-26T18:16:17.167 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: CURRENT file: CURRENT 2024-04-26T18:16:17.167 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T18:16:17.167 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T18:16:17.167 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T18:16:17.167 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105682 ; 2024-04-26T18:16:17.167 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.error_if_exists: 0 2024-04-26T18:16:17.168 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.create_if_missing: 0 2024-04-26T18:16:17.168 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.paranoid_checks: 1 2024-04-26T18:16:17.168 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T18:16:17.168 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T18:16:17.169 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T18:16:17.169 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.env: 0x5599d11a7cc0 2024-04-26T18:16:17.169 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T18:16:17.169 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.info_log: 0x5599d1f7dd80 2024-04-26T18:16:17.169 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T18:16:17.169 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.statistics: (nil) 2024-04-26T18:16:17.169 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.use_fsync: 0 2024-04-26T18:16:17.169 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_log_file_size: 0 2024-04-26T18:16:17.169 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T18:16:17.169 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T18:16:17.169 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T18:16:17.170 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T18:16:17.170 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.allow_fallocate: 1 2024-04-26T18:16:17.170 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T18:16:17.170 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T18:16:17.170 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.use_direct_reads: 0 2024-04-26T18:16:17.170 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T18:16:17.170 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T18:16:17.170 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.db_log_dir: 2024-04-26T18:16:17.170 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.wal_dir: 2024-04-26T18:16:17.170 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T18:16:17.170 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T18:16:17.170 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T18:16:17.170 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T18:16:17.171 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T18:16:17.171 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T18:16:17.171 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T18:16:17.171 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T18:16:17.171 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.write_buffer_manager: 0x5599d1f4f0e0 2024-04-26T18:16:17.171 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T18:16:17.171 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T18:16:17.171 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T18:16:17.171 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T18:16:17.171 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T18:16:17.171 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T18:16:17.171 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T18:16:17.172 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T18:16:17.172 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.unordered_write: 0 2024-04-26T18:16:17.172 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T18:16:17.172 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T18:16:17.172 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T18:16:17.172 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T18:16:17.172 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.row_cache: None 2024-04-26T18:16:17.172 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.wal_filter: None 2024-04-26T18:16:17.172 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T18:16:17.172 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T18:16:17.172 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.two_write_queues: 0 2024-04-26T18:16:17.172 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T18:16:17.172 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.wal_compression: 0 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.atomic_flush: 0 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.log_readahead_size: 0 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_background_jobs: 2 2024-04-26T18:16:17.173 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_background_compactions: -1 2024-04-26T18:16:17.174 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_subcompactions: 1 2024-04-26T18:16:17.174 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T18:16:17.174 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T18:16:17.174 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T18:16:17.174 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T18:16:17.174 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T18:16:17.174 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T18:16:17.174 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T18:16:17.174 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T18:16:17.174 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_open_files: -1 2024-04-26T18:16:17.174 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T18:16:17.174 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_background_flushes: -1 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Compression algorithms supported: 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: kZSTD supported: 0 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: kXpressCompression supported: 0 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: kZlibCompression supported: 1 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: kSnappyCompression supported: 1 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: kLZ4Compression supported: 1 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: kBZip2Compression supported: 0 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T18:16:17.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.merge_operator: 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compaction_filter: None 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5599d1f7df40) 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout: cache_index_and_filter_blocks: 1 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T18:16:17.176 INFO:journalctl@ceph.mon.a.smithi084.stdout: pin_top_level_index_and_filter: 1 2024-04-26T18:16:17.177 INFO:journalctl@ceph.mon.a.smithi084.stdout: index_type: 0 2024-04-26T18:16:17.177 INFO:journalctl@ceph.mon.a.smithi084.stdout: data_block_index_type: 0 2024-04-26T18:16:17.177 INFO:journalctl@ceph.mon.a.smithi084.stdout: index_shortening: 1 2024-04-26T18:16:17.177 INFO:journalctl@ceph.mon.a.smithi084.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T18:16:17.177 INFO:journalctl@ceph.mon.a.smithi084.stdout: checksum: 4 2024-04-26T18:16:17.177 INFO:journalctl@ceph.mon.a.smithi084.stdout: no_block_cache: 0 2024-04-26T18:16:17.177 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_cache: 0x5599d1f811f0 2024-04-26T18:16:17.177 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_cache_name: BinnedLRUCache 2024-04-26T18:16:17.177 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_cache_options: 2024-04-26T18:16:17.177 INFO:journalctl@ceph.mon.a.smithi084.stdout: capacity : 536870912 2024-04-26T18:16:17.177 INFO:journalctl@ceph.mon.a.smithi084.stdout: num_shard_bits : 4 2024-04-26T18:16:17.177 INFO:journalctl@ceph.mon.a.smithi084.stdout: strict_capacity_limit : 0 2024-04-26T18:16:17.178 INFO:journalctl@ceph.mon.a.smithi084.stdout: high_pri_pool_ratio: 0.000 2024-04-26T18:16:17.178 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_cache_compressed: (nil) 2024-04-26T18:16:17.178 INFO:journalctl@ceph.mon.a.smithi084.stdout: persistent_cache: (nil) 2024-04-26T18:16:17.178 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_size: 4096 2024-04-26T18:16:17.178 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_size_deviation: 10 2024-04-26T18:16:17.178 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_restart_interval: 16 2024-04-26T18:16:17.178 INFO:journalctl@ceph.mon.a.smithi084.stdout: index_block_restart_interval: 1 2024-04-26T18:16:17.178 INFO:journalctl@ceph.mon.a.smithi084.stdout: metadata_block_size: 4096 2024-04-26T18:16:17.178 INFO:journalctl@ceph.mon.a.smithi084.stdout: partition_filters: 0 2024-04-26T18:16:17.178 INFO:journalctl@ceph.mon.a.smithi084.stdout: use_delta_encoding: 1 2024-04-26T18:16:17.178 INFO:journalctl@ceph.mon.a.smithi084.stdout: filter_policy: bloomfilter 2024-04-26T18:16:17.178 INFO:journalctl@ceph.mon.a.smithi084.stdout: whole_key_filtering: 1 2024-04-26T18:16:17.178 INFO:journalctl@ceph.mon.a.smithi084.stdout: verify_compression: 0 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout: read_amp_bytes_per_bit: 0 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout: format_version: 5 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout: enable_index_compression: 1 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout: block_align: 0 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout: max_auto_readahead_size: 262144 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout: prepopulate_block_cache: 0 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout: initial_auto_readahead_size: 8192 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compression: NoCompression 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T18:16:17.179 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T18:16:17.180 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.num_levels: 7 2024-04-26T18:16:17.180 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T18:16:17.180 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T18:16:17.180 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T18:16:17.180 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T18:16:17.180 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T18:16:17.180 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T18:16:17.180 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T18:16:17.180 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:16:17.180 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T18:16:17.180 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T18:16:17.180 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:16:17.180 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T18:16:17.181 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T18:16:17.182 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T18:16:17.183 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T18:16:17.183 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T18:16:17.183 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T18:16:17.183 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T18:16:17.183 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T18:16:17.183 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T18:16:17.183 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T18:16:17.183 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T18:16:17.183 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T18:16:17.183 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T18:16:17.183 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T18:16:17.183 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T18:16:17.183 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T18:16:17.184 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T18:16:17.184 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.inplace_update_support: 0 2024-04-26T18:16:17.184 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T18:16:17.184 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T18:16:17.184 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T18:16:17.185 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T18:16:17.185 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.bloom_locality: 0 2024-04-26T18:16:17.185 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.max_successive_merges: 0 2024-04-26T18:16:17.185 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T18:16:17.185 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T18:16:17.185 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T18:16:17.185 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T18:16:17.185 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.ttl: 2592000 2024-04-26T18:16:17.185 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T18:16:17.185 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T18:16:17.185 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T18:16:17.185 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.enable_blob_files: false 2024-04-26T18:16:17.186 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.min_blob_size: 0 2024-04-26T18:16:17.186 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T18:16:17.186 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T18:16:17.186 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T18:16:17.186 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T18:16:17.186 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T18:16:17.186 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T18:16:17.186 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T18:16:17.186 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T18:16:17.186 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T18:16:17.186 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T18:16:17.186 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 328e6a2b-bbc5-4f79-930e-c8dbcd028526 2024-04-26T18:16:17.186 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155377122449, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155377123443, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100789, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98677, "index_size": 304, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14302, "raw_average_key_size": 49, "raw_value_size": 90895, "raw_average_value_size": 316, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714155377, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "328e6a2b-bbc5-4f79-930e-c8dbcd028526", "db_session_id": "ENQ7ALZ54NG2WMYW566F", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155377123524, "job": 1, "event": "recovery_finished"} 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5599d209e000 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: DB pointer 0x5599d208a000 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout: ** DB Stats ** 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:16:17.187 INFO:journalctl@ceph.mon.a.smithi084.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:16:17.188 INFO:journalctl@ceph.mon.a.smithi084.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T18:16:17.188 INFO:journalctl@ceph.mon.a.smithi084.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:16:17.188 INFO:journalctl@ceph.mon.a.smithi084.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:16:17.188 INFO:journalctl@ceph.mon.a.smithi084.stdout: 2024-04-26T18:16:17.188 INFO:journalctl@ceph.mon.a.smithi084.stdout: ** Compaction Stats [default] ** 2024-04-26T18:16:17.188 INFO:journalctl@ceph.mon.a.smithi084.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:16:17.188 INFO:journalctl@ceph.mon.a.smithi084.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T18:16:17.188 INFO:journalctl@ceph.mon.a.smithi084.stdout: L0 2/0 100.29 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:16:17.188 INFO:journalctl@ceph.mon.a.smithi084.stdout: Sum 2/0 100.29 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:16:17.188 INFO:journalctl@ceph.mon.a.smithi084.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 155.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:16:17.188 INFO:journalctl@ceph.mon.a.smithi084.stdout: 2024-04-26T18:16:17.188 INFO:journalctl@ceph.mon.a.smithi084.stdout: ** Compaction Stats [default] ** 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 155.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: Cumulative compaction: 0.00 GB write, 32.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:16:17.189 INFO:journalctl@ceph.mon.a.smithi084.stdout: Interval compaction: 0.00 GB write, 32.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:16:17.190 INFO:journalctl@ceph.mon.a.smithi084.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T18:16:17.190 INFO:journalctl@ceph.mon.a.smithi084.stdout: Block cache BinnedLRUCache@0x5599d1f811f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-26T18:16:17.190 INFO:journalctl@ceph.mon.a.smithi084.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-26T18:16:17.190 INFO:journalctl@ceph.mon.a.smithi084.stdout: 2024-04-26T18:16:17.190 INFO:journalctl@ceph.mon.a.smithi084.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T18:16:17.190 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: starting mon.a rank 0 at public addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] at bind addrs [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:16:17.190 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: mon.a@-1(???) e1 preinit fsid df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:16:17.190 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: mon.a@-1(???).mds e1 new map 2024-04-26T18:16:17.190 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: mon.a@-1(???).mds e1 print_map 2024-04-26T18:16:17.190 INFO:journalctl@ceph.mon.a.smithi084.stdout: e1 2024-04-26T18:16:17.190 INFO:journalctl@ceph.mon.a.smithi084.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T18:16:17.190 INFO:journalctl@ceph.mon.a.smithi084.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T18:16:17.190 INFO:journalctl@ceph.mon.a.smithi084.stdout: legacy client fscid: -1 2024-04-26T18:16:17.191 INFO:journalctl@ceph.mon.a.smithi084.stdout: 2024-04-26T18:16:17.191 INFO:journalctl@ceph.mon.a.smithi084.stdout: No filesystems configured 2024-04-26T18:16:17.191 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T18:16:17.191 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:16:17.191 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:16:17.191 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:16:17.191 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T18:16:17.191 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 bash[29122]: b25e6427f5bac1cabae0495c3c24e5ae335036bcf695e962e0ab151853ef5bfd 2024-04-26T18:16:17.191 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 systemd[1]: Started Ceph mon.a for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:16:17.191 INFO:teuthology.orchestra.run.smithi084.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T18:16:17.468 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-26T18:16:17.468 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: monmap epoch 1 2024-04-26T18:16:17.468 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: fsid df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:16:17.468 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: last_changed 2024-04-26T18:16:03.305539+0000 2024-04-26T18:16:17.468 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: created 2024-04-26T18:16:03.305539+0000 2024-04-26T18:16:17.469 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: min_mon_release 19 (squid) 2024-04-26T18:16:17.469 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: election_strategy: 1 2024-04-26T18:16:17.469 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-04-26T18:16:17.469 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: fsmap 2024-04-26T18:16:17.469 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T18:16:17.469 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:17 smithi084 ceph-mon[29141]: mgrmap e1: no daemons active 2024-04-26T18:16:18.213 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.569+0000 7f20bbdad700 1 Processor -- start 2024-04-26T18:16:18.213 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.569+0000 7f20bbdad700 1 -- start start 2024-04-26T18:16:18.214 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.569+0000 7f20bbdad700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 0x7f20b41081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:18.214 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.569+0000 7f20bbdad700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41087b0 con 0x7f20b4107dc0 2024-04-26T18:16:18.214 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.569+0000 7f20b9b49700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 0x7f20b41081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:18.214 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.570+0000 7f20b9b49700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 0x7f20b41081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58406/0 (socket says 172.21.15.84:58406) 2024-04-26T18:16:18.214 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.570+0000 7f20b9b49700 1 -- 172.21.15.84:0/3350313124 learned_addr learned my addr 172.21.15.84:0/3350313124 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:18.214 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.570+0000 7f20b9b49700 1 -- 172.21.15.84:0/3350313124 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b4108f70 con 0x7f20b4107dc0 2024-04-26T18:16:18.214 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.570+0000 7f20b9b49700 1 --2- 172.21.15.84:0/3350313124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 0x7f20b41081e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f20b00098a0 tx=0x7f20b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66fed2112b291dce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:18.214 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.570+0000 7f20b8b47700 1 -- 172.21.15.84:0/3350313124 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20b0004160 con 0x7f20b4107dc0 2024-04-26T18:16:18.214 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.570+0000 7f20b8b47700 1 -- 172.21.15.84:0/3350313124 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f20b0004300 con 0x7f20b4107dc0 2024-04-26T18:16:18.214 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.570+0000 7f20b8b47700 1 -- 172.21.15.84:0/3350313124 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20b0011af0 con 0x7f20b4107dc0 2024-04-26T18:16:18.214 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.571+0000 7f20bbdad700 1 -- 172.21.15.84:0/3350313124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 msgr2=0x7f20b41081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:18.214 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.571+0000 7f20bbdad700 1 --2- 172.21.15.84:0/3350313124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 0x7f20b41081e0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f20b00098a0 tx=0x7f20b0009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:18.214 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.571+0000 7f20bbdad700 1 -- 172.21.15.84:0/3350313124 shutdown_connections 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.571+0000 7f20bbdad700 1 --2- 172.21.15.84:0/3350313124 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 0x7f20b41081e0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.571+0000 7f20bbdad700 1 -- 172.21.15.84:0/3350313124 >> 172.21.15.84:0/3350313124 conn(0x7f20b4103380 msgr2=0x7f20b41057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.571+0000 7f20bbdad700 1 -- 172.21.15.84:0/3350313124 shutdown_connections 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.571+0000 7f20bbdad700 1 -- 172.21.15.84:0/3350313124 wait complete. 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.571+0000 7f20bbdad700 1 Processor -- start 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.571+0000 7f20bbdad700 1 -- start start 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.572+0000 7f20bbdad700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 0x7f20b407d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.572+0000 7f20bbdad700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f20b410c320 con 0x7f20b4107dc0 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.572+0000 7f20b9b49700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 0x7f20b407d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.572+0000 7f20b9b49700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 0x7f20b407d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58414/0 (socket says 172.21.15.84:58414) 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.572+0000 7f20b9b49700 1 -- 172.21.15.84:0/6566298 learned_addr learned my addr 172.21.15.84:0/6566298 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.572+0000 7f20b9b49700 1 -- 172.21.15.84:0/6566298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b419fb70 con 0x7f20b4107dc0 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.572+0000 7f20b9b49700 1 --2- 172.21.15.84:0/6566298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 0x7f20b407d290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f20b0003f40 tx=0x7f20b0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.572+0000 7f20aaffd700 1 -- 172.21.15.84:0/6566298 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20b00038b0 con 0x7f20b4107dc0 2024-04-26T18:16:18.215 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.572+0000 7f20aaffd700 1 -- 172.21.15.84:0/6566298 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f20b0003a50 con 0x7f20b4107dc0 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.572+0000 7f20bbdad700 1 -- 172.21.15.84:0/6566298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b419fd40 con 0x7f20b4107dc0 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.572+0000 7f20aaffd700 1 -- 172.21.15.84:0/6566298 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20b001a5c0 con 0x7f20b4107dc0 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.572+0000 7f20bbdad700 1 -- 172.21.15.84:0/6566298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b41a00e0 con 0x7f20b4107dc0 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.573+0000 7f20aaffd700 1 -- 172.21.15.84:0/6566298 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f20b001ac50 con 0x7f20b4107dc0 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.573+0000 7f20aaffd700 1 -- 172.21.15.84:0/6566298 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f20b0019700 con 0x7f20b4107dc0 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.573+0000 7f20bbdad700 1 -- 172.21.15.84:0/6566298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b4064a80 con 0x7f20b4107dc0 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.576+0000 7f20aaffd700 1 -- 172.21.15.84:0/6566298 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f20b0019900 con 0x7f20b4107dc0 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.645+0000 7f20bbdad700 1 -- 172.21.15.84:0/6566298 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f20b41a03e0 con 0x7f20b4107dc0 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.650+0000 7f20aaffd700 1 -- 172.21.15.84:0/6566298 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f20b0019ae0 con 0x7f20b4107dc0 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.651+0000 7f20bbdad700 1 -- 172.21.15.84:0/6566298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 msgr2=0x7f20b407d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.651+0000 7f20bbdad700 1 --2- 172.21.15.84:0/6566298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 0x7f20b407d290 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f20b0003f40 tx=0x7f20b0011890 comp rx=0 tx=0).stop 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.651+0000 7f20bbdad700 1 -- 172.21.15.84:0/6566298 shutdown_connections 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.651+0000 7f20bbdad700 1 --2- 172.21.15.84:0/6566298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f20b4107dc0 0x7f20b407d290 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:18.216 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.651+0000 7f20bbdad700 1 -- 172.21.15.84:0/6566298 >> 172.21.15.84:0/6566298 conn(0x7f20b4103380 msgr2=0x7f20b4103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:18.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.651+0000 7f20bbdad700 1 -- 172.21.15.84:0/6566298 shutdown_connections 2024-04-26T18:16:18.217 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:17.651+0000 7f20bbdad700 1 -- 172.21.15.84:0/6566298 wait complete. 2024-04-26T18:16:18.217 INFO:teuthology.orchestra.run.smithi084.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T18:16:18.238 INFO:teuthology.orchestra.run.smithi084.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:16:18.238 INFO:teuthology.orchestra.run.smithi084.stdout:Creating mgr... 2024-04-26T18:16:18.238 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T18:16:18.239 INFO:teuthology.orchestra.run.smithi084.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T18:16:18.481 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mgr.x 2024-04-26T18:16:18.482 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Failed to reset failed state of unit ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mgr.x.service: Unit ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-26T18:16:18.623 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-df3f3816-03f8-11ef-bc93-c7b262605968.target.wants/ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-df3f3816-03f8-11ef-bc93-c7b262605968@.service. 2024-04-26T18:16:18.945 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:18 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/6566298' entity='client.admin' 2024-04-26T18:16:19.122 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T18:16:19.122 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout disabled 2024-04-26T18:16:19.129 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T18:16:19.129 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout inactive 2024-04-26T18:16:19.129 INFO:teuthology.orchestra.run.smithi084.stdout:firewalld.service is not enabled 2024-04-26T18:16:19.129 INFO:teuthology.orchestra.run.smithi084.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T18:16:19.137 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T18:16:19.137 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout disabled 2024-04-26T18:16:19.144 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T18:16:19.145 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout inactive 2024-04-26T18:16:19.145 INFO:teuthology.orchestra.run.smithi084.stdout:firewalld.service is not enabled 2024-04-26T18:16:19.145 INFO:teuthology.orchestra.run.smithi084.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-26T18:16:19.145 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for mgr to start... 2024-04-26T18:16:19.145 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for mgr... 2024-04-26T18:16:19.205 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:19 smithi084 systemd[1]: Started Ceph mgr.x for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:16:19.494 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:19 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:19.203+0000 7ffba35b7200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:16:19.494 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:19 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:19.389+0000 7ffba35b7200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:16:19.825 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:19 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1096202932' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T18:16:19.825 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:19 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:19.496+0000 7ffba35b7200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:16:19.825 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:19 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:19.823+0000 7ffba35b7200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:16:20.094 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:19 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:19.909+0000 7ffba35b7200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:16:20.286 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-04-26T18:16:20.286 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:16:20.286 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:16:20.286 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T18:16:20.286 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T18:16:20.286 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T18:16:20.286 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T18:16:20.286 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:20.286 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T18:16:20.287 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T18:16:20.287 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 0 2024-04-26T18:16:20.287 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:16:20.287 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T18:16:20.292 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T18:16:20.293 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:16:20.293 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-26T18:16:20.293 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T18:16:20.293 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:16:20.293 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T18:16:20.293 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T18:16:20.293 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:20.293 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T18:16:20.293 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:16:20.293 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T18:16:20.293 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T18:16:20.293 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T18:16:20.293 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:20.294 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:16:20.295 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T18:16:06.268175+0000", 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.593+0000 7ffa41ab3700 1 Processor -- start 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.593+0000 7ffa41ab3700 1 -- start start 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.593+0000 7ffa41ab3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 0x7ffa3c10bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.593+0000 7ffa41ab3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c07cdb0 con 0x7ffa3c1099b0 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.594+0000 7ffa3b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 0x7ffa3c10bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.594+0000 7ffa3b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 0x7ffa3c10bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58450/0 (socket says 172.21.15.84:58450) 2024-04-26T18:16:20.296 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.594+0000 7ffa3b7fe700 1 -- 172.21.15.84:0/2118980384 learned_addr learned my addr 172.21.15.84:0/2118980384 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.594+0000 7ffa3b7fe700 1 -- 172.21.15.84:0/2118980384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c10c2e0 con 0x7ffa3c1099b0 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.594+0000 7ffa3b7fe700 1 --2- 172.21.15.84:0/2118980384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 0x7ffa3c10bda0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffa300098a0 tx=0x7ffa30009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c8dd7e096890 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.594+0000 7ffa3a7fc700 1 -- 172.21.15.84:0/2118980384 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa30004160 con 0x7ffa3c1099b0 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.594+0000 7ffa3a7fc700 1 -- 172.21.15.84:0/2118980384 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffa30004300 con 0x7ffa3c1099b0 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.594+0000 7ffa3a7fc700 1 -- 172.21.15.84:0/2118980384 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa30011af0 con 0x7ffa3c1099b0 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.595+0000 7ffa41ab3700 1 -- 172.21.15.84:0/2118980384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 msgr2=0x7ffa3c10bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.595+0000 7ffa41ab3700 1 --2- 172.21.15.84:0/2118980384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 0x7ffa3c10bda0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ffa300098a0 tx=0x7ffa30009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.595+0000 7ffa41ab3700 1 -- 172.21.15.84:0/2118980384 shutdown_connections 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.595+0000 7ffa41ab3700 1 --2- 172.21.15.84:0/2118980384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 0x7ffa3c10bda0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.595+0000 7ffa41ab3700 1 -- 172.21.15.84:0/2118980384 >> 172.21.15.84:0/2118980384 conn(0x7ffa3c1033c0 msgr2=0x7ffa3c1057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.595+0000 7ffa41ab3700 1 -- 172.21.15.84:0/2118980384 shutdown_connections 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.595+0000 7ffa41ab3700 1 -- 172.21.15.84:0/2118980384 wait complete. 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.596+0000 7ffa41ab3700 1 Processor -- start 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.596+0000 7ffa41ab3700 1 -- start start 2024-04-26T18:16:20.297 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.596+0000 7ffa41ab3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 0x7ffa3c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:20.298 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.596+0000 7ffa41ab3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffa3c10d240 con 0x7ffa3c1099b0 2024-04-26T18:16:20.298 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.596+0000 7ffa3b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 0x7ffa3c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:20.298 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.596+0000 7ffa3b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 0x7ffa3c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58460/0 (socket says 172.21.15.84:58460) 2024-04-26T18:16:20.298 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.596+0000 7ffa3b7fe700 1 -- 172.21.15.84:0/1096202932 learned_addr learned my addr 172.21.15.84:0/1096202932 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:20.298 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.597+0000 7ffa3b7fe700 1 -- 172.21.15.84:0/1096202932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa3c19d3c0 con 0x7ffa3c1099b0 2024-04-26T18:16:20.298 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.597+0000 7ffa3b7fe700 1 --2- 172.21.15.84:0/1096202932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 0x7ffa3c19ce80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009fb0 tx=0x7ffa30011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:20.298 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.597+0000 7ffa38ff9700 1 -- 172.21.15.84:0/1096202932 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa300036a0 con 0x7ffa3c1099b0 2024-04-26T18:16:20.298 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.597+0000 7ffa38ff9700 1 -- 172.21.15.84:0/1096202932 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7ffa3001a430 con 0x7ffa3c1099b0 2024-04-26T18:16:20.298 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.597+0000 7ffa41ab3700 1 -- 172.21.15.84:0/1096202932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa3c19d650 con 0x7ffa3c1099b0 2024-04-26T18:16:20.298 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.597+0000 7ffa38ff9700 1 -- 172.21.15.84:0/1096202932 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa30019550 con 0x7ffa3c1099b0 2024-04-26T18:16:20.298 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.597+0000 7ffa41ab3700 1 -- 172.21.15.84:0/1096202932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa3c1a0340 con 0x7ffa3c1099b0 2024-04-26T18:16:20.298 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.597+0000 7ffa38ff9700 1 -- 172.21.15.84:0/1096202932 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ffa30019b90 con 0x7ffa3c1099b0 2024-04-26T18:16:20.299 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.597+0000 7ffa38ff9700 1 -- 172.21.15.84:0/1096202932 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ffa30024730 con 0x7ffa3c1099b0 2024-04-26T18:16:20.299 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.598+0000 7ffa41ab3700 1 -- 172.21.15.84:0/1096202932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa3c064a80 con 0x7ffa3c1099b0 2024-04-26T18:16:20.299 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.602+0000 7ffa38ff9700 1 -- 172.21.15.84:0/1096202932 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7ffa30032070 con 0x7ffa3c1099b0 2024-04-26T18:16:20.299 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.676+0000 7ffa41ab3700 1 -- 172.21.15.84:0/1096202932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7ffa3c1a08c0 con 0x7ffa3c1099b0 2024-04-26T18:16:20.299 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.677+0000 7ffa38ff9700 1 -- 172.21.15.84:0/1096202932 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7ffa3001a5e0 con 0x7ffa3c1099b0 2024-04-26T18:16:20.299 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.678+0000 7ffa41ab3700 1 -- 172.21.15.84:0/1096202932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 msgr2=0x7ffa3c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:20.299 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.678+0000 7ffa41ab3700 1 --2- 172.21.15.84:0/1096202932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 0x7ffa3c19ce80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ffa30009fb0 tx=0x7ffa30011940 comp rx=0 tx=0).stop 2024-04-26T18:16:20.299 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.678+0000 7ffa41ab3700 1 -- 172.21.15.84:0/1096202932 shutdown_connections 2024-04-26T18:16:20.299 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.678+0000 7ffa41ab3700 1 --2- 172.21.15.84:0/1096202932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa3c1099b0 0x7ffa3c19ce80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:20.299 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.678+0000 7ffa41ab3700 1 -- 172.21.15.84:0/1096202932 >> 172.21.15.84:0/1096202932 conn(0x7ffa3c1033c0 msgr2=0x7ffa3c103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:20.299 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.678+0000 7ffa41ab3700 1 -- 172.21.15.84:0/1096202932 shutdown_connections 2024-04-26T18:16:20.299 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:19.678+0000 7ffa41ab3700 1 -- 172.21.15.84:0/1096202932 wait complete. 2024-04-26T18:16:20.300 INFO:teuthology.orchestra.run.smithi084.stdout:mgr not available, waiting (1/15)... 2024-04-26T18:16:21.431 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:21 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:21.096+0000 7ffba35b7200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:16:21.431 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:21 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:21.263+0000 7ffba35b7200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:16:21.431 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:21 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:21.429+0000 7ffba35b7200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:16:22.010 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:21 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:21.716+0000 7ffba35b7200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:16:22.010 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:21 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:21.807+0000 7ffba35b7200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:16:22.010 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:22.009+0000 7ffba35b7200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:16:22.344 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:22.201+0000 7ffba35b7200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:16:23.094 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:22.798+0000 7ffba35b7200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:16:23.558 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:23 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:23.216+0000 7ffba35b7200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:16:23.558 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:23 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:23.311+0000 7ffba35b7200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:16:23.558 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:23 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:23.513+0000 7ffba35b7200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:16:24.344 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:24.037+0000 7ffba35b7200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:16:24.344 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:24.130+0000 7ffba35b7200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:16:24.712 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:24.371+0000 7ffba35b7200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:16:24.713 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:24.504+0000 7ffba35b7200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:16:24.713 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:24.598+0000 7ffba35b7200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:16:24.713 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:24.712+0000 7ffba35b7200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:16:25.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:25 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1453073370' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T18:16:25.345 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:25 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:25.015+0000 7ffba35b7200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:16:25.743 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-04-26T18:16:25.743 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:16:25.743 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:16:25.743 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 0 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_age": 7, 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T18:16:25.744 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:16:25.745 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T18:16:25.745 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T18:16:25.745 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:25.745 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T18:16:25.745 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:16:25.745 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T18:16:25.745 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T18:16:25.745 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T18:16:25.745 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T18:16:25.745 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T18:16:25.745 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T18:16:25.745 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:25.745 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T18:16:25.746 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T18:16:25.746 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T18:16:25.746 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T18:16:25.746 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T18:16:25.746 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T18:16:25.746 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T18:16:25.746 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T18:16:25.746 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T18:16:25.746 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:25.746 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T18:16:25.746 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:16:25.746 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T18:16:25.746 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:16:25.747 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T18:16:06.268175+0000", 2024-04-26T18:16:25.748 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:16:25.748 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:25.748 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T18:16:25.748 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:16:25.748 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.034+0000 7f3b82cda700 1 Processor -- start 2024-04-26T18:16:25.748 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.034+0000 7f3b82cda700 1 -- start start 2024-04-26T18:16:25.748 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.034+0000 7f3b82cda700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 0x7f3b7c10a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:25.748 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.034+0000 7f3b82cda700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c10ae40 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.748 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.034+0000 7f3b80a76700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 0x7f3b7c10a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:25.748 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.034+0000 7f3b80a76700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 0x7f3b7c10a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58466/0 (socket says 172.21.15.84:58466) 2024-04-26T18:16:25.748 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.034+0000 7f3b80a76700 1 -- 172.21.15.84:0/733516132 learned_addr learned my addr 172.21.15.84:0/733516132 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:25.748 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.035+0000 7f3b80a76700 1 -- 172.21.15.84:0/733516132 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7c10afc0 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.748 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.035+0000 7f3b80a76700 1 --2- 172.21.15.84:0/733516132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 0x7f3b7c10a900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3b700098a0 tx=0x7f3b70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b684bbaaae7a75b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.035+0000 7f3b7b7fe700 1 -- 172.21.15.84:0/733516132 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b70004160 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.035+0000 7f3b7b7fe700 1 -- 172.21.15.84:0/733516132 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3b70004300 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.035+0000 7f3b7b7fe700 1 -- 172.21.15.84:0/733516132 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b70011af0 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.036+0000 7f3b82cda700 1 -- 172.21.15.84:0/733516132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 msgr2=0x7f3b7c10a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.036+0000 7f3b82cda700 1 --2- 172.21.15.84:0/733516132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 0x7f3b7c10a900 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3b700098a0 tx=0x7f3b70009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.036+0000 7f3b82cda700 1 -- 172.21.15.84:0/733516132 shutdown_connections 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.036+0000 7f3b82cda700 1 --2- 172.21.15.84:0/733516132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 0x7f3b7c10a900 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.036+0000 7f3b82cda700 1 -- 172.21.15.84:0/733516132 >> 172.21.15.84:0/733516132 conn(0x7f3b7c102b40 msgr2=0x7f3b7c104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.036+0000 7f3b82cda700 1 -- 172.21.15.84:0/733516132 shutdown_connections 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.036+0000 7f3b82cda700 1 -- 172.21.15.84:0/733516132 wait complete. 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.037+0000 7f3b82cda700 1 Processor -- start 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.037+0000 7f3b82cda700 1 -- start start 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.037+0000 7f3b82cda700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 0x7f3b7c19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:25.749 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.037+0000 7f3b82cda700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3b7c107cf0 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.750 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.037+0000 7f3b80a76700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 0x7f3b7c19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:25.750 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.037+0000 7f3b80a76700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 0x7f3b7c19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58482/0 (socket says 172.21.15.84:58482) 2024-04-26T18:16:25.750 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.037+0000 7f3b80a76700 1 -- 172.21.15.84:0/1453073370 learned_addr learned my addr 172.21.15.84:0/1453073370 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:25.750 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.037+0000 7f3b80a76700 1 -- 172.21.15.84:0/1453073370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b7c19d3c0 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.750 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.038+0000 7f3b80a76700 1 --2- 172.21.15.84:0/1453073370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 0x7f3b7c19ce80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b700117b0 tx=0x7f3b70011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:25.750 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.038+0000 7f3b79ffb700 1 -- 172.21.15.84:0/1453073370 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b70003af0 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.750 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.038+0000 7f3b79ffb700 1 -- 172.21.15.84:0/1453073370 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3b70020070 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.750 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.038+0000 7f3b79ffb700 1 -- 172.21.15.84:0/1453073370 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3b70019630 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.750 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.038+0000 7f3b82cda700 1 -- 172.21.15.84:0/1453073370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b7c19d650 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.750 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.038+0000 7f3b82cda700 1 -- 172.21.15.84:0/1453073370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b7c1a0340 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.750 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.038+0000 7f3b79ffb700 1 -- 172.21.15.84:0/1453073370 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f3b70019c80 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.750 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.038+0000 7f3b79ffb700 1 -- 172.21.15.84:0/1453073370 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f3b7000b720 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.750 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.039+0000 7f3b82cda700 1 -- 172.21.15.84:0/1453073370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b7c064a80 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.751 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.043+0000 7f3b79ffb700 1 -- 172.21.15.84:0/1453073370 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f3b700042a0 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.751 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.115+0000 7f3b82cda700 1 -- 172.21.15.84:0/1453073370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f3b7c1a0880 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.751 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.115+0000 7f3b79ffb700 1 -- 172.21.15.84:0/1453073370 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f3b7001a9c0 con 0x7f3b7c10a4e0 2024-04-26T18:16:25.751 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.116+0000 7f3b82cda700 1 -- 172.21.15.84:0/1453073370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 msgr2=0x7f3b7c19ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:25.751 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.116+0000 7f3b82cda700 1 --2- 172.21.15.84:0/1453073370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 0x7f3b7c19ce80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b700117b0 tx=0x7f3b70011b70 comp rx=0 tx=0).stop 2024-04-26T18:16:25.751 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.116+0000 7f3b82cda700 1 -- 172.21.15.84:0/1453073370 shutdown_connections 2024-04-26T18:16:25.751 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.116+0000 7f3b82cda700 1 --2- 172.21.15.84:0/1453073370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b7c10a4e0 0x7f3b7c19ce80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:25.751 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.116+0000 7f3b82cda700 1 -- 172.21.15.84:0/1453073370 >> 172.21.15.84:0/1453073370 conn(0x7f3b7c102b40 msgr2=0x7f3b7c103670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:25.751 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.117+0000 7f3b82cda700 1 -- 172.21.15.84:0/1453073370 shutdown_connections 2024-04-26T18:16:25.751 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:25.117+0000 7f3b82cda700 1 -- 172.21.15.84:0/1453073370 wait complete. 2024-04-26T18:16:25.751 INFO:teuthology.orchestra.run.smithi084.stdout:mgr not available, waiting (2/15)... 2024-04-26T18:16:25.844 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:25 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:25.642+0000 7ffba35b7200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:16:26.167 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:25 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:25.859+0000 7ffba35b7200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:16:26.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:26 smithi084 ceph-mon[29141]: Activating manager daemon x 2024-04-26T18:16:26.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:26 smithi084 ceph-mon[29141]: mgrmap e2: x(active, starting, since 0.00325815s) 2024-04-26T18:16:26.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:26 smithi084 ceph-mon[29141]: from='mgr.14100 172.21.15.84:0/4029521113' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:16:26.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:26 smithi084 ceph-mon[29141]: from='mgr.14100 172.21.15.84:0/4029521113' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:16:26.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:26 smithi084 ceph-mon[29141]: from='mgr.14100 172.21.15.84:0/4029521113' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:16:26.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:26 smithi084 ceph-mon[29141]: from='mgr.14100 172.21.15.84:0/4029521113' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:16:26.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:26 smithi084 ceph-mon[29141]: from='mgr.14100 172.21.15.84:0/4029521113' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:16:26.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:26 smithi084 ceph-mon[29141]: Manager daemon x is now available 2024-04-26T18:16:26.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:26 smithi084 ceph-mon[29141]: from='mgr.14100 172.21.15.84:0/4029521113' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:16:26.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:26 smithi084 ceph-mon[29141]: from='mgr.14100 172.21.15.84:0/4029521113' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:16:26.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:26 smithi084 ceph-mon[29141]: from='mgr.14100 172.21.15.84:0/4029521113' entity='mgr.x' 2024-04-26T18:16:26.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:26 smithi084 ceph-mon[29141]: from='mgr.14100 172.21.15.84:0/4029521113' entity='mgr.x' 2024-04-26T18:16:26.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:26 smithi084 ceph-mon[29141]: from='mgr.14100 172.21.15.84:0/4029521113' entity='mgr.x' 2024-04-26T18:16:28.140 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:27 smithi084 ceph-mon[29141]: mgrmap e3: x(active, since 1.00678s) 2024-04-26T18:16:29.048 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-04-26T18:16:29.048 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:16:29.048 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 0 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "a" 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T18:16:29.049 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:16:29.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-26T18:16:29.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T18:16:29.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:29.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T18:16:29.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:16:29.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T18:16:29.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T18:16:29.050 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T18:16:29.058 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:29.059 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ], 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:29.060 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T18:16:06.268175+0000", 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout }, 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.154+0000 7eff12b79700 1 Processor -- start 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.154+0000 7eff12b79700 1 -- start start 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.154+0000 7eff12b79700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c07d530 0x7eff0c10ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.154+0000 7eff12b79700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c10bff0 con 0x7eff0c07d530 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.154+0000 7eff10915700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c07d530 0x7eff0c10ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.154+0000 7eff10915700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c07d530 0x7eff0c10ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56764/0 (socket says 172.21.15.84:56764) 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.154+0000 7eff10915700 1 -- 172.21.15.84:0/4192593490 learned_addr learned my addr 172.21.15.84:0/4192593490 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:29.061 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.155+0000 7eff10915700 1 -- 172.21.15.84:0/4192593490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0c10c170 con 0x7eff0c07d530 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.155+0000 7eff10915700 1 --2- 172.21.15.84:0/4192593490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c07d530 0x7eff0c10ba20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7eff000098a0 tx=0x7eff00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e7857f3a403f35 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.155+0000 7eff0b7fe700 1 -- 172.21.15.84:0/4192593490 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff00004160 con 0x7eff0c07d530 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.155+0000 7eff0b7fe700 1 -- 172.21.15.84:0/4192593490 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7eff00004300 con 0x7eff0c07d530 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.155+0000 7eff0b7fe700 1 -- 172.21.15.84:0/4192593490 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff00011af0 con 0x7eff0c07d530 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.155+0000 7eff12b79700 1 -- 172.21.15.84:0/4192593490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c07d530 msgr2=0x7eff0c10ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.155+0000 7eff12b79700 1 --2- 172.21.15.84:0/4192593490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c07d530 0x7eff0c10ba20 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7eff000098a0 tx=0x7eff00009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.155+0000 7eff12b79700 1 -- 172.21.15.84:0/4192593490 shutdown_connections 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.155+0000 7eff12b79700 1 --2- 172.21.15.84:0/4192593490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c07d530 0x7eff0c10ba20 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.155+0000 7eff12b79700 1 -- 172.21.15.84:0/4192593490 >> 172.21.15.84:0/4192593490 conn(0x7eff0c103380 msgr2=0x7eff0c1057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.156+0000 7eff12b79700 1 -- 172.21.15.84:0/4192593490 shutdown_connections 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.156+0000 7eff12b79700 1 -- 172.21.15.84:0/4192593490 wait complete. 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.156+0000 7eff12b79700 1 Processor -- start 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.156+0000 7eff12b79700 1 -- start start 2024-04-26T18:16:29.062 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.156+0000 7eff12b79700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c109250 0x7eff0c1079c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.156+0000 7eff12b79700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7eff0c10d1f0 con 0x7eff0c109250 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.156+0000 7eff10915700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c109250 0x7eff0c1079c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.156+0000 7eff10915700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c109250 0x7eff0c1079c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56780/0 (socket says 172.21.15.84:56780) 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.156+0000 7eff10915700 1 -- 172.21.15.84:0/3291212554 learned_addr learned my addr 172.21.15.84:0/3291212554 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.157+0000 7eff10915700 1 -- 172.21.15.84:0/3291212554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0c109670 con 0x7eff0c109250 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.157+0000 7eff10915700 1 --2- 172.21.15.84:0/3291212554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c109250 0x7eff0c1079c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7eff00011ac0 tx=0x7eff00011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.157+0000 7eff09ffb700 1 -- 172.21.15.84:0/3291212554 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff00003b20 con 0x7eff0c109250 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.157+0000 7eff09ffb700 1 -- 172.21.15.84:0/3291212554 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7eff0001a8c0 con 0x7eff0c109250 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.157+0000 7eff12b79700 1 -- 172.21.15.84:0/3291212554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff0c107f00 con 0x7eff0c109250 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.157+0000 7eff09ffb700 1 -- 172.21.15.84:0/3291212554 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7eff000248d0 con 0x7eff0c109250 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.157+0000 7eff12b79700 1 -- 172.21.15.84:0/3291212554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0c1083e0 con 0x7eff0c109250 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.158+0000 7eff09ffb700 1 -- 172.21.15.84:0/3291212554 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7eff00022020 con 0x7eff0c109250 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.158+0000 7eff09ffb700 1 --2- 172.21.15.84:0/3291212554 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7efefc0395b0 0x7efefc03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:29.063 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.158+0000 7eff09ffb700 1 -- 172.21.15.84:0/3291212554 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7eff0004d4c0 con 0x7eff0c109250 2024-04-26T18:16:29.064 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.158+0000 7eff0bfff700 1 --2- 172.21.15.84:0/3291212554 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7efefc0395b0 0x7efefc03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:29.064 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.159+0000 7eff12b79700 1 -- 172.21.15.84:0/3291212554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0c064a80 con 0x7eff0c109250 2024-04-26T18:16:29.064 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.159+0000 7eff0bfff700 1 --2- 172.21.15.84:0/3291212554 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7efefc0395b0 0x7efefc03ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efef8006fd0 tx=0x7efef8006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:29.065 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.168+0000 7eff09ffb700 1 -- 172.21.15.84:0/3291212554 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff0001ade0 con 0x7eff0c109250 2024-04-26T18:16:29.065 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.460+0000 7eff12b79700 1 -- 172.21.15.84:0/3291212554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7eff0c07d530 con 0x7eff0c109250 2024-04-26T18:16:29.065 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.460+0000 7eff09ffb700 1 -- 172.21.15.84:0/3291212554 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7eff0004d220 con 0x7eff0c109250 2024-04-26T18:16:29.065 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.463+0000 7eff12b79700 1 -- 172.21.15.84:0/3291212554 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7efefc0395b0 msgr2=0x7efefc03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:29.065 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.463+0000 7eff12b79700 1 --2- 172.21.15.84:0/3291212554 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7efefc0395b0 0x7efefc03ba70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efef8006fd0 tx=0x7efef8006e40 comp rx=0 tx=0).stop 2024-04-26T18:16:29.065 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.463+0000 7eff12b79700 1 -- 172.21.15.84:0/3291212554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c109250 msgr2=0x7eff0c1079c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:29.065 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.463+0000 7eff12b79700 1 --2- 172.21.15.84:0/3291212554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c109250 0x7eff0c1079c0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7eff00011ac0 tx=0x7eff00011ba0 comp rx=0 tx=0).stop 2024-04-26T18:16:29.066 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.463+0000 7eff12b79700 1 -- 172.21.15.84:0/3291212554 shutdown_connections 2024-04-26T18:16:29.066 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.463+0000 7eff12b79700 1 --2- 172.21.15.84:0/3291212554 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7efefc0395b0 0x7efefc03ba70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:29.066 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.463+0000 7eff12b79700 1 --2- 172.21.15.84:0/3291212554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7eff0c109250 0x7eff0c1079c0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:29.066 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.463+0000 7eff12b79700 1 -- 172.21.15.84:0/3291212554 >> 172.21.15.84:0/3291212554 conn(0x7eff0c103380 msgr2=0x7eff0c103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:29.066 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.463+0000 7eff12b79700 1 -- 172.21.15.84:0/3291212554 shutdown_connections 2024-04-26T18:16:29.066 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:28.463+0000 7eff12b79700 1 -- 172.21.15.84:0/3291212554 wait complete. 2024-04-26T18:16:29.066 INFO:teuthology.orchestra.run.smithi084.stdout:mgr is available 2024-04-26T18:16:29.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:28 smithi084 ceph-mon[29141]: mgrmap e4: x(active, since 2s) 2024-04-26T18:16:29.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:28 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3291212554' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T18:16:30.307 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:29 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/993874975' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T18:16:30.318 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-04-26T18:16:30.319 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T18:16:30.319 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout fsid = df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:16:30.319 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-26T18:16:30.319 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T18:16:30.319 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T18:16:30.319 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T18:16:30.319 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T18:16:30.319 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-04-26T18:16:30.319 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T18:16:30.319 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T18:16:30.319 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 2024-04-26T18:16:30.319 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T18:16:30.319 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T18:16:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T18:16:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T18:16:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T18:16:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T18:16:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.466+0000 7f74ad79c700 1 Processor -- start 2024-04-26T18:16:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.466+0000 7f74ad79c700 1 -- start start 2024-04-26T18:16:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.466+0000 7f74ad79c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 0x7f74a810bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.466+0000 7f74ad79c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f74a807d640 con 0x7f74a8109980 2024-04-26T18:16:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.467+0000 7f74a6ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 0x7f74a810bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.467+0000 7f74a6ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 0x7f74a810bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56782/0 (socket says 172.21.15.84:56782) 2024-04-26T18:16:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.467+0000 7f74a6ffd700 1 -- 172.21.15.84:0/3470198839 learned_addr learned my addr 172.21.15.84:0/3470198839 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.467+0000 7f74a6ffd700 1 -- 172.21.15.84:0/3470198839 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a810c2b0 con 0x7f74a8109980 2024-04-26T18:16:30.320 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.467+0000 7f74a6ffd700 1 --2- 172.21.15.84:0/3470198839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 0x7f74a810bd70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f74980098a0 tx=0x7f7498009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10c5d05812cb944b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:30.321 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.467+0000 7f74a67fc700 1 -- 172.21.15.84:0/3470198839 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7498004160 con 0x7f74a8109980 2024-04-26T18:16:30.321 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.467+0000 7f74a67fc700 1 -- 172.21.15.84:0/3470198839 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7498004300 con 0x7f74a8109980 2024-04-26T18:16:30.321 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.467+0000 7f74a67fc700 1 -- 172.21.15.84:0/3470198839 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7498011af0 con 0x7f74a8109980 2024-04-26T18:16:30.321 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.468+0000 7f74ad79c700 1 -- 172.21.15.84:0/3470198839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 msgr2=0x7f74a810bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:30.321 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.468+0000 7f74ad79c700 1 --2- 172.21.15.84:0/3470198839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 0x7f74a810bd70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f74980098a0 tx=0x7f7498009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:30.321 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.468+0000 7f74ad79c700 1 -- 172.21.15.84:0/3470198839 shutdown_connections 2024-04-26T18:16:30.321 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.468+0000 7f74ad79c700 1 --2- 172.21.15.84:0/3470198839 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 0x7f74a810bd70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:30.321 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.468+0000 7f74ad79c700 1 -- 172.21.15.84:0/3470198839 >> 172.21.15.84:0/3470198839 conn(0x7f74a81033b0 msgr2=0x7f74a81057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:30.321 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.468+0000 7f74ad79c700 1 -- 172.21.15.84:0/3470198839 shutdown_connections 2024-04-26T18:16:30.321 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.468+0000 7f74ad79c700 1 -- 172.21.15.84:0/3470198839 wait complete. 2024-04-26T18:16:30.321 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.468+0000 7f74ad79c700 1 Processor -- start 2024-04-26T18:16:30.321 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.469+0000 7f74ad79c700 1 -- start start 2024-04-26T18:16:30.321 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.469+0000 7f74ad79c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 0x7f74a819cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:30.322 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.469+0000 7f74ad79c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f74a810d210 con 0x7f74a8109980 2024-04-26T18:16:30.322 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.469+0000 7f74a6ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 0x7f74a819cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:30.322 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.469+0000 7f74a6ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 0x7f74a819cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56796/0 (socket says 172.21.15.84:56796) 2024-04-26T18:16:30.322 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.469+0000 7f74a6ffd700 1 -- 172.21.15.84:0/993874975 learned_addr learned my addr 172.21.15.84:0/993874975 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:30.322 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.469+0000 7f74a6ffd700 1 -- 172.21.15.84:0/993874975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74a819d2f0 con 0x7f74a8109980 2024-04-26T18:16:30.322 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.469+0000 7f74a6ffd700 1 --2- 172.21.15.84:0/993874975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 0x7f74a819cdb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7498003f40 tx=0x7f7498011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:30.322 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.469+0000 7f74a4ff9700 1 -- 172.21.15.84:0/993874975 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f74980038b0 con 0x7f74a8109980 2024-04-26T18:16:30.322 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.469+0000 7f74a4ff9700 1 -- 172.21.15.84:0/993874975 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7498003a50 con 0x7f74a8109980 2024-04-26T18:16:30.322 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.469+0000 7f74a4ff9700 1 -- 172.21.15.84:0/993874975 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f749801a5c0 con 0x7f74a8109980 2024-04-26T18:16:30.322 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.469+0000 7f74ad79c700 1 -- 172.21.15.84:0/993874975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74a819d580 con 0x7f74a8109980 2024-04-26T18:16:30.322 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.469+0000 7f74ad79c700 1 -- 172.21.15.84:0/993874975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74a81a0230 con 0x7f74a8109980 2024-04-26T18:16:30.322 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.470+0000 7f74a4ff9700 1 -- 172.21.15.84:0/993874975 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f749801a870 con 0x7f74a8109980 2024-04-26T18:16:30.323 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.470+0000 7f74ad79c700 1 -- 172.21.15.84:0/993874975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74a8064a80 con 0x7f74a8109980 2024-04-26T18:16:30.323 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.470+0000 7f74a4ff9700 1 --2- 172.21.15.84:0/993874975 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f7494039560 0x7f749403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:30.323 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.471+0000 7f74a4ff9700 1 -- 172.21.15.84:0/993874975 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f749804dab0 con 0x7f74a8109980 2024-04-26T18:16:30.323 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.471+0000 7f749e7fc700 1 --2- 172.21.15.84:0/993874975 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f7494039560 0x7f749403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:30.323 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.471+0000 7f749e7fc700 1 --2- 172.21.15.84:0/993874975 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f7494039560 0x7f749403ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7490006fd0 tx=0x7f7490006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:30.323 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.475+0000 7f74a4ff9700 1 -- 172.21.15.84:0/993874975 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f74980248c0 con 0x7f74a8109980 2024-04-26T18:16:30.323 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.671+0000 7f74ad79c700 1 -- 172.21.15.84:0/993874975 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f74a81a0400 con 0x7f74a8109980 2024-04-26T18:16:30.323 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.672+0000 7f74a4ff9700 1 -- 172.21.15.84:0/993874975 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f7498019c70 con 0x7f74a8109980 2024-04-26T18:16:30.323 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.674+0000 7f74ad79c700 1 -- 172.21.15.84:0/993874975 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f7494039560 msgr2=0x7f749403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:30.323 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.674+0000 7f74ad79c700 1 --2- 172.21.15.84:0/993874975 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f7494039560 0x7f749403ba20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7490006fd0 tx=0x7f7490006e40 comp rx=0 tx=0).stop 2024-04-26T18:16:30.323 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.674+0000 7f74ad79c700 1 -- 172.21.15.84:0/993874975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 msgr2=0x7f74a819cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:30.323 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.674+0000 7f74ad79c700 1 --2- 172.21.15.84:0/993874975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 0x7f74a819cdb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7498003f40 tx=0x7f7498011890 comp rx=0 tx=0).stop 2024-04-26T18:16:30.324 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.674+0000 7f74ad79c700 1 -- 172.21.15.84:0/993874975 shutdown_connections 2024-04-26T18:16:30.324 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.674+0000 7f74ad79c700 1 --2- 172.21.15.84:0/993874975 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f7494039560 0x7f749403ba20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:30.324 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.674+0000 7f74ad79c700 1 --2- 172.21.15.84:0/993874975 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74a8109980 0x7f74a819cdb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:30.324 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.674+0000 7f74ad79c700 1 -- 172.21.15.84:0/993874975 >> 172.21.15.84:0/993874975 conn(0x7f74a81033b0 msgr2=0x7f74a8103f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:30.324 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.674+0000 7f74ad79c700 1 -- 172.21.15.84:0/993874975 shutdown_connections 2024-04-26T18:16:30.324 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:29.674+0000 7f74ad79c700 1 -- 172.21.15.84:0/993874975 wait complete. 2024-04-26T18:16:30.324 INFO:teuthology.orchestra.run.smithi084.stdout:Enabling cephadm module... 2024-04-26T18:16:31.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:31 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1321689159' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T18:16:31.345 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:31 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: ignoring --setuser ceph since I am not root 2024-04-26T18:16:31.345 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:31 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: ignoring --setgroup ceph since I am not root 2024-04-26T18:16:31.345 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:31 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:31.191+0000 7f4d4854c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:16:31.630 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:31 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:31.378+0000 7f4d4854c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:16:31.630 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:31 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:31.480+0000 7f4d4854c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:16:31.638 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.706+0000 7f5fb4c4a700 1 Processor -- start 2024-04-26T18:16:31.639 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.706+0000 7f5fb4c4a700 1 -- start start 2024-04-26T18:16:31.639 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.706+0000 7f5fb4c4a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 0x7f5fb01092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:31.639 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.706+0000 7f5fb4c4a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb0109800 con 0x7f5fb0108ea0 2024-04-26T18:16:31.639 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.706+0000 7f5fae59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 0x7f5fb01092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:31.639 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.707+0000 7f5fae59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 0x7f5fb01092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56800/0 (socket says 172.21.15.84:56800) 2024-04-26T18:16:31.639 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.707+0000 7f5fae59c700 1 -- 172.21.15.84:0/1614290486 learned_addr learned my addr 172.21.15.84:0/1614290486 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:31.639 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.707+0000 7f5fae59c700 1 -- 172.21.15.84:0/1614290486 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb010a040 con 0x7f5fb0108ea0 2024-04-26T18:16:31.639 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.707+0000 7f5fae59c700 1 --2- 172.21.15.84:0/1614290486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 0x7f5fb01092c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009e80 tx=0x7f5fa800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98709801ae7fa15 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:31.639 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.707+0000 7f5fad59a700 1 -- 172.21.15.84:0/1614290486 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa8017070 con 0x7f5fb0108ea0 2024-04-26T18:16:31.639 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.707+0000 7f5fad59a700 1 -- 172.21.15.84:0/1614290486 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5fa800b8c0 con 0x7f5fb0108ea0 2024-04-26T18:16:31.639 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.707+0000 7f5fad59a700 1 -- 172.21.15.84:0/1614290486 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa8004490 con 0x7f5fb0108ea0 2024-04-26T18:16:31.639 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.708+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1614290486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 msgr2=0x7f5fb01092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:31.640 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.708+0000 7f5fb4c4a700 1 --2- 172.21.15.84:0/1614290486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 0x7f5fb01092c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8009e80 tx=0x7f5fa800b330 comp rx=0 tx=0).stop 2024-04-26T18:16:31.640 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.708+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1614290486 shutdown_connections 2024-04-26T18:16:31.640 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.708+0000 7f5fb4c4a700 1 --2- 172.21.15.84:0/1614290486 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 0x7f5fb01092c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:31.640 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.708+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1614290486 >> 172.21.15.84:0/1614290486 conn(0x7f5fb0104470 msgr2=0x7f5fb01068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:31.640 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.708+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1614290486 shutdown_connections 2024-04-26T18:16:31.640 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.708+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1614290486 wait complete. 2024-04-26T18:16:31.640 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.709+0000 7f5fb4c4a700 1 Processor -- start 2024-04-26T18:16:31.640 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.709+0000 7f5fb4c4a700 1 -- start start 2024-04-26T18:16:31.640 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.709+0000 7f5fb4c4a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 0x7f5fb007f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:31.640 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.709+0000 7f5fb4c4a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5fb010d300 con 0x7f5fb0108ea0 2024-04-26T18:16:31.640 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.709+0000 7f5fae59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 0x7f5fb007f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:31.640 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.709+0000 7f5fae59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 0x7f5fb007f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56802/0 (socket says 172.21.15.84:56802) 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.709+0000 7f5fae59c700 1 -- 172.21.15.84:0/1321689159 learned_addr learned my addr 172.21.15.84:0/1321689159 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.709+0000 7f5fae59c700 1 -- 172.21.15.84:0/1321689159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fb007f8f0 con 0x7f5fb0108ea0 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.709+0000 7f5fae59c700 1 --2- 172.21.15.84:0/1321689159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 0x7f5fb007f3b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8004a60 tx=0x7f5fa800bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.709+0000 7f5fa77fe700 1 -- 172.21.15.84:0/1321689159 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa8020020 con 0x7f5fb0108ea0 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.709+0000 7f5fa77fe700 1 -- 172.21.15.84:0/1321689159 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5fa8003c60 con 0x7f5fb0108ea0 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.709+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1321689159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fb007dae0 con 0x7f5fb0108ea0 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.709+0000 7f5fa77fe700 1 -- 172.21.15.84:0/1321689159 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5fa8017040 con 0x7f5fb0108ea0 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.710+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1321689159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fb007e020 con 0x7f5fb0108ea0 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.710+0000 7f5fa77fe700 1 -- 172.21.15.84:0/1321689159 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f5fa8023020 con 0x7f5fb0108ea0 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.710+0000 7f5fa77fe700 1 --2- 172.21.15.84:0/1321689159 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f5f9c0395b0 0x7f5f9c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.710+0000 7f5fa77fe700 1 -- 172.21.15.84:0/1321689159 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5fa800f050 con 0x7f5fb0108ea0 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.710+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1321689159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fb0064a80 con 0x7f5fb0108ea0 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.711+0000 7f5fadd9b700 1 --2- 172.21.15.84:0/1321689159 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f5f9c0395b0 0x7f5f9c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:31.641 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.711+0000 7f5fadd9b700 1 --2- 172.21.15.84:0/1321689159 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f5f9c0395b0 0x7f5f9c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5f98006fd0 tx=0x7f5f98006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.716+0000 7f5fa77fe700 1 -- 172.21.15.84:0/1321689159 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5fa802a020 con 0x7f5fb0108ea0 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:30.974+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1321689159 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f5fb007e240 con 0x7f5fb0108ea0 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:31.025+0000 7f5fa77fe700 1 -- 172.21.15.84:0/1321689159 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f5fa8003e10 con 0x7f5fb0108ea0 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:31.027+0000 7f5fa77fe700 1 -- 172.21.15.84:0/1321689159 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f5fa8025400 con 0x7f5fb0108ea0 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:31.028+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1321689159 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f5f9c0395b0 msgr2=0x7f5f9c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:31.028+0000 7f5fb4c4a700 1 --2- 172.21.15.84:0/1321689159 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f5f9c0395b0 0x7f5f9c03ba70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5f98006fd0 tx=0x7f5f98006e40 comp rx=0 tx=0).stop 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:31.028+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1321689159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 msgr2=0x7f5fb007f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:31.028+0000 7f5fb4c4a700 1 --2- 172.21.15.84:0/1321689159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 0x7f5fb007f3b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8004a60 tx=0x7f5fa800bd50 comp rx=0 tx=0).stop 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:31.028+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1321689159 shutdown_connections 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:31.028+0000 7f5fb4c4a700 1 --2- 172.21.15.84:0/1321689159 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f5f9c0395b0 0x7f5f9c03ba70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:31.028+0000 7f5fb4c4a700 1 --2- 172.21.15.84:0/1321689159 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5fb0108ea0 0x7f5fb007f3b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:31.028+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1321689159 >> 172.21.15.84:0/1321689159 conn(0x7f5fb0104470 msgr2=0x7f5fb0105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:31.028+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1321689159 shutdown_connections 2024-04-26T18:16:31.642 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:31.028+0000 7f5fb4c4a700 1 -- 172.21.15.84:0/1321689159 wait complete. 2024-04-26T18:16:31.885 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:31 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:31.800+0000 7f4d4854c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:16:31.885 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:31 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:31.885+0000 7f4d4854c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:16:32.306 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:32 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1321689159' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T18:16:32.306 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:32 smithi084 ceph-mon[29141]: mgrmap e5: x(active, since 5s) 2024-04-26T18:16:32.939 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:16:32.941 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T18:16:32.941 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T18:16:32.941 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T18:16:32.941 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T18:16:32.941 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:16:32.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.100+0000 7f012b4aa700 1 Processor -- start 2024-04-26T18:16:32.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.100+0000 7f012b4aa700 1 -- start start 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.100+0000 7f012b4aa700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 0x7f012407db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.100+0000 7f012b4aa700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f012407e080 con 0x7f012407f640 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.100+0000 7f0129246700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 0x7f012407db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.101+0000 7f0129246700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 0x7f012407db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56826/0 (socket says 172.21.15.84:56826) 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.101+0000 7f0129246700 1 -- 172.21.15.84:0/2331878880 learned_addr learned my addr 172.21.15.84:0/2331878880 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.101+0000 7f0129246700 1 -- 172.21.15.84:0/2331878880 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012407e200 con 0x7f012407f640 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.101+0000 7f0129246700 1 --2- 172.21.15.84:0/2331878880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 0x7f012407db40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f01140098a0 tx=0x7f0114009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be163d68d3dd4b86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.101+0000 7f011bfff700 1 -- 172.21.15.84:0/2331878880 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0114004160 con 0x7f012407f640 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.101+0000 7f011bfff700 1 -- 172.21.15.84:0/2331878880 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0114004300 con 0x7f012407f640 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.101+0000 7f011bfff700 1 -- 172.21.15.84:0/2331878880 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0114011af0 con 0x7f012407f640 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.102+0000 7f012b4aa700 1 -- 172.21.15.84:0/2331878880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 msgr2=0x7f012407db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.102+0000 7f012b4aa700 1 --2- 172.21.15.84:0/2331878880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 0x7f012407db40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f01140098a0 tx=0x7f0114009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.102+0000 7f012b4aa700 1 -- 172.21.15.84:0/2331878880 shutdown_connections 2024-04-26T18:16:32.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.102+0000 7f012b4aa700 1 --2- 172.21.15.84:0/2331878880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 0x7f012407db40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.102+0000 7f012b4aa700 1 -- 172.21.15.84:0/2331878880 >> 172.21.15.84:0/2331878880 conn(0x7f0124104530 msgr2=0x7f0124106950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.102+0000 7f012b4aa700 1 -- 172.21.15.84:0/2331878880 shutdown_connections 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.102+0000 7f012b4aa700 1 -- 172.21.15.84:0/2331878880 wait complete. 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.102+0000 7f012b4aa700 1 Processor -- start 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.103+0000 7f012b4aa700 1 -- start start 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.103+0000 7f012b4aa700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 0x7f012419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.103+0000 7f012b4aa700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f012410f4c0 con 0x7f012407f640 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.103+0000 7f0129246700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 0x7f012419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.103+0000 7f0129246700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 0x7f012419eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56842/0 (socket says 172.21.15.84:56842) 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.103+0000 7f0129246700 1 -- 172.21.15.84:0/3890915641 learned_addr learned my addr 172.21.15.84:0/3890915641 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.103+0000 7f0129246700 1 -- 172.21.15.84:0/3890915641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f012419f530 con 0x7f012407f640 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.103+0000 7f0129246700 1 --2- 172.21.15.84:0/3890915641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 0x7f012419eff0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0114003f40 tx=0x7f0114011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.103+0000 7f011a7fc700 1 -- 172.21.15.84:0/3890915641 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01140038b0 con 0x7f012407f640 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.103+0000 7f012b4aa700 1 -- 172.21.15.84:0/3890915641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f012419f7c0 con 0x7f012407f640 2024-04-26T18:16:32.948 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.104+0000 7f011a7fc700 1 -- 172.21.15.84:0/3890915641 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0114003a50 con 0x7f012407f640 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.104+0000 7f011a7fc700 1 -- 172.21.15.84:0/3890915641 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f011401a5c0 con 0x7f012407f640 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.104+0000 7f012b4aa700 1 -- 172.21.15.84:0/3890915641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01241a24b0 con 0x7f012407f640 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.105+0000 7f011a7fc700 1 -- 172.21.15.84:0/3890915641 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f011401a8c0 con 0x7f012407f640 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.105+0000 7f012b4aa700 1 -- 172.21.15.84:0/3890915641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0124064a80 con 0x7f012407f640 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.105+0000 7f011a7fc700 1 --2- 172.21.15.84:0/3890915641 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f0110039560 0x7f011003ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.105+0000 7f011a7fc700 1 -- 172.21.15.84:0/3890915641 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f011404db30 con 0x7f012407f640 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.105+0000 7f0128a45700 1 -- 172.21.15.84:0/3890915641 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f0110039560 msgr2=0x7f011003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/1448727861 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.105+0000 7f0128a45700 1 --2- 172.21.15.84:0/3890915641 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f0110039560 0x7f011003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.114+0000 7f011a7fc700 1 -- 172.21.15.84:0/3890915641 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0114019c70 con 0x7f012407f640 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.306+0000 7f0128a45700 1 -- 172.21.15.84:0/3890915641 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f0110039560 msgr2=0x7f011003ba20 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/1448727861 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.306+0000 7f0128a45700 1 --2- 172.21.15.84:0/3890915641 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f0110039560 0x7f011003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.412+0000 7f012b4aa700 1 -- 172.21.15.84:0/3890915641 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f01241a2680 con 0x7f012407f640 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.412+0000 7f011a7fc700 1 -- 172.21.15.84:0/3890915641 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f01241a2680 con 0x7f012407f640 2024-04-26T18:16:32.949 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.414+0000 7f012b4aa700 1 -- 172.21.15.84:0/3890915641 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f0110039560 msgr2=0x7f011003ba20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:16:32.950 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.414+0000 7f012b4aa700 1 --2- 172.21.15.84:0/3890915641 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f0110039560 0x7f011003ba20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:32.950 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.414+0000 7f012b4aa700 1 -- 172.21.15.84:0/3890915641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 msgr2=0x7f012419eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:32.950 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.414+0000 7f012b4aa700 1 --2- 172.21.15.84:0/3890915641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 0x7f012419eff0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f0114003f40 tx=0x7f0114011890 comp rx=0 tx=0).stop 2024-04-26T18:16:32.950 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.414+0000 7f012b4aa700 1 -- 172.21.15.84:0/3890915641 shutdown_connections 2024-04-26T18:16:32.950 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.414+0000 7f012b4aa700 1 --2- 172.21.15.84:0/3890915641 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f0110039560 0x7f011003ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:32.950 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.414+0000 7f012b4aa700 1 --2- 172.21.15.84:0/3890915641 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f012407f640 0x7f012419eff0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:32.950 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.414+0000 7f012b4aa700 1 -- 172.21.15.84:0/3890915641 >> 172.21.15.84:0/3890915641 conn(0x7f0124104530 msgr2=0x7f0124105090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:32.950 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.414+0000 7f012b4aa700 1 -- 172.21.15.84:0/3890915641 shutdown_connections 2024-04-26T18:16:32.950 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:32.414+0000 7f012b4aa700 1 -- 172.21.15.84:0/3890915641 wait complete. 2024-04-26T18:16:32.950 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for the mgr to restart... 2024-04-26T18:16:32.950 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for mgr epoch 5... 2024-04-26T18:16:33.304 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:33 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3890915641' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T18:16:33.305 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:33 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:33.127+0000 7f4d4854c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:16:33.573 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:33 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:33.304+0000 7f4d4854c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:16:33.573 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:33 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:33.478+0000 7f4d4854c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:16:33.844 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:33 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:33.764+0000 7f4d4854c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:16:34.246 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:33 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:33.855+0000 7f4d4854c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:16:34.246 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:34 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:34.056+0000 7f4d4854c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:16:34.594 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:34 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:34.246+0000 7f4d4854c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:16:35.094 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:34 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:34.835+0000 7f4d4854c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:16:35.554 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:35 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:35.255+0000 7f4d4854c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:16:35.554 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:35 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:35.353+0000 7f4d4854c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:16:35.554 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:35 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:35.554+0000 7f4d4854c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:16:36.344 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:36 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:36.071+0000 7f4d4854c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:16:36.344 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:36 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:36.163+0000 7f4d4854c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:16:36.632 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:36 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:36.406+0000 7f4d4854c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:16:36.632 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:36 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:36.537+0000 7f4d4854c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:16:36.632 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:36 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:36.632+0000 7f4d4854c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:16:37.043 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:36 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:36.747+0000 7f4d4854c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:16:37.043 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:37.042+0000 7f4d4854c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:16:37.910 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:37.640+0000 7f4d4854c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:16:37.910 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:37.857+0000 7f4d4854c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:16:38.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: Active manager daemon x restarted 2024-04-26T18:16:38.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: Activating manager daemon x 2024-04-26T18:16:38.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T18:16:38.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: mgrmap e6: x(active, starting, since 0.00366081s) 2024-04-26T18:16:38.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:16:38.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:16:38.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:16:38.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:16:38.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:16:38.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: Manager daemon x is now available 2024-04-26T18:16:38.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:38.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:38.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:38.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:38.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:37 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:16:39.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:38 smithi084 ceph-mon[29141]: Found migration_current of "None". Setting to last migration. 2024-04-26T18:16:39.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:38 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:16:39.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:38 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:39.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:38 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:39.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:38 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:39.175 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:38 smithi084 ceph-mon[29141]: mgrmap e7: x(active, since 1.00578s) 2024-04-26T18:16:39.469 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:16:39.470 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T18:16:39.470 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T18:16:39.470 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:16:39.470 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.366+0000 7f560c92a700 1 Processor -- start 2024-04-26T18:16:39.470 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.367+0000 7f560c92a700 1 -- start start 2024-04-26T18:16:39.470 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.367+0000 7f560c92a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 0x7f560810bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:39.470 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.367+0000 7f560c92a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f560807d660 con 0x7f5608109970 2024-04-26T18:16:39.470 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.367+0000 7f560659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 0x7f560810bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:39.470 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.367+0000 7f560659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 0x7f560810bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56858/0 (socket says 172.21.15.84:56858) 2024-04-26T18:16:39.471 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.367+0000 7f560659c700 1 -- 172.21.15.84:0/3662536576 learned_addr learned my addr 172.21.15.84:0/3662536576 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:39.471 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.367+0000 7f560659c700 1 -- 172.21.15.84:0/3662536576 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560810c2a0 con 0x7f5608109970 2024-04-26T18:16:39.471 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.367+0000 7f560659c700 1 --2- 172.21.15.84:0/3662536576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 0x7f560810bd60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f55f80098a0 tx=0x7f55f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1636c058f0417d2a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:39.472 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.368+0000 7f560559a700 1 -- 172.21.15.84:0/3662536576 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55f8004160 con 0x7f5608109970 2024-04-26T18:16:39.472 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.368+0000 7f560559a700 1 -- 172.21.15.84:0/3662536576 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f55f8004300 con 0x7f5608109970 2024-04-26T18:16:39.472 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.368+0000 7f560559a700 1 -- 172.21.15.84:0/3662536576 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55f8011af0 con 0x7f5608109970 2024-04-26T18:16:39.472 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.368+0000 7f560c92a700 1 -- 172.21.15.84:0/3662536576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 msgr2=0x7f560810bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:39.472 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.368+0000 7f560c92a700 1 --2- 172.21.15.84:0/3662536576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 0x7f560810bd60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f55f80098a0 tx=0x7f55f8009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:39.472 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.369+0000 7f560c92a700 1 -- 172.21.15.84:0/3662536576 shutdown_connections 2024-04-26T18:16:39.472 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.369+0000 7f560c92a700 1 --2- 172.21.15.84:0/3662536576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 0x7f560810bd60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:39.472 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.369+0000 7f560c92a700 1 -- 172.21.15.84:0/3662536576 >> 172.21.15.84:0/3662536576 conn(0x7f56081033e0 msgr2=0x7f5608105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:39.472 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.369+0000 7f560c92a700 1 -- 172.21.15.84:0/3662536576 shutdown_connections 2024-04-26T18:16:39.472 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.369+0000 7f560c92a700 1 -- 172.21.15.84:0/3662536576 wait complete. 2024-04-26T18:16:39.472 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.369+0000 7f560c92a700 1 Processor -- start 2024-04-26T18:16:39.472 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.369+0000 7f560c92a700 1 -- start start 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.370+0000 7f560c92a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 0x7f560819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.370+0000 7f560c92a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f560810d200 con 0x7f5608109970 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.370+0000 7f560659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 0x7f560819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.370+0000 7f560659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 0x7f560819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56872/0 (socket says 172.21.15.84:56872) 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.370+0000 7f560659c700 1 -- 172.21.15.84:0/3837928445 learned_addr learned my addr 172.21.15.84:0/3837928445 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.370+0000 7f560659c700 1 -- 172.21.15.84:0/3837928445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f560819d2e0 con 0x7f5608109970 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.370+0000 7f560659c700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 0x7f560819cda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f55f8011ac0 tx=0x7f55f8011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.370+0000 7f55f77fe700 1 -- 172.21.15.84:0/3837928445 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55f8003b20 con 0x7f5608109970 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.370+0000 7f560c92a700 1 -- 172.21.15.84:0/3837928445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f560819d570 con 0x7f5608109970 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.370+0000 7f55f77fe700 1 -- 172.21.15.84:0/3837928445 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f55f801a8c0 con 0x7f5608109970 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.370+0000 7f55f77fe700 1 -- 172.21.15.84:0/3837928445 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f55f80248d0 con 0x7f5608109970 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.370+0000 7f560c92a700 1 -- 172.21.15.84:0/3837928445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56081a0260 con 0x7f5608109970 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.371+0000 7f55f77fe700 1 -- 172.21.15.84:0/3837928445 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f55f8022020 con 0x7f5608109970 2024-04-26T18:16:39.473 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.371+0000 7f55f77fe700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 0x7f55f003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.371+0000 7f55f77fe700 1 -- 172.21.15.84:0/3837928445 --> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f55f003c180 con 0x7f55f00395b0 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.372+0000 7f55f77fe700 1 -- 172.21.15.84:0/3837928445 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f55f804d6b0 con 0x7f5608109970 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.372+0000 7f5605d9b700 1 -- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 msgr2=0x7f55f003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/1448727861 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.372+0000 7f5605d9b700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 0x7f55f003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.573+0000 7f5605d9b700 1 -- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 msgr2=0x7f55f003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/1448727861 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.573+0000 7f5605d9b700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 0x7f55f003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.974+0000 7f5605d9b700 1 -- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 msgr2=0x7f55f003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/1448727861 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:33.974+0000 7f5605d9b700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 0x7f55f003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:34.775+0000 7f5605d9b700 1 -- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 msgr2=0x7f55f003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/1448727861 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:34.775+0000 7f5605d9b700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 0x7f55f003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:36.376+0000 7f5605d9b700 1 -- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 msgr2=0x7f55f003ba70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/1448727861 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:36.376+0000 7f5605d9b700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 0x7f55f003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:37.861+0000 7f55f77fe700 1 -- 172.21.15.84:0/3837928445 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f55f801a5c0 con 0x7f5608109970 2024-04-26T18:16:39.474 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:37.861+0000 7f55f77fe700 1 -- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 msgr2=0x7f55f003ba70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:37.861+0000 7f55f77fe700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 0x7f55f003ba70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.862+0000 7f55f77fe700 1 -- 172.21.15.84:0/3837928445 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f55f801aa70 con 0x7f5608109970 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.862+0000 7f55f77fe700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f55f003cea0 0x7f55f003f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.862+0000 7f55f77fe700 1 -- 172.21.15.84:0/3837928445 --> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f55f8001d30 con 0x7f55f003cea0 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.863+0000 7f5605d9b700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f55f003cea0 0x7f55f003f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.863+0000 7f5605d9b700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f55f003cea0 0x7f55f003f290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f55fc003a10 tx=0x7f55fc009290 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.864+0000 7f55f77fe700 1 -- 172.21.15.84:0/3837928445 <== mgr.14118 v2:172.21.15.84:6800/3919933286 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f55f8001d30 con 0x7f55f003cea0 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.872+0000 7f560c92a700 1 -- 172.21.15.84:0/3837928445 --> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f560806a570 con 0x7f55f003cea0 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.872+0000 7f55f77fe700 1 -- 172.21.15.84:0/3837928445 <== mgr.14118 v2:172.21.15.84:6800/3919933286 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f560806a570 con 0x7f55f003cea0 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.873+0000 7f560c92a700 1 -- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f55f003cea0 msgr2=0x7f55f003f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.873+0000 7f560c92a700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f55f003cea0 0x7f55f003f290 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f55fc003a10 tx=0x7f55fc009290 comp rx=0 tx=0).stop 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.873+0000 7f560c92a700 1 -- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 msgr2=0x7f560819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.873+0000 7f560c92a700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 0x7f560819cda0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f55f8011ac0 tx=0x7f55f8011ba0 comp rx=0 tx=0).stop 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.873+0000 7f560c92a700 1 -- 172.21.15.84:0/3837928445 shutdown_connections 2024-04-26T18:16:39.475 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.873+0000 7f560c92a700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f55f003cea0 0x7f55f003f290 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:39.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.873+0000 7f560c92a700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5608109970 0x7f560819cda0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:39.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.873+0000 7f560c92a700 1 --2- 172.21.15.84:0/3837928445 >> [v2:172.21.15.84:6800/1448727861,v1:172.21.15.84:6801/1448727861] conn(0x7f55f00395b0 0x7f55f003ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:39.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.873+0000 7f560c92a700 1 -- 172.21.15.84:0/3837928445 >> 172.21.15.84:0/3837928445 conn(0x7f56081033e0 msgr2=0x7f5608103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:39.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.873+0000 7f560c92a700 1 -- 172.21.15.84:0/3837928445 shutdown_connections 2024-04-26T18:16:39.476 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:38.873+0000 7f560c92a700 1 -- 172.21.15.84:0/3837928445 wait complete. 2024-04-26T18:16:39.476 INFO:teuthology.orchestra.run.smithi084.stdout:mgr epoch 5 is available 2024-04-26T18:16:39.476 INFO:teuthology.orchestra.run.smithi084.stdout:Setting orchestrator backend to cephadm... 2024-04-26T18:16:40.079 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:39 smithi084 ceph-mon[29141]: [26/Apr/2024:18:16:38] ENGINE Bus STARTING 2024-04-26T18:16:40.079 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:39 smithi084 ceph-mon[29141]: [26/Apr/2024:18:16:38] ENGINE Serving on https://172.21.15.84:7150 2024-04-26T18:16:40.079 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:39 smithi084 ceph-mon[29141]: [26/Apr/2024:18:16:38] ENGINE Serving on http://172.21.15.84:8765 2024-04-26T18:16:40.079 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:39 smithi084 ceph-mon[29141]: [26/Apr/2024:18:16:38] ENGINE Bus STARTED 2024-04-26T18:16:40.079 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:39 smithi084 ceph-mon[29141]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T18:16:40.079 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:39 smithi084 ceph-mon[29141]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T18:16:40.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.854+0000 7f8068676700 1 Processor -- start 2024-04-26T18:16:40.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.854+0000 7f8068676700 1 -- start start 2024-04-26T18:16:40.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.855+0000 7f8068676700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806007cca0 0x7f806010ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:40.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.855+0000 7f8068676700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f806010c010 con 0x7f806007cca0 2024-04-26T18:16:40.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.855+0000 7f8066412700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806007cca0 0x7f806010ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:40.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.855+0000 7f8066412700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806007cca0 0x7f806010ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33350/0 (socket says 172.21.15.84:33350) 2024-04-26T18:16:40.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.855+0000 7f8066412700 1 -- 172.21.15.84:0/4049414790 learned_addr learned my addr 172.21.15.84:0/4049414790 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:40.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.855+0000 7f8066412700 1 -- 172.21.15.84:0/4049414790 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806010c190 con 0x7f806007cca0 2024-04-26T18:16:40.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.855+0000 7f8066412700 1 --2- 172.21.15.84:0/4049414790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806007cca0 0x7f806010ba40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f805c0098a0 tx=0x7f805c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465a30747cdfb2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:40.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.855+0000 7f8065410700 1 -- 172.21.15.84:0/4049414790 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f805c004160 con 0x7f806007cca0 2024-04-26T18:16:40.714 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.855+0000 7f8065410700 1 -- 172.21.15.84:0/4049414790 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f805c004300 con 0x7f806007cca0 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.855+0000 7f8065410700 1 -- 172.21.15.84:0/4049414790 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f805c011af0 con 0x7f806007cca0 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.856+0000 7f8068676700 1 -- 172.21.15.84:0/4049414790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806007cca0 msgr2=0x7f806010ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.856+0000 7f8068676700 1 --2- 172.21.15.84:0/4049414790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806007cca0 0x7f806010ba40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f805c0098a0 tx=0x7f805c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.856+0000 7f8068676700 1 -- 172.21.15.84:0/4049414790 shutdown_connections 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.856+0000 7f8068676700 1 --2- 172.21.15.84:0/4049414790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806007cca0 0x7f806010ba40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.856+0000 7f8068676700 1 -- 172.21.15.84:0/4049414790 >> 172.21.15.84:0/4049414790 conn(0x7f80601033c0 msgr2=0x7f80601057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.856+0000 7f8068676700 1 -- 172.21.15.84:0/4049414790 shutdown_connections 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.856+0000 7f8068676700 1 -- 172.21.15.84:0/4049414790 wait complete. 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.857+0000 7f8068676700 1 Processor -- start 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.857+0000 7f8068676700 1 -- start start 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.857+0000 7f8068676700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806019ce60 0x7f806019d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.857+0000 7f8068676700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f806010d210 con 0x7f806019ce60 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.857+0000 7f8066412700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806019ce60 0x7f806019d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:40.715 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.857+0000 7f8066412700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806019ce60 0x7f806019d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33356/0 (socket says 172.21.15.84:33356) 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.857+0000 7f8066412700 1 -- 172.21.15.84:0/1691410243 learned_addr learned my addr 172.21.15.84:0/1691410243 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.857+0000 7f8066412700 1 -- 172.21.15.84:0/1691410243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f806019d7c0 con 0x7f806019ce60 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.857+0000 7f8066412700 1 --2- 172.21.15.84:0/1691410243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806019ce60 0x7f806019d280 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f805c011ac0 tx=0x7f805c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.857+0000 7f80577fe700 1 -- 172.21.15.84:0/1691410243 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f805c003970 con 0x7f806019ce60 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.858+0000 7f8068676700 1 -- 172.21.15.84:0/1691410243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f806019da50 con 0x7f806019ce60 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.858+0000 7f80577fe700 1 -- 172.21.15.84:0/1691410243 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f805c003b10 con 0x7f806019ce60 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.858+0000 7f80577fe700 1 -- 172.21.15.84:0/1691410243 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f805c024610 con 0x7f806019ce60 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.858+0000 7f8068676700 1 -- 172.21.15.84:0/1691410243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80601a0530 con 0x7f806019ce60 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.859+0000 7f80577fe700 1 -- 172.21.15.84:0/1691410243 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f805c024890 con 0x7f806019ce60 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.859+0000 7f8068676700 1 -- 172.21.15.84:0/1691410243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8060064a80 con 0x7f806019ce60 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.859+0000 7f80577fe700 1 --2- 172.21.15.84:0/1691410243 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f804c039490 0x7f804c03b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.859+0000 7f80577fe700 1 -- 172.21.15.84:0/1691410243 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f805c04d770 con 0x7f806019ce60 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.859+0000 7f8065c11700 1 --2- 172.21.15.84:0/1691410243 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f804c039490 0x7f804c03b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.859+0000 7f8065c11700 1 --2- 172.21.15.84:0/1691410243 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f804c039490 0x7f804c03b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8050006fd0 tx=0x7f8050006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:40.716 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.864+0000 7f80577fe700 1 -- 172.21.15.84:0/1691410243 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f805c0172b0 con 0x7f806019ce60 2024-04-26T18:16:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:39.913+0000 7f80577fe700 1 -- 172.21.15.84:0/1691410243 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f805c04fd00 con 0x7f806019ce60 2024-04-26T18:16:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:40.078+0000 7f8068676700 1 -- 172.21.15.84:0/1691410243 --> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f80601046d0 con 0x7f804c039490 2024-04-26T18:16:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:40.083+0000 7f80577fe700 1 -- 172.21.15.84:0/1691410243 <== mgr.14118 v2:172.21.15.84:6800/3919933286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f80601046d0 con 0x7f804c039490 2024-04-26T18:16:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:40.086+0000 7f8068676700 1 -- 172.21.15.84:0/1691410243 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f804c039490 msgr2=0x7f804c03b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:40.086+0000 7f8068676700 1 --2- 172.21.15.84:0/1691410243 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f804c039490 0x7f804c03b950 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f8050006fd0 tx=0x7f8050006e40 comp rx=0 tx=0).stop 2024-04-26T18:16:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:40.086+0000 7f8068676700 1 -- 172.21.15.84:0/1691410243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806019ce60 msgr2=0x7f806019d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:40.086+0000 7f8068676700 1 --2- 172.21.15.84:0/1691410243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806019ce60 0x7f806019d280 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f805c011ac0 tx=0x7f805c011ba0 comp rx=0 tx=0).stop 2024-04-26T18:16:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:40.086+0000 7f8068676700 1 -- 172.21.15.84:0/1691410243 shutdown_connections 2024-04-26T18:16:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:40.086+0000 7f8068676700 1 --2- 172.21.15.84:0/1691410243 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f804c039490 0x7f804c03b950 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:40.086+0000 7f8068676700 1 --2- 172.21.15.84:0/1691410243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f806019ce60 0x7f806019d280 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:40.086+0000 7f8068676700 1 -- 172.21.15.84:0/1691410243 >> 172.21.15.84:0/1691410243 conn(0x7f80601033c0 msgr2=0x7f8060103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:40.086+0000 7f8068676700 1 -- 172.21.15.84:0/1691410243 shutdown_connections 2024-04-26T18:16:40.717 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:40.086+0000 7f8068676700 1 -- 172.21.15.84:0/1691410243 wait complete. 2024-04-26T18:16:41.042 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:40 smithi084 ceph-mon[29141]: mgrmap e8: x(active, since 2s) 2024-04-26T18:16:41.042 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:40 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:41.042 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:40 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:41.970 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:41 smithi084 ceph-mon[29141]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:41.980 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T18:16:41.980 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.152+0000 7f869d2df700 1 Processor -- start 2024-04-26T18:16:41.980 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.152+0000 7f869d2df700 1 -- start start 2024-04-26T18:16:41.980 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.153+0000 7f869d2df700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 0x7f8698108290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:41.980 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.153+0000 7f869d2df700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f86981087d0 con 0x7f8698107e70 2024-04-26T18:16:41.980 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.153+0000 7f8696ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 0x7f8698108290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:41.980 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.153+0000 7f8696ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 0x7f8698108290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33372/0 (socket says 172.21.15.84:33372) 2024-04-26T18:16:41.981 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.153+0000 7f8696ffd700 1 -- 172.21.15.84:0/3333588382 learned_addr learned my addr 172.21.15.84:0/3333588382 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:41.981 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.153+0000 7f8696ffd700 1 -- 172.21.15.84:0/3333588382 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8698108f80 con 0x7f8698107e70 2024-04-26T18:16:41.981 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.153+0000 7f8696ffd700 1 --2- 172.21.15.84:0/3333588382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 0x7f8698108290 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f86880098a0 tx=0x7f8688009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e19a28fc198c8664 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:41.981 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.153+0000 7f8695ffb700 1 -- 172.21.15.84:0/3333588382 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8688004160 con 0x7f8698107e70 2024-04-26T18:16:41.981 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.153+0000 7f8695ffb700 1 -- 172.21.15.84:0/3333588382 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8688004300 con 0x7f8698107e70 2024-04-26T18:16:41.981 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.153+0000 7f8695ffb700 1 -- 172.21.15.84:0/3333588382 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8688011af0 con 0x7f8698107e70 2024-04-26T18:16:41.981 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.154+0000 7f869d2df700 1 -- 172.21.15.84:0/3333588382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 msgr2=0x7f8698108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:41.981 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.154+0000 7f869d2df700 1 --2- 172.21.15.84:0/3333588382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 0x7f8698108290 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f86880098a0 tx=0x7f8688009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:41.981 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.154+0000 7f869d2df700 1 -- 172.21.15.84:0/3333588382 shutdown_connections 2024-04-26T18:16:41.981 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.154+0000 7f869d2df700 1 --2- 172.21.15.84:0/3333588382 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 0x7f8698108290 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:41.981 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.154+0000 7f869d2df700 1 -- 172.21.15.84:0/3333588382 >> 172.21.15.84:0/3333588382 conn(0x7f8698103470 msgr2=0x7f8698105890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:41.981 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.154+0000 7f869d2df700 1 -- 172.21.15.84:0/3333588382 shutdown_connections 2024-04-26T18:16:41.982 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.154+0000 7f869d2df700 1 -- 172.21.15.84:0/3333588382 wait complete. 2024-04-26T18:16:41.982 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.155+0000 7f869d2df700 1 Processor -- start 2024-04-26T18:16:41.982 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.155+0000 7f869d2df700 1 -- start start 2024-04-26T18:16:41.982 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.155+0000 7f869d2df700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 0x7f869819be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:41.982 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.155+0000 7f869d2df700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f869810c3c0 con 0x7f8698107e70 2024-04-26T18:16:41.982 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.155+0000 7f8696ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 0x7f869819be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:41.982 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.155+0000 7f8696ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 0x7f869819be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33384/0 (socket says 172.21.15.84:33384) 2024-04-26T18:16:41.982 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.155+0000 7f8696ffd700 1 -- 172.21.15.84:0/911194967 learned_addr learned my addr 172.21.15.84:0/911194967 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:41.982 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.156+0000 7f8696ffd700 1 -- 172.21.15.84:0/911194967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f869819c360 con 0x7f8698107e70 2024-04-26T18:16:41.982 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.156+0000 7f8696ffd700 1 --2- 172.21.15.84:0/911194967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 0x7f869819be20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8688009fb0 tx=0x7f8688011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:41.983 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.156+0000 7f868ffff700 1 -- 172.21.15.84:0/911194967 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f86880036a0 con 0x7f8698107e70 2024-04-26T18:16:41.983 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.156+0000 7f869d2df700 1 -- 172.21.15.84:0/911194967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f869819c5f0 con 0x7f8698107e70 2024-04-26T18:16:41.983 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.156+0000 7f868ffff700 1 -- 172.21.15.84:0/911194967 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f868801a430 con 0x7f8698107e70 2024-04-26T18:16:41.983 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.156+0000 7f868ffff700 1 -- 172.21.15.84:0/911194967 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8688019550 con 0x7f8698107e70 2024-04-26T18:16:41.983 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.156+0000 7f869d2df700 1 -- 172.21.15.84:0/911194967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f869819f2e0 con 0x7f8698107e70 2024-04-26T18:16:41.983 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.157+0000 7f868ffff700 1 -- 172.21.15.84:0/911194967 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f8688019850 con 0x7f8698107e70 2024-04-26T18:16:41.983 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.157+0000 7f869d2df700 1 -- 172.21.15.84:0/911194967 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f869806a570 con 0x7f8698107e70 2024-04-26T18:16:41.983 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.157+0000 7f868ffff700 1 --2- 172.21.15.84:0/911194967 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f8684039560 0x7f868403ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:41.983 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.157+0000 7f868ffff700 1 -- 172.21.15.84:0/911194967 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f868804da80 con 0x7f8698107e70 2024-04-26T18:16:41.983 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.157+0000 7f86967fc700 1 --2- 172.21.15.84:0/911194967 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f8684039560 0x7f868403ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:41.984 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.157+0000 7f86967fc700 1 --2- 172.21.15.84:0/911194967 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f8684039560 0x7f868403ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8680006fd0 tx=0x7f8680006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:41.984 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.162+0000 7f868ffff700 1 -- 172.21.15.84:0/911194967 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f868800b3e0 con 0x7f8698107e70 2024-04-26T18:16:41.984 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.369+0000 7f869d2df700 1 -- 172.21.15.84:0/911194967 --> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f86981046e0 con 0x7f8684039560 2024-04-26T18:16:41.984 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.370+0000 7f868ffff700 1 -- 172.21.15.84:0/911194967 <== mgr.14118 v2:172.21.15.84:6800/3919933286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f86981046e0 con 0x7f8684039560 2024-04-26T18:16:41.984 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.372+0000 7f869d2df700 1 -- 172.21.15.84:0/911194967 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f8684039560 msgr2=0x7f868403ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:41.984 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.372+0000 7f869d2df700 1 --2- 172.21.15.84:0/911194967 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f8684039560 0x7f868403ba20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8680006fd0 tx=0x7f8680006e40 comp rx=0 tx=0).stop 2024-04-26T18:16:41.984 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.372+0000 7f869d2df700 1 -- 172.21.15.84:0/911194967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 msgr2=0x7f869819be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:41.984 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.372+0000 7f869d2df700 1 --2- 172.21.15.84:0/911194967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 0x7f869819be20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8688009fb0 tx=0x7f8688011940 comp rx=0 tx=0).stop 2024-04-26T18:16:41.984 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.372+0000 7f869d2df700 1 -- 172.21.15.84:0/911194967 shutdown_connections 2024-04-26T18:16:41.984 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.372+0000 7f869d2df700 1 --2- 172.21.15.84:0/911194967 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f8684039560 0x7f868403ba20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:41.984 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.372+0000 7f869d2df700 1 --2- 172.21.15.84:0/911194967 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8698107e70 0x7f869819be20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:41.985 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.372+0000 7f869d2df700 1 -- 172.21.15.84:0/911194967 >> 172.21.15.84:0/911194967 conn(0x7f8698103470 msgr2=0x7f8698103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:41.985 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.372+0000 7f869d2df700 1 -- 172.21.15.84:0/911194967 shutdown_connections 2024-04-26T18:16:41.985 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:41.372+0000 7f869d2df700 1 -- 172.21.15.84:0/911194967 wait complete. 2024-04-26T18:16:41.985 INFO:teuthology.orchestra.run.smithi084.stdout:Generating ssh key... 2024-04-26T18:16:42.917 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: Generating public/private rsa key pair. 2024-04-26T18:16:42.917 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: Your identification has been saved in /tmp/tmp9ilgbxk2/key. 2024-04-26T18:16:42.917 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: Your public key has been saved in /tmp/tmp9ilgbxk2/key.pub. 2024-04-26T18:16:42.917 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: The key fingerprint is: 2024-04-26T18:16:42.918 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: SHA256:fvZi0GSCoChsG2YOs2r9XIdPUJ4K87mf9vp22PC2KgA ceph-df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:16:42.918 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: The key's randomart image is: 2024-04-26T18:16:42.918 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: +---[RSA 3072]----+ 2024-04-26T18:16:42.918 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: | | 2024-04-26T18:16:42.918 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: | . | 2024-04-26T18:16:42.918 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: |.. . . . . | 2024-04-26T18:16:42.918 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: |=*. E. + + | 2024-04-26T18:16:42.918 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: |Boo o..SB | 2024-04-26T18:16:42.918 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: |.o +o* .. | 2024-04-26T18:16:42.918 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: |. . *o+o = | 2024-04-26T18:16:42.918 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: |.. .. . ==++ = | 2024-04-26T18:16:42.918 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: |. .o .o**==.. | 2024-04-26T18:16:42.918 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: +----[SHA256]-----+ 2024-04-26T18:16:43.343 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-mon[29141]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:43.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:43.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:42 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:43.351 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.423+0000 7f94f04c4700 1 Processor -- start 2024-04-26T18:16:43.351 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.423+0000 7f94f04c4700 1 -- start start 2024-04-26T18:16:43.351 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.423+0000 7f94f04c4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 0x7f94e81081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:43.352 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.423+0000 7f94f04c4700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f94e81087b0 con 0x7f94e8107dc0 2024-04-26T18:16:43.352 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.423+0000 7f94ee260700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 0x7f94e81081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:43.352 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.423+0000 7f94ee260700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 0x7f94e81081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33386/0 (socket says 172.21.15.84:33386) 2024-04-26T18:16:43.352 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.423+0000 7f94ee260700 1 -- 172.21.15.84:0/350040111 learned_addr learned my addr 172.21.15.84:0/350040111 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:43.352 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.424+0000 7f94ee260700 1 -- 172.21.15.84:0/350040111 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e8108f70 con 0x7f94e8107dc0 2024-04-26T18:16:43.352 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.424+0000 7f94ee260700 1 --2- 172.21.15.84:0/350040111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 0x7f94e81081e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f94e40098a0 tx=0x7f94e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f0c54eccaa53 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:43.352 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.424+0000 7f94ed25e700 1 -- 172.21.15.84:0/350040111 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94e4004160 con 0x7f94e8107dc0 2024-04-26T18:16:43.352 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.424+0000 7f94ed25e700 1 -- 172.21.15.84:0/350040111 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f94e4004300 con 0x7f94e8107dc0 2024-04-26T18:16:43.352 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.424+0000 7f94ed25e700 1 -- 172.21.15.84:0/350040111 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94e4011af0 con 0x7f94e8107dc0 2024-04-26T18:16:43.352 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.425+0000 7f94f04c4700 1 -- 172.21.15.84:0/350040111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 msgr2=0x7f94e81081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:43.352 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.425+0000 7f94f04c4700 1 --2- 172.21.15.84:0/350040111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 0x7f94e81081e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f94e40098a0 tx=0x7f94e4009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:43.352 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.425+0000 7f94f04c4700 1 -- 172.21.15.84:0/350040111 shutdown_connections 2024-04-26T18:16:43.352 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.425+0000 7f94f04c4700 1 --2- 172.21.15.84:0/350040111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 0x7f94e81081e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.425+0000 7f94f04c4700 1 -- 172.21.15.84:0/350040111 >> 172.21.15.84:0/350040111 conn(0x7f94e8103380 msgr2=0x7f94e81057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.425+0000 7f94f04c4700 1 -- 172.21.15.84:0/350040111 shutdown_connections 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.425+0000 7f94f04c4700 1 -- 172.21.15.84:0/350040111 wait complete. 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.425+0000 7f94f04c4700 1 Processor -- start 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.425+0000 7f94f04c4700 1 -- start start 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.426+0000 7f94f04c4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 0x7f94e819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.426+0000 7f94f04c4700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f94e810c320 con 0x7f94e8107dc0 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.426+0000 7f94ee260700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 0x7f94e819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.426+0000 7f94ee260700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 0x7f94e819bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33398/0 (socket says 172.21.15.84:33398) 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.426+0000 7f94ee260700 1 -- 172.21.15.84:0/590269562 learned_addr learned my addr 172.21.15.84:0/590269562 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.426+0000 7f94ee260700 1 -- 172.21.15.84:0/590269562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e819c260 con 0x7f94e8107dc0 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.426+0000 7f94ee260700 1 --2- 172.21.15.84:0/590269562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 0x7f94e819bd20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f94e4000c00 tx=0x7f94e4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.426+0000 7f94df7fe700 1 -- 172.21.15.84:0/590269562 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94e4003b20 con 0x7f94e8107dc0 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.426+0000 7f94df7fe700 1 -- 172.21.15.84:0/590269562 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f94e401a8c0 con 0x7f94e8107dc0 2024-04-26T18:16:43.353 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.426+0000 7f94df7fe700 1 -- 172.21.15.84:0/590269562 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94e4024940 con 0x7f94e8107dc0 2024-04-26T18:16:43.354 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.426+0000 7f94f04c4700 1 -- 172.21.15.84:0/590269562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e819c4f0 con 0x7f94e8107dc0 2024-04-26T18:16:43.354 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.426+0000 7f94f04c4700 1 -- 172.21.15.84:0/590269562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e819f1d0 con 0x7f94e8107dc0 2024-04-26T18:16:43.354 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.427+0000 7f94df7fe700 1 -- 172.21.15.84:0/590269562 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f94e4022020 con 0x7f94e8107dc0 2024-04-26T18:16:43.354 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.427+0000 7f94f04c4700 1 -- 172.21.15.84:0/590269562 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94e8064a80 con 0x7f94e8107dc0 2024-04-26T18:16:43.354 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.427+0000 7f94df7fe700 1 --2- 172.21.15.84:0/590269562 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f94d4039600 0x7f94d403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:43.354 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.427+0000 7f94df7fe700 1 -- 172.21.15.84:0/590269562 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f94e404d7e0 con 0x7f94e8107dc0 2024-04-26T18:16:43.354 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.428+0000 7f94eda5f700 1 --2- 172.21.15.84:0/590269562 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f94d4039600 0x7f94d403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:43.354 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.428+0000 7f94eda5f700 1 --2- 172.21.15.84:0/590269562 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f94d4039600 0x7f94d403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f94d8006fd0 tx=0x7f94d8006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:43.354 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.433+0000 7f94df7fe700 1 -- 172.21.15.84:0/590269562 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f94e400be50 con 0x7f94e8107dc0 2024-04-26T18:16:43.354 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.637+0000 7f94f04c4700 1 -- 172.21.15.84:0/590269562 --> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f94e81045c0 con 0x7f94d4039600 2024-04-26T18:16:43.354 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.770+0000 7f94df7fe700 1 -- 172.21.15.84:0/590269562 <== mgr.14118 v2:172.21.15.84:6800/3919933286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f94e81045c0 con 0x7f94d4039600 2024-04-26T18:16:43.354 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.772+0000 7f94f04c4700 1 -- 172.21.15.84:0/590269562 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f94d4039600 msgr2=0x7f94d403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:43.354 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.772+0000 7f94f04c4700 1 --2- 172.21.15.84:0/590269562 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f94d4039600 0x7f94d403bac0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f94d8006fd0 tx=0x7f94d8006e40 comp rx=0 tx=0).stop 2024-04-26T18:16:43.355 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.772+0000 7f94f04c4700 1 -- 172.21.15.84:0/590269562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 msgr2=0x7f94e819bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:43.355 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.773+0000 7f94f04c4700 1 --2- 172.21.15.84:0/590269562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 0x7f94e819bd20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f94e4000c00 tx=0x7f94e4011b70 comp rx=0 tx=0).stop 2024-04-26T18:16:43.355 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.773+0000 7f94f04c4700 1 -- 172.21.15.84:0/590269562 shutdown_connections 2024-04-26T18:16:43.355 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.773+0000 7f94f04c4700 1 --2- 172.21.15.84:0/590269562 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f94d4039600 0x7f94d403bac0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:43.355 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.773+0000 7f94f04c4700 1 --2- 172.21.15.84:0/590269562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f94e8107dc0 0x7f94e819bd20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:43.355 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.773+0000 7f94f04c4700 1 -- 172.21.15.84:0/590269562 >> 172.21.15.84:0/590269562 conn(0x7f94e8103380 msgr2=0x7f94e8103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:43.355 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.773+0000 7f94f04c4700 1 -- 172.21.15.84:0/590269562 shutdown_connections 2024-04-26T18:16:43.355 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:42.773+0000 7f94f04c4700 1 -- 172.21.15.84:0/590269562 wait complete. 2024-04-26T18:16:43.995 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:43 smithi084 ceph-mon[29141]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:43.995 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:43 smithi084 ceph-mon[29141]: Generating ssh key... 2024-04-26T18:16:44.567 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3TgI8lfZoWvnZDqhzWIFbexROjohsqW+yAPxfeER5LLpaC7M+xPqszQcVU6HRwSlRZz3nfSYmf7Y5IwZSX05Klgc2pDFjsX4aDsEURGr9lsu+zXN+mJDZxN+UCuKOJFDB3A3y2YNH7mL/w0t89ZneruoraDfNI5O2cbMZpkJoETXZzuvOPWTQ1wrhu+s/2mclLdBLq5E3FIQNi2qdk7Bu1pTGWI+u6z5aNJyHrlV3XJ+qZ5jHs68IrskUzxxR7qEfOXvBZ+QQyBEVutLvewz300+Y9+HFGUGVndVTAE3iZgCLywFmnec2jnZsUGlrvSLfvWg6ejjQPVU9no+0l5Iex8/Zz8zJXPzdHY7sacV0KQlcVaf0xeeKsGwkPphLvXPkg+azSTFTR8SqlefmMnmOvoTqK5f3rhdbkkGT8DVLLOFv73PyiJhre0bWGdPXGEa1MBRSesAtzJy1XPo9kg08kl/5vJfx1J1rXluPH3TfdVpKGOWmsC7tLHJsTUd6k8U= ceph-df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:16:44.567 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.783+0000 7f417a322700 1 Processor -- start 2024-04-26T18:16:44.568 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.783+0000 7f417a322700 1 -- start start 2024-04-26T18:16:44.568 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.783+0000 7f417a322700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 0x7f41741081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:44.568 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.784+0000 7f417a322700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f41741087b0 con 0x7f4174107dc0 2024-04-26T18:16:44.568 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.784+0000 7f4173fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 0x7f41741081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:44.568 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.784+0000 7f4173fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 0x7f41741081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33402/0 (socket says 172.21.15.84:33402) 2024-04-26T18:16:44.568 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.784+0000 7f4173fff700 1 -- 172.21.15.84:0/3228484462 learned_addr learned my addr 172.21.15.84:0/3228484462 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:44.568 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.784+0000 7f4173fff700 1 -- 172.21.15.84:0/3228484462 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4174108f70 con 0x7f4174107dc0 2024-04-26T18:16:44.568 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.784+0000 7f4173fff700 1 --2- 172.21.15.84:0/3228484462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 0x7f41741081e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f41680098a0 tx=0x7f4168009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91dc989e8bdf9a7a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:44.568 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.784+0000 7f4172ffd700 1 -- 172.21.15.84:0/3228484462 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4168004160 con 0x7f4174107dc0 2024-04-26T18:16:44.568 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.784+0000 7f4172ffd700 1 -- 172.21.15.84:0/3228484462 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4168004300 con 0x7f4174107dc0 2024-04-26T18:16:44.568 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.784+0000 7f4172ffd700 1 -- 172.21.15.84:0/3228484462 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4168011af0 con 0x7f4174107dc0 2024-04-26T18:16:44.568 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.785+0000 7f417a322700 1 -- 172.21.15.84:0/3228484462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 msgr2=0x7f41741081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:44.568 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.785+0000 7f417a322700 1 --2- 172.21.15.84:0/3228484462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 0x7f41741081e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f41680098a0 tx=0x7f4168009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.785+0000 7f417a322700 1 -- 172.21.15.84:0/3228484462 shutdown_connections 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.785+0000 7f417a322700 1 --2- 172.21.15.84:0/3228484462 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 0x7f41741081e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.785+0000 7f417a322700 1 -- 172.21.15.84:0/3228484462 >> 172.21.15.84:0/3228484462 conn(0x7f4174103380 msgr2=0x7f41741057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.785+0000 7f417a322700 1 -- 172.21.15.84:0/3228484462 shutdown_connections 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.785+0000 7f417a322700 1 -- 172.21.15.84:0/3228484462 wait complete. 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.785+0000 7f417a322700 1 Processor -- start 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.786+0000 7f417a322700 1 -- start start 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.786+0000 7f417a322700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 0x7f417407d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.786+0000 7f417a322700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f417410c320 con 0x7f4174107dc0 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.786+0000 7f4173fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 0x7f417407d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.786+0000 7f4173fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 0x7f417407d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33404/0 (socket says 172.21.15.84:33404) 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.786+0000 7f4173fff700 1 -- 172.21.15.84:0/3864418226 learned_addr learned my addr 172.21.15.84:0/3864418226 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.786+0000 7f4173fff700 1 -- 172.21.15.84:0/3864418226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f417419fb50 con 0x7f4174107dc0 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.786+0000 7f4173fff700 1 --2- 172.21.15.84:0/3864418226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 0x7f417407d2b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4168003f40 tx=0x7f4168011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:44.569 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.787+0000 7f41717fa700 1 -- 172.21.15.84:0/3864418226 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f41680038b0 con 0x7f4174107dc0 2024-04-26T18:16:44.570 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.787+0000 7f41717fa700 1 -- 172.21.15.84:0/3864418226 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4168003a50 con 0x7f4174107dc0 2024-04-26T18:16:44.570 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.787+0000 7f417a322700 1 -- 172.21.15.84:0/3864418226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f417419fd20 con 0x7f4174107dc0 2024-04-26T18:16:44.570 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.787+0000 7f41717fa700 1 -- 172.21.15.84:0/3864418226 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f416801a5c0 con 0x7f4174107dc0 2024-04-26T18:16:44.570 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.787+0000 7f417a322700 1 -- 172.21.15.84:0/3864418226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41741a0140 con 0x7f4174107dc0 2024-04-26T18:16:44.570 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.787+0000 7f41717fa700 1 -- 172.21.15.84:0/3864418226 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f416801a8c0 con 0x7f4174107dc0 2024-04-26T18:16:44.570 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.788+0000 7f41717fa700 1 --2- 172.21.15.84:0/3864418226 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f415c039560 0x7f415c03ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:44.571 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.788+0000 7f417a322700 1 -- 172.21.15.84:0/3864418226 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4174064a80 con 0x7f4174107dc0 2024-04-26T18:16:44.571 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.788+0000 7f41717fa700 1 -- 172.21.15.84:0/3864418226 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f416804db10 con 0x7f4174107dc0 2024-04-26T18:16:44.571 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.788+0000 7f41737fe700 1 --2- 172.21.15.84:0/3864418226 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f415c039560 0x7f415c03ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:44.571 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.788+0000 7f41737fe700 1 --2- 172.21.15.84:0/3864418226 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f415c039560 0x7f415c03ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4164006fd0 tx=0x7f4164006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:44.571 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.793+0000 7f41717fa700 1 -- 172.21.15.84:0/3864418226 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4168019c70 con 0x7f4174107dc0 2024-04-26T18:16:44.571 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.995+0000 7f417a322700 1 -- 172.21.15.84:0/3864418226 --> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f41741045c0 con 0x7f415c039560 2024-04-26T18:16:44.571 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.996+0000 7f41717fa700 1 -- 172.21.15.84:0/3864418226 <== mgr.14118 v2:172.21.15.84:6800/3919933286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f41741045c0 con 0x7f415c039560 2024-04-26T18:16:44.571 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.998+0000 7f417a322700 1 -- 172.21.15.84:0/3864418226 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f415c039560 msgr2=0x7f415c03ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:44.571 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.998+0000 7f417a322700 1 --2- 172.21.15.84:0/3864418226 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f415c039560 0x7f415c03ba20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4164006fd0 tx=0x7f4164006e40 comp rx=0 tx=0).stop 2024-04-26T18:16:44.572 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.998+0000 7f417a322700 1 -- 172.21.15.84:0/3864418226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 msgr2=0x7f417407d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:44.572 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.998+0000 7f417a322700 1 --2- 172.21.15.84:0/3864418226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 0x7f417407d2b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4168003f40 tx=0x7f4168011890 comp rx=0 tx=0).stop 2024-04-26T18:16:44.572 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.998+0000 7f417a322700 1 -- 172.21.15.84:0/3864418226 shutdown_connections 2024-04-26T18:16:44.572 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.998+0000 7f417a322700 1 --2- 172.21.15.84:0/3864418226 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f415c039560 0x7f415c03ba20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:44.572 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.998+0000 7f417a322700 1 --2- 172.21.15.84:0/3864418226 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4174107dc0 0x7f417407d2b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:44.572 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.998+0000 7f417a322700 1 -- 172.21.15.84:0/3864418226 >> 172.21.15.84:0/3864418226 conn(0x7f4174103380 msgr2=0x7f4174103eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:44.572 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.999+0000 7f417a322700 1 -- 172.21.15.84:0/3864418226 shutdown_connections 2024-04-26T18:16:44.572 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:43.999+0000 7f417a322700 1 -- 172.21.15.84:0/3864418226 wait complete. 2024-04-26T18:16:44.572 INFO:teuthology.orchestra.run.smithi084.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T18:16:44.572 INFO:teuthology.orchestra.run.smithi084.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T18:16:44.572 INFO:teuthology.orchestra.run.smithi084.stdout:Adding host smithi084... 2024-04-26T18:16:44.960 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:44 smithi084 ceph-mon[29141]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:46.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:46 smithi084 ceph-mon[29141]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi084", "addr": "172.21.15.84", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:46.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:46 smithi084 ceph-mon[29141]: Deploying cephadm binary to smithi084 2024-04-26T18:16:49.898 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout Added host 'smithi084' with addr '172.21.15.84' 2024-04-26T18:16:49.898 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.996+0000 7f2ba76bb700 1 Processor -- start 2024-04-26T18:16:49.899 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.996+0000 7f2ba76bb700 1 -- start start 2024-04-26T18:16:49.899 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.997+0000 7f2ba76bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 0x7f2ba01097d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.899 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.997+0000 7f2ba76bb700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba0109da0 con 0x7f2ba01093b0 2024-04-26T18:16:49.899 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.997+0000 7f2ba5457700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 0x7f2ba01097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.899 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.997+0000 7f2ba5457700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 0x7f2ba01097d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33410/0 (socket says 172.21.15.84:33410) 2024-04-26T18:16:49.899 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.997+0000 7f2ba5457700 1 -- 172.21.15.84:0/487503467 learned_addr learned my addr 172.21.15.84:0/487503467 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:49.900 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.997+0000 7f2ba5457700 1 -- 172.21.15.84:0/487503467 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba0109f20 con 0x7f2ba01093b0 2024-04-26T18:16:49.900 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.997+0000 7f2ba5457700 1 --2- 172.21.15.84:0/487503467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 0x7f2ba01097d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2b900098a0 tx=0x7f2b90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b07b204c084e6a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:49.900 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.997+0000 7f2b97fff700 1 -- 172.21.15.84:0/487503467 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b90004160 con 0x7f2ba01093b0 2024-04-26T18:16:49.900 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.997+0000 7f2b97fff700 1 -- 172.21.15.84:0/487503467 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2b90004300 con 0x7f2ba01093b0 2024-04-26T18:16:49.900 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.997+0000 7f2b97fff700 1 -- 172.21.15.84:0/487503467 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b90011af0 con 0x7f2ba01093b0 2024-04-26T18:16:49.901 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.998+0000 7f2ba76bb700 1 -- 172.21.15.84:0/487503467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 msgr2=0x7f2ba01097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:49.901 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.998+0000 7f2ba76bb700 1 --2- 172.21.15.84:0/487503467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 0x7f2ba01097d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2b900098a0 tx=0x7f2b90009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.901 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.998+0000 7f2ba76bb700 1 -- 172.21.15.84:0/487503467 shutdown_connections 2024-04-26T18:16:49.901 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.998+0000 7f2ba76bb700 1 --2- 172.21.15.84:0/487503467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 0x7f2ba01097d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.901 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.998+0000 7f2ba76bb700 1 -- 172.21.15.84:0/487503467 >> 172.21.15.84:0/487503467 conn(0x7f2ba0102ac0 msgr2=0x7f2ba0104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:49.901 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.998+0000 7f2ba76bb700 1 -- 172.21.15.84:0/487503467 shutdown_connections 2024-04-26T18:16:49.902 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.998+0000 7f2ba76bb700 1 -- 172.21.15.84:0/487503467 wait complete. 2024-04-26T18:16:49.902 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.999+0000 7f2ba76bb700 1 Processor -- start 2024-04-26T18:16:49.902 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.999+0000 7f2ba76bb700 1 -- start start 2024-04-26T18:16:49.902 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.999+0000 7f2ba76bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 0x7f2ba019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.902 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.999+0000 7f2ba76bb700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2ba0107c50 con 0x7f2ba01093b0 2024-04-26T18:16:49.903 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.999+0000 7f2ba5457700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 0x7f2ba019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.903 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.999+0000 7f2ba5457700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 0x7f2ba019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33414/0 (socket says 172.21.15.84:33414) 2024-04-26T18:16:49.903 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.999+0000 7f2ba5457700 1 -- 172.21.15.84:0/3597585385 learned_addr learned my addr 172.21.15.84:0/3597585385 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:49.903 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.999+0000 7f2ba5457700 1 -- 172.21.15.84:0/3597585385 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ba019f4e0 con 0x7f2ba01093b0 2024-04-26T18:16:49.903 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.999+0000 7f2ba5457700 1 --2- 172.21.15.84:0/3597585385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 0x7f2ba019efa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2b90000c00 tx=0x7f2b90011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:49.904 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.999+0000 7f2b967fc700 1 -- 172.21.15.84:0/3597585385 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b90003b20 con 0x7f2ba01093b0 2024-04-26T18:16:49.904 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.999+0000 7f2b967fc700 1 -- 172.21.15.84:0/3597585385 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2b9001a8c0 con 0x7f2ba01093b0 2024-04-26T18:16:49.904 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.999+0000 7f2ba76bb700 1 -- 172.21.15.84:0/3597585385 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ba019f7d0 con 0x7f2ba01093b0 2024-04-26T18:16:49.904 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:44.999+0000 7f2b967fc700 1 -- 172.21.15.84:0/3597585385 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b90024940 con 0x7f2ba01093b0 2024-04-26T18:16:49.904 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:45.000+0000 7f2ba76bb700 1 -- 172.21.15.84:0/3597585385 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ba01a2460 con 0x7f2ba01093b0 2024-04-26T18:16:49.904 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:45.000+0000 7f2b967fc700 1 -- 172.21.15.84:0/3597585385 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2b90022020 con 0x7f2ba01093b0 2024-04-26T18:16:49.904 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:45.000+0000 7f2b967fc700 1 --2- 172.21.15.84:0/3597585385 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f2b8c0395b0 0x7f2b8c03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:49.905 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:45.001+0000 7f2b967fc700 1 -- 172.21.15.84:0/3597585385 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2b9004d910 con 0x7f2ba01093b0 2024-04-26T18:16:49.905 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:45.001+0000 7f2ba4c56700 1 --2- 172.21.15.84:0/3597585385 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f2b8c0395b0 0x7f2b8c03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:49.905 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:45.001+0000 7f2ba76bb700 1 -- 172.21.15.84:0/3597585385 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ba0064a80 con 0x7f2ba01093b0 2024-04-26T18:16:49.905 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:45.001+0000 7f2ba4c56700 1 --2- 172.21.15.84:0/3597585385 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f2b8c0395b0 0x7f2b8c03ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c006fd0 tx=0x7f2b9c006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:49.905 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:45.006+0000 7f2b967fc700 1 -- 172.21.15.84:0/3597585385 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2b90029210 con 0x7f2ba01093b0 2024-04-26T18:16:49.905 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:45.208+0000 7f2ba76bb700 1 -- 172.21.15.84:0/3597585385 --> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi084", "addr": "172.21.15.84", "target": ["mon-mgr", ""]}) -- 0x7f2ba00008d0 con 0x7f2b8c0395b0 2024-04-26T18:16:49.906 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:49.292+0000 7f2b967fc700 1 -- 172.21.15.84:0/3597585385 <== mgr.14118 v2:172.21.15.84:6800/3919933286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f2ba00008d0 con 0x7f2b8c0395b0 2024-04-26T18:16:49.906 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:49.295+0000 7f2ba76bb700 1 -- 172.21.15.84:0/3597585385 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f2b8c0395b0 msgr2=0x7f2b8c03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:49.906 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:49.295+0000 7f2ba76bb700 1 --2- 172.21.15.84:0/3597585385 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f2b8c0395b0 0x7f2b8c03ba70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f2b9c006fd0 tx=0x7f2b9c006e40 comp rx=0 tx=0).stop 2024-04-26T18:16:49.906 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:49.295+0000 7f2ba76bb700 1 -- 172.21.15.84:0/3597585385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 msgr2=0x7f2ba019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:49.906 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:49.295+0000 7f2ba76bb700 1 --2- 172.21.15.84:0/3597585385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 0x7f2ba019efa0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2b90000c00 tx=0x7f2b90011b70 comp rx=0 tx=0).stop 2024-04-26T18:16:49.907 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:49.295+0000 7f2ba76bb700 1 -- 172.21.15.84:0/3597585385 shutdown_connections 2024-04-26T18:16:49.907 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:49.295+0000 7f2ba76bb700 1 --2- 172.21.15.84:0/3597585385 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f2b8c0395b0 0x7f2b8c03ba70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.907 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:49.295+0000 7f2ba76bb700 1 --2- 172.21.15.84:0/3597585385 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ba01093b0 0x7f2ba019efa0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:49.907 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:49.295+0000 7f2ba76bb700 1 -- 172.21.15.84:0/3597585385 >> 172.21.15.84:0/3597585385 conn(0x7f2ba0102ac0 msgr2=0x7f2ba0195be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:49.907 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:49.295+0000 7f2ba76bb700 1 -- 172.21.15.84:0/3597585385 shutdown_connections 2024-04-26T18:16:49.907 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:49.295+0000 7f2ba76bb700 1 -- 172.21.15.84:0/3597585385 wait complete. 2024-04-26T18:16:49.908 INFO:teuthology.orchestra.run.smithi084.stdout:Deploying unmanaged mon service... 2024-04-26T18:16:50.527 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:50 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:50.527 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:50 smithi084 ceph-mon[29141]: Added host smithi084 2024-04-26T18:16:50.527 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:50 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:16:51.173 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T18:16:51.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.303+0000 7fcbbc035700 1 Processor -- start 2024-04-26T18:16:51.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.303+0000 7fcbbc035700 1 -- start start 2024-04-26T18:16:51.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.303+0000 7fcbbc035700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 0x7fcbb407db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:51.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.303+0000 7fcbbc035700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb407e130 con 0x7fcbb407f6b0 2024-04-26T18:16:51.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.304+0000 7fcbb9dd1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 0x7fcbb407db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:51.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.304+0000 7fcbb9dd1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 0x7fcbb407db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48202/0 (socket says 172.21.15.84:48202) 2024-04-26T18:16:51.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.304+0000 7fcbb9dd1700 1 -- 172.21.15.84:0/2078463585 learned_addr learned my addr 172.21.15.84:0/2078463585 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:51.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.304+0000 7fcbb9dd1700 1 -- 172.21.15.84:0/2078463585 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb407e2b0 con 0x7fcbb407f6b0 2024-04-26T18:16:51.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.304+0000 7fcbb9dd1700 1 --2- 172.21.15.84:0/2078463585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 0x7fcbb407db60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00098a0 tx=0x7fcbb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5cfcd0b4416ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:51.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.304+0000 7fcbb8dcf700 1 -- 172.21.15.84:0/2078463585 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbb0004160 con 0x7fcbb407f6b0 2024-04-26T18:16:51.174 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.304+0000 7fcbb8dcf700 1 -- 172.21.15.84:0/2078463585 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcbb0004300 con 0x7fcbb407f6b0 2024-04-26T18:16:51.175 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.304+0000 7fcbb8dcf700 1 -- 172.21.15.84:0/2078463585 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbb0011af0 con 0x7fcbb407f6b0 2024-04-26T18:16:51.175 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.305+0000 7fcbbc035700 1 -- 172.21.15.84:0/2078463585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 msgr2=0x7fcbb407db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:51.175 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.305+0000 7fcbbc035700 1 --2- 172.21.15.84:0/2078463585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 0x7fcbb407db60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcbb00098a0 tx=0x7fcbb0009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:51.175 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.305+0000 7fcbbc035700 1 -- 172.21.15.84:0/2078463585 shutdown_connections 2024-04-26T18:16:51.175 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.305+0000 7fcbbc035700 1 --2- 172.21.15.84:0/2078463585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 0x7fcbb407db60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:51.175 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.305+0000 7fcbbc035700 1 -- 172.21.15.84:0/2078463585 >> 172.21.15.84:0/2078463585 conn(0x7fcbb4104470 msgr2=0x7fcbb41068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:51.175 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.305+0000 7fcbbc035700 1 -- 172.21.15.84:0/2078463585 shutdown_connections 2024-04-26T18:16:51.177 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.305+0000 7fcbbc035700 1 -- 172.21.15.84:0/2078463585 wait complete. 2024-04-26T18:16:51.177 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.305+0000 7fcbbc035700 1 Processor -- start 2024-04-26T18:16:51.177 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.306+0000 7fcbbc035700 1 -- start start 2024-04-26T18:16:51.178 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.306+0000 7fcbbc035700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 0x7fcbb419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:51.178 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.306+0000 7fcbbc035700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcbb410f420 con 0x7fcbb407f6b0 2024-04-26T18:16:51.178 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.306+0000 7fcbb9dd1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 0x7fcbb419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:51.178 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.306+0000 7fcbb9dd1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 0x7fcbb419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48218/0 (socket says 172.21.15.84:48218) 2024-04-26T18:16:51.178 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.306+0000 7fcbb9dd1700 1 -- 172.21.15.84:0/507008701 learned_addr learned my addr 172.21.15.84:0/507008701 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:51.178 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.306+0000 7fcbb9dd1700 1 -- 172.21.15.84:0/507008701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbb419f540 con 0x7fcbb407f6b0 2024-04-26T18:16:51.178 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.306+0000 7fcbb9dd1700 1 --2- 172.21.15.84:0/507008701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 0x7fcbb419f000 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0003f40 tx=0x7fcbb0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:51.178 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.306+0000 7fcbaaffd700 1 -- 172.21.15.84:0/507008701 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbb00038b0 con 0x7fcbb407f6b0 2024-04-26T18:16:51.178 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.306+0000 7fcbaaffd700 1 -- 172.21.15.84:0/507008701 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fcbb0003a50 con 0x7fcbb407f6b0 2024-04-26T18:16:51.178 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.306+0000 7fcbbc035700 1 -- 172.21.15.84:0/507008701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbb419f7d0 con 0x7fcbb407f6b0 2024-04-26T18:16:51.179 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.306+0000 7fcbaaffd700 1 -- 172.21.15.84:0/507008701 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcbb001a5c0 con 0x7fcbb407f6b0 2024-04-26T18:16:51.179 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.306+0000 7fcbbc035700 1 -- 172.21.15.84:0/507008701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbb41a24c0 con 0x7fcbb407f6b0 2024-04-26T18:16:51.179 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.307+0000 7fcbaaffd700 1 -- 172.21.15.84:0/507008701 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fcbb0024430 con 0x7fcbb407f6b0 2024-04-26T18:16:51.179 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.307+0000 7fcbbc035700 1 -- 172.21.15.84:0/507008701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbb4064a80 con 0x7fcbb407f6b0 2024-04-26T18:16:51.179 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.307+0000 7fcbaaffd700 1 --2- 172.21.15.84:0/507008701 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fcba00395b0 0x7fcba003ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:51.179 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.307+0000 7fcbaaffd700 1 -- 172.21.15.84:0/507008701 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fcbb004da80 con 0x7fcbb407f6b0 2024-04-26T18:16:51.179 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.307+0000 7fcbb95d0700 1 --2- 172.21.15.84:0/507008701 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fcba00395b0 0x7fcba003ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:51.179 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.308+0000 7fcbb95d0700 1 --2- 172.21.15.84:0/507008701 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fcba00395b0 0x7fcba003ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcba4006fd0 tx=0x7fcba4006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:51.179 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.313+0000 7fcbaaffd700 1 -- 172.21.15.84:0/507008701 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcbb002d910 con 0x7fcbb407f6b0 2024-04-26T18:16:51.179 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.527+0000 7fcbbc035700 1 -- 172.21.15.84:0/507008701 --> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fcbb41056c0 con 0x7fcba00395b0 2024-04-26T18:16:51.179 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.531+0000 7fcbaaffd700 1 -- 172.21.15.84:0/507008701 <== mgr.14118 v2:172.21.15.84:6800/3919933286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fcbb41056c0 con 0x7fcba00395b0 2024-04-26T18:16:51.180 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.534+0000 7fcbbc035700 1 -- 172.21.15.84:0/507008701 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fcba00395b0 msgr2=0x7fcba003ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:51.180 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.534+0000 7fcbbc035700 1 --2- 172.21.15.84:0/507008701 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fcba00395b0 0x7fcba003ba70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fcba4006fd0 tx=0x7fcba4006e40 comp rx=0 tx=0).stop 2024-04-26T18:16:51.180 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.534+0000 7fcbbc035700 1 -- 172.21.15.84:0/507008701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 msgr2=0x7fcbb419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:51.180 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.534+0000 7fcbbc035700 1 --2- 172.21.15.84:0/507008701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 0x7fcbb419f000 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcbb0003f40 tx=0x7fcbb0011890 comp rx=0 tx=0).stop 2024-04-26T18:16:51.180 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.534+0000 7fcbbc035700 1 -- 172.21.15.84:0/507008701 shutdown_connections 2024-04-26T18:16:51.180 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.534+0000 7fcbbc035700 1 --2- 172.21.15.84:0/507008701 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fcba00395b0 0x7fcba003ba70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:51.180 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.534+0000 7fcbbc035700 1 --2- 172.21.15.84:0/507008701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcbb407f6b0 0x7fcbb419f000 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:51.180 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.534+0000 7fcbbc035700 1 -- 172.21.15.84:0/507008701 >> 172.21.15.84:0/507008701 conn(0x7fcbb4104470 msgr2=0x7fcbb4104fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:51.180 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.534+0000 7fcbbc035700 1 -- 172.21.15.84:0/507008701 shutdown_connections 2024-04-26T18:16:51.180 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:50.534+0000 7fcbbc035700 1 -- 172.21.15.84:0/507008701 wait complete. 2024-04-26T18:16:51.180 INFO:teuthology.orchestra.run.smithi084.stdout:Deploying unmanaged mgr service... 2024-04-26T18:16:51.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:51 smithi084 ceph-mon[29141]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:51.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:51 smithi084 ceph-mon[29141]: Saving service mon spec with placement count:5 2024-04-26T18:16:51.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:51 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:53.396 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T18:16:53.396 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.580+0000 7fa6c7748700 1 Processor -- start 2024-04-26T18:16:53.397 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.580+0000 7fa6c7748700 1 -- start start 2024-04-26T18:16:53.397 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.580+0000 7fa6c7748700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 0x7fa6c010bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:53.397 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.580+0000 7fa6c7748700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c007cd90 con 0x7fa6c01099d0 2024-04-26T18:16:53.397 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.581+0000 7fa6c54e4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 0x7fa6c010bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:53.397 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.581+0000 7fa6c54e4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 0x7fa6c010bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48230/0 (socket says 172.21.15.84:48230) 2024-04-26T18:16:53.397 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.581+0000 7fa6c54e4700 1 -- 172.21.15.84:0/1297813623 learned_addr learned my addr 172.21.15.84:0/1297813623 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:53.398 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.581+0000 7fa6c54e4700 1 -- 172.21.15.84:0/1297813623 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c010c300 con 0x7fa6c01099d0 2024-04-26T18:16:53.398 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.581+0000 7fa6c54e4700 1 --2- 172.21.15.84:0/1297813623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 0x7fa6c010bdc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00098a0 tx=0x7fa6b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2167fe705f4381f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:53.398 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.581+0000 7fa6b7fff700 1 -- 172.21.15.84:0/1297813623 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6b0004160 con 0x7fa6c01099d0 2024-04-26T18:16:53.398 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.581+0000 7fa6b7fff700 1 -- 172.21.15.84:0/1297813623 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b0004300 con 0x7fa6c01099d0 2024-04-26T18:16:53.398 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.581+0000 7fa6b7fff700 1 -- 172.21.15.84:0/1297813623 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6b0011a20 con 0x7fa6c01099d0 2024-04-26T18:16:53.398 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.582+0000 7fa6c7748700 1 -- 172.21.15.84:0/1297813623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 msgr2=0x7fa6c010bdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:53.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.582+0000 7fa6c7748700 1 --2- 172.21.15.84:0/1297813623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 0x7fa6c010bdc0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00098a0 tx=0x7fa6b0009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:53.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.582+0000 7fa6c7748700 1 -- 172.21.15.84:0/1297813623 shutdown_connections 2024-04-26T18:16:53.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.582+0000 7fa6c7748700 1 --2- 172.21.15.84:0/1297813623 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 0x7fa6c010bdc0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:53.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.582+0000 7fa6c7748700 1 -- 172.21.15.84:0/1297813623 >> 172.21.15.84:0/1297813623 conn(0x7fa6c01033c0 msgr2=0x7fa6c01057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:53.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.582+0000 7fa6c7748700 1 -- 172.21.15.84:0/1297813623 shutdown_connections 2024-04-26T18:16:53.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.582+0000 7fa6c7748700 1 -- 172.21.15.84:0/1297813623 wait complete. 2024-04-26T18:16:53.399 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.583+0000 7fa6c7748700 1 Processor -- start 2024-04-26T18:16:53.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.583+0000 7fa6c7748700 1 -- start start 2024-04-26T18:16:53.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.583+0000 7fa6c7748700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 0x7fa6c0109250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:53.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.583+0000 7fa6c7748700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c010d240 con 0x7fa6c01099d0 2024-04-26T18:16:53.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.583+0000 7fa6c54e4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 0x7fa6c0109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:53.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.583+0000 7fa6c54e4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 0x7fa6c0109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48244/0 (socket says 172.21.15.84:48244) 2024-04-26T18:16:53.400 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.583+0000 7fa6c54e4700 1 -- 172.21.15.84:0/3198037848 learned_addr learned my addr 172.21.15.84:0/3198037848 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:53.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.583+0000 7fa6c54e4700 1 -- 172.21.15.84:0/3198037848 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c0109790 con 0x7fa6c01099d0 2024-04-26T18:16:53.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.583+0000 7fa6c54e4700 1 --2- 172.21.15.84:0/3198037848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 0x7fa6c0109250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0004660 tx=0x7fa6b0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:53.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.584+0000 7fa6b67fc700 1 -- 172.21.15.84:0/3198037848 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6b0003aa0 con 0x7fa6c01099d0 2024-04-26T18:16:53.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.584+0000 7fa6b67fc700 1 -- 172.21.15.84:0/3198037848 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b001b8f0 con 0x7fa6c01099d0 2024-04-26T18:16:53.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.584+0000 7fa6b67fc700 1 -- 172.21.15.84:0/3198037848 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6b0023900 con 0x7fa6c01099d0 2024-04-26T18:16:53.401 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.584+0000 7fa6c7748700 1 -- 172.21.15.84:0/3198037848 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c0107980 con 0x7fa6c01099d0 2024-04-26T18:16:53.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.584+0000 7fa6c7748700 1 -- 172.21.15.84:0/3198037848 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c0107ec0 con 0x7fa6c01099d0 2024-04-26T18:16:53.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.585+0000 7fa6b67fc700 1 -- 172.21.15.84:0/3198037848 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fa6b0023bc0 con 0x7fa6c01099d0 2024-04-26T18:16:53.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.585+0000 7fa6b67fc700 1 --2- 172.21.15.84:0/3198037848 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fa6ac0395b0 0x7fa6ac03ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:53.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.585+0000 7fa6b67fc700 1 -- 172.21.15.84:0/3198037848 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fa6b004e0a0 con 0x7fa6c01099d0 2024-04-26T18:16:53.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.585+0000 7fa6c7748700 1 -- 172.21.15.84:0/3198037848 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c0064a80 con 0x7fa6c01099d0 2024-04-26T18:16:53.402 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.585+0000 7fa6c4ce3700 1 --2- 172.21.15.84:0/3198037848 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fa6ac0395b0 0x7fa6ac03ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:53.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.585+0000 7fa6c4ce3700 1 --2- 172.21.15.84:0/3198037848 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fa6ac0395b0 0x7fa6ac03ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc006fd0 tx=0x7fa6bc006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:53.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.590+0000 7fa6b67fc700 1 -- 172.21.15.84:0/3198037848 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6b0031020 con 0x7fa6c01099d0 2024-04-26T18:16:53.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.808+0000 7fa6c7748700 1 -- 172.21.15.84:0/3198037848 --> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fa6c00008d0 con 0x7fa6ac0395b0 2024-04-26T18:16:53.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.816+0000 7fa6b67fc700 1 -- 172.21.15.84:0/3198037848 <== mgr.14118 v2:172.21.15.84:6800/3919933286 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa6c00008d0 con 0x7fa6ac0395b0 2024-04-26T18:16:53.403 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.818+0000 7fa6c7748700 1 -- 172.21.15.84:0/3198037848 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fa6ac0395b0 msgr2=0x7fa6ac03ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:53.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.818+0000 7fa6c7748700 1 --2- 172.21.15.84:0/3198037848 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fa6ac0395b0 0x7fa6ac03ba70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc006fd0 tx=0x7fa6bc006e40 comp rx=0 tx=0).stop 2024-04-26T18:16:53.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.818+0000 7fa6c7748700 1 -- 172.21.15.84:0/3198037848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 msgr2=0x7fa6c0109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:53.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.818+0000 7fa6c7748700 1 --2- 172.21.15.84:0/3198037848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 0x7fa6c0109250 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0004660 tx=0x7fa6b0011aa0 comp rx=0 tx=0).stop 2024-04-26T18:16:53.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.818+0000 7fa6c7748700 1 -- 172.21.15.84:0/3198037848 shutdown_connections 2024-04-26T18:16:53.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.818+0000 7fa6c7748700 1 --2- 172.21.15.84:0/3198037848 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fa6ac0395b0 0x7fa6ac03ba70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:53.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.818+0000 7fa6c7748700 1 --2- 172.21.15.84:0/3198037848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6c01099d0 0x7fa6c0109250 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:53.404 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.818+0000 7fa6c7748700 1 -- 172.21.15.84:0/3198037848 >> 172.21.15.84:0/3198037848 conn(0x7fa6c01033c0 msgr2=0x7fa6c0103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:53.405 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.818+0000 7fa6c7748700 1 -- 172.21.15.84:0/3198037848 shutdown_connections 2024-04-26T18:16:53.405 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:52.818+0000 7fa6c7748700 1 -- 172.21.15.84:0/3198037848 wait complete. 2024-04-26T18:16:53.642 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:53 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:53.642 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:53 smithi084 ceph-mon[29141]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:16:53.642 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:53 smithi084 ceph-mon[29141]: Saving service mgr spec with placement count:2 2024-04-26T18:16:53.642 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:53 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:54.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:54 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:54.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:54 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/533516440' entity='client.admin' 2024-04-26T18:16:55.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.833+0000 7fd9b22d6700 1 Processor -- start 2024-04-26T18:16:55.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.833+0000 7fd9b22d6700 1 -- start start 2024-04-26T18:16:55.268 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.834+0000 7fd9b22d6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 0x7fd9ac10bd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.834+0000 7fd9b22d6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac07cdf0 con 0x7fd9ac109970 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.834+0000 7fd9abfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 0x7fd9ac10bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.834+0000 7fd9abfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 0x7fd9ac10bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48260/0 (socket says 172.21.15.84:48260) 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.834+0000 7fd9abfff700 1 -- 172.21.15.84:0/1634574613 learned_addr learned my addr 172.21.15.84:0/1634574613 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.834+0000 7fd9abfff700 1 -- 172.21.15.84:0/1634574613 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac10c2a0 con 0x7fd9ac109970 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.834+0000 7fd9abfff700 1 --2- 172.21.15.84:0/1634574613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 0x7fd9ac10bd60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0098a0 tx=0x7fd99c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3180d8673e270dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.834+0000 7fd9aaffd700 1 -- 172.21.15.84:0/1634574613 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd99c004160 con 0x7fd9ac109970 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.834+0000 7fd9aaffd700 1 -- 172.21.15.84:0/1634574613 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c004300 con 0x7fd9ac109970 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.834+0000 7fd9aaffd700 1 -- 172.21.15.84:0/1634574613 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd99c011a20 con 0x7fd9ac109970 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.835+0000 7fd9b22d6700 1 -- 172.21.15.84:0/1634574613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 msgr2=0x7fd9ac10bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.835+0000 7fd9b22d6700 1 --2- 172.21.15.84:0/1634574613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 0x7fd9ac10bd60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0098a0 tx=0x7fd99c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.835+0000 7fd9b22d6700 1 -- 172.21.15.84:0/1634574613 shutdown_connections 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.835+0000 7fd9b22d6700 1 --2- 172.21.15.84:0/1634574613 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 0x7fd9ac10bd60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:55.269 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.835+0000 7fd9b22d6700 1 -- 172.21.15.84:0/1634574613 >> 172.21.15.84:0/1634574613 conn(0x7fd9ac1033c0 msgr2=0x7fd9ac1057e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:55.270 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.835+0000 7fd9b22d6700 1 -- 172.21.15.84:0/1634574613 shutdown_connections 2024-04-26T18:16:55.270 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.835+0000 7fd9b22d6700 1 -- 172.21.15.84:0/1634574613 wait complete. 2024-04-26T18:16:55.270 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.836+0000 7fd9b22d6700 1 Processor -- start 2024-04-26T18:16:55.270 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.836+0000 7fd9b22d6700 1 -- start start 2024-04-26T18:16:55.270 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.836+0000 7fd9b22d6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 0x7fd9ac19ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:55.270 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.836+0000 7fd9b22d6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd9ac10d240 con 0x7fd9ac109970 2024-04-26T18:16:55.270 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.836+0000 7fd9abfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 0x7fd9ac19ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:55.270 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.836+0000 7fd9abfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 0x7fd9ac19ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48266/0 (socket says 172.21.15.84:48266) 2024-04-26T18:16:55.270 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.836+0000 7fd9abfff700 1 -- 172.21.15.84:0/533516440 learned_addr learned my addr 172.21.15.84:0/533516440 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:55.270 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.837+0000 7fd9abfff700 1 -- 172.21.15.84:0/533516440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9ac19d3b0 con 0x7fd9ac109970 2024-04-26T18:16:55.270 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.837+0000 7fd9abfff700 1 --2- 172.21.15.84:0/533516440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 0x7fd9ac19ce70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0116e0 tx=0x7fd99c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:55.270 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.837+0000 7fd9a97fa700 1 -- 172.21.15.84:0/533516440 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd99c003b20 con 0x7fd9ac109970 2024-04-26T18:16:55.270 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.837+0000 7fd9a97fa700 1 -- 172.21.15.84:0/533516440 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd99c01b8f0 con 0x7fd9ac109970 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.837+0000 7fd9b22d6700 1 -- 172.21.15.84:0/533516440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9ac19d640 con 0x7fd9ac109970 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.837+0000 7fd9a97fa700 1 -- 172.21.15.84:0/533516440 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd99c0238c0 con 0x7fd9ac109970 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.837+0000 7fd9b22d6700 1 -- 172.21.15.84:0/533516440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9ac1a0330 con 0x7fd9ac109970 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.838+0000 7fd9a97fa700 1 -- 172.21.15.84:0/533516440 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd99c023a80 con 0x7fd9ac109970 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.838+0000 7fd9a97fa700 1 --2- 172.21.15.84:0/533516440 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fd9940395b0 0x7fd99403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.838+0000 7fd9b22d6700 1 -- 172.21.15.84:0/533516440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9ac064a80 con 0x7fd9ac109970 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.838+0000 7fd9a97fa700 1 -- 172.21.15.84:0/533516440 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd99c04e000 con 0x7fd9ac109970 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.838+0000 7fd9ab7fe700 1 --2- 172.21.15.84:0/533516440 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fd9940395b0 0x7fd99403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.838+0000 7fd9ab7fe700 1 --2- 172.21.15.84:0/533516440 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fd9940395b0 0x7fd99403ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a0006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:53.843+0000 7fd9a97fa700 1 -- 172.21.15.84:0/533516440 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd99c01f030 con 0x7fd9ac109970 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:54.063+0000 7fd9b22d6700 1 -- 172.21.15.84:0/533516440 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fd9ac1a08d0 con 0x7fd9ac109970 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:54.067+0000 7fd9a97fa700 1 -- 172.21.15.84:0/533516440 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fd99c023d70 con 0x7fd9ac109970 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:54.069+0000 7fd9b22d6700 1 -- 172.21.15.84:0/533516440 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fd9940395b0 msgr2=0x7fd99403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:54.069+0000 7fd9b22d6700 1 --2- 172.21.15.84:0/533516440 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fd9940395b0 0x7fd99403ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd9a0006fd0 tx=0x7fd9a0006e40 comp rx=0 tx=0).stop 2024-04-26T18:16:55.271 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:54.069+0000 7fd9b22d6700 1 -- 172.21.15.84:0/533516440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 msgr2=0x7fd9ac19ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:55.272 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:54.069+0000 7fd9b22d6700 1 --2- 172.21.15.84:0/533516440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 0x7fd9ac19ce70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0116e0 tx=0x7fd99c011aa0 comp rx=0 tx=0).stop 2024-04-26T18:16:55.272 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:54.069+0000 7fd9b22d6700 1 -- 172.21.15.84:0/533516440 shutdown_connections 2024-04-26T18:16:55.272 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:54.069+0000 7fd9b22d6700 1 --2- 172.21.15.84:0/533516440 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7fd9940395b0 0x7fd99403ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:55.272 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:54.069+0000 7fd9b22d6700 1 --2- 172.21.15.84:0/533516440 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9ac109970 0x7fd9ac19ce70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:55.272 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:54.069+0000 7fd9b22d6700 1 -- 172.21.15.84:0/533516440 >> 172.21.15.84:0/533516440 conn(0x7fd9ac1033c0 msgr2=0x7fd9ac103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:55.272 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:54.070+0000 7fd9b22d6700 1 -- 172.21.15.84:0/533516440 shutdown_connections 2024-04-26T18:16:55.272 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:54.070+0000 7fd9b22d6700 1 -- 172.21.15.84:0/533516440 wait complete. 2024-04-26T18:16:56.537 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:56 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:56.537 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:56 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2491071849' entity='client.admin' 2024-04-26T18:16:56.659 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.742+0000 7efe90444700 1 Processor -- start 2024-04-26T18:16:56.659 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.742+0000 7efe90444700 1 -- start start 2024-04-26T18:16:56.659 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.742+0000 7efe90444700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 0x7efe88109880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:56.659 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.742+0000 7efe90444700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe88109e90 con 0x7efe88109460 2024-04-26T18:16:56.659 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.742+0000 7efe8e1e0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 0x7efe88109880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:56.659 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.742+0000 7efe8e1e0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 0x7efe88109880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48280/0 (socket says 172.21.15.84:48280) 2024-04-26T18:16:56.660 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.742+0000 7efe8e1e0700 1 -- 172.21.15.84:0/813891705 learned_addr learned my addr 172.21.15.84:0/813891705 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:56.660 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.742+0000 7efe8e1e0700 1 -- 172.21.15.84:0/813891705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8810e650 con 0x7efe88109460 2024-04-26T18:16:56.660 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.743+0000 7efe8e1e0700 1 --2- 172.21.15.84:0/813891705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 0x7efe88109880 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe840098a0 tx=0x7efe84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ae3216400172d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:56.660 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.743+0000 7efe8d1de700 1 -- 172.21.15.84:0/813891705 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe84004160 con 0x7efe88109460 2024-04-26T18:16:56.660 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.743+0000 7efe8d1de700 1 -- 172.21.15.84:0/813891705 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe84004300 con 0x7efe88109460 2024-04-26T18:16:56.661 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.743+0000 7efe8d1de700 1 -- 172.21.15.84:0/813891705 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe84011a20 con 0x7efe88109460 2024-04-26T18:16:56.661 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.743+0000 7efe90444700 1 -- 172.21.15.84:0/813891705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 msgr2=0x7efe88109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:56.661 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.743+0000 7efe90444700 1 --2- 172.21.15.84:0/813891705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 0x7efe88109880 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efe840098a0 tx=0x7efe84009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:56.661 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.744+0000 7efe90444700 1 -- 172.21.15.84:0/813891705 shutdown_connections 2024-04-26T18:16:56.661 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.744+0000 7efe90444700 1 --2- 172.21.15.84:0/813891705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 0x7efe88109880 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:56.661 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.744+0000 7efe90444700 1 -- 172.21.15.84:0/813891705 >> 172.21.15.84:0/813891705 conn(0x7efe88102b40 msgr2=0x7efe88104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:56.661 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.744+0000 7efe90444700 1 -- 172.21.15.84:0/813891705 shutdown_connections 2024-04-26T18:16:56.661 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.744+0000 7efe90444700 1 -- 172.21.15.84:0/813891705 wait complete. 2024-04-26T18:16:56.661 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.744+0000 7efe90444700 1 Processor -- start 2024-04-26T18:16:56.662 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.745+0000 7efe90444700 1 -- start start 2024-04-26T18:16:56.662 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.745+0000 7efe90444700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 0x7efe8819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:56.662 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.745+0000 7efe90444700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe8810f590 con 0x7efe88109460 2024-04-26T18:16:56.662 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.745+0000 7efe8e1e0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 0x7efe8819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:56.662 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.745+0000 7efe8e1e0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 0x7efe8819f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48292/0 (socket says 172.21.15.84:48292) 2024-04-26T18:16:56.662 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.745+0000 7efe8e1e0700 1 -- 172.21.15.84:0/2491071849 learned_addr learned my addr 172.21.15.84:0/2491071849 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:56.662 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.745+0000 7efe8e1e0700 1 -- 172.21.15.84:0/2491071849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8819f550 con 0x7efe88109460 2024-04-26T18:16:56.662 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.745+0000 7efe8e1e0700 1 --2- 172.21.15.84:0/2491071849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 0x7efe8819f010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe84004660 tx=0x7efe84011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:56.662 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.745+0000 7efe7f7fe700 1 -- 172.21.15.84:0/2491071849 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe84003aa0 con 0x7efe88109460 2024-04-26T18:16:56.662 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.745+0000 7efe7f7fe700 1 -- 172.21.15.84:0/2491071849 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe8401b8f0 con 0x7efe88109460 2024-04-26T18:16:56.662 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.745+0000 7efe90444700 1 -- 172.21.15.84:0/2491071849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8819f7e0 con 0x7efe88109460 2024-04-26T18:16:56.662 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.746+0000 7efe7f7fe700 1 -- 172.21.15.84:0/2491071849 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe84023900 con 0x7efe88109460 2024-04-26T18:16:56.663 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.746+0000 7efe90444700 1 -- 172.21.15.84:0/2491071849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe881a24d0 con 0x7efe88109460 2024-04-26T18:16:56.663 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.747+0000 7efe7f7fe700 1 -- 172.21.15.84:0/2491071849 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7efe84023b70 con 0x7efe88109460 2024-04-26T18:16:56.663 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.747+0000 7efe7f7fe700 1 --2- 172.21.15.84:0/2491071849 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7efe74039600 0x7efe7403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:56.663 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.747+0000 7efe90444700 1 -- 172.21.15.84:0/2491071849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe88064a80 con 0x7efe88109460 2024-04-26T18:16:56.663 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.747+0000 7efe7f7fe700 1 -- 172.21.15.84:0/2491071849 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7efe8404de20 con 0x7efe88109460 2024-04-26T18:16:56.663 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.747+0000 7efe8d9df700 1 --2- 172.21.15.84:0/2491071849 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7efe74039600 0x7efe7403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:56.663 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.748+0000 7efe8d9df700 1 --2- 172.21.15.84:0/2491071849 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7efe74039600 0x7efe7403bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efe78006fd0 tx=0x7efe78006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:56.663 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.753+0000 7efe7f7fe700 1 -- 172.21.15.84:0/2491071849 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe8401f030 con 0x7efe88109460 2024-04-26T18:16:56.663 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.960+0000 7efe90444700 1 -- 172.21.15.84:0/2491071849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7efe881a26a0 con 0x7efe88109460 2024-04-26T18:16:56.663 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.963+0000 7efe7f7fe700 1 -- 172.21.15.84:0/2491071849 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7efe8401bde0 con 0x7efe88109460 2024-04-26T18:16:56.664 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.965+0000 7efe90444700 1 -- 172.21.15.84:0/2491071849 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7efe74039600 msgr2=0x7efe7403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:56.664 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.965+0000 7efe90444700 1 --2- 172.21.15.84:0/2491071849 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7efe74039600 0x7efe7403bac0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7efe78006fd0 tx=0x7efe78006e40 comp rx=0 tx=0).stop 2024-04-26T18:16:56.664 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.965+0000 7efe90444700 1 -- 172.21.15.84:0/2491071849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 msgr2=0x7efe8819f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:56.664 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.965+0000 7efe90444700 1 --2- 172.21.15.84:0/2491071849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 0x7efe8819f010 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efe84004660 tx=0x7efe84011aa0 comp rx=0 tx=0).stop 2024-04-26T18:16:56.664 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.966+0000 7efe90444700 1 -- 172.21.15.84:0/2491071849 shutdown_connections 2024-04-26T18:16:56.664 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.966+0000 7efe90444700 1 --2- 172.21.15.84:0/2491071849 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7efe74039600 0x7efe7403bac0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:56.664 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.966+0000 7efe90444700 1 --2- 172.21.15.84:0/2491071849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe88109460 0x7efe8819f010 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:56.664 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.966+0000 7efe90444700 1 -- 172.21.15.84:0/2491071849 >> 172.21.15.84:0/2491071849 conn(0x7efe88102b40 msgr2=0x7efe88103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:56.664 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.966+0000 7efe90444700 1 -- 172.21.15.84:0/2491071849 shutdown_connections 2024-04-26T18:16:56.664 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:55.966+0000 7efe90444700 1 -- 172.21.15.84:0/2491071849 wait complete. 2024-04-26T18:16:56.664 INFO:teuthology.orchestra.run.smithi084.stdout:Enabling the dashboard module... 2024-04-26T18:16:58.307 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:58 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: ignoring --setuser ceph since I am not root 2024-04-26T18:16:58.308 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:58 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: ignoring --setgroup ceph since I am not root 2024-04-26T18:16:58.308 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:58 smithi084 ceph-mon[29141]: from='mgr.14118 172.21.15.84:0/3747034629' entity='mgr.x' 2024-04-26T18:16:58.308 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:58 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2306954250' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T18:16:58.308 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:58 smithi084 ceph-mon[29141]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:16:58.594 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:58 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:58.340+0000 7f90fdf2d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:16:58.595 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:58 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:58.526+0000 7f90fdf2d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:16:58.935 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:58 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:58.632+0000 7f90fdf2d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:16:58.938 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.070+0000 7f5f9ee4f700 1 Processor -- start 2024-04-26T18:16:58.943 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.070+0000 7f5f9ee4f700 1 -- start start 2024-04-26T18:16:58.944 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.070+0000 7f5f9ee4f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 0x7f5f9807db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:58.944 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.070+0000 7f5f9ee4f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9807e060 con 0x7f5f9807f670 2024-04-26T18:16:58.944 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.070+0000 7f5f9cbeb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 0x7f5f9807db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:58.944 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.070+0000 7f5f9cbeb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 0x7f5f9807db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40008/0 (socket says 172.21.15.84:40008) 2024-04-26T18:16:58.944 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.070+0000 7f5f9cbeb700 1 -- 172.21.15.84:0/2683846242 learned_addr learned my addr 172.21.15.84:0/2683846242 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:58.944 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.071+0000 7f5f9cbeb700 1 -- 172.21.15.84:0/2683846242 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9807e1e0 con 0x7f5f9807f670 2024-04-26T18:16:58.944 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.071+0000 7f5f9cbeb700 1 --2- 172.21.15.84:0/2683846242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 0x7f5f9807db20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0098a0 tx=0x7f5f8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8036e79ea8675e60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:58.944 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.071+0000 7f5f977fe700 1 -- 172.21.15.84:0/2683846242 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f8c004160 con 0x7f5f9807f670 2024-04-26T18:16:58.944 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.071+0000 7f5f977fe700 1 -- 172.21.15.84:0/2683846242 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f8c004300 con 0x7f5f9807f670 2024-04-26T18:16:58.944 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.071+0000 7f5f977fe700 1 -- 172.21.15.84:0/2683846242 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f8c011a20 con 0x7f5f9807f670 2024-04-26T18:16:58.944 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.071+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2683846242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 msgr2=0x7f5f9807db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:58.944 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.071+0000 7f5f9ee4f700 1 --2- 172.21.15.84:0/2683846242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 0x7f5f9807db20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0098a0 tx=0x7f5f8c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:16:58.944 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.072+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2683846242 shutdown_connections 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.072+0000 7f5f9ee4f700 1 --2- 172.21.15.84:0/2683846242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 0x7f5f9807db20 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.072+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2683846242 >> 172.21.15.84:0/2683846242 conn(0x7f5f98104470 msgr2=0x7f5f981068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.072+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2683846242 shutdown_connections 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.072+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2683846242 wait complete. 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.072+0000 7f5f9ee4f700 1 Processor -- start 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.072+0000 7f5f9ee4f700 1 -- start start 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.072+0000 7f5f9ee4f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 0x7f5f9819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.072+0000 7f5f9ee4f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5f9810f450 con 0x7f5f9807f670 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.072+0000 7f5f9cbeb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 0x7f5f9819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.072+0000 7f5f9cbeb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 0x7f5f9819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40012/0 (socket says 172.21.15.84:40012) 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.072+0000 7f5f9cbeb700 1 -- 172.21.15.84:0/2306954250 learned_addr learned my addr 172.21.15.84:0/2306954250 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.073+0000 7f5f9cbeb700 1 -- 172.21.15.84:0/2306954250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f9819f4c0 con 0x7f5f9807f670 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.073+0000 7f5f9cbeb700 1 --2- 172.21.15.84:0/2306954250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 0x7f5f9819ef80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0116e0 tx=0x7f5f8c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.073+0000 7f5f95ffb700 1 -- 172.21.15.84:0/2306954250 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f8c003a60 con 0x7f5f9807f670 2024-04-26T18:16:58.945 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.073+0000 7f5f95ffb700 1 -- 172.21.15.84:0/2306954250 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f8c01b8f0 con 0x7f5f9807f670 2024-04-26T18:16:58.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.073+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2306954250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f9819f750 con 0x7f5f9807f670 2024-04-26T18:16:58.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.073+0000 7f5f95ffb700 1 -- 172.21.15.84:0/2306954250 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5f8c019860 con 0x7f5f9807f670 2024-04-26T18:16:58.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.073+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2306954250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f981a2440 con 0x7f5f9807f670 2024-04-26T18:16:58.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.074+0000 7f5f95ffb700 1 -- 172.21.15.84:0/2306954250 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5f8c019b10 con 0x7f5f9807f670 2024-04-26T18:16:58.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.074+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2306954250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f98064a80 con 0x7f5f9807f670 2024-04-26T18:16:58.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.074+0000 7f5f95ffb700 1 --2- 172.21.15.84:0/2306954250 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f5f880395b0 0x7f5f8803ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:16:58.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.074+0000 7f5f95ffb700 1 -- 172.21.15.84:0/2306954250 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5f8c04dde0 con 0x7f5f9807f670 2024-04-26T18:16:58.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.075+0000 7f5f97fff700 1 --2- 172.21.15.84:0/2306954250 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f5f880395b0 0x7f5f8803ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:16:58.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.075+0000 7f5f97fff700 1 --2- 172.21.15.84:0/2306954250 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f5f880395b0 0x7f5f8803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f84006fd0 tx=0x7f5f84006e40 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:16:58.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.079+0000 7f5f95ffb700 1 -- 172.21.15.84:0/2306954250 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5f8c01f030 con 0x7f5f9807f670 2024-04-26T18:16:58.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:57.364+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2306954250 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f5f981a29f0 con 0x7f5f9807f670 2024-04-26T18:16:58.946 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:58.163+0000 7f5f95ffb700 1 -- 172.21.15.84:0/2306954250 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f5f8c022a00 con 0x7f5f9807f670 2024-04-26T18:16:58.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:58.165+0000 7f5f95ffb700 1 -- 172.21.15.84:0/2306954250 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f5f8c053020 con 0x7f5f9807f670 2024-04-26T18:16:58.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:58.166+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2306954250 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f5f880395b0 msgr2=0x7f5f8803ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:58.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:58.166+0000 7f5f9ee4f700 1 --2- 172.21.15.84:0/2306954250 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f5f880395b0 0x7f5f8803ba70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5f84006fd0 tx=0x7f5f84006e40 comp rx=0 tx=0).stop 2024-04-26T18:16:58.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:58.166+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2306954250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 msgr2=0x7f5f9819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:16:58.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:58.166+0000 7f5f9ee4f700 1 --2- 172.21.15.84:0/2306954250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 0x7f5f9819ef80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c0116e0 tx=0x7f5f8c011aa0 comp rx=0 tx=0).stop 2024-04-26T18:16:58.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:58.166+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2306954250 shutdown_connections 2024-04-26T18:16:58.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:58.166+0000 7f5f9ee4f700 1 --2- 172.21.15.84:0/2306954250 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f5f880395b0 0x7f5f8803ba70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:58.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:58.166+0000 7f5f9ee4f700 1 --2- 172.21.15.84:0/2306954250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5f9807f670 0x7f5f9819ef80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:16:58.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:58.166+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2306954250 >> 172.21.15.84:0/2306954250 conn(0x7f5f98104470 msgr2=0x7f5f98104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:16:58.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:58.166+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2306954250 shutdown_connections 2024-04-26T18:16:58.947 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:58.166+0000 7f5f9ee4f700 1 -- 172.21.15.84:0/2306954250 wait complete. 2024-04-26T18:16:59.288 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:59 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2306954250' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T18:16:59.288 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:16:59 smithi084 ceph-mon[29141]: mgrmap e9: x(active, since 20s) 2024-04-26T18:16:59.289 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:58 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:58.942+0000 7f90fdf2d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:16:59.289 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:16:59 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:16:59.031+0000 7f90fdf2d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:17:00.431 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:00 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3377901063' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T18:17:00.431 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:00 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:00.258+0000 7f90fdf2d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:17:00.461 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:17:00.461 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T18:17:00.461 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T18:17:00.461 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-26T18:17:00.461 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T18:17:00.462 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:17:00.462 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.389+0000 7fedc3b1b700 1 Processor -- start 2024-04-26T18:17:00.462 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.389+0000 7fedc3b1b700 1 -- start start 2024-04-26T18:17:00.462 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.390+0000 7fedc3b1b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 0x7fedbc10bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:00.462 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.390+0000 7fedc3b1b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc07d660 con 0x7fedbc109960 2024-04-26T18:17:00.462 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.390+0000 7fedc18b7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 0x7fedbc10bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:00.462 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.390+0000 7fedc18b7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 0x7fedbc10bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40048/0 (socket says 172.21.15.84:40048) 2024-04-26T18:17:00.462 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.390+0000 7fedc18b7700 1 -- 172.21.15.84:0/578816482 learned_addr learned my addr 172.21.15.84:0/578816482 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:00.462 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.390+0000 7fedc18b7700 1 -- 172.21.15.84:0/578816482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedbc10c290 con 0x7fedbc109960 2024-04-26T18:17:00.462 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.390+0000 7fedc18b7700 1 --2- 172.21.15.84:0/578816482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 0x7fedbc10bd50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fedb80098a0 tx=0x7fedb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45900e1252d89527 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:00.462 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.391+0000 7fedc08b5700 1 -- 172.21.15.84:0/578816482 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb8004160 con 0x7fedbc109960 2024-04-26T18:17:00.462 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.391+0000 7fedc08b5700 1 -- 172.21.15.84:0/578816482 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedb8004300 con 0x7fedbc109960 2024-04-26T18:17:00.462 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.391+0000 7fedc08b5700 1 -- 172.21.15.84:0/578816482 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb8011a20 con 0x7fedbc109960 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.391+0000 7fedc3b1b700 1 -- 172.21.15.84:0/578816482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 msgr2=0x7fedbc10bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.391+0000 7fedc3b1b700 1 --2- 172.21.15.84:0/578816482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 0x7fedbc10bd50 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fedb80098a0 tx=0x7fedb8009bb0 comp rx=0 tx=0).stop 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.391+0000 7fedc3b1b700 1 -- 172.21.15.84:0/578816482 shutdown_connections 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.391+0000 7fedc3b1b700 1 --2- 172.21.15.84:0/578816482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 0x7fedbc10bd50 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.391+0000 7fedc3b1b700 1 -- 172.21.15.84:0/578816482 >> 172.21.15.84:0/578816482 conn(0x7fedbc1033b0 msgr2=0x7fedbc1057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.391+0000 7fedc3b1b700 1 -- 172.21.15.84:0/578816482 shutdown_connections 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.392+0000 7fedc3b1b700 1 -- 172.21.15.84:0/578816482 wait complete. 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.392+0000 7fedc3b1b700 1 Processor -- start 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.392+0000 7fedc3b1b700 1 -- start start 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.392+0000 7fedc3b1b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 0x7fedbc19cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.392+0000 7fedc3b1b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fedbc10d230 con 0x7fedbc109960 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.392+0000 7fedc18b7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 0x7fedbc19cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.392+0000 7fedc18b7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 0x7fedbc19cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40054/0 (socket says 172.21.15.84:40054) 2024-04-26T18:17:00.463 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.392+0000 7fedc18b7700 1 -- 172.21.15.84:0/3377901063 learned_addr learned my addr 172.21.15.84:0/3377901063 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:00.464 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.393+0000 7fedc18b7700 1 -- 172.21.15.84:0/3377901063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedbc19d2a0 con 0x7fedbc109960 2024-04-26T18:17:00.464 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.393+0000 7fedc18b7700 1 --2- 172.21.15.84:0/3377901063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 0x7fedbc19cd60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fedb80116e0 tx=0x7fedb8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:00.464 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.393+0000 7fedb2ffd700 1 -- 172.21.15.84:0/3377901063 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb8003a60 con 0x7fedbc109960 2024-04-26T18:17:00.464 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.393+0000 7fedb2ffd700 1 -- 172.21.15.84:0/3377901063 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fedb801b8f0 con 0x7fedbc109960 2024-04-26T18:17:00.464 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.393+0000 7fedc3b1b700 1 -- 172.21.15.84:0/3377901063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedbc19d530 con 0x7fedbc109960 2024-04-26T18:17:00.464 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.393+0000 7fedb2ffd700 1 -- 172.21.15.84:0/3377901063 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedb8019860 con 0x7fedbc109960 2024-04-26T18:17:00.464 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.393+0000 7fedc3b1b700 1 -- 172.21.15.84:0/3377901063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedbc1a0220 con 0x7fedbc109960 2024-04-26T18:17:00.464 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.394+0000 7fedb2ffd700 1 -- 172.21.15.84:0/3377901063 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fedb8019b30 con 0x7fedbc109960 2024-04-26T18:17:00.464 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.394+0000 7fedb2ffd700 1 --2- 172.21.15.84:0/3377901063 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7feda8039610 0x7feda803bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:00.464 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.394+0000 7fedb2ffd700 1 -- 172.21.15.84:0/3377901063 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fedb804dcd0 con 0x7fedbc109960 2024-04-26T18:17:00.464 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.394+0000 7fedc10b6700 1 -- 172.21.15.84:0/3377901063 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7feda8039610 msgr2=0x7feda803bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3919933286 2024-04-26T18:17:00.464 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.394+0000 7fedc10b6700 1 --2- 172.21.15.84:0/3377901063 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7feda8039610 0x7feda803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:17:00.464 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.394+0000 7fedc3b1b700 1 -- 172.21.15.84:0/3377901063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedbc064a80 con 0x7fedbc109960 2024-04-26T18:17:00.465 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.399+0000 7fedb2ffd700 1 -- 172.21.15.84:0/3377901063 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fedb801f030 con 0x7fedbc109960 2024-04-26T18:17:00.465 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.595+0000 7fedc10b6700 1 -- 172.21.15.84:0/3377901063 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7feda8039610 msgr2=0x7feda803bad0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3919933286 2024-04-26T18:17:00.465 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.595+0000 7fedc10b6700 1 --2- 172.21.15.84:0/3377901063 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7feda8039610 0x7feda803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:17:00.465 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.696+0000 7fedc3b1b700 1 -- 172.21.15.84:0/3377901063 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fedbc107c20 con 0x7fedbc109960 2024-04-26T18:17:00.465 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.696+0000 7fedb2ffd700 1 -- 172.21.15.84:0/3377901063 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fedb8029260 con 0x7fedbc109960 2024-04-26T18:17:00.465 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.698+0000 7fedc3b1b700 1 -- 172.21.15.84:0/3377901063 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7feda8039610 msgr2=0x7feda803bad0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:17:00.465 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.698+0000 7fedc3b1b700 1 --2- 172.21.15.84:0/3377901063 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7feda8039610 0x7feda803bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:00.465 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.698+0000 7fedc3b1b700 1 -- 172.21.15.84:0/3377901063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 msgr2=0x7fedbc19cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:00.465 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.698+0000 7fedc3b1b700 1 --2- 172.21.15.84:0/3377901063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 0x7fedbc19cd60 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fedb80116e0 tx=0x7fedb8011aa0 comp rx=0 tx=0).stop 2024-04-26T18:17:00.465 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.698+0000 7fedc3b1b700 1 -- 172.21.15.84:0/3377901063 shutdown_connections 2024-04-26T18:17:00.465 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.698+0000 7fedc3b1b700 1 --2- 172.21.15.84:0/3377901063 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7feda8039610 0x7feda803bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:00.465 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.698+0000 7fedc3b1b700 1 --2- 172.21.15.84:0/3377901063 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedbc109960 0x7fedbc19cd60 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:00.465 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.698+0000 7fedc3b1b700 1 -- 172.21.15.84:0/3377901063 >> 172.21.15.84:0/3377901063 conn(0x7fedbc1033b0 msgr2=0x7fedbc103ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:00.466 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.699+0000 7fedc3b1b700 1 -- 172.21.15.84:0/3377901063 shutdown_connections 2024-04-26T18:17:00.466 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:16:59.699+0000 7fedc3b1b700 1 -- 172.21.15.84:0/3377901063 wait complete. 2024-04-26T18:17:00.466 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for the mgr to restart... 2024-04-26T18:17:00.466 INFO:teuthology.orchestra.run.smithi084.stdout:Waiting for mgr epoch 9... 2024-04-26T18:17:00.700 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:00 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:00.431+0000 7f90fdf2d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:17:00.700 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:00 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:00.610+0000 7f90fdf2d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:17:01.003 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:00 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:00.909+0000 7f90fdf2d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:17:01.004 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:01 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:01.003+0000 7f90fdf2d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:17:01.344 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:01 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:01.210+0000 7f90fdf2d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:17:01.844 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:01 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:01.404+0000 7f90fdf2d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:17:02.305 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:02 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:02.007+0000 7f90fdf2d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:17:02.594 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:02 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:02.433+0000 7f90fdf2d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:17:02.594 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:02 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:02.530+0000 7f90fdf2d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:17:03.094 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:02 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:02.734+0000 7f90fdf2d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:17:03.594 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:03 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:03.270+0000 7f90fdf2d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:17:03.594 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:03 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:03.363+0000 7f90fdf2d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:17:03.907 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:03 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:03.608+0000 7f90fdf2d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:17:03.907 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:03 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:03.744+0000 7f90fdf2d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:17:03.907 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:03 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:03.840+0000 7f90fdf2d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:17:04.258 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:03 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:03.956+0000 7f90fdf2d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:17:04.594 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:04 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:04.258+0000 7f90fdf2d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:17:05.136 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:04 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:04.861+0000 7f90fdf2d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:17:05.136 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:17:05.080+0000 7f90fdf2d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:17:05.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-mon[29141]: Active manager daemon x restarted 2024-04-26T18:17:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-mon[29141]: Activating manager daemon x 2024-04-26T18:17:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-mon[29141]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T18:17:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-mon[29141]: mgrmap e10: x(active, starting, since 0.00341501s) 2024-04-26T18:17:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:17:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:17:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:17:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:17:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:17:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-mon[29141]: Manager daemon x is now available 2024-04-26T18:17:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:17:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:17:06.652 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout { 2024-04-26T18:17:06.652 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T18:17:06.652 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T18:17:06.652 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout } 2024-04-26T18:17:06.653 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.898+0000 7f044ee9e700 1 Processor -- start 2024-04-26T18:17:06.653 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.898+0000 7f044ee9e700 1 -- start start 2024-04-26T18:17:06.653 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.898+0000 7f044ee9e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 0x7f044810bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:06.653 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.898+0000 7f044ee9e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f044807d640 con 0x7f0448109980 2024-04-26T18:17:06.653 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.898+0000 7f044cc3a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 0x7f044810bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:06.653 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.898+0000 7f044cc3a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 0x7f044810bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40056/0 (socket says 172.21.15.84:40056) 2024-04-26T18:17:06.653 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.898+0000 7f044cc3a700 1 -- 172.21.15.84:0/2468259754 learned_addr learned my addr 172.21.15.84:0/2468259754 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:06.653 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.899+0000 7f044cc3a700 1 -- 172.21.15.84:0/2468259754 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044810c2b0 con 0x7f0448109980 2024-04-26T18:17:06.653 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.899+0000 7f044cc3a700 1 --2- 172.21.15.84:0/2468259754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 0x7f044810bd70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f043c0098a0 tx=0x7f043c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956f4ae725824099 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:06.653 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.899+0000 7f04477fe700 1 -- 172.21.15.84:0/2468259754 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f043c004160 con 0x7f0448109980 2024-04-26T18:17:06.653 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.899+0000 7f04477fe700 1 -- 172.21.15.84:0/2468259754 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f043c004300 con 0x7f0448109980 2024-04-26T18:17:06.653 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.899+0000 7f04477fe700 1 -- 172.21.15.84:0/2468259754 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f043c011a20 con 0x7f0448109980 2024-04-26T18:17:06.653 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.900+0000 7f044ee9e700 1 -- 172.21.15.84:0/2468259754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 msgr2=0x7f044810bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.900+0000 7f044ee9e700 1 --2- 172.21.15.84:0/2468259754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 0x7f044810bd70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f043c0098a0 tx=0x7f043c009bb0 comp rx=0 tx=0).stop 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.900+0000 7f044ee9e700 1 -- 172.21.15.84:0/2468259754 shutdown_connections 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.900+0000 7f044ee9e700 1 --2- 172.21.15.84:0/2468259754 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 0x7f044810bd70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.900+0000 7f044ee9e700 1 -- 172.21.15.84:0/2468259754 >> 172.21.15.84:0/2468259754 conn(0x7f04481033b0 msgr2=0x7f04481057d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.900+0000 7f044ee9e700 1 -- 172.21.15.84:0/2468259754 shutdown_connections 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.900+0000 7f044ee9e700 1 -- 172.21.15.84:0/2468259754 wait complete. 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.900+0000 7f044ee9e700 1 Processor -- start 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.900+0000 7f044ee9e700 1 -- start start 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.900+0000 7f044ee9e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 0x7f044819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.900+0000 7f044ee9e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f044810d250 con 0x7f0448109980 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.901+0000 7f044cc3a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 0x7f044819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.901+0000 7f044cc3a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 0x7f044819cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40062/0 (socket says 172.21.15.84:40062) 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.901+0000 7f044cc3a700 1 -- 172.21.15.84:0/3207869820 learned_addr learned my addr 172.21.15.84:0/3207869820 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:06.654 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.901+0000 7f044cc3a700 1 -- 172.21.15.84:0/3207869820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f044819d2a0 con 0x7f0448109980 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.901+0000 7f044cc3a700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 0x7f044819cd60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f043c000c00 tx=0x7f043c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.901+0000 7f0445ffb700 1 -- 172.21.15.84:0/3207869820 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f043c003aa0 con 0x7f0448109980 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.901+0000 7f0445ffb700 1 -- 172.21.15.84:0/3207869820 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f043c01b8f0 con 0x7f0448109980 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.901+0000 7f044ee9e700 1 -- 172.21.15.84:0/3207869820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f044819d530 con 0x7f0448109980 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.901+0000 7f0445ffb700 1 -- 172.21.15.84:0/3207869820 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f043c023870 con 0x7f0448109980 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.901+0000 7f044ee9e700 1 -- 172.21.15.84:0/3207869820 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04481a0220 con 0x7f0448109980 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.902+0000 7f0445ffb700 1 -- 172.21.15.84:0/3207869820 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f043c023b70 con 0x7f0448109980 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.902+0000 7f0445ffb700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 0x7f043803bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.902+0000 7f044ee9e700 1 -- 172.21.15.84:0/3207869820 --> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f044806a570 con 0x7f0438039600 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.902+0000 7f0445ffb700 1 -- 172.21.15.84:0/3207869820 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f043c04ee80 con 0x7f0448109980 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.902+0000 7f0447fff700 1 -- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 msgr2=0x7f043803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3919933286 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:00.902+0000 7f0447fff700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 0x7f043803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:01.103+0000 7f0447fff700 1 -- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 msgr2=0x7f043803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3919933286 2024-04-26T18:17:06.655 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:01.103+0000 7f0447fff700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 0x7f043803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:01.504+0000 7f0447fff700 1 -- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 msgr2=0x7f043803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3919933286 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:01.504+0000 7f0447fff700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 0x7f043803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:02.305+0000 7f0447fff700 1 -- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 msgr2=0x7f043803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3919933286 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:02.305+0000 7f0447fff700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 0x7f043803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:03.906+0000 7f0447fff700 1 -- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 msgr2=0x7f043803bac0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/3919933286 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:03.906+0000 7f0447fff700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 0x7f043803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:05.084+0000 7f0445ffb700 1 -- 172.21.15.84:0/3207869820 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f043c02c910 con 0x7f0448109980 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:05.084+0000 7f0445ffb700 1 -- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 msgr2=0x7f043803bac0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:05.084+0000 7f0445ffb700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 0x7f043803bac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.086+0000 7f0445ffb700 1 -- 172.21.15.84:0/3207869820 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f043c02b350 con 0x7f0448109980 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.086+0000 7f0445ffb700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f043803ce90 0x7f043803f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.087+0000 7f0445ffb700 1 -- 172.21.15.84:0/3207869820 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f044806a570 con 0x7f043803ce90 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.087+0000 7f0447fff700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f043803ce90 0x7f043803f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:06.656 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.088+0000 7f0447fff700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f043803ce90 0x7f043803f280 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0434003a10 tx=0x7f04340092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.088+0000 7f0445ffb700 1 -- 172.21.15.84:0/3207869820 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f044806a570 con 0x7f043803ce90 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.096+0000 7f044ee9e700 1 -- 172.21.15.84:0/3207869820 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f0448104620 con 0x7f043803ce90 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.097+0000 7f0445ffb700 1 -- 172.21.15.84:0/3207869820 <== mgr.14150 v2:172.21.15.84:6800/2059208901 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f0448104620 con 0x7f043803ce90 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.097+0000 7f044ee9e700 1 -- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f043803ce90 msgr2=0x7f043803f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.097+0000 7f044ee9e700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f043803ce90 0x7f043803f280 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0434003a10 tx=0x7f04340092b0 comp rx=0 tx=0).stop 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.097+0000 7f044ee9e700 1 -- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 msgr2=0x7f044819cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.097+0000 7f044ee9e700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 0x7f044819cd60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f043c000c00 tx=0x7f043c011aa0 comp rx=0 tx=0).stop 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.097+0000 7f044ee9e700 1 -- 172.21.15.84:0/3207869820 shutdown_connections 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.097+0000 7f044ee9e700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f043803ce90 0x7f043803f280 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.097+0000 7f044ee9e700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0448109980 0x7f044819cd60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.097+0000 7f044ee9e700 1 --2- 172.21.15.84:0/3207869820 >> [v2:172.21.15.84:6800/3919933286,v1:172.21.15.84:6801/3919933286] conn(0x7f0438039600 0x7f043803bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.097+0000 7f044ee9e700 1 -- 172.21.15.84:0/3207869820 >> 172.21.15.84:0/3207869820 conn(0x7f04481033b0 msgr2=0x7f0448103f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.097+0000 7f044ee9e700 1 -- 172.21.15.84:0/3207869820 shutdown_connections 2024-04-26T18:17:06.657 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:06.097+0000 7f044ee9e700 1 -- 172.21.15.84:0/3207869820 wait complete. 2024-04-26T18:17:06.658 INFO:teuthology.orchestra.run.smithi084.stdout:mgr epoch 9 is available 2024-04-26T18:17:06.658 INFO:teuthology.orchestra.run.smithi084.stdout:Generating a dashboard self-signed certificate... 2024-04-26T18:17:07.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:07 smithi084 ceph-mon[29141]: [26/Apr/2024:18:17:05] ENGINE Bus STARTING 2024-04-26T18:17:07.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:07 smithi084 ceph-mon[29141]: [26/Apr/2024:18:17:05] ENGINE Serving on http://172.21.15.84:8765 2024-04-26T18:17:07.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:07 smithi084 ceph-mon[29141]: [26/Apr/2024:18:17:05] ENGINE Serving on https://172.21.15.84:7150 2024-04-26T18:17:07.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:07 smithi084 ceph-mon[29141]: [26/Apr/2024:18:17:05] ENGINE Bus STARTED 2024-04-26T18:17:07.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:07 smithi084 ceph-mon[29141]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T18:17:07.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:07 smithi084 ceph-mon[29141]: mgrmap e11: x(active, since 1.00637s) 2024-04-26T18:17:07.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:07 smithi084 ceph-mon[29141]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T18:17:08.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:08 smithi084 ceph-mon[29141]: mgrmap e12: x(active, since 2s) 2024-04-26T18:17:08.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:08 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:08.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:08 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:08.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:08 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:08.508 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T18:17:08.508 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.557+0000 7f30d1058700 1 Processor -- start 2024-04-26T18:17:08.508 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.557+0000 7f30d1058700 1 -- start start 2024-04-26T18:17:08.508 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.557+0000 7f30d1058700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 0x7f30cc1092e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:08.509 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.557+0000 7f30d1058700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc109820 con 0x7f30cc108ec0 2024-04-26T18:17:08.509 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.557+0000 7f30cad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 0x7f30cc1092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:08.509 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.557+0000 7f30cad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 0x7f30cc1092e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41848/0 (socket says 172.21.15.84:41848) 2024-04-26T18:17:08.509 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.557+0000 7f30cad9d700 1 -- 172.21.15.84:0/351914537 learned_addr learned my addr 172.21.15.84:0/351914537 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:08.509 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.558+0000 7f30cad9d700 1 -- 172.21.15.84:0/351914537 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc109fd0 con 0x7f30cc108ec0 2024-04-26T18:17:08.509 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.558+0000 7f30cad9d700 1 --2- 172.21.15.84:0/351914537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 0x7f30cc1092e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009e80 tx=0x7f30c400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f09ac276be56200 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:08.509 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.558+0000 7f30c9d9b700 1 -- 172.21.15.84:0/351914537 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30c4017070 con 0x7f30cc108ec0 2024-04-26T18:17:08.509 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.558+0000 7f30c9d9b700 1 -- 172.21.15.84:0/351914537 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c400b8c0 con 0x7f30cc108ec0 2024-04-26T18:17:08.509 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.558+0000 7f30c9d9b700 1 -- 172.21.15.84:0/351914537 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30c4004490 con 0x7f30cc108ec0 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.559+0000 7f30d1058700 1 -- 172.21.15.84:0/351914537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 msgr2=0x7f30cc1092e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.559+0000 7f30d1058700 1 --2- 172.21.15.84:0/351914537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 0x7f30cc1092e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009e80 tx=0x7f30c400b330 comp rx=0 tx=0).stop 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.559+0000 7f30d1058700 1 -- 172.21.15.84:0/351914537 shutdown_connections 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.559+0000 7f30d1058700 1 --2- 172.21.15.84:0/351914537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 0x7f30cc1092e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.559+0000 7f30d1058700 1 -- 172.21.15.84:0/351914537 >> 172.21.15.84:0/351914537 conn(0x7f30cc104580 msgr2=0x7f30cc1069a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.559+0000 7f30d1058700 1 -- 172.21.15.84:0/351914537 shutdown_connections 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.559+0000 7f30d1058700 1 -- 172.21.15.84:0/351914537 wait complete. 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.560+0000 7f30d1058700 1 Processor -- start 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.560+0000 7f30d1058700 1 -- start start 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.560+0000 7f30d1058700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 0x7f30cc19cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.560+0000 7f30d1058700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc10d410 con 0x7f30cc108ec0 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.560+0000 7f30cad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 0x7f30cc19cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.560+0000 7f30cad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 0x7f30cc19cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41856/0 (socket says 172.21.15.84:41856) 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.560+0000 7f30cad9d700 1 -- 172.21.15.84:0/2187934567 learned_addr learned my addr 172.21.15.84:0/2187934567 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:08.510 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.560+0000 7f30cad9d700 1 -- 172.21.15.84:0/2187934567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc19d2e0 con 0x7f30cc108ec0 2024-04-26T18:17:08.511 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.561+0000 7f30cad9d700 1 --2- 172.21.15.84:0/2187934567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 0x7f30cc19cda0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f30c4004220 tx=0x7f30c400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:08.511 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.561+0000 7f30b3fff700 1 -- 172.21.15.84:0/2187934567 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30c4017040 con 0x7f30cc108ec0 2024-04-26T18:17:08.511 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.561+0000 7f30d1058700 1 -- 172.21.15.84:0/2187934567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30cc19d570 con 0x7f30cc108ec0 2024-04-26T18:17:08.511 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.561+0000 7f30b3fff700 1 -- 172.21.15.84:0/2187934567 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c4022070 con 0x7f30cc108ec0 2024-04-26T18:17:08.511 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.561+0000 7f30b3fff700 1 -- 172.21.15.84:0/2187934567 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f30c4012430 con 0x7f30cc108ec0 2024-04-26T18:17:08.511 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.561+0000 7f30d1058700 1 -- 172.21.15.84:0/2187934567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30cc1a0220 con 0x7f30cc108ec0 2024-04-26T18:17:08.511 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.562+0000 7f30b3fff700 1 -- 172.21.15.84:0/2187934567 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f30c4012730 con 0x7f30cc108ec0 2024-04-26T18:17:08.511 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.562+0000 7f30d1058700 1 -- 172.21.15.84:0/2187934567 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30cc10d590 con 0x7f30cc108ec0 2024-04-26T18:17:08.511 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.562+0000 7f30b3fff700 1 --2- 172.21.15.84:0/2187934567 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f30b4039600 0x7f30b403bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:08.511 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.562+0000 7f30b3fff700 1 -- 172.21.15.84:0/2187934567 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f30c400f050 con 0x7f30cc108ec0 2024-04-26T18:17:08.511 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.562+0000 7f30ca59c700 1 --2- 172.21.15.84:0/2187934567 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f30b4039600 0x7f30b403bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:08.511 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.563+0000 7f30ca59c700 1 --2- 172.21.15.84:0/2187934567 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f30b4039600 0x7f30b403bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30c0006fd0 tx=0x7f30c0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:08.511 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.568+0000 7f30b3fff700 1 -- 172.21.15.84:0/2187934567 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30c402c200 con 0x7f30cc108ec0 2024-04-26T18:17:08.512 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.784+0000 7f30d1058700 1 -- 172.21.15.84:0/2187934567 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f30cc105730 con 0x7f30b4039600 2024-04-26T18:17:08.512 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.904+0000 7f30b3fff700 1 -- 172.21.15.84:0/2187934567 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f30cc105730 con 0x7f30b4039600 2024-04-26T18:17:08.512 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.907+0000 7f30d1058700 1 -- 172.21.15.84:0/2187934567 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f30b4039600 msgr2=0x7f30b403bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:08.512 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.907+0000 7f30d1058700 1 --2- 172.21.15.84:0/2187934567 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f30b4039600 0x7f30b403bac0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f30c0006fd0 tx=0x7f30c0006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:08.512 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.907+0000 7f30d1058700 1 -- 172.21.15.84:0/2187934567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 msgr2=0x7f30cc19cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:08.512 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.907+0000 7f30d1058700 1 --2- 172.21.15.84:0/2187934567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 0x7f30cc19cda0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f30c4004220 tx=0x7f30c400bd30 comp rx=0 tx=0).stop 2024-04-26T18:17:08.512 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.907+0000 7f30d1058700 1 -- 172.21.15.84:0/2187934567 shutdown_connections 2024-04-26T18:17:08.512 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.907+0000 7f30d1058700 1 --2- 172.21.15.84:0/2187934567 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f30b4039600 0x7f30b403bac0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:08.512 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.907+0000 7f30d1058700 1 --2- 172.21.15.84:0/2187934567 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f30cc108ec0 0x7f30cc19cda0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:08.512 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.907+0000 7f30d1058700 1 -- 172.21.15.84:0/2187934567 >> 172.21.15.84:0/2187934567 conn(0x7f30cc104580 msgr2=0x7f30cc105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:08.512 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.907+0000 7f30d1058700 1 -- 172.21.15.84:0/2187934567 shutdown_connections 2024-04-26T18:17:08.512 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:07.907+0000 7f30d1058700 1 -- 172.21.15.84:0/2187934567 wait complete. 2024-04-26T18:17:08.512 INFO:teuthology.orchestra.run.smithi084.stdout:Creating initial admin user... 2024-04-26T18:17:09.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:09 smithi084 ceph-mon[29141]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:09.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:09 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:10.354 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:10 smithi084 ceph-mon[29141]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:10.354 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:10 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:10.361 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$IVgOuyVl9OHPN1cXjwRCre1IRiH1xW7BzD6LcAS3c4HZ8mN0/dp8q", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714155429, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T18:17:10.361 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.942+0000 7f4c07aad700 1 Processor -- start 2024-04-26T18:17:10.361 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.942+0000 7f4c07aad700 1 -- start start 2024-04-26T18:17:10.361 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.942+0000 7f4c07aad700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 0x7f4c0010a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:10.362 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.942+0000 7f4c07aad700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4c0010ade0 con 0x7f4c0010a480 2024-04-26T18:17:10.362 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.942+0000 7f4c05849700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 0x7f4c0010a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:10.362 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.942+0000 7f4c05849700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 0x7f4c0010a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41864/0 (socket says 172.21.15.84:41864) 2024-04-26T18:17:10.362 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.942+0000 7f4c05849700 1 -- 172.21.15.84:0/3290295219 learned_addr learned my addr 172.21.15.84:0/3290295219 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:10.362 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.943+0000 7f4c05849700 1 -- 172.21.15.84:0/3290295219 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0010af60 con 0x7f4c0010a480 2024-04-26T18:17:10.362 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.943+0000 7f4c05849700 1 --2- 172.21.15.84:0/3290295219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 0x7f4c0010a8a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0098a0 tx=0x7f4bfc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d33a62ecef8348aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:10.362 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.943+0000 7f4c04847700 1 -- 172.21.15.84:0/3290295219 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bfc004160 con 0x7f4c0010a480 2024-04-26T18:17:10.362 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.943+0000 7f4c04847700 1 -- 172.21.15.84:0/3290295219 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bfc004300 con 0x7f4c0010a480 2024-04-26T18:17:10.362 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.943+0000 7f4c04847700 1 -- 172.21.15.84:0/3290295219 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bfc011a20 con 0x7f4c0010a480 2024-04-26T18:17:10.362 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.943+0000 7f4c07aad700 1 -- 172.21.15.84:0/3290295219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 msgr2=0x7f4c0010a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:10.362 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.943+0000 7f4c07aad700 1 --2- 172.21.15.84:0/3290295219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 0x7f4c0010a8a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc0098a0 tx=0x7f4bfc009bb0 comp rx=0 tx=0).stop 2024-04-26T18:17:10.362 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.944+0000 7f4c07aad700 1 -- 172.21.15.84:0/3290295219 shutdown_connections 2024-04-26T18:17:10.362 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.944+0000 7f4c07aad700 1 --2- 172.21.15.84:0/3290295219 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 0x7f4c0010a8a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:10.363 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.944+0000 7f4c07aad700 1 -- 172.21.15.84:0/3290295219 >> 172.21.15.84:0/3290295219 conn(0x7f4c00102b40 msgr2=0x7f4c00104f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:10.363 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.944+0000 7f4c07aad700 1 -- 172.21.15.84:0/3290295219 shutdown_connections 2024-04-26T18:17:10.363 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.944+0000 7f4c07aad700 1 -- 172.21.15.84:0/3290295219 wait complete. 2024-04-26T18:17:10.363 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.944+0000 7f4c07aad700 1 Processor -- start 2024-04-26T18:17:10.363 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.944+0000 7f4c07aad700 1 -- start start 2024-04-26T18:17:10.363 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.944+0000 7f4c07aad700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 0x7f4c00109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:10.363 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.944+0000 7f4c07aad700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4c00107d30 con 0x7f4c0010a480 2024-04-26T18:17:10.363 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.945+0000 7f4c05849700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 0x7f4c00109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:10.363 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.945+0000 7f4c05849700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 0x7f4c00109aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41872/0 (socket says 172.21.15.84:41872) 2024-04-26T18:17:10.363 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.945+0000 7f4c05849700 1 -- 172.21.15.84:0/1675112760 learned_addr learned my addr 172.21.15.84:0/1675112760 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:10.363 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.945+0000 7f4c05849700 1 -- 172.21.15.84:0/1675112760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c00109fe0 con 0x7f4c0010a480 2024-04-26T18:17:10.363 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.945+0000 7f4c05849700 1 --2- 172.21.15.84:0/1675112760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 0x7f4c00109aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc004660 tx=0x7f4bfc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:10.363 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.945+0000 7f4bf6ffd700 1 -- 172.21.15.84:0/1675112760 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bfc003aa0 con 0x7f4c0010a480 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.945+0000 7f4bf6ffd700 1 -- 172.21.15.84:0/1675112760 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bfc01b8f0 con 0x7f4c0010a480 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.945+0000 7f4bf6ffd700 1 -- 172.21.15.84:0/1675112760 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4bfc023900 con 0x7f4c0010a480 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.945+0000 7f4c07aad700 1 -- 172.21.15.84:0/1675112760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c001081d0 con 0x7f4c0010a480 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.945+0000 7f4c07aad700 1 -- 172.21.15.84:0/1675112760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c00108710 con 0x7f4c0010a480 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.946+0000 7f4bf6ffd700 1 -- 172.21.15.84:0/1675112760 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f4bfc023b70 con 0x7f4c0010a480 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.946+0000 7f4c07aad700 1 -- 172.21.15.84:0/1675112760 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c00064a80 con 0x7f4c0010a480 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.946+0000 7f4bf6ffd700 1 --2- 172.21.15.84:0/1675112760 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4bec039660 0x7f4bec03bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.946+0000 7f4bf6ffd700 1 -- 172.21.15.84:0/1675112760 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4bfc04e000 con 0x7f4c0010a480 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.947+0000 7f4c05048700 1 --2- 172.21.15.84:0/1675112760 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4bec039660 0x7f4bec03bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.947+0000 7f4c05048700 1 --2- 172.21.15.84:0/1675112760 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4bec039660 0x7f4bec03bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0006fd0 tx=0x7f4bf0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:08.952+0000 7f4bf6ffd700 1 -- 172.21.15.84:0/1675112760 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4bfc01f030 con 0x7f4c0010a480 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:09.186+0000 7f4c07aad700 1 -- 172.21.15.84:0/1675112760 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f4c00108eb0 con 0x7f4bec039660 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:09.410+0000 7f4bf6ffd700 1 -- 172.21.15.84:0/1675112760 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f4c00108eb0 con 0x7f4bec039660 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:09.412+0000 7f4c07aad700 1 -- 172.21.15.84:0/1675112760 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4bec039660 msgr2=0x7f4bec03bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:10.364 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:09.412+0000 7f4c07aad700 1 --2- 172.21.15.84:0/1675112760 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4bec039660 0x7f4bec03bb20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0006fd0 tx=0x7f4bf0006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:10.365 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:09.412+0000 7f4c07aad700 1 -- 172.21.15.84:0/1675112760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 msgr2=0x7f4c00109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:10.365 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:09.412+0000 7f4c07aad700 1 --2- 172.21.15.84:0/1675112760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 0x7f4c00109aa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bfc004660 tx=0x7f4bfc011aa0 comp rx=0 tx=0).stop 2024-04-26T18:17:10.365 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:09.412+0000 7f4c07aad700 1 -- 172.21.15.84:0/1675112760 shutdown_connections 2024-04-26T18:17:10.365 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:09.412+0000 7f4c07aad700 1 --2- 172.21.15.84:0/1675112760 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4bec039660 0x7f4bec03bb20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:10.365 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:09.412+0000 7f4c07aad700 1 --2- 172.21.15.84:0/1675112760 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c0010a480 0x7f4c00109aa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:10.365 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:09.412+0000 7f4c07aad700 1 -- 172.21.15.84:0/1675112760 >> 172.21.15.84:0/1675112760 conn(0x7f4c00102b40 msgr2=0x7f4c001036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:10.365 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:09.412+0000 7f4c07aad700 1 -- 172.21.15.84:0/1675112760 shutdown_connections 2024-04-26T18:17:10.365 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:09.412+0000 7f4c07aad700 1 -- 172.21.15.84:0/1675112760 wait complete. 2024-04-26T18:17:10.365 INFO:teuthology.orchestra.run.smithi084.stdout:Fetching dashboard port number... 2024-04-26T18:17:11.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:11 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/76197046' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T18:17:11.780 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T18:17:11.780 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.782+0000 7fd2eaf71700 1 Processor -- start 2024-04-26T18:17:11.780 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.782+0000 7fd2eaf71700 1 -- start start 2024-04-26T18:17:11.780 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.782+0000 7fd2eaf71700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 0x7fd2e41092c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:11.780 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.782+0000 7fd2eaf71700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e4109800 con 0x7fd2e4108ea0 2024-04-26T18:17:11.780 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.782+0000 7fd2e8d0d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 0x7fd2e41092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:11.780 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.782+0000 7fd2e8d0d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 0x7fd2e41092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41886/0 (socket says 172.21.15.84:41886) 2024-04-26T18:17:11.780 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.782+0000 7fd2e8d0d700 1 -- 172.21.15.84:0/4033425123 learned_addr learned my addr 172.21.15.84:0/4033425123 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.782+0000 7fd2e8d0d700 1 -- 172.21.15.84:0/4033425123 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e410a040 con 0x7fd2e4108ea0 2024-04-26T18:17:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.783+0000 7fd2e8d0d700 1 --2- 172.21.15.84:0/4033425123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 0x7fd2e41092c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80098a0 tx=0x7fd2d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe43ae41a1fc811 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.783+0000 7fd2e37fe700 1 -- 172.21.15.84:0/4033425123 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2d8004160 con 0x7fd2e4108ea0 2024-04-26T18:17:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.783+0000 7fd2e37fe700 1 -- 172.21.15.84:0/4033425123 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2d8004300 con 0x7fd2e4108ea0 2024-04-26T18:17:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.783+0000 7fd2e37fe700 1 -- 172.21.15.84:0/4033425123 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2d8011a20 con 0x7fd2e4108ea0 2024-04-26T18:17:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.783+0000 7fd2eaf71700 1 -- 172.21.15.84:0/4033425123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 msgr2=0x7fd2e41092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.783+0000 7fd2eaf71700 1 --2- 172.21.15.84:0/4033425123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 0x7fd2e41092c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80098a0 tx=0x7fd2d8009bb0 comp rx=0 tx=0).stop 2024-04-26T18:17:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.783+0000 7fd2eaf71700 1 -- 172.21.15.84:0/4033425123 shutdown_connections 2024-04-26T18:17:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.783+0000 7fd2eaf71700 1 --2- 172.21.15.84:0/4033425123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 0x7fd2e41092c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.783+0000 7fd2eaf71700 1 -- 172.21.15.84:0/4033425123 >> 172.21.15.84:0/4033425123 conn(0x7fd2e4104470 msgr2=0x7fd2e41068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:11.781 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.783+0000 7fd2eaf71700 1 -- 172.21.15.84:0/4033425123 shutdown_connections 2024-04-26T18:17:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.783+0000 7fd2eaf71700 1 -- 172.21.15.84:0/4033425123 wait complete. 2024-04-26T18:17:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.784+0000 7fd2eaf71700 1 Processor -- start 2024-04-26T18:17:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.784+0000 7fd2eaf71700 1 -- start start 2024-04-26T18:17:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.784+0000 7fd2eaf71700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 0x7fd2e419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.784+0000 7fd2eaf71700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd2e410d300 con 0x7fd2e4108ea0 2024-04-26T18:17:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.784+0000 7fd2e8d0d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 0x7fd2e419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.784+0000 7fd2e8d0d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 0x7fd2e419cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41892/0 (socket says 172.21.15.84:41892) 2024-04-26T18:17:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.784+0000 7fd2e8d0d700 1 -- 172.21.15.84:0/76197046 learned_addr learned my addr 172.21.15.84:0/76197046 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.785+0000 7fd2e8d0d700 1 -- 172.21.15.84:0/76197046 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2e419d300 con 0x7fd2e4108ea0 2024-04-26T18:17:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.785+0000 7fd2e8d0d700 1 --2- 172.21.15.84:0/76197046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 0x7fd2e419cdc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80116e0 tx=0x7fd2d8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.785+0000 7fd2e1ffb700 1 -- 172.21.15.84:0/76197046 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2d80039e0 con 0x7fd2e4108ea0 2024-04-26T18:17:11.782 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.785+0000 7fd2e1ffb700 1 -- 172.21.15.84:0/76197046 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2d801b8f0 con 0x7fd2e4108ea0 2024-04-26T18:17:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.785+0000 7fd2eaf71700 1 -- 172.21.15.84:0/76197046 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2e419d590 con 0x7fd2e4108ea0 2024-04-26T18:17:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.785+0000 7fd2e1ffb700 1 -- 172.21.15.84:0/76197046 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd2d8023760 con 0x7fd2e4108ea0 2024-04-26T18:17:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.785+0000 7fd2eaf71700 1 -- 172.21.15.84:0/76197046 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2e41a0280 con 0x7fd2e4108ea0 2024-04-26T18:17:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.786+0000 7fd2e1ffb700 1 -- 172.21.15.84:0/76197046 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fd2d80239d0 con 0x7fd2e4108ea0 2024-04-26T18:17:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.786+0000 7fd2eaf71700 1 -- 172.21.15.84:0/76197046 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2e4064a80 con 0x7fd2e4108ea0 2024-04-26T18:17:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.786+0000 7fd2e1ffb700 1 --2- 172.21.15.84:0/76197046 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd2d40395b0 0x7fd2d403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.786+0000 7fd2e1ffb700 1 -- 172.21.15.84:0/76197046 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd2d804e110 con 0x7fd2e4108ea0 2024-04-26T18:17:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.786+0000 7fd2e3fff700 1 --2- 172.21.15.84:0/76197046 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd2d40395b0 0x7fd2d403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.787+0000 7fd2e3fff700 1 --2- 172.21.15.84:0/76197046 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd2d40395b0 0x7fd2d403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0006fd0 tx=0x7fd2d0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.791+0000 7fd2e1ffb700 1 -- 172.21.15.84:0/76197046 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd2d801f030 con 0x7fd2e4108ea0 2024-04-26T18:17:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.990+0000 7fd2eaf71700 1 -- 172.21.15.84:0/76197046 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fd2e41a0830 con 0x7fd2e4108ea0 2024-04-26T18:17:11.783 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.991+0000 7fd2e1ffb700 1 -- 172.21.15.84:0/76197046 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fd2d8023dc0 con 0x7fd2e4108ea0 2024-04-26T18:17:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.993+0000 7fd2eaf71700 1 -- 172.21.15.84:0/76197046 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd2d40395b0 msgr2=0x7fd2d403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.993+0000 7fd2eaf71700 1 --2- 172.21.15.84:0/76197046 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd2d40395b0 0x7fd2d403ba70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd2d0006fd0 tx=0x7fd2d0006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.993+0000 7fd2eaf71700 1 -- 172.21.15.84:0/76197046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 msgr2=0x7fd2e419cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.993+0000 7fd2eaf71700 1 --2- 172.21.15.84:0/76197046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 0x7fd2e419cdc0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd2d80116e0 tx=0x7fd2d8011aa0 comp rx=0 tx=0).stop 2024-04-26T18:17:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.993+0000 7fd2eaf71700 1 -- 172.21.15.84:0/76197046 shutdown_connections 2024-04-26T18:17:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.993+0000 7fd2eaf71700 1 --2- 172.21.15.84:0/76197046 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd2d40395b0 0x7fd2d403ba70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.993+0000 7fd2eaf71700 1 --2- 172.21.15.84:0/76197046 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2e4108ea0 0x7fd2e419cdc0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.993+0000 7fd2eaf71700 1 -- 172.21.15.84:0/76197046 >> 172.21.15.84:0/76197046 conn(0x7fd2e4104470 msgr2=0x7fd2e4104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.993+0000 7fd2eaf71700 1 -- 172.21.15.84:0/76197046 shutdown_connections 2024-04-26T18:17:11.784 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:10.993+0000 7fd2eaf71700 1 -- 172.21.15.84:0/76197046 wait complete. 2024-04-26T18:17:11.786 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T18:17:11.786 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout disabled 2024-04-26T18:17:11.793 INFO:teuthology.orchestra.run.smithi084.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T18:17:11.793 INFO:teuthology.orchestra.run.smithi084.stdout:systemctl: stdout inactive 2024-04-26T18:17:11.793 INFO:teuthology.orchestra.run.smithi084.stdout:firewalld.service is not enabled 2024-04-26T18:17:11.793 INFO:teuthology.orchestra.run.smithi084.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T18:17:11.795 INFO:teuthology.orchestra.run.smithi084.stdout:Ceph Dashboard is now available at: 2024-04-26T18:17:11.795 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:17:11.795 INFO:teuthology.orchestra.run.smithi084.stdout: URL: https://smithi084:8443/ 2024-04-26T18:17:11.795 INFO:teuthology.orchestra.run.smithi084.stdout: User: admin 2024-04-26T18:17:11.795 INFO:teuthology.orchestra.run.smithi084.stdout: Password: 5mguj17zb1 2024-04-26T18:17:11.795 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:17:11.796 INFO:teuthology.orchestra.run.smithi084.stdout:Saving cluster configuration to /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config directory 2024-04-26T18:17:11.796 INFO:teuthology.orchestra.run.smithi084.stdout:Enabling autotune for osd_memory_target 2024-04-26T18:17:12.408 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:12 smithi084 ceph-mon[29141]: mgrmap e13: x(active, since 6s) 2024-04-26T18:17:13.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.201+0000 7fac024db700 1 Processor -- start 2024-04-26T18:17:13.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.201+0000 7fac024db700 1 -- start start 2024-04-26T18:17:13.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.201+0000 7fac024db700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 0x7fabfc1081e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:13.010 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.201+0000 7fac024db700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc1087b0 con 0x7fabfc107dc0 2024-04-26T18:17:13.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.202+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 0x7fabfc1081e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:13.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.202+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 0x7fabfc1081e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41896/0 (socket says 172.21.15.84:41896) 2024-04-26T18:17:13.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.202+0000 7fabfbfff700 1 -- 172.21.15.84:0/747953681 learned_addr learned my addr 172.21.15.84:0/747953681 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:13.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.202+0000 7fabfbfff700 1 -- 172.21.15.84:0/747953681 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabfc108f70 con 0x7fabfc107dc0 2024-04-26T18:17:13.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.202+0000 7fabfbfff700 1 --2- 172.21.15.84:0/747953681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 0x7fabfc1081e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fabec00bb40 tx=0x7fabec00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=828632e566d8369f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:13.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.202+0000 7fabfaffd700 1 -- 172.21.15.84:0/747953681 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabec00dbe0 con 0x7fabfc107dc0 2024-04-26T18:17:13.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.202+0000 7fabfaffd700 1 -- 172.21.15.84:0/747953681 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabec0044e0 con 0x7fabfc107dc0 2024-04-26T18:17:13.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.202+0000 7fabfaffd700 1 -- 172.21.15.84:0/747953681 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabec010470 con 0x7fabfc107dc0 2024-04-26T18:17:13.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.203+0000 7fac024db700 1 -- 172.21.15.84:0/747953681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 msgr2=0x7fabfc1081e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:13.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.203+0000 7fac024db700 1 --2- 172.21.15.84:0/747953681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 0x7fabfc1081e0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fabec00bb40 tx=0x7fabec00be50 comp rx=0 tx=0).stop 2024-04-26T18:17:13.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.203+0000 7fac024db700 1 -- 172.21.15.84:0/747953681 shutdown_connections 2024-04-26T18:17:13.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.203+0000 7fac024db700 1 --2- 172.21.15.84:0/747953681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 0x7fabfc1081e0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:13.011 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.203+0000 7fac024db700 1 -- 172.21.15.84:0/747953681 >> 172.21.15.84:0/747953681 conn(0x7fabfc103380 msgr2=0x7fabfc1057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:13.012 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.203+0000 7fac024db700 1 -- 172.21.15.84:0/747953681 shutdown_connections 2024-04-26T18:17:13.012 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.203+0000 7fac024db700 1 -- 172.21.15.84:0/747953681 wait complete. 2024-04-26T18:17:13.012 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.203+0000 7fac024db700 1 Processor -- start 2024-04-26T18:17:13.012 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.203+0000 7fac024db700 1 -- start start 2024-04-26T18:17:13.012 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.204+0000 7fac024db700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 0x7fabfc19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:13.012 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.204+0000 7fac024db700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fabfc07c810 con 0x7fabfc107dc0 2024-04-26T18:17:13.012 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.204+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 0x7fabfc19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:13.012 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.204+0000 7fabfbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 0x7fabfc19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41906/0 (socket says 172.21.15.84:41906) 2024-04-26T18:17:13.012 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.204+0000 7fabfbfff700 1 -- 172.21.15.84:0/732161991 learned_addr learned my addr 172.21.15.84:0/732161991 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:13.013 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.204+0000 7fabfbfff700 1 -- 172.21.15.84:0/732161991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabfc19b120 con 0x7fabfc107dc0 2024-04-26T18:17:13.013 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.204+0000 7fabfbfff700 1 --2- 172.21.15.84:0/732161991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 0x7fabfc19abe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabec00d650 tx=0x7fabec003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:13.013 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.204+0000 7fabf97fa700 1 -- 172.21.15.84:0/732161991 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabec027070 con 0x7fabfc107dc0 2024-04-26T18:17:13.013 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.204+0000 7fabf97fa700 1 -- 172.21.15.84:0/732161991 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabec010c20 con 0x7fabfc107dc0 2024-04-26T18:17:13.013 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.204+0000 7fac024db700 1 -- 172.21.15.84:0/732161991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabfc19b3b0 con 0x7fabfc107dc0 2024-04-26T18:17:13.013 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.204+0000 7fabf97fa700 1 -- 172.21.15.84:0/732161991 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fabec022b10 con 0x7fabfc107dc0 2024-04-26T18:17:13.013 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.204+0000 7fac024db700 1 -- 172.21.15.84:0/732161991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabfc19e0a0 con 0x7fabfc107dc0 2024-04-26T18:17:13.013 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.205+0000 7fabf97fa700 1 -- 172.21.15.84:0/732161991 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fabec02a070 con 0x7fabfc107dc0 2024-04-26T18:17:13.013 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.205+0000 7fabf97fa700 1 --2- 172.21.15.84:0/732161991 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fabe40396b0 0x7fabe403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:13.013 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.205+0000 7fabf97fa700 1 -- 172.21.15.84:0/732161991 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fabec01f070 con 0x7fabfc107dc0 2024-04-26T18:17:13.013 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.205+0000 7fac024db700 1 -- 172.21.15.84:0/732161991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabfc064a80 con 0x7fabfc107dc0 2024-04-26T18:17:13.013 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.206+0000 7fabfb7fe700 1 --2- 172.21.15.84:0/732161991 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fabe40396b0 0x7fabe403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:13.014 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.206+0000 7fabfb7fe700 1 --2- 172.21.15.84:0/732161991 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fabe40396b0 0x7fabe403bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fabf0006fd0 tx=0x7fabf0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:13.014 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.211+0000 7fabf97fa700 1 -- 172.21.15.84:0/732161991 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fabec015070 con 0x7fabfc107dc0 2024-04-26T18:17:13.014 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.408+0000 7fac024db700 1 -- 172.21.15.84:0/732161991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fabfc02e6c0 con 0x7fabfc107dc0 2024-04-26T18:17:13.014 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.408+0000 7fabf97fa700 1 -- 172.21.15.84:0/732161991 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fabec004490 con 0x7fabfc107dc0 2024-04-26T18:17:13.014 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.410+0000 7fac024db700 1 -- 172.21.15.84:0/732161991 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fabe40396b0 msgr2=0x7fabe403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:13.014 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.410+0000 7fac024db700 1 --2- 172.21.15.84:0/732161991 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fabe40396b0 0x7fabe403bb70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fabf0006fd0 tx=0x7fabf0006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:13.014 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.410+0000 7fac024db700 1 -- 172.21.15.84:0/732161991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 msgr2=0x7fabfc19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:13.014 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.410+0000 7fac024db700 1 --2- 172.21.15.84:0/732161991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 0x7fabfc19abe0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fabec00d650 tx=0x7fabec003ff0 comp rx=0 tx=0).stop 2024-04-26T18:17:13.014 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.410+0000 7fac024db700 1 -- 172.21.15.84:0/732161991 shutdown_connections 2024-04-26T18:17:13.014 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.410+0000 7fac024db700 1 --2- 172.21.15.84:0/732161991 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fabe40396b0 0x7fabe403bb70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:13.014 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.410+0000 7fac024db700 1 --2- 172.21.15.84:0/732161991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabfc107dc0 0x7fabfc19abe0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:13.014 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.410+0000 7fac024db700 1 -- 172.21.15.84:0/732161991 >> 172.21.15.84:0/732161991 conn(0x7fabfc103380 msgr2=0x7fabfc191740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:13.014 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.411+0000 7fac024db700 1 -- 172.21.15.84:0/732161991 shutdown_connections 2024-04-26T18:17:13.015 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:12.411+0000 7fac024db700 1 -- 172.21.15.84:0/732161991 wait complete. 2024-04-26T18:17:14.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.425+0000 7f3a39af6700 1 Processor -- start 2024-04-26T18:17:14.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.425+0000 7f3a39af6700 1 -- start start 2024-04-26T18:17:14.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.425+0000 7f3a39af6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 0x7f3a3410bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:14.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.425+0000 7f3a39af6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3407d680 con 0x7f3a34109950 2024-04-26T18:17:14.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.425+0000 7f3a337fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 0x7f3a3410bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:14.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.425+0000 7f3a337fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 0x7f3a3410bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41920/0 (socket says 172.21.15.84:41920) 2024-04-26T18:17:14.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.425+0000 7f3a337fe700 1 -- 172.21.15.84:0/171633920 learned_addr learned my addr 172.21.15.84:0/171633920 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:14.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.425+0000 7f3a337fe700 1 -- 172.21.15.84:0/171633920 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a3410c280 con 0x7f3a34109950 2024-04-26T18:17:14.334 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.425+0000 7f3a337fe700 1 --2- 172.21.15.84:0/171633920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 0x7f3a3410bd40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3a240098a0 tx=0x7f3a24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a3fdd7644daf13 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.425+0000 7f3a327fc700 1 -- 172.21.15.84:0/171633920 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a24004160 con 0x7f3a34109950 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.425+0000 7f3a327fc700 1 -- 172.21.15.84:0/171633920 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a24004300 con 0x7f3a34109950 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.426+0000 7f3a327fc700 1 -- 172.21.15.84:0/171633920 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a24011a20 con 0x7f3a34109950 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.426+0000 7f3a39af6700 1 -- 172.21.15.84:0/171633920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 msgr2=0x7f3a3410bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.426+0000 7f3a39af6700 1 --2- 172.21.15.84:0/171633920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 0x7f3a3410bd40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3a240098a0 tx=0x7f3a24009bb0 comp rx=0 tx=0).stop 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.426+0000 7f3a39af6700 1 -- 172.21.15.84:0/171633920 shutdown_connections 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.426+0000 7f3a39af6700 1 --2- 172.21.15.84:0/171633920 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 0x7f3a3410bd40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.426+0000 7f3a39af6700 1 -- 172.21.15.84:0/171633920 >> 172.21.15.84:0/171633920 conn(0x7f3a34103380 msgr2=0x7f3a341057a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.426+0000 7f3a39af6700 1 -- 172.21.15.84:0/171633920 shutdown_connections 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.426+0000 7f3a39af6700 1 -- 172.21.15.84:0/171633920 wait complete. 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.427+0000 7f3a39af6700 1 Processor -- start 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.427+0000 7f3a39af6700 1 -- start start 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.427+0000 7f3a39af6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 0x7f3a341091d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.427+0000 7f3a39af6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3a3410d220 con 0x7f3a34109950 2024-04-26T18:17:14.335 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.427+0000 7f3a337fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 0x7f3a341091d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.427+0000 7f3a337fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 0x7f3a341091d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41930/0 (socket says 172.21.15.84:41930) 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.427+0000 7f3a337fe700 1 -- 172.21.15.84:0/1906030522 learned_addr learned my addr 172.21.15.84:0/1906030522 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.427+0000 7f3a337fe700 1 -- 172.21.15.84:0/1906030522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a34109710 con 0x7f3a34109950 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.428+0000 7f3a337fe700 1 --2- 172.21.15.84:0/1906030522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 0x7f3a341091d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a24000c00 tx=0x7f3a24011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.428+0000 7f3a30ff9700 1 -- 172.21.15.84:0/1906030522 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a24003aa0 con 0x7f3a34109950 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.428+0000 7f3a30ff9700 1 -- 172.21.15.84:0/1906030522 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a2401b8f0 con 0x7f3a34109950 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.428+0000 7f3a39af6700 1 -- 172.21.15.84:0/1906030522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a34107900 con 0x7f3a34109950 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.428+0000 7f3a30ff9700 1 -- 172.21.15.84:0/1906030522 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a24023870 con 0x7f3a34109950 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.428+0000 7f3a39af6700 1 -- 172.21.15.84:0/1906030522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a34107e40 con 0x7f3a34109950 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.429+0000 7f3a30ff9700 1 -- 172.21.15.84:0/1906030522 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3a24023b70 con 0x7f3a34109950 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.429+0000 7f3a39af6700 1 -- 172.21.15.84:0/1906030522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a34064a80 con 0x7f3a34109950 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.429+0000 7f3a30ff9700 1 --2- 172.21.15.84:0/1906030522 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3a1c0396a0 0x7f3a1c03bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.429+0000 7f3a30ff9700 1 -- 172.21.15.84:0/1906030522 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3a2404e0b0 con 0x7f3a34109950 2024-04-26T18:17:14.336 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.429+0000 7f3a32ffd700 1 --2- 172.21.15.84:0/1906030522 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3a1c0396a0 0x7f3a1c03bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.429+0000 7f3a32ffd700 1 --2- 172.21.15.84:0/1906030522 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3a1c0396a0 0x7f3a1c03bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3a28006fd0 tx=0x7f3a28006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.434+0000 7f3a30ff9700 1 -- 172.21.15.84:0/1906030522 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3a2401f030 con 0x7f3a34109950 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.741+0000 7f3a39af6700 1 -- 172.21.15.84:0/1906030522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f3a341080c0 con 0x7f3a34109950 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.743+0000 7f3a30ff9700 1 -- 172.21.15.84:0/1906030522 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f3a24018c10 con 0x7f3a34109950 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.745+0000 7f3a39af6700 1 -- 172.21.15.84:0/1906030522 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3a1c0396a0 msgr2=0x7f3a1c03bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.745+0000 7f3a39af6700 1 --2- 172.21.15.84:0/1906030522 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3a1c0396a0 0x7f3a1c03bb60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3a28006fd0 tx=0x7f3a28006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.745+0000 7f3a39af6700 1 -- 172.21.15.84:0/1906030522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 msgr2=0x7f3a341091d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.745+0000 7f3a39af6700 1 --2- 172.21.15.84:0/1906030522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 0x7f3a341091d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a24000c00 tx=0x7f3a24011aa0 comp rx=0 tx=0).stop 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.746+0000 7f3a39af6700 1 -- 172.21.15.84:0/1906030522 shutdown_connections 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.746+0000 7f3a39af6700 1 --2- 172.21.15.84:0/1906030522 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3a1c0396a0 0x7f3a1c03bb60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.746+0000 7f3a39af6700 1 --2- 172.21.15.84:0/1906030522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a34109950 0x7f3a341091d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.746+0000 7f3a39af6700 1 -- 172.21.15.84:0/1906030522 >> 172.21.15.84:0/1906030522 conn(0x7f3a34103380 msgr2=0x7f3a34103e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.746+0000 7f3a39af6700 1 -- 172.21.15.84:0/1906030522 shutdown_connections 2024-04-26T18:17:14.337 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr 2024-04-26T18:17:13.746+0000 7f3a39af6700 1 -- 172.21.15.84:0/1906030522 wait complete. 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout: ceph telemetry on 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout:For more information see: 2024-04-26T18:17:14.338 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:17:14.339 INFO:teuthology.orchestra.run.smithi084.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T18:17:14.339 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:17:14.339 INFO:teuthology.orchestra.run.smithi084.stdout:Bootstrap complete. 2024-04-26T18:17:14.372 INFO:tasks.cephadm:Fetching config... 2024-04-26T18:17:14.372 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:17:14.372 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T18:17:14.388 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T18:17:14.388 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:17:14.389 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T18:17:14.443 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T18:17:14.444 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:17:14.444 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-26T18:17:14.507 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T18:17:14.507 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:17:14.507 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T18:17:14.563 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T18:17:14.563 DEBUG:teuthology.orchestra.run.smithi084:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3TgI8lfZoWvnZDqhzWIFbexROjohsqW+yAPxfeER5LLpaC7M+xPqszQcVU6HRwSlRZz3nfSYmf7Y5IwZSX05Klgc2pDFjsX4aDsEURGr9lsu+zXN+mJDZxN+UCuKOJFDB3A3y2YNH7mL/w0t89ZneruoraDfNI5O2cbMZpkJoETXZzuvOPWTQ1wrhu+s/2mclLdBLq5E3FIQNi2qdk7Bu1pTGWI+u6z5aNJyHrlV3XJ+qZ5jHs68IrskUzxxR7qEfOXvBZ+QQyBEVutLvewz300+Y9+HFGUGVndVTAE3iZgCLywFmnec2jnZsUGlrvSLfvWg6ejjQPVU9no+0l5Iex8/Zz8zJXPzdHY7sacV0KQlcVaf0xeeKsGwkPphLvXPkg+azSTFTR8SqlefmMnmOvoTqK5f3rhdbkkGT8DVLLOFv73PyiJhre0bWGdPXGEa1MBRSesAtzJy1XPo9kg08kl/5vJfx1J1rXluPH3TfdVpKGOWmsC7tLHJsTUd6k8U= ceph-df3f3816-03f8-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T18:17:14.635 INFO:teuthology.orchestra.run.smithi084.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3TgI8lfZoWvnZDqhzWIFbexROjohsqW+yAPxfeER5LLpaC7M+xPqszQcVU6HRwSlRZz3nfSYmf7Y5IwZSX05Klgc2pDFjsX4aDsEURGr9lsu+zXN+mJDZxN+UCuKOJFDB3A3y2YNH7mL/w0t89ZneruoraDfNI5O2cbMZpkJoETXZzuvOPWTQ1wrhu+s/2mclLdBLq5E3FIQNi2qdk7Bu1pTGWI+u6z5aNJyHrlV3XJ+qZ5jHs68IrskUzxxR7qEfOXvBZ+QQyBEVutLvewz300+Y9+HFGUGVndVTAE3iZgCLywFmnec2jnZsUGlrvSLfvWg6ejjQPVU9no+0l5Iex8/Zz8zJXPzdHY7sacV0KQlcVaf0xeeKsGwkPphLvXPkg+azSTFTR8SqlefmMnmOvoTqK5f3rhdbkkGT8DVLLOFv73PyiJhre0bWGdPXGEa1MBRSesAtzJy1XPo9kg08kl/5vJfx1J1rXluPH3TfdVpKGOWmsC7tLHJsTUd6k8U= ceph-df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:17:14.646 DEBUG:teuthology.orchestra.run.smithi103:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3TgI8lfZoWvnZDqhzWIFbexROjohsqW+yAPxfeER5LLpaC7M+xPqszQcVU6HRwSlRZz3nfSYmf7Y5IwZSX05Klgc2pDFjsX4aDsEURGr9lsu+zXN+mJDZxN+UCuKOJFDB3A3y2YNH7mL/w0t89ZneruoraDfNI5O2cbMZpkJoETXZzuvOPWTQ1wrhu+s/2mclLdBLq5E3FIQNi2qdk7Bu1pTGWI+u6z5aNJyHrlV3XJ+qZ5jHs68IrskUzxxR7qEfOXvBZ+QQyBEVutLvewz300+Y9+HFGUGVndVTAE3iZgCLywFmnec2jnZsUGlrvSLfvWg6ejjQPVU9no+0l5Iex8/Zz8zJXPzdHY7sacV0KQlcVaf0xeeKsGwkPphLvXPkg+azSTFTR8SqlefmMnmOvoTqK5f3rhdbkkGT8DVLLOFv73PyiJhre0bWGdPXGEa1MBRSesAtzJy1XPo9kg08kl/5vJfx1J1rXluPH3TfdVpKGOWmsC7tLHJsTUd6k8U= ceph-df3f3816-03f8-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T18:17:14.685 INFO:teuthology.orchestra.run.smithi103.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3TgI8lfZoWvnZDqhzWIFbexROjohsqW+yAPxfeER5LLpaC7M+xPqszQcVU6HRwSlRZz3nfSYmf7Y5IwZSX05Klgc2pDFjsX4aDsEURGr9lsu+zXN+mJDZxN+UCuKOJFDB3A3y2YNH7mL/w0t89ZneruoraDfNI5O2cbMZpkJoETXZzuvOPWTQ1wrhu+s/2mclLdBLq5E3FIQNi2qdk7Bu1pTGWI+u6z5aNJyHrlV3XJ+qZ5jHs68IrskUzxxR7qEfOXvBZ+QQyBEVutLvewz300+Y9+HFGUGVndVTAE3iZgCLywFmnec2jnZsUGlrvSLfvWg6ejjQPVU9no+0l5Iex8/Zz8zJXPzdHY7sacV0KQlcVaf0xeeKsGwkPphLvXPkg+azSTFTR8SqlefmMnmOvoTqK5f3rhdbkkGT8DVLLOFv73PyiJhre0bWGdPXGEa1MBRSesAtzJy1XPo9kg08kl/5vJfx1J1rXluPH3TfdVpKGOWmsC7tLHJsTUd6k8U= ceph-df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:17:14.697 DEBUG:teuthology.orchestra.run.smithi174:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3TgI8lfZoWvnZDqhzWIFbexROjohsqW+yAPxfeER5LLpaC7M+xPqszQcVU6HRwSlRZz3nfSYmf7Y5IwZSX05Klgc2pDFjsX4aDsEURGr9lsu+zXN+mJDZxN+UCuKOJFDB3A3y2YNH7mL/w0t89ZneruoraDfNI5O2cbMZpkJoETXZzuvOPWTQ1wrhu+s/2mclLdBLq5E3FIQNi2qdk7Bu1pTGWI+u6z5aNJyHrlV3XJ+qZ5jHs68IrskUzxxR7qEfOXvBZ+QQyBEVutLvewz300+Y9+HFGUGVndVTAE3iZgCLywFmnec2jnZsUGlrvSLfvWg6ejjQPVU9no+0l5Iex8/Zz8zJXPzdHY7sacV0KQlcVaf0xeeKsGwkPphLvXPkg+azSTFTR8SqlefmMnmOvoTqK5f3rhdbkkGT8DVLLOFv73PyiJhre0bWGdPXGEa1MBRSesAtzJy1XPo9kg08kl/5vJfx1J1rXluPH3TfdVpKGOWmsC7tLHJsTUd6k8U= ceph-df3f3816-03f8-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T18:17:14.736 INFO:teuthology.orchestra.run.smithi174.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC3TgI8lfZoWvnZDqhzWIFbexROjohsqW+yAPxfeER5LLpaC7M+xPqszQcVU6HRwSlRZz3nfSYmf7Y5IwZSX05Klgc2pDFjsX4aDsEURGr9lsu+zXN+mJDZxN+UCuKOJFDB3A3y2YNH7mL/w0t89ZneruoraDfNI5O2cbMZpkJoETXZzuvOPWTQ1wrhu+s/2mclLdBLq5E3FIQNi2qdk7Bu1pTGWI+u6z5aNJyHrlV3XJ+qZ5jHs68IrskUzxxR7qEfOXvBZ+QQyBEVutLvewz300+Y9+HFGUGVndVTAE3iZgCLywFmnec2jnZsUGlrvSLfvWg6ejjQPVU9no+0l5Iex8/Zz8zJXPzdHY7sacV0KQlcVaf0xeeKsGwkPphLvXPkg+azSTFTR8SqlefmMnmOvoTqK5f3rhdbkkGT8DVLLOFv73PyiJhre0bWGdPXGEa1MBRSesAtzJy1XPo9kg08kl/5vJfx1J1rXluPH3TfdVpKGOWmsC7tLHJsTUd6k8U= ceph-df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:17:14.748 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T18:17:14.993 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:17:15.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:14 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1906030522' entity='client.admin' 2024-04-26T18:17:17.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.325+0000 7fd401984700 1 -- 172.21.15.84:0/374021406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3fc101170 msgr2=0x7fd3fc101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:17.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.325+0000 7fd401984700 1 --2- 172.21.15.84:0/374021406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3fc101170 0x7fd3fc101590 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec009910 tx=0x7fd3ec009c20 comp rx=0 tx=0).stop 2024-04-26T18:17:17.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.325+0000 7fd401984700 1 -- 172.21.15.84:0/374021406 shutdown_connections 2024-04-26T18:17:17.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.325+0000 7fd401984700 1 --2- 172.21.15.84:0/374021406 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3fc101170 0x7fd3fc101590 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:17.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.325+0000 7fd401984700 1 -- 172.21.15.84:0/374021406 >> 172.21.15.84:0/374021406 conn(0x7fd3fc0fcaf0 msgr2=0x7fd3fc0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:17.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.325+0000 7fd401984700 1 -- 172.21.15.84:0/374021406 shutdown_connections 2024-04-26T18:17:17.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.325+0000 7fd401984700 1 -- 172.21.15.84:0/374021406 wait complete. 2024-04-26T18:17:17.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.326+0000 7fd401984700 1 Processor -- start 2024-04-26T18:17:17.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.326+0000 7fd401984700 1 -- start start 2024-04-26T18:17:17.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.326+0000 7fd401984700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3fc101170 0x7fd3fc1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:17.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.326+0000 7fd401984700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd3fc108df0 con 0x7fd3fc101170 2024-04-26T18:17:17.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.327+0000 7fd3faffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3fc101170 0x7fd3fc1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:17.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.327+0000 7fd3faffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3fc101170 0x7fd3fc1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44938/0 (socket says 172.21.15.84:44938) 2024-04-26T18:17:17.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.327+0000 7fd3faffd700 1 -- 172.21.15.84:0/3172388886 learned_addr learned my addr 172.21.15.84:0/3172388886 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:17.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.327+0000 7fd3faffd700 1 -- 172.21.15.84:0/3172388886 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3fc198a20 con 0x7fd3fc101170 2024-04-26T18:17:17.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.327+0000 7fd3faffd700 1 --2- 172.21.15.84:0/3172388886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3fc101170 0x7fd3fc1984e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec005950 tx=0x7fd3ec004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:17.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.327+0000 7fd400982700 1 -- 172.21.15.84:0/3172388886 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3ec026070 con 0x7fd3fc101170 2024-04-26T18:17:17.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.327+0000 7fd401984700 1 -- 172.21.15.84:0/3172388886 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3fc198cb0 con 0x7fd3fc101170 2024-04-26T18:17:17.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.327+0000 7fd400982700 1 -- 172.21.15.84:0/3172388886 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3ec00bc00 con 0x7fd3fc101170 2024-04-26T18:17:17.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.327+0000 7fd400982700 1 -- 172.21.15.84:0/3172388886 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3ec003a10 con 0x7fd3fc101170 2024-04-26T18:17:17.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.327+0000 7fd401984700 1 -- 172.21.15.84:0/3172388886 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3fc19b9a0 con 0x7fd3fc101170 2024-04-26T18:17:17.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.328+0000 7fd400982700 1 -- 172.21.15.84:0/3172388886 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd3ec029070 con 0x7fd3fc101170 2024-04-26T18:17:17.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.328+0000 7fd401984700 1 -- 172.21.15.84:0/3172388886 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3fc109570 con 0x7fd3fc101170 2024-04-26T18:17:17.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.328+0000 7fd400982700 1 --2- 172.21.15.84:0/3172388886 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd3e4039650 0x7fd3e403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:17.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.328+0000 7fd400982700 1 -- 172.21.15.84:0/3172388886 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd3ec01e070 con 0x7fd3fc101170 2024-04-26T18:17:17.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.329+0000 7fd3fa7fc700 1 --2- 172.21.15.84:0/3172388886 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd3e4039650 0x7fd3e403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:17.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.329+0000 7fd3fa7fc700 1 --2- 172.21.15.84:0/3172388886 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd3e4039650 0x7fd3e403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0006fd0 tx=0x7fd3f0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:17.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.333+0000 7fd400982700 1 -- 172.21.15.84:0/3172388886 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd3ec013070 con 0x7fd3fc101170 2024-04-26T18:17:17.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:17.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:17.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi084", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:17.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:17.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:17.422 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:17.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.536+0000 7fd401984700 1 -- 172.21.15.84:0/3172388886 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fd3fc02e5a0 con 0x7fd3fc101170 2024-04-26T18:17:17.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.539+0000 7fd400982700 1 -- 172.21.15.84:0/3172388886 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fd3ec030020 con 0x7fd3fc101170 2024-04-26T18:17:17.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.541+0000 7fd401984700 1 -- 172.21.15.84:0/3172388886 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd3e4039650 msgr2=0x7fd3e403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:17.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.541+0000 7fd401984700 1 --2- 172.21.15.84:0/3172388886 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd3e4039650 0x7fd3e403bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd3f0006fd0 tx=0x7fd3f0006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:17.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.541+0000 7fd401984700 1 -- 172.21.15.84:0/3172388886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3fc101170 msgr2=0x7fd3fc1984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:17.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.541+0000 7fd401984700 1 --2- 172.21.15.84:0/3172388886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3fc101170 0x7fd3fc1984e0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec005950 tx=0x7fd3ec004fa0 comp rx=0 tx=0).stop 2024-04-26T18:17:17.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.541+0000 7fd401984700 1 -- 172.21.15.84:0/3172388886 shutdown_connections 2024-04-26T18:17:17.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.541+0000 7fd401984700 1 --2- 172.21.15.84:0/3172388886 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd3e4039650 0x7fd3e403bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:17.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.541+0000 7fd401984700 1 --2- 172.21.15.84:0/3172388886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd3fc101170 0x7fd3fc1984e0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:17.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.542+0000 7fd401984700 1 -- 172.21.15.84:0/3172388886 >> 172.21.15.84:0/3172388886 conn(0x7fd3fc0fcaf0 msgr2=0x7fd3fc0fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:17.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.542+0000 7fd401984700 1 -- 172.21.15.84:0/3172388886 shutdown_connections 2024-04-26T18:17:17.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:17.542+0000 7fd401984700 1 -- 172.21.15.84:0/3172388886 wait complete. 2024-04-26T18:17:18.269 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T18:17:18.269 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T18:17:18.802 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:18 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3172388886' entity='client.admin' 2024-04-26T18:17:18.803 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:18.803 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:18.934 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:17:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.400+0000 7f6f26084700 1 -- 172.21.15.84:0/1635847473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f20101540 msgr2=0x7f6f20101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.400+0000 7f6f26084700 1 --2- 172.21.15.84:0/1635847473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f20101540 0x7f6f20101960 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6f10009910 tx=0x7f6f10009c20 comp rx=0 tx=0).stop 2024-04-26T18:17:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.400+0000 7f6f26084700 1 -- 172.21.15.84:0/1635847473 shutdown_connections 2024-04-26T18:17:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.400+0000 7f6f26084700 1 --2- 172.21.15.84:0/1635847473 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f20101540 0x7f6f20101960 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.400+0000 7f6f26084700 1 -- 172.21.15.84:0/1635847473 >> 172.21.15.84:0/1635847473 conn(0x7f6f200fcac0 msgr2=0x7f6f200fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:20.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.400+0000 7f6f26084700 1 -- 172.21.15.84:0/1635847473 shutdown_connections 2024-04-26T18:17:20.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.400+0000 7f6f26084700 1 -- 172.21.15.84:0/1635847473 wait complete. 2024-04-26T18:17:20.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.400+0000 7f6f26084700 1 Processor -- start 2024-04-26T18:17:20.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.400+0000 7f6f26084700 1 -- start start 2024-04-26T18:17:20.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.401+0000 7f6f26084700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f20101540 0x7f6f2019a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:20.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.401+0000 7f6f26084700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20107480 con 0x7f6f20101540 2024-04-26T18:17:20.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.401+0000 7f6f1f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f20101540 0x7f6f2019a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:20.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.401+0000 7f6f1f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f20101540 0x7f6f2019a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44950/0 (socket says 172.21.15.84:44950) 2024-04-26T18:17:20.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.401+0000 7f6f1f7fe700 1 -- 172.21.15.84:0/2930164101 learned_addr learned my addr 172.21.15.84:0/2930164101 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:20.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.401+0000 7f6f1f7fe700 1 -- 172.21.15.84:0/2930164101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2019ac30 con 0x7f6f20101540 2024-04-26T18:17:20.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.401+0000 7f6f1f7fe700 1 --2- 172.21.15.84:0/2930164101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f20101540 0x7f6f2019a6f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f10005950 tx=0x7f6f10004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:20.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.401+0000 7f6f1d7fa700 1 -- 172.21.15.84:0/2930164101 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f10026070 con 0x7f6f20101540 2024-04-26T18:17:20.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.401+0000 7f6f1d7fa700 1 -- 172.21.15.84:0/2930164101 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f1000bc00 con 0x7f6f20101540 2024-04-26T18:17:20.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.401+0000 7f6f26084700 1 -- 172.21.15.84:0/2930164101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f2019aec0 con 0x7f6f20101540 2024-04-26T18:17:20.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.401+0000 7f6f1d7fa700 1 -- 172.21.15.84:0/2930164101 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6f10003a10 con 0x7f6f20101540 2024-04-26T18:17:20.404 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.401+0000 7f6f26084700 1 -- 172.21.15.84:0/2930164101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f2019dbb0 con 0x7f6f20101540 2024-04-26T18:17:20.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.402+0000 7f6f1d7fa700 1 -- 172.21.15.84:0/2930164101 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6f10029070 con 0x7f6f20101540 2024-04-26T18:17:20.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.402+0000 7f6f26084700 1 -- 172.21.15.84:0/2930164101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f20107c00 con 0x7f6f20101540 2024-04-26T18:17:20.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.402+0000 7f6f1d7fa700 1 --2- 172.21.15.84:0/2930164101 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6f0c039650 0x7f6f0c03bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:20.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.402+0000 7f6f1d7fa700 1 -- 172.21.15.84:0/2930164101 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f6f1001e070 con 0x7f6f20101540 2024-04-26T18:17:20.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.402+0000 7f6f16ffd700 1 --2- 172.21.15.84:0/2930164101 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6f0c039650 0x7f6f0c03bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:20.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.403+0000 7f6f16ffd700 1 --2- 172.21.15.84:0/2930164101 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6f0c039650 0x7f6f0c03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6f08006fd0 tx=0x7f6f08006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:20.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.407+0000 7f6f1d7fa700 1 -- 172.21.15.84:0/2930164101 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6f10013070 con 0x7f6f20101540 2024-04-26T18:17:20.608 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.607+0000 7f6f26084700 1 -- 172.21.15.84:0/2930164101 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f6f2018cfe0 con 0x7f6f0c039650 2024-04-26T18:17:20.611 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.610+0000 7f6f1d7fa700 1 -- 172.21.15.84:0/2930164101 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6f2018cfe0 con 0x7f6f0c039650 2024-04-26T18:17:20.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.612+0000 7f6f26084700 1 -- 172.21.15.84:0/2930164101 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6f0c039650 msgr2=0x7f6f0c03bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:20.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.612+0000 7f6f26084700 1 --2- 172.21.15.84:0/2930164101 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6f0c039650 0x7f6f0c03bb10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6f08006fd0 tx=0x7f6f08006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:20.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.612+0000 7f6f26084700 1 -- 172.21.15.84:0/2930164101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f20101540 msgr2=0x7f6f2019a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:20.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.612+0000 7f6f26084700 1 --2- 172.21.15.84:0/2930164101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f20101540 0x7f6f2019a6f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f6f10005950 tx=0x7f6f10004fa0 comp rx=0 tx=0).stop 2024-04-26T18:17:20.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.612+0000 7f6f26084700 1 -- 172.21.15.84:0/2930164101 shutdown_connections 2024-04-26T18:17:20.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.612+0000 7f6f26084700 1 --2- 172.21.15.84:0/2930164101 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6f0c039650 0x7f6f0c03bb10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:20.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.612+0000 7f6f26084700 1 --2- 172.21.15.84:0/2930164101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6f20101540 0x7f6f2019a6f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:20.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.612+0000 7f6f26084700 1 -- 172.21.15.84:0/2930164101 >> 172.21.15.84:0/2930164101 conn(0x7f6f200fcac0 msgr2=0x7f6f200fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:20.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.612+0000 7f6f26084700 1 -- 172.21.15.84:0/2930164101 shutdown_connections 2024-04-26T18:17:20.614 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:20.612+0000 7f6f26084700 1 -- 172.21.15.84:0/2930164101 wait complete. 2024-04-26T18:17:21.382 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi103 2024-04-26T18:17:21.382 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T18:17:21.382 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.conf 2024-04-26T18:17:21.400 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T18:17:21.400 DEBUG:teuthology.orchestra.run.smithi103:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:17:21.457 INFO:tasks.cephadm:Adding host smithi103 to orchestrator... 2024-04-26T18:17:21.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch host add smithi103 2024-04-26T18:17:21.695 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:17:21.906 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:21 smithi084 ceph-mon[29141]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:21.906 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:21.906 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:21.906 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:21.906 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:21.906 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:21 smithi084 ceph-mon[29141]: Updating smithi084:/etc/ceph/ceph.conf 2024-04-26T18:17:21.907 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:21 smithi084 ceph-mon[29141]: Updating smithi084:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:17:21.907 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:21 smithi084 ceph-mon[29141]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:17:21.907 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:21.907 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:21.907 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:22.910 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:22 smithi084 ceph-mon[29141]: Updating smithi084:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:17:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.164+0000 7f8f76de7700 1 -- 172.21.15.84:0/1568921603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70101fd0 msgr2=0x7f8f701023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.164+0000 7f8f76de7700 1 --2- 172.21.15.84:0/1568921603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70101fd0 0x7f8f701023b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00b0a0 tx=0x7f8f6c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:17:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.164+0000 7f8f76de7700 1 -- 172.21.15.84:0/1568921603 shutdown_connections 2024-04-26T18:17:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.164+0000 7f8f76de7700 1 --2- 172.21.15.84:0/1568921603 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70101fd0 0x7f8f701023b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.164+0000 7f8f76de7700 1 -- 172.21.15.84:0/1568921603 >> 172.21.15.84:0/1568921603 conn(0x7f8f70077880 msgr2=0x7f8f70077c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.164+0000 7f8f76de7700 1 -- 172.21.15.84:0/1568921603 shutdown_connections 2024-04-26T18:17:23.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.164+0000 7f8f76de7700 1 -- 172.21.15.84:0/1568921603 wait complete. 2024-04-26T18:17:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.164+0000 7f8f76de7700 1 Processor -- start 2024-04-26T18:17:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.165+0000 7f8f76de7700 1 -- start start 2024-04-26T18:17:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.165+0000 7f8f76de7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70101fd0 0x7f8f70199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.165+0000 7f8f76de7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8f7010c200 con 0x7f8f70101fd0 2024-04-26T18:17:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.165+0000 7f8f74b83700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70101fd0 0x7f8f70199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.165+0000 7f8f74b83700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70101fd0 0x7f8f70199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44970/0 (socket says 172.21.15.84:44970) 2024-04-26T18:17:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.165+0000 7f8f74b83700 1 -- 172.21.15.84:0/3906078918 learned_addr learned my addr 172.21.15.84:0/3906078918 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.165+0000 7f8f74b83700 1 -- 172.21.15.84:0/3906078918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f70199bb0 con 0x7f8f70101fd0 2024-04-26T18:17:23.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.165+0000 7f8f74b83700 1 --2- 172.21.15.84:0/3906078918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70101fd0 0x7f8f70199670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c006130 tx=0x7f8f6c0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:23.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.165+0000 7f8f65ffb700 1 -- 172.21.15.84:0/3906078918 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f6c021020 con 0x7f8f70101fd0 2024-04-26T18:17:23.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.165+0000 7f8f76de7700 1 -- 172.21.15.84:0/3906078918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f70199e40 con 0x7f8f70101fd0 2024-04-26T18:17:23.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.165+0000 7f8f65ffb700 1 -- 172.21.15.84:0/3906078918 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f6c0079b0 con 0x7f8f70101fd0 2024-04-26T18:17:23.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.166+0000 7f8f65ffb700 1 -- 172.21.15.84:0/3906078918 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8f6c017040 con 0x7f8f70101fd0 2024-04-26T18:17:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.166+0000 7f8f76de7700 1 -- 172.21.15.84:0/3906078918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f7019a300 con 0x7f8f70101fd0 2024-04-26T18:17:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.166+0000 7f8f65ffb700 1 -- 172.21.15.84:0/3906078918 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8f6c024020 con 0x7f8f70101fd0 2024-04-26T18:17:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.167+0000 7f8f76de7700 1 -- 172.21.15.84:0/3906078918 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f70103460 con 0x7f8f70101fd0 2024-04-26T18:17:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.167+0000 7f8f65ffb700 1 --2- 172.21.15.84:0/3906078918 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8f58039650 0x7f8f5803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.167+0000 7f8f65ffb700 1 -- 172.21.15.84:0/3906078918 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f8f6c00f050 con 0x7f8f70101fd0 2024-04-26T18:17:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.167+0000 7f8f67fff700 1 --2- 172.21.15.84:0/3906078918 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8f58039650 0x7f8f5803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:23.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.167+0000 7f8f67fff700 1 --2- 172.21.15.84:0/3906078918 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8f58039650 0x7f8f5803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8f60006fd0 tx=0x7f8f60006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:23.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.172+0000 7f8f65ffb700 1 -- 172.21.15.84:0/3906078918 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f6c02b020 con 0x7f8f70101fd0 2024-04-26T18:17:23.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:23.371+0000 7f8f76de7700 1 -- 172.21.15.84:0/3906078918 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}) -- 0x7f8f70064a80 con 0x7f8f58039650 2024-04-26T18:17:24.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:24 smithi084 ceph-mon[29141]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi103", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:24.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:24 smithi084 ceph-mon[29141]: Deploying cephadm binary to smithi103 2024-04-26T18:17:25.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:25 smithi084 ceph-mon[29141]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:27.524 INFO:teuthology.orchestra.run.smithi084.stdout:Added host 'smithi103' with addr '172.21.15.103' 2024-04-26T18:17:27.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:27.524+0000 7f8f65ffb700 1 -- 172.21.15.84:0/3906078918 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f8f70064a80 con 0x7f8f58039650 2024-04-26T18:17:27.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:27.527+0000 7f8f76de7700 1 -- 172.21.15.84:0/3906078918 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8f58039650 msgr2=0x7f8f5803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:27.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:27.527+0000 7f8f76de7700 1 --2- 172.21.15.84:0/3906078918 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8f58039650 0x7f8f5803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f8f60006fd0 tx=0x7f8f60006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:27.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:27.527+0000 7f8f76de7700 1 -- 172.21.15.84:0/3906078918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70101fd0 msgr2=0x7f8f70199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:27.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:27.527+0000 7f8f76de7700 1 --2- 172.21.15.84:0/3906078918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70101fd0 0x7f8f70199670 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c006130 tx=0x7f8f6c0097d0 comp rx=0 tx=0).stop 2024-04-26T18:17:27.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:27.527+0000 7f8f76de7700 1 -- 172.21.15.84:0/3906078918 shutdown_connections 2024-04-26T18:17:27.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:27.527+0000 7f8f76de7700 1 --2- 172.21.15.84:0/3906078918 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8f58039650 0x7f8f5803bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:27.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:27.527+0000 7f8f76de7700 1 --2- 172.21.15.84:0/3906078918 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f70101fd0 0x7f8f70199670 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:27.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:27.527+0000 7f8f76de7700 1 -- 172.21.15.84:0/3906078918 >> 172.21.15.84:0/3906078918 conn(0x7f8f70077880 msgr2=0x7f8f70103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:27.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:27.527+0000 7f8f76de7700 1 -- 172.21.15.84:0/3906078918 shutdown_connections 2024-04-26T18:17:27.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:27.527+0000 7f8f76de7700 1 -- 172.21.15.84:0/3906078918 wait complete. 2024-04-26T18:17:28.181 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T18:17:28.412 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:17:28.533 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:28 smithi084 ceph-mon[29141]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:28.534 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:28 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:28.534 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:28 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:28.534 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:28 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:29.512 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:29 smithi084 ceph-mon[29141]: Added host smithi103 2024-04-26T18:17:29.512 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:30.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:29.999+0000 7f644e719700 1 -- 172.21.15.84:0/2140243325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6448104080 msgr2=0x7f6448104460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:30.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:29.999+0000 7f644e719700 1 --2- 172.21.15.84:0/2140243325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6448104080 0x7f6448104460 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6438009910 tx=0x7f6438009c20 comp rx=0 tx=0).stop 2024-04-26T18:17:30.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:29.999+0000 7f644e719700 1 -- 172.21.15.84:0/2140243325 shutdown_connections 2024-04-26T18:17:30.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:29.999+0000 7f644e719700 1 --2- 172.21.15.84:0/2140243325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6448104080 0x7f6448104460 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:30.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:29.999+0000 7f644e719700 1 -- 172.21.15.84:0/2140243325 >> 172.21.15.84:0/2140243325 conn(0x7f64480fd830 msgr2=0x7f64480ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:29.999+0000 7f644e719700 1 -- 172.21.15.84:0/2140243325 shutdown_connections 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:29.999+0000 7f644e719700 1 -- 172.21.15.84:0/2140243325 wait complete. 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.000+0000 7f644e719700 1 Processor -- start 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.000+0000 7f644e719700 1 -- start start 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.000+0000 7f644e719700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6448104080 0x7f644819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.000+0000 7f644e719700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f644810d380 con 0x7f6448104080 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.000+0000 7f6447fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6448104080 0x7f644819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.000+0000 7f6447fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6448104080 0x7f644819e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53904/0 (socket says 172.21.15.84:53904) 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.000+0000 7f6447fff700 1 -- 172.21.15.84:0/1826940949 learned_addr learned my addr 172.21.15.84:0/1826940949 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.001+0000 7f6447fff700 1 -- 172.21.15.84:0/1826940949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f644819eb40 con 0x7f6448104080 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.001+0000 7f6447fff700 1 --2- 172.21.15.84:0/1826940949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6448104080 0x7f644819e600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f64380056d0 tx=0x7f64380057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.001+0000 7f64457fa700 1 -- 172.21.15.84:0/1826940949 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6438026070 con 0x7f6448104080 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.001+0000 7f644e719700 1 -- 172.21.15.84:0/1826940949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f644819edd0 con 0x7f6448104080 2024-04-26T18:17:30.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.001+0000 7f644e719700 1 -- 172.21.15.84:0/1826940949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6448198780 con 0x7f6448104080 2024-04-26T18:17:30.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.001+0000 7f64457fa700 1 -- 172.21.15.84:0/1826940949 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6438004370 con 0x7f6448104080 2024-04-26T18:17:30.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.001+0000 7f64457fa700 1 -- 172.21.15.84:0/1826940949 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64380039b0 con 0x7f6448104080 2024-04-26T18:17:30.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.002+0000 7f644e719700 1 -- 172.21.15.84:0/1826940949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f644806bb50 con 0x7f6448104080 2024-04-26T18:17:30.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.002+0000 7f64457fa700 1 -- 172.21.15.84:0/1826940949 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f6438029070 con 0x7f6448104080 2024-04-26T18:17:30.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.002+0000 7f64457fa700 1 --2- 172.21.15.84:0/1826940949 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6430039650 0x7f643003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:30.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.002+0000 7f64457fa700 1 -- 172.21.15.84:0/1826940949 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f643801e070 con 0x7f6448104080 2024-04-26T18:17:30.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.002+0000 7f64477fe700 1 --2- 172.21.15.84:0/1826940949 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6430039650 0x7f643003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:30.004 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.003+0000 7f64477fe700 1 --2- 172.21.15.84:0/1826940949 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6430039650 0x7f643003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f643c006fd0 tx=0x7f643c006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:30.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.007+0000 7f64457fa700 1 -- 172.21.15.84:0/1826940949 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6438013070 con 0x7f6448104080 2024-04-26T18:17:30.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.216+0000 7f644e719700 1 -- 172.21.15.84:0/1826940949 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6448064a80 con 0x7f6430039650 2024-04-26T18:17:30.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.217+0000 7f64457fa700 1 -- 172.21.15.84:0/1826940949 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+160 (secure 0 0 0) 0x7f6448064a80 con 0x7f6430039650 2024-04-26T18:17:30.217 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:17:30.218 INFO:teuthology.orchestra.run.smithi084.stdout:[{"addr": "172.21.15.84", "hostname": "smithi084", "labels": [], "status": ""}, {"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}] 2024-04-26T18:17:30.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.219+0000 7f644e719700 1 -- 172.21.15.84:0/1826940949 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6430039650 msgr2=0x7f643003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:30.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.219+0000 7f644e719700 1 --2- 172.21.15.84:0/1826940949 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6430039650 0x7f643003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f643c006fd0 tx=0x7f643c006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:30.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.219+0000 7f644e719700 1 -- 172.21.15.84:0/1826940949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6448104080 msgr2=0x7f644819e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:30.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.219+0000 7f644e719700 1 --2- 172.21.15.84:0/1826940949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6448104080 0x7f644819e600 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f64380056d0 tx=0x7f64380057b0 comp rx=0 tx=0).stop 2024-04-26T18:17:30.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.220+0000 7f644e719700 1 -- 172.21.15.84:0/1826940949 shutdown_connections 2024-04-26T18:17:30.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.220+0000 7f644e719700 1 --2- 172.21.15.84:0/1826940949 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6430039650 0x7f643003bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:30.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.220+0000 7f644e719700 1 --2- 172.21.15.84:0/1826940949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6448104080 0x7f644819e600 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:30.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.220+0000 7f644e719700 1 -- 172.21.15.84:0/1826940949 >> 172.21.15.84:0/1826940949 conn(0x7f64480fd830 msgr2=0x7f6448104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:30.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.220+0000 7f644e719700 1 -- 172.21.15.84:0/1826940949 shutdown_connections 2024-04-26T18:17:30.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:30.220+0000 7f644e719700 1 -- 172.21.15.84:0/1826940949 wait complete. 2024-04-26T18:17:30.509 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:30 smithi084 ceph-mon[29141]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:30.509 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:30 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:30.795 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi174 2024-04-26T18:17:30.795 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T18:17:30.795 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.conf 2024-04-26T18:17:30.813 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T18:17:30.813 DEBUG:teuthology.orchestra.run.smithi174:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:17:30.870 INFO:tasks.cephadm:Adding host smithi174 to orchestrator... 2024-04-26T18:17:30.870 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch host add smithi174 2024-04-26T18:17:31.103 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:17:31.428 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:31 smithi084 ceph-mon[29141]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:17:32.418 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:32 smithi084 ceph-mon[29141]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.631+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/469761319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6bc103090 msgr2=0x7ff6bc103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.631+0000 7ff6c0c9d700 1 --2- 172.21.15.84:0/469761319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6bc103090 0x7ff6bc103470 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac009910 tx=0x7ff6ac009c20 comp rx=0 tx=0).stop 2024-04-26T18:17:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.631+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/469761319 shutdown_connections 2024-04-26T18:17:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.631+0000 7ff6c0c9d700 1 --2- 172.21.15.84:0/469761319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6bc103090 0x7ff6bc103470 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.631+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/469761319 >> 172.21.15.84:0/469761319 conn(0x7ff6bc0fe960 msgr2=0x7ff6bc100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.632+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/469761319 shutdown_connections 2024-04-26T18:17:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.632+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/469761319 wait complete. 2024-04-26T18:17:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.632+0000 7ff6c0c9d700 1 Processor -- start 2024-04-26T18:17:32.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.632+0000 7ff6c0c9d700 1 -- start start 2024-04-26T18:17:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.632+0000 7ff6c0c9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6bc103090 0x7ff6bc1a0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.632+0000 7ff6c0c9d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff6bc10d310 con 0x7ff6bc103090 2024-04-26T18:17:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.633+0000 7ff6ba59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6bc103090 0x7ff6bc1a0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.633+0000 7ff6ba59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6bc103090 0x7ff6bc1a0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53918/0 (socket says 172.21.15.84:53918) 2024-04-26T18:17:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.633+0000 7ff6ba59c700 1 -- 172.21.15.84:0/2189858181 learned_addr learned my addr 172.21.15.84:0/2189858181 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:32.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.633+0000 7ff6ba59c700 1 -- 172.21.15.84:0/2189858181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6bc1a0c60 con 0x7ff6bc103090 2024-04-26T18:17:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.633+0000 7ff6ba59c700 1 --2- 172.21.15.84:0/2189858181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6bc103090 0x7ff6bc1a0720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0056d0 tx=0x7ff6ac0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.633+0000 7ff6ab7fe700 1 -- 172.21.15.84:0/2189858181 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6ac026070 con 0x7ff6bc103090 2024-04-26T18:17:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.633+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/2189858181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6bc1a0f50 con 0x7ff6bc103090 2024-04-26T18:17:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.633+0000 7ff6ab7fe700 1 -- 172.21.15.84:0/2189858181 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6ac00bba0 con 0x7ff6bc103090 2024-04-26T18:17:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.633+0000 7ff6ab7fe700 1 -- 172.21.15.84:0/2189858181 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff6ac003920 con 0x7ff6bc103090 2024-04-26T18:17:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.633+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/2189858181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6bc19a8e0 con 0x7ff6bc103090 2024-04-26T18:17:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.634+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/2189858181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6bc104570 con 0x7ff6bc103090 2024-04-26T18:17:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.634+0000 7ff6ab7fe700 1 -- 172.21.15.84:0/2189858181 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff6ac029070 con 0x7ff6bc103090 2024-04-26T18:17:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.634+0000 7ff6ab7fe700 1 --2- 172.21.15.84:0/2189858181 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff6a4039650 0x7ff6a403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.634+0000 7ff6ab7fe700 1 -- 172.21.15.84:0/2189858181 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff6ac01e070 con 0x7ff6bc103090 2024-04-26T18:17:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.634+0000 7ff6b9d9b700 1 --2- 172.21.15.84:0/2189858181 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff6a4039650 0x7ff6a403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:32.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.635+0000 7ff6b9d9b700 1 --2- 172.21.15.84:0/2189858181 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff6a4039650 0x7ff6a403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0006fd0 tx=0x7ff6b0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:32.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.639+0000 7ff6ab7fe700 1 -- 172.21.15.84:0/2189858181 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff6ac013070 con 0x7ff6bc103090 2024-04-26T18:17:32.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:32.841+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/2189858181 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}) -- 0x7ff6bc19bf20 con 0x7ff6a4039650 2024-04-26T18:17:34.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:34 smithi084 ceph-mon[29141]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:34 smithi084 ceph-mon[29141]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:34 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:34 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:34 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:34 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:34 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:34 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:34 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:35.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:35 smithi084 ceph-mon[29141]: Deploying cephadm binary to smithi174 2024-04-26T18:17:35.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:35 smithi084 ceph-mon[29141]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-26T18:17:35.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:35 smithi084 ceph-mon[29141]: Updating smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:17:35.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:35 smithi084 ceph-mon[29141]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:17:36.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:36 smithi084 ceph-mon[29141]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:37.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:37.083+0000 7ff6ab7fe700 1 -- 172.21.15.84:0/2189858181 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7ff6bc19bf20 con 0x7ff6a4039650 2024-04-26T18:17:37.084 INFO:teuthology.orchestra.run.smithi084.stdout:Added host 'smithi174' with addr '172.21.15.174' 2024-04-26T18:17:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:37.085+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/2189858181 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff6a4039650 msgr2=0x7ff6a403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:37.085+0000 7ff6c0c9d700 1 --2- 172.21.15.84:0/2189858181 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff6a4039650 0x7ff6a403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0006fd0 tx=0x7ff6b0006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:37.085+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/2189858181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6bc103090 msgr2=0x7ff6bc1a0720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:37.085+0000 7ff6c0c9d700 1 --2- 172.21.15.84:0/2189858181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6bc103090 0x7ff6bc1a0720 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7ff6ac0056d0 tx=0x7ff6ac0057b0 comp rx=0 tx=0).stop 2024-04-26T18:17:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:37.085+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/2189858181 shutdown_connections 2024-04-26T18:17:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:37.085+0000 7ff6c0c9d700 1 --2- 172.21.15.84:0/2189858181 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff6a4039650 0x7ff6a403bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:37.085+0000 7ff6c0c9d700 1 --2- 172.21.15.84:0/2189858181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff6bc103090 0x7ff6bc1a0720 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:37.085+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/2189858181 >> 172.21.15.84:0/2189858181 conn(0x7ff6bc0fe960 msgr2=0x7ff6bc075650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:37.086+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/2189858181 shutdown_connections 2024-04-26T18:17:37.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:37.086+0000 7ff6c0c9d700 1 -- 172.21.15.84:0/2189858181 wait complete. 2024-04-26T18:17:37.708 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T18:17:37.940 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:17:38.062 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:37 smithi084 ceph-mon[29141]: Updating smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:17:38.062 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:38.062 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:38.062 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:38.062 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:38.062 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:37 smithi084 ceph-mon[29141]: Added host smithi174 2024-04-26T18:17:38.062 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:37 smithi084 ceph-mon[29141]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:38.062 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:38.063 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:39.413 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:39 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:39.413 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:39 smithi084 ceph-mon[29141]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.640+0000 7fd6e7886700 1 -- 172.21.15.84:0/2952419718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6e0101f90 msgr2=0x7fd6e0102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:39.641 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.640+0000 7fd6e7886700 1 --2- 172.21.15.84:0/2952419718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6e0101f90 0x7fd6e0102370 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009910 tx=0x7fd6d0009c20 comp rx=0 tx=0).stop 2024-04-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.640+0000 7fd6e7886700 1 -- 172.21.15.84:0/2952419718 shutdown_connections 2024-04-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.640+0000 7fd6e7886700 1 --2- 172.21.15.84:0/2952419718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6e0101f90 0x7fd6e0102370 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.640+0000 7fd6e7886700 1 -- 172.21.15.84:0/2952419718 >> 172.21.15.84:0/2952419718 conn(0x7fd6e00fd820 msgr2=0x7fd6e00ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.640+0000 7fd6e7886700 1 -- 172.21.15.84:0/2952419718 shutdown_connections 2024-04-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.640+0000 7fd6e7886700 1 -- 172.21.15.84:0/2952419718 wait complete. 2024-04-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.641+0000 7fd6e7886700 1 Processor -- start 2024-04-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.641+0000 7fd6e7886700 1 -- start start 2024-04-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.641+0000 7fd6e7886700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6e0101f90 0x7fd6e0103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.641+0000 7fd6e7886700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e010c210 con 0x7fd6e0101f90 2024-04-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.641+0000 7fd6e5622700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6e0101f90 0x7fd6e0103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.641+0000 7fd6e5622700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6e0101f90 0x7fd6e0103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57884/0 (socket says 172.21.15.84:57884) 2024-04-26T18:17:39.642 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.641+0000 7fd6e5622700 1 -- 172.21.15.84:0/1638848932 learned_addr learned my addr 172.21.15.84:0/1638848932 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.642+0000 7fd6e5622700 1 -- 172.21.15.84:0/1638848932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e0105920 con 0x7fd6e0101f90 2024-04-26T18:17:39.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.642+0000 7fd6e5622700 1 --2- 172.21.15.84:0/1638848932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6e0101f90 0x7fd6e0103a30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00056d0 tx=0x7fd6d00057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.642+0000 7fd6d67fc700 1 -- 172.21.15.84:0/1638848932 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d0026070 con 0x7fd6e0101f90 2024-04-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.642+0000 7fd6d67fc700 1 -- 172.21.15.84:0/1638848932 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d000bc00 con 0x7fd6e0101f90 2024-04-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.642+0000 7fd6d67fc700 1 -- 172.21.15.84:0/1638848932 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd6d0003920 con 0x7fd6e0101f90 2024-04-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.642+0000 7fd6e7886700 1 -- 172.21.15.84:0/1638848932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e0103fd0 con 0x7fd6e0101f90 2024-04-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.642+0000 7fd6e7886700 1 -- 172.21.15.84:0/1638848932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e0104490 con 0x7fd6e0101f90 2024-04-26T18:17:39.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.643+0000 7fd6d67fc700 1 -- 172.21.15.84:0/1638848932 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd6d0029070 con 0x7fd6e0101f90 2024-04-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.643+0000 7fd6e7886700 1 -- 172.21.15.84:0/1638848932 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6e0103470 con 0x7fd6e0101f90 2024-04-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.643+0000 7fd6d67fc700 1 --2- 172.21.15.84:0/1638848932 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd6cc0396b0 0x7fd6cc03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.643+0000 7fd6d67fc700 1 -- 172.21.15.84:0/1638848932 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd6d001e070 con 0x7fd6e0101f90 2024-04-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.643+0000 7fd6e4e21700 1 --2- 172.21.15.84:0/1638848932 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd6cc0396b0 0x7fd6cc03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:39.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.643+0000 7fd6e4e21700 1 --2- 172.21.15.84:0/1638848932 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd6cc0396b0 0x7fd6cc03bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc006fd0 tx=0x7fd6dc006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:39.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.648+0000 7fd6d67fc700 1 -- 172.21.15.84:0/1638848932 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6d0013070 con 0x7fd6e0101f90 2024-04-26T18:17:39.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.853+0000 7fd6e7886700 1 -- 172.21.15.84:0/1638848932 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd6e0104fd0 con 0x7fd6cc0396b0 2024-04-26T18:17:39.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.854+0000 7fd6d67fc700 1 -- 172.21.15.84:0/1638848932 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+240 (secure 0 0 0) 0x7fd6e0104fd0 con 0x7fd6cc0396b0 2024-04-26T18:17:39.854 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:17:39.854 INFO:teuthology.orchestra.run.smithi084.stdout:[{"addr": "172.21.15.84", "hostname": "smithi084", "labels": [], "status": ""}, {"addr": "172.21.15.103", "hostname": "smithi103", "labels": [], "status": ""}, {"addr": "172.21.15.174", "hostname": "smithi174", "labels": [], "status": ""}] 2024-04-26T18:17:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.856+0000 7fd6e7886700 1 -- 172.21.15.84:0/1638848932 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd6cc0396b0 msgr2=0x7fd6cc03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.856+0000 7fd6e7886700 1 --2- 172.21.15.84:0/1638848932 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd6cc0396b0 0x7fd6cc03bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc006fd0 tx=0x7fd6dc006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.856+0000 7fd6e7886700 1 -- 172.21.15.84:0/1638848932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6e0101f90 msgr2=0x7fd6e0103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.856+0000 7fd6e7886700 1 --2- 172.21.15.84:0/1638848932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6e0101f90 0x7fd6e0103a30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00056d0 tx=0x7fd6d00057b0 comp rx=0 tx=0).stop 2024-04-26T18:17:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.856+0000 7fd6e7886700 1 -- 172.21.15.84:0/1638848932 shutdown_connections 2024-04-26T18:17:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.856+0000 7fd6e7886700 1 --2- 172.21.15.84:0/1638848932 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd6cc0396b0 0x7fd6cc03bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.856+0000 7fd6e7886700 1 --2- 172.21.15.84:0/1638848932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd6e0101f90 0x7fd6e0103a30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.856+0000 7fd6e7886700 1 -- 172.21.15.84:0/1638848932 >> 172.21.15.84:0/1638848932 conn(0x7fd6e00fd820 msgr2=0x7fd6e006b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.856+0000 7fd6e7886700 1 -- 172.21.15.84:0/1638848932 shutdown_connections 2024-04-26T18:17:39.857 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:39.856+0000 7fd6e7886700 1 -- 172.21.15.84:0/1638848932 wait complete. 2024-04-26T18:17:40.472 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T18:17:40.472 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T18:17:40.706 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:17:41.003 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:40 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:41.004 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:40 smithi084 ceph-mon[29141]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:17:41.869 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:41 smithi084 ceph-mon[29141]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:42.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.095+0000 7fe9e20ea700 1 -- 172.21.15.84:0/3725488897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9dc103090 msgr2=0x7fe9dc103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:42.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.095+0000 7fe9e20ea700 1 --2- 172.21.15.84:0/3725488897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9dc103090 0x7fe9dc103470 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc009910 tx=0x7fe9cc009c20 comp rx=0 tx=0).stop 2024-04-26T18:17:42.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.095+0000 7fe9e20ea700 1 -- 172.21.15.84:0/3725488897 shutdown_connections 2024-04-26T18:17:42.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.095+0000 7fe9e20ea700 1 --2- 172.21.15.84:0/3725488897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9dc103090 0x7fe9dc103470 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:42.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.095+0000 7fe9e20ea700 1 -- 172.21.15.84:0/3725488897 >> 172.21.15.84:0/3725488897 conn(0x7fe9dc0fe940 msgr2=0x7fe9dc100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:42.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.095+0000 7fe9e20ea700 1 -- 172.21.15.84:0/3725488897 shutdown_connections 2024-04-26T18:17:42.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.095+0000 7fe9e20ea700 1 -- 172.21.15.84:0/3725488897 wait complete. 2024-04-26T18:17:42.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.096+0000 7fe9e20ea700 1 Processor -- start 2024-04-26T18:17:42.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.096+0000 7fe9e20ea700 1 -- start start 2024-04-26T18:17:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.096+0000 7fe9e20ea700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9dc103090 0x7fe9dc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.096+0000 7fe9e20ea700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe9dc10d310 con 0x7fe9dc103090 2024-04-26T18:17:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.096+0000 7fe9db7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9dc103090 0x7fe9dc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.096+0000 7fe9db7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9dc103090 0x7fe9dc19e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57900/0 (socket says 172.21.15.84:57900) 2024-04-26T18:17:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.096+0000 7fe9db7fe700 1 -- 172.21.15.84:0/2329779228 learned_addr learned my addr 172.21.15.84:0/2329779228 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:17:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.096+0000 7fe9db7fe700 1 -- 172.21.15.84:0/2329779228 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9dc19eb00 con 0x7fe9dc103090 2024-04-26T18:17:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.097+0000 7fe9db7fe700 1 --2- 172.21.15.84:0/2329779228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9dc103090 0x7fe9dc19e5c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0056d0 tx=0x7fe9cc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.097+0000 7fe9d8ff9700 1 -- 172.21.15.84:0/2329779228 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc026070 con 0x7fe9dc103090 2024-04-26T18:17:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.097+0000 7fe9d8ff9700 1 -- 172.21.15.84:0/2329779228 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9cc00bba0 con 0x7fe9dc103090 2024-04-26T18:17:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.097+0000 7fe9e20ea700 1 -- 172.21.15.84:0/2329779228 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9dc19edf0 con 0x7fe9dc103090 2024-04-26T18:17:42.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.097+0000 7fe9d8ff9700 1 -- 172.21.15.84:0/2329779228 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe9cc003920 con 0x7fe9dc103090 2024-04-26T18:17:42.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.097+0000 7fe9e20ea700 1 -- 172.21.15.84:0/2329779228 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9dc198860 con 0x7fe9dc103090 2024-04-26T18:17:42.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.098+0000 7fe9d8ff9700 1 -- 172.21.15.84:0/2329779228 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe9cc029070 con 0x7fe9dc103090 2024-04-26T18:17:42.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.098+0000 7fe9e20ea700 1 -- 172.21.15.84:0/2329779228 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9dc104570 con 0x7fe9dc103090 2024-04-26T18:17:42.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.098+0000 7fe9d8ff9700 1 --2- 172.21.15.84:0/2329779228 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe9c4039650 0x7fe9c403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:42.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.098+0000 7fe9d8ff9700 1 -- 172.21.15.84:0/2329779228 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe9cc01e070 con 0x7fe9dc103090 2024-04-26T18:17:42.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.098+0000 7fe9daffd700 1 --2- 172.21.15.84:0/2329779228 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe9c4039650 0x7fe9c403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:42.099 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.098+0000 7fe9daffd700 1 --2- 172.21.15.84:0/2329779228 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe9c4039650 0x7fe9c403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0006fd0 tx=0x7fe9d0006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:42.103 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.103+0000 7fe9d8ff9700 1 -- 172.21.15.84:0/2329779228 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe9cc013070 con 0x7fe9dc103090 2024-04-26T18:17:42.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.307+0000 7fe9e20ea700 1 -- 172.21.15.84:0/2329779228 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fe9dc19efc0 con 0x7fe9dc103090 2024-04-26T18:17:42.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.694+0000 7fe9d8ff9700 1 -- 172.21.15.84:0/2329779228 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fe9cc030090 con 0x7fe9dc103090 2024-04-26T18:17:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.697+0000 7fe9e20ea700 1 -- 172.21.15.84:0/2329779228 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe9c4039650 msgr2=0x7fe9c403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.697+0000 7fe9e20ea700 1 --2- 172.21.15.84:0/2329779228 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe9c4039650 0x7fe9c403bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe9d0006fd0 tx=0x7fe9d0006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.697+0000 7fe9e20ea700 1 -- 172.21.15.84:0/2329779228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9dc103090 msgr2=0x7fe9dc19e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.697+0000 7fe9e20ea700 1 --2- 172.21.15.84:0/2329779228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9dc103090 0x7fe9dc19e5c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc0056d0 tx=0x7fe9cc0057b0 comp rx=0 tx=0).stop 2024-04-26T18:17:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.697+0000 7fe9e20ea700 1 -- 172.21.15.84:0/2329779228 shutdown_connections 2024-04-26T18:17:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.697+0000 7fe9e20ea700 1 --2- 172.21.15.84:0/2329779228 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe9c4039650 0x7fe9c403bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.697+0000 7fe9e20ea700 1 --2- 172.21.15.84:0/2329779228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9dc103090 0x7fe9dc19e5c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.697+0000 7fe9e20ea700 1 -- 172.21.15.84:0/2329779228 >> 172.21.15.84:0/2329779228 conn(0x7fe9dc0fe940 msgr2=0x7fe9dc104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.697+0000 7fe9e20ea700 1 -- 172.21.15.84:0/2329779228 shutdown_connections 2024-04-26T18:17:42.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:17:42.697+0000 7fe9e20ea700 1 -- 172.21.15.84:0/2329779228 wait complete. 2024-04-26T18:17:42.698 INFO:teuthology.orchestra.run.smithi084.stderr:adjusted tunables profile to default 2024-04-26T18:17:43.080 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:42 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2329779228' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T18:17:43.080 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:42 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:43.080 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:42 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:43.080 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:42 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:43.080 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:42 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:43.080 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:42 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:43.081 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:42 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:43.081 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:42 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:43.423 INFO:tasks.cephadm:Adding mon.a on smithi084 2024-04-26T18:17:43.423 INFO:tasks.cephadm:Adding mon.b on smithi103 2024-04-26T18:17:43.424 INFO:tasks.cephadm:Adding mon.c on smithi174 2024-04-26T18:17:43.424 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch apply mon '3;smithi084:172.21.15.84=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c' 2024-04-26T18:17:43.664 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:17:44.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:43 smithi084 ceph-mon[29141]: Updating smithi174:/etc/ceph/ceph.conf 2024-04-26T18:17:44.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:43 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2329779228' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T18:17:44.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:43 smithi084 ceph-mon[29141]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:17:44.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:43 smithi084 ceph-mon[29141]: Updating smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:17:44.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:43 smithi084 ceph-mon[29141]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:44.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:43 smithi084 ceph-mon[29141]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:17:44.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:43 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:44.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:43 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:44.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:43 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:45.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:44 smithi084 ceph-mon[29141]: Updating smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:17:45.184 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.183+0000 7faf1f042700 1 -- 172.21.15.174:0/1824937177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf18103090 msgr2=0x7faf18103470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:45.184 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.183+0000 7faf1f042700 1 --2- 172.21.15.174:0/1824937177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf18103090 0x7faf18103470 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faf1400b0a0 tx=0x7faf1400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:17:45.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.183+0000 7faf1f042700 1 -- 172.21.15.174:0/1824937177 shutdown_connections 2024-04-26T18:17:45.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.183+0000 7faf1f042700 1 --2- 172.21.15.174:0/1824937177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf18103090 0x7faf18103470 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:45.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.183+0000 7faf1f042700 1 -- 172.21.15.174:0/1824937177 >> 172.21.15.174:0/1824937177 conn(0x7faf180fe960 msgr2=0x7faf18100d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:45.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.184+0000 7faf1f042700 1 -- 172.21.15.174:0/1824937177 shutdown_connections 2024-04-26T18:17:45.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.184+0000 7faf1f042700 1 -- 172.21.15.174:0/1824937177 wait complete. 2024-04-26T18:17:45.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.184+0000 7faf1f042700 1 Processor -- start 2024-04-26T18:17:45.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.184+0000 7faf1f042700 1 -- start start 2024-04-26T18:17:45.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.184+0000 7faf1f042700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf18103090 0x7faf1819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:45.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.184+0000 7faf1f042700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faf1810d310 con 0x7faf18103090 2024-04-26T18:17:45.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.184+0000 7faf1cdde700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf18103090 0x7faf1819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:45.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.185+0000 7faf1cdde700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf18103090 0x7faf1819e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.174:54008/0 (socket says 172.21.15.174:54008) 2024-04-26T18:17:45.185 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.185+0000 7faf1cdde700 1 -- 172.21.15.174:0/2221152806 learned_addr learned my addr 172.21.15.174:0/2221152806 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T18:17:45.187 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.185+0000 7faf1cdde700 1 -- 172.21.15.174:0/2221152806 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1819eab0 con 0x7faf18103090 2024-04-26T18:17:45.187 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.185+0000 7faf1cdde700 1 --2- 172.21.15.174:0/2221152806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf18103090 0x7faf1819e570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faf14006130 tx=0x7faf140097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:45.187 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.185+0000 7faf0dffb700 1 -- 172.21.15.174:0/2221152806 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf14021020 con 0x7faf18103090 2024-04-26T18:17:45.187 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.185+0000 7faf0dffb700 1 -- 172.21.15.174:0/2221152806 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf140079b0 con 0x7faf18103090 2024-04-26T18:17:45.187 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.185+0000 7faf1f042700 1 -- 172.21.15.174:0/2221152806 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1819ed40 con 0x7faf18103090 2024-04-26T18:17:45.187 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.185+0000 7faf0dffb700 1 -- 172.21.15.174:0/2221152806 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7faf14017040 con 0x7faf18103090 2024-04-26T18:17:45.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.185+0000 7faf1f042700 1 -- 172.21.15.174:0/2221152806 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf18198640 con 0x7faf18103090 2024-04-26T18:17:45.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.186+0000 7faf0dffb700 1 -- 172.21.15.174:0/2221152806 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7faf14024020 con 0x7faf18103090 2024-04-26T18:17:45.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.186+0000 7faf1f042700 1 -- 172.21.15.174:0/2221152806 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf18104570 con 0x7faf18103090 2024-04-26T18:17:45.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.187+0000 7faf0dffb700 1 --2- 172.21.15.174:0/2221152806 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faf00039650 0x7faf0003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:45.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.187+0000 7faf0dffb700 1 -- 172.21.15.174:0/2221152806 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7faf1400f050 con 0x7faf18103090 2024-04-26T18:17:45.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.187+0000 7faf0ffff700 1 --2- 172.21.15.174:0/2221152806 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faf00039650 0x7faf0003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:45.188 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.188+0000 7faf0ffff700 1 --2- 172.21.15.174:0/2221152806 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faf00039650 0x7faf0003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faf08006fd0 tx=0x7faf08006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:45.192 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.192+0000 7faf0dffb700 1 -- 172.21.15.174:0/2221152806 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faf1402b020 con 0x7faf18103090 2024-04-26T18:17:45.399 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.399+0000 7faf1f042700 1 -- 172.21.15.174:0/2221152806 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi084:172.21.15.84=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}) -- 0x7faf18199b70 con 0x7faf00039650 2024-04-26T18:17:45.406 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.406+0000 7faf0dffb700 1 -- 172.21.15.174:0/2221152806 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7faf18199b70 con 0x7faf00039650 2024-04-26T18:17:45.406 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mon update... 2024-04-26T18:17:45.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.408+0000 7faf1f042700 1 -- 172.21.15.174:0/2221152806 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faf00039650 msgr2=0x7faf0003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:45.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.408+0000 7faf1f042700 1 --2- 172.21.15.174:0/2221152806 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faf00039650 0x7faf0003bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7faf08006fd0 tx=0x7faf08006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:45.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.408+0000 7faf1f042700 1 -- 172.21.15.174:0/2221152806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf18103090 msgr2=0x7faf1819e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:45.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.408+0000 7faf1f042700 1 --2- 172.21.15.174:0/2221152806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf18103090 0x7faf1819e570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faf14006130 tx=0x7faf140097d0 comp rx=0 tx=0).stop 2024-04-26T18:17:45.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.408+0000 7faf1f042700 1 -- 172.21.15.174:0/2221152806 shutdown_connections 2024-04-26T18:17:45.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.408+0000 7faf1f042700 1 --2- 172.21.15.174:0/2221152806 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faf00039650 0x7faf0003bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:45.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.408+0000 7faf1f042700 1 --2- 172.21.15.174:0/2221152806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faf18103090 0x7faf1819e570 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:45.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.408+0000 7faf1f042700 1 -- 172.21.15.174:0/2221152806 >> 172.21.15.174:0/2221152806 conn(0x7faf180fe960 msgr2=0x7faf18104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:45.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.408+0000 7faf1f042700 1 -- 172.21.15.174:0/2221152806 shutdown_connections 2024-04-26T18:17:45.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:45.408+0000 7faf1f042700 1 -- 172.21.15.174:0/2221152806 wait complete. 2024-04-26T18:17:46.049 DEBUG:teuthology.orchestra.run.smithi103:mon.b> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.b.service 2024-04-26T18:17:46.052 DEBUG:teuthology.orchestra.run.smithi174:mon.c> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.c.service 2024-04-26T18:17:46.054 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T18:17:46.054 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T18:17:46.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:45 smithi084 ceph-mon[29141]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:46.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:17:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:46.855 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:17:47.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:46 smithi084 ceph-mon[29141]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi084:172.21.15.84=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:47.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:46 smithi084 ceph-mon[29141]: Saving service mon spec with placement smithi084:172.21.15.84=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c;count:3 2024-04-26T18:17:47.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:46 smithi084 ceph-mon[29141]: Deploying daemon mon.c on smithi174 2024-04-26T18:17:48.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:47 smithi084 ceph-mon[29141]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:49.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.938+0000 7f3aafd24700 1 -- 172.21.15.174:0/3602611204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa8101f90 msgr2=0x7f3aa8102370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:49.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.938+0000 7f3aafd24700 1 --2- 172.21.15.174:0/3602611204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa8101f90 0x7f3aa8102370 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000b0a0 tx=0x7f3aa000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:17:49.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.938+0000 7f3aafd24700 1 -- 172.21.15.174:0/3602611204 shutdown_connections 2024-04-26T18:17:49.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.938+0000 7f3aafd24700 1 --2- 172.21.15.174:0/3602611204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa8101f90 0x7f3aa8102370 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:49.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.938+0000 7f3aafd24700 1 -- 172.21.15.174:0/3602611204 >> 172.21.15.174:0/3602611204 conn(0x7f3aa80fd840 msgr2=0x7f3aa80ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:49.940 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.938+0000 7f3aafd24700 1 -- 172.21.15.174:0/3602611204 shutdown_connections 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.938+0000 7f3aafd24700 1 -- 172.21.15.174:0/3602611204 wait complete. 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.938+0000 7f3aafd24700 1 Processor -- start 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.938+0000 7f3aafd24700 1 -- start start 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.939+0000 7f3aafd24700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa8101f90 0x7f3aa81a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.939+0000 7f3aafd24700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3aa810c210 con 0x7f3aa8101f90 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.939+0000 7f3aadac0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa8101f90 0x7f3aa81a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.939+0000 7f3aadac0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa8101f90 0x7f3aa81a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.174:54176/0 (socket says 172.21.15.174:54176) 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.939+0000 7f3aadac0700 1 -- 172.21.15.174:0/3262962442 learned_addr learned my addr 172.21.15.174:0/3262962442 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.939+0000 7f3aadac0700 1 -- 172.21.15.174:0/3262962442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3aa81a0c10 con 0x7f3aa8101f90 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.939+0000 7f3aadac0700 1 --2- 172.21.15.174:0/3262962442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa8101f90 0x7f3aa81a06d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000d010 tx=0x7f3aa0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.940+0000 7f3a9effd700 1 -- 172.21.15.174:0/3262962442 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa0021020 con 0x7f3aa8101f90 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.940+0000 7f3a9effd700 1 -- 172.21.15.174:0/3262962442 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa00044b0 con 0x7f3aa8101f90 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.940+0000 7f3aafd24700 1 -- 172.21.15.174:0/3262962442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3aa81a0ea0 con 0x7f3aa8101f90 2024-04-26T18:17:49.941 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.940+0000 7f3a9effd700 1 -- 172.21.15.174:0/3262962442 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3aa0017040 con 0x7f3aa8101f90 2024-04-26T18:17:49.942 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.940+0000 7f3aafd24700 1 -- 172.21.15.174:0/3262962442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3aa819a850 con 0x7f3aa8101f90 2024-04-26T18:17:49.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.941+0000 7f3a9effd700 1 -- 172.21.15.174:0/3262962442 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3aa0024020 con 0x7f3aa8101f90 2024-04-26T18:17:49.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.941+0000 7f3a9effd700 1 --2- 172.21.15.174:0/3262962442 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3a940396a0 0x7f3a9403bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:49.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.941+0000 7f3a9effd700 1 -- 172.21.15.174:0/3262962442 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3aa000f050 con 0x7f3aa8101f90 2024-04-26T18:17:49.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.941+0000 7f3aafd24700 1 -- 172.21.15.174:0/3262962442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3aa8103470 con 0x7f3aa8101f90 2024-04-26T18:17:49.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.941+0000 7f3aad2bf700 1 --2- 172.21.15.174:0/3262962442 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3a940396a0 0x7f3a9403bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:49.943 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.942+0000 7f3aad2bf700 1 --2- 172.21.15.174:0/3262962442 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3a940396a0 0x7f3a9403bb60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3a98006fd0 tx=0x7f3a98006e40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:49.946 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:49.947+0000 7f3a9effd700 1 -- 172.21.15.174:0/3262962442 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3aa002b020 con 0x7f3aa8101f90 2024-04-26T18:17:50.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:50.239+0000 7f3aafd24700 1 -- 172.21.15.174:0/3262962442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3aa8064a80 con 0x7f3aa8101f90 2024-04-26T18:17:50.239 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:50.239+0000 7f3a9effd700 1 -- 172.21.15.174:0/3262962442 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f3aa002e580 con 0x7f3aa8101f90 2024-04-26T18:17:50.240 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:17:50.240 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":1,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","modified":"2024-04-26T18:16:03.305539Z","created":"2024-04-26T18:16:03.305539Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T18:17:50.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:50.242+0000 7f3aafd24700 1 -- 172.21.15.174:0/3262962442 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3a940396a0 msgr2=0x7f3a9403bb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:50.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:50.242+0000 7f3aafd24700 1 --2- 172.21.15.174:0/3262962442 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3a940396a0 0x7f3a9403bb60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3a98006fd0 tx=0x7f3a98006e40 comp rx=0 tx=0).stop 2024-04-26T18:17:50.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:50.242+0000 7f3aafd24700 1 -- 172.21.15.174:0/3262962442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa8101f90 msgr2=0x7f3aa81a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:50.242 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:50.242+0000 7f3aafd24700 1 --2- 172.21.15.174:0/3262962442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa8101f90 0x7f3aa81a06d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3aa000d010 tx=0x7f3aa0003ce0 comp rx=0 tx=0).stop 2024-04-26T18:17:50.243 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:50.242+0000 7f3aafd24700 1 -- 172.21.15.174:0/3262962442 shutdown_connections 2024-04-26T18:17:50.243 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:50.242+0000 7f3aafd24700 1 --2- 172.21.15.174:0/3262962442 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3a940396a0 0x7f3a9403bb60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:50.243 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:50.242+0000 7f3aafd24700 1 --2- 172.21.15.174:0/3262962442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3aa8101f90 0x7f3aa81a06d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:50.243 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:50.242+0000 7f3aafd24700 1 -- 172.21.15.174:0/3262962442 >> 172.21.15.174:0/3262962442 conn(0x7f3aa80fd840 msgr2=0x7f3aa806b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:50.243 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:50.242+0000 7f3aafd24700 1 -- 172.21.15.174:0/3262962442 shutdown_connections 2024-04-26T18:17:50.243 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:50.242+0000 7f3aafd24700 1 -- 172.21.15.174:0/3262962442 wait complete. 2024-04-26T18:17:50.243 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 1 2024-04-26T18:17:50.362 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 podman[32672]: 2024-04-26 18:17:50.362923427 +0000 UTC m=+0.372308524 container init 8f4977f04f29ed9fb432fbeeb2c0f68b1682083a2d6635d736687467f13d5e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T18:17:50.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:50 smithi084 ceph-mon[29141]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:50.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 podman[32672]: 2024-04-26 18:17:50.366354001 +0000 UTC m=+0.375739102 container start 8f4977f04f29ed9fb432fbeeb2c0f68b1682083a2d6635d736687467f13d5e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:17:50.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T18:17:50.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: ceph version 19.0.0-3267-gbc94a2a9 (bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1) squid (dev), process ceph-mon, pid 2 2024-04-26T18:17:50.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: pidfile_write: ignore empty --pid-file 2024-04-26T18:17:50.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: load: jerasure load: lrc 2024-04-26T18:17:50.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: RocksDB version: 7.9.2 2024-04-26T18:17:50.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Git sha 0 2024-04-26T18:17:50.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Compile date 2024-04-26 07:20:26 2024-04-26T18:17:50.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: DB SUMMARY 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: DB Session ID: YS06RD2WRRUPHTBS1WEI 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: CURRENT file: CURRENT 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.error_if_exists: 0 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.create_if_missing: 0 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.paranoid_checks: 1 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.env: 0x55cffc322cc0 2024-04-26T18:17:50.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T18:17:50.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.info_log: 0x55cffdab9d80 2024-04-26T18:17:50.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T18:17:50.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.statistics: (nil) 2024-04-26T18:17:50.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.use_fsync: 0 2024-04-26T18:17:50.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_log_file_size: 0 2024-04-26T18:17:50.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T18:17:50.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T18:17:50.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T18:17:50.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T18:17:50.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.allow_fallocate: 1 2024-04-26T18:17:50.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T18:17:50.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T18:17:50.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.use_direct_reads: 0 2024-04-26T18:17:50.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T18:17:50.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T18:17:50.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.db_log_dir: 2024-04-26T18:17:50.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.wal_dir: 2024-04-26T18:17:50.694 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T18:17:50.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T18:17:50.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T18:17:50.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T18:17:50.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T18:17:50.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T18:17:50.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T18:17:50.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T18:17:50.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.write_buffer_manager: 0x55cffda8b0e0 2024-04-26T18:17:50.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T18:17:50.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T18:17:50.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T18:17:50.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T18:17:50.695 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T18:17:50.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T18:17:50.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T18:17:50.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T18:17:50.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.unordered_write: 0 2024-04-26T18:17:50.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T18:17:50.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T18:17:50.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T18:17:50.696 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T18:17:50.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.row_cache: None 2024-04-26T18:17:50.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.wal_filter: None 2024-04-26T18:17:50.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T18:17:50.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T18:17:50.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.two_write_queues: 0 2024-04-26T18:17:50.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T18:17:50.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.wal_compression: 0 2024-04-26T18:17:50.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.atomic_flush: 0 2024-04-26T18:17:50.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T18:17:50.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T18:17:50.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T18:17:50.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.log_readahead_size: 0 2024-04-26T18:17:50.697 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T18:17:50.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T18:17:50.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T18:17:50.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T18:17:50.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T18:17:50.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T18:17:50.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T18:17:50.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_background_jobs: 2 2024-04-26T18:17:50.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_background_compactions: -1 2024-04-26T18:17:50.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_subcompactions: 1 2024-04-26T18:17:50.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T18:17:50.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T18:17:50.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T18:17:50.698 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T18:17:50.699 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T18:17:50.699 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T18:17:50.699 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T18:17:50.699 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T18:17:50.699 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_open_files: -1 2024-04-26T18:17:50.699 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T18:17:50.699 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T18:17:50.699 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T18:17:50.699 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T18:17:50.699 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_background_flushes: -1 2024-04-26T18:17:50.699 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Compression algorithms supported: 2024-04-26T18:17:50.700 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T18:17:50.700 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: kZSTD supported: 0 2024-04-26T18:17:50.700 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: kXpressCompression supported: 0 2024-04-26T18:17:50.700 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T18:17:50.700 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: kZlibCompression supported: 1 2024-04-26T18:17:50.700 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: kSnappyCompression supported: 1 2024-04-26T18:17:50.700 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: kLZ4Compression supported: 1 2024-04-26T18:17:50.700 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: kBZip2Compression supported: 0 2024-04-26T18:17:50.700 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T18:17:50.700 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T18:17:50.700 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-26T18:17:50.700 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.merge_operator: 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compaction_filter: None 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cffdab9f40) 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks: 1 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout: pin_top_level_index_and_filter: 1 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_type: 0 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_index_type: 0 2024-04-26T18:17:50.701 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_shortening: 1 2024-04-26T18:17:50.702 INFO:journalctl@ceph.mon.c.smithi174.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T18:17:50.702 INFO:journalctl@ceph.mon.c.smithi174.stdout: checksum: 4 2024-04-26T18:17:50.702 INFO:journalctl@ceph.mon.c.smithi174.stdout: no_block_cache: 0 2024-04-26T18:17:50.702 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache: 0x55cffdabd1f0 2024-04-26T18:17:50.702 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_name: BinnedLRUCache 2024-04-26T18:17:50.702 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_options: 2024-04-26T18:17:50.702 INFO:journalctl@ceph.mon.c.smithi174.stdout: capacity : 536870912 2024-04-26T18:17:50.702 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_shard_bits : 4 2024-04-26T18:17:50.702 INFO:journalctl@ceph.mon.c.smithi174.stdout: strict_capacity_limit : 0 2024-04-26T18:17:50.702 INFO:journalctl@ceph.mon.c.smithi174.stdout: high_pri_pool_ratio: 0.000 2024-04-26T18:17:50.702 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_cache_compressed: (nil) 2024-04-26T18:17:50.702 INFO:journalctl@ceph.mon.c.smithi174.stdout: persistent_cache: (nil) 2024-04-26T18:17:50.703 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size: 4096 2024-04-26T18:17:50.703 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_size_deviation: 10 2024-04-26T18:17:50.703 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_restart_interval: 16 2024-04-26T18:17:50.703 INFO:journalctl@ceph.mon.c.smithi174.stdout: index_block_restart_interval: 1 2024-04-26T18:17:50.703 INFO:journalctl@ceph.mon.c.smithi174.stdout: metadata_block_size: 4096 2024-04-26T18:17:50.703 INFO:journalctl@ceph.mon.c.smithi174.stdout: partition_filters: 0 2024-04-26T18:17:50.703 INFO:journalctl@ceph.mon.c.smithi174.stdout: use_delta_encoding: 1 2024-04-26T18:17:50.703 INFO:journalctl@ceph.mon.c.smithi174.stdout: filter_policy: bloomfilter 2024-04-26T18:17:50.704 INFO:journalctl@ceph.mon.c.smithi174.stdout: whole_key_filtering: 1 2024-04-26T18:17:50.704 INFO:journalctl@ceph.mon.c.smithi174.stdout: verify_compression: 0 2024-04-26T18:17:50.704 INFO:journalctl@ceph.mon.c.smithi174.stdout: read_amp_bytes_per_bit: 0 2024-04-26T18:17:50.704 INFO:journalctl@ceph.mon.c.smithi174.stdout: format_version: 5 2024-04-26T18:17:50.704 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_index_compression: 1 2024-04-26T18:17:50.704 INFO:journalctl@ceph.mon.c.smithi174.stdout: block_align: 0 2024-04-26T18:17:50.704 INFO:journalctl@ceph.mon.c.smithi174.stdout: max_auto_readahead_size: 262144 2024-04-26T18:17:50.704 INFO:journalctl@ceph.mon.c.smithi174.stdout: prepopulate_block_cache: 0 2024-04-26T18:17:50.704 INFO:journalctl@ceph.mon.c.smithi174.stdout: initial_auto_readahead_size: 8192 2024-04-26T18:17:50.704 INFO:journalctl@ceph.mon.c.smithi174.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T18:17:50.704 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compression: NoCompression 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.num_levels: 7 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T18:17:50.705 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:17:50.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T18:17:50.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T18:17:50.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:17:50.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:17:50.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T18:17:50.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T18:17:50.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T18:17:50.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T18:17:50.706 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T18:17:50.707 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T18:17:50.707 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T18:17:50.707 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T18:17:50.707 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T18:17:50.707 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T18:17:50.707 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T18:17:50.707 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T18:17:50.707 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T18:17:50.707 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T18:17:50.707 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T18:17:50.707 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T18:17:50.708 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T18:17:50.709 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T18:17:50.709 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T18:17:50.709 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T18:17:50.709 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T18:17:50.709 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T18:17:50.709 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T18:17:50.709 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T18:17:50.709 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T18:17:50.710 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T18:17:50.710 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T18:17:50.710 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T18:17:50.710 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-26T18:17:50.710 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.inplace_update_support: 0 2024-04-26T18:17:50.710 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T18:17:50.710 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T18:17:50.710 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T18:17:50.710 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T18:17:50.710 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.bloom_locality: 0 2024-04-26T18:17:50.710 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.max_successive_merges: 0 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.ttl: 2592000 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.enable_blob_files: false 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.min_blob_size: 0 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T18:17:50.711 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T18:17:50.712 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T18:17:50.712 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T18:17:50.712 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T18:17:50.712 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T18:17:50.712 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-26T18:17:50.712 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-26T18:17:50.712 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4bb54a79-9b6e-4921-938e-57abe6be321c 2024-04-26T18:17:50.712 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155470396164, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-26T18:17:50.712 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-26T18:17:50.712 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155470396573, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714155470, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4bb54a79-9b6e-4921-938e-57abe6be321c", "db_session_id": "YS06RD2WRRUPHTBS1WEI", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-26T18:17:50.712 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714155470396652, "job": 1, "event": "recovery_finished"} 2024-04-26T18:17:50.712 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-26T18:17:50.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T18:17:50.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cffdbda000 2024-04-26T18:17:50.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: DB pointer 0x55cffdbc6000 2024-04-26T18:17:50.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T18:17:50.713 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T18:17:50.713 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** DB Stats ** 2024-04-26T18:17:50.713 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:17:50.713 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T18:17:50.713 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:17:50.714 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:17:50.714 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T18:17:50.714 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T18:17:50.714 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T18:17:50.714 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-04-26T18:17:50.714 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-04-26T18:17:50.714 INFO:journalctl@ceph.mon.c.smithi174.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:17:50.714 INFO:journalctl@ceph.mon.c.smithi174.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T18:17:50.714 INFO:journalctl@ceph.mon.c.smithi174.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T18:17:50.714 INFO:journalctl@ceph.mon.c.smithi174.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T18:17:50.714 INFO:journalctl@ceph.mon.c.smithi174.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** Compaction Stats [default] ** 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T18:17:50.715 INFO:journalctl@ceph.mon.c.smithi174.stdout: Cumulative compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:17:50.716 INFO:journalctl@ceph.mon.c.smithi174.stdout: Interval compaction: 0.00 GB write, 0.66 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T18:17:50.716 INFO:journalctl@ceph.mon.c.smithi174.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T18:17:50.716 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache BinnedLRUCache@0x55cffdabd1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-26T18:17:50.717 INFO:journalctl@ceph.mon.c.smithi174.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-26T18:17:50.717 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-04-26T18:17:50.717 INFO:journalctl@ceph.mon.c.smithi174.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T18:17:50.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-26T18:17:50.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: using public_addr v2:172.21.15.174:0/0 -> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-04-26T18:17:50.717 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: starting mon.c rank -1 at public addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] at bind addrs [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:17:50.718 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(???) e0 preinit fsid df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:17:50.718 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(synchronizing).mds e1 new map 2024-04-26T18:17:50.718 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-26T18:17:50.718 INFO:journalctl@ceph.mon.c.smithi174.stdout: e1 2024-04-26T18:17:50.718 INFO:journalctl@ceph.mon.c.smithi174.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T18:17:50.718 INFO:journalctl@ceph.mon.c.smithi174.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T18:17:50.718 INFO:journalctl@ceph.mon.c.smithi174.stdout: legacy client fscid: -1 2024-04-26T18:17:50.718 INFO:journalctl@ceph.mon.c.smithi174.stdout: 2024-04-26T18:17:50.718 INFO:journalctl@ceph.mon.c.smithi174.stdout: No filesystems configured 2024-04-26T18:17:50.718 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-26T18:17:50.718 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-26T18:17:50.718 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-26T18:17:50.718 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-26T18:17:50.719 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-26T18:17:50.719 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-26T18:17:50.719 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T18:17:50.719 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:17:50.719 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:17:50.719 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T18:17:50.719 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:50.719 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi174", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:50.719 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:50.720 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.720 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.720 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.720 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.720 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi103", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:50.720 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:50.720 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:50.720 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: Deploying cephadm binary to smithi174 2024-04-26T18:17:50.720 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-26T18:17:50.720 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: Updating smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:17:50.720 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:17:50.720 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:50.721 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: Updating smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:17:50.721 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.721 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.721 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.721 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.721 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: Added host smithi174 2024-04-26T18:17:50.721 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:50.721 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:50.721 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.721 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.721 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:50.721 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.722 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:17:50.722 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:50.722 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2329779228' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T18:17:50.722 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.722 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.722 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.722 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.722 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi174", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:17:50.722 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: Updating smithi174:/etc/ceph/ceph.conf 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2329779228' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: Updating smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: Updating smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:17:50.723 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:50.724 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:17:50.724 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:50.724 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:17:50.724 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:50.724 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi084:172.21.15.84=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:17:50.724 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: Saving service mon spec with placement smithi084:172.21.15.84=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c;count:3 2024-04-26T18:17:50.724 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: Deploying daemon mon.c on smithi174 2024-04-26T18:17:50.724 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:50.724 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:50.724 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 ceph-mon[32705]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-26T18:17:50.724 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 bash[32672]: 8f4977f04f29ed9fb432fbeeb2c0f68b1682083a2d6635d736687467f13d5e32 2024-04-26T18:17:50.724 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:50 smithi174 systemd[1]: Started Ceph mon.c for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:17:52.019 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T18:17:52.019 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T18:17:52.264 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:17:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:17:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: mon.a calling monitor election 2024-04-26T18:17:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: mon.c calling monitor election 2024-04-26T18:17:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:17:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: monmap epoch 2 2024-04-26T18:17:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: fsid df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:17:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: last_changed 2024-04-26T18:17:50.419122+0000 2024-04-26T18:17:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: created 2024-04-26T18:16:03.305539+0000 2024-04-26T18:17:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: min_mon_release 19 (squid) 2024-04-26T18:17:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: election_strategy: 1 2024-04-26T18:17:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-04-26T18:17:55.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-04-26T18:17:55.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: fsmap 2024-04-26T18:17:55.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:17:55.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: mgrmap e13: x(active, since 50s) 2024-04-26T18:17:55.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: overall HEALTH_OK 2024-04-26T18:17:55.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:55.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:55.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:55.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:17:55.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:17:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: mon.a calling monitor election 2024-04-26T18:17:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: mon.c calling monitor election 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: monmap epoch 2 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: fsid df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: last_changed 2024-04-26T18:17:50.419122+0000 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: created 2024-04-26T18:16:03.305539+0000 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: min_mon_release 19 (squid) 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: election_strategy: 1 2024-04-26T18:17:55.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-04-26T18:17:55.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-04-26T18:17:55.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: fsmap 2024-04-26T18:17:55.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:17:55.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: mgrmap e13: x(active, since 50s) 2024-04-26T18:17:55.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: overall HEALTH_OK 2024-04-26T18:17:55.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:55.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:55.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:17:55.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:17:55.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:17:56.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.634+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1070120173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da00035c0 msgr2=0x7f8da0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:56.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.634+0000 7f8dbb86b700 1 --2- 172.21.15.174:0/1070120173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da00035c0 0x7f8da0005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f8db4102e70 tx=0x7f8db000feb0 comp rx=0 tx=0).stop 2024-04-26T18:17:56.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.634+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1070120173 shutdown_connections 2024-04-26T18:17:56.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.634+0000 7f8dbb86b700 1 --2- 172.21.15.174:0/1070120173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8da00035c0 0x7f8da0005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:56.634 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.634+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1070120173 >> 172.21.15.174:0/1070120173 conn(0x7f8db40fea30 msgr2=0x7f8db4100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:56.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.634+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1070120173 shutdown_connections 2024-04-26T18:17:56.635 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.634+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1070120173 wait complete. 2024-04-26T18:17:56.636 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.635+0000 7f8dbb86b700 1 Processor -- start 2024-04-26T18:17:56.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.635+0000 7f8dbb86b700 1 -- start start 2024-04-26T18:17:56.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.635+0000 7f8dbb86b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da00035c0 0x7f8db419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:56.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.635+0000 7f8dbb86b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8db419eb90 0x7f8db41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:56.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.635+0000 7f8dbb86b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8db410ae90 con 0x7f8db419eb90 2024-04-26T18:17:56.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.635+0000 7f8dbb86b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8db410ad10 con 0x7f8da00035c0 2024-04-26T18:17:56.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.635+0000 7f8db9607700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da00035c0 0x7f8db419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:56.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.635+0000 7f8db9607700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da00035c0 0x7f8db419e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:50764/0 (socket says 172.21.15.174:50764) 2024-04-26T18:17:56.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.635+0000 7f8db9607700 1 -- 172.21.15.174:0/1279597581 learned_addr learned my addr 172.21.15.174:0/1279597581 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T18:17:56.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.636+0000 7f8db8e06700 1 --2- 172.21.15.174:0/1279597581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8db419eb90 0x7f8db41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:56.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.636+0000 7f8db9607700 1 -- 172.21.15.174:0/1279597581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da00035c0 msgr2=0x7f8db419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T18:17:56.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.636+0000 7f8db9607700 1 -- 172.21.15.174:0/1279597581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da00035c0 msgr2=0x7f8db419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T18:17:56.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.636+0000 7f8db9607700 1 --2- 172.21.15.174:0/1279597581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da00035c0 0x7f8db419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T18:17:56.637 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.636+0000 7f8db9607700 1 --2- 172.21.15.174:0/1279597581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da00035c0 0x7f8db419e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:17:56.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.636+0000 7f8db8e06700 1 -- 172.21.15.174:0/1279597581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da00035c0 msgr2=0x7f8db419e650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:17:56.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.636+0000 7f8db8e06700 1 --2- 172.21.15.174:0/1279597581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da00035c0 0x7f8db419e650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:56.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.636+0000 7f8db8e06700 1 -- 172.21.15.174:0/1279597581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8db419f1d0 con 0x7f8db419eb90 2024-04-26T18:17:56.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.636+0000 7f8db8e06700 1 --2- 172.21.15.174:0/1279597581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8db419eb90 0x7f8db41987d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8da400b4d0 tx=0x7f8da400b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:56.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.636+0000 7f8daa7fc700 1 -- 172.21.15.174:0/1279597581 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8da4011630 con 0x7f8db419eb90 2024-04-26T18:17:56.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.636+0000 7f8daa7fc700 1 -- 172.21.15.174:0/1279597581 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8da4011c80 con 0x7f8db419eb90 2024-04-26T18:17:56.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.636+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1279597581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8db4198d70 con 0x7f8db419eb90 2024-04-26T18:17:56.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.636+0000 7f8daa7fc700 1 -- 172.21.15.174:0/1279597581 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f8da4010da0 con 0x7f8db419eb90 2024-04-26T18:17:56.638 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.637+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1279597581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8db41992d0 con 0x7f8db419eb90 2024-04-26T18:17:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.637+0000 7f8daa7fc700 1 -- 172.21.15.174:0/1279597581 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f8da4018020 con 0x7f8db419eb90 2024-04-26T18:17:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.638+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1279597581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8db4104680 con 0x7f8db419eb90 2024-04-26T18:17:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.638+0000 7f8daa7fc700 1 --2- 172.21.15.174:0/1279597581 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8d98039440 0x7f8d9803b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:17:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.638+0000 7f8daa7fc700 1 -- 172.21.15.174:0/1279597581 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8da404c2a0 con 0x7f8db419eb90 2024-04-26T18:17:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.638+0000 7f8db9607700 1 --2- 172.21.15.174:0/1279597581 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8d98039440 0x7f8d9803b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:17:56.639 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.638+0000 7f8db9607700 1 --2- 172.21.15.174:0/1279597581 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8d98039440 0x7f8d9803b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8db000fa60 tx=0x7f8db0005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:17:56.643 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.644+0000 7f8daa7fc700 1 -- 172.21.15.174:0/1279597581 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8da401f020 con 0x7f8db419eb90 2024-04-26T18:17:56.747 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:56 smithi084 ceph-mon[29141]: Deploying daemon mon.b on smithi103 2024-04-26T18:17:56.747 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:56 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:56.834 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:56 smithi174 ceph-mon[32705]: Deploying daemon mon.b on smithi103 2024-04-26T18:17:56.834 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:56 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:17:56.931 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.931+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1279597581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f8db4199640 con 0x7f8db419eb90 2024-04-26T18:17:56.931 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.931+0000 7f8daa7fc700 1 -- 172.21.15.174:0/1279597581 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f8da4010410 con 0x7f8db419eb90 2024-04-26T18:17:56.932 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:17:56.932 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":2,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","modified":"2024-04-26T18:17:50.419122Z","created":"2024-04-26T18:16:03.305539Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T18:17:56.934 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.933+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1279597581 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8d98039440 msgr2=0x7f8d9803b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:56.934 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.933+0000 7f8dbb86b700 1 --2- 172.21.15.174:0/1279597581 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8d98039440 0x7f8d9803b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8db000fa60 tx=0x7f8db0005960 comp rx=0 tx=0).stop 2024-04-26T18:17:56.934 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.933+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1279597581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8db419eb90 msgr2=0x7f8db41987d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:17:56.934 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.933+0000 7f8dbb86b700 1 --2- 172.21.15.174:0/1279597581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8db419eb90 0x7f8db41987d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f8da400b4d0 tx=0x7f8da400b890 comp rx=0 tx=0).stop 2024-04-26T18:17:56.934 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.933+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1279597581 shutdown_connections 2024-04-26T18:17:56.934 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.933+0000 7f8dbb86b700 1 --2- 172.21.15.174:0/1279597581 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8d98039440 0x7f8d9803b900 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:56.934 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.933+0000 7f8dbb86b700 1 --2- 172.21.15.174:0/1279597581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8da00035c0 0x7f8db419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:56.935 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.933+0000 7f8dbb86b700 1 --2- 172.21.15.174:0/1279597581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8db419eb90 0x7f8db41987d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:17:56.935 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.933+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1279597581 >> 172.21.15.174:0/1279597581 conn(0x7f8db40fea30 msgr2=0x7f8db40ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:17:56.935 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.933+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1279597581 shutdown_connections 2024-04-26T18:17:56.935 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:17:56.933+0000 7f8dbb86b700 1 -- 172.21.15.174:0/1279597581 wait complete. 2024-04-26T18:17:56.935 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 2 2024-04-26T18:17:57.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:57 smithi084 ceph-mon[29141]: from='client.? 172.21.15.174:0/1279597581' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:17:57.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:17:57 smithi084 ceph-mon[29141]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:57.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:57 smithi174 ceph-mon[32705]: from='client.? 172.21.15.174:0/1279597581' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:17:57.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:17:57 smithi174 ceph-mon[32705]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:17:58.518 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-26T18:17:58.519 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T18:17:58.762 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:18:04.552 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.551+0000 7f3c8da6f700 1 -- 172.21.15.174:0/1602561412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700035c0 msgr2=0x7f3c70005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:04.552 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.551+0000 7f3c8da6f700 1 --2- 172.21.15.174:0/1602561412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700035c0 0x7f3c70005a50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3c8807a660 tx=0x7f3c7800fe60 comp rx=0 tx=0).stop 2024-04-26T18:18:04.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.551+0000 7f3c8da6f700 1 -- 172.21.15.174:0/1602561412 shutdown_connections 2024-04-26T18:18:04.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.551+0000 7f3c8da6f700 1 --2- 172.21.15.174:0/1602561412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700035c0 0x7f3c70005a50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:04.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.551+0000 7f3c8da6f700 1 -- 172.21.15.174:0/1602561412 >> 172.21.15.174:0/1602561412 conn(0x7f3c880784d0 msgr2=0x7f3c880788e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:04.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.551+0000 7f3c8da6f700 1 -- 172.21.15.174:0/1602561412 shutdown_connections 2024-04-26T18:18:04.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.551+0000 7f3c8da6f700 1 -- 172.21.15.174:0/1602561412 wait complete. 2024-04-26T18:18:04.553 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.552+0000 7f3c8da6f700 1 Processor -- start 2024-04-26T18:18:04.557 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.553+0000 7f3c8da6f700 1 -- start start 2024-04-26T18:18:04.557 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.553+0000 7f3c8da6f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700035c0 0x7f3c8819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:04.557 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.553+0000 7f3c8da6f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8819db60 0x7f3c881977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:04.557 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.553+0000 7f3c8da6f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88197d30 0x7f3c881981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:04.557 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.553+0000 7f3c8da6f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3c88109dc0 con 0x7f3c700035c0 2024-04-26T18:18:04.557 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.553+0000 7f3c8da6f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c88109c40 con 0x7f3c88197d30 2024-04-26T18:18:04.557 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.553+0000 7f3c8da6f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c88109f40 con 0x7f3c8819db60 2024-04-26T18:18:04.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.553+0000 7f3c877fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700035c0 0x7f3c8819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:04.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.553+0000 7f3c877fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700035c0 0x7f3c8819d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.174:38292/0 (socket says 172.21.15.174:38292) 2024-04-26T18:18:04.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.553+0000 7f3c877fe700 1 -- 172.21.15.174:0/3736468742 learned_addr learned my addr 172.21.15.174:0/3736468742 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T18:18:04.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.553+0000 7f3c86ffd700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8819db60 0x7f3c881977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:04.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.553+0000 7f3c87fff700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88197d30 0x7f3c881981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:04.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.553+0000 7f3c86ffd700 1 -- 172.21.15.174:0/3736468742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8819db60 msgr2=0x7f3c881977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T18:18:04.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c86ffd700 1 -- 172.21.15.174:0/3736468742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8819db60 msgr2=0x7f3c881977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T18:18:04.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c86ffd700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8819db60 0x7f3c881977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T18:18:04.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c877fe700 1 -- 172.21.15.174:0/3736468742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8819db60 msgr2=0x7f3c881977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:04.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c86ffd700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8819db60 0x7f3c881977f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:18:04.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c877fe700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8819db60 0x7f3c881977f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:04.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c87fff700 1 -- 172.21.15.174:0/3736468742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88197d30 msgr2=0x7f3c881981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-26T18:18:04.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c87fff700 1 -- 172.21.15.174:0/3736468742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88197d30 msgr2=0x7f3c881981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T18:18:04.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c877fe700 1 -- 172.21.15.174:0/3736468742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88197d30 msgr2=0x7f3c881981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:04.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c87fff700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88197d30 0x7f3c881981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T18:18:04.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c87fff700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88197d30 0x7f3c881981e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:18:04.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c877fe700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88197d30 0x7f3c881981e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:04.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c877fe700 1 -- 172.21.15.174:0/3736468742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c881a4480 con 0x7f3c700035c0 2024-04-26T18:18:04.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c877fe700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700035c0 0x7f3c8819d620 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3c78003ca0 tx=0x7f3c78003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:04.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c84ff9700 1 -- 172.21.15.174:0/3736468742 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c7802b020 con 0x7f3c700035c0 2024-04-26T18:18:04.559 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c84ff9700 1 -- 172.21.15.174:0/3736468742 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c7801d440 con 0x7f3c700035c0 2024-04-26T18:18:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c8da6f700 1 -- 172.21.15.174:0/3736468742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c881a46b0 con 0x7f3c700035c0 2024-04-26T18:18:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c84ff9700 1 -- 172.21.15.174:0/3736468742 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78022040 con 0x7f3c700035c0 2024-04-26T18:18:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.554+0000 7f3c8da6f700 1 -- 172.21.15.174:0/3736468742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c881a4c10 con 0x7f3c700035c0 2024-04-26T18:18:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.555+0000 7f3c84ff9700 1 -- 172.21.15.174:0/3736468742 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f3c7802e020 con 0x7f3c700035c0 2024-04-26T18:18:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.555+0000 7f3c84ff9700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c74039ae0 0x7f3c7403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.555+0000 7f3c84ff9700 1 -- 172.21.15.174:0/3736468742 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3c78012b50 con 0x7f3c700035c0 2024-04-26T18:18:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.556+0000 7f3c86ffd700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c74039ae0 0x7f3c7403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.556+0000 7f3c8da6f700 1 -- 172.21.15.174:0/3736468742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c88103730 con 0x7f3c700035c0 2024-04-26T18:18:04.560 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.556+0000 7f3c86ffd700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c74039ae0 0x7f3c7403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009850 tx=0x7f3c7c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:04.561 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.561+0000 7f3c84ff9700 1 -- 172.21.15.174:0/3736468742 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c78035020 con 0x7f3c700035c0 2024-04-26T18:18:04.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:18:04.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: mon.a calling monitor election 2024-04-26T18:18:04.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: mon.c calling monitor election 2024-04-26T18:18:04.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:04.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:18:04.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:04.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:04.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:04.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:04.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:04.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:04.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:04.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:18:04.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: monmap epoch 3 2024-04-26T18:18:04.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: fsid df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:18:04.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: last_changed 2024-04-26T18:17:59.445928+0000 2024-04-26T18:18:04.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: created 2024-04-26T18:16:03.305539+0000 2024-04-26T18:18:04.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: min_mon_release 19 (squid) 2024-04-26T18:18:04.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: election_strategy: 1 2024-04-26T18:18:04.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-04-26T18:18:04.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-04-26T18:18:04.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-04-26T18:18:04.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: fsmap 2024-04-26T18:18:04.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:18:04.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: mgrmap e13: x(active, since 59s) 2024-04-26T18:18:04.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: overall HEALTH_OK 2024-04-26T18:18:04.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:04.847 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:04 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:04.851 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.851+0000 7f3c8da6f700 1 -- 172.21.15.174:0/3736468742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3c8802e490 con 0x7f3c700035c0 2024-04-26T18:18:04.852 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:18:04.852 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: mon.a calling monitor election 2024-04-26T18:18:04.852 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: mon.c calling monitor election 2024-04-26T18:18:04.852 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:04.852 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:18:04.852 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:04.852 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:04.853 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:04.853 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:04.853 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:04.853 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:04.853 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:04.853 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-26T18:18:04.853 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: monmap epoch 3 2024-04-26T18:18:04.853 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: fsid df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:18:04.853 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: last_changed 2024-04-26T18:17:59.445928+0000 2024-04-26T18:18:04.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: created 2024-04-26T18:16:03.305539+0000 2024-04-26T18:18:04.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: min_mon_release 19 (squid) 2024-04-26T18:18:04.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: election_strategy: 1 2024-04-26T18:18:04.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-04-26T18:18:04.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-04-26T18:18:04.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-04-26T18:18:04.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: fsmap 2024-04-26T18:18:04.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:18:04.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: mgrmap e13: x(active, since 59s) 2024-04-26T18:18:04.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: overall HEALTH_OK 2024-04-26T18:18:04.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:04.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:04 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:04.855 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.852+0000 7f3c84ff9700 1 -- 172.21.15.174:0/3736468742 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1283 (secure 0 0 0) 0x7f3c780252d0 con 0x7f3c700035c0 2024-04-26T18:18:04.856 INFO:teuthology.orchestra.run.smithi174.stdout: 2024-04-26T18:18:04.856 INFO:teuthology.orchestra.run.smithi174.stdout:{"epoch":3,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","modified":"2024-04-26T18:17:59.445928Z","created":"2024-04-26T18:16:03.305539Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:3300","nonce":0},{"type":"v1","addr":"172.21.15.84:6789","nonce":0}]},"addr":"172.21.15.84:6789/0","public_addr":"172.21.15.84:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:3300","nonce":0},{"type":"v1","addr":"172.21.15.174:6789","nonce":0}]},"addr":"172.21.15.174:6789/0","public_addr":"172.21.15.174:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:3300","nonce":0},{"type":"v1","addr":"172.21.15.103:6789","nonce":0}]},"addr":"172.21.15.103:6789/0","public_addr":"172.21.15.103:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T18:18:04.857 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.855+0000 7f3c8da6f700 1 -- 172.21.15.174:0/3736468742 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c74039ae0 msgr2=0x7f3c7403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:04.857 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.855+0000 7f3c8da6f700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c74039ae0 0x7f3c7403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c009850 tx=0x7f3c7c008040 comp rx=0 tx=0).stop 2024-04-26T18:18:04.857 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.855+0000 7f3c8da6f700 1 -- 172.21.15.174:0/3736468742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700035c0 msgr2=0x7f3c8819d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:04.857 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.855+0000 7f3c8da6f700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700035c0 0x7f3c8819d620 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3c78003ca0 tx=0x7f3c78003d80 comp rx=0 tx=0).stop 2024-04-26T18:18:04.857 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.855+0000 7f3c8da6f700 1 -- 172.21.15.174:0/3736468742 shutdown_connections 2024-04-26T18:18:04.857 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.855+0000 7f3c8da6f700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c74039ae0 0x7f3c7403bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:04.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.855+0000 7f3c8da6f700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c700035c0 0x7f3c8819d620 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:04.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.855+0000 7f3c8da6f700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c88197d30 0x7f3c881981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:04.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.855+0000 7f3c8da6f700 1 --2- 172.21.15.174:0/3736468742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8819db60 0x7f3c881977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:04.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.855+0000 7f3c8da6f700 1 -- 172.21.15.174:0/3736468742 >> 172.21.15.174:0/3736468742 conn(0x7f3c880784d0 msgr2=0x7f3c880fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:04.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.855+0000 7f3c8da6f700 1 -- 172.21.15.174:0/3736468742 shutdown_connections 2024-04-26T18:18:04.858 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:04.855+0000 7f3c8da6f700 1 -- 172.21.15.174:0/3736468742 wait complete. 2024-04-26T18:18:04.858 INFO:teuthology.orchestra.run.smithi174.stderr:dumped monmap epoch 3 2024-04-26T18:18:05.395 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T18:18:05.395 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T18:18:05.643 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:18:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:05 smithi174 ceph-mon[32705]: from='client.? 172.21.15.174:0/3736468742' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:18:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:05 smithi174 ceph-mon[32705]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:05.764 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:05.764 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:05.764 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:05.764 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:05 smithi084 ceph-mon[29141]: from='client.? 172.21.15.174:0/3736468742' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T18:18:05.764 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:05 smithi084 ceph-mon[29141]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:05.765 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:05.765 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:06.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:07.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: mon.b calling monitor election 2024-04-26T18:18:07.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: mon.b calling monitor election 2024-04-26T18:18:07.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: mon.c calling monitor election 2024-04-26T18:18:07.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: mon.a calling monitor election 2024-04-26T18:18:07.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T18:18:07.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: monmap epoch 3 2024-04-26T18:18:07.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: fsid df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:18:07.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: last_changed 2024-04-26T18:17:59.445928+0000 2024-04-26T18:18:07.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: created 2024-04-26T18:16:03.305539+0000 2024-04-26T18:18:07.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: min_mon_release 19 (squid) 2024-04-26T18:18:07.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: election_strategy: 1 2024-04-26T18:18:07.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-04-26T18:18:07.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-04-26T18:18:07.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-04-26T18:18:07.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: fsmap 2024-04-26T18:18:07.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:18:07.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: mgrmap e13: x(active, since 61s) 2024-04-26T18:18:07.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: overall HEALTH_OK 2024-04-26T18:18:07.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.348 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.348 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.348 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.348 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.348 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.348 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.348 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.348 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.348 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.349 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:18:07.349 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:18:07.349 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: mon.b calling monitor election 2024-04-26T18:18:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: mon.b calling monitor election 2024-04-26T18:18:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: mon.c calling monitor election 2024-04-26T18:18:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: mon.a calling monitor election 2024-04-26T18:18:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: monmap epoch 3 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: fsid df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: last_changed 2024-04-26T18:17:59.445928+0000 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: created 2024-04-26T18:16:03.305539+0000 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: min_mon_release 19 (squid) 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: election_strategy: 1 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: fsmap 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: mgrmap e13: x(active, since 61s) 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: overall HEALTH_OK 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:18:07.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:18:07.441 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:07.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.459+0000 7fe16f9bb700 1 -- 172.21.15.84:0/203168917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe168103d30 msgr2=0x7fe168104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:07.465 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.459+0000 7fe16f9bb700 1 --2- 172.21.15.84:0/203168917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe168103d30 0x7fe168104110 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fe164009910 tx=0x7fe164009c20 comp rx=0 tx=0).stop 2024-04-26T18:18:07.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.459+0000 7fe16f9bb700 1 -- 172.21.15.84:0/203168917 shutdown_connections 2024-04-26T18:18:07.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.459+0000 7fe16f9bb700 1 --2- 172.21.15.84:0/203168917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe168103d30 0x7fe168104110 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:07.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.459+0000 7fe16f9bb700 1 -- 172.21.15.84:0/203168917 >> 172.21.15.84:0/203168917 conn(0x7fe1680fcf90 msgr2=0x7fe1680ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:07.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.460+0000 7fe16f9bb700 1 -- 172.21.15.84:0/203168917 shutdown_connections 2024-04-26T18:18:07.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.460+0000 7fe16f9bb700 1 -- 172.21.15.84:0/203168917 wait complete. 2024-04-26T18:18:07.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.460+0000 7fe16f9bb700 1 Processor -- start 2024-04-26T18:18:07.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.460+0000 7fe16f9bb700 1 -- start start 2024-04-26T18:18:07.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.460+0000 7fe16f9bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe168103d30 0x7fe168199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:07.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.460+0000 7fe16f9bb700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe168199f80 0x7fe16819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:07.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.460+0000 7fe16f9bb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1681a2ab0 0x7fe1681a4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:07.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.460+0000 7fe16f9bb700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe16810c460 con 0x7fe168103d30 2024-04-26T18:18:07.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.460+0000 7fe16f9bb700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe16810c2e0 con 0x7fe168199f80 2024-04-26T18:18:07.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.460+0000 7fe16f9bb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe16810c5e0 con 0x7fe1681a2ab0 2024-04-26T18:18:07.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16d757700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe168103d30 0x7fe168199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:07.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16d757700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe168103d30 0x7fe168199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35978/0 (socket says 172.21.15.84:35978) 2024-04-26T18:18:07.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16d757700 1 -- 172.21.15.84:0/2602265616 learned_addr learned my addr 172.21.15.84:0/2602265616 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:18:07.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16cf56700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe168199f80 0x7fe16819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:07.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16df58700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1681a2ab0 0x7fe1681a4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:07.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16d757700 1 -- 172.21.15.84:0/2602265616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1681a2ab0 msgr2=0x7fe1681a4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16d757700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1681a2ab0 0x7fe1681a4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16d757700 1 -- 172.21.15.84:0/2602265616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe168199f80 msgr2=0x7fe16819a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16d757700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe168199f80 0x7fe16819a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16d757700 1 -- 172.21.15.84:0/2602265616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1681a5670 con 0x7fe168103d30 2024-04-26T18:18:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16df58700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1681a2ab0 0x7fe1681a4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:18:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16cf56700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe168199f80 0x7fe16819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:18:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16d757700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe168103d30 0x7fe168199a40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe164004f80 tx=0x7fe164005060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe15e7fc700 1 -- 172.21.15.84:0/2602265616 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe164026070 con 0x7fe168103d30 2024-04-26T18:18:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe15e7fc700 1 -- 172.21.15.84:0/2602265616 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe16400b750 con 0x7fe168103d30 2024-04-26T18:18:07.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16f9bb700 1 -- 172.21.15.84:0/2602265616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1681a5900 con 0x7fe168103d30 2024-04-26T18:18:07.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe15e7fc700 1 -- 172.21.15.84:0/2602265616 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1640039e0 con 0x7fe168103d30 2024-04-26T18:18:07.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.461+0000 7fe16f9bb700 1 -- 172.21.15.84:0/2602265616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1681a5de0 con 0x7fe168103d30 2024-04-26T18:18:07.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.462+0000 7fe15e7fc700 1 -- 172.21.15.84:0/2602265616 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe164029070 con 0x7fe168103d30 2024-04-26T18:18:07.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.462+0000 7fe16f9bb700 1 -- 172.21.15.84:0/2602265616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe168064a80 con 0x7fe168103d30 2024-04-26T18:18:07.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.462+0000 7fe15e7fc700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe154039ae0 0x7fe15403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:07.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.462+0000 7fe15e7fc700 1 -- 172.21.15.84:0/2602265616 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe16404d660 con 0x7fe168103d30 2024-04-26T18:18:07.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.462+0000 7fe16cf56700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe154039ae0 0x7fe15403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:07.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.463+0000 7fe16cf56700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe154039ae0 0x7fe15403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe158009910 tx=0x7fe158008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:07.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.467+0000 7fe15e7fc700 1 -- 172.21.15.84:0/2602265616 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe164013070 con 0x7fe168103d30 2024-04-26T18:18:07.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: mon.b calling monitor election 2024-04-26T18:18:07.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: mon.b calling monitor election 2024-04-26T18:18:07.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: mon.c calling monitor election 2024-04-26T18:18:07.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: mon.a calling monitor election 2024-04-26T18:18:07.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-26T18:18:07.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: monmap epoch 3 2024-04-26T18:18:07.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: fsid df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:18:07.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: last_changed 2024-04-26T18:17:59.445928+0000 2024-04-26T18:18:07.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: created 2024-04-26T18:16:03.305539+0000 2024-04-26T18:18:07.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: min_mon_release 19 (squid) 2024-04-26T18:18:07.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: election_strategy: 1 2024-04-26T18:18:07.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: 0: [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] mon.a 2024-04-26T18:18:07.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: 1: [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] mon.c 2024-04-26T18:18:07.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: 2: [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] mon.b 2024-04-26T18:18:07.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: fsmap 2024-04-26T18:18:07.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T18:18:07.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: mgrmap e13: x(active, since 61s) 2024-04-26T18:18:07.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: overall HEALTH_OK 2024-04-26T18:18:07.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:07.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:18:07.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:18:07.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:07.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.678+0000 7fe16f9bb700 1 -- 172.21.15.84:0/2602265616 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fe1681a5fe0 con 0x7fe168103d30 2024-04-26T18:18:07.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.678+0000 7fe15e7fc700 1 -- 172.21.15.84:0/2602265616 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+275 (secure 0 0 0) 0x7fe164030090 con 0x7fe168103d30 2024-04-26T18:18:07.679 INFO:teuthology.orchestra.run.smithi084.stdout:# minimal ceph.conf for df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:18:07.679 INFO:teuthology.orchestra.run.smithi084.stdout:[global] 2024-04-26T18:18:07.679 INFO:teuthology.orchestra.run.smithi084.stdout: fsid = df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:18:07.679 INFO:teuthology.orchestra.run.smithi084.stdout: mon_host = [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] 2024-04-26T18:18:07.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.681+0000 7fe16f9bb700 1 -- 172.21.15.84:0/2602265616 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe154039ae0 msgr2=0x7fe15403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:07.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.681+0000 7fe16f9bb700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe154039ae0 0x7fe15403bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe158009910 tx=0x7fe158008040 comp rx=0 tx=0).stop 2024-04-26T18:18:07.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.681+0000 7fe16f9bb700 1 -- 172.21.15.84:0/2602265616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe168103d30 msgr2=0x7fe168199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:07.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.681+0000 7fe16f9bb700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe168103d30 0x7fe168199a40 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fe164004f80 tx=0x7fe164005060 comp rx=0 tx=0).stop 2024-04-26T18:18:07.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.681+0000 7fe16f9bb700 1 -- 172.21.15.84:0/2602265616 shutdown_connections 2024-04-26T18:18:07.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.681+0000 7fe16f9bb700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe154039ae0 0x7fe15403bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:07.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.681+0000 7fe16f9bb700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe168103d30 0x7fe168199a40 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:07.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.681+0000 7fe16f9bb700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1681a2ab0 0x7fe1681a4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:07.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.681+0000 7fe16f9bb700 1 --2- 172.21.15.84:0/2602265616 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe168199f80 0x7fe16819a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:07.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.681+0000 7fe16f9bb700 1 -- 172.21.15.84:0/2602265616 >> 172.21.15.84:0/2602265616 conn(0x7fe1680fcf90 msgr2=0x7fe168104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:07.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.681+0000 7fe16f9bb700 1 -- 172.21.15.84:0/2602265616 shutdown_connections 2024-04-26T18:18:07.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:07.681+0000 7fe16f9bb700 1 -- 172.21.15.84:0/2602265616 wait complete. 2024-04-26T18:18:08.282 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:08 smithi084 ceph-mon[29141]: Updating smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:18:08.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:08 smithi084 ceph-mon[29141]: Updating smithi084:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:18:08.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:08 smithi084 ceph-mon[29141]: Updating smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:18:08.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:08 smithi084 ceph-mon[29141]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T18:18:08.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:08 smithi084 ceph-mon[29141]: Reconfiguring daemon mon.a on smithi084 2024-04-26T18:18:08.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:08 smithi084 ceph-mon[29141]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:08.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:08 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:08.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:08 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2602265616' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:08.314 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T18:18:08.314 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:18:08.314 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T18:18:08.340 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:18:08.340 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:18:08.405 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T18:18:08.406 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T18:18:08.431 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:08 smithi103 ceph-mon[32727]: Updating smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:18:08.432 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:08 smithi103 ceph-mon[32727]: Updating smithi084:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:18:08.432 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:08 smithi103 ceph-mon[32727]: Updating smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:18:08.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:08 smithi103 ceph-mon[32727]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T18:18:08.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:08 smithi103 ceph-mon[32727]: Reconfiguring daemon mon.a on smithi084 2024-04-26T18:18:08.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:08 smithi103 ceph-mon[32727]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:08.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:08 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:08.433 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:08 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2602265616' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:08.437 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T18:18:08.437 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:18:08.438 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:08 smithi174 ceph-mon[32705]: Updating smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:18:08.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:08 smithi174 ceph-mon[32705]: Updating smithi084:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:18:08.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:08 smithi174 ceph-mon[32705]: Updating smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:18:08.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:08 smithi174 ceph-mon[32705]: Reconfiguring mon.a (unknown last config time)... 2024-04-26T18:18:08.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:08 smithi174 ceph-mon[32705]: Reconfiguring daemon mon.a on smithi084 2024-04-26T18:18:08.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:08 smithi174 ceph-mon[32705]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:08.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:08 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:18:08.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:08 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2602265616' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:08.503 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T18:18:08.504 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T18:18:08.535 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T18:18:08.535 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:18:08.602 INFO:tasks.cephadm:Adding mgr.x on smithi084 2024-04-26T18:18:08.602 INFO:tasks.cephadm:Adding mgr.y on smithi103 2024-04-26T18:18:08.602 INFO:tasks.cephadm:Adding mgr.z on smithi174 2024-04-26T18:18:08.603 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi084=x;smithi103=y;smithi174=z' 2024-04-26T18:18:08.887 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:18:10.330 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.330+0000 7f4f98af8700 1 -- 172.21.15.174:0/2026344682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f94102fb0 msgr2=0x7f4f941033d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:10.331 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.330+0000 7f4f98af8700 1 --2- 172.21.15.174:0/2026344682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f94102fb0 0x7f4f941033d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4f84009910 tx=0x7f4f84009c20 comp rx=0 tx=0).stop 2024-04-26T18:18:10.331 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.330+0000 7f4f98af8700 1 -- 172.21.15.174:0/2026344682 shutdown_connections 2024-04-26T18:18:10.331 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.330+0000 7f4f98af8700 1 --2- 172.21.15.174:0/2026344682 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f94102fb0 0x7f4f941033d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:10.331 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.330+0000 7f4f98af8700 1 -- 172.21.15.174:0/2026344682 >> 172.21.15.174:0/2026344682 conn(0x7f4f940fe550 msgr2=0x7f4f94100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:10.331 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.330+0000 7f4f98af8700 1 -- 172.21.15.174:0/2026344682 shutdown_connections 2024-04-26T18:18:10.331 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.330+0000 7f4f98af8700 1 -- 172.21.15.174:0/2026344682 wait complete. 2024-04-26T18:18:10.331 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:10 smithi174 ceph-mon[32705]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:10.331 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:10 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:10.331 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:10 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:10.331 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:10 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:18:10.331 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:10 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:18:10.332 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:10 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:10.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.331+0000 7f4f98af8700 1 Processor -- start 2024-04-26T18:18:10.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.331+0000 7f4f98af8700 1 -- start start 2024-04-26T18:18:10.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.331+0000 7f4f98af8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f94102fb0 0x7f4f941986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:10.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.331+0000 7f4f98af8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f94198c30 0x7f4f9419dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:10.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.331+0000 7f4f98af8700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f941990b0 0x7f4f941a0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:10.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.331+0000 7f4f98af8700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4f94108ef0 con 0x7f4f941990b0 2024-04-26T18:18:10.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.331+0000 7f4f98af8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4f94108d70 con 0x7f4f94198c30 2024-04-26T18:18:10.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.331+0000 7f4f98af8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f94109070 con 0x7f4f94102fb0 2024-04-26T18:18:10.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.331+0000 7f4f9259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f94102fb0 0x7f4f941986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:10.334 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.331+0000 7f4f9259c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f94102fb0 0x7f4f941986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:39944/0 (socket says 172.21.15.174:39944) 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.331+0000 7f4f9259c700 1 -- 172.21.15.174:0/3789769468 learned_addr learned my addr 172.21.15.174:0/3789769468 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.332+0000 7f4f91d9b700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f94198c30 0x7f4f9419dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.332+0000 7f4f92d9d700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f941990b0 0x7f4f941a0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.332+0000 7f4f9259c700 1 -- 172.21.15.174:0/3789769468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f94198c30 msgr2=0x7f4f9419dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.332+0000 7f4f9259c700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f94198c30 0x7f4f9419dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.332+0000 7f4f9259c700 1 -- 172.21.15.174:0/3789769468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f941990b0 msgr2=0x7f4f941a0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.332+0000 7f4f9259c700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f941990b0 0x7f4f941a0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.332+0000 7f4f9259c700 1 -- 172.21.15.174:0/3789769468 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f941a0b00 con 0x7f4f94102fb0 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.332+0000 7f4f9259c700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f94102fb0 0x7f4f941986f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4f84005e00 tx=0x7f4f84005020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.332+0000 7f4f837fe700 1 -- 172.21.15.174:0/3789769468 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f84026070 con 0x7f4f94102fb0 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.332+0000 7f4f837fe700 1 -- 172.21.15.174:0/3789769468 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8400b750 con 0x7f4f94102fb0 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.332+0000 7f4f98af8700 1 -- 172.21.15.174:0/3789769468 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f941a0d90 con 0x7f4f94102fb0 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.332+0000 7f4f837fe700 1 -- 172.21.15.174:0/3789769468 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f840038a0 con 0x7f4f94102fb0 2024-04-26T18:18:10.335 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.332+0000 7f4f98af8700 1 -- 172.21.15.174:0/3789769468 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f941a1240 con 0x7f4f94102fb0 2024-04-26T18:18:10.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.333+0000 7f4f837fe700 1 -- 172.21.15.174:0/3789769468 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f4f84029070 con 0x7f4f94102fb0 2024-04-26T18:18:10.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.333+0000 7f4f837fe700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4f7c039ae0 0x7f4f7c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:10.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.333+0000 7f4f837fe700 1 -- 172.21.15.174:0/3789769468 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4f8404e410 con 0x7f4f94102fb0 2024-04-26T18:18:10.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.333+0000 7f4f98af8700 1 -- 172.21.15.174:0/3789769468 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f9402e490 con 0x7f4f94102fb0 2024-04-26T18:18:10.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.334+0000 7f4f91d9b700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4f7c039ae0 0x7f4f7c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:10.336 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.334+0000 7f4f91d9b700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4f7c039ae0 0x7f4f7c03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009ad0 tx=0x7f4f88009210 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:10.339 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.339+0000 7f4f837fe700 1 -- 172.21.15.174:0/3789769468 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f84013070 con 0x7f4f94102fb0 2024-04-26T18:18:10.546 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.546+0000 7f4f98af8700 1 -- 172.21.15.174:0/3789769468 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi084=x;smithi103=y;smithi174=z", "target": ["mon-mgr", ""]}) -- 0x7f4f94063860 con 0x7f4f7c039ae0 2024-04-26T18:18:10.548 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:10 smithi103 ceph-mon[32727]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:10.548 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:10 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:10.549 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:10 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:10.549 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:10 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:18:10.549 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:10 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:18:10.549 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:10 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:10.555 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.555+0000 7f4f837fe700 1 -- 172.21.15.174:0/3789769468 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f4f94063860 con 0x7f4f7c039ae0 2024-04-26T18:18:10.555 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mgr update... 2024-04-26T18:18:10.557 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.557+0000 7f4f98af8700 1 -- 172.21.15.174:0/3789769468 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4f7c039ae0 msgr2=0x7f4f7c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:10.557 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.557+0000 7f4f98af8700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4f7c039ae0 0x7f4f7c03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f4f88009ad0 tx=0x7f4f88009210 comp rx=0 tx=0).stop 2024-04-26T18:18:10.557 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.557+0000 7f4f98af8700 1 -- 172.21.15.174:0/3789769468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f94102fb0 msgr2=0x7f4f941986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:10.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.557+0000 7f4f98af8700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f94102fb0 0x7f4f941986f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f4f84005e00 tx=0x7f4f84005020 comp rx=0 tx=0).stop 2024-04-26T18:18:10.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.557+0000 7f4f98af8700 1 -- 172.21.15.174:0/3789769468 shutdown_connections 2024-04-26T18:18:10.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.557+0000 7f4f98af8700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4f7c039ae0 0x7f4f7c03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:10.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.557+0000 7f4f98af8700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f941990b0 0x7f4f941a0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:10.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.557+0000 7f4f98af8700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f94102fb0 0x7f4f941986f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:10.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.557+0000 7f4f98af8700 1 --2- 172.21.15.174:0/3789769468 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f94198c30 0x7f4f9419dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:10.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.557+0000 7f4f98af8700 1 -- 172.21.15.174:0/3789769468 >> 172.21.15.174:0/3789769468 conn(0x7f4f940fe550 msgr2=0x7f4f940ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:10.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.557+0000 7f4f98af8700 1 -- 172.21.15.174:0/3789769468 shutdown_connections 2024-04-26T18:18:10.558 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:18:10.557+0000 7f4f98af8700 1 -- 172.21.15.174:0/3789769468 wait complete. 2024-04-26T18:18:10.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:10 smithi084 ceph-mon[29141]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:10.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:10 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:10.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:10 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:10.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:10 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:18:10.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:10 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:18:10.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:10 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:11.168 DEBUG:teuthology.orchestra.run.smithi103:mgr.y> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mgr.y.service 2024-04-26T18:18:11.171 DEBUG:teuthology.orchestra.run.smithi174:mgr.z> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mgr.z.service 2024-04-26T18:18:11.173 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T18:18:11.173 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:18:11.173 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T18:18:11.193 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T18:18:11.194 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_1 2024-04-26T18:18:11.252 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T18:18:11.252 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:18:11.252 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-26T18:18:11.252 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:18:11.252 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:18:11.252 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-04-26 18:17:17.919999556 +0000 2024-04-26T18:18:11.252 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-04-26 18:12:07.213853353 +0000 2024-04-26T18:18:11.252 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-04-26 18:12:07.213853353 +0000 2024-04-26T18:18:11.253 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-04-26 18:12:07.213853353 +0000 2024-04-26T18:18:11.253 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T18:18:11.307 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:11 smithi174 ceph-mon[32705]: Reconfiguring mon.b (monmap changed)... 2024-04-26T18:18:11.307 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:11 smithi174 ceph-mon[32705]: Reconfiguring daemon mon.b on smithi103 2024-04-26T18:18:11.307 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:11.308 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:11.308 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:11.308 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:18:11.308 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:18:11.308 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:11.320 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-04-26T18:18:11.320 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-04-26T18:18:11.320 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000233515 s, 2.2 MB/s 2024-04-26T18:18:11.322 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T18:18:11.380 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_2 2024-04-26T18:18:11.436 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-04-26 18:17:17.947999080 +0000 2024-04-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-04-26 18:12:07.550847305 +0000 2024-04-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-04-26 18:12:07.550847305 +0000 2024-04-26T18:18:11.437 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-04-26 18:12:07.550847305 +0000 2024-04-26T18:18:11.437 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T18:18:11.499 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:11 smithi084 ceph-mon[29141]: Reconfiguring mon.b (monmap changed)... 2024-04-26T18:18:11.499 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:11 smithi084 ceph-mon[29141]: Reconfiguring daemon mon.b on smithi103 2024-04-26T18:18:11.499 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:11.499 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:11.499 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:11.499 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:18:11.499 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:18:11.499 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:11.501 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-04-26T18:18:11.501 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-04-26T18:18:11.501 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000220329 s, 2.3 MB/s 2024-04-26T18:18:11.502 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T18:18:11.560 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_3 2024-04-26T18:18:11.620 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T18:18:11.620 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:18:11.620 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-26T18:18:11.620 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:18:11.620 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:18:11.620 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-04-26 18:17:17.971998672 +0000 2024-04-26T18:18:11.620 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-04-26 18:12:07.912840810 +0000 2024-04-26T18:18:11.620 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-04-26 18:12:07.912840810 +0000 2024-04-26T18:18:11.621 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-04-26 18:12:07.912840810 +0000 2024-04-26T18:18:11.621 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T18:18:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:11 smithi103 ceph-mon[32727]: Reconfiguring mon.b (monmap changed)... 2024-04-26T18:18:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:11 smithi103 ceph-mon[32727]: Reconfiguring daemon mon.b on smithi103 2024-04-26T18:18:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T18:18:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T18:18:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:11.683 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-04-26T18:18:11.683 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-04-26T18:18:11.684 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000238728 s, 2.1 MB/s 2024-04-26T18:18:11.685 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T18:18:11.742 DEBUG:teuthology.orchestra.run.smithi084:> stat /dev/vg_nvme/lv_4 2024-04-26T18:18:11.798 INFO:teuthology.orchestra.run.smithi084.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T18:18:11.798 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:18:11.798 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-26T18:18:11.798 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:18:11.798 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:18:11.799 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-04-26 18:17:17.996998247 +0000 2024-04-26T18:18:11.799 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-04-26 18:12:08.247834799 +0000 2024-04-26T18:18:11.799 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-04-26 18:12:08.247834799 +0000 2024-04-26T18:18:11.799 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-04-26 18:12:08.247834799 +0000 2024-04-26T18:18:11.799 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T18:18:11.861 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records in 2024-04-26T18:18:11.861 INFO:teuthology.orchestra.run.smithi084.stderr:1+0 records out 2024-04-26T18:18:11.861 INFO:teuthology.orchestra.run.smithi084.stderr:512 bytes copied, 0.000217934 s, 2.3 MB/s 2024-04-26T18:18:11.862 DEBUG:teuthology.orchestra.run.smithi084:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T18:18:11.919 DEBUG:teuthology.orchestra.run.smithi103:> set -ex 2024-04-26T18:18:11.919 DEBUG:teuthology.orchestra.run.smithi103:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T18:18:11.937 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T18:18:11.938 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_1 2024-04-26T18:18:11.995 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T18:18:11.995 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:18:11.995 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-26T18:18:11.995 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:18:11.995 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:18:11.995 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-26 18:17:58.899641491 +0000 2024-04-26T18:18:11.995 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-26 18:12:43.789524143 +0000 2024-04-26T18:18:11.995 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-26 18:12:43.789524143 +0000 2024-04-26T18:18:11.995 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-26 18:12:43.789524143 +0000 2024-04-26T18:18:11.996 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T18:18:12.061 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-26T18:18:12.061 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-26T18:18:12.061 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000272151 s, 1.9 MB/s 2024-04-26T18:18:12.063 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T18:18:12.121 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_2 2024-04-26T18:18:12.178 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T18:18:12.179 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:18:12.179 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-26T18:18:12.179 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:18:12.179 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:18:12.179 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-26 18:17:58.899641491 +0000 2024-04-26T18:18:12.179 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-26 18:12:44.121517518 +0000 2024-04-26T18:18:12.179 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-26 18:12:44.121517518 +0000 2024-04-26T18:18:12.179 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-26 18:12:44.121517518 +0000 2024-04-26T18:18:12.179 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T18:18:12.242 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-26T18:18:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-26T18:18:12.243 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000231221 s, 2.2 MB/s 2024-04-26T18:18:12.244 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T18:18:12.301 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_3 2024-04-26T18:18:12.357 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T18:18:12.357 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:18:12.357 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-26T18:18:12.358 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:18:12.358 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:18:12.358 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-26 18:17:58.899641491 +0000 2024-04-26T18:18:12.358 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-26 18:12:44.463510692 +0000 2024-04-26T18:18:12.358 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-26 18:12:44.463510692 +0000 2024-04-26T18:18:12.358 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-26 18:12:44.463510692 +0000 2024-04-26T18:18:12.358 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T18:18:12.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:12 smithi103 ceph-mon[32727]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi084=x;smithi103=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:12.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:12 smithi103 ceph-mon[32727]: Saving service mgr spec with placement smithi084=x;smithi103=y;smithi174=z;count:3 2024-04-26T18:18:12.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:12 smithi103 ceph-mon[32727]: Reconfiguring mon.c (monmap changed)... 2024-04-26T18:18:12.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:12 smithi103 ceph-mon[32727]: Reconfiguring daemon mon.c on smithi174 2024-04-26T18:18:12.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:12 smithi103 ceph-mon[32727]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:12.399 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-26T18:18:12.399 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-26T18:18:12.399 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000231117 s, 2.2 MB/s 2024-04-26T18:18:12.401 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T18:18:12.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:12 smithi174 ceph-mon[32705]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi084=x;smithi103=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:12.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:12 smithi174 ceph-mon[32705]: Saving service mgr spec with placement smithi084=x;smithi103=y;smithi174=z;count:3 2024-04-26T18:18:12.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:12 smithi174 ceph-mon[32705]: Reconfiguring mon.c (monmap changed)... 2024-04-26T18:18:12.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:12 smithi174 ceph-mon[32705]: Reconfiguring daemon mon.c on smithi174 2024-04-26T18:18:12.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:12 smithi174 ceph-mon[32705]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:12.458 DEBUG:teuthology.orchestra.run.smithi103:> stat /dev/vg_nvme/lv_4 2024-04-26T18:18:12.514 INFO:teuthology.orchestra.run.smithi103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T18:18:12.514 INFO:teuthology.orchestra.run.smithi103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:18:12.514 INFO:teuthology.orchestra.run.smithi103.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-04-26T18:18:12.514 INFO:teuthology.orchestra.run.smithi103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:18:12.514 INFO:teuthology.orchestra.run.smithi103.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:18:12.514 INFO:teuthology.orchestra.run.smithi103.stdout:Access: 2024-04-26 18:17:58.900641473 +0000 2024-04-26T18:18:12.515 INFO:teuthology.orchestra.run.smithi103.stdout:Modify: 2024-04-26 18:12:44.790504167 +0000 2024-04-26T18:18:12.515 INFO:teuthology.orchestra.run.smithi103.stdout:Change: 2024-04-26 18:12:44.790504167 +0000 2024-04-26T18:18:12.515 INFO:teuthology.orchestra.run.smithi103.stdout: Birth: 2024-04-26 18:12:44.790504167 +0000 2024-04-26T18:18:12.515 DEBUG:teuthology.orchestra.run.smithi103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T18:18:12.577 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records in 2024-04-26T18:18:12.577 INFO:teuthology.orchestra.run.smithi103.stderr:1+0 records out 2024-04-26T18:18:12.577 INFO:teuthology.orchestra.run.smithi103.stderr:512 bytes copied, 0.000216925 s, 2.4 MB/s 2024-04-26T18:18:12.578 DEBUG:teuthology.orchestra.run.smithi103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T18:18:12.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:12 smithi084 ceph-mon[29141]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi084=x;smithi103=y;smithi174=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:12.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:12 smithi084 ceph-mon[29141]: Saving service mgr spec with placement smithi084=x;smithi103=y;smithi174=z;count:3 2024-04-26T18:18:12.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:12 smithi084 ceph-mon[29141]: Reconfiguring mon.c (monmap changed)... 2024-04-26T18:18:12.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:12 smithi084 ceph-mon[29141]: Reconfiguring daemon mon.c on smithi174 2024-04-26T18:18:12.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:12 smithi084 ceph-mon[29141]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:12.635 DEBUG:teuthology.orchestra.run.smithi174:> set -ex 2024-04-26T18:18:12.635 DEBUG:teuthology.orchestra.run.smithi174:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T18:18:12.654 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T18:18:12.654 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_1 2024-04-26T18:18:12.710 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T18:18:12.710 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:18:12.710 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-26T18:18:12.710 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:18:12.710 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:18:12.710 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-04-26 18:17:49.675831710 +0000 2024-04-26T18:18:12.710 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-04-26 18:12:25.100997489 +0000 2024-04-26T18:18:12.711 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-04-26 18:12:25.100997489 +0000 2024-04-26T18:18:12.711 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-04-26 18:12:25.100997489 +0000 2024-04-26T18:18:12.711 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T18:18:12.774 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-04-26T18:18:12.775 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-04-26T18:18:12.775 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000261979 s, 2.0 MB/s 2024-04-26T18:18:12.776 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T18:18:12.832 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_2 2024-04-26T18:18:12.888 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T18:18:12.888 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:18:12.888 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-04-26T18:18:12.888 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:18:12.888 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:18:12.888 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-04-26 18:17:49.675831710 +0000 2024-04-26T18:18:12.888 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-04-26 18:12:25.449993294 +0000 2024-04-26T18:18:12.888 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-04-26 18:12:25.449993294 +0000 2024-04-26T18:18:12.889 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-04-26 18:12:25.449993294 +0000 2024-04-26T18:18:12.889 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T18:18:12.950 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-04-26T18:18:12.950 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-04-26T18:18:12.950 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000212119 s, 2.4 MB/s 2024-04-26T18:18:12.951 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T18:18:13.007 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_3 2024-04-26T18:18:13.061 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T18:18:13.062 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:18:13.062 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T18:18:13.062 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:18:13.062 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:18:13.062 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-04-26 18:17:49.675831710 +0000 2024-04-26T18:18:13.062 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-04-26 18:12:25.800989075 +0000 2024-04-26T18:18:13.062 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-04-26 18:12:25.800989075 +0000 2024-04-26T18:18:13.062 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-04-26 18:12:25.800989075 +0000 2024-04-26T18:18:13.063 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T18:18:13.124 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-04-26T18:18:13.124 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-04-26T18:18:13.124 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000194778 s, 2.6 MB/s 2024-04-26T18:18:13.126 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T18:18:13.181 DEBUG:teuthology.orchestra.run.smithi174:> stat /dev/vg_nvme/lv_4 2024-04-26T18:18:13.238 INFO:teuthology.orchestra.run.smithi174.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T18:18:13.238 INFO:teuthology.orchestra.run.smithi174.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T18:18:13.238 INFO:teuthology.orchestra.run.smithi174.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-26T18:18:13.238 INFO:teuthology.orchestra.run.smithi174.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:18:13.238 INFO:teuthology.orchestra.run.smithi174.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T18:18:13.238 INFO:teuthology.orchestra.run.smithi174.stdout:Access: 2024-04-26 18:17:49.676831699 +0000 2024-04-26T18:18:13.238 INFO:teuthology.orchestra.run.smithi174.stdout:Modify: 2024-04-26 18:12:26.144984940 +0000 2024-04-26T18:18:13.239 INFO:teuthology.orchestra.run.smithi174.stdout:Change: 2024-04-26 18:12:26.144984940 +0000 2024-04-26T18:18:13.239 INFO:teuthology.orchestra.run.smithi174.stdout: Birth: 2024-04-26 18:12:26.144984940 +0000 2024-04-26T18:18:13.239 DEBUG:teuthology.orchestra.run.smithi174:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T18:18:13.302 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records in 2024-04-26T18:18:13.302 INFO:teuthology.orchestra.run.smithi174.stderr:1+0 records out 2024-04-26T18:18:13.302 INFO:teuthology.orchestra.run.smithi174.stderr:512 bytes copied, 0.000236962 s, 2.2 MB/s 2024-04-26T18:18:13.304 DEBUG:teuthology.orchestra.run.smithi174:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T18:18:13.360 INFO:tasks.cephadm:Deploying osd.0 on smithi084 with /dev/vg_nvme/lv_4... 2024-04-26T18:18:13.360 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T18:18:13.418 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:13.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:13.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:13.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:13.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:13.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:13.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:18:13.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:18:13.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:18:13.419 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:13.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:13.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:13.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:13.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:13.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:13.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:13.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:18:13.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:18:13.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:18:13.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:13.596 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:18:13.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:13.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:13.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:13.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:13.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:13.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:13.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:18:13.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:18:13.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:18:13.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:14.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:14 smithi084 ceph-mon[29141]: Deploying daemon mgr.z on smithi174 2024-04-26T18:18:14.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:14 smithi084 ceph-mon[29141]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:14.438 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:14 smithi174 ceph-mon[32705]: Deploying daemon mgr.z on smithi174 2024-04-26T18:18:14.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:14 smithi174 ceph-mon[32705]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:14.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:14 smithi103 ceph-mon[32727]: Deploying daemon mgr.z on smithi174 2024-04-26T18:18:14.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:14 smithi103 ceph-mon[32727]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:14.905 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:14 smithi174 systemd[1]: Starting Ceph mgr.z for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:18:15.166 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:14 smithi174 podman[34547]: 2024-04-26 18:18:14.807569082 +0000 UTC m=+0.017145528 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:18:15.166 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:14 smithi174 podman[34547]: 2024-04-26 18:18:14.919815547 +0000 UTC m=+0.129392036 container create 04b25399f61fe7ca2dfb6b58cb0f5e509c066aee0e91f75c1f63a87ea424b3f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T18:18:15.166 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:15 smithi174 podman[34547]: 2024-04-26 18:18:15.139518474 +0000 UTC m=+0.349094922 container init 04b25399f61fe7ca2dfb6b58cb0f5e509c066aee0e91f75c1f63a87ea424b3f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-26T18:18:15.166 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:15 smithi174 podman[34547]: 2024-04-26 18:18:15.142086174 +0000 UTC m=+0.351662622 container start 04b25399f61fe7ca2dfb6b58cb0f5e509c066aee0e91f75c1f63a87ea424b3f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T18:18:15.439 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:15 smithi174 ceph-mgr[34566]: -- 172.21.15.174:0/2680056500 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555d2ee5c5a0 con 0x555d2ee30800 2024-04-26T18:18:15.439 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:15 smithi174 bash[34547]: 04b25399f61fe7ca2dfb6b58cb0f5e509c066aee0e91f75c1f63a87ea424b3f5 2024-04-26T18:18:15.439 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:15 smithi174 systemd[1]: Started Ceph mgr.z for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:18:15.439 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:15 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:15.295+0000 7fefa53bf200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:18:15.527 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T18:18:15.528 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T18:18:15.528 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-04-26T18:18:15.528 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-04-26T18:18:15.528 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113294 s, 926 MB/s 2024-04-26T18:18:15.528 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-04-26T18:18:15.887 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:15 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:15.480+0000 7fefa53bf200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:18:15.887 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:15 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:15.580+0000 7fefa53bf200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:18:15.887 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:15 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:15.886+0000 7fefa53bf200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:18:16.140 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_4 2024-04-26T18:18:16.189 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:15 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:15.970+0000 7fefa53bf200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:18:16.251 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:16 smithi103 ceph-mon[32727]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:16.251 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:16.251 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:16.251 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:16.251 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:18:16.251 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:18:16.251 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:18:16.251 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:16.374 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:18:16.495 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:16 smithi084 ceph-mon[29141]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:16.496 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:16.496 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:16.496 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:16.496 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:18:16.496 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:18:16.496 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:18:16.496 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:16.604 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:16 smithi174 ceph-mon[32705]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:16.604 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:16.604 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:16.604 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:16.604 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:18:16.604 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T18:18:16.604 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:18:16.604 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:17.473 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:17 smithi103 ceph-mon[32727]: Deploying daemon mgr.y on smithi103 2024-04-26T18:18:17.474 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:17 smithi103 systemd[1]: Starting Ceph mgr.y for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:18:17.489 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:17 smithi084 ceph-mon[29141]: Deploying daemon mgr.y on smithi103 2024-04-26T18:18:17.560 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:17 smithi174 ceph-mon[32705]: Deploying daemon mgr.y on smithi103 2024-04-26T18:18:17.561 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:17 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:17.213+0000 7fefa53bf200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:18:17.561 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:17 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:17.384+0000 7fefa53bf200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:18:17.561 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:17 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:17.560+0000 7fefa53bf200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:18:17.786 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:17 smithi103 podman[34103]: 2024-04-26 18:18:17.474078271 +0000 UTC m=+0.103389000 container create e973bf7c756de0be52e200c13ddc2e9305b1ae7ddcd210a157f828eb6a57f656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-26T18:18:17.787 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:17 smithi103 podman[34103]: 2024-04-26 18:18:17.388543172 +0000 UTC m=+0.017853869 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:18:17.787 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:17 smithi103 podman[34103]: 2024-04-26 18:18:17.678887555 +0000 UTC m=+0.308198239 container init e973bf7c756de0be52e200c13ddc2e9305b1ae7ddcd210a157f828eb6a57f656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-04-26T18:18:17.787 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:17 smithi103 podman[34103]: 2024-04-26 18:18:17.681450759 +0000 UTC m=+0.310761444 container start e973bf7c756de0be52e200c13ddc2e9305b1ae7ddcd210a157f828eb6a57f656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-04-26T18:18:17.787 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:17 smithi103 ceph-mgr[34122]: -- 172.21.15.103:0/3342707824 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5627df0c45a0 con 0x5627df099000 2024-04-26T18:18:17.787 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:17 smithi103 bash[34103]: e973bf7c756de0be52e200c13ddc2e9305b1ae7ddcd210a157f828eb6a57f656 2024-04-26T18:18:17.827 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.824+0000 7fdb8dda1700 1 -- 172.21.15.84:0/2448755940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb88109140 msgr2=0x7fdb8810b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.824+0000 7fdb8dda1700 1 --2- 172.21.15.84:0/2448755940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb88109140 0x7fdb8810b530 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000b0a0 tx=0x7fdb8000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:18:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.824+0000 7fdb8dda1700 1 -- 172.21.15.84:0/2448755940 shutdown_connections 2024-04-26T18:18:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.824+0000 7fdb8dda1700 1 --2- 172.21.15.84:0/2448755940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb88109140 0x7fdb8810b530 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.824+0000 7fdb8dda1700 1 --2- 172.21.15.84:0/2448755940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb88103ee0 0x7fdb881062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.824+0000 7fdb8dda1700 1 --2- 172.21.15.84:0/2448755940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb88106810 0x7fdb88108c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.824+0000 7fdb8dda1700 1 -- 172.21.15.84:0/2448755940 >> 172.21.15.84:0/2448755940 conn(0x7fdb880fd830 msgr2=0x7fdb880ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.824+0000 7fdb8dda1700 1 -- 172.21.15.84:0/2448755940 shutdown_connections 2024-04-26T18:18:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.824+0000 7fdb8dda1700 1 -- 172.21.15.84:0/2448755940 wait complete. 2024-04-26T18:18:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb8dda1700 1 Processor -- start 2024-04-26T18:18:17.828 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb8dda1700 1 -- start start 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb8dda1700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb88103ee0 0x7fdb881993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb8dda1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb88106810 0x7fdb88199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb8dda1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb88109140 0x7fdb881a09b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb8dda1700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8810ee10 con 0x7fdb88106810 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb8dda1700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8810ec90 con 0x7fdb88103ee0 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb8dda1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8810ef90 con 0x7fdb88109140 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb87fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb88106810 0x7fdb88199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb87fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb88106810 0x7fdb88199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42652/0 (socket says 172.21.15.84:42652) 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb87fff700 1 -- 172.21.15.84:0/3097158786 learned_addr learned my addr 172.21.15.84:0/3097158786 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb8cd9f700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb88103ee0 0x7fdb881993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb8d5a0700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb88109140 0x7fdb881a09b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb87fff700 1 -- 172.21.15.84:0/3097158786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb88109140 msgr2=0x7fdb881a09b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb87fff700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb88109140 0x7fdb881a09b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb87fff700 1 -- 172.21.15.84:0/3097158786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb88103ee0 msgr2=0x7fdb881993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:17.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb87fff700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb88103ee0 0x7fdb881993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.825+0000 7fdb87fff700 1 -- 172.21.15.84:0/3097158786 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb881a1050 con 0x7fdb88106810 2024-04-26T18:18:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.826+0000 7fdb87fff700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb88106810 0x7fdb88199930 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400d980 tx=0x7fdb7400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.826+0000 7fdb85ffb700 1 -- 172.21.15.84:0/3097158786 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb74016070 con 0x7fdb88106810 2024-04-26T18:18:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.826+0000 7fdb85ffb700 1 -- 172.21.15.84:0/3097158786 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb74009d80 con 0x7fdb88106810 2024-04-26T18:18:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.826+0000 7fdb8dda1700 1 -- 172.21.15.84:0/3097158786 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb881a1340 con 0x7fdb88106810 2024-04-26T18:18:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.826+0000 7fdb85ffb700 1 -- 172.21.15.84:0/3097158786 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7400f3c0 con 0x7fdb88106810 2024-04-26T18:18:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.826+0000 7fdb8dda1700 1 -- 172.21.15.84:0/3097158786 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb881a18d0 con 0x7fdb88106810 2024-04-26T18:18:17.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.827+0000 7fdb85ffb700 1 -- 172.21.15.84:0/3097158786 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fdb74019070 con 0x7fdb88106810 2024-04-26T18:18:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.827+0000 7fdb85ffb700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdb78039890 0x7fdb7803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.827+0000 7fdb85ffb700 1 -- 172.21.15.84:0/3097158786 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fdb7404c520 con 0x7fdb88106810 2024-04-26T18:18:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.827+0000 7fdb8dda1700 1 -- 172.21.15.84:0/3097158786 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb8806a110 con 0x7fdb88106810 2024-04-26T18:18:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.827+0000 7fdb8cd9f700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdb78039890 0x7fdb7803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:17.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.827+0000 7fdb8cd9f700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdb78039890 0x7fdb7803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c005950 tx=0x7fdb7c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:17.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:17.832+0000 7fdb85ffb700 1 -- 172.21.15.84:0/3097158786 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb74010460 con 0x7fdb88106810 2024-04-26T18:18:18.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:18.061+0000 7fdb8dda1700 1 -- 172.21.15.84:0/3097158786 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fdb8802e800 con 0x7fdb78039890 2024-04-26T18:18:18.108 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:17 smithi103 systemd[1]: Started Ceph mgr.y for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:18:18.109 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:17 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:17.835+0000 7fd25099b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:18:18.109 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:18 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:18.023+0000 7fd25099b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:18:18.149 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:17 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:17.845+0000 7fefa53bf200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:18:18.149 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:17 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:17.937+0000 7fefa53bf200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:18:18.150 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:18 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:18.149+0000 7fefa53bf200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:18:18.373 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:18 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:18.127+0000 7fd25099b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:18:18.373 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:18 smithi103 ceph-mon[32727]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:18.373 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:18 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:18.373 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:18 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:18.373 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:18 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:18.373 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:18 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:18.373 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:18 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:18.373 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:18 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:18.374 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:18 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:18.374 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:18 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:18.439 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:18 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:18.346+0000 7fefa53bf200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:18:18.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:18 smithi174 ceph-mon[32705]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:18.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:18 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:18.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:18 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:18.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:18 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:18.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:18 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:18.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:18 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:18.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:18 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:18.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:18 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:18.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:18 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:18.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:18 smithi084 ceph-mon[29141]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:18.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:18.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:18.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:18.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:18.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:18.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:18.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:18.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:18.627 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:18 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:18.460+0000 7fd25099b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:18:18.627 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:18 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:18.554+0000 7fd25099b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:18:19.189 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:18 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:18.982+0000 7fefa53bf200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:18:19.296 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:19 smithi084 ceph-mon[29141]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:19.506 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:19 smithi174 ceph-mon[32705]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:19.506 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:19 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:19.408+0000 7fefa53bf200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:18:19.506 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:19 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:19.506+0000 7fefa53bf200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:18:19.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:19 smithi103 ceph-mon[32727]: from='client.14223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:19.939 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:19 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:19.703+0000 7fefa53bf200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:18:20.127 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:19 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:19.828+0000 7fd25099b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:18:20.127 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:19 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:19.993+0000 7fd25099b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:18:20.441 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:20.160+0000 7fd25099b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:18:20.442 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:20.442+0000 7fd25099b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:18:20.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-mon[32727]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:20.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:20.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:20.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.442 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:18:20.443 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:18:20.443 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:20.443 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.540 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:20.217+0000 7fefa53bf200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:18:20.540 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:20.306+0000 7fefa53bf200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:18:20.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-mon[32705]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:20.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:20.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:20.541 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.542 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:18:20.542 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:18:20.542 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:20.542 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:20 smithi084 ceph-mon[29141]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T18:18:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T18:18:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:20.596 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:20.732 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:20.533+0000 7fd25099b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:18:20.732 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:20.732+0000 7fd25099b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:18:20.872 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:20.540+0000 7fefa53bf200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:18:20.872 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:20.669+0000 7fefa53bf200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:18:20.872 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:20.761+0000 7fefa53bf200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:18:20.873 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:20.872+0000 7fefa53bf200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:18:21.127 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:20 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:20.921+0000 7fd25099b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:18:21.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:21 smithi084 ceph-mon[29141]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T18:18:21.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:21 smithi084 ceph-mon[29141]: Reconfiguring daemon mgr.x on smithi084 2024-04-26T18:18:21.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:21 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1100088749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1076912-986e-4825-b1d1-5fa9d69bc07d"}]: dispatch 2024-04-26T18:18:21.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:21 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1100088749' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1076912-986e-4825-b1d1-5fa9d69bc07d"}]': finished 2024-04-26T18:18:21.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:21 smithi084 ceph-mon[29141]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T18:18:21.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:21.439 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:21 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:21.171+0000 7fefa53bf200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:18:21.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:21 smithi174 ceph-mon[32705]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T18:18:21.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:21 smithi174 ceph-mon[32705]: Reconfiguring daemon mgr.x on smithi084 2024-04-26T18:18:21.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:21 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1100088749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1076912-986e-4825-b1d1-5fa9d69bc07d"}]: dispatch 2024-04-26T18:18:21.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:21 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1100088749' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1076912-986e-4825-b1d1-5fa9d69bc07d"}]': finished 2024-04-26T18:18:21.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:21 smithi174 ceph-mon[32705]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T18:18:21.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:21.511 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:21 smithi103 ceph-mon[32727]: Reconfiguring mgr.x (unknown last config time)... 2024-04-26T18:18:21.512 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:21 smithi103 ceph-mon[32727]: Reconfiguring daemon mgr.x on smithi084 2024-04-26T18:18:21.512 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:21 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1100088749' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b1076912-986e-4825-b1d1-5fa9d69bc07d"}]: dispatch 2024-04-26T18:18:21.512 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:21 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1100088749' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b1076912-986e-4825-b1d1-5fa9d69bc07d"}]': finished 2024-04-26T18:18:21.512 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:21 smithi103 ceph-mon[32727]: osdmap e5: 1 total, 0 up, 1 in 2024-04-26T18:18:21.512 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:21.877 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:21 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:21.512+0000 7fd25099b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:18:22.189 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:21 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:21.757+0000 7fefa53bf200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:18:22.189 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:18:21 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:18:21.966+0000 7fefa53bf200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:18:22.205 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:21 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:21.924+0000 7fd25099b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:18:22.205 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:22.019+0000 7fd25099b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:18:22.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:22 smithi084 ceph-mon[29141]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:22.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:22 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1463883329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:22.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:22 smithi084 ceph-mon[29141]: Standby manager daemon z started 2024-04-26T18:18:22.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:22 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.174:0/2180180351' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:18:22.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:22 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.174:0/2180180351' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:18:22.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:22 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.174:0/2180180351' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:18:22.596 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:22 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.174:0/2180180351' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:18:22.627 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:22.216+0000 7fd25099b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:18:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:22 smithi103 ceph-mon[32727]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:22 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1463883329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:22 smithi103 ceph-mon[32727]: Standby manager daemon z started 2024-04-26T18:18:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:22 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.174:0/2180180351' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:18:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:22 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.174:0/2180180351' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:18:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:22 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.174:0/2180180351' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:18:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:22 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.174:0/2180180351' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:18:22.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:22 smithi174 ceph-mon[32705]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:22.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:22 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1463883329' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:22.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:22 smithi174 ceph-mon[32705]: Standby manager daemon z started 2024-04-26T18:18:22.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:22 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.174:0/2180180351' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:18:22.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:22 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.174:0/2180180351' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:18:22.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:22 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.174:0/2180180351' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:18:22.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:22 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.174:0/2180180351' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:18:22.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:22.762+0000 7fdb85ffb700 1 -- 172.21.15.84:0/3097158786 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7fdb740098d0 con 0x7fdb88106810 2024-04-26T18:18:23.068 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:22.730+0000 7fd25099b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:18:23.068 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:22.823+0000 7fd25099b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:18:23.377 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:23 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:23.068+0000 7fd25099b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:18:23.377 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:23 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:23.199+0000 7fd25099b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:18:23.377 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:23 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:23.292+0000 7fd25099b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:18:23.703 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:23 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:23.406+0000 7fd25099b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:18:23.703 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:23 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:23.703+0000 7fd25099b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:18:24.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:23 smithi084 ceph-mon[29141]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-26T18:18:24.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:23 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:18:24.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:23 smithi084 ceph-mon[29141]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:24.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:23 smithi103 ceph-mon[32727]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-26T18:18:24.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:23 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:18:24.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:23 smithi103 ceph-mon[32727]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:24.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:23 smithi174 ceph-mon[32705]: mgrmap e14: x(active, since 77s), standbys: z 2024-04-26T18:18:24.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:23 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:18:24.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:23 smithi174 ceph-mon[32705]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:24.627 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:24 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:24.295+0000 7fd25099b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:18:24.627 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:18:24 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:18:24.506+0000 7fd25099b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:18:24.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:24.776+0000 7fdb85ffb700 1 -- 172.21.15.84:0/3097158786 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fdb74036b50 con 0x7fdb88106810 2024-04-26T18:18:25.062 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:24 smithi084 ceph-mon[29141]: Standby manager daemon y started 2024-04-26T18:18:25.062 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:24 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.103:0/574866928' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:18:25.062 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:24 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.103:0/574866928' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:18:25.062 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:24 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.103:0/574866928' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:18:25.062 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:24 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.103:0/574866928' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:18:25.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:24 smithi103 ceph-mon[32727]: Standby manager daemon y started 2024-04-26T18:18:25.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:24 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.103:0/574866928' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:18:25.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:24 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.103:0/574866928' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:18:25.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:24 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.103:0/574866928' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:18:25.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:24 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.103:0/574866928' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:18:25.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:24 smithi174 ceph-mon[32705]: Standby manager daemon y started 2024-04-26T18:18:25.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:24 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.103:0/574866928' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:18:25.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:24 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.103:0/574866928' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:18:25.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:24 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.103:0/574866928' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:18:25.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:24 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.103:0/574866928' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:18:26.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:25 smithi084 ceph-mon[29141]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-26T18:18:26.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:25 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:18:26.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:25 smithi084 ceph-mon[29141]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:26.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:25 smithi103 ceph-mon[32727]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-26T18:18:26.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:25 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:18:26.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:25 smithi103 ceph-mon[32727]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:26.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:25 smithi174 ceph-mon[32705]: mgrmap e15: x(active, since 79s), standbys: z, y 2024-04-26T18:18:26.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:25 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:18:26.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:25 smithi174 ceph-mon[32705]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:27.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:27 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:27.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:27 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:27.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:27 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:27.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:27 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:27.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:27 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:27.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:27 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:27.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:27 smithi174 ceph-mon[32705]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:27.457 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:27 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:27.457 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:27 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:27.457 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:27 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:27.457 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:27 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:27.457 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:27 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:27.457 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:27 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:27.458 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:27 smithi084 ceph-mon[29141]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:27.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:27 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:27.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:27 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:27.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:27 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:27.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:27 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:27.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:27 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:27.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:27 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:27.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:27 smithi103 ceph-mon[32727]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:28.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:28 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T18:18:28.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:28 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:28.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:28 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T18:18:28.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:28 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:29.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:28 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T18:18:29.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:28 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:29.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:29 smithi084 ceph-mon[29141]: Deploying daemon osd.0 on smithi084 2024-04-26T18:18:29.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:29 smithi084 ceph-mon[29141]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:29 smithi103 ceph-mon[32727]: Deploying daemon osd.0 on smithi084 2024-04-26T18:18:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:29 smithi103 ceph-mon[32727]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:29 smithi174 ceph-mon[32705]: Deploying daemon osd.0 on smithi084 2024-04-26T18:18:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:29 smithi174 ceph-mon[32705]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:32.417 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:32 smithi084 ceph-mon[29141]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:32.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:32 smithi174 ceph-mon[32705]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:32.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:32 smithi103 ceph-mon[32727]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:34.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:34 smithi084 ceph-mon[29141]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:34.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:34 smithi174 ceph-mon[32705]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:34.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:34 smithi103 ceph-mon[32727]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:35.318 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:35 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:35.318 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:35 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:35.318 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:35 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:35.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:35 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:35.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:35 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:35.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:35 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:35 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:35 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:35 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:36.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:36 smithi084 ceph-mon[29141]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:36 smithi103 ceph-mon[32727]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:36.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:36 smithi174 ceph-mon[32705]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:37.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:37.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:37.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:37.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:37.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:37.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:37 smithi084 ceph-mon[29141]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:37.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:37.967+0000 7fdb85ffb700 1 -- 172.21.15.84:0/3097158786 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fdb8802e800 con 0x7fdb78039890 2024-04-26T18:18:37.968 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 0 on host 'smithi084' 2024-04-26T18:18:37.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:37.970+0000 7fdb8dda1700 1 -- 172.21.15.84:0/3097158786 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdb78039890 msgr2=0x7fdb7803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:37.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:37.970+0000 7fdb8dda1700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdb78039890 0x7fdb7803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c005950 tx=0x7fdb7c00b410 comp rx=0 tx=0).stop 2024-04-26T18:18:37.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:37.970+0000 7fdb8dda1700 1 -- 172.21.15.84:0/3097158786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb88106810 msgr2=0x7fdb88199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:37.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:37.970+0000 7fdb8dda1700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb88106810 0x7fdb88199930 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fdb7400d980 tx=0x7fdb7400dc90 comp rx=0 tx=0).stop 2024-04-26T18:18:37.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:37.970+0000 7fdb8dda1700 1 -- 172.21.15.84:0/3097158786 shutdown_connections 2024-04-26T18:18:37.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:37.970+0000 7fdb8dda1700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdb78039890 0x7fdb7803bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:37.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:37.970+0000 7fdb8dda1700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb88109140 0x7fdb881a09b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:37.971 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:37.970+0000 7fdb8dda1700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb88103ee0 0x7fdb881993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:37.971 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:37.970+0000 7fdb8dda1700 1 --2- 172.21.15.84:0/3097158786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb88106810 0x7fdb88199930 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:37.971 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:37.970+0000 7fdb8dda1700 1 -- 172.21.15.84:0/3097158786 >> 172.21.15.84:0/3097158786 conn(0x7fdb880fd830 msgr2=0x7fdb88104820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:37.971 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:37.970+0000 7fdb8dda1700 1 -- 172.21.15.84:0/3097158786 shutdown_connections 2024-04-26T18:18:37.971 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:18:37.970+0000 7fdb8dda1700 1 -- 172.21.15.84:0/3097158786 wait complete. 2024-04-26T18:18:38.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:38.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:38.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:38.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:38.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:38.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:37 smithi103 ceph-mon[32727]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:38.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:38.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:38.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:38.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:38.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:38.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:37 smithi174 ceph-mon[32705]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:38.663 DEBUG:teuthology.orchestra.run.smithi084:osd.0> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T18:18:38.681 INFO:tasks.cephadm:Deploying osd.1 on smithi103 with /dev/vg_nvme/lv_4... 2024-04-26T18:18:38.682 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T18:18:38.929 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:18:39.050 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:38 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:39.051 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:38 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:39.051 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:38 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:39.051 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:38 smithi103 ceph-mon[32727]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T18:18:39.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:38 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:39.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:38 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:39.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:38 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:39.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:38 smithi174 ceph-mon[32705]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T18:18:39.207 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:38 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:39.207 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:38 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:39.207 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:38 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:39.207 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:38 smithi084 ceph-mon[29141]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T18:18:40.227 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:39 smithi084 ceph-mon[29141]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T18:18:40.228 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:39 smithi084 ceph-mon[29141]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T18:18:40.228 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:39 smithi084 ceph-mon[29141]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-04-26T18:18:40.228 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:39 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:40.228 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:39 smithi084 ceph-mon[29141]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:40.253 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:39 smithi103 ceph-mon[32727]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T18:18:40.253 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:39 smithi103 ceph-mon[32727]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T18:18:40.253 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:39 smithi103 ceph-mon[32727]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-04-26T18:18:40.253 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:39 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:40.253 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:39 smithi103 ceph-mon[32727]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:40.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:39 smithi174 ceph-mon[32705]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T18:18:40.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:39 smithi174 ceph-mon[32705]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T18:18:40.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:39 smithi174 ceph-mon[32705]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-04-26T18:18:40.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:39 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:40.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:39 smithi174 ceph-mon[32705]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:40.594 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:18:40 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0[43734]: 2024-04-26T18:18:40.391+0000 7feeec05b700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T18:18:40.595 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:18:40 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0[43734]: 2024-04-26T18:18:40.395+0000 7feee3e46700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:18:40.822 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T18:18:40.823 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T18:18:40.823 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-04-26T18:18:40.823 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-04-26T18:18:40.823 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127327 s, 824 MB/s 2024-04-26T18:18:40.823 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-04-26T18:18:41.316 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:40 smithi084 ceph-mon[29141]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-04-26T18:18:41.316 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:40 smithi084 ceph-mon[29141]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T18:18:41.317 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:40 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:41.317 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:40 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:41.317 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:40 smithi084 ceph-mon[29141]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' 2024-04-26T18:18:41.366 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:40 smithi103 ceph-mon[32727]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-04-26T18:18:41.366 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:40 smithi103 ceph-mon[32727]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T18:18:41.366 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:40 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:41.366 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:40 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:41.366 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:40 smithi103 ceph-mon[32727]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' 2024-04-26T18:18:41.394 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_4 2024-04-26T18:18:41.442 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:40 smithi174 ceph-mon[32705]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-04-26T18:18:41.442 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:40 smithi174 ceph-mon[32705]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T18:18:41.442 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:40 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:41.442 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:40 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:41.442 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:40 smithi174 ceph-mon[32705]: from='osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011]' entity='osd.0' 2024-04-26T18:18:41.640 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:18:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: purged_snaps scrub starts 2024-04-26T18:18:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: purged_snaps scrub ok 2024-04-26T18:18:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011] boot 2024-04-26T18:18:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T18:18:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:42.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:42.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:42.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:42.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:42.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:42.330 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: purged_snaps scrub starts 2024-04-26T18:18:42.330 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: purged_snaps scrub ok 2024-04-26T18:18:42.330 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:42.331 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:42.331 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011] boot 2024-04-26T18:18:42.331 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T18:18:42.331 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:42.331 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:42.331 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:42.331 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:42.331 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:42.331 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:42.331 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:42.331 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: purged_snaps scrub starts 2024-04-26T18:18:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: purged_snaps scrub ok 2024-04-26T18:18:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T18:18:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: osd.0 [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011] boot 2024-04-26T18:18:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: osdmap e8: 1 total, 1 up, 1 in 2024-04-26T18:18:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:18:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:18:42.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:42.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:42.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:18:42.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:43.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.144+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/2302179634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9f4103f10 msgr2=0x7fa9f410a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:43.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.144+0000 7fa9f8ec2700 1 --2- 172.21.15.103:0/2302179634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9f4103f10 0x7fa9f410a7c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00b0d0 tx=0x7fa9ec00b3e0 comp rx=0 tx=0).stop 2024-04-26T18:18:43.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.145+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/2302179634 shutdown_connections 2024-04-26T18:18:43.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.145+0000 7fa9f8ec2700 1 --2- 172.21.15.103:0/2302179634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9f4103f10 0x7fa9f410a7c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:43.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.145+0000 7fa9f8ec2700 1 --2- 172.21.15.103:0/2302179634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9f4102370 0x7fa9f4102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:43.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.145+0000 7fa9f8ec2700 1 --2- 172.21.15.103:0/2302179634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9f4103570 0x7fa9f41039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:43.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.145+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/2302179634 >> 172.21.15.103:0/2302179634 conn(0x7fa9f40fd950 msgr2=0x7fa9f40ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:18:43.148 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.145+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/2302179634 shutdown_connections 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.145+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/2302179634 wait complete. 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.145+0000 7fa9f8ec2700 1 Processor -- start 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.145+0000 7fa9f8ec2700 1 -- start start 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f8ec2700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9f4102370 0x7fa9f41983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f8ec2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9f4103570 0x7fa9f4198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f8ec2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9f4103f10 0x7fa9f419f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f8ec2700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f410de40 con 0x7fa9f4103570 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f8ec2700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f410dcc0 con 0x7fa9f4102370 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f8ec2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa9f410dfc0 con 0x7fa9f4103f10 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f259c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9f4102370 0x7fa9f41983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f259c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9f4102370 0x7fa9f41983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:42218/0 (socket says 172.21.15.103:42218) 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f259c700 1 -- 172.21.15.103:0/1215007037 learned_addr learned my addr 172.21.15.103:0/1215007037 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f1d9b700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9f4103570 0x7fa9f4198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:43.149 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f2d9d700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9f4103f10 0x7fa9f419f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:43.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f259c700 1 -- 172.21.15.103:0/1215007037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9f4103f10 msgr2=0x7fa9f419f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:43.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f259c700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9f4103f10 0x7fa9f419f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:43.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f259c700 1 -- 172.21.15.103:0/1215007037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9f4103570 msgr2=0x7fa9f4198900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:18:43.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f259c700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9f4103570 0x7fa9f4198900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:18:43.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f259c700 1 -- 172.21.15.103:0/1215007037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9f41a0020 con 0x7fa9f4102370 2024-04-26T18:18:43.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f2d9d700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9f4103f10 0x7fa9f419f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:18:43.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f259c700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9f4102370 0x7fa9f41983c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800c990 tx=0x7fa9e800cca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:43.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9e37fe700 1 -- 172.21.15.103:0/1215007037 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8007b00 con 0x7fa9f4102370 2024-04-26T18:18:43.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/1215007037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9f41a0310 con 0x7fa9f4102370 2024-04-26T18:18:43.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9e37fe700 1 -- 172.21.15.103:0/1215007037 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9e8007ca0 con 0x7fa9f4102370 2024-04-26T18:18:43.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.146+0000 7fa9e37fe700 1 -- 172.21.15.103:0/1215007037 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8018660 con 0x7fa9f4102370 2024-04-26T18:18:43.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.147+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/1215007037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9f41a08a0 con 0x7fa9f4102370 2024-04-26T18:18:43.150 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.148+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/1215007037 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9f406a110 con 0x7fa9f4102370 2024-04-26T18:18:43.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.148+0000 7fa9e37fe700 1 -- 172.21.15.103:0/1215007037 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fa9e800f450 con 0x7fa9f4102370 2024-04-26T18:18:43.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.149+0000 7fa9e37fe700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa9dc0a3bb0 0x7fa9dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:18:43.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.149+0000 7fa9e37fe700 1 -- 172.21.15.103:0/1215007037 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fa9e80ce880 con 0x7fa9f4102370 2024-04-26T18:18:43.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.149+0000 7fa9f1d9b700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa9dc0a3bb0 0x7fa9dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:18:43.151 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.150+0000 7fa9f1d9b700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa9dc0a3bb0 0x7fa9dc0a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4005950 tx=0x7fa9e400a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:18:43.154 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.154+0000 7fa9e37fe700 1 -- 172.21.15.103:0/1215007037 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa9e8014030 con 0x7fa9f4102370 2024-04-26T18:18:43.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:42 smithi084 ceph-mon[29141]: Detected new or changed devices on smithi084 2024-04-26T18:18:43.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:42 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi084 to 17088M 2024-04-26T18:18:43.357 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:18:43.356+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/1215007037 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa9f4063860 con 0x7fa9dc0a3bb0 2024-04-26T18:18:43.357 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:42 smithi103 ceph-mon[32727]: Detected new or changed devices on smithi084 2024-04-26T18:18:43.357 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:42 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi084 to 17088M 2024-04-26T18:18:43.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:42 smithi174 ceph-mon[32705]: Detected new or changed devices on smithi084 2024-04-26T18:18:43.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:42 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi084 to 17088M 2024-04-26T18:18:44.246 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:43 smithi103 ceph-mon[32727]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T18:18:44.246 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:43 smithi103 ceph-mon[32727]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:44.246 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:43 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:44.246 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:43 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:44.246 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:43 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:44.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:43 smithi084 ceph-mon[29141]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T18:18:44.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:43 smithi084 ceph-mon[29141]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:44.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:43 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:44.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:43 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:44.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:43 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:44.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:43 smithi174 ceph-mon[32705]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T18:18:44.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:43 smithi174 ceph-mon[32705]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:44.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:43 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:18:44.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:43 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:18:44.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:43 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:45.118 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:44 smithi103 ceph-mon[32727]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:45.340 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:44 smithi174 ceph-mon[32705]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:45.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:44 smithi084 ceph-mon[29141]: from='client.24146 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:18:46.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:45 smithi103 ceph-mon[32727]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:46.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:45 smithi103 ceph-mon[32727]: from='client.? 172.21.15.103:0/1004128774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa93293d-9058-4804-ba4b-7e1e674b0fb8"}]: dispatch 2024-04-26T18:18:46.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:45 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa93293d-9058-4804-ba4b-7e1e674b0fb8"}]: dispatch 2024-04-26T18:18:46.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:45 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa93293d-9058-4804-ba4b-7e1e674b0fb8"}]': finished 2024-04-26T18:18:46.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:45 smithi103 ceph-mon[32727]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T18:18:46.279 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:18:46.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:45 smithi084 ceph-mon[29141]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:46.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:45 smithi084 ceph-mon[29141]: from='client.? 172.21.15.103:0/1004128774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa93293d-9058-4804-ba4b-7e1e674b0fb8"}]: dispatch 2024-04-26T18:18:46.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:45 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa93293d-9058-4804-ba4b-7e1e674b0fb8"}]: dispatch 2024-04-26T18:18:46.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:45 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa93293d-9058-4804-ba4b-7e1e674b0fb8"}]': finished 2024-04-26T18:18:46.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:45 smithi084 ceph-mon[29141]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T18:18:46.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:18:46.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:45 smithi174 ceph-mon[32705]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:46.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:45 smithi174 ceph-mon[32705]: from='client.? 172.21.15.103:0/1004128774' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa93293d-9058-4804-ba4b-7e1e674b0fb8"}]: dispatch 2024-04-26T18:18:46.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:45 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa93293d-9058-4804-ba4b-7e1e674b0fb8"}]: dispatch 2024-04-26T18:18:46.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:45 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa93293d-9058-4804-ba4b-7e1e674b0fb8"}]': finished 2024-04-26T18:18:46.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:45 smithi174 ceph-mon[32705]: osdmap e10: 2 total, 1 up, 2 in 2024-04-26T18:18:46.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:18:47.288 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:46 smithi103 ceph-mon[32727]: from='client.? 172.21.15.103:0/2237117338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:47.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:46 smithi084 ceph-mon[29141]: from='client.? 172.21.15.103:0/2237117338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:47.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:46 smithi174 ceph-mon[32705]: from='client.? 172.21.15.103:0/2237117338' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:18:48.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:47 smithi084 ceph-mon[29141]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:48.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:47 smithi103 ceph-mon[32727]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:48.438 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:47 smithi174 ceph-mon[32705]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:50.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:50 smithi174 ceph-mon[32705]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:50.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:50 smithi084 ceph-mon[29141]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:50 smithi103 ceph-mon[32727]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:52.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:52 smithi103 ceph-mon[32727]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:52.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:52 smithi174 ceph-mon[32705]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:52.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:52 smithi084 ceph-mon[29141]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:53.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:53 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T18:18:53.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:53 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:53.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:53 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T18:18:53.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:53 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:53.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:53 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T18:18:53.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:53 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:18:54.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:54 smithi103 ceph-mon[32727]: Deploying daemon osd.1 on smithi103 2024-04-26T18:18:54.278 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:54 smithi103 ceph-mon[32727]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:54.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:54 smithi174 ceph-mon[32705]: Deploying daemon osd.1 on smithi103 2024-04-26T18:18:54.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:54 smithi174 ceph-mon[32705]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:54.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:54 smithi084 ceph-mon[29141]: Deploying daemon osd.1 on smithi103 2024-04-26T18:18:54.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:54 smithi084 ceph-mon[29141]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:56.305 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:56 smithi103 ceph-mon[32727]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:56.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:56 smithi174 ceph-mon[32705]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:56.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:56 smithi084 ceph-mon[29141]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:58.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:58 smithi103 ceph-mon[32727]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:58.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:58 smithi174 ceph-mon[32705]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:58.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:58 smithi084 ceph-mon[29141]: pgmap v56: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:18:59 smithi103 ceph-mon[32727]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:18:59 smithi174 ceph-mon[32705]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:18:59.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:18:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:18:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:18:59 smithi084 ceph-mon[29141]: pgmap v57: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:19:02.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:02.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:02.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:02.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:02.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:02.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:02 smithi103 ceph-mon[32727]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:19:02.275 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:02.275+0000 7fa9e37fe700 1 -- 172.21.15.103:0/1215007037 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa9f4063860 con 0x7fa9dc0a3bb0 2024-04-26T18:19:02.275 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 1 on host 'smithi103' 2024-04-26T18:19:02.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:02.278+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/1215007037 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa9dc0a3bb0 msgr2=0x7fa9dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:02.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:02.278+0000 7fa9f8ec2700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa9dc0a3bb0 0x7fa9dc0a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4005950 tx=0x7fa9e400a300 comp rx=0 tx=0).stop 2024-04-26T18:19:02.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:02.278+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/1215007037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9f4102370 msgr2=0x7fa9f41983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:02.278 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:02.278+0000 7fa9f8ec2700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9f4102370 0x7fa9f41983c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800c990 tx=0x7fa9e800cca0 comp rx=0 tx=0).stop 2024-04-26T18:19:02.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:02.278+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/1215007037 shutdown_connections 2024-04-26T18:19:02.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:02.278+0000 7fa9f8ec2700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa9dc0a3bb0 0x7fa9dc0a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:02.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:02.278+0000 7fa9f8ec2700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9f4103f10 0x7fa9f419f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:02.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:02.278+0000 7fa9f8ec2700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9f4102370 0x7fa9f41983c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:02.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:02.278+0000 7fa9f8ec2700 1 --2- 172.21.15.103:0/1215007037 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9f4103570 0x7fa9f4198900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:02.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:02.278+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/1215007037 >> 172.21.15.103:0/1215007037 conn(0x7fa9f40fd950 msgr2=0x7fa9f4106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:02.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:02.278+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/1215007037 shutdown_connections 2024-04-26T18:19:02.279 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:02.279+0000 7fa9f8ec2700 1 -- 172.21.15.103:0/1215007037 wait complete. 2024-04-26T18:19:02.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:02.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:02.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:02.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:02.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:02.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:02 smithi084 ceph-mon[29141]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:19:02.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:02.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:02.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:02.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:02.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:02.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:02 smithi174 ceph-mon[32705]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:19:02.870 DEBUG:teuthology.orchestra.run.smithi103:osd.1> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T18:19:02.872 INFO:tasks.cephadm:Deploying osd.2 on smithi174 with /dev/vg_nvme/lv_4... 2024-04-26T18:19:02.872 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T18:19:03.114 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:19:03.532 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:03 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:03.532 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:03 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:03.533 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:03 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:03.533 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:03 smithi174 ceph-mon[32705]: from='osd.1 [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:19:03.533 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:03 smithi174 ceph-mon[32705]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:19:03.533 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:03 smithi174 ceph-mon[32705]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:19:03.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:03 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:03.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:03 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:03.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:03 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:03.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:03 smithi084 ceph-mon[29141]: from='osd.1 [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:19:03.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:03 smithi084 ceph-mon[29141]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:19:03.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:03 smithi084 ceph-mon[29141]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:19:03.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:03 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:03.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:03 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:03.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:03 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:03.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:03 smithi103 ceph-mon[32727]: from='osd.1 [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:19:03.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:03 smithi103 ceph-mon[32727]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T18:19:03.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:03 smithi103 ceph-mon[32727]: pgmap v59: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:19:04.374 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:04 smithi103 ceph-mon[32727]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T18:19:04.374 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:04 smithi103 ceph-mon[32727]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T18:19:04.374 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:04 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:04.374 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:04 smithi103 ceph-mon[32727]: from='osd.1 [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:19:04.374 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:04 smithi103 ceph-mon[32727]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:19:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:04 smithi174 ceph-mon[32705]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T18:19:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:04 smithi174 ceph-mon[32705]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T18:19:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:04 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:04.416 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:04 smithi174 ceph-mon[32705]: from='osd.1 [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:19:04.417 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:04 smithi174 ceph-mon[32705]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:19:04.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:04 smithi084 ceph-mon[29141]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T18:19:04.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:04 smithi084 ceph-mon[29141]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T18:19:04.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:04 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:04.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:04 smithi084 ceph-mon[29141]: from='osd.1 [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:19:04.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:04 smithi084 ceph-mon[29141]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:19:04.996 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T18:19:04.997 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T18:19:04.997 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-04-26T18:19:04.997 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-04-26T18:19:04.997 INFO:teuthology.orchestra.run.smithi174.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114105 s, 919 MB/s 2024-04-26T18:19:04.997 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-04-26T18:19:05.128 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:19:04 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1[37335]: 2024-04-26T18:19:04.696+0000 7fc3e19a6700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T18:19:05.128 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:19:04 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1[37335]: 2024-04-26T18:19:04.702+0000 7fc3d9791700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:19:05.584 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:05 smithi103 ceph-mon[32727]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-26T18:19:05.584 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:05 smithi103 ceph-mon[32727]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T18:19:05.585 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:05.585 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:05.585 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:05 smithi103 ceph-mon[32727]: from='osd.1 ' entity='osd.1' 2024-04-26T18:19:05.585 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:05 smithi103 ceph-mon[32727]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:19:05.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:05 smithi084 ceph-mon[29141]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-26T18:19:05.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:05 smithi084 ceph-mon[29141]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T18:19:05.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:05 smithi084 ceph-mon[29141]: from='osd.1 ' entity='osd.1' 2024-04-26T18:19:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:05 smithi084 ceph-mon[29141]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:19:05.599 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:05 smithi174 ceph-mon[32705]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-26T18:19:05.599 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:05 smithi174 ceph-mon[32705]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T18:19:05.599 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:05.599 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:05.599 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:05 smithi174 ceph-mon[32705]: from='osd.1 ' entity='osd.1' 2024-04-26T18:19:05.599 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:05 smithi174 ceph-mon[32705]: pgmap v62: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-04-26T18:19:05.626 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_4 2024-04-26T18:19:05.872 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:19:06.554 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: purged_snaps scrub starts 2024-04-26T18:19:06.554 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: purged_snaps scrub ok 2024-04-26T18:19:06.554 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:06.554 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: osd.1 [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657] boot 2024-04-26T18:19:06.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T18:19:06.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:06.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: Detected new or changed devices on smithi103 2024-04-26T18:19:06.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:06.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:06.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:06.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi103 to 17088M 2024-04-26T18:19:06.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:06.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:06.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:06.555 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:06.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: purged_snaps scrub starts 2024-04-26T18:19:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: purged_snaps scrub ok 2024-04-26T18:19:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: osd.1 [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657] boot 2024-04-26T18:19:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T18:19:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: Detected new or changed devices on smithi103 2024-04-26T18:19:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi103 to 17088M 2024-04-26T18:19:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:06.596 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: purged_snaps scrub starts 2024-04-26T18:19:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: purged_snaps scrub ok 2024-04-26T18:19:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: osd.1 [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657] boot 2024-04-26T18:19:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: osdmap e13: 2 total, 2 up, 2 in 2024-04-26T18:19:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:19:06.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: Detected new or changed devices on smithi103 2024-04-26T18:19:06.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:06.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:06.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:06.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi103 to 17088M 2024-04-26T18:19:06.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:06.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:06.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:06.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:07.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.321+0000 7f3b23f21700 1 -- 172.21.15.174:0/4140974420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b1c1035a0 msgr2=0x7f3b1c103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:07.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.321+0000 7f3b23f21700 1 --2- 172.21.15.174:0/4140974420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b1c1035a0 0x7f3b1c103a00 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3b18009990 tx=0x7f3b18009ca0 comp rx=0 tx=0).stop 2024-04-26T18:19:07.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.321+0000 7f3b23f21700 1 -- 172.21.15.174:0/4140974420 shutdown_connections 2024-04-26T18:19:07.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.321+0000 7f3b23f21700 1 --2- 172.21.15.174:0/4140974420 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c103f40 0x7f3b1c10a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:07.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.321+0000 7f3b23f21700 1 --2- 172.21.15.174:0/4140974420 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b1c1023a0 0x7f3b1c1027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:07.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.321+0000 7f3b23f21700 1 --2- 172.21.15.174:0/4140974420 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b1c1035a0 0x7f3b1c103a00 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:07.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.321+0000 7f3b23f21700 1 -- 172.21.15.174:0/4140974420 >> 172.21.15.174:0/4140974420 conn(0x7f3b1c0fd940 msgr2=0x7f3b1c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:07.322 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.322+0000 7f3b23f21700 1 -- 172.21.15.174:0/4140974420 shutdown_connections 2024-04-26T18:19:07.323 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.322+0000 7f3b23f21700 1 -- 172.21.15.174:0/4140974420 wait complete. 2024-04-26T18:19:07.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.322+0000 7f3b23f21700 1 Processor -- start 2024-04-26T18:19:07.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.322+0000 7f3b23f21700 1 -- start start 2024-04-26T18:19:07.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.322+0000 7f3b23f21700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b1c1023a0 0x7f3b1c106330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:07.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.322+0000 7f3b23f21700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b1c1035a0 0x7f3b1c104980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:07.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.322+0000 7f3b23f21700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c103f40 0x7f3b1c104ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:07.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.322+0000 7f3b23f21700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c10deb0 con 0x7f3b1c103f40 2024-04-26T18:19:07.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.322+0000 7f3b23f21700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c10dd30 con 0x7f3b1c1023a0 2024-04-26T18:19:07.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.322+0000 7f3b23f21700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b1c10e030 con 0x7f3b1c1035a0 2024-04-26T18:19:07.325 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b214bc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b1c1035a0 0x7f3b1c104980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b214bc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b1c1035a0 0x7f3b1c104980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:34890/0 (socket says 172.21.15.174:34890) 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b214bc700 1 -- 172.21.15.174:0/1055297123 learned_addr learned my addr 172.21.15.174:0/1055297123 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b224be700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c103f40 0x7f3b1c104ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b21cbd700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b1c1023a0 0x7f3b1c106330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b214bc700 1 -- 172.21.15.174:0/1055297123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b1c1023a0 msgr2=0x7f3b1c106330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b214bc700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b1c1023a0 0x7f3b1c106330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b214bc700 1 -- 172.21.15.174:0/1055297123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c103f40 msgr2=0x7f3b1c104ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b214bc700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c103f40 0x7f3b1c104ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b214bc700 1 -- 172.21.15.174:0/1055297123 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b1c105770 con 0x7f3b1c1035a0 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b214bc700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b1c1035a0 0x7f3b1c104980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b18006010 tx=0x7f3b180047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b12ffd700 1 -- 172.21.15.174:0/1055297123 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18027070 con 0x7f3b1c1035a0 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b23f21700 1 -- 172.21.15.174:0/1055297123 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b1c1a37a0 con 0x7f3b1c1035a0 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b12ffd700 1 -- 172.21.15.174:0/1055297123 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b18004100 con 0x7f3b1c1035a0 2024-04-26T18:19:07.326 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b12ffd700 1 -- 172.21.15.174:0/1055297123 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b18022d70 con 0x7f3b1c1035a0 2024-04-26T18:19:07.327 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.323+0000 7f3b23f21700 1 -- 172.21.15.174:0/1055297123 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b1c1a3bf0 con 0x7f3b1c1035a0 2024-04-26T18:19:07.327 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.324+0000 7f3b23f21700 1 -- 172.21.15.174:0/1055297123 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b1c06a110 con 0x7f3b1c1035a0 2024-04-26T18:19:07.327 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.325+0000 7f3b12ffd700 1 -- 172.21.15.174:0/1055297123 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f3b1802a070 con 0x7f3b1c1035a0 2024-04-26T18:19:07.327 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.326+0000 7f3b12ffd700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3b080a3bb0 0x7f3b080a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:07.328 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.326+0000 7f3b12ffd700 1 -- 172.21.15.174:0/1055297123 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f3b180cfa20 con 0x7f3b1c1035a0 2024-04-26T18:19:07.328 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.326+0000 7f3b21cbd700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3b080a3bb0 0x7f3b080a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:07.328 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.327+0000 7f3b21cbd700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3b080a3bb0 0x7f3b080a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c006fd0 tx=0x7f3b0c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:07.329 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.330+0000 7f3b12ffd700 1 -- 172.21.15.174:0/1055297123 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b18013070 con 0x7f3b1c1035a0 2024-04-26T18:19:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:07 smithi174 ceph-mon[32705]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:07.537 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:07.537+0000 7f3b23f21700 1 -- 172.21.15.174:0/1055297123 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f3b1c0ffb40 con 0x7f3b080a3bb0 2024-04-26T18:19:07.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:07 smithi084 ceph-mon[29141]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:07.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:07 smithi103 ceph-mon[32727]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:08.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:08 smithi174 ceph-mon[32705]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T18:19:08.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:08 smithi174 ceph-mon[32705]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:08.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:08 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:08.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:08 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:08.393 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:08 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:08.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:08 smithi084 ceph-mon[29141]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T18:19:08.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:08 smithi084 ceph-mon[29141]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:08.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:08 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:08.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:08 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:08.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:08 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:08.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:08 smithi103 ceph-mon[32727]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T18:19:08.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:08 smithi103 ceph-mon[32727]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:08.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:08 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:08.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:08 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:08.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:08 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:09.533 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:09 smithi174 ceph-mon[32705]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:09.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:09 smithi084 ceph-mon[29141]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:09.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:09 smithi103 ceph-mon[32727]: pgmap v66: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:10.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:10 smithi084 ceph-mon[29141]: from='client.? 172.21.15.174:0/382821691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf78a44f-80a2-4cd7-974f-5abdf771e650"}]: dispatch 2024-04-26T18:19:10.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:10 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf78a44f-80a2-4cd7-974f-5abdf771e650"}]: dispatch 2024-04-26T18:19:10.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:10 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf78a44f-80a2-4cd7-974f-5abdf771e650"}]': finished 2024-04-26T18:19:10.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:10 smithi084 ceph-mon[29141]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T18:19:10.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:10 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:10.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:10 smithi103 ceph-mon[32727]: from='client.? 172.21.15.174:0/382821691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf78a44f-80a2-4cd7-974f-5abdf771e650"}]: dispatch 2024-04-26T18:19:10.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:10 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf78a44f-80a2-4cd7-974f-5abdf771e650"}]: dispatch 2024-04-26T18:19:10.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:10 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf78a44f-80a2-4cd7-974f-5abdf771e650"}]': finished 2024-04-26T18:19:10.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:10 smithi103 ceph-mon[32727]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T18:19:10.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:10 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:10 smithi174 ceph-mon[32705]: from='client.? 172.21.15.174:0/382821691' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf78a44f-80a2-4cd7-974f-5abdf771e650"}]: dispatch 2024-04-26T18:19:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:10 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bf78a44f-80a2-4cd7-974f-5abdf771e650"}]: dispatch 2024-04-26T18:19:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:10 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bf78a44f-80a2-4cd7-974f-5abdf771e650"}]': finished 2024-04-26T18:19:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:10 smithi174 ceph-mon[32705]: osdmap e15: 3 total, 2 up, 3 in 2024-04-26T18:19:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:10 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:11.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:11 smithi084 ceph-mon[29141]: from='client.? 172.21.15.174:0/4039776285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:19:11.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:11 smithi084 ceph-mon[29141]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:11 smithi103 ceph-mon[32727]: from='client.? 172.21.15.174:0/4039776285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:19:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:11 smithi103 ceph-mon[32727]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:11 smithi174 ceph-mon[32705]: from='client.? 172.21.15.174:0/4039776285' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:19:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:11 smithi174 ceph-mon[32705]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:14.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:14 smithi174 ceph-mon[32705]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:14.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:14 smithi084 ceph-mon[29141]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:14.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:14 smithi103 ceph-mon[32727]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:16.387 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:16 smithi174 ceph-mon[32705]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:16.429 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:16 smithi103 ceph-mon[32727]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:16.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:16 smithi084 ceph-mon[29141]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:18.384 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:18 smithi174 ceph-mon[32705]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:18.384 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:18 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T18:19:18.384 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:18 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:18.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:18 smithi084 ceph-mon[29141]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:18.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T18:19:18.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:18.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:18 smithi103 ceph-mon[32727]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:18.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:18 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T18:19:18.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:18 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:19.287 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:19 smithi174 ceph-mon[32705]: Deploying daemon osd.2 on smithi174 2024-04-26T18:19:19.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:19 smithi084 ceph-mon[29141]: Deploying daemon osd.2 on smithi174 2024-04-26T18:19:19.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:19 smithi103 ceph-mon[32727]: Deploying daemon osd.2 on smithi174 2024-04-26T18:19:20.438 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:20 smithi174 ceph-mon[32705]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:20.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:20 smithi084 ceph-mon[29141]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:20 smithi103 ceph-mon[32727]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:22.267 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:22 smithi174 ceph-mon[32705]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:22.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:22 smithi084 ceph-mon[29141]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:22 smithi103 ceph-mon[32727]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:24.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:23 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:24.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:23 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:24.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:23 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:24.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:23 smithi084 ceph-mon[29141]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:24.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:23 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:24.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:23 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:24.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:23 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:24.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:23 smithi103 ceph-mon[32727]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:24.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:23 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:24.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:23 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:24.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:23 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:24.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:23 smithi174 ceph-mon[32705]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:26.201 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:26.201+0000 7f3b12ffd700 1 -- 172.21.15.174:0/1055297123 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3b1c0ffb40 con 0x7f3b080a3bb0 2024-04-26T18:19:26.202 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 2 on host 'smithi174' 2024-04-26T18:19:26.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:26.203+0000 7f3b23f21700 1 -- 172.21.15.174:0/1055297123 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3b080a3bb0 msgr2=0x7f3b080a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:26.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:26.203+0000 7f3b23f21700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3b080a3bb0 0x7f3b080a6070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c006fd0 tx=0x7f3b0c008040 comp rx=0 tx=0).stop 2024-04-26T18:19:26.204 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:26.203+0000 7f3b23f21700 1 -- 172.21.15.174:0/1055297123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b1c1035a0 msgr2=0x7f3b1c104980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:26.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:26.203+0000 7f3b23f21700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b1c1035a0 0x7f3b1c104980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3b18006010 tx=0x7f3b180047b0 comp rx=0 tx=0).stop 2024-04-26T18:19:26.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:26.203+0000 7f3b23f21700 1 -- 172.21.15.174:0/1055297123 shutdown_connections 2024-04-26T18:19:26.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:26.203+0000 7f3b23f21700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3b080a3bb0 0x7f3b080a6070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:26.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:26.203+0000 7f3b23f21700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b1c103f40 0x7f3b1c104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:26.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:26.203+0000 7f3b23f21700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b1c1023a0 0x7f3b1c106330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:26.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:26.203+0000 7f3b23f21700 1 --2- 172.21.15.174:0/1055297123 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b1c1035a0 0x7f3b1c104980 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:26.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:26.203+0000 7f3b23f21700 1 -- 172.21.15.174:0/1055297123 >> 172.21.15.174:0/1055297123 conn(0x7f3b1c0fd940 msgr2=0x7f3b1c0ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:26.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:26.204+0000 7f3b23f21700 1 -- 172.21.15.174:0/1055297123 shutdown_connections 2024-04-26T18:19:26.205 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:19:26.204+0000 7f3b23f21700 1 -- 172.21.15.174:0/1055297123 wait complete. 2024-04-26T18:19:26.438 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:26 smithi174 ceph-mon[32705]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:26.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:26 smithi084 ceph-mon[29141]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:26.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:26 smithi103 ceph-mon[32727]: pgmap v75: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:26.863 DEBUG:teuthology.orchestra.run.smithi174:osd.2> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T18:19:26.866 INFO:tasks.cephadm:Deploying osd.3 on smithi084 with /dev/vg_nvme/lv_3... 2024-04-26T18:19:26.866 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T18:19:27.116 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:19:27.438 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:27 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:27.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:27 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:27.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:27 smithi174 ceph-mon[32705]: from='osd.2 [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:19:27.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:27 smithi174 ceph-mon[32705]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:19:27.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:27 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:27.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:27 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:27.443 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:27 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:27.444 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:27 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:27.444 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:27 smithi084 ceph-mon[29141]: from='osd.2 [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:19:27.444 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:27 smithi084 ceph-mon[29141]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:19:27.444 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:27 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:27.444 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:27 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:27.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:27 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:27.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:27 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:27.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:27 smithi103 ceph-mon[32727]: from='osd.2 [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:19:27.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:27 smithi103 ceph-mon[32727]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T18:19:27.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:27 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:27.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:27 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:28.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:28 smithi174 ceph-mon[32705]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:28.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:28 smithi174 ceph-mon[32705]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T18:19:28.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:28 smithi174 ceph-mon[32705]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T18:19:28.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:28 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:28.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:28 smithi174 ceph-mon[32705]: from='osd.2 [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:19:28.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:28 smithi174 ceph-mon[32705]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:19:28.440 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:19:28 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2[37862]: 2024-04-26T18:19:28.193+0000 7f936da08700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T18:19:28.440 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:19:28 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2[37862]: 2024-04-26T18:19:28.196+0000 7f93697fb700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:19:28.566 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:28 smithi084 ceph-mon[29141]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:28.566 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:28 smithi084 ceph-mon[29141]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T18:19:28.566 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:28 smithi084 ceph-mon[29141]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T18:19:28.566 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:28 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:28.566 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:28 smithi084 ceph-mon[29141]: from='osd.2 [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:19:28.566 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:28 smithi084 ceph-mon[29141]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:19:28.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:28 smithi103 ceph-mon[32727]: pgmap v76: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:28.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:28 smithi103 ceph-mon[32727]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T18:19:28.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:28 smithi103 ceph-mon[32727]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T18:19:28.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:28 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:28.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:28 smithi103 ceph-mon[32727]: from='osd.2 [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:19:28.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:28 smithi103 ceph-mon[32727]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:19:29.307 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T18:19:29.307 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T18:19:29.307 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-04-26T18:19:29.307 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-04-26T18:19:29.307 INFO:teuthology.orchestra.run.smithi084.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124526 s, 842 MB/s 2024-04-26T18:19:29.307 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-04-26T18:19:29.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:29 smithi084 ceph-mon[29141]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-04-26T18:19:29.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:29 smithi084 ceph-mon[29141]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T18:19:29.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:29.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:29.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:29 smithi103 ceph-mon[32727]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-04-26T18:19:29.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:29 smithi103 ceph-mon[32727]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T18:19:29.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:29.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:29.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:29 smithi174 ceph-mon[32705]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-04-26T18:19:29.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:29 smithi174 ceph-mon[32705]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T18:19:29.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:29.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:30.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_3 2024-04-26T18:19:30.286 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:19:30.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:30 smithi174 ceph-mon[32705]: purged_snaps scrub starts 2024-04-26T18:19:30.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:30 smithi174 ceph-mon[32705]: purged_snaps scrub ok 2024-04-26T18:19:30.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:30 smithi174 ceph-mon[32705]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:30.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:30 smithi174 ceph-mon[32705]: osd.2 [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924] boot 2024-04-26T18:19:30.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:30 smithi174 ceph-mon[32705]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T18:19:30.335 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:30 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:30.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:30 smithi084 ceph-mon[29141]: purged_snaps scrub starts 2024-04-26T18:19:30.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:30 smithi084 ceph-mon[29141]: purged_snaps scrub ok 2024-04-26T18:19:30.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:30 smithi084 ceph-mon[29141]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:30.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:30 smithi084 ceph-mon[29141]: osd.2 [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924] boot 2024-04-26T18:19:30.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:30 smithi084 ceph-mon[29141]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T18:19:30.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:30 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:30 smithi103 ceph-mon[32727]: purged_snaps scrub starts 2024-04-26T18:19:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:30 smithi103 ceph-mon[32727]: purged_snaps scrub ok 2024-04-26T18:19:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:30 smithi103 ceph-mon[32727]: pgmap v79: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-26T18:19:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:30 smithi103 ceph-mon[32727]: osd.2 [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924] boot 2024-04-26T18:19:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:30 smithi103 ceph-mon[32727]: osdmap e18: 3 total, 3 up, 3 in 2024-04-26T18:19:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:30 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:19:31.404 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:31 smithi084 ceph-mon[29141]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T18:19:31.404 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:31 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.404 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:31 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.404 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:31 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:31.404 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:31 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.404 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:31 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:31.404 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:31 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:31.405 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:31 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.405 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:31 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:31.405 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:31 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:31.405 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:31 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:31.405 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:31 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.405 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:31 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:19:31.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:31 smithi103 ceph-mon[32727]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T18:19:31.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:31 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:31 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:31 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:31.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:31 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:31 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:31.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:31 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:31.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:31 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:31 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:31.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:31 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:31.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:31 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:31.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:31 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:31 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:19:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:31 smithi174 ceph-mon[32705]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T18:19:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:31 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:31 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:31 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:31 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:31 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:31 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:31.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:31 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:31 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:31.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:31 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:31.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:31 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:31.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:31 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:31.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:31 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:19:31.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.917+0000 7f899039b700 1 -- 172.21.15.84:0/3399245476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89881034a0 msgr2=0x7f89881038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:31.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.917+0000 7f899039b700 1 --2- 172.21.15.84:0/3399245476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89881034a0 0x7f89881038c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f89840099c0 tx=0x7f8984009cd0 comp rx=0 tx=0).stop 2024-04-26T18:19:31.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.917+0000 7f899039b700 1 -- 172.21.15.84:0/3399245476 shutdown_connections 2024-04-26T18:19:31.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.917+0000 7f899039b700 1 --2- 172.21.15.84:0/3399245476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f89881034a0 0x7f89881038c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:31.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.917+0000 7f899039b700 1 --2- 172.21.15.84:0/3399245476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8988105040 0x7f898810b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:31.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.917+0000 7f899039b700 1 --2- 172.21.15.84:0/3399245476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89881046a0 0x7f8988104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:31.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.917+0000 7f899039b700 1 -- 172.21.15.84:0/3399245476 >> 172.21.15.84:0/3399245476 conn(0x7f89880fea40 msgr2=0x7f8988100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:31.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.917+0000 7f899039b700 1 -- 172.21.15.84:0/3399245476 shutdown_connections 2024-04-26T18:19:31.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.917+0000 7f899039b700 1 -- 172.21.15.84:0/3399245476 wait complete. 2024-04-26T18:19:31.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f899039b700 1 Processor -- start 2024-04-26T18:19:31.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f899039b700 1 -- start start 2024-04-26T18:19:31.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f899039b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89881034a0 0x7f89880771c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:31.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f899039b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89881046a0 0x7f8988075810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:31.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f899039b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8988105040 0x7f8988075d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:31.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f899039b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f898810ef50 con 0x7f8988105040 2024-04-26T18:19:31.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f899039b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f898810edd0 con 0x7f89881046a0 2024-04-26T18:19:31.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f899039b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f898810f0d0 con 0x7f89881034a0 2024-04-26T18:19:31.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f898e938700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8988105040 0x7f8988075d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:31.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f898e938700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8988105040 0x7f8988075d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44616/0 (socket says 172.21.15.84:44616) 2024-04-26T18:19:31.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f898e938700 1 -- 172.21.15.84:0/2341976524 learned_addr learned my addr 172.21.15.84:0/2341976524 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:19:31.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f898d936700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89881046a0 0x7f8988075810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:31.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f898e137700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89881034a0 0x7f89880771c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:31.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f898e938700 1 -- 172.21.15.84:0/2341976524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89881034a0 msgr2=0x7f89880771c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:31.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f898e938700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89881034a0 0x7f89880771c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:31.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f898e938700 1 -- 172.21.15.84:0/2341976524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89881046a0 msgr2=0x7f8988075810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:31.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f898e938700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89881046a0 0x7f8988075810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:31.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f898e938700 1 -- 172.21.15.84:0/2341976524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8988076600 con 0x7f8988105040 2024-04-26T18:19:31.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f898e938700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8988105040 0x7f8988075d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f898000ef30 tx=0x7f898000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:31.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.918+0000 7f897f7fe700 1 -- 172.21.15.84:0/2341976524 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89800092e0 con 0x7f8988105040 2024-04-26T18:19:31.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.919+0000 7f899039b700 1 -- 172.21.15.84:0/2341976524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89881a3720 con 0x7f8988105040 2024-04-26T18:19:31.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.919+0000 7f897f7fe700 1 -- 172.21.15.84:0/2341976524 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8980004bb0 con 0x7f8988105040 2024-04-26T18:19:31.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.919+0000 7f899039b700 1 -- 172.21.15.84:0/2341976524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89881a3c30 con 0x7f8988105040 2024-04-26T18:19:31.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.919+0000 7f897f7fe700 1 -- 172.21.15.84:0/2341976524 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980004280 con 0x7f8988105040 2024-04-26T18:19:31.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.919+0000 7f899039b700 1 -- 172.21.15.84:0/2341976524 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f898806a110 con 0x7f8988105040 2024-04-26T18:19:31.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.921+0000 7f897f7fe700 1 -- 172.21.15.84:0/2341976524 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f898001b030 con 0x7f8988105040 2024-04-26T18:19:31.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.922+0000 7f897f7fe700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f89740a3bb0 0x7f89740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:31.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.922+0000 7f898e137700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f89740a3bb0 0x7f89740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:31.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.922+0000 7f897f7fe700 1 -- 172.21.15.84:0/2341976524 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7f89800cf420 con 0x7f8988105040 2024-04-26T18:19:31.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.922+0000 7f898e137700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f89740a3bb0 0x7f89740a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f898400b580 tx=0x7f8984005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:31.938 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:31.926+0000 7f897f7fe700 1 -- 172.21.15.84:0/2341976524 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8980022020 con 0x7f8988105040 2024-04-26T18:19:32.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:32.134+0000 7f899039b700 1 -- 172.21.15.84:0/2341976524 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f898802e8a0 con 0x7f89740a3bb0 2024-04-26T18:19:32.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 ceph-mon[29141]: Detected new or changed devices on smithi174 2024-04-26T18:19:32.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi174 to 17088M 2024-04-26T18:19:32.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 ceph-mon[29141]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:32.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T18:19:32.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 ceph-mon[29141]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T18:19:32.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:19:32.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:32.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:32.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:32.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:32 smithi103 ceph-mon[32727]: Detected new or changed devices on smithi174 2024-04-26T18:19:32.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:32 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi174 to 17088M 2024-04-26T18:19:32.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:32 smithi103 ceph-mon[32727]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:32.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:32 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T18:19:32.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:32 smithi103 ceph-mon[32727]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T18:19:32.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:32 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:19:32.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:32 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:32.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:32 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:32.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:32 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:32.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 ceph-mon[32705]: Detected new or changed devices on smithi174 2024-04-26T18:19:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi174 to 17088M 2024-04-26T18:19:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 ceph-mon[32705]: pgmap v82: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T18:19:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 ceph-mon[32705]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T18:19:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:19:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:33.011 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48296]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:19:33.011 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48296]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:19:33.011 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48296]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:19:33.011 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48296]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:19:33.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48299]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:19:33.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48299]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:19:33.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48299]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:19:33.012 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48299]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:19:33.012 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48286]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:19:33.012 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48286]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:19:33.012 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48286]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:19:33.012 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48286]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:19:33.013 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48289]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:19:33.013 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48289]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:19:33.013 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48289]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:19:33.013 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:19:32 smithi084 sudo[48289]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:19:33.127 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:19:32 smithi103 sudo[41311]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:19:33.127 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:19:32 smithi103 sudo[41311]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:19:33.127 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:19:32 smithi103 sudo[41311]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:19:33.127 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:19:32 smithi103 sudo[41311]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:19:33.127 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:19:32 smithi103 sudo[41314]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:19:33.127 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:19:32 smithi103 sudo[41314]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:19:33.127 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:19:32 smithi103 sudo[41314]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:19:33.127 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:19:32 smithi103 sudo[41314]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:19:33.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:32 smithi103 sudo[41317]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:19:33.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:32 smithi103 sudo[41317]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:19:33.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:32 smithi103 sudo[41317]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:19:33.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:32 smithi103 sudo[41317]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:19:33.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 sudo[41320]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:19:33.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 sudo[41320]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:19:33.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 sudo[41320]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:19:33.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 sudo[41320]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:19:33.189 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41915]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:19:33.189 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41915]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:19:33.189 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41915]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:19:33.189 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41915]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:19:33.189 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41918]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:19:33.189 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41918]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:19:33.189 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41918]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:19:33.190 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41918]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:19:33.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41921]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T18:19:33.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41921]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:19:33.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41921]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:19:33.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41921]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:19:33.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41924]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T18:19:33.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41924]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T18:19:33.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41924]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-26T18:19:33.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:32 smithi174 sudo[41924]: pam_unix(sudo:session): session closed for user root 2024-04-26T18:19:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:19:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T18:19:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:19:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:19:33.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:19:33.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:19:33.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:19:33.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:19:33.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:19:33.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:19:33.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:19:33.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:19:33.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:19:33.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:19:33.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:19:33.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:19:33.347 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:33 smithi084 ceph-mon[29141]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:19:33.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:33.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:19:33.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T18:19:33.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:33.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:33.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:19:33.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:33 smithi103 ceph-mon[32727]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:19:33.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='client.14289 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:33.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:19:33.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:19:33.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T18:19:33.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:19:33.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:19:33.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:19:33.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:33 smithi174 ceph-mon[32705]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T18:19:33.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:33.907+0000 7f897f7fe700 1 -- 172.21.15.84:0/2341976524 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8980012050 con 0x7f8988105040 2024-04-26T18:19:34.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:34 smithi084 ceph-mon[29141]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:34.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:34 smithi084 ceph-mon[29141]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T18:19:34.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:34 smithi084 ceph-mon[29141]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T18:19:34.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:34 smithi103 ceph-mon[32727]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:34.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:34 smithi103 ceph-mon[32727]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T18:19:34.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:34 smithi103 ceph-mon[32727]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T18:19:34.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:34 smithi174 ceph-mon[32705]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:34.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:34 smithi174 ceph-mon[32705]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T18:19:34.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:34 smithi174 ceph-mon[32705]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-26T18:19:35.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:35 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/791432379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55fd0035-bc76-4235-b8b5-5cf5ef770139"}]: dispatch 2024-04-26T18:19:35.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:35 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/791432379' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55fd0035-bc76-4235-b8b5-5cf5ef770139"}]': finished 2024-04-26T18:19:35.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:35 smithi084 ceph-mon[29141]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T18:19:35.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:35 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:35.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:35 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/791432379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55fd0035-bc76-4235-b8b5-5cf5ef770139"}]: dispatch 2024-04-26T18:19:35.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:35 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/791432379' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55fd0035-bc76-4235-b8b5-5cf5ef770139"}]': finished 2024-04-26T18:19:35.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:35 smithi103 ceph-mon[32727]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T18:19:35.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:35 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:35 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/791432379' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "55fd0035-bc76-4235-b8b5-5cf5ef770139"}]: dispatch 2024-04-26T18:19:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:35 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/791432379' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "55fd0035-bc76-4235-b8b5-5cf5ef770139"}]': finished 2024-04-26T18:19:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:35 smithi174 ceph-mon[32705]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T18:19:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:35 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:36.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:36 smithi084 ceph-mon[29141]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:36.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:36 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3423362037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:19:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:36 smithi103 ceph-mon[32727]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:36 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3423362037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:19:36.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:36 smithi174 ceph-mon[32705]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:36.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:36 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3423362037' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:19:38.573 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:38 smithi084 ceph-mon[29141]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:38 smithi103 ceph-mon[32727]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:38.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:38 smithi174 ceph-mon[32705]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:40.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:40 smithi084 ceph-mon[29141]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:40 smithi103 ceph-mon[32727]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:40 smithi174 ceph-mon[32705]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:42.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:42 smithi084 ceph-mon[29141]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:42.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:42 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T18:19:42.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:42 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:42.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:42 smithi103 ceph-mon[32727]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:42.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:42 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T18:19:42.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:42 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:42.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:42 smithi174 ceph-mon[32705]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:42.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:42 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T18:19:42.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:42 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:43.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:43 smithi084 ceph-mon[29141]: Deploying daemon osd.3 on smithi084 2024-04-26T18:19:43.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:43 smithi103 ceph-mon[32727]: Deploying daemon osd.3 on smithi084 2024-04-26T18:19:43.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:43 smithi174 ceph-mon[32705]: Deploying daemon osd.3 on smithi084 2024-04-26T18:19:44.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:44 smithi084 ceph-mon[29141]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:44.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:44 smithi103 ceph-mon[32727]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:44.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:44 smithi174 ceph-mon[32705]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:46.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:46 smithi174 ceph-mon[32705]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:46.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:46 smithi084 ceph-mon[29141]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:46.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:46 smithi103 ceph-mon[32727]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:48.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:48 smithi084 ceph-mon[29141]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:48.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:48 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:48.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:48 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:48.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:48 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:48 smithi103 ceph-mon[32727]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:48 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:48 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:48 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:48 smithi174 ceph-mon[32705]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:48 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:48 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:48 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:50.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:50 smithi084 ceph-mon[29141]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:50 smithi103 ceph-mon[32727]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:50.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:50 smithi174 ceph-mon[32705]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:51.009 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:51.008+0000 7f897f7fe700 1 -- 172.21.15.84:0/2341976524 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f898802e8a0 con 0x7f89740a3bb0 2024-04-26T18:19:51.009 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 3 on host 'smithi084' 2024-04-26T18:19:51.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:51.010+0000 7f899039b700 1 -- 172.21.15.84:0/2341976524 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f89740a3bb0 msgr2=0x7f89740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:51.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:51.010+0000 7f899039b700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f89740a3bb0 0x7f89740a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f898400b580 tx=0x7f8984005fb0 comp rx=0 tx=0).stop 2024-04-26T18:19:51.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:51.010+0000 7f899039b700 1 -- 172.21.15.84:0/2341976524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8988105040 msgr2=0x7f8988075d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:51.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:51.010+0000 7f899039b700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8988105040 0x7f8988075d50 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f898000ef30 tx=0x7f898000c3a0 comp rx=0 tx=0).stop 2024-04-26T18:19:51.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:51.011+0000 7f899039b700 1 -- 172.21.15.84:0/2341976524 shutdown_connections 2024-04-26T18:19:51.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:51.011+0000 7f899039b700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f89740a3bb0 0x7f89740a6070 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:51.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:51.011+0000 7f899039b700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8988105040 0x7f8988075d50 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:51.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:51.011+0000 7f899039b700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f89881034a0 0x7f89880771c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:51.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:51.011+0000 7f899039b700 1 --2- 172.21.15.84:0/2341976524 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f89881046a0 0x7f8988075810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:51.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:51.011+0000 7f899039b700 1 -- 172.21.15.84:0/2341976524 >> 172.21.15.84:0/2341976524 conn(0x7f89880fea40 msgr2=0x7f8988107a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:51.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:51.011+0000 7f899039b700 1 -- 172.21.15.84:0/2341976524 shutdown_connections 2024-04-26T18:19:51.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:19:51.011+0000 7f899039b700 1 -- 172.21.15.84:0/2341976524 wait complete. 2024-04-26T18:19:51.740 DEBUG:teuthology.orchestra.run.smithi084:osd.3> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T18:19:51.742 INFO:tasks.cephadm:Deploying osd.4 on smithi103 with /dev/vg_nvme/lv_3... 2024-04-26T18:19:51.742 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T18:19:51.988 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:19:52.111 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:52.111 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:52.111 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:51 smithi103 ceph-mon[32727]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:52.111 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:51 smithi103 ceph-mon[32727]: from='osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T18:19:52.111 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:52.111 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:52.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:52.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:52.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:51 smithi084 ceph-mon[29141]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:52.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:51 smithi084 ceph-mon[29141]: from='osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T18:19:52.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:52.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:52.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:52.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:52.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:51 smithi174 ceph-mon[32705]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:52.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:51 smithi174 ceph-mon[32705]: from='osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T18:19:52.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:52.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:53.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:53 smithi103 ceph-mon[32727]: from='osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T18:19:53.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:53 smithi103 ceph-mon[32727]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T18:19:53.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:53 smithi103 ceph-mon[32727]: from='osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-04-26T18:19:53.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:53 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:53.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:53 smithi084 ceph-mon[29141]: from='osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T18:19:53.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:53 smithi084 ceph-mon[29141]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T18:19:53.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:53 smithi084 ceph-mon[29141]: from='osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-04-26T18:19:53.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:53 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:53.346 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:19:53 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3[50414]: 2024-04-26T18:19:53.005+0000 7f4a36661700 -1 osd.3 0 waiting for initial osdmap 2024-04-26T18:19:53.346 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:19:53 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3[50414]: 2024-04-26T18:19:53.011+0000 7f4a32454700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:19:53.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:53 smithi174 ceph-mon[32705]: from='osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T18:19:53.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:53 smithi174 ceph-mon[32705]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T18:19:53.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:53 smithi174 ceph-mon[32705]: from='osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-04-26T18:19:53.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:53 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:53.998 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T18:19:53.998 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T18:19:53.998 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-04-26T18:19:53.998 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-04-26T18:19:53.998 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112636 s, 931 MB/s 2024-04-26T18:19:53.998 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-04-26T18:19:54.315 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:54 smithi103 ceph-mon[32727]: from='osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-04-26T18:19:54.315 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:54 smithi103 ceph-mon[32727]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T18:19:54.316 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:54 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:54.316 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:54 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:54.316 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:54 smithi103 ceph-mon[32727]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:54.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:54 smithi084 ceph-mon[29141]: from='osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-04-26T18:19:54.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:54 smithi084 ceph-mon[29141]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T18:19:54.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:54 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:54.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:54 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:54.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:54 smithi084 ceph-mon[29141]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:54.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:54 smithi174 ceph-mon[32705]: from='osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-04-26T18:19:54.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:54 smithi174 ceph-mon[32705]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T18:19:54.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:54 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:54.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:54 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:54.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:54 smithi174 ceph-mon[32705]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 88 MiB used, 268 GiB / 268 GiB avail 2024-04-26T18:19:54.665 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_3 2024-04-26T18:19:54.907 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:19:55.240 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:55 smithi084 ceph-mon[29141]: purged_snaps scrub starts 2024-04-26T18:19:55.240 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:55 smithi084 ceph-mon[29141]: purged_snaps scrub ok 2024-04-26T18:19:55.240 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:55 smithi084 ceph-mon[29141]: osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256] boot 2024-04-26T18:19:55.240 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:55 smithi084 ceph-mon[29141]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T18:19:55.241 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:55.241 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:55 smithi084 ceph-mon[29141]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T18:19:55.320 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:55 smithi103 ceph-mon[32727]: purged_snaps scrub starts 2024-04-26T18:19:55.320 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:55 smithi103 ceph-mon[32727]: purged_snaps scrub ok 2024-04-26T18:19:55.320 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:55 smithi103 ceph-mon[32727]: osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256] boot 2024-04-26T18:19:55.320 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:55 smithi103 ceph-mon[32727]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T18:19:55.320 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:55.321 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:55 smithi103 ceph-mon[32727]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T18:19:55.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:55 smithi174 ceph-mon[32705]: purged_snaps scrub starts 2024-04-26T18:19:55.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:55 smithi174 ceph-mon[32705]: purged_snaps scrub ok 2024-04-26T18:19:55.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:55 smithi174 ceph-mon[32705]: osd.3 [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256] boot 2024-04-26T18:19:55.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:55 smithi174 ceph-mon[32705]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T18:19:55.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:19:55.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:55 smithi174 ceph-mon[32705]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T18:19:56.248 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:19:56.248 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.248 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.248 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:56.248 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:56.248 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.248 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:56.249 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:56.249 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.249 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:56.249 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:56.249 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:56.249 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.249 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:56 smithi103 ceph-mon[32727]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T18:19:56.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:19:56.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:56.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:56.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:56.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:56.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:56.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:56.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:56.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:56 smithi084 ceph-mon[29141]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T18:19:56.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:19:56.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:56.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:19:56.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:56.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:56.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:19:56.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:56.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:19:56.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:19:56.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:56 smithi174 ceph-mon[32705]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T18:19:56.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.473+0000 7f1239b51700 1 -- 172.21.15.103:0/3324036215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234109170 msgr2=0x7f123410b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:56.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.473+0000 7f1239b51700 1 --2- 172.21.15.103:0/3324036215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234109170 0x7f123410b5e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f122c00b0a0 tx=0x7f122c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:19:56.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.473+0000 7f1239b51700 1 -- 172.21.15.103:0/3324036215 shutdown_connections 2024-04-26T18:19:56.472 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.473+0000 7f1239b51700 1 --2- 172.21.15.103:0/3324036215 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234109170 0x7f123410b5e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:56.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.473+0000 7f1239b51700 1 --2- 172.21.15.103:0/3324036215 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1234103f10 0x7f1234106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:56.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.473+0000 7f1239b51700 1 --2- 172.21.15.103:0/3324036215 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1234106840 0x7f1234108c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:56.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.473+0000 7f1239b51700 1 -- 172.21.15.103:0/3324036215 >> 172.21.15.103:0/3324036215 conn(0x7f12340fd920 msgr2=0x7f12340ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:19:56.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.473+0000 7f1239b51700 1 -- 172.21.15.103:0/3324036215 shutdown_connections 2024-04-26T18:19:56.473 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.473+0000 7f1239b51700 1 -- 172.21.15.103:0/3324036215 wait complete. 2024-04-26T18:19:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.473+0000 7f1239b51700 1 Processor -- start 2024-04-26T18:19:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.473+0000 7f1239b51700 1 -- start start 2024-04-26T18:19:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f1239b51700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1234103f10 0x7f1234199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f1239b51700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234106840 0x7f1234199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f1239b51700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1234109170 0x7f12341a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f1239b51700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f123410efe0 con 0x7f1234103f10 2024-04-26T18:19:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f1239b51700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f123410ee60 con 0x7f1234106840 2024-04-26T18:19:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f1239b51700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f123410f160 con 0x7f1234109170 2024-04-26T18:19:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f122affd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234106840 0x7f1234199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f122affd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234106840 0x7f1234199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:58086/0 (socket says 172.21.15.103:58086) 2024-04-26T18:19:56.475 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f122affd700 1 -- 172.21.15.103:0/3423481314 learned_addr learned my addr 172.21.15.103:0/3423481314 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T18:19:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f12337fe700 1 --2- 172.21.15.103:0/3423481314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1234103f10 0x7f1234199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f122affd700 1 -- 172.21.15.103:0/3423481314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1234109170 msgr2=0x7f12341a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f122affd700 1 --2- 172.21.15.103:0/3423481314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1234109170 0x7f12341a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f122affd700 1 -- 172.21.15.103:0/3423481314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1234103f10 msgr2=0x7f1234199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:19:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f122affd700 1 --2- 172.21.15.103:0/3423481314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1234103f10 0x7f1234199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:19:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f122affd700 1 -- 172.21.15.103:0/3423481314 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12341a13a0 con 0x7f1234106840 2024-04-26T18:19:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f122affd700 1 --2- 172.21.15.103:0/3423481314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234106840 0x7f1234199c80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f121c00b530 tx=0x7f121c00b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f12317fa700 1 -- 172.21.15.103:0/3423481314 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c015020 con 0x7f1234106840 2024-04-26T18:19:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f12317fa700 1 -- 172.21.15.103:0/3423481314 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f121c011ce0 con 0x7f1234106840 2024-04-26T18:19:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f12317fa700 1 -- 172.21.15.103:0/3423481314 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c010d50 con 0x7f1234106840 2024-04-26T18:19:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.474+0000 7f1239b51700 1 -- 172.21.15.103:0/3423481314 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12341a1690 con 0x7f1234106840 2024-04-26T18:19:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.475+0000 7f1239b51700 1 -- 172.21.15.103:0/3423481314 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12341a1ca0 con 0x7f1234106840 2024-04-26T18:19:56.476 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.475+0000 7f1239b51700 1 -- 172.21.15.103:0/3423481314 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f123406a110 con 0x7f1234106840 2024-04-26T18:19:56.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.477+0000 7f12317fa700 1 -- 172.21.15.103:0/3423481314 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f121c018020 con 0x7f1234106840 2024-04-26T18:19:56.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.477+0000 7f12317fa700 1 --2- 172.21.15.103:0/3423481314 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12200a3cc0 0x7f12200a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:19:56.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.477+0000 7f12317fa700 1 -- 172.21.15.103:0/3423481314 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(28..28 src has 1..28) ==== 3698+0+0 (secure 0 0 0) 0x7f121c0ceef0 con 0x7f1234106840 2024-04-26T18:19:56.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.477+0000 7f12337fe700 1 --2- 172.21.15.103:0/3423481314 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12200a3cc0 0x7f12200a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:19:56.477 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.478+0000 7f12337fe700 1 --2- 172.21.15.103:0/3423481314 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12200a3cc0 0x7f12200a6180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1224006fd0 tx=0x7f1224008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:19:56.481 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.481+0000 7f12317fa700 1 -- 172.21.15.103:0/3423481314 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f121c01f020 con 0x7f1234106840 2024-04-26T18:19:56.691 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:19:56.692+0000 7f1239b51700 1 -- 172.21.15.103:0/3423481314 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f1234063860 con 0x7f12200a3cc0 2024-04-26T18:19:57.302 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:57 smithi103 ceph-mon[32727]: Detected new or changed devices on smithi084 2024-04-26T18:19:57.302 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:57 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi084 to 8544M 2024-04-26T18:19:57.302 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:57.303 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:57.303 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:57.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:57 smithi084 ceph-mon[29141]: Detected new or changed devices on smithi084 2024-04-26T18:19:57.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:57 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi084 to 8544M 2024-04-26T18:19:57.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:57.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:57.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:57 smithi174 ceph-mon[32705]: Detected new or changed devices on smithi084 2024-04-26T18:19:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:57 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi084 to 8544M 2024-04-26T18:19:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:19:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:19:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:19:58.327 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:58 smithi103 ceph-mon[32727]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:58.327 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:58 smithi103 ceph-mon[32727]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T18:19:58.327 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:58 smithi103 ceph-mon[32727]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T18:19:58.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:58 smithi084 ceph-mon[29141]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:58.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:58 smithi084 ceph-mon[29141]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T18:19:58.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:58 smithi084 ceph-mon[29141]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T18:19:58.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:58 smithi174 ceph-mon[32705]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:19:58.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:58 smithi174 ceph-mon[32705]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T18:19:58.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:58 smithi174 ceph-mon[32705]: pgmap v105: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T18:19:59.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:19:59 smithi084 ceph-mon[29141]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T18:19:59.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:19:59 smithi103 ceph-mon[32727]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T18:19:59.438 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:19:59 smithi174 ceph-mon[32705]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T18:20:00.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:00 smithi103 ceph-mon[32727]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-04-26T18:20:00.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:00 smithi103 ceph-mon[32727]: from='client.? 172.21.15.103:0/2027556575' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c278a0f3-e436-4d1c-a009-f9165ebf3c60"}]: dispatch 2024-04-26T18:20:00.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:00 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c278a0f3-e436-4d1c-a009-f9165ebf3c60"}]: dispatch 2024-04-26T18:20:00.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:00 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c278a0f3-e436-4d1c-a009-f9165ebf3c60"}]': finished 2024-04-26T18:20:00.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:00 smithi103 ceph-mon[32727]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T18:20:00.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:00 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:00.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:00 smithi103 ceph-mon[32727]: overall HEALTH_OK 2024-04-26T18:20:00.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:00 smithi084 ceph-mon[29141]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-04-26T18:20:00.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:00 smithi084 ceph-mon[29141]: from='client.? 172.21.15.103:0/2027556575' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c278a0f3-e436-4d1c-a009-f9165ebf3c60"}]: dispatch 2024-04-26T18:20:00.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:00 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c278a0f3-e436-4d1c-a009-f9165ebf3c60"}]: dispatch 2024-04-26T18:20:00.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:00 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c278a0f3-e436-4d1c-a009-f9165ebf3c60"}]': finished 2024-04-26T18:20:00.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:00 smithi084 ceph-mon[29141]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T18:20:00.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:00 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:00.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:00 smithi084 ceph-mon[29141]: overall HEALTH_OK 2024-04-26T18:20:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:00 smithi174 ceph-mon[32705]: pgmap v107: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-04-26T18:20:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:00 smithi174 ceph-mon[32705]: from='client.? 172.21.15.103:0/2027556575' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c278a0f3-e436-4d1c-a009-f9165ebf3c60"}]: dispatch 2024-04-26T18:20:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:00 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c278a0f3-e436-4d1c-a009-f9165ebf3c60"}]: dispatch 2024-04-26T18:20:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:00 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c278a0f3-e436-4d1c-a009-f9165ebf3c60"}]': finished 2024-04-26T18:20:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:00 smithi174 ceph-mon[32705]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T18:20:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:00 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:00 smithi174 ceph-mon[32705]: overall HEALTH_OK 2024-04-26T18:20:01.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:01 smithi084 ceph-mon[29141]: from='client.? 172.21.15.103:0/1040413163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:20:01.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:01 smithi103 ceph-mon[32727]: from='client.? 172.21.15.103:0/1040413163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:20:01.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:01 smithi174 ceph-mon[32705]: from='client.? 172.21.15.103:0/1040413163' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:20:02.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:02 smithi084 ceph-mon[29141]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-26T18:20:02.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:02 smithi103 ceph-mon[32727]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-26T18:20:02.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:02 smithi174 ceph-mon[32705]: pgmap v109: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 113 KiB/s, 0 objects/s recovering 2024-04-26T18:20:04.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:04 smithi103 ceph-mon[32727]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:04.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:04 smithi174 ceph-mon[32705]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:04.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:04 smithi084 ceph-mon[29141]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:06.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:06 smithi174 ceph-mon[32705]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:06 smithi084 ceph-mon[29141]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:06.603 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:06 smithi103 ceph-mon[32727]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T18:20:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:07.553 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T18:20:07.553 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:07.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T18:20:07.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:08.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:08 smithi084 ceph-mon[29141]: Deploying daemon osd.4 on smithi103 2024-04-26T18:20:08.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:08 smithi084 ceph-mon[29141]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:08.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:08 smithi174 ceph-mon[32705]: Deploying daemon osd.4 on smithi103 2024-04-26T18:20:08.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:08 smithi174 ceph-mon[32705]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:08.583 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:08 smithi103 ceph-mon[32727]: Deploying daemon osd.4 on smithi103 2024-04-26T18:20:08.583 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:08 smithi103 ceph-mon[32727]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:10.323 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:10 smithi103 ceph-mon[32727]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:10.438 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:10 smithi174 ceph-mon[32705]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:10.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:10 smithi084 ceph-mon[29141]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:12.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:12 smithi103 ceph-mon[32727]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:12.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:12 smithi174 ceph-mon[32705]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:12.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:12 smithi084 ceph-mon[29141]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:14.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:14.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:14.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:14.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:13 smithi084 ceph-mon[29141]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:14.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:14.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:14.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:14.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:13 smithi103 ceph-mon[32727]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:14.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:14.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:14.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:14.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:13 smithi174 ceph-mon[32705]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:16.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:16 smithi103 ceph-mon[32727]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:16.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:16.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:16.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:16.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:16.272 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:16.314 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:20:16.314+0000 7f12317fa700 1 -- 172.21.15.103:0/3423481314 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f1234063860 con 0x7f12200a3cc0 2024-04-26T18:20:16.314 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 4 on host 'smithi103' 2024-04-26T18:20:16.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:20:16.317+0000 7f1239b51700 1 -- 172.21.15.103:0/3423481314 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12200a3cc0 msgr2=0x7f12200a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:16.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:20:16.317+0000 7f1239b51700 1 --2- 172.21.15.103:0/3423481314 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12200a3cc0 0x7f12200a6180 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f1224006fd0 tx=0x7f1224008040 comp rx=0 tx=0).stop 2024-04-26T18:20:16.316 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:20:16.317+0000 7f1239b51700 1 -- 172.21.15.103:0/3423481314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234106840 msgr2=0x7f1234199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:16.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:20:16.317+0000 7f1239b51700 1 --2- 172.21.15.103:0/3423481314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234106840 0x7f1234199c80 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f121c00b530 tx=0x7f121c00b8f0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:20:16.317+0000 7f1239b51700 1 -- 172.21.15.103:0/3423481314 shutdown_connections 2024-04-26T18:20:16.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:20:16.317+0000 7f1239b51700 1 --2- 172.21.15.103:0/3423481314 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12200a3cc0 0x7f12200a6180 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:20:16.317+0000 7f1239b51700 1 --2- 172.21.15.103:0/3423481314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1234103f10 0x7f1234199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:20:16.317+0000 7f1239b51700 1 --2- 172.21.15.103:0/3423481314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1234109170 0x7f12341a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:20:16.317+0000 7f1239b51700 1 --2- 172.21.15.103:0/3423481314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1234106840 0x7f1234199c80 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:16.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:20:16.317+0000 7f1239b51700 1 -- 172.21.15.103:0/3423481314 >> 172.21.15.103:0/3423481314 conn(0x7f12340fd920 msgr2=0x7f1234104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:16.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:20:16.317+0000 7f1239b51700 1 -- 172.21.15.103:0/3423481314 shutdown_connections 2024-04-26T18:20:16.317 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:20:16.317+0000 7f1239b51700 1 -- 172.21.15.103:0/3423481314 wait complete. 2024-04-26T18:20:16.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:16 smithi174 ceph-mon[32705]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:16.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:16.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:16.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:16.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:16.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:16.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:16 smithi084 ceph-mon[29141]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:16.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:16.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:16.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:16.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:16.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:16.957 DEBUG:teuthology.orchestra.run.smithi103:osd.4> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T18:20:16.959 INFO:tasks.cephadm:Deploying osd.5 on smithi174 with /dev/vg_nvme/lv_3... 2024-04-26T18:20:16.960 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T18:20:17.200 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:20:17.534 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:17.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:17.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:17.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:17 smithi103 ceph-mon[32727]: from='osd.4 [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:20:17.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:17 smithi103 ceph-mon[32727]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:20:17.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:17 smithi103 ceph-mon[32727]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:17.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:17.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:17.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:17.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:17 smithi084 ceph-mon[29141]: from='osd.4 [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:20:17.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:17 smithi084 ceph-mon[29141]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:20:17.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:17 smithi084 ceph-mon[29141]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:17.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:17.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:17.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:17.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:17 smithi174 ceph-mon[32705]: from='osd.4 [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:20:17.642 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:17 smithi174 ceph-mon[32705]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T18:20:17.643 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:17 smithi174 ceph-mon[32705]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:18.479 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:18 smithi103 ceph-mon[32727]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T18:20:18.479 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:18 smithi103 ceph-mon[32727]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T18:20:18.479 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:18 smithi103 ceph-mon[32727]: from='osd.4 [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:20:18.479 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:18 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:18.480 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:18 smithi103 ceph-mon[32727]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:20:18.480 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:20:18 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4[44113]: 2024-04-26T18:20:18.321+0000 7f4fe09d6700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T18:20:18.480 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:20:18 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4[44113]: 2024-04-26T18:20:18.326+0000 7f4fdc7c9700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:20:18.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:18 smithi174 ceph-mon[32705]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T18:20:18.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:18 smithi174 ceph-mon[32705]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T18:20:18.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:18 smithi174 ceph-mon[32705]: from='osd.4 [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:20:18.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:18 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:18.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:18 smithi174 ceph-mon[32705]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:20:18.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:18 smithi084 ceph-mon[29141]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T18:20:18.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:18 smithi084 ceph-mon[29141]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T18:20:18.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:18 smithi084 ceph-mon[29141]: from='osd.4 [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:20:18.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:18.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:18 smithi084 ceph-mon[29141]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:20:19.146 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T18:20:19.146 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T18:20:19.146 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-04-26T18:20:19.147 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-04-26T18:20:19.147 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115725 s, 906 MB/s 2024-04-26T18:20:19.147 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-04-26T18:20:19.468 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:19 smithi174 ceph-mon[32705]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-26T18:20:19.469 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:19 smithi174 ceph-mon[32705]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T18:20:19.469 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:19 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:19.469 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:19 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:19.469 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:19 smithi174 ceph-mon[32705]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:19.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:19 smithi084 ceph-mon[29141]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-26T18:20:19.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:19 smithi084 ceph-mon[29141]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T18:20:19.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:19 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:19.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:19 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:19.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:19 smithi084 ceph-mon[29141]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:19.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:19 smithi103 ceph-mon[32727]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-26T18:20:19.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:19 smithi103 ceph-mon[32727]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T18:20:19.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:19 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:19.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:19 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:19.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:19 smithi103 ceph-mon[32727]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-04-26T18:20:19.805 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_3 2024-04-26T18:20:20.038 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:20:20.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: purged_snaps scrub starts 2024-04-26T18:20:20.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: purged_snaps scrub ok 2024-04-26T18:20:20.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: osd.4 [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252] boot 2024-04-26T18:20:20.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T18:20:20.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:20.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: Detected new or changed devices on smithi103 2024-04-26T18:20:20.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:20.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:20.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:20.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:20.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi103 to 8544M 2024-04-26T18:20:20.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:20.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:20.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:20.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: purged_snaps scrub starts 2024-04-26T18:20:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: purged_snaps scrub ok 2024-04-26T18:20:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: osd.4 [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252] boot 2024-04-26T18:20:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T18:20:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: Detected new or changed devices on smithi103 2024-04-26T18:20:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi103 to 8544M 2024-04-26T18:20:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:20.596 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:20.596 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: purged_snaps scrub starts 2024-04-26T18:20:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: purged_snaps scrub ok 2024-04-26T18:20:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: osd.4 [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252] boot 2024-04-26T18:20:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T18:20:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:20:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: Detected new or changed devices on smithi103 2024-04-26T18:20:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:20.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:20.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi103 to 8544M 2024-04-26T18:20:20.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:20.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:20.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:20.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:21.550 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:21 smithi174 ceph-mon[32705]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T18:20:21.551 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:21 smithi174 ceph-mon[32705]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:21.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:21 smithi084 ceph-mon[29141]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T18:20:21.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:21 smithi084 ceph-mon[29141]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:21.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:21 smithi103 ceph-mon[32727]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T18:20:21.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:21 smithi103 ceph-mon[32727]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:21.807 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.806+0000 7f15997e6700 1 -- 172.21.15.174:0/2070541222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1594102380 msgr2=0x7f15941027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:21.807 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.806+0000 7f15997e6700 1 --2- 172.21.15.174:0/2070541222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1594102380 0x7f15941027a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1584009990 tx=0x7f1584009ca0 comp rx=0 tx=0).stop 2024-04-26T18:20:21.807 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.806+0000 7f15997e6700 1 -- 172.21.15.174:0/2070541222 shutdown_connections 2024-04-26T18:20:21.807 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.806+0000 7f15997e6700 1 --2- 172.21.15.174:0/2070541222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1594103f20 0x7f159410a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:21.807 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.806+0000 7f15997e6700 1 --2- 172.21.15.174:0/2070541222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1594102380 0x7f15941027a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:21.807 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.806+0000 7f15997e6700 1 --2- 172.21.15.174:0/2070541222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1594103580 0x7f15941039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:21.807 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.806+0000 7f15997e6700 1 -- 172.21.15.174:0/2070541222 >> 172.21.15.174:0/2070541222 conn(0x7f15940fd920 msgr2=0x7f15940ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:21.807 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.806+0000 7f15997e6700 1 -- 172.21.15.174:0/2070541222 shutdown_connections 2024-04-26T18:20:21.807 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.807+0000 7f15997e6700 1 -- 172.21.15.174:0/2070541222 wait complete. 2024-04-26T18:20:21.809 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.807+0000 7f15997e6700 1 Processor -- start 2024-04-26T18:20:21.809 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f15997e6700 1 -- start start 2024-04-26T18:20:21.809 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f15997e6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1594102380 0x7f159419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:21.809 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f15997e6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1594103580 0x7f159419bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:21.809 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f15997e6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1594103f20 0x7f15941a2e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:21.809 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f15997e6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f159410de60 con 0x7f1594103f20 2024-04-26T18:20:21.809 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f15997e6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f159410dce0 con 0x7f1594103580 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f15997e6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f159410dfe0 con 0x7f1594102380 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f1592ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1594102380 0x7f159419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f1592ffd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1594102380 0x7f159419b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:48786/0 (socket says 172.21.15.174:48786) 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f1592ffd700 1 -- 172.21.15.174:0/4033340586 learned_addr learned my addr 172.21.15.174:0/4033340586 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f1592ffd700 1 -- 172.21.15.174:0/4033340586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1594103580 msgr2=0x7f159419bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f15927fc700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1594103580 0x7f159419bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f15937fe700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1594103f20 0x7f15941a2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f1592ffd700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1594103580 0x7f159419bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f1592ffd700 1 -- 172.21.15.174:0/4033340586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1594103f20 msgr2=0x7f15941a2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f1592ffd700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1594103f20 0x7f15941a2e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f1592ffd700 1 -- 172.21.15.174:0/4033340586 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15941a34c0 con 0x7f1594102380 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f1592ffd700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1594102380 0x7f159419b860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f15840047b0 tx=0x7f1584003940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.808+0000 7f157bfff700 1 -- 172.21.15.174:0/4033340586 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1584027070 con 0x7f1594102380 2024-04-26T18:20:21.810 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.809+0000 7f15997e6700 1 -- 172.21.15.174:0/4033340586 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15941a3750 con 0x7f1594102380 2024-04-26T18:20:21.811 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.809+0000 7f157bfff700 1 -- 172.21.15.174:0/4033340586 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1584022470 con 0x7f1594102380 2024-04-26T18:20:21.811 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.809+0000 7f157bfff700 1 -- 172.21.15.174:0/4033340586 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f158400f6c0 con 0x7f1594102380 2024-04-26T18:20:21.811 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.809+0000 7f15997e6700 1 -- 172.21.15.174:0/4033340586 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15941a3c00 con 0x7f1594102380 2024-04-26T18:20:21.812 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.809+0000 7f15997e6700 1 -- 172.21.15.174:0/4033340586 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f159406a110 con 0x7f1594102380 2024-04-26T18:20:21.812 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.811+0000 7f157bfff700 1 -- 172.21.15.174:0/4033340586 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f158402a070 con 0x7f1594102380 2024-04-26T18:20:21.812 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.811+0000 7f157bfff700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f157c0a3c40 0x7f157c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:21.812 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.811+0000 7f157bfff700 1 -- 172.21.15.174:0/4033340586 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 4101+0+0 (secure 0 0 0) 0x7f15840d03a0 con 0x7f1594102380 2024-04-26T18:20:21.812 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.811+0000 7f15927fc700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f157c0a3c40 0x7f157c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:21.812 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.812+0000 7f15927fc700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f157c0a3c40 0x7f157c0a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1588005950 tx=0x7f1588005cf0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:21.816 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:21.816+0000 7f157bfff700 1 -- 172.21.15.174:0/4033340586 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f158405eff0 con 0x7f1594102380 2024-04-26T18:20:22.022 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:22.022+0000 7f15997e6700 1 -- 172.21.15.174:0/4033340586 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f15941a3f20 con 0x7f157c0a3c40 2024-04-26T18:20:22.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:22 smithi084 ceph-mon[29141]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:20:22.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:22 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:20:22.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:22 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:20:22.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:22 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:22 smithi103 ceph-mon[32727]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:20:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:22 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:20:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:22 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:20:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:22 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:22.639 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:22 smithi174 ceph-mon[32705]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:20:22.639 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:22 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:20:22.639 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:22 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:20:22.639 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:22 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:23.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:23 smithi084 ceph-mon[29141]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:23.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:23 smithi103 ceph-mon[32727]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:23.636 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:23 smithi174 ceph-mon[32705]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:25.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:24 smithi084 ceph-mon[29141]: from='client.? 172.21.15.174:0/385829711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6505be9b-c1b0-4d8b-944e-9920f90cb88c"}]: dispatch 2024-04-26T18:20:25.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:24 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6505be9b-c1b0-4d8b-944e-9920f90cb88c"}]: dispatch 2024-04-26T18:20:25.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:24 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6505be9b-c1b0-4d8b-944e-9920f90cb88c"}]': finished 2024-04-26T18:20:25.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:24 smithi084 ceph-mon[29141]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T18:20:25.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:24 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:25.117 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:24 smithi174 ceph-mon[32705]: from='client.? 172.21.15.174:0/385829711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6505be9b-c1b0-4d8b-944e-9920f90cb88c"}]: dispatch 2024-04-26T18:20:25.117 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:24 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6505be9b-c1b0-4d8b-944e-9920f90cb88c"}]: dispatch 2024-04-26T18:20:25.117 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:24 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6505be9b-c1b0-4d8b-944e-9920f90cb88c"}]': finished 2024-04-26T18:20:25.117 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:24 smithi174 ceph-mon[32705]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T18:20:25.117 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:24 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:25.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:24 smithi103 ceph-mon[32727]: from='client.? 172.21.15.174:0/385829711' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6505be9b-c1b0-4d8b-944e-9920f90cb88c"}]: dispatch 2024-04-26T18:20:25.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:24 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6505be9b-c1b0-4d8b-944e-9920f90cb88c"}]: dispatch 2024-04-26T18:20:25.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:24 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6505be9b-c1b0-4d8b-944e-9920f90cb88c"}]': finished 2024-04-26T18:20:25.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:24 smithi103 ceph-mon[32727]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T18:20:25.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:24 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:26.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:25 smithi084 ceph-mon[29141]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:26.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:25 smithi084 ceph-mon[29141]: from='client.? 172.21.15.174:0/4184390456' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:20:26.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:25 smithi103 ceph-mon[32727]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:26.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:25 smithi103 ceph-mon[32727]: from='client.? 172.21.15.174:0/4184390456' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:20:26.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:25 smithi174 ceph-mon[32705]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:26.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:25 smithi174 ceph-mon[32705]: from='client.? 172.21.15.174:0/4184390456' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:20:28.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:28 smithi103 ceph-mon[32727]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:28.434 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:28 smithi174 ceph-mon[32705]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:28.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:28 smithi084 ceph-mon[29141]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:30.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:30 smithi174 ceph-mon[32705]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:30.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:30 smithi084 ceph-mon[29141]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:30 smithi103 ceph-mon[32727]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:32.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:32 smithi174 ceph-mon[32705]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:32.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:32 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T18:20:32.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:32 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:32.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:32 smithi084 ceph-mon[29141]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:32.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:32 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T18:20:32.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:32 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:32.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:32 smithi103 ceph-mon[32727]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:32.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:32 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T18:20:32.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:32 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:33.345 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:33 smithi174 ceph-mon[32705]: Deploying daemon osd.5 on smithi174 2024-04-26T18:20:33.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:33 smithi084 ceph-mon[29141]: Deploying daemon osd.5 on smithi174 2024-04-26T18:20:33.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:33 smithi103 ceph-mon[32727]: Deploying daemon osd.5 on smithi174 2024-04-26T18:20:34.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:34 smithi174 ceph-mon[32705]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:34.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:34 smithi084 ceph-mon[29141]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:34.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:34 smithi103 ceph-mon[32727]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:36.546 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:36 smithi174 ceph-mon[32705]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:36.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:36 smithi084 ceph-mon[29141]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:36.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:36 smithi103 ceph-mon[32727]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:38.449 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:38 smithi174 ceph-mon[32705]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:38.449 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:38 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:38.450 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:38 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:38.450 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:38 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:38.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:38 smithi084 ceph-mon[29141]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:38.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:38 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:38.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:38 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:38.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:38 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:38 smithi103 ceph-mon[32727]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:38 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:38 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:38 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:40.518 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:40 smithi174 ceph-mon[32705]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:40.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:40 smithi084 ceph-mon[29141]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:40 smithi103 ceph-mon[32727]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:41.494 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 5 on host 'smithi174' 2024-04-26T18:20:41.494 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:41.494+0000 7f157bfff700 1 -- 172.21.15.174:0/4033340586 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f15941a3f20 con 0x7f157c0a3c40 2024-04-26T18:20:41.496 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:41.496+0000 7f15997e6700 1 -- 172.21.15.174:0/4033340586 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f157c0a3c40 msgr2=0x7f157c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:41.496 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:41.496+0000 7f15997e6700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f157c0a3c40 0x7f157c0a6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1588005950 tx=0x7f1588005cf0 comp rx=0 tx=0).stop 2024-04-26T18:20:41.496 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:41.496+0000 7f15997e6700 1 -- 172.21.15.174:0/4033340586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1594102380 msgr2=0x7f159419b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:41.496 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:41.496+0000 7f15997e6700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1594102380 0x7f159419b860 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f15840047b0 tx=0x7f1584003940 comp rx=0 tx=0).stop 2024-04-26T18:20:41.497 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:41.496+0000 7f15997e6700 1 -- 172.21.15.174:0/4033340586 shutdown_connections 2024-04-26T18:20:41.497 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:41.496+0000 7f15997e6700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f157c0a3c40 0x7f157c0a6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:41.497 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:41.496+0000 7f15997e6700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1594103f20 0x7f15941a2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:41.497 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:41.496+0000 7f15997e6700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1594102380 0x7f159419b860 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:41.497 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:41.496+0000 7f15997e6700 1 --2- 172.21.15.174:0/4033340586 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1594103580 0x7f159419bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:41.497 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:41.496+0000 7f15997e6700 1 -- 172.21.15.174:0/4033340586 >> 172.21.15.174:0/4033340586 conn(0x7f15940fd920 msgr2=0x7f159406bb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:41.497 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:41.496+0000 7f15997e6700 1 -- 172.21.15.174:0/4033340586 shutdown_connections 2024-04-26T18:20:41.497 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:20:41.496+0000 7f15997e6700 1 -- 172.21.15.174:0/4033340586 wait complete. 2024-04-26T18:20:41.766 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:41.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:41.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:41.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:41.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:41.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:41 smithi174 ceph-mon[32705]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:41.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:41.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:41.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:41.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:41 smithi174 ceph-mon[32705]: from='osd.5 [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:20:41.767 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:41 smithi174 ceph-mon[32705]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:20:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:41 smithi084 ceph-mon[29141]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:42.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:42.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:42.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:41 smithi084 ceph-mon[29141]: from='osd.5 [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:20:42.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:41 smithi084 ceph-mon[29141]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:20:42.111 DEBUG:teuthology.orchestra.run.smithi174:osd.5> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T18:20:42.113 INFO:tasks.cephadm:Deploying osd.6 on smithi084 with /dev/vg_nvme/lv_2... 2024-04-26T18:20:42.114 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T18:20:42.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:42.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:42.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:42.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:42.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:42.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:41 smithi103 ceph-mon[32727]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:42.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:42.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:42.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:20:42.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:41 smithi103 ceph-mon[32727]: from='osd.5 [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:20:42.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:41 smithi103 ceph-mon[32727]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T18:20:42.361 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:43.600 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:43 smithi174 ceph-mon[32705]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T18:20:43.600 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:43 smithi174 ceph-mon[32705]: from='osd.5 [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:20:43.600 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:43 smithi174 ceph-mon[32705]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T18:20:43.600 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:43 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:43.600 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:43 smithi174 ceph-mon[32705]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:20:43.600 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:43 smithi174 ceph-mon[32705]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:43.707 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:43 smithi084 ceph-mon[29141]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T18:20:43.707 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:43 smithi084 ceph-mon[29141]: from='osd.5 [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:20:43.707 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:43 smithi084 ceph-mon[29141]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T18:20:43.707 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:43 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:43.708 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:43 smithi084 ceph-mon[29141]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:20:43.708 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:43 smithi084 ceph-mon[29141]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:43.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:43 smithi103 ceph-mon[32727]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T18:20:43.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:43 smithi103 ceph-mon[32727]: from='osd.5 [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:20:43.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:43 smithi103 ceph-mon[32727]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T18:20:43.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:43 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:43.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:43 smithi103 ceph-mon[32727]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:20:43.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:43 smithi103 ceph-mon[32727]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-04-26T18:20:44.277 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T18:20:44.277 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T18:20:44.277 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-04-26T18:20:44.277 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-04-26T18:20:44.277 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114262 s, 918 MB/s 2024-04-26T18:20:44.278 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-04-26T18:20:44.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:20:43 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:20:43.943+0000 7f619501f700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T18:20:44.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:20:43 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:20:43.952+0000 7f618ce0a700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:20:44.657 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:44 smithi084 ceph-mon[29141]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-04-26T18:20:44.657 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:44 smithi084 ceph-mon[29141]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T18:20:44.657 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:44 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:44.657 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:44 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:44.657 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:44 smithi084 ceph-mon[29141]: from='osd.5 ' entity='osd.5' 2024-04-26T18:20:44.657 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:44 smithi084 ceph-mon[29141]: osd.5 [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768] boot 2024-04-26T18:20:44.657 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:44 smithi084 ceph-mon[29141]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T18:20:44.658 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:44 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:44.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:44 smithi174 ceph-mon[32705]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-04-26T18:20:44.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:44 smithi174 ceph-mon[32705]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T18:20:44.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:44 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:44.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:44 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:44.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:44 smithi174 ceph-mon[32705]: from='osd.5 ' entity='osd.5' 2024-04-26T18:20:44.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:44 smithi174 ceph-mon[32705]: osd.5 [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768] boot 2024-04-26T18:20:44.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:44 smithi174 ceph-mon[32705]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T18:20:44.780 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:44 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:44.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:44 smithi103 ceph-mon[32727]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-04-26T18:20:44.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:44 smithi103 ceph-mon[32727]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T18:20:44.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:44 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:44.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:44 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:44.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:44 smithi103 ceph-mon[32727]: from='osd.5 ' entity='osd.5' 2024-04-26T18:20:44.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:44 smithi103 ceph-mon[32727]: osd.5 [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768] boot 2024-04-26T18:20:44.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:44 smithi103 ceph-mon[32727]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T18:20:44.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:44 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:20:44.985 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_2 2024-04-26T18:20:45.227 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:20:45.578 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:45 smithi084 ceph-mon[29141]: purged_snaps scrub starts 2024-04-26T18:20:45.578 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:45 smithi084 ceph-mon[29141]: purged_snaps scrub ok 2024-04-26T18:20:45.578 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:45 smithi084 ceph-mon[29141]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:20:45.578 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:45.578 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:45.578 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:45.578 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:45.578 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:45.578 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:45.578 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:45.579 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:45.579 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:45 smithi084 ceph-mon[29141]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T18:20:45.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:45 smithi174 ceph-mon[32705]: purged_snaps scrub starts 2024-04-26T18:20:45.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:45 smithi174 ceph-mon[32705]: purged_snaps scrub ok 2024-04-26T18:20:45.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:45 smithi174 ceph-mon[32705]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:20:45.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:45.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:45.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:45.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:45.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:45.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:45.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:45.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:45.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:45 smithi174 ceph-mon[32705]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T18:20:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:45 smithi103 ceph-mon[32727]: purged_snaps scrub starts 2024-04-26T18:20:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:45 smithi103 ceph-mon[32727]: purged_snaps scrub ok 2024-04-26T18:20:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:45 smithi103 ceph-mon[32727]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:20:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:20:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:45.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:45.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:20:45.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:20:45.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:45 smithi103 ceph-mon[32727]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T18:20:46.754 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:46 smithi084 ceph-mon[29141]: Detected new or changed devices on smithi174 2024-04-26T18:20:46.755 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:46 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi174 to 8544M 2024-04-26T18:20:46.755 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:46 smithi084 ceph-mon[29141]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T18:20:46.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.754+0000 7f67fee70700 1 -- 172.21.15.84:0/3581389960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f8105040 msgr2=0x7f67f810b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.754+0000 7f67fee70700 1 --2- 172.21.15.84:0/3581389960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f8105040 0x7f67f810b8f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f67f00099c0 tx=0x7f67f0009cd0 comp rx=0 tx=0).stop 2024-04-26T18:20:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.754+0000 7f67fee70700 1 -- 172.21.15.84:0/3581389960 shutdown_connections 2024-04-26T18:20:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.754+0000 7f67fee70700 1 --2- 172.21.15.84:0/3581389960 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f8105040 0x7f67f810b8f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.754+0000 7f67fee70700 1 --2- 172.21.15.84:0/3581389960 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f81034a0 0x7f67f81038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.754+0000 7f67fee70700 1 --2- 172.21.15.84:0/3581389960 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67f81046a0 0x7f67f8104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.754+0000 7f67fee70700 1 -- 172.21.15.84:0/3581389960 >> 172.21.15.84:0/3581389960 conn(0x7f67f80fea40 msgr2=0x7f67f8100e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:20:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.754+0000 7f67fee70700 1 -- 172.21.15.84:0/3581389960 shutdown_connections 2024-04-26T18:20:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.754+0000 7f67fee70700 1 -- 172.21.15.84:0/3581389960 wait complete. 2024-04-26T18:20:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fee70700 1 Processor -- start 2024-04-26T18:20:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fee70700 1 -- start start 2024-04-26T18:20:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fee70700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f81034a0 0x7f67f81994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fee70700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f81046a0 0x7f67f8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:46.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fee70700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67f8105040 0x7f67f81a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fee70700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67f810ef40 con 0x7f67f81034a0 2024-04-26T18:20:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fee70700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f67f810edc0 con 0x7f67f81046a0 2024-04-26T18:20:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fee70700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f67f810f0c0 con 0x7f67f8105040 2024-04-26T18:20:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fcc0c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f81034a0 0x7f67f81994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fcc0c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f81034a0 0x7f67f81994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37150/0 (socket says 172.21.15.84:37150) 2024-04-26T18:20:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fcc0c700 1 -- 172.21.15.84:0/3408087881 learned_addr learned my addr 172.21.15.84:0/3408087881 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:20:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67effff700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f81046a0 0x7f67f8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fd40d700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67f8105040 0x7f67f81a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fcc0c700 1 -- 172.21.15.84:0/3408087881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67f8105040 msgr2=0x7f67f81a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fcc0c700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67f8105040 0x7f67f81a0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fcc0c700 1 -- 172.21.15.84:0/3408087881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f81046a0 msgr2=0x7f67f8199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:20:46.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fcc0c700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f81046a0 0x7f67f8199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:20:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.755+0000 7f67fcc0c700 1 -- 172.21.15.84:0/3408087881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f81a1150 con 0x7f67f81034a0 2024-04-26T18:20:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.756+0000 7f67fcc0c700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f81034a0 0x7f67f81994f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67f400add0 tx=0x7f67f400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.756+0000 7f67edffb700 1 -- 172.21.15.84:0/3408087881 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4017020 con 0x7f67f81034a0 2024-04-26T18:20:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.756+0000 7f67fee70700 1 -- 172.21.15.84:0/3408087881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f81a1440 con 0x7f67f81034a0 2024-04-26T18:20:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.756+0000 7f67edffb700 1 -- 172.21.15.84:0/3408087881 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67f400c890 con 0x7f67f81034a0 2024-04-26T18:20:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.756+0000 7f67edffb700 1 -- 172.21.15.84:0/3408087881 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67f4004260 con 0x7f67f81034a0 2024-04-26T18:20:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.756+0000 7f67fee70700 1 -- 172.21.15.84:0/3408087881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f81a19d0 con 0x7f67f81034a0 2024-04-26T18:20:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.757+0000 7f67fee70700 1 -- 172.21.15.84:0/3408087881 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f806a110 con 0x7f67f81034a0 2024-04-26T18:20:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.758+0000 7f67edffb700 1 -- 172.21.15.84:0/3408087881 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f67f400ca40 con 0x7f67f81034a0 2024-04-26T18:20:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.758+0000 7f67edffb700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f67e00a3c40 0x7f67e00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:20:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.759+0000 7f67effff700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f67e00a3c40 0x7f67e00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:20:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.759+0000 7f67edffb700 1 -- 172.21.15.84:0/3408087881 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(41..41 src has 1..41) ==== 4562+0+0 (secure 0 0 0) 0x7f67f40ceb80 con 0x7f67f81034a0 2024-04-26T18:20:46.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.759+0000 7f67effff700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f67e00a3c40 0x7f67e00a6100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009c60 tx=0x7f67e8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:20:46.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.763+0000 7f67edffb700 1 -- 172.21.15.84:0/3408087881 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67f4021020 con 0x7f67f81034a0 2024-04-26T18:20:46.778 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:46 smithi103 ceph-mon[32727]: Detected new or changed devices on smithi174 2024-04-26T18:20:46.778 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:46 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi174 to 8544M 2024-04-26T18:20:46.778 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:46 smithi103 ceph-mon[32727]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T18:20:46.866 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:46 smithi174 ceph-mon[32705]: Detected new or changed devices on smithi174 2024-04-26T18:20:46.866 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:46 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi174 to 8544M 2024-04-26T18:20:46.866 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:46 smithi174 ceph-mon[32705]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T18:20:46.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:20:46.968+0000 7f67fee70700 1 -- 172.21.15.84:0/3408087881 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f67f8063860 con 0x7f67e00a3c40 2024-04-26T18:20:47.757 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:47 smithi084 ceph-mon[29141]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:20:47.757 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:47 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:20:47.757 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:47 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:20:47.757 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:47 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:47.757 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:47 smithi084 ceph-mon[29141]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:20:47.757 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:47 smithi084 ceph-mon[29141]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T18:20:47.785 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:47 smithi174 ceph-mon[32705]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:20:47.785 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:47 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:20:47.785 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:47 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:20:47.785 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:47 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:47.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:47 smithi174 ceph-mon[32705]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:20:47.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:47 smithi174 ceph-mon[32705]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T18:20:47.802 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:47 smithi103 ceph-mon[32727]: from='client.14358 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:20:47.802 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:47 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:20:47.802 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:47 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:20:47.802 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:47 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:47.802 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:47 smithi103 ceph-mon[32727]: pgmap v142: 1 pgs: 1 remapped+peering; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:20:47.802 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:47 smithi103 ceph-mon[32727]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T18:20:49.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:49 smithi103 ceph-mon[32727]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T18:20:49.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:49 smithi174 ceph-mon[32705]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T18:20:49.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:49 smithi084 ceph-mon[29141]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T18:20:50.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:50 smithi084 ceph-mon[29141]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:20:50.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:50 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1796748354' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a037b66a-7488-469d-b73a-14cc0d35cb5e"}]: dispatch 2024-04-26T18:20:50.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:50 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1796748354' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a037b66a-7488-469d-b73a-14cc0d35cb5e"}]': finished 2024-04-26T18:20:50.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:50 smithi084 ceph-mon[29141]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T18:20:50.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:50 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:20:50.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:50 smithi174 ceph-mon[32705]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:20:50.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:50 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1796748354' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a037b66a-7488-469d-b73a-14cc0d35cb5e"}]: dispatch 2024-04-26T18:20:50.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:50 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1796748354' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a037b66a-7488-469d-b73a-14cc0d35cb5e"}]': finished 2024-04-26T18:20:50.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:50 smithi174 ceph-mon[32705]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T18:20:50.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:20:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:50 smithi103 ceph-mon[32727]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 206 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:20:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:50 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1796748354' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a037b66a-7488-469d-b73a-14cc0d35cb5e"}]: dispatch 2024-04-26T18:20:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:50 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1796748354' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a037b66a-7488-469d-b73a-14cc0d35cb5e"}]': finished 2024-04-26T18:20:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:50 smithi103 ceph-mon[32727]: osdmap e44: 7 total, 6 up, 7 in 2024-04-26T18:20:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:50 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:20:51.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:51 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1291437090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:20:51.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:51 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1291437090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:20:51.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:51 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1291437090' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:20:52.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:52 smithi174 ceph-mon[32705]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:20:52.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:52 smithi084 ceph-mon[29141]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:20:52.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:52 smithi103 ceph-mon[32727]: pgmap v147: 1 pgs: 1 remapped+peering; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:20:54.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:54 smithi174 ceph-mon[32705]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:20:54.577 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:54 smithi084 ceph-mon[29141]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:20:54.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:54 smithi103 ceph-mon[32727]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-26T18:20:56.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:56 smithi084 ceph-mon[29141]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:20:56.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:56 smithi103 ceph-mon[32727]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:20:56.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:56 smithi174 ceph-mon[32705]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:20:57.485 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T18:20:57.485 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:57.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T18:20:57.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:57.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-26T18:20:57.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:20:58.498 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:58 smithi103 ceph-mon[32727]: Deploying daemon osd.6 on smithi084 2024-04-26T18:20:58.498 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:20:58 smithi103 ceph-mon[32727]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T18:20:58.503 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:58 smithi084 ceph-mon[29141]: Deploying daemon osd.6 on smithi084 2024-04-26T18:20:58.503 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:20:58 smithi084 ceph-mon[29141]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T18:20:58.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:58 smithi174 ceph-mon[32705]: Deploying daemon osd.6 on smithi084 2024-04-26T18:20:58.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:20:58 smithi174 ceph-mon[32705]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-26T18:21:00.469 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:00 smithi084 ceph-mon[29141]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T18:21:00.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:00 smithi103 ceph-mon[32727]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T18:21:00.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:00 smithi174 ceph-mon[32705]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-26T18:21:02.484 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:02 smithi084 ceph-mon[29141]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-26T18:21:02.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:02 smithi103 ceph-mon[32727]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-26T18:21:02.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:02 smithi174 ceph-mon[32705]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 50 KiB/s, 0 objects/s recovering 2024-04-26T18:21:04.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:03 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:04.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:03 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:04.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:03 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:04.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:03 smithi084 ceph-mon[29141]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-26T18:21:04.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:03 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:04.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:03 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:04.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:03 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:04.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:03 smithi103 ceph-mon[32727]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-26T18:21:04.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:03 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:04.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:03 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:04.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:03 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:04.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:03 smithi174 ceph-mon[32705]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-04-26T18:21:06.309 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:06 smithi084 ceph-mon[29141]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:21:06.309 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:06.309 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:06.309 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:06.309 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:06.309 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:06.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:06 smithi174 ceph-mon[32705]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:21:06.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:06.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:06.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:06.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:06.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:06 smithi103 ceph-mon[32727]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:21:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:06.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:21:06.773+0000 7f67edffb700 1 -- 172.21.15.84:0/3408087881 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f67f8063860 con 0x7f67e00a3c40 2024-04-26T18:21:06.774 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 6 on host 'smithi084' 2024-04-26T18:21:06.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:21:06.775+0000 7f67fee70700 1 -- 172.21.15.84:0/3408087881 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f67e00a3c40 msgr2=0x7f67e00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:06.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:21:06.775+0000 7f67fee70700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f67e00a3c40 0x7f67e00a6100 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009c60 tx=0x7f67e8009260 comp rx=0 tx=0).stop 2024-04-26T18:21:06.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:21:06.775+0000 7f67fee70700 1 -- 172.21.15.84:0/3408087881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f81034a0 msgr2=0x7f67f81994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:06.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:21:06.775+0000 7f67fee70700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f81034a0 0x7f67f81994f0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f67f400add0 tx=0x7f67f400c330 comp rx=0 tx=0).stop 2024-04-26T18:21:06.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:21:06.775+0000 7f67fee70700 1 -- 172.21.15.84:0/3408087881 shutdown_connections 2024-04-26T18:21:06.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:21:06.775+0000 7f67fee70700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f67e00a3c40 0x7f67e00a6100 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:06.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:21:06.775+0000 7f67fee70700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67f81034a0 0x7f67f81994f0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:06.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:21:06.775+0000 7f67fee70700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67f8105040 0x7f67f81a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:06.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:21:06.775+0000 7f67fee70700 1 --2- 172.21.15.84:0/3408087881 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67f81046a0 0x7f67f8199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:06.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:21:06.775+0000 7f67fee70700 1 -- 172.21.15.84:0/3408087881 >> 172.21.15.84:0/3408087881 conn(0x7f67f80fea40 msgr2=0x7f67f81079a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:06.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:21:06.776+0000 7f67fee70700 1 -- 172.21.15.84:0/3408087881 shutdown_connections 2024-04-26T18:21:06.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:21:06.776+0000 7f67fee70700 1 -- 172.21.15.84:0/3408087881 wait complete. 2024-04-26T18:21:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:07 smithi174 ceph-mon[32705]: from='osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T18:21:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:07.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:07 smithi084 ceph-mon[29141]: from='osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T18:21:07.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:07.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:07.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:07.592 DEBUG:teuthology.orchestra.run.smithi084:osd.6> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.6.service 2024-04-26T18:21:07.594 INFO:tasks.cephadm:Deploying osd.7 on smithi103 with /dev/vg_nvme/lv_2... 2024-04-26T18:21:07.594 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T18:21:07.621 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:07 smithi103 ceph-mon[32727]: from='osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-26T18:21:07.622 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:07.622 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:07.622 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:07.842 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:21:08.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:08 smithi103 ceph-mon[32727]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:21:08.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:08 smithi103 ceph-mon[32727]: from='osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T18:21:08.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:08 smithi103 ceph-mon[32727]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T18:21:08.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:08 smithi103 ceph-mon[32727]: from='osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-04-26T18:21:08.293 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:08 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:21:08.505 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:08 smithi084 ceph-mon[29141]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:21:08.505 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:08 smithi084 ceph-mon[29141]: from='osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T18:21:08.505 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:08 smithi084 ceph-mon[29141]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T18:21:08.505 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:08 smithi084 ceph-mon[29141]: from='osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-04-26T18:21:08.505 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:08 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:21:08.506 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:21:08 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:21:08.194+0000 7f0503c80700 -1 osd.6 0 waiting for initial osdmap 2024-04-26T18:21:08.506 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:21:08 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:21:08.202+0000 7f04ffa73700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:21:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:08 smithi174 ceph-mon[32705]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:21:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:08 smithi174 ceph-mon[32705]: from='osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-26T18:21:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:08 smithi174 ceph-mon[32705]: osdmap e45: 7 total, 6 up, 7 in 2024-04-26T18:21:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:08 smithi174 ceph-mon[32705]: from='osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-04-26T18:21:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:08 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:21:09.281 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:09 smithi084 ceph-mon[29141]: from='osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-04-26T18:21:09.282 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:09 smithi084 ceph-mon[29141]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T18:21:09.282 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:09 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:21:09.282 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:09 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:21:09.373 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:09 smithi103 ceph-mon[32727]: from='osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-04-26T18:21:09.373 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:09 smithi103 ceph-mon[32727]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T18:21:09.374 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:09 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:21:09.374 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:09 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:21:09.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:09 smithi174 ceph-mon[32705]: from='osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-04-26T18:21:09.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:09 smithi174 ceph-mon[32705]: osdmap e46: 7 total, 6 up, 7 in 2024-04-26T18:21:09.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:09 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:21:09.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:09 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:21:10.107 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T18:21:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T18:21:10.108 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-04-26T18:21:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-04-26T18:21:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112259 s, 934 MB/s 2024-04-26T18:21:10.108 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-04-26T18:21:10.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:10 smithi103 ceph-mon[32727]: purged_snaps scrub starts 2024-04-26T18:21:10.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:10 smithi103 ceph-mon[32727]: purged_snaps scrub ok 2024-04-26T18:21:10.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:10 smithi103 ceph-mon[32727]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:21:10.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:10 smithi103 ceph-mon[32727]: osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297] boot 2024-04-26T18:21:10.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:10 smithi103 ceph-mon[32727]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T18:21:10.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:10 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:21:10.464 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:10 smithi084 ceph-mon[29141]: purged_snaps scrub starts 2024-04-26T18:21:10.464 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:10 smithi084 ceph-mon[29141]: purged_snaps scrub ok 2024-04-26T18:21:10.464 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:10 smithi084 ceph-mon[29141]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:21:10.464 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:10 smithi084 ceph-mon[29141]: osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297] boot 2024-04-26T18:21:10.464 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:10 smithi084 ceph-mon[29141]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T18:21:10.464 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:10 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:21:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:10 smithi174 ceph-mon[32705]: purged_snaps scrub starts 2024-04-26T18:21:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:10 smithi174 ceph-mon[32705]: purged_snaps scrub ok 2024-04-26T18:21:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:10 smithi174 ceph-mon[32705]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-04-26T18:21:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:10 smithi174 ceph-mon[32705]: osd.6 [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297] boot 2024-04-26T18:21:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:10 smithi174 ceph-mon[32705]: osdmap e47: 7 total, 7 up, 7 in 2024-04-26T18:21:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:10 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:21:10.826 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_2 2024-04-26T18:21:11.113 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:21:11.459 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:11 smithi103 ceph-mon[32727]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T18:21:11.459 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:11.459 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:11.459 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:11.459 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:11.459 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:11.459 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:11.460 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:11.460 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:11.460 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:11.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:11 smithi084 ceph-mon[29141]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T18:21:11.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:11.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:11.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:11.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:11.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:11.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:11.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:11.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:11.596 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:11 smithi174 ceph-mon[32705]: osdmap e48: 7 total, 7 up, 7 in 2024-04-26T18:21:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:12.593 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:12 smithi103 ceph-mon[32727]: Detected new or changed devices on smithi084 2024-04-26T18:21:12.593 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:12 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi084 to 5696M 2024-04-26T18:21:12.593 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:12 smithi103 ceph-mon[32727]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:12.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:12 smithi084 ceph-mon[29141]: Detected new or changed devices on smithi084 2024-04-26T18:21:12.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:12 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi084 to 5696M 2024-04-26T18:21:12.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:12 smithi084 ceph-mon[29141]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:12.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:12 smithi174 ceph-mon[32705]: Detected new or changed devices on smithi084 2024-04-26T18:21:12.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:12 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi084 to 5696M 2024-04-26T18:21:12.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:12 smithi174 ceph-mon[32705]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:12.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.814+0000 7f8319871700 1 -- 172.21.15.103:0/4267080595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8314106840 msgr2=0x7f8314108c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:12.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.814+0000 7f8319871700 1 --2- 172.21.15.103:0/4267080595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8314106840 0x7f8314108c30 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8308009990 tx=0x7f8308009ca0 comp rx=0 tx=0).stop 2024-04-26T18:21:12.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.814+0000 7f8319871700 1 -- 172.21.15.103:0/4267080595 shutdown_connections 2024-04-26T18:21:12.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.814+0000 7f8319871700 1 --2- 172.21.15.103:0/4267080595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8314103f10 0x7f8314106300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:12.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.814+0000 7f8319871700 1 --2- 172.21.15.103:0/4267080595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8314109170 0x7f831410b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:12.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.814+0000 7f8319871700 1 --2- 172.21.15.103:0/4267080595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8314106840 0x7f8314108c30 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:12.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.814+0000 7f8319871700 1 -- 172.21.15.103:0/4267080595 >> 172.21.15.103:0/4267080595 conn(0x7f83140fd920 msgr2=0x7f83140ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:12.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.814+0000 7f8319871700 1 -- 172.21.15.103:0/4267080595 shutdown_connections 2024-04-26T18:21:12.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.814+0000 7f8319871700 1 -- 172.21.15.103:0/4267080595 wait complete. 2024-04-26T18:21:12.814 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.814+0000 7f8319871700 1 Processor -- start 2024-04-26T18:21:12.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8319871700 1 -- start start 2024-04-26T18:21:12.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8319871700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8314103f10 0x7f83141995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:12.816 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8319871700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8314106840 0x7f8314199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:12.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8319871700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8314109170 0x7f83141a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:12.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8319871700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f831410ef40 con 0x7f8314109170 2024-04-26T18:21:12.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8319871700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f831410edc0 con 0x7f8314103f10 2024-04-26T18:21:12.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8319871700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f831410f0c0 con 0x7f8314106840 2024-04-26T18:21:12.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8312ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8314103f10 0x7f83141995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:12.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8312ffd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8314103f10 0x7f83141995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:43956/0 (socket says 172.21.15.103:43956) 2024-04-26T18:21:12.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8312ffd700 1 -- 172.21.15.103:0/1037383335 learned_addr learned my addr 172.21.15.103:0/1037383335 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T18:21:12.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f83127fc700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8314106840 0x7f8314199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:12.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f83137fe700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8314109170 0x7f83141a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:12.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8312ffd700 1 -- 172.21.15.103:0/1037383335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8314106840 msgr2=0x7f8314199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:12.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8312ffd700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8314106840 0x7f8314199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:12.817 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8312ffd700 1 -- 172.21.15.103:0/1037383335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8314109170 msgr2=0x7f83141a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:12.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8312ffd700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8314109170 0x7f83141a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:12.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8312ffd700 1 -- 172.21.15.103:0/1037383335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83141a1240 con 0x7f8314103f10 2024-04-26T18:21:12.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f8312ffd700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8314103f10 0x7f83141995e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f830400ada0 tx=0x7f830400c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:12.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.815+0000 7f831886f700 1 -- 172.21.15.103:0/1037383335 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304017020 con 0x7f8314103f10 2024-04-26T18:21:12.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.816+0000 7f8319871700 1 -- 172.21.15.103:0/1037383335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83141a1530 con 0x7f8314103f10 2024-04-26T18:21:12.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.816+0000 7f831886f700 1 -- 172.21.15.103:0/1037383335 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f830400ce30 con 0x7f8314103f10 2024-04-26T18:21:12.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.816+0000 7f831886f700 1 -- 172.21.15.103:0/1037383335 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8304005560 con 0x7f8314103f10 2024-04-26T18:21:12.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.816+0000 7f8319871700 1 -- 172.21.15.103:0/1037383335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83141a1a70 con 0x7f8314103f10 2024-04-26T18:21:12.818 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.817+0000 7f8319871700 1 -- 172.21.15.103:0/1037383335 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831406a110 con 0x7f8314103f10 2024-04-26T18:21:12.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.818+0000 7f831886f700 1 -- 172.21.15.103:0/1037383335 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8304004ad0 con 0x7f8314103f10 2024-04-26T18:21:12.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.818+0000 7f831886f700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f82fc0a3c40 0x7f82fc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:12.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.818+0000 7f831886f700 1 -- 172.21.15.103:0/1037383335 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(48..48 src has 1..48) ==== 4965+0+0 (secure 0 0 0) 0x7f83040ceff0 con 0x7f8314103f10 2024-04-26T18:21:12.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.818+0000 7f83127fc700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f82fc0a3c40 0x7f82fc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:12.820 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.819+0000 7f83127fc700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f82fc0a3c40 0x7f82fc0a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f830800b5c0 tx=0x7f8308005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:12.823 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:12.823+0000 7f831886f700 1 -- 172.21.15.103:0/1037383335 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8304021020 con 0x7f8314103f10 2024-04-26T18:21:13.030 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:13.030+0000 7f8319871700 1 -- 172.21.15.103:0/1037383335 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8314063920 con 0x7f82fc0a3c40 2024-04-26T18:21:13.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:21:13.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:21:13.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:13.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:21:13.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:21:13.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:21:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:21:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:14.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:14 smithi084 ceph-mon[29141]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:14.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:14 smithi084 ceph-mon[29141]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:14.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:14 smithi103 ceph-mon[32727]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:14.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:14 smithi103 ceph-mon[32727]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:14.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:14 smithi174 ceph-mon[32705]: from='client.24269 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:14.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:14 smithi174 ceph-mon[32705]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:16.491 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:16 smithi174 ceph-mon[32705]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:16.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:16 smithi174 ceph-mon[32705]: from='client.? 172.21.15.103:0/1858378712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c425a13f-8019-4484-86a0-a34fa4d8de81"}]: dispatch 2024-04-26T18:21:16.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:16 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c425a13f-8019-4484-86a0-a34fa4d8de81"}]: dispatch 2024-04-26T18:21:16.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:16 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c425a13f-8019-4484-86a0-a34fa4d8de81"}]': finished 2024-04-26T18:21:16.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:16 smithi174 ceph-mon[32705]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T18:21:16.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:16.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:16 smithi084 ceph-mon[29141]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:16.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:16 smithi084 ceph-mon[29141]: from='client.? 172.21.15.103:0/1858378712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c425a13f-8019-4484-86a0-a34fa4d8de81"}]: dispatch 2024-04-26T18:21:16.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:16 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c425a13f-8019-4484-86a0-a34fa4d8de81"}]: dispatch 2024-04-26T18:21:16.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:16 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c425a13f-8019-4484-86a0-a34fa4d8de81"}]': finished 2024-04-26T18:21:16.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:16 smithi084 ceph-mon[29141]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T18:21:16.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:16.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:16 smithi103 ceph-mon[32727]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:16.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:16 smithi103 ceph-mon[32727]: from='client.? 172.21.15.103:0/1858378712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c425a13f-8019-4484-86a0-a34fa4d8de81"}]: dispatch 2024-04-26T18:21:16.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:16 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c425a13f-8019-4484-86a0-a34fa4d8de81"}]: dispatch 2024-04-26T18:21:16.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:16 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c425a13f-8019-4484-86a0-a34fa4d8de81"}]': finished 2024-04-26T18:21:16.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:16 smithi103 ceph-mon[32727]: osdmap e49: 8 total, 7 up, 8 in 2024-04-26T18:21:16.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:17.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:17 smithi084 ceph-mon[29141]: from='client.? 172.21.15.103:0/4070771960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:21:17.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:17 smithi103 ceph-mon[32727]: from='client.? 172.21.15.103:0/4070771960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:21:17.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:17 smithi174 ceph-mon[32705]: from='client.? 172.21.15.103:0/4070771960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:21:18.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:18 smithi084 ceph-mon[29141]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:18.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:18 smithi103 ceph-mon[32727]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:18.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:18 smithi174 ceph-mon[32705]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:20.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:20 smithi084 ceph-mon[29141]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:20.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:20 smithi103 ceph-mon[32727]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:20.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:20 smithi174 ceph-mon[32705]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:22.460 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:22 smithi103 ceph-mon[32727]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:22.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:22 smithi084 ceph-mon[29141]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:22.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:22 smithi174 ceph-mon[32705]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:24.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:24 smithi103 ceph-mon[32727]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:24.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:24 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T18:21:24.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:24 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:24.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:24 smithi084 ceph-mon[29141]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:24.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:24 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T18:21:24.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:24 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:24.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:24 smithi174 ceph-mon[32705]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:24.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:24 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-26T18:21:24.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:24 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:25.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:25 smithi103 ceph-mon[32727]: Deploying daemon osd.7 on smithi103 2024-04-26T18:21:25.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:25 smithi084 ceph-mon[29141]: Deploying daemon osd.7 on smithi103 2024-04-26T18:21:25.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:25 smithi174 ceph-mon[32705]: Deploying daemon osd.7 on smithi103 2024-04-26T18:21:26.356 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:26 smithi103 ceph-mon[32727]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:26.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:26 smithi084 ceph-mon[29141]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:26.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:26 smithi174 ceph-mon[32705]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:28.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:28 smithi084 ceph-mon[29141]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:28.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:28 smithi103 ceph-mon[32727]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:28.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:28 smithi174 ceph-mon[32705]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:30.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:30 smithi084 ceph-mon[29141]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:30.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:30 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:30.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:30 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:30.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:30 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:30 smithi103 ceph-mon[32727]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:30 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:30 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:30 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:30.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:30 smithi174 ceph-mon[32705]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:30.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:30 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:30.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:30 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:30.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:30 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:32.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:32 smithi103 ceph-mon[32727]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:32.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:32 smithi084 ceph-mon[29141]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:32.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:32 smithi174 ceph-mon[32705]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:32.896 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:32.895+0000 7f831886f700 1 -- 172.21.15.103:0/1037383335 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8314063920 con 0x7f82fc0a3c40 2024-04-26T18:21:32.897 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 7 on host 'smithi103' 2024-04-26T18:21:32.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:32.898+0000 7f8319871700 1 -- 172.21.15.103:0/1037383335 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f82fc0a3c40 msgr2=0x7f82fc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:32.898 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:32.898+0000 7f8319871700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f82fc0a3c40 0x7f82fc0a6100 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f830800b5c0 tx=0x7f8308005fd0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:32.898+0000 7f8319871700 1 -- 172.21.15.103:0/1037383335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8314103f10 msgr2=0x7f83141995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:32.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:32.898+0000 7f8319871700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8314103f10 0x7f83141995e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f830400ada0 tx=0x7f830400c330 comp rx=0 tx=0).stop 2024-04-26T18:21:32.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:32.898+0000 7f8319871700 1 -- 172.21.15.103:0/1037383335 shutdown_connections 2024-04-26T18:21:32.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:32.898+0000 7f8319871700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f82fc0a3c40 0x7f82fc0a6100 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:32.898+0000 7f8319871700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8314109170 0x7f83141a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:32.898+0000 7f8319871700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8314103f10 0x7f83141995e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:32.898+0000 7f8319871700 1 --2- 172.21.15.103:0/1037383335 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8314106840 0x7f8314199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:32.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:32.898+0000 7f8319871700 1 -- 172.21.15.103:0/1037383335 >> 172.21.15.103:0/1037383335 conn(0x7f83140fd920 msgr2=0x7f8314104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:32.899 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:32.898+0000 7f8319871700 1 -- 172.21.15.103:0/1037383335 shutdown_connections 2024-04-26T18:21:32.900 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:21:32.899+0000 7f8319871700 1 -- 172.21.15.103:0/1037383335 wait complete. 2024-04-26T18:21:33.607 DEBUG:teuthology.orchestra.run.smithi103:osd.7> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.7.service 2024-04-26T18:21:33.609 INFO:tasks.cephadm:Deploying osd.8 on smithi174 with /dev/vg_nvme/lv_2... 2024-04-26T18:21:33.609 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T18:21:33.854 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:21:33.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:33.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:33.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:33.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:33.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:33 smithi174 ceph-mon[32705]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:33.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:33 smithi174 ceph-mon[32705]: from='osd.7 [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:21:33.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:33 smithi174 ceph-mon[32705]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:21:34.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:34.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:34.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:34.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:34.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:33 smithi084 ceph-mon[29141]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:34.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:33 smithi084 ceph-mon[29141]: from='osd.7 [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:21:34.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:33 smithi084 ceph-mon[29141]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:21:34.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:34.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:34.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:34.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:34.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:33 smithi103 ceph-mon[32727]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:34.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:33 smithi103 ceph-mon[32727]: from='osd.7 [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:21:34.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:33 smithi103 ceph-mon[32727]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-26T18:21:35.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:35 smithi103 ceph-mon[32727]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T18:21:35.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:35 smithi103 ceph-mon[32727]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T18:21:35.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:35 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:35.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:35 smithi103 ceph-mon[32727]: from='osd.7 [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:21:35.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:35 smithi103 ceph-mon[32727]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:21:35.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:35 smithi174 ceph-mon[32705]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T18:21:35.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:35 smithi174 ceph-mon[32705]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T18:21:35.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:35 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:35.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:35 smithi174 ceph-mon[32705]: from='osd.7 [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:21:35.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:35 smithi174 ceph-mon[32705]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:21:35.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:35 smithi084 ceph-mon[29141]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-26T18:21:35.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:35 smithi084 ceph-mon[29141]: osdmap e50: 8 total, 7 up, 8 in 2024-04-26T18:21:35.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:35 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:35.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:35 smithi084 ceph-mon[29141]: from='osd.7 [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:21:35.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:35 smithi084 ceph-mon[29141]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:21:35.768 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T18:21:35.769 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T18:21:35.769 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-04-26T18:21:35.769 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-04-26T18:21:35.769 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114268 s, 918 MB/s 2024-04-26T18:21:35.769 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-04-26T18:21:35.877 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:21:35 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:21:35.467+0000 7f09a4ddb700 -1 osd.7 0 waiting for initial osdmap 2024-04-26T18:21:35.877 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:21:35 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:21:35.482+0000 7f099cbc6700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:21:36.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:36 smithi084 ceph-mon[29141]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-26T18:21:36.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:36 smithi084 ceph-mon[29141]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T18:21:36.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:36 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:36.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:36 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:36.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:36 smithi084 ceph-mon[29141]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:36.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:36 smithi084 ceph-mon[29141]: from='osd.7 ' entity='osd.7' 2024-04-26T18:21:36.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:36 smithi084 ceph-mon[29141]: osd.7 [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079] boot 2024-04-26T18:21:36.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:36 smithi084 ceph-mon[29141]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T18:21:36.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:36 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:36.367 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:36 smithi103 ceph-mon[32727]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-26T18:21:36.367 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:36 smithi103 ceph-mon[32727]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T18:21:36.367 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:36 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:36.367 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:36 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:36.367 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:36 smithi103 ceph-mon[32727]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:36.367 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:36 smithi103 ceph-mon[32727]: from='osd.7 ' entity='osd.7' 2024-04-26T18:21:36.367 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:36 smithi103 ceph-mon[32727]: osd.7 [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079] boot 2024-04-26T18:21:36.367 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:36 smithi103 ceph-mon[32727]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T18:21:36.367 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:36 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:36.424 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:36 smithi174 ceph-mon[32705]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-26T18:21:36.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:36 smithi174 ceph-mon[32705]: osdmap e51: 8 total, 7 up, 8 in 2024-04-26T18:21:36.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:36 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:36.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:36 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:36.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:36 smithi174 ceph-mon[32705]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 200 MiB used, 626 GiB / 626 GiB avail 2024-04-26T18:21:36.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:36 smithi174 ceph-mon[32705]: from='osd.7 ' entity='osd.7' 2024-04-26T18:21:36.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:36 smithi174 ceph-mon[32705]: osd.7 [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079] boot 2024-04-26T18:21:36.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:36 smithi174 ceph-mon[32705]: osdmap e52: 8 total, 8 up, 8 in 2024-04-26T18:21:36.425 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:36 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:21:36.452 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_2 2024-04-26T18:21:36.693 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:21:37.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: purged_snaps scrub starts 2024-04-26T18:21:37.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: purged_snaps scrub ok 2024-04-26T18:21:37.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:37.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:37.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:37.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:37.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:37.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:37.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:37.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:37.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:37 smithi084 ceph-mon[29141]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T18:21:37.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: purged_snaps scrub starts 2024-04-26T18:21:37.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: purged_snaps scrub ok 2024-04-26T18:21:37.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:37.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:37.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:37.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:37.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:37.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:37.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:37.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:37.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:37 smithi103 ceph-mon[32727]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T18:21:37.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: purged_snaps scrub starts 2024-04-26T18:21:37.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: purged_snaps scrub ok 2024-04-26T18:21:37.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:37.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:37.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:21:37.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:37.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:37.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:37.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:37.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:37.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:37.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:37 smithi174 ceph-mon[32705]: osdmap e53: 8 total, 8 up, 8 in 2024-04-26T18:21:38.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:38 smithi174 ceph-mon[32705]: Detected new or changed devices on smithi103 2024-04-26T18:21:38.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:38 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi103 to 5696M 2024-04-26T18:21:38.173 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:38 smithi174 ceph-mon[32705]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:38.174 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:38 smithi174 ceph-mon[32705]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T18:21:38.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:38 smithi084 ceph-mon[29141]: Detected new or changed devices on smithi103 2024-04-26T18:21:38.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:38 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi103 to 5696M 2024-04-26T18:21:38.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:38 smithi084 ceph-mon[29141]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:38.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:38 smithi084 ceph-mon[29141]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T18:21:38.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:38 smithi103 ceph-mon[32727]: Detected new or changed devices on smithi103 2024-04-26T18:21:38.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:38 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi103 to 5696M 2024-04-26T18:21:38.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:38 smithi103 ceph-mon[32727]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:38.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:38 smithi103 ceph-mon[32727]: osdmap e54: 8 total, 8 up, 8 in 2024-04-26T18:21:38.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.396+0000 7f52594be700 1 -- 172.21.15.174:0/3726141532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254103480 msgr2=0x7f52541038a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:38.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.396+0000 7f52594be700 1 --2- 172.21.15.174:0/3726141532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254103480 0x7f52541038a0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5244009990 tx=0x7f5244009ca0 comp rx=0 tx=0).stop 2024-04-26T18:21:38.396 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.396+0000 7f52594be700 1 -- 172.21.15.174:0/3726141532 shutdown_connections 2024-04-26T18:21:38.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.396+0000 7f52594be700 1 --2- 172.21.15.174:0/3726141532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254105020 0x7f525410b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:38.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.396+0000 7f52594be700 1 --2- 172.21.15.174:0/3726141532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254103480 0x7f52541038a0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:38.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.396+0000 7f52594be700 1 --2- 172.21.15.174:0/3726141532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5254104680 0x7f5254104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:38.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.396+0000 7f52594be700 1 -- 172.21.15.174:0/3726141532 >> 172.21.15.174:0/3726141532 conn(0x7f52540fea20 msgr2=0x7f5254100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:38.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.396+0000 7f52594be700 1 -- 172.21.15.174:0/3726141532 shutdown_connections 2024-04-26T18:21:38.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.396+0000 7f52594be700 1 -- 172.21.15.174:0/3726141532 wait complete. 2024-04-26T18:21:38.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.396+0000 7f52594be700 1 Processor -- start 2024-04-26T18:21:38.397 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f52594be700 1 -- start start 2024-04-26T18:21:38.399 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f52594be700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5254103480 0x7f5254199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:38.399 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f52594be700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254104680 0x7f5254199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:38.399 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f52594be700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254105020 0x7f52541a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:38.399 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f52594be700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f525410ef20 con 0x7f5254105020 2024-04-26T18:21:38.399 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f52594be700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f525410eda0 con 0x7f5254103480 2024-04-26T18:21:38.399 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f52594be700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f525410f0a0 con 0x7f5254104680 2024-04-26T18:21:38.399 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f524a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254104680 0x7f5254199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f524a7fc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254104680 0x7f5254199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:49592/0 (socket says 172.21.15.174:49592) 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f524a7fc700 1 -- 172.21.15.174:0/3761632457 learned_addr learned my addr 172.21.15.174:0/3761632457 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f52537fe700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254105020 0x7f52541a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f5252ffd700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5254103480 0x7f5254199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f524a7fc700 1 -- 172.21.15.174:0/3761632457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5254103480 msgr2=0x7f5254199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f524a7fc700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5254103480 0x7f5254199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f524a7fc700 1 -- 172.21.15.174:0/3761632457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254105020 msgr2=0x7f52541a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f524a7fc700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254105020 0x7f52541a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f524a7fc700 1 -- 172.21.15.174:0/3761632457 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52541a1270 con 0x7f5254104680 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f524a7fc700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254104680 0x7f5254199ba0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f523c00c9a0 tx=0x7f523c00ccb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.397+0000 7f5250ff9700 1 -- 172.21.15.174:0/3761632457 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c007a60 con 0x7f5254104680 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.398+0000 7f5250ff9700 1 -- 172.21.15.174:0/3761632457 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f523c004ce0 con 0x7f5254104680 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.398+0000 7f5250ff9700 1 -- 172.21.15.174:0/3761632457 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c005560 con 0x7f5254104680 2024-04-26T18:21:38.400 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.398+0000 7f52594be700 1 -- 172.21.15.174:0/3761632457 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52541a1560 con 0x7f5254104680 2024-04-26T18:21:38.401 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.398+0000 7f52594be700 1 -- 172.21.15.174:0/3761632457 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52541a1a50 con 0x7f5254104680 2024-04-26T18:21:38.401 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.398+0000 7f52594be700 1 -- 172.21.15.174:0/3761632457 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f525406a110 con 0x7f5254104680 2024-04-26T18:21:38.401 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.400+0000 7f5250ff9700 1 -- 172.21.15.174:0/3761632457 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f523c007c00 con 0x7f5254104680 2024-04-26T18:21:38.401 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.400+0000 7f5250ff9700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f52400a3c40 0x7f52400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:21:38.401 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.401+0000 7f5252ffd700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f52400a3c40 0x7f52400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:21:38.402 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.401+0000 7f5250ff9700 1 -- 172.21.15.174:0/3761632457 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5426+0+0 (secure 0 0 0) 0x7f523c0cfaa0 con 0x7f5254104680 2024-04-26T18:21:38.402 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.401+0000 7f5252ffd700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f52400a3c40 0x7f52400a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52440038a0 tx=0x7f5244005fd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:21:38.406 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.404+0000 7f5250ff9700 1 -- 172.21.15.174:0/3761632457 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f523c09cfb0 con 0x7f5254104680 2024-04-26T18:21:38.608 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:38.608+0000 7f52594be700 1 -- 172.21.15.174:0/3761632457 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5254063860 con 0x7f52400a3c40 2024-04-26T18:21:39.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:39 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:21:39.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:39 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:21:39.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:39 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:39.358 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:39 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:21:39.358 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:39 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:21:39.358 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:39 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:39.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:39 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:21:39.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:39 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:21:39.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:39 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:40.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:40 smithi084 ceph-mon[29141]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:40.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:40 smithi084 ceph-mon[29141]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T18:21:40.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:40 smithi084 ceph-mon[29141]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:40.376 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:40 smithi103 ceph-mon[32727]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:40.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:40 smithi103 ceph-mon[32727]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T18:21:40.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:40 smithi103 ceph-mon[32727]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:40.434 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:40 smithi174 ceph-mon[32705]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:21:40.434 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:40 smithi174 ceph-mon[32705]: osdmap e55: 8 total, 8 up, 8 in 2024-04-26T18:21:40.434 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:40 smithi174 ceph-mon[32705]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:41.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:41 smithi084 ceph-mon[29141]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T18:21:41.376 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:41 smithi103 ceph-mon[32727]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T18:21:41.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:41 smithi174 ceph-mon[32705]: osdmap e56: 8 total, 8 up, 8 in 2024-04-26T18:21:42.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:42 smithi084 ceph-mon[29141]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:42.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:42 smithi084 ceph-mon[29141]: from='client.? 172.21.15.174:0/3837018832' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "687e6c08-bfa4-440c-b3bc-90a5748989d3"}]: dispatch 2024-04-26T18:21:42.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:42 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "687e6c08-bfa4-440c-b3bc-90a5748989d3"}]: dispatch 2024-04-26T18:21:42.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:42 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "687e6c08-bfa4-440c-b3bc-90a5748989d3"}]': finished 2024-04-26T18:21:42.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:42 smithi084 ceph-mon[29141]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T18:21:42.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:42 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:21:42.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:42 smithi103 ceph-mon[32727]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:42.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:42 smithi103 ceph-mon[32727]: from='client.? 172.21.15.174:0/3837018832' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "687e6c08-bfa4-440c-b3bc-90a5748989d3"}]: dispatch 2024-04-26T18:21:42.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:42 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "687e6c08-bfa4-440c-b3bc-90a5748989d3"}]: dispatch 2024-04-26T18:21:42.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:42 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "687e6c08-bfa4-440c-b3bc-90a5748989d3"}]': finished 2024-04-26T18:21:42.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:42 smithi103 ceph-mon[32727]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T18:21:42.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:42 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:21:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:42 smithi174 ceph-mon[32705]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:42 smithi174 ceph-mon[32705]: from='client.? 172.21.15.174:0/3837018832' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "687e6c08-bfa4-440c-b3bc-90a5748989d3"}]: dispatch 2024-04-26T18:21:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:42 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "687e6c08-bfa4-440c-b3bc-90a5748989d3"}]: dispatch 2024-04-26T18:21:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:42 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "687e6c08-bfa4-440c-b3bc-90a5748989d3"}]': finished 2024-04-26T18:21:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:42 smithi174 ceph-mon[32705]: osdmap e57: 9 total, 8 up, 9 in 2024-04-26T18:21:42.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:42 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:21:43.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:43 smithi084 ceph-mon[29141]: from='client.? 172.21.15.174:0/1787779201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:21:43.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:43 smithi103 ceph-mon[32727]: from='client.? 172.21.15.174:0/1787779201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:21:43.438 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:43 smithi174 ceph-mon[32705]: from='client.? 172.21.15.174:0/1787779201' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:21:44.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:44 smithi084 ceph-mon[29141]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:44.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:44 smithi103 ceph-mon[32727]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:44.438 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:44 smithi174 ceph-mon[32705]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:46.487 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:46 smithi174 ceph-mon[32705]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:46.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:46 smithi084 ceph-mon[29141]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:46.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:46 smithi103 ceph-mon[32727]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:48.397 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:48 smithi174 ceph-mon[32705]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:48.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:48 smithi084 ceph-mon[29141]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:48.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:48 smithi103 ceph-mon[32727]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:49.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:49 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T18:21:49.492 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:49 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:49.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:49 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T18:21:49.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:49 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:49.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:49 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-26T18:21:49.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:49 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:50.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:50 smithi174 ceph-mon[32705]: Deploying daemon osd.8 on smithi174 2024-04-26T18:21:50.396 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:50 smithi174 ceph-mon[32705]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:50.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:50 smithi084 ceph-mon[29141]: Deploying daemon osd.8 on smithi174 2024-04-26T18:21:50.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:50 smithi084 ceph-mon[29141]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:50 smithi103 ceph-mon[32727]: Deploying daemon osd.8 on smithi174 2024-04-26T18:21:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:50 smithi103 ceph-mon[32727]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:52.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:52 smithi174 ceph-mon[32705]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:52.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:52 smithi084 ceph-mon[29141]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:52.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:52 smithi103 ceph-mon[32727]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:54.514 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:54 smithi174 ceph-mon[32705]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:54.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:54 smithi084 ceph-mon[29141]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:54.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:54 smithi103 ceph-mon[32727]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:55.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:55.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:55.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:55 smithi084 ceph-mon[29141]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:55.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:55 smithi103 ceph-mon[32727]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:55 smithi174 ceph-mon[32705]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:58.340 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:58 smithi174 ceph-mon[32705]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:58.340 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:58 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:58.341 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:58 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:58.341 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:58 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:58.341 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:58 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:58.341 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:58 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:58.429 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 8 on host 'smithi174' 2024-04-26T18:21:58.429 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:58.428+0000 7f5250ff9700 1 -- 172.21.15.174:0/3761632457 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5254063860 con 0x7f52400a3c40 2024-04-26T18:21:58.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:58.430+0000 7f52594be700 1 -- 172.21.15.174:0/3761632457 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f52400a3c40 msgr2=0x7f52400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:58.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:58.430+0000 7f52594be700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f52400a3c40 0x7f52400a6100 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f52440038a0 tx=0x7f5244005fd0 comp rx=0 tx=0).stop 2024-04-26T18:21:58.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:58.430+0000 7f52594be700 1 -- 172.21.15.174:0/3761632457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254104680 msgr2=0x7f5254199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:21:58.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:58.430+0000 7f52594be700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254104680 0x7f5254199ba0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f523c00c9a0 tx=0x7f523c00ccb0 comp rx=0 tx=0).stop 2024-04-26T18:21:58.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:58.430+0000 7f52594be700 1 -- 172.21.15.174:0/3761632457 shutdown_connections 2024-04-26T18:21:58.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:58.430+0000 7f52594be700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f52400a3c40 0x7f52400a6100 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:58.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:58.430+0000 7f52594be700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254105020 0x7f52541a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:58.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:58.430+0000 7f52594be700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5254103480 0x7f5254199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:58.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:58.430+0000 7f52594be700 1 --2- 172.21.15.174:0/3761632457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254104680 0x7f5254199ba0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:21:58.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:58.430+0000 7f52594be700 1 -- 172.21.15.174:0/3761632457 >> 172.21.15.174:0/3761632457 conn(0x7f52540fea20 msgr2=0x7f5254107ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:21:58.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:58.430+0000 7f52594be700 1 -- 172.21.15.174:0/3761632457 shutdown_connections 2024-04-26T18:21:58.431 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:21:58.430+0000 7f52594be700 1 -- 172.21.15.174:0/3761632457 wait complete. 2024-04-26T18:21:58.548 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:58 smithi084 ceph-mon[29141]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:58.548 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:58 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:58.548 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:58 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:58.548 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:58 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:58.548 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:58 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:58.548 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:58 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:58 smithi103 ceph-mon[32727]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:58 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:58 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:58 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:21:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:58 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:21:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:58 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:59.065 DEBUG:teuthology.orchestra.run.smithi174:osd.8> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.8.service 2024-04-26T18:21:59.067 INFO:tasks.cephadm:Deploying osd.9 on smithi084 with /dev/vg_nvme/lv_1... 2024-04-26T18:21:59.067 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T18:21:59.316 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:21:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:59 smithi103 ceph-mon[32727]: from='osd.8 [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:21:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:59 smithi103 ceph-mon[32727]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:21:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:21:59 smithi103 ceph-mon[32727]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:59.679 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:59.679 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:59.679 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:59.679 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:59 smithi084 ceph-mon[29141]: from='osd.8 [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:21:59.679 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:59 smithi084 ceph-mon[29141]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:21:59.679 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:21:59 smithi084 ceph-mon[29141]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:21:59.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:59.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:21:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:21:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:59 smithi174 ceph-mon[32705]: from='osd.8 [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:21:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:59 smithi174 ceph-mon[32705]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-26T18:21:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:21:59 smithi174 ceph-mon[32705]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:22:00.543 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:00 smithi174 ceph-mon[32705]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T18:22:00.543 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:00 smithi174 ceph-mon[32705]: from='osd.8 [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:22:00.543 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:00 smithi174 ceph-mon[32705]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T18:22:00.543 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:00 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:00.543 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:00 smithi174 ceph-mon[32705]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:22:00.743 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:00 smithi084 ceph-mon[29141]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T18:22:00.744 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:00 smithi084 ceph-mon[29141]: from='osd.8 [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:22:00.744 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:00 smithi084 ceph-mon[29141]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T18:22:00.744 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:00 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:00.744 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:00 smithi084 ceph-mon[29141]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:22:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:00 smithi103 ceph-mon[32727]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-26T18:22:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:00 smithi103 ceph-mon[32727]: from='osd.8 [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:22:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:00 smithi103 ceph-mon[32727]: osdmap e58: 9 total, 8 up, 9 in 2024-04-26T18:22:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:00 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:00 smithi103 ceph-mon[32727]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:22:01.189 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:22:00 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:22:00.895+0000 7f2291215700 -1 osd.8 0 waiting for initial osdmap 2024-04-26T18:22:01.189 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:22:00 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:22:00.904+0000 7f2289000700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:22:01.411 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T18:22:01.411 INFO:teuthology.orchestra.run.smithi084.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T18:22:01.411 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10+0 records in 2024-04-26T18:22:01.411 INFO:teuthology.orchestra.run.smithi084.stderr:10+0 records out 2024-04-26T18:22:01.411 INFO:teuthology.orchestra.run.smithi084.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127591 s, 822 MB/s 2024-04-26T18:22:01.411 INFO:teuthology.orchestra.run.smithi084.stderr:--> Zapping successful for: 2024-04-26T18:22:01.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:01 smithi174 ceph-mon[32705]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-04-26T18:22:01.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:01 smithi174 ceph-mon[32705]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T18:22:01.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:01 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:01.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:01 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:01.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:01 smithi174 ceph-mon[32705]: from='osd.8 ' entity='osd.8' 2024-04-26T18:22:01.675 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:01 smithi174 ceph-mon[32705]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:22:01.717 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:01 smithi084 ceph-mon[29141]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-04-26T18:22:01.717 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:01 smithi084 ceph-mon[29141]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T18:22:01.717 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:01 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:01.717 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:01 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:01.718 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:01 smithi084 ceph-mon[29141]: from='osd.8 ' entity='osd.8' 2024-04-26T18:22:01.718 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:01 smithi084 ceph-mon[29141]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:22:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:01 smithi103 ceph-mon[32727]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-04-26T18:22:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:01 smithi103 ceph-mon[32727]: osdmap e59: 9 total, 8 up, 9 in 2024-04-26T18:22:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:01 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:01 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:01 smithi103 ceph-mon[32727]: from='osd.8 ' entity='osd.8' 2024-04-26T18:22:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:01 smithi103 ceph-mon[32727]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-04-26T18:22:02.029 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi084:vg_nvme/lv_1 2024-04-26T18:22:02.292 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:22:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: purged_snaps scrub starts 2024-04-26T18:22:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: purged_snaps scrub ok 2024-04-26T18:22:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: osd.8 [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548] boot 2024-04-26T18:22:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T18:22:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: Detected new or changed devices on smithi174 2024-04-26T18:22:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi174 to 5696M 2024-04-26T18:22:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:02 smithi174 ceph-mon[32705]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T18:22:02.696 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: purged_snaps scrub starts 2024-04-26T18:22:02.696 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: purged_snaps scrub ok 2024-04-26T18:22:02.696 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:02.696 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: osd.8 [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548] boot 2024-04-26T18:22:02.696 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T18:22:02.696 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:02.696 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: Detected new or changed devices on smithi174 2024-04-26T18:22:02.697 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:02.697 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:02.697 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:02.697 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:02.697 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:02.697 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi174 to 5696M 2024-04-26T18:22:02.697 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:02.697 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:02.697 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:02.697 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:02.697 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:02 smithi084 ceph-mon[29141]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T18:22:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: purged_snaps scrub starts 2024-04-26T18:22:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: purged_snaps scrub ok 2024-04-26T18:22:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: osd.8 [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548] boot 2024-04-26T18:22:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: osdmap e60: 9 total, 9 up, 9 in 2024-04-26T18:22:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:22:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: Detected new or changed devices on smithi174 2024-04-26T18:22:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi174 to 5696M 2024-04-26T18:22:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:02 smithi103 ceph-mon[32727]: osdmap e61: 9 total, 9 up, 9 in 2024-04-26T18:22:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.007+0000 7f5e70761700 1 -- 172.21.15.84:0/3260831144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e681035a0 msgr2=0x7f5e68103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.007+0000 7f5e70761700 1 --2- 172.21.15.84:0/3260831144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e681035a0 0x7f5e68103a00 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f5e580099c0 tx=0x7f5e58009cd0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 -- 172.21.15.84:0/3260831144 shutdown_connections 2024-04-26T18:22:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 --2- 172.21.15.84:0/3260831144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e68103f40 0x7f5e6810a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 --2- 172.21.15.84:0/3260831144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e681023a0 0x7f5e681027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 --2- 172.21.15.84:0/3260831144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e681035a0 0x7f5e68103a00 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 -- 172.21.15.84:0/3260831144 >> 172.21.15.84:0/3260831144 conn(0x7f5e680fd980 msgr2=0x7f5e680ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 -- 172.21.15.84:0/3260831144 shutdown_connections 2024-04-26T18:22:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 -- 172.21.15.84:0/3260831144 wait complete. 2024-04-26T18:22:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 Processor -- start 2024-04-26T18:22:04.010 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 -- start start 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e681023a0 0x7f5e6806c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e681035a0 0x7f5e6819e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e68103f40 0x7f5e681a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6810de40 con 0x7f5e68103f40 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6810dcc0 con 0x7f5e681023a0 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.008+0000 7f5e70761700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6810dfc0 con 0x7f5e681035a0 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e6ecfe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e68103f40 0x7f5e681a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e6ecfe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e68103f40 0x7f5e681a0950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45900/0 (socket says 172.21.15.84:45900) 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e6ecfe700 1 -- 172.21.15.84:0/1304047160 learned_addr learned my addr 172.21.15.84:0/1304047160 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e6e4fd700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e681023a0 0x7f5e6806c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e6dcfc700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e681035a0 0x7f5e6819e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e6ecfe700 1 -- 172.21.15.84:0/1304047160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e681035a0 msgr2=0x7f5e6819e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e6ecfe700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e681035a0 0x7f5e6819e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e6ecfe700 1 -- 172.21.15.84:0/1304047160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e681023a0 msgr2=0x7f5e6806c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:04.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e6ecfe700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e681023a0 0x7f5e6806c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:04.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e6ecfe700 1 -- 172.21.15.84:0/1304047160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e681a1220 con 0x7f5e68103f40 2024-04-26T18:22:04.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e6dcfc700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e681035a0 0x7f5e6819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:22:04.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e6ecfe700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e68103f40 0x7f5e681a0950 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000ada0 tx=0x7f5e6000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:04.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e5f7fe700 1 -- 172.21.15.84:0/1304047160 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60017020 con 0x7f5e68103f40 2024-04-26T18:22:04.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e70761700 1 -- 172.21.15.84:0/1304047160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e681a1510 con 0x7f5e68103f40 2024-04-26T18:22:04.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e5f7fe700 1 -- 172.21.15.84:0/1304047160 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e6000ce30 con 0x7f5e68103f40 2024-04-26T18:22:04.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e5f7fe700 1 -- 172.21.15.84:0/1304047160 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e60005350 con 0x7f5e68103f40 2024-04-26T18:22:04.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.009+0000 7f5e70761700 1 -- 172.21.15.84:0/1304047160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e681a1a70 con 0x7f5e68103f40 2024-04-26T18:22:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.010+0000 7f5e70761700 1 -- 172.21.15.84:0/1304047160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e6806a110 con 0x7f5e68103f40 2024-04-26T18:22:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.011+0000 7f5e5f7fe700 1 -- 172.21.15.84:0/1304047160 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5e6000c980 con 0x7f5e68103f40 2024-04-26T18:22:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.012+0000 7f5e5f7fe700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f5e540a3c40 0x7f5e540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.012+0000 7f5e6e4fd700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f5e540a3c40 0x7f5e540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.012+0000 7f5e5f7fe700 1 -- 172.21.15.84:0/1304047160 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5858+0+0 (secure 0 0 0) 0x7f5e600cf730 con 0x7f5e68103f40 2024-04-26T18:22:04.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.012+0000 7f5e6e4fd700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f5e540a3c40 0x7f5e540a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5e68103400 tx=0x7f5e6400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:04.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.016+0000 7f5e5f7fe700 1 -- 172.21.15.84:0/1304047160 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e60021020 con 0x7f5e68103f40 2024-04-26T18:22:04.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:04.227+0000 7f5e70761700 1 -- 172.21.15.84:0/1304047160 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f5e68063860 con 0x7f5e540a3c40 2024-04-26T18:22:04.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:04 smithi084 ceph-mon[29141]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:04.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:04 smithi084 ceph-mon[29141]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T18:22:04.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:04 smithi103 ceph-mon[32727]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:04.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:04 smithi103 ceph-mon[32727]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T18:22:04.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:04 smithi174 ceph-mon[32705]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:04.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:04 smithi174 ceph-mon[32705]: osdmap e62: 9 total, 9 up, 9 in 2024-04-26T18:22:05.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:05 smithi084 ceph-mon[29141]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:22:05.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:22:05.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:22:05.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:05.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:05 smithi084 ceph-mon[29141]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T18:22:05.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:05 smithi103 ceph-mon[32727]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:22:05.645 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:22:05.645 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:22:05.645 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:05.645 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:05 smithi103 ceph-mon[32727]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T18:22:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:05 smithi174 ceph-mon[32705]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi084:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:22:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:22:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:22:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:05 smithi174 ceph-mon[32705]: osdmap e63: 9 total, 9 up, 9 in 2024-04-26T18:22:06.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:06 smithi084 ceph-mon[29141]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:06.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:06 smithi084 ceph-mon[29141]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T18:22:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:06 smithi103 ceph-mon[32727]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:06 smithi103 ceph-mon[32727]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T18:22:06.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:06 smithi174 ceph-mon[32705]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:06.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:06 smithi174 ceph-mon[32705]: osdmap e64: 9 total, 9 up, 9 in 2024-04-26T18:22:07.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:07 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3221469387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "059eb22d-6dfb-4dd0-b192-84ed42ee1fbc"}]: dispatch 2024-04-26T18:22:07.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:07 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3221469387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "059eb22d-6dfb-4dd0-b192-84ed42ee1fbc"}]': finished 2024-04-26T18:22:07.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:07 smithi084 ceph-mon[29141]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T18:22:07.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:07.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:07 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3221469387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "059eb22d-6dfb-4dd0-b192-84ed42ee1fbc"}]: dispatch 2024-04-26T18:22:07.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:07 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3221469387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "059eb22d-6dfb-4dd0-b192-84ed42ee1fbc"}]': finished 2024-04-26T18:22:07.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:07 smithi103 ceph-mon[32727]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T18:22:07.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:07.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:07 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3221469387' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "059eb22d-6dfb-4dd0-b192-84ed42ee1fbc"}]: dispatch 2024-04-26T18:22:07.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:07 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3221469387' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "059eb22d-6dfb-4dd0-b192-84ed42ee1fbc"}]': finished 2024-04-26T18:22:07.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:07 smithi174 ceph-mon[32705]: osdmap e65: 10 total, 9 up, 10 in 2024-04-26T18:22:07.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:08.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:08 smithi084 ceph-mon[29141]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T18:22:08.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:08 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2448887510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:22:08.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:08 smithi103 ceph-mon[32727]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T18:22:08.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:08 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2448887510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:22:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:08 smithi174 ceph-mon[32705]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T18:22:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:08 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2448887510' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:22:10.547 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:10 smithi084 ceph-mon[29141]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-26T18:22:10.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:10 smithi103 ceph-mon[32727]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-26T18:22:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:10 smithi174 ceph-mon[32705]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-04-26T18:22:12.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:12 smithi084 ceph-mon[29141]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-26T18:22:12.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:12 smithi103 ceph-mon[32727]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-26T18:22:12.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:12 smithi174 ceph-mon[32705]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-26T18:22:14.476 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:14 smithi084 ceph-mon[29141]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:22:14.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:14 smithi103 ceph-mon[32727]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:22:14.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:14 smithi174 ceph-mon[32705]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T18:22:15.393 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:15 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T18:22:15.394 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:15 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:15.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:15 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T18:22:15.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:15 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:15.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:15 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-26T18:22:15.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:15 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:16.415 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:16 smithi084 ceph-mon[29141]: Deploying daemon osd.9 on smithi084 2024-04-26T18:22:16.415 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:16 smithi084 ceph-mon[29141]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-26T18:22:16.528 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:16 smithi103 ceph-mon[32727]: Deploying daemon osd.9 on smithi084 2024-04-26T18:22:16.528 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:16 smithi103 ceph-mon[32727]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-26T18:22:16.590 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:16 smithi174 ceph-mon[32705]: Deploying daemon osd.9 on smithi084 2024-04-26T18:22:16.591 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:16 smithi174 ceph-mon[32705]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-04-26T18:22:18.470 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:18 smithi084 ceph-mon[29141]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:18.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:18 smithi103 ceph-mon[32727]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:18.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:18 smithi174 ceph-mon[32705]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:20.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:20 smithi084 ceph-mon[29141]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:20.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:20 smithi103 ceph-mon[32727]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:20.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:20 smithi174 ceph-mon[32705]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:21.814 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:21.814 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:21.815 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:21.815 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:21 smithi084 ceph-mon[29141]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:21 smithi103 ceph-mon[32727]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:21.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:21.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:21.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:21.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:21 smithi174 ceph-mon[32705]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:24.341 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:24 smithi084 ceph-mon[29141]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:24.341 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:24 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:24.341 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:24 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:24.341 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:24 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:24.341 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:24 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:24.341 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:24 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:24.435 INFO:teuthology.orchestra.run.smithi084.stdout:Created osd(s) 9 on host 'smithi084' 2024-04-26T18:22:24.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:24.434+0000 7f5e5f7fe700 1 -- 172.21.15.84:0/1304047160 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5e68063860 con 0x7f5e540a3c40 2024-04-26T18:22:24.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:24.436+0000 7f5e70761700 1 -- 172.21.15.84:0/1304047160 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f5e540a3c40 msgr2=0x7f5e540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:24.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:24.436+0000 7f5e70761700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f5e540a3c40 0x7f5e540a6100 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5e68103400 tx=0x7f5e6400a380 comp rx=0 tx=0).stop 2024-04-26T18:22:24.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:24.436+0000 7f5e70761700 1 -- 172.21.15.84:0/1304047160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e68103f40 msgr2=0x7f5e681a0950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:24.436+0000 7f5e70761700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e68103f40 0x7f5e681a0950 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5e6000ada0 tx=0x7f5e6000c330 comp rx=0 tx=0).stop 2024-04-26T18:22:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:24.436+0000 7f5e70761700 1 -- 172.21.15.84:0/1304047160 shutdown_connections 2024-04-26T18:22:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:24.436+0000 7f5e70761700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f5e540a3c40 0x7f5e540a6100 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:24.436+0000 7f5e70761700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5e68103f40 0x7f5e681a0950 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:24.436+0000 7f5e70761700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5e681023a0 0x7f5e6806c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:24.436+0000 7f5e70761700 1 --2- 172.21.15.84:0/1304047160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5e681035a0 0x7f5e6819e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:24.436+0000 7f5e70761700 1 -- 172.21.15.84:0/1304047160 >> 172.21.15.84:0/1304047160 conn(0x7f5e680fd980 msgr2=0x7f5e68106920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:24.436+0000 7f5e70761700 1 -- 172.21.15.84:0/1304047160 shutdown_connections 2024-04-26T18:22:24.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:22:24.436+0000 7f5e70761700 1 -- 172.21.15.84:0/1304047160 wait complete. 2024-04-26T18:22:24.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:24 smithi103 ceph-mon[32727]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:24.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:24 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:24.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:24 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:24.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:24 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:24.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:24 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:24.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:24 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:24.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:24 smithi174 ceph-mon[32705]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:24.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:24 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:24.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:24 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:24.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:24 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:24.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:24 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:24.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:24 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:25.103 DEBUG:teuthology.orchestra.run.smithi084:osd.9> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.9.service 2024-04-26T18:22:25.105 INFO:tasks.cephadm:Deploying osd.10 on smithi103 with /dev/vg_nvme/lv_1... 2024-04-26T18:22:25.106 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T18:22:25.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:25 smithi084 ceph-mon[29141]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T18:22:25.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:25 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:25.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:25 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:25.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:25 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:25.353 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:22:25.475 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:25 smithi103 ceph-mon[32727]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T18:22:25.476 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:25 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:25.476 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:25 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:25.476 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:25 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:25.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:25 smithi174 ceph-mon[32705]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-26T18:22:25.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:25 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:25.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:25 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:25.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:25 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:26.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:26 smithi084 ceph-mon[29141]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:26.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:26 smithi084 ceph-mon[29141]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T18:22:26.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:26 smithi084 ceph-mon[29141]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T18:22:26.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:26 smithi084 ceph-mon[29141]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-04-26T18:22:26.283 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:26 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:26.534 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:26 smithi103 ceph-mon[32727]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:26.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:26 smithi103 ceph-mon[32727]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T18:22:26.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:26 smithi103 ceph-mon[32727]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T18:22:26.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:26 smithi103 ceph-mon[32727]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-04-26T18:22:26.535 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:26 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:26.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:26 smithi174 ceph-mon[32705]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:26.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:26 smithi174 ceph-mon[32705]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-26T18:22:26.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:26 smithi174 ceph-mon[32705]: osdmap e66: 10 total, 9 up, 10 in 2024-04-26T18:22:26.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:26 smithi174 ceph-mon[32705]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]: dispatch 2024-04-26T18:22:26.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:26 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:27.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:22:26 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:22:26.652+0000 7f35691fb700 -1 osd.9 0 waiting for initial osdmap 2024-04-26T18:22:27.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:22:26 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:22:26.664+0000 7f3560fe6700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:22:27.402 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:27 smithi103 ceph-mon[32727]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-04-26T18:22:27.402 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:27 smithi103 ceph-mon[32727]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T18:22:27.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:27 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:27.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:27 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:27.403 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:27 smithi103 ceph-mon[32727]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' 2024-04-26T18:22:27.478 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T18:22:27.478 INFO:teuthology.orchestra.run.smithi103.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T18:22:27.478 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10+0 records in 2024-04-26T18:22:27.478 INFO:teuthology.orchestra.run.smithi103.stderr:10+0 records out 2024-04-26T18:22:27.478 INFO:teuthology.orchestra.run.smithi103.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0128864 s, 814 MB/s 2024-04-26T18:22:27.478 INFO:teuthology.orchestra.run.smithi103.stderr:--> Zapping successful for: 2024-04-26T18:22:27.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:27 smithi084 ceph-mon[29141]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-04-26T18:22:27.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:27 smithi084 ceph-mon[29141]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T18:22:27.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:27 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:27.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:27 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:27.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:27 smithi084 ceph-mon[29141]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' 2024-04-26T18:22:27.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:27 smithi174 ceph-mon[32705]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi084", "root=default"]}]': finished 2024-04-26T18:22:27.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:27 smithi174 ceph-mon[32705]: osdmap e67: 10 total, 9 up, 10 in 2024-04-26T18:22:27.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:27 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:27.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:27 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:27.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:27 smithi174 ceph-mon[32705]: from='osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927]' entity='osd.9' 2024-04-26T18:22:28.215 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi103:vg_nvme/lv_1 2024-04-26T18:22:28.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:28 smithi084 ceph-mon[29141]: purged_snaps scrub starts 2024-04-26T18:22:28.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:28 smithi084 ceph-mon[29141]: purged_snaps scrub ok 2024-04-26T18:22:28.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:28 smithi084 ceph-mon[29141]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:28.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:28 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:28.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:28 smithi084 ceph-mon[29141]: osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927] boot 2024-04-26T18:22:28.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:28 smithi084 ceph-mon[29141]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T18:22:28.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:28 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:28.456 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:22:28.578 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:28 smithi103 ceph-mon[32727]: purged_snaps scrub starts 2024-04-26T18:22:28.578 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:28 smithi103 ceph-mon[32727]: purged_snaps scrub ok 2024-04-26T18:22:28.578 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:28 smithi103 ceph-mon[32727]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:28.578 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:28 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:28.579 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:28 smithi103 ceph-mon[32727]: osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927] boot 2024-04-26T18:22:28.579 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:28 smithi103 ceph-mon[32727]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T18:22:28.579 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:28 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:28.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:28 smithi174 ceph-mon[32705]: purged_snaps scrub starts 2024-04-26T18:22:28.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:28 smithi174 ceph-mon[32705]: purged_snaps scrub ok 2024-04-26T18:22:28.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:28 smithi174 ceph-mon[32705]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-26T18:22:28.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:28 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:28.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:28 smithi174 ceph-mon[32705]: osd.9 [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927] boot 2024-04-26T18:22:28.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:28 smithi174 ceph-mon[32705]: osdmap e68: 10 total, 10 up, 10 in 2024-04-26T18:22:28.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:28 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:22:29.579 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: Detected new or changed devices on smithi084 2024-04-26T18:22:29.579 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:29.580 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:29.580 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:29.580 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:29.580 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:29.580 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:29.580 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi084 to 4272M 2024-04-26T18:22:29.580 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:29.580 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:29.580 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:29.580 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:29.580 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:29.580 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:29 smithi103 ceph-mon[32727]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T18:22:29.793 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: Detected new or changed devices on smithi084 2024-04-26T18:22:29.794 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:29.794 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:29.794 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:29.794 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:29.794 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:29.794 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:29.794 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi084 to 4272M 2024-04-26T18:22:29.794 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:29.794 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:29.794 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:29.794 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:29.795 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:29.795 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:29 smithi084 ceph-mon[29141]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T18:22:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: Detected new or changed devices on smithi084 2024-04-26T18:22:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:29.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:29.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:29.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi084 to 4272M 2024-04-26T18:22:29.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:29.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:29.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:29.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:29.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:29.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:29 smithi174 ceph-mon[32705]: osdmap e69: 10 total, 10 up, 10 in 2024-04-26T18:22:30.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.117+0000 7fd01ceab700 1 -- 172.21.15.103:0/1061530870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd018103fd0 msgr2=0x7fd01810a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:30.118 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.117+0000 7fd01ceab700 1 --2- 172.21.15.103:0/1061530870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd018103fd0 0x7fd01810a860 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fd01000d330 tx=0x7fd01000d640 comp rx=0 tx=0).stop 2024-04-26T18:22:30.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.117+0000 7fd01ceab700 1 -- 172.21.15.103:0/1061530870 shutdown_connections 2024-04-26T18:22:30.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.117+0000 7fd01ceab700 1 --2- 172.21.15.103:0/1061530870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0181028f0 0x7fd018102d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:30.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.117+0000 7fd01ceab700 1 --2- 172.21.15.103:0/1061530870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd018103fd0 0x7fd01810a860 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:30.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.117+0000 7fd01ceab700 1 --2- 172.21.15.103:0/1061530870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd018103610 0x7fd018103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:30.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.117+0000 7fd01ceab700 1 -- 172.21.15.103:0/1061530870 >> 172.21.15.103:0/1061530870 conn(0x7fd0180fd0f0 msgr2=0x7fd0180ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:30.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.117+0000 7fd01ceab700 1 -- 172.21.15.103:0/1061530870 shutdown_connections 2024-04-26T18:22:30.119 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.117+0000 7fd01ceab700 1 -- 172.21.15.103:0/1061530870 wait complete. 2024-04-26T18:22:30.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.118+0000 7fd01ceab700 1 Processor -- start 2024-04-26T18:22:30.121 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.118+0000 7fd01ceab700 1 -- start start 2024-04-26T18:22:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01ceab700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0181028f0 0x7fd018197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01ceab700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018103610 0x7fd018197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01ceab700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd018103fd0 0x7fd01819e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01ceab700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810ce10 con 0x7fd018103610 2024-04-26T18:22:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01ceab700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810cc90 con 0x7fd0181028f0 2024-04-26T18:22:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01ceab700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd01810cf90 con 0x7fd018103fd0 2024-04-26T18:22:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0181028f0 0x7fd018197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0181028f0 0x7fd018197300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.103:34080/0 (socket says 172.21.15.103:34080) 2024-04-26T18:22:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01659c700 1 -- 172.21.15.103:0/624313469 learned_addr learned my addr 172.21.15.103:0/624313469 (peer_addr_for_me v2:172.21.15.103:0/0) 2024-04-26T18:22:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd015d9b700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018103610 0x7fd018197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd016d9d700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd018103fd0 0x7fd01819e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01659c700 1 -- 172.21.15.103:0/624313469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd018103fd0 msgr2=0x7fd01819e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:30.122 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01659c700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd018103fd0 0x7fd01819e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:30.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01659c700 1 -- 172.21.15.103:0/624313469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018103610 msgr2=0x7fd018197840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:30.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01659c700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018103610 0x7fd018197840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:30.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01659c700 1 -- 172.21.15.103:0/624313469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd01819ef60 con 0x7fd0181028f0 2024-04-26T18:22:30.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01659c700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0181028f0 0x7fd018197300 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd00800e970 tx=0x7fd00800ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:30.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd0077fe700 1 -- 172.21.15.103:0/624313469 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008004220 con 0x7fd0181028f0 2024-04-26T18:22:30.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd0077fe700 1 -- 172.21.15.103:0/624313469 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0080043c0 con 0x7fd0181028f0 2024-04-26T18:22:30.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01ceab700 1 -- 172.21.15.103:0/624313469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd01819f250 con 0x7fd0181028f0 2024-04-26T18:22:30.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd0077fe700 1 -- 172.21.15.103:0/624313469 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd008003ce0 con 0x7fd0181028f0 2024-04-26T18:22:30.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.119+0000 7fd01ceab700 1 -- 172.21.15.103:0/624313469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd01819f790 con 0x7fd0181028f0 2024-04-26T18:22:30.123 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.120+0000 7fd01ceab700 1 -- 172.21.15.103:0/624313469 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd01806a110 con 0x7fd0181028f0 2024-04-26T18:22:30.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.121+0000 7fd0077fe700 1 -- 172.21.15.103:0/624313469 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd00800c7e0 con 0x7fd0181028f0 2024-04-26T18:22:30.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.122+0000 7fd0077fe700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd0000a3c40 0x7fd0000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:30.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.122+0000 7fd0077fe700 1 -- 172.21.15.103:0/624313469 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 6261+0+0 (secure 0 0 0) 0x7fd0080cff30 con 0x7fd0181028f0 2024-04-26T18:22:30.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.122+0000 7fd015d9b700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd0000a3c40 0x7fd0000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:30.124 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.123+0000 7fd015d9b700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd0000a3c40 0x7fd0000a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd018102620 tx=0x7fd00c007400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:30.126 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.126+0000 7fd0077fe700 1 -- 172.21.15.103:0/624313469 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd00809d1a0 con 0x7fd0181028f0 2024-04-26T18:22:30.338 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:30.334+0000 7fd01ceab700 1 -- 172.21.15.103:0/624313469 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fd018063860 con 0x7fd0000a3c40 2024-04-26T18:22:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:30 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:22:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:30 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:22:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:30 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:30.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:30 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:22:30.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:30 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:22:30.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:30 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:30.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:30 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:22:30.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:30 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:22:30.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:30 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:31.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:31 smithi103 ceph-mon[32727]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:22:31.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:31 smithi103 ceph-mon[32727]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:31.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:31 smithi084 ceph-mon[29141]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:22:31.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:31 smithi084 ceph-mon[29141]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:31.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:31 smithi174 ceph-mon[32705]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi103:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:22:31.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:31 smithi174 ceph-mon[32705]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:34.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:34 smithi084 ceph-mon[29141]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:34 smithi084 ceph-mon[29141]: from='client.? 172.21.15.103:0/4059358401' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da31e305-5220-4a87-a131-005b7c7f76b6"}]: dispatch 2024-04-26T18:22:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:34 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da31e305-5220-4a87-a131-005b7c7f76b6"}]: dispatch 2024-04-26T18:22:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:34 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da31e305-5220-4a87-a131-005b7c7f76b6"}]': finished 2024-04-26T18:22:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:34 smithi084 ceph-mon[29141]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T18:22:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:34 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:34 smithi084 ceph-mon[29141]: from='client.? 172.21.15.103:0/3734340406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:22:34.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:34 smithi103 ceph-mon[32727]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:34.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:34 smithi103 ceph-mon[32727]: from='client.? 172.21.15.103:0/4059358401' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da31e305-5220-4a87-a131-005b7c7f76b6"}]: dispatch 2024-04-26T18:22:34.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:34 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da31e305-5220-4a87-a131-005b7c7f76b6"}]: dispatch 2024-04-26T18:22:34.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:34 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da31e305-5220-4a87-a131-005b7c7f76b6"}]': finished 2024-04-26T18:22:34.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:34 smithi103 ceph-mon[32727]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T18:22:34.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:34 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:34.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:34 smithi103 ceph-mon[32727]: from='client.? 172.21.15.103:0/3734340406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:22:34.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:34 smithi174 ceph-mon[32705]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:34.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:34 smithi174 ceph-mon[32705]: from='client.? 172.21.15.103:0/4059358401' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da31e305-5220-4a87-a131-005b7c7f76b6"}]: dispatch 2024-04-26T18:22:34.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:34 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "da31e305-5220-4a87-a131-005b7c7f76b6"}]: dispatch 2024-04-26T18:22:34.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:34 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "da31e305-5220-4a87-a131-005b7c7f76b6"}]': finished 2024-04-26T18:22:34.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:34 smithi174 ceph-mon[32705]: osdmap e70: 11 total, 10 up, 11 in 2024-04-26T18:22:34.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:34 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:34.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:34 smithi174 ceph-mon[32705]: from='client.? 172.21.15.103:0/3734340406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:22:36.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:36 smithi084 ceph-mon[29141]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:36.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:36 smithi103 ceph-mon[32727]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:36.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:36 smithi174 ceph-mon[32705]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:38.477 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:38 smithi103 ceph-mon[32727]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:38.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:38 smithi084 ceph-mon[29141]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:38.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:38 smithi174 ceph-mon[32705]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:40.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:40 smithi103 ceph-mon[32727]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:40.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:40 smithi084 ceph-mon[29141]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:40 smithi174 ceph-mon[32705]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:41.502 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T18:22:41.502 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:41.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T18:22:41.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:41.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-26T18:22:41.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:42.586 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:42 smithi103 ceph-mon[32727]: Deploying daemon osd.10 on smithi103 2024-04-26T18:22:42.586 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:42 smithi103 ceph-mon[32727]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:42.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:42 smithi084 ceph-mon[29141]: Deploying daemon osd.10 on smithi103 2024-04-26T18:22:42.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:42 smithi084 ceph-mon[29141]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:42.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:42 smithi174 ceph-mon[32705]: Deploying daemon osd.10 on smithi103 2024-04-26T18:22:42.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:42 smithi174 ceph-mon[32705]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:44.553 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:44 smithi103 ceph-mon[32727]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:44.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:44 smithi084 ceph-mon[29141]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:44.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:44 smithi174 ceph-mon[32705]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:46.264 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:46 smithi103 ceph-mon[32727]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:46.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:46 smithi084 ceph-mon[29141]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:46.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:46 smithi174 ceph-mon[32705]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:47.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:47 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:47.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:47 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:47.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:47 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:47.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:47 smithi103 ceph-mon[32727]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:47.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:47 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:47.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:47 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:47.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:47 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:47.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:47 smithi174 ceph-mon[32705]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:48.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:47 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:48.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:47 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:48.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:47 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:48.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:47 smithi084 ceph-mon[29141]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:50.517 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:50 smithi103 ceph-mon[32727]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:50.517 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:50 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:50.518 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:50 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:50.518 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:50 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:50.518 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:50 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:50.518 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:50 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:50.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:50 smithi084 ceph-mon[29141]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:50.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:50 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:50.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:50 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:50.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:50 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:50.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:50 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:50.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:50 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:50.606 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:50.604+0000 7fd0077fe700 1 -- 172.21.15.103:0/624313469 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fd018063860 con 0x7fd0000a3c40 2024-04-26T18:22:50.606 INFO:teuthology.orchestra.run.smithi103.stdout:Created osd(s) 10 on host 'smithi103' 2024-04-26T18:22:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:50.606+0000 7fd01ceab700 1 -- 172.21.15.103:0/624313469 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd0000a3c40 msgr2=0x7fd0000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:50.606+0000 7fd01ceab700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd0000a3c40 0x7fd0000a6100 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd018102620 tx=0x7fd00c007400 comp rx=0 tx=0).stop 2024-04-26T18:22:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:50.606+0000 7fd01ceab700 1 -- 172.21.15.103:0/624313469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0181028f0 msgr2=0x7fd018197300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:50.606+0000 7fd01ceab700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0181028f0 0x7fd018197300 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd00800e970 tx=0x7fd00800ec80 comp rx=0 tx=0).stop 2024-04-26T18:22:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:50.606+0000 7fd01ceab700 1 -- 172.21.15.103:0/624313469 shutdown_connections 2024-04-26T18:22:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:50.606+0000 7fd01ceab700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd0000a3c40 0x7fd0000a6100 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:50.606+0000 7fd01ceab700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd018103fd0 0x7fd01819e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:50.606+0000 7fd01ceab700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0181028f0 0x7fd018197300 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.607 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:50.606+0000 7fd01ceab700 1 --2- 172.21.15.103:0/624313469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd018103610 0x7fd018197840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:50.606+0000 7fd01ceab700 1 -- 172.21.15.103:0/624313469 >> 172.21.15.103:0/624313469 conn(0x7fd0180fd0f0 msgr2=0x7fd018106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:50.606+0000 7fd01ceab700 1 -- 172.21.15.103:0/624313469 shutdown_connections 2024-04-26T18:22:50.608 INFO:teuthology.orchestra.run.smithi103.stderr:2024-04-26T18:22:50.606+0000 7fd01ceab700 1 -- 172.21.15.103:0/624313469 wait complete. 2024-04-26T18:22:50.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:50 smithi174 ceph-mon[32705]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:50.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:50.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:50.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:50.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:50.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:50 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:51.308 DEBUG:teuthology.orchestra.run.smithi103:osd.10> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.10.service 2024-04-26T18:22:51.310 INFO:tasks.cephadm:Deploying osd.11 on smithi174 with /dev/vg_nvme/lv_1... 2024-04-26T18:22:51.311 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-26T18:22:51.557 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:22:51.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:51.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:51 smithi174 ceph-mon[32705]: from='osd.10 [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:22:51.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:51 smithi174 ceph-mon[32705]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:22:51.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:51.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:51.682 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:51 smithi174 ceph-mon[32705]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:51.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:51.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:51 smithi084 ceph-mon[29141]: from='osd.10 [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:22:51.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:51 smithi084 ceph-mon[29141]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:22:51.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:51.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:51.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:51 smithi084 ceph-mon[29141]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:51.860 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:51.860 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:51 smithi103 ceph-mon[32727]: from='osd.10 [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:22:51.860 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:51 smithi103 ceph-mon[32727]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-26T18:22:51.860 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:51.860 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:22:51.860 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:51 smithi103 ceph-mon[32727]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:52.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:52 smithi084 ceph-mon[29141]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T18:22:52.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:52 smithi084 ceph-mon[29141]: from='osd.10 [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:22:52.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:52 smithi084 ceph-mon[29141]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T18:22:52.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:52 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:52.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:52 smithi084 ceph-mon[29141]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:22:52.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:52 smithi084 ceph-mon[29141]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-26T18:22:52.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:52 smithi084 ceph-mon[29141]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T18:22:52.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:52 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:52.854 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:52 smithi103 ceph-mon[32727]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T18:22:52.854 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:52 smithi103 ceph-mon[32727]: from='osd.10 [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:22:52.855 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:52 smithi103 ceph-mon[32727]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T18:22:52.855 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:52 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:52.855 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:52 smithi103 ceph-mon[32727]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:22:52.855 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:52 smithi103 ceph-mon[32727]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-26T18:22:52.855 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:52 smithi103 ceph-mon[32727]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T18:22:52.855 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:52 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:52.889 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:52 smithi174 ceph-mon[32705]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-26T18:22:52.889 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:52 smithi174 ceph-mon[32705]: from='osd.10 [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:22:52.889 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:52 smithi174 ceph-mon[32705]: osdmap e71: 11 total, 10 up, 11 in 2024-04-26T18:22:52.889 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:52 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:52.890 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:52 smithi174 ceph-mon[32705]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]: dispatch 2024-04-26T18:22:52.890 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:52 smithi174 ceph-mon[32705]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi103", "root=default"]}]': finished 2024-04-26T18:22:52.890 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:52 smithi174 ceph-mon[32705]: osdmap e72: 11 total, 10 up, 11 in 2024-04-26T18:22:52.890 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:52 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:53.111 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:22:53 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:22:53.011+0000 7faa8e8d2700 -1 osd.10 0 waiting for initial osdmap 2024-04-26T18:22:53.111 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:22:53 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:22:53.025+0000 7faa866bd700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:22:53.786 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:53 smithi174 ceph-mon[32705]: purged_snaps scrub starts 2024-04-26T18:22:53.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:53 smithi174 ceph-mon[32705]: purged_snaps scrub ok 2024-04-26T18:22:53.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:53 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:53.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:53 smithi174 ceph-mon[32705]: from='osd.10 ' entity='osd.10' 2024-04-26T18:22:53.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:53 smithi174 ceph-mon[32705]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:53.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:53 smithi174 ceph-mon[32705]: osd.10 [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915] boot 2024-04-26T18:22:53.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:53 smithi174 ceph-mon[32705]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T18:22:53.787 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:53 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:53.787 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-26T18:22:53.787 INFO:teuthology.orchestra.run.smithi174.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-26T18:22:53.787 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 10+0 records in 2024-04-26T18:22:53.787 INFO:teuthology.orchestra.run.smithi174.stderr:10+0 records out 2024-04-26T18:22:53.787 INFO:teuthology.orchestra.run.smithi174.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114821 s, 913 MB/s 2024-04-26T18:22:53.788 INFO:teuthology.orchestra.run.smithi174.stderr: stderr: 2024-04-26T18:22:53.788 INFO:teuthology.orchestra.run.smithi174.stderr:--> Zapping successful for: 2024-04-26T18:22:53.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:53 smithi084 ceph-mon[29141]: purged_snaps scrub starts 2024-04-26T18:22:53.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:53 smithi084 ceph-mon[29141]: purged_snaps scrub ok 2024-04-26T18:22:53.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:53 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:53.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:53 smithi084 ceph-mon[29141]: from='osd.10 ' entity='osd.10' 2024-04-26T18:22:53.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:53 smithi084 ceph-mon[29141]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:53.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:53 smithi084 ceph-mon[29141]: osd.10 [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915] boot 2024-04-26T18:22:53.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:53 smithi084 ceph-mon[29141]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T18:22:53.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:53 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:53.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:53 smithi103 ceph-mon[32727]: purged_snaps scrub starts 2024-04-26T18:22:53.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:53 smithi103 ceph-mon[32727]: purged_snaps scrub ok 2024-04-26T18:22:53.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:53 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:53.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:53 smithi103 ceph-mon[32727]: from='osd.10 ' entity='osd.10' 2024-04-26T18:22:53.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:53 smithi103 ceph-mon[32727]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 295 MiB used, 894 GiB / 894 GiB avail 2024-04-26T18:22:53.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:53 smithi103 ceph-mon[32727]: osd.10 [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915] boot 2024-04-26T18:22:53.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:53 smithi103 ceph-mon[32727]: osdmap e73: 11 total, 11 up, 11 in 2024-04-26T18:22:53.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:53 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:22:54.509 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi174:vg_nvme/lv_1 2024-04-26T18:22:54.756 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:22:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: Detected new or changed devices on smithi103 2024-04-26T18:22:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi103 to 4272M 2024-04-26T18:22:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T18:22:55.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:55 smithi084 ceph-mon[29141]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:22:55.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: Detected new or changed devices on smithi103 2024-04-26T18:22:55.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:55.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:55.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:55.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:55.854 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:55.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:55.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi103 to 4272M 2024-04-26T18:22:55.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:55.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:55.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:55.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:55.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T18:22:55.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:55 smithi174 ceph-mon[32705]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:22:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: Detected new or changed devices on smithi103 2024-04-26T18:22:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:22:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi103 to 4272M 2024-04-26T18:22:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:55.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:22:55.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:22:55.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: osdmap e74: 11 total, 11 up, 11 in 2024-04-26T18:22:55.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:55 smithi103 ceph-mon[32727]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:22:56.404 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.403+0000 7f3b9cf19700 1 -- 172.21.15.174:0/3301965776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b981050f0 msgr2=0x7f3b9810b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:56.404 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.404+0000 7f3b9cf19700 1 --2- 172.21.15.174:0/3301965776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b981050f0 0x7f3b9810b9a0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3b9000b0a0 tx=0x7f3b9000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.404 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.404+0000 7f3b9cf19700 1 -- 172.21.15.174:0/3301965776 shutdown_connections 2024-04-26T18:22:56.404 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.404+0000 7f3b9cf19700 1 --2- 172.21.15.174:0/3301965776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b98101770 0x7f3b98101b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.405 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.404+0000 7f3b9cf19700 1 --2- 172.21.15.174:0/3301965776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b981050f0 0x7f3b9810b9a0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.405 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.404+0000 7f3b9cf19700 1 --2- 172.21.15.174:0/3301965776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b98104750 0x7f3b98104bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.405 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.404+0000 7f3b9cf19700 1 -- 172.21.15.174:0/3301965776 >> 172.21.15.174:0/3301965776 conn(0x7f3b980fd0d0 msgr2=0x7f3b980ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:22:56.405 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.404+0000 7f3b9cf19700 1 -- 172.21.15.174:0/3301965776 shutdown_connections 2024-04-26T18:22:56.405 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.404+0000 7f3b9cf19700 1 -- 172.21.15.174:0/3301965776 wait complete. 2024-04-26T18:22:56.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.404+0000 7f3b9cf19700 1 Processor -- start 2024-04-26T18:22:56.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9cf19700 1 -- start start 2024-04-26T18:22:56.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9cf19700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b98101770 0x7f3b981995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:56.407 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9cf19700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b98104750 0x7f3b98199b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:56.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9cf19700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b981050f0 0x7f3b981a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:56.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9cf19700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9810ef90 con 0x7f3b98104750 2024-04-26T18:22:56.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9cf19700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9810ee10 con 0x7f3b981050f0 2024-04-26T18:22:56.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9cf19700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3b9810f110 con 0x7f3b98101770 2024-04-26T18:22:56.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b98101770 0x7f3b981995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:56.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9659c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b98101770 0x7f3b981995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:55496/0 (socket says 172.21.15.174:55496) 2024-04-26T18:22:56.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9659c700 1 -- 172.21.15.174:0/1063862746 learned_addr learned my addr 172.21.15.174:0/1063862746 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T18:22:56.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b95d9b700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b98104750 0x7f3b98199b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:56.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b96d9d700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b981050f0 0x7f3b981a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:56.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9659c700 1 -- 172.21.15.174:0/1063862746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b981050f0 msgr2=0x7f3b981a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:56.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9659c700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b981050f0 0x7f3b981a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.408 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9659c700 1 -- 172.21.15.174:0/1063862746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b98104750 msgr2=0x7f3b98199b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:22:56.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9659c700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b98104750 0x7f3b98199b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:22:56.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9659c700 1 -- 172.21.15.174:0/1063862746 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b981a1230 con 0x7f3b98101770 2024-04-26T18:22:56.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9659c700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b98101770 0x7f3b981995d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800d9b0 tx=0x7f3b8800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:56.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b877fe700 1 -- 172.21.15.174:0/1063862746 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b88017030 con 0x7f3b98101770 2024-04-26T18:22:56.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b9cf19700 1 -- 172.21.15.174:0/1063862746 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b981a1520 con 0x7f3b98101770 2024-04-26T18:22:56.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b877fe700 1 -- 172.21.15.174:0/1063862746 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b880044d0 con 0x7f3b98101770 2024-04-26T18:22:56.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.405+0000 7f3b877fe700 1 -- 172.21.15.174:0/1063862746 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b880039e0 con 0x7f3b98101770 2024-04-26T18:22:56.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.406+0000 7f3b9cf19700 1 -- 172.21.15.174:0/1063862746 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b981a1ae0 con 0x7f3b98101770 2024-04-26T18:22:56.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.407+0000 7f3b9cf19700 1 -- 172.21.15.174:0/1063862746 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b9806a110 con 0x7f3b98101770 2024-04-26T18:22:56.409 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.407+0000 7f3b877fe700 1 -- 172.21.15.174:0/1063862746 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3b8801a070 con 0x7f3b98101770 2024-04-26T18:22:56.410 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.408+0000 7f3b877fe700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3b800a3c40 0x7f3b800a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:22:56.410 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.408+0000 7f3b877fe700 1 -- 172.21.15.174:0/1063862746 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6723+0+0 (secure 0 0 0) 0x7f3b880cf720 con 0x7f3b98101770 2024-04-26T18:22:56.410 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.408+0000 7f3b95d9b700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3b800a3c40 0x7f3b800a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:22:56.410 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.409+0000 7f3b95d9b700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3b800a3c40 0x7f3b800a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009a10 tx=0x7f3b8c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:22:56.412 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.413+0000 7f3b877fe700 1 -- 172.21.15.174:0/1063862746 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3b880232f0 con 0x7f3b98101770 2024-04-26T18:22:56.623 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:22:56.623+0000 7f3b9cf19700 1 -- 172.21.15.174:0/1063862746 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f3b98063860 con 0x7f3b800a3c40 2024-04-26T18:22:56.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:56 smithi084 ceph-mon[29141]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T18:22:56.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:56 smithi103 ceph-mon[32727]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T18:22:56.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:56 smithi174 ceph-mon[32705]: osdmap e75: 11 total, 11 up, 11 in 2024-04-26T18:22:57.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:57 smithi084 ceph-mon[29141]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T18:22:57.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:57 smithi084 ceph-mon[29141]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:22:57.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:22:57.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:22:57.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:57.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:57 smithi084 ceph-mon[29141]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T18:22:57.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:57 smithi103 ceph-mon[32727]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T18:22:57.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:57 smithi103 ceph-mon[32727]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:22:57.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:22:57.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:22:57.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:57.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:57 smithi103 ceph-mon[32727]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T18:22:57.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:57 smithi174 ceph-mon[32705]: osdmap e76: 11 total, 11 up, 11 in 2024-04-26T18:22:57.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:57 smithi174 ceph-mon[32705]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi174:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:22:57.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T18:22:57.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T18:22:57.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:22:57.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:57 smithi174 ceph-mon[32705]: pgmap v242: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-26T18:22:58.812 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:58 smithi174 ceph-mon[32705]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T18:22:58.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:58 smithi084 ceph-mon[29141]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T18:22:58.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:58 smithi103 ceph-mon[32727]: osdmap e77: 11 total, 11 up, 11 in 2024-04-26T18:22:59.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:59 smithi084 ceph-mon[29141]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-26T18:22:59.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:59 smithi084 ceph-mon[29141]: from='client.? 172.21.15.174:0/3889959548' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fde6c8f-ee02-41ee-a34f-73072b4761e0"}]: dispatch 2024-04-26T18:22:59.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:59 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fde6c8f-ee02-41ee-a34f-73072b4761e0"}]: dispatch 2024-04-26T18:22:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:59 smithi084 ceph-mon[29141]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fde6c8f-ee02-41ee-a34f-73072b4761e0"}]': finished 2024-04-26T18:22:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:59 smithi084 ceph-mon[29141]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T18:22:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:22:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:22:59.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:59 smithi103 ceph-mon[32727]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-26T18:22:59.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:59 smithi103 ceph-mon[32727]: from='client.? 172.21.15.174:0/3889959548' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fde6c8f-ee02-41ee-a34f-73072b4761e0"}]: dispatch 2024-04-26T18:22:59.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:59 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fde6c8f-ee02-41ee-a34f-73072b4761e0"}]: dispatch 2024-04-26T18:22:59.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:59 smithi103 ceph-mon[32727]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fde6c8f-ee02-41ee-a34f-73072b4761e0"}]': finished 2024-04-26T18:22:59.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:59 smithi103 ceph-mon[32727]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T18:22:59.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:22:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:22:59.903 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:59 smithi174 ceph-mon[32705]: pgmap v244: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 126 KiB/s, 0 objects/s recovering 2024-04-26T18:22:59.903 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:59 smithi174 ceph-mon[32705]: from='client.? 172.21.15.174:0/3889959548' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fde6c8f-ee02-41ee-a34f-73072b4761e0"}]: dispatch 2024-04-26T18:22:59.903 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:59 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3fde6c8f-ee02-41ee-a34f-73072b4761e0"}]: dispatch 2024-04-26T18:22:59.903 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:59 smithi174 ceph-mon[32705]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3fde6c8f-ee02-41ee-a34f-73072b4761e0"}]': finished 2024-04-26T18:22:59.903 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:59 smithi174 ceph-mon[32705]: osdmap e78: 12 total, 11 up, 12 in 2024-04-26T18:22:59.903 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:22:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:00.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:00 smithi084 ceph-mon[29141]: from='client.? 172.21.15.174:0/3588195699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:23:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:00 smithi103 ceph-mon[32727]: from='client.? 172.21.15.174:0/3588195699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:23:00.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:00 smithi174 ceph-mon[32705]: from='client.? 172.21.15.174:0/3588195699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T18:23:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:01 smithi103 ceph-mon[32727]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-26T18:23:01.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:01 smithi174 ceph-mon[32705]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-26T18:23:02.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:01 smithi084 ceph-mon[29141]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-04-26T18:23:04.438 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:04 smithi174 ceph-mon[32705]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:04.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:04 smithi084 ceph-mon[29141]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:04.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:04 smithi103 ceph-mon[32727]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:06.500 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:06 smithi174 ceph-mon[32705]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:06.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:06 smithi084 ceph-mon[29141]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:06.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:06 smithi103 ceph-mon[32727]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T18:23:07.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:07.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T18:23:07.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:07.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-26T18:23:07.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:08.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:08 smithi174 ceph-mon[32705]: Deploying daemon osd.11 on smithi174 2024-04-26T18:23:08.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:08 smithi174 ceph-mon[32705]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:08.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:08 smithi084 ceph-mon[29141]: Deploying daemon osd.11 on smithi174 2024-04-26T18:23:08.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:08 smithi084 ceph-mon[29141]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:08.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:08 smithi103 ceph-mon[32727]: Deploying daemon osd.11 on smithi174 2024-04-26T18:23:08.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:08 smithi103 ceph-mon[32727]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:10.314 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:10 smithi174 ceph-mon[32705]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:10.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:10 smithi084 ceph-mon[29141]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:10.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:10 smithi103 ceph-mon[32727]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:12.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:12 smithi084 ceph-mon[29141]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:12.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:12 smithi103 ceph-mon[32727]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:12.654 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:12 smithi174 ceph-mon[32705]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:14.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:14 smithi174 ceph-mon[32705]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:14.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:14 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:14.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:14 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:14.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:14 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:14.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:14 smithi084 ceph-mon[29141]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:14.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:14 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:14.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:14 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:14.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:14 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:14.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:14 smithi103 ceph-mon[32727]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:14.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:14 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:14.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:14 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:14.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:14 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:16.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:16 smithi174 ceph-mon[32705]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:16.569 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:16 smithi103 ceph-mon[32727]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:16.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:16 smithi084 ceph-mon[29141]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:17.292 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:17.292+0000 7f3b877fe700 1 -- 172.21.15.174:0/1063862746 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f3b98063860 con 0x7f3b800a3c40 2024-04-26T18:23:17.292 INFO:teuthology.orchestra.run.smithi174.stdout:Created osd(s) 11 on host 'smithi174' 2024-04-26T18:23:17.294 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:17.294+0000 7f3b9cf19700 1 -- 172.21.15.174:0/1063862746 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3b800a3c40 msgr2=0x7f3b800a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:17.294 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:17.294+0000 7f3b9cf19700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3b800a3c40 0x7f3b800a6100 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c009a10 tx=0x7f3b8c009190 comp rx=0 tx=0).stop 2024-04-26T18:23:17.294 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:17.294+0000 7f3b9cf19700 1 -- 172.21.15.174:0/1063862746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b98101770 msgr2=0x7f3b981995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:17.294 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:17.294+0000 7f3b9cf19700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b98101770 0x7f3b981995d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f3b8800d9b0 tx=0x7f3b8800dcc0 comp rx=0 tx=0).stop 2024-04-26T18:23:17.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:17.294+0000 7f3b9cf19700 1 -- 172.21.15.174:0/1063862746 shutdown_connections 2024-04-26T18:23:17.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:17.294+0000 7f3b9cf19700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3b800a3c40 0x7f3b800a6100 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:17.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:17.294+0000 7f3b9cf19700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3b981050f0 0x7f3b981a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:17.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:17.294+0000 7f3b9cf19700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3b98101770 0x7f3b981995d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:17.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:17.294+0000 7f3b9cf19700 1 --2- 172.21.15.174:0/1063862746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3b98104750 0x7f3b98199b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:17.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:17.294+0000 7f3b9cf19700 1 -- 172.21.15.174:0/1063862746 >> 172.21.15.174:0/1063862746 conn(0x7f3b980fd0d0 msgr2=0x7f3b98107a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:17.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:17.294+0000 7f3b9cf19700 1 -- 172.21.15.174:0/1063862746 shutdown_connections 2024-04-26T18:23:17.295 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:17.294+0000 7f3b9cf19700 1 -- 172.21.15.174:0/1063862746 wait complete. 2024-04-26T18:23:17.751 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.751 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.751 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:17.751 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:17.751 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.751 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:17 smithi103 ceph-mon[32727]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:17.751 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:17 smithi103 ceph-mon[32727]: from='osd.11 [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:23:17.751 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:17 smithi103 ceph-mon[32727]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:23:17.752 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.752 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.752 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:17 smithi084 ceph-mon[29141]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:17 smithi084 ceph-mon[29141]: from='osd.11 [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:23:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:17 smithi084 ceph-mon[29141]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:23:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:17.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:17.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:17.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:17 smithi174 ceph-mon[32705]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 327 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:17.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:17 smithi174 ceph-mon[32705]: from='osd.11 [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:23:17.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:17 smithi174 ceph-mon[32705]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-26T18:23:17.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:17.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:18.003 DEBUG:teuthology.orchestra.run.smithi174:osd.11> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.11.service 2024-04-26T18:23:18.005 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-26T18:23:18.006 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T18:23:18.255 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:18.633 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:18 smithi084 ceph-mon[29141]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T18:23:18.633 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:18 smithi084 ceph-mon[29141]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T18:23:18.633 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:18 smithi084 ceph-mon[29141]: from='osd.11 [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:23:18.634 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:18.634 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:18 smithi084 ceph-mon[29141]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:23:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:18 smithi103 ceph-mon[32727]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T18:23:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:18 smithi103 ceph-mon[32727]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T18:23:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:18 smithi103 ceph-mon[32727]: from='osd.11 [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:23:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:18 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:18 smithi103 ceph-mon[32727]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:23:18.903 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:18 smithi174 ceph-mon[32705]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-26T18:23:18.903 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:18 smithi174 ceph-mon[32705]: osdmap e79: 12 total, 11 up, 12 in 2024-04-26T18:23:18.904 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:18 smithi174 ceph-mon[32705]: from='osd.11 [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:23:18.904 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:18 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:18.904 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:18 smithi174 ceph-mon[32705]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]: dispatch 2024-04-26T18:23:19.189 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:23:18 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:23:18.902+0000 7f3306439700 -1 osd.11 0 waiting for initial osdmap 2024-04-26T18:23:19.189 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:23:18 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:23:18.915+0000 7f32fda23700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T18:23:19.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:19 smithi174 ceph-mon[32705]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-04-26T18:23:19.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:19 smithi174 ceph-mon[32705]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T18:23:19.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:19 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:19.582 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:19 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:19.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:19 smithi174 ceph-mon[32705]: from='osd.11 ' entity='osd.11' 2024-04-26T18:23:19.583 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:19 smithi174 ceph-mon[32705]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:19.713 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:19 smithi084 ceph-mon[29141]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-04-26T18:23:19.713 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:19 smithi084 ceph-mon[29141]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T18:23:19.713 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:19 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:19.713 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:19 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:19.713 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:19 smithi084 ceph-mon[29141]: from='osd.11 ' entity='osd.11' 2024-04-26T18:23:19.713 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:19 smithi084 ceph-mon[29141]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:19 smithi103 ceph-mon[32727]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi174", "root=default"]}]': finished 2024-04-26T18:23:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:19 smithi103 ceph-mon[32727]: osdmap e80: 12 total, 11 up, 12 in 2024-04-26T18:23:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:19 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:19 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:19 smithi103 ceph-mon[32727]: from='osd.11 ' entity='osd.11' 2024-04-26T18:23:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:19 smithi103 ceph-mon[32727]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-26T18:23:20.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.076+0000 7fd831c69700 1 -- 172.21.15.84:0/2266173802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd82c105020 msgr2=0x7fd82c10b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:20.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.076+0000 7fd831c69700 1 --2- 172.21.15.84:0/2266173802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd82c105020 0x7fd82c10b8d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd82400b0d0 tx=0x7fd82400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:23:20.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.077+0000 7fd831c69700 1 -- 172.21.15.84:0/2266173802 shutdown_connections 2024-04-26T18:23:20.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.077+0000 7fd831c69700 1 --2- 172.21.15.84:0/2266173802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd82c105020 0x7fd82c10b8d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:20.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.077+0000 7fd831c69700 1 --2- 172.21.15.84:0/2266173802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd82c103480 0x7fd82c1038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:20.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.077+0000 7fd831c69700 1 --2- 172.21.15.84:0/2266173802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd82c104680 0x7fd82c104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:20.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.077+0000 7fd831c69700 1 -- 172.21.15.84:0/2266173802 >> 172.21.15.84:0/2266173802 conn(0x7fd82c0fea20 msgr2=0x7fd82c100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:20.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.077+0000 7fd831c69700 1 -- 172.21.15.84:0/2266173802 shutdown_connections 2024-04-26T18:23:20.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.077+0000 7fd831c69700 1 -- 172.21.15.84:0/2266173802 wait complete. 2024-04-26T18:23:20.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.077+0000 7fd831c69700 1 Processor -- start 2024-04-26T18:23:20.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.077+0000 7fd831c69700 1 -- start start 2024-04-26T18:23:20.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd831c69700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd82c103480 0x7fd82c199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:20.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd831c69700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd82c104680 0x7fd82c199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:20.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd831c69700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd82c105020 0x7fd82c1a0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:20.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd831c69700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c10ef00 con 0x7fd82c103480 2024-04-26T18:23:20.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd831c69700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c10ed80 con 0x7fd82c104680 2024-04-26T18:23:20.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd831c69700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd82c10f080 con 0x7fd82c105020 2024-04-26T18:23:20.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd82b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd82c103480 0x7fd82c199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:20.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd82b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd82c103480 0x7fd82c199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41240/0 (socket says 172.21.15.84:41240) 2024-04-26T18:23:20.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd82b7fe700 1 -- 172.21.15.84:0/545676059 learned_addr learned my addr 172.21.15.84:0/545676059 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:23:20.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd82bfff700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd82c105020 0x7fd82c1a0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:20.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd82affd700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd82c104680 0x7fd82c199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:20.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd82b7fe700 1 -- 172.21.15.84:0/545676059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd82c105020 msgr2=0x7fd82c1a0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:20.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd82b7fe700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd82c105020 0x7fd82c1a0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:20.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd82b7fe700 1 -- 172.21.15.84:0/545676059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd82c104680 msgr2=0x7fd82c199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:20.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd82b7fe700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd82c104680 0x7fd82c199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:20.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd82b7fe700 1 -- 172.21.15.84:0/545676059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd82c1a1170 con 0x7fd82c103480 2024-04-26T18:23:20.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd82b7fe700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd82c103480 0x7fd82c199510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00d9b0 tx=0x7fd81c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:20.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd828ff9700 1 -- 172.21.15.84:0/545676059 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c017030 con 0x7fd82c103480 2024-04-26T18:23:20.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd828ff9700 1 -- 172.21.15.84:0/545676059 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd81c0044d0 con 0x7fd82c103480 2024-04-26T18:23:20.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd828ff9700 1 -- 172.21.15.84:0/545676059 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd81c0039e0 con 0x7fd82c103480 2024-04-26T18:23:20.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd831c69700 1 -- 172.21.15.84:0/545676059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd82c1a1460 con 0x7fd82c103480 2024-04-26T18:23:20.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.078+0000 7fd831c69700 1 -- 172.21.15.84:0/545676059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd82c1a1950 con 0x7fd82c103480 2024-04-26T18:23:20.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.079+0000 7fd831c69700 1 -- 172.21.15.84:0/545676059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd82c06a110 con 0x7fd82c103480 2024-04-26T18:23:20.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.080+0000 7fd828ff9700 1 -- 172.21.15.84:0/545676059 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fd81c01a070 con 0x7fd82c103480 2024-04-26T18:23:20.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.081+0000 7fd828ff9700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd8140a3cc0 0x7fd8140a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:20.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.081+0000 7fd82affd700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd8140a3cc0 0x7fd8140a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:20.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.081+0000 7fd828ff9700 1 -- 172.21.15.84:0/545676059 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fd81c0cf7e0 con 0x7fd82c103480 2024-04-26T18:23:20.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.081+0000 7fd82affd700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd8140a3cc0 0x7fd8140a6180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd820005950 tx=0x7fd8200058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:20.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.086+0000 7fd828ff9700 1 -- 172.21.15.84:0/545676059 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd81c09c660 con 0x7fd82c103480 2024-04-26T18:23:20.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.283+0000 7fd831c69700 1 -- 172.21.15.84:0/545676059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fd82c064030 con 0x7fd82c103480 2024-04-26T18:23:20.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.284+0000 7fd828ff9700 1 -- 172.21.15.84:0/545676059 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) ==== 74+0+133 (secure 0 0 0) 0x7fd81c09bf60 con 0x7fd82c103480 2024-04-26T18:23:20.284 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:23:20.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.286+0000 7fd831c69700 1 -- 172.21.15.84:0/545676059 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd8140a3cc0 msgr2=0x7fd8140a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:20.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.286+0000 7fd831c69700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd8140a3cc0 0x7fd8140a6180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd820005950 tx=0x7fd8200058e0 comp rx=0 tx=0).stop 2024-04-26T18:23:20.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.286+0000 7fd831c69700 1 -- 172.21.15.84:0/545676059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd82c103480 msgr2=0x7fd82c199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:20.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.286+0000 7fd831c69700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd82c103480 0x7fd82c199510 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fd81c00d9b0 tx=0x7fd81c00dcc0 comp rx=0 tx=0).stop 2024-04-26T18:23:20.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.286+0000 7fd831c69700 1 -- 172.21.15.84:0/545676059 shutdown_connections 2024-04-26T18:23:20.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.286+0000 7fd831c69700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd8140a3cc0 0x7fd8140a6180 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:20.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.286+0000 7fd831c69700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd82c103480 0x7fd82c199510 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:20.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.286+0000 7fd831c69700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd82c105020 0x7fd82c1a0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:20.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.286+0000 7fd831c69700 1 --2- 172.21.15.84:0/545676059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd82c104680 0x7fd82c199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:20.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.286+0000 7fd831c69700 1 -- 172.21.15.84:0/545676059 >> 172.21.15.84:0/545676059 conn(0x7fd82c0fea20 msgr2=0x7fd82c107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:20.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.286+0000 7fd831c69700 1 -- 172.21.15.84:0/545676059 shutdown_connections 2024-04-26T18:23:20.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:20.286+0000 7fd831c69700 1 -- 172.21.15.84:0/545676059 wait complete. 2024-04-26T18:23:20.572 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:20 smithi084 ceph-mon[29141]: purged_snaps scrub starts 2024-04-26T18:23:20.572 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:20 smithi084 ceph-mon[29141]: purged_snaps scrub ok 2024-04-26T18:23:20.572 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:20.572 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:20 smithi084 ceph-mon[29141]: osd.11 [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128] boot 2024-04-26T18:23:20.572 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:20 smithi084 ceph-mon[29141]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T18:23:20.572 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:20.572 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:20 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/545676059' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T18:23:20.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:20 smithi103 ceph-mon[32727]: purged_snaps scrub starts 2024-04-26T18:23:20.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:20 smithi103 ceph-mon[32727]: purged_snaps scrub ok 2024-04-26T18:23:20.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:20.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:20 smithi103 ceph-mon[32727]: osd.11 [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128] boot 2024-04-26T18:23:20.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:20 smithi103 ceph-mon[32727]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T18:23:20.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:20.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:20 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/545676059' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T18:23:20.926 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:20 smithi174 ceph-mon[32705]: purged_snaps scrub starts 2024-04-26T18:23:20.926 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:20 smithi174 ceph-mon[32705]: purged_snaps scrub ok 2024-04-26T18:23:20.926 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:20.926 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:20 smithi174 ceph-mon[32705]: osd.11 [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128] boot 2024-04-26T18:23:20.926 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:20 smithi174 ceph-mon[32705]: osdmap e81: 12 total, 12 up, 12 in 2024-04-26T18:23:20.926 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:23:20.926 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:20 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/545676059' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T18:23:21.008 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1714155799,"num_in_osds":12,"osd_in_since":1714155779,"num_remapped_pgs":0} 2024-04-26T18:23:21.009 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:23:21.253 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:22.073 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:21 smithi084 ceph-mon[29141]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T18:23:22.073 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:21 smithi084 ceph-mon[29141]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:22.073 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:22.074 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:22.074 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:23:22.074 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:23:22.074 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:23:22.074 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:23:22.074 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:22.074 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:22.074 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:22.074 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:22.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:21 smithi174 ceph-mon[32705]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T18:23:22.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:21 smithi174 ceph-mon[32705]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:22.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:22.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:22.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:23:22.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:23:22.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:23:22.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:23:22.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:22.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:22.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:22.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:22.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:21 smithi103 ceph-mon[32727]: osdmap e82: 12 total, 12 up, 12 in 2024-04-26T18:23:22.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:21 smithi103 ceph-mon[32727]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:22.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:22.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:22.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:23:22.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:23:22.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:23:22.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:23:22.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:22.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:22.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:22.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:22.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.920+0000 7fa68ce3e700 1 -- 172.21.15.84:0/2076341296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa68806b9c0 msgr2=0x7fa68806be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.920+0000 7fa68ce3e700 1 --2- 172.21.15.84:0/2076341296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa68806b9c0 0x7fa68806be20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa67c009990 tx=0x7fa67c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.920+0000 7fa68ce3e700 1 -- 172.21.15.84:0/2076341296 shutdown_connections 2024-04-26T18:23:22.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.920+0000 7fa68ce3e700 1 --2- 172.21.15.84:0/2076341296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa68806c360 0x7fa6881102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.920+0000 7fa68ce3e700 1 --2- 172.21.15.84:0/2076341296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa688109490 0x7fa688109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.920+0000 7fa68ce3e700 1 --2- 172.21.15.84:0/2076341296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa68806b9c0 0x7fa68806be20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.920+0000 7fa68ce3e700 1 -- 172.21.15.84:0/2076341296 >> 172.21.15.84:0/2076341296 conn(0x7fa688079290 msgr2=0x7fa6880796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:22.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.920+0000 7fa68ce3e700 1 -- 172.21.15.84:0/2076341296 shutdown_connections 2024-04-26T18:23:22.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.920+0000 7fa68ce3e700 1 -- 172.21.15.84:0/2076341296 wait complete. 2024-04-26T18:23:22.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.921+0000 7fa68ce3e700 1 Processor -- start 2024-04-26T18:23:22.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.921+0000 7fa68ce3e700 1 -- start start 2024-04-26T18:23:22.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.921+0000 7fa68ce3e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa68806b9c0 0x7fa6881023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.921+0000 7fa68ce3e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa68806c360 0x7fa688102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.921+0000 7fa68ce3e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa688109490 0x7fa688102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.921+0000 7fa68ce3e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa688113680 con 0x7fa688109490 2024-04-26T18:23:22.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.921+0000 7fa68ce3e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa688113500 con 0x7fa68806b9c0 2024-04-26T18:23:22.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.921+0000 7fa68ce3e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa688113800 con 0x7fa68806c360 2024-04-26T18:23:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.922+0000 7fa68659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa68806b9c0 0x7fa6881023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.922+0000 7fa685d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa68806c360 0x7fa688102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.922+0000 7fa686d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa688109490 0x7fa688102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.922+0000 7fa68659c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa68806b9c0 0x7fa6881023e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:53864/0 (socket says 172.21.15.84:53864) 2024-04-26T18:23:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.922+0000 7fa685d9b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa68806c360 0x7fa688102920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.84:33892/0 (socket says 172.21.15.84:33892) 2024-04-26T18:23:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.922+0000 7fa68659c700 1 -- 172.21.15.84:0/275014632 learned_addr learned my addr 172.21.15.84:0/275014632 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:23:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.922+0000 7fa686d9d700 1 -- 172.21.15.84:0/275014632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa68806c360 msgr2=0x7fa688102920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.922+0000 7fa686d9d700 1 --2- 172.21.15.84:0/275014632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa68806c360 0x7fa688102920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.922+0000 7fa686d9d700 1 -- 172.21.15.84:0/275014632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa68806b9c0 msgr2=0x7fa6881023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.922+0000 7fa686d9d700 1 --2- 172.21.15.84:0/275014632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa68806b9c0 0x7fa6881023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:22.926 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.922+0000 7fa686d9d700 1 -- 172.21.15.84:0/275014632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6881a8230 con 0x7fa688109490 2024-04-26T18:23:22.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.922+0000 7fa686d9d700 1 --2- 172.21.15.84:0/275014632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa688109490 0x7fa688102e60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa67800b3f0 tx=0x7fa67800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:22.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.924+0000 7fa6777fe700 1 -- 172.21.15.84:0/275014632 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678015020 con 0x7fa688109490 2024-04-26T18:23:22.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.924+0000 7fa6777fe700 1 -- 172.21.15.84:0/275014632 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa678011ae0 con 0x7fa688109490 2024-04-26T18:23:22.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.924+0000 7fa68ce3e700 1 -- 172.21.15.84:0/275014632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6881a8520 con 0x7fa688109490 2024-04-26T18:23:22.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.925+0000 7fa6777fe700 1 -- 172.21.15.84:0/275014632 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa678010bd0 con 0x7fa688109490 2024-04-26T18:23:22.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.925+0000 7fa68ce3e700 1 -- 172.21.15.84:0/275014632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6881a8a30 con 0x7fa688109490 2024-04-26T18:23:22.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.925+0000 7fa68ce3e700 1 -- 172.21.15.84:0/275014632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa68810ba00 con 0x7fa688109490 2024-04-26T18:23:22.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.926+0000 7fa6777fe700 1 -- 172.21.15.84:0/275014632 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa678018020 con 0x7fa688109490 2024-04-26T18:23:22.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.927+0000 7fa6777fe700 1 --2- 172.21.15.84:0/275014632 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa6700a3cc0 0x7fa6700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:22.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.927+0000 7fa68659c700 1 --2- 172.21.15.84:0/275014632 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa6700a3cc0 0x7fa6700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:22.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.927+0000 7fa6777fe700 1 -- 172.21.15.84:0/275014632 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fa6780cf480 con 0x7fa688109490 2024-04-26T18:23:22.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.927+0000 7fa68659c700 1 --2- 172.21.15.84:0/275014632 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa6700a3cc0 0x7fa6700a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa68000e250 tx=0x7fa68000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:22.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:22.932+0000 7fa6777fe700 1 -- 172.21.15.84:0/275014632 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa67801f020 con 0x7fa688109490 2024-04-26T18:23:23.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:22 smithi084 ceph-mon[29141]: Detected new or changed devices on smithi174 2024-04-26T18:23:23.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:22 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi174 to 4272M 2024-04-26T18:23:23.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.138+0000 7fa68ce3e700 1 -- 172.21.15.84:0/275014632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa688103910 con 0x7fa688109490 2024-04-26T18:23:23.139 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.139+0000 7fa6777fe700 1 -- 172.21.15.84:0/275014632 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+18420 (secure 0 0 0) 0x7fa67809c300 con 0x7fa688109490 2024-04-26T18:23:23.139 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:23:23.140 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":82,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:16:06.267694+0000","modified":"2024-04-26T18:23:20.912504+0000","last_up_change":"2024-04-26T18:23:19.906511+0000","last_in_change":"2024-04-26T18:22:59.541869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:19:31.174024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1076912-986e-4825-b1d1-5fa9d69bc07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6803","nonce":140050011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6805","nonce":140050011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6809","nonce":140050011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6807","nonce":140050011}]},"public_addr":"172.21.15.84:6803/140050011","cluster_addr":"172.21.15.84:6805/140050011","heartbeat_back_addr":"172.21.15.84:6809/140050011","heartbeat_front_addr":"172.21.15.84:6807/140050011","state":["exists","up"]},{"osd":1,"uuid":"fa93293d-9058-4804-ba4b-7e1e674b0fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6801","nonce":2512857657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6803","nonce":2512857657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6807","nonce":2512857657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6805","nonce":2512857657}]},"public_addr":"172.21.15.103:6801/2512857657","cluster_addr":"172.21.15.103:6803/2512857657","heartbeat_back_addr":"172.21.15.103:6807/2512857657","heartbeat_front_addr":"172.21.15.103:6805/2512857657","state":["exists","up"]},{"osd":2,"uuid":"bf78a44f-80a2-4cd7-974f-5abdf771e650","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6801","nonce":76340924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6803","nonce":76340924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6807","nonce":76340924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6805","nonce":76340924}]},"public_addr":"172.21.15.174:6801/76340924","cluster_addr":"172.21.15.174:6803/76340924","heartbeat_back_addr":"172.21.15.174:6807/76340924","heartbeat_front_addr":"172.21.15.174:6805/76340924","state":["exists","up"]},{"osd":3,"uuid":"55fd0035-bc76-4235-b8b5-5cf5ef770139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6811","nonce":2352789256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6813","nonce":2352789256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6817","nonce":2352789256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6815","nonce":2352789256}]},"public_addr":"172.21.15.84:6811/2352789256","cluster_addr":"172.21.15.84:6813/2352789256","heartbeat_back_addr":"172.21.15.84:6817/2352789256","heartbeat_front_addr":"172.21.15.84:6815/2352789256","state":["exists","up"]},{"osd":4,"uuid":"c278a0f3-e436-4d1c-a009-f9165ebf3c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6809","nonce":1619258252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6811","nonce":1619258252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6815","nonce":1619258252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6813","nonce":1619258252}]},"public_addr":"172.21.15.103:6809/1619258252","cluster_addr":"172.21.15.103:6811/1619258252","heartbeat_back_addr":"172.21.15.103:6815/1619258252","heartbeat_front_addr":"172.21.15.103:6813/1619258252","state":["exists","up"]},{"osd":5,"uuid":"6505be9b-c1b0-4d8b-944e-9920f90cb88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6809","nonce":3168291768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6811","nonce":3168291768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6815","nonce":3168291768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6813","nonce":3168291768}]},"public_addr":"172.21.15.174:6809/3168291768","cluster_addr":"172.21.15.174:6811/3168291768","heartbeat_back_addr":"172.21.15.174:6815/3168291768","heartbeat_front_addr":"172.21.15.174:6813/3168291768","state":["exists","up"]},{"osd":6,"uuid":"a037b66a-7488-469d-b73a-14cc0d35cb5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6819","nonce":886635297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6821","nonce":886635297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6825","nonce":886635297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6823","nonce":886635297}]},"public_addr":"172.21.15.84:6819/886635297","cluster_addr":"172.21.15.84:6821/886635297","heartbeat_back_addr":"172.21.15.84:6825/886635297","heartbeat_front_addr":"172.21.15.84:6823/886635297","state":["exists","up"]},{"osd":7,"uuid":"c425a13f-8019-4484-86a0-a34fa4d8de81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6817","nonce":1738278079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6819","nonce":1738278079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6823","nonce":1738278079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6821","nonce":1738278079}]},"public_addr":"172.21.15.103:6817/1738278079","cluster_addr":"172.21.15.103:6819/1738278079","heartbeat_back_addr":"172.21.15.103:6823/1738278079","heartbeat_front_addr":"172.21.15.103:6821/1738278079","state":["exists","up"]},{"osd":8,"uuid":"687e6c08-bfa4-440c-b3bc-90a5748989d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6817","nonce":38410548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6819","nonce":38410548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6823","nonce":38410548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6821","nonce":38410548}]},"public_addr":"172.21.15.174:6817/38410548","cluster_addr":"172.21.15.174:6819/38410548","heartbeat_back_addr":"172.21.15.174:6823/38410548","heartbeat_front_addr":"172.21.15.174:6821/38410548","state":["exists","up"]},{"osd":9,"uuid":"059eb22d-6dfb-4dd0-b192-84ed42ee1fbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6827","nonce":2926393927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6829","nonce":2926393927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6833","nonce":2926393927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6831","nonce":2926393927}]},"public_addr":"172.21.15.84:6827/2926393927","cluster_addr":"172.21.15.84:6829/2926393927","heartbeat_back_addr":"172.21.15.84:6833/2926393927","heartbeat_front_addr":"172.21.15.84:6831/2926393927","state":["exists","up"]},{"osd":10,"uuid":"da31e305-5220-4a87-a131-005b7c7f76b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6825","nonce":2705726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6827","nonce":2705726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6831","nonce":2705726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6829","nonce":2705726915}]},"public_addr":"172.21.15.103:6825/2705726915","cluster_addr":"172.21.15.103:6827/2705726915","heartbeat_back_addr":"172.21.15.103:6831/2705726915","heartbeat_front_addr":"172.21.15.103:6829/2705726915","state":["exists","up"]},{"osd":11,"uuid":"3fde6c8f-ee02-41ee-a34f-73072b4761e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6825","nonce":1668672128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6827","nonce":1668672128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6831","nonce":1668672128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6829","nonce":1668672128}]},"public_addr":"172.21.15.174:6825/1668672128","cluster_addr":"172.21.15.174:6827/1668672128","heartbeat_back_addr":"172.21.15.174:6831/1668672128","heartbeat_front_addr":"172.21.15.174:6829/1668672128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:39.099188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:03.327608+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:27.714108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:52.506015+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:17.631670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:42.702489+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:07.606697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:34.123214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:59.534658+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:25.357720+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:51.543106+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:23:18.208230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1604680842":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6800/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/538363975":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/952052990":"2024-04-27T18:17:05.083192+0000","172.21.15.84:0/1266942705":"2024-04-27T18:16:37.859669+0000","172.21.15.84:6800/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3970542149":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3607125049":"2024-04-27T18:17:05.083192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:23:23.141 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.141+0000 7fa68ce3e700 1 -- 172.21.15.84:0/275014632 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa6700a3cc0 msgr2=0x7fa6700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:23.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.141+0000 7fa68ce3e700 1 --2- 172.21.15.84:0/275014632 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa6700a3cc0 0x7fa6700a6180 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa68000e250 tx=0x7fa68000b040 comp rx=0 tx=0).stop 2024-04-26T18:23:23.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.141+0000 7fa68ce3e700 1 -- 172.21.15.84:0/275014632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa688109490 msgr2=0x7fa688102e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:23.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.141+0000 7fa68ce3e700 1 --2- 172.21.15.84:0/275014632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa688109490 0x7fa688102e60 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fa67800b3f0 tx=0x7fa67800b700 comp rx=0 tx=0).stop 2024-04-26T18:23:23.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.141+0000 7fa68ce3e700 1 -- 172.21.15.84:0/275014632 shutdown_connections 2024-04-26T18:23:23.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.141+0000 7fa68ce3e700 1 --2- 172.21.15.84:0/275014632 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa6700a3cc0 0x7fa6700a6180 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:23.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.141+0000 7fa68ce3e700 1 --2- 172.21.15.84:0/275014632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa688109490 0x7fa688102e60 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:23.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.141+0000 7fa68ce3e700 1 --2- 172.21.15.84:0/275014632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa68806b9c0 0x7fa6881023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:23.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.141+0000 7fa68ce3e700 1 --2- 172.21.15.84:0/275014632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa68806c360 0x7fa688102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:23.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.141+0000 7fa68ce3e700 1 -- 172.21.15.84:0/275014632 >> 172.21.15.84:0/275014632 conn(0x7fa688079290 msgr2=0x7fa6881004c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:23.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.141+0000 7fa68ce3e700 1 -- 172.21.15.84:0/275014632 shutdown_connections 2024-04-26T18:23:23.142 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:23.141+0000 7fa68ce3e700 1 -- 172.21.15.84:0/275014632 wait complete. 2024-04-26T18:23:23.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:22 smithi174 ceph-mon[32705]: Detected new or changed devices on smithi174 2024-04-26T18:23:23.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:22 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi174 to 4272M 2024-04-26T18:23:23.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:22 smithi103 ceph-mon[32727]: Detected new or changed devices on smithi174 2024-04-26T18:23:23.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:22 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi174 to 4272M 2024-04-26T18:23:23.796 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T18:19:31.174024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T18:23:23.796 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T18:23:24.034 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:24.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:23 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/275014632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:23:24.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:23 smithi084 ceph-mon[29141]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:24.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:23 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/275014632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:23:24.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:23 smithi174 ceph-mon[32705]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:24.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:23 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/275014632' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:23:24.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:23 smithi103 ceph-mon[32727]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:25.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.767+0000 7fb93db93700 1 -- 172.21.15.84:0/3519732139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9381094c0 msgr2=0x7fb9381098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:25.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.767+0000 7fb93db93700 1 --2- 172.21.15.84:0/3519732139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9381094c0 0x7fb9381098a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fb928009990 tx=0x7fb928009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.767+0000 7fb93db93700 1 -- 172.21.15.84:0/3519732139 shutdown_connections 2024-04-26T18:23:25.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.767+0000 7fb93db93700 1 --2- 172.21.15.84:0/3519732139 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb9381094c0 0x7fb9381098a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.767+0000 7fb93db93700 1 --2- 172.21.15.84:0/3519732139 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb938103e30 0x7fb938110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.767+0000 7fb93db93700 1 --2- 172.21.15.84:0/3519732139 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb938103470 0x7fb9381038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.767+0000 7fb93db93700 1 -- 172.21.15.84:0/3519732139 >> 172.21.15.84:0/3519732139 conn(0x7fb93807a580 msgr2=0x7fb93807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:25.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.767+0000 7fb93db93700 1 -- 172.21.15.84:0/3519732139 shutdown_connections 2024-04-26T18:23:25.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.767+0000 7fb93db93700 1 -- 172.21.15.84:0/3519732139 wait complete. 2024-04-26T18:23:25.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb93db93700 1 Processor -- start 2024-04-26T18:23:25.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb93db93700 1 -- start start 2024-04-26T18:23:25.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb93db93700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb938103470 0x7fb93819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:25.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb93db93700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb938103e30 0x7fb93819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:25.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb93db93700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9381094c0 0x7fb938199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:25.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb93db93700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb938113640 con 0x7fb938103470 2024-04-26T18:23:25.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb93db93700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381134c0 con 0x7fb938103e30 2024-04-26T18:23:25.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb93db93700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381137c0 con 0x7fb9381094c0 2024-04-26T18:23:25.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb9377fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb938103470 0x7fb93819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:25.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb9377fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb938103470 0x7fb93819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41280/0 (socket says 172.21.15.84:41280) 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb9377fe700 1 -- 172.21.15.84:0/1573350358 learned_addr learned my addr 172.21.15.84:0/1573350358 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb936ffd700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb938103e30 0x7fb93819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb937fff700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9381094c0 0x7fb938199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb9377fe700 1 -- 172.21.15.84:0/1573350358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9381094c0 msgr2=0x7fb938199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb9377fe700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9381094c0 0x7fb938199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb9377fe700 1 -- 172.21.15.84:0/1573350358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb938103e30 msgr2=0x7fb93819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb9377fe700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb938103e30 0x7fb93819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.768+0000 7fb9377fe700 1 -- 172.21.15.84:0/1573350358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93819a0e0 con 0x7fb938103470 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.769+0000 7fb9377fe700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb938103470 0x7fb93819f7c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb928000c00 tx=0x7fb9280047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.769+0000 7fb934ff9700 1 -- 172.21.15.84:0/1573350358 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928027070 con 0x7fb938103470 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.769+0000 7fb934ff9700 1 -- 172.21.15.84:0/1573350358 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb928003f20 con 0x7fb938103470 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.769+0000 7fb93db93700 1 -- 172.21.15.84:0/1573350358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93819a370 con 0x7fb938103470 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.769+0000 7fb934ff9700 1 -- 172.21.15.84:0/1573350358 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928022cf0 con 0x7fb938103470 2024-04-26T18:23:25.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.769+0000 7fb93db93700 1 -- 172.21.15.84:0/1573350358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb938075650 con 0x7fb938103470 2024-04-26T18:23:25.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.770+0000 7fb93db93700 1 -- 172.21.15.84:0/1573350358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb938104b60 con 0x7fb938103470 2024-04-26T18:23:25.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.772+0000 7fb934ff9700 1 -- 172.21.15.84:0/1573350358 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb92802a070 con 0x7fb938103470 2024-04-26T18:23:25.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.772+0000 7fb934ff9700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb9200a3c40 0x7fb9200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:25.772 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.772+0000 7fb936ffd700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb9200a3c40 0x7fb9200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:25.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.772+0000 7fb934ff9700 1 -- 172.21.15.84:0/1573350358 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb9280d0f60 con 0x7fb938103470 2024-04-26T18:23:25.773 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.773+0000 7fb936ffd700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb9200a3c40 0x7fb9200a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb93807ac60 tx=0x7fb92c00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:25.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.776+0000 7fb934ff9700 1 -- 172.21.15.84:0/1573350358 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb928031080 con 0x7fb938103470 2024-04-26T18:23:25.971 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.971+0000 7fb93db93700 1 -- 172.21.15.84:0/1573350358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb93806a110 con 0x7fb938103470 2024-04-26T18:23:25.971 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.971+0000 7fb934ff9700 1 -- 172.21.15.84:0/1573350358 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) ==== 93+0+10 (secure 0 0 0) 0x7fb928013070 con 0x7fb938103470 2024-04-26T18:23:25.971 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-04-26T18:23:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.973+0000 7fb93db93700 1 -- 172.21.15.84:0/1573350358 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb9200a3c40 msgr2=0x7fb9200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:25.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.973+0000 7fb93db93700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb9200a3c40 0x7fb9200a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb93807ac60 tx=0x7fb92c00b410 comp rx=0 tx=0).stop 2024-04-26T18:23:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.973+0000 7fb93db93700 1 -- 172.21.15.84:0/1573350358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb938103470 msgr2=0x7fb93819f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.973+0000 7fb93db93700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb938103470 0x7fb93819f7c0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb928000c00 tx=0x7fb9280047b0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.973+0000 7fb93db93700 1 -- 172.21.15.84:0/1573350358 shutdown_connections 2024-04-26T18:23:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.973+0000 7fb93db93700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb9200a3c40 0x7fb9200a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.973+0000 7fb93db93700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb938103470 0x7fb93819f7c0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.973+0000 7fb93db93700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb9381094c0 0x7fb938199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.973+0000 7fb93db93700 1 --2- 172.21.15.84:0/1573350358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb938103e30 0x7fb93819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.973+0000 7fb93db93700 1 -- 172.21.15.84:0/1573350358 >> 172.21.15.84:0/1573350358 conn(0x7fb93807a580 msgr2=0x7fb938102270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.973+0000 7fb93db93700 1 -- 172.21.15.84:0/1573350358 shutdown_connections 2024-04-26T18:23:25.974 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:25.973+0000 7fb93db93700 1 -- 172.21.15.84:0/1573350358 wait complete. 2024-04-26T18:23:26.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:26 smithi084 ceph-mon[29141]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:26.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:26 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1573350358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:23:26.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:26 smithi103 ceph-mon[32727]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:26.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:26 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1573350358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:23:26.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:26 smithi174 ceph-mon[32705]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:26.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:26 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1573350358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:23:26.814 INFO:tasks.cephadm:Adding mds.a on smithi084 2024-04-26T18:23:26.815 INFO:tasks.cephadm:Adding mds.d on smithi084 2024-04-26T18:23:26.815 INFO:tasks.cephadm:Adding mds.g on smithi084 2024-04-26T18:23:26.815 INFO:tasks.cephadm:Adding mds.j on smithi084 2024-04-26T18:23:26.815 INFO:tasks.cephadm:Adding mds.b on smithi103 2024-04-26T18:23:26.815 INFO:tasks.cephadm:Adding mds.e on smithi103 2024-04-26T18:23:26.815 INFO:tasks.cephadm:Adding mds.h on smithi103 2024-04-26T18:23:26.815 INFO:tasks.cephadm:Adding mds.k on smithi103 2024-04-26T18:23:26.815 INFO:tasks.cephadm:Adding mds.c on smithi174 2024-04-26T18:23:26.815 INFO:tasks.cephadm:Adding mds.f on smithi174 2024-04-26T18:23:26.815 INFO:tasks.cephadm:Adding mds.i on smithi174 2024-04-26T18:23:26.815 INFO:tasks.cephadm:Adding mds.l on smithi174 2024-04-26T18:23:26.815 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l' 2024-04-26T18:23:27.070 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:23:28.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:28 smithi174 ceph-mon[32705]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:28.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:28 smithi084 ceph-mon[29141]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:28.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:28 smithi103 ceph-mon[32727]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:28.685 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.682+0000 7fc756f2c700 1 -- 172.21.15.174:0/503741714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc750109ac0 msgr2=0x7fc750109ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.685 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.682+0000 7fc756f2c700 1 --2- 172.21.15.174:0/503741714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc750109ac0 0x7fc750109ee0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc740009990 tx=0x7fc740009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.685 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.682+0000 7fc756f2c700 1 -- 172.21.15.174:0/503741714 shutdown_connections 2024-04-26T18:23:28.685 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.682+0000 7fc756f2c700 1 --2- 172.21.15.174:0/503741714 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc75010b660 0x7fc750111f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.685 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.682+0000 7fc756f2c700 1 --2- 172.21.15.174:0/503741714 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc750109ac0 0x7fc750109ee0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.685 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.682+0000 7fc756f2c700 1 --2- 172.21.15.174:0/503741714 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc75010acc0 0x7fc75010b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.685 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.682+0000 7fc756f2c700 1 -- 172.21.15.174:0/503741714 >> 172.21.15.174:0/503741714 conn(0x7fc750078ad0 msgr2=0x7fc75007aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:28.685 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.682+0000 7fc756f2c700 1 -- 172.21.15.174:0/503741714 shutdown_connections 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.682+0000 7fc756f2c700 1 -- 172.21.15.174:0/503741714 wait complete. 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc756f2c700 1 Processor -- start 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc756f2c700 1 -- start start 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc756f2c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc750109ac0 0x7fc75019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc756f2c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc75010acc0 0x7fc7501a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc756f2c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc75010b660 0x7fc7501a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc756f2c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc750115550 con 0x7fc750109ac0 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc756f2c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501153d0 con 0x7fc75010acc0 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc756f2c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc7501156d0 con 0x7fc75010b660 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc7554c9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc75010b660 0x7fc7501a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc7554c9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc75010b660 0x7fc7501a7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.174:58382/0 (socket says 172.21.15.174:58382) 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc7554c9700 1 -- 172.21.15.174:0/3289029634 learned_addr learned my addr 172.21.15.174:0/3289029634 (peer_addr_for_me v2:172.21.15.174:0/0) 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc754cc8700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc750109ac0 0x7fc75019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.686 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc74ffff700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc75010acc0 0x7fc7501a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc7554c9700 1 -- 172.21.15.174:0/3289029634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc75010acc0 msgr2=0x7fc7501a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc7554c9700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc75010acc0 0x7fc7501a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc7554c9700 1 -- 172.21.15.174:0/3289029634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc750109ac0 msgr2=0x7fc75019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc7554c9700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc750109ac0 0x7fc75019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc7554c9700 1 -- 172.21.15.174:0/3289029634 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7501a7860 con 0x7fc75010b660 2024-04-26T18:23:28.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.683+0000 7fc7554c9700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc75010b660 0x7fc7501a7190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc74800ef30 tx=0x7fc74800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:28.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.684+0000 7fc74dffb700 1 -- 172.21.15.174:0/3289029634 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7480092e0 con 0x7fc75010b660 2024-04-26T18:23:28.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.684+0000 7fc74dffb700 1 -- 172.21.15.174:0/3289029634 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc748004bb0 con 0x7fc75010b660 2024-04-26T18:23:28.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.684+0000 7fc756f2c700 1 -- 172.21.15.174:0/3289029634 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7501a7af0 con 0x7fc75010b660 2024-04-26T18:23:28.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.684+0000 7fc74dffb700 1 -- 172.21.15.174:0/3289029634 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc748004020 con 0x7fc75010b660 2024-04-26T18:23:28.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.684+0000 7fc756f2c700 1 -- 172.21.15.174:0/3289029634 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7501a8030 con 0x7fc75010b660 2024-04-26T18:23:28.687 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.685+0000 7fc756f2c700 1 -- 172.21.15.174:0/3289029634 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc75006a110 con 0x7fc75010b660 2024-04-26T18:23:28.688 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.686+0000 7fc74dffb700 1 -- 172.21.15.174:0/3289029634 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc74801b030 con 0x7fc75010b660 2024-04-26T18:23:28.688 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.687+0000 7fc74dffb700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc73c0a3c40 0x7fc73c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:28.688 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.687+0000 7fc74dffb700 1 -- 172.21.15.174:0/3289029634 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fc7480cf9c0 con 0x7fc75010b660 2024-04-26T18:23:28.688 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.687+0000 7fc754cc8700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc73c0a3c40 0x7fc73c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:28.688 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.687+0000 7fc754cc8700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc73c0a3c40 0x7fc73c0a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc7501a10f0 tx=0x7fc74001a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:28.691 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.691+0000 7fc74dffb700 1 -- 172.21.15.174:0/3289029634 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc7480d1e20 con 0x7fc75010b660 2024-04-26T18:23:28.898 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.898+0000 7fc756f2c700 1 -- 172.21.15.174:0/3289029634 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}) -- 0x7fc750063b20 con 0x7fc73c0a3c40 2024-04-26T18:23:28.906 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.906+0000 7fc74dffb700 1 -- 172.21.15.174:0/3289029634 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fc750063b20 con 0x7fc73c0a3c40 2024-04-26T18:23:28.906 INFO:teuthology.orchestra.run.smithi174.stdout:Scheduled mds.all update... 2024-04-26T18:23:28.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.908+0000 7fc756f2c700 1 -- 172.21.15.174:0/3289029634 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc73c0a3c40 msgr2=0x7fc73c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.908+0000 7fc756f2c700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc73c0a3c40 0x7fc73c0a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc7501a10f0 tx=0x7fc74001a040 comp rx=0 tx=0).stop 2024-04-26T18:23:28.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.908+0000 7fc756f2c700 1 -- 172.21.15.174:0/3289029634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc75010b660 msgr2=0x7fc7501a7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:28.908 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.908+0000 7fc756f2c700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc75010b660 0x7fc7501a7190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc74800ef30 tx=0x7fc74800c3a0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.908+0000 7fc756f2c700 1 -- 172.21.15.174:0/3289029634 shutdown_connections 2024-04-26T18:23:28.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.908+0000 7fc756f2c700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc73c0a3c40 0x7fc73c0a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.908+0000 7fc756f2c700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc750109ac0 0x7fc75019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.908+0000 7fc756f2c700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc75010b660 0x7fc7501a7190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.908+0000 7fc756f2c700 1 --2- 172.21.15.174:0/3289029634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc75010acc0 0x7fc7501a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:28.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.908+0000 7fc756f2c700 1 -- 172.21.15.174:0/3289029634 >> 172.21.15.174:0/3289029634 conn(0x7fc750078ad0 msgr2=0x7fc75010e060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:28.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.908+0000 7fc756f2c700 1 -- 172.21.15.174:0/3289029634 shutdown_connections 2024-04-26T18:23:28.909 INFO:teuthology.orchestra.run.smithi174.stderr:2024-04-26T18:23:28.908+0000 7fc756f2c700 1 -- 172.21.15.174:0/3289029634 wait complete. 2024-04-26T18:23:29.532 DEBUG:teuthology.orchestra.run.smithi084:mds.a> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.a.service 2024-04-26T18:23:29.535 DEBUG:teuthology.orchestra.run.smithi084:mds.d> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.d.service 2024-04-26T18:23:29.578 DEBUG:teuthology.orchestra.run.smithi084:mds.g> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.g.service 2024-04-26T18:23:29.621 DEBUG:teuthology.orchestra.run.smithi084:mds.j> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.j.service 2024-04-26T18:23:29.664 DEBUG:teuthology.orchestra.run.smithi103:mds.b> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.b.service 2024-04-26T18:23:29.666 DEBUG:teuthology.orchestra.run.smithi103:mds.e> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.e.service 2024-04-26T18:23:29.709 DEBUG:teuthology.orchestra.run.smithi103:mds.h> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.h.service 2024-04-26T18:23:29.752 DEBUG:teuthology.orchestra.run.smithi103:mds.k> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.k.service 2024-04-26T18:23:29.795 DEBUG:teuthology.orchestra.run.smithi174:mds.c> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.c.service 2024-04-26T18:23:29.798 DEBUG:teuthology.orchestra.run.smithi174:mds.f> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.f.service 2024-04-26T18:23:29.841 DEBUG:teuthology.orchestra.run.smithi174:mds.i> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.i.service 2024-04-26T18:23:29.885 DEBUG:teuthology.orchestra.run.smithi174:mds.l> sudo journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.l.service 2024-04-26T18:23:29.929 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-26T18:23:29.930 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:23:30.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:29 smithi103 ceph-mon[32727]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:30.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:29 smithi103 ceph-mon[32727]: Saving service mds.all spec with placement smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-04-26T18:23:30.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:30.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:30.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:29 smithi103 ceph-mon[32727]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:30.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:30.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:30.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:30.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:30.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:30.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:29 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:30.159 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:29 smithi174 ceph-mon[32705]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:30.159 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:29 smithi174 ceph-mon[32705]: Saving service mds.all spec with placement smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-04-26T18:23:30.159 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:30.159 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:30.159 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:29 smithi174 ceph-mon[32705]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:30.159 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:30.159 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:30.159 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:30.159 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:30.159 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:30.159 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:29 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:30.168 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:30.290 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:29 smithi084 ceph-mon[29141]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:23:30.291 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:29 smithi084 ceph-mon[29141]: Saving service mds.all spec with placement smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-04-26T18:23:30.291 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:30.291 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:23:30.291 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:29 smithi084 ceph-mon[29141]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:30.291 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:30.291 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:23:30.291 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:30.291 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:30.291 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:30.291 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:29 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:31.175 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:30 smithi174 ceph-mon[32705]: Deploying daemon mds.l on smithi174 2024-04-26T18:23:31.323 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:30 smithi084 ceph-mon[29141]: Deploying daemon mds.l on smithi174 2024-04-26T18:23:31.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:30 smithi103 ceph-mon[32727]: Deploying daemon mds.l on smithi174 2024-04-26T18:23:31.839 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:23:31 smithi174 systemd[1]: Starting Ceph mds.l for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:23:31.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.942+0000 7fb736996700 1 -- 172.21.15.84:0/1546393882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb730075650 msgr2=0x7fb730075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:31.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.942+0000 7fb736996700 1 --2- 172.21.15.84:0/1546393882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb730075650 0x7fb730075a30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb720009990 tx=0x7fb720009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:31.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.942+0000 7fb736996700 1 -- 172.21.15.84:0/1546393882 shutdown_connections 2024-04-26T18:23:31.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.942+0000 7fb736996700 1 --2- 172.21.15.84:0/1546393882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb730075650 0x7fb730075a30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:31.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.942+0000 7fb736996700 1 --2- 172.21.15.84:0/1546393882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb7301145f0 0x7fb7301169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:31.944 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.942+0000 7fb736996700 1 --2- 172.21.15.84:0/1546393882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb730075f70 0x7fb7301140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.942+0000 7fb736996700 1 -- 172.21.15.84:0/1546393882 >> 172.21.15.84:0/1546393882 conn(0x7fb73007a580 msgr2=0x7fb73007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.942+0000 7fb736996700 1 -- 172.21.15.84:0/1546393882 shutdown_connections 2024-04-26T18:23:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.942+0000 7fb736996700 1 -- 172.21.15.84:0/1546393882 wait complete. 2024-04-26T18:23:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.942+0000 7fb736996700 1 Processor -- start 2024-04-26T18:23:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.942+0000 7fb736996700 1 -- start start 2024-04-26T18:23:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb736996700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb730075650 0x7fb7301a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb736996700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb730075f70 0x7fb7301a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb736996700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7301145f0 0x7fb7301a49c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb736996700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb730119c00 con 0x7fb7301145f0 2024-04-26T18:23:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb736996700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb730119a80 con 0x7fb730075f70 2024-04-26T18:23:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb736996700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb730119d80 con 0x7fb730075650 2024-04-26T18:23:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb734f33700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7301145f0 0x7fb7301a49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:31.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb72ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb730075650 0x7fb7301a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb72ffff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb730075650 0x7fb7301a00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.84:49220/0 (socket says 172.21.15.84:49220) 2024-04-26T18:23:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb72ffff700 1 -- 172.21.15.84:0/343169912 learned_addr learned my addr 172.21.15.84:0/343169912 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:23:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb72f7fe700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb730075f70 0x7fb7301a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb734f33700 1 -- 172.21.15.84:0/343169912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb730075650 msgr2=0x7fb7301a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb734f33700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb730075650 0x7fb7301a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb734f33700 1 -- 172.21.15.84:0/343169912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb730075f70 msgr2=0x7fb7301a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb734f33700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb730075f70 0x7fb7301a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb734f33700 1 -- 172.21.15.84:0/343169912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7301a5100 con 0x7fb7301145f0 2024-04-26T18:23:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb72f7fe700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb730075f70 0x7fb7301a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:23:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb72ffff700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb730075650 0x7fb7301a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:23:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.943+0000 7fb734f33700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7301145f0 0x7fb7301a49c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb728007ee0 tx=0x7fb72800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.944+0000 7fb72d7fa700 1 -- 172.21.15.84:0/343169912 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb72800ddd0 con 0x7fb7301145f0 2024-04-26T18:23:31.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.944+0000 7fb72d7fa700 1 -- 172.21.15.84:0/343169912 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb728009e20 con 0x7fb7301145f0 2024-04-26T18:23:31.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.944+0000 7fb736996700 1 -- 172.21.15.84:0/343169912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7301a53f0 con 0x7fb7301145f0 2024-04-26T18:23:31.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.944+0000 7fb72d7fa700 1 -- 172.21.15.84:0/343169912 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb728015400 con 0x7fb7301145f0 2024-04-26T18:23:31.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.944+0000 7fb736996700 1 -- 172.21.15.84:0/343169912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7301acc70 con 0x7fb7301145f0 2024-04-26T18:23:31.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.945+0000 7fb736996700 1 -- 172.21.15.84:0/343169912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb73006a110 con 0x7fb7301145f0 2024-04-26T18:23:31.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.946+0000 7fb72d7fa700 1 -- 172.21.15.84:0/343169912 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb728005020 con 0x7fb7301145f0 2024-04-26T18:23:31.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.946+0000 7fb72d7fa700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7180a3cc0 0x7fb7180a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:31.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.946+0000 7fb72ffff700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7180a3cc0 0x7fb7180a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:31.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.947+0000 7fb72d7fa700 1 -- 172.21.15.84:0/343169912 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fb7280d00b0 con 0x7fb7301145f0 2024-04-26T18:23:31.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.947+0000 7fb72ffff700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7180a3cc0 0x7fb7180a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb720003800 tx=0x7fb720005c00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:31.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:31.950+0000 7fb72d7fa700 1 -- 172.21.15.84:0/343169912 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb728010070 con 0x7fb7301145f0 2024-04-26T18:23:32.114 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:31 smithi174 ceph-mon[32705]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:32.115 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:23:31 smithi174 podman[62521]: 2024-04-26 18:23:31.742514931 +0000 UTC m=+0.017671494 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:23:32.115 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:23:31 smithi174 podman[62521]: 2024-04-26 18:23:31.862118402 +0000 UTC m=+0.137274964 container create e004f7cad9de9d5f9353b50f5c20d286b70783aec9123ba63d1cd0c7d622b743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-26T18:23:32.115 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:23:32 smithi174 podman[62521]: 2024-04-26 18:23:32.115606315 +0000 UTC m=+0.390762888 container init e004f7cad9de9d5f9353b50f5c20d286b70783aec9123ba63d1cd0c7d622b743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T18:23:32.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.204+0000 7fb736996700 1 -- 172.21.15.84:0/343169912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7301acf60 con 0x7fb7301145f0 2024-04-26T18:23:32.205 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:31 smithi084 ceph-mon[29141]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:32.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.205+0000 7fb72d7fa700 1 -- 172.21.15.84:0/343169912 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1393 (secure 0 0 0) 0x7fb72809cf30 con 0x7fb7301145f0 2024-04-26T18:23:32.205 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:23:32.206 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-26T18:23:32.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.207+0000 7fb736996700 1 -- 172.21.15.84:0/343169912 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7180a3cc0 msgr2=0x7fb7180a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:32.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.207+0000 7fb736996700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7180a3cc0 0x7fb7180a6180 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb720003800 tx=0x7fb720005c00 comp rx=0 tx=0).stop 2024-04-26T18:23:32.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.207+0000 7fb736996700 1 -- 172.21.15.84:0/343169912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7301145f0 msgr2=0x7fb7301a49c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:32.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.207+0000 7fb736996700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7301145f0 0x7fb7301a49c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb728007ee0 tx=0x7fb72800d3b0 comp rx=0 tx=0).stop 2024-04-26T18:23:32.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.207+0000 7fb736996700 1 -- 172.21.15.84:0/343169912 shutdown_connections 2024-04-26T18:23:32.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.207+0000 7fb736996700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7180a3cc0 0x7fb7180a6180 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:32.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.207+0000 7fb736996700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb7301145f0 0x7fb7301a49c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:32.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.207+0000 7fb736996700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb730075650 0x7fb7301a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:32.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.207+0000 7fb736996700 1 --2- 172.21.15.84:0/343169912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb730075f70 0x7fb7301a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:32.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.207+0000 7fb736996700 1 -- 172.21.15.84:0/343169912 >> 172.21.15.84:0/343169912 conn(0x7fb73007a580 msgr2=0x7fb730113420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:32.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.208+0000 7fb736996700 1 -- 172.21.15.84:0/343169912 shutdown_connections 2024-04-26T18:23:32.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:32.208+0000 7fb736996700 1 -- 172.21.15.84:0/343169912 wait complete. 2024-04-26T18:23:32.209 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 2 2024-04-26T18:23:32.376 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:31 smithi103 ceph-mon[32727]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:32.439 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:23:32 smithi174 podman[62521]: 2024-04-26 18:23:32.118442328 +0000 UTC m=+0.393598894 container start e004f7cad9de9d5f9353b50f5c20d286b70783aec9123ba63d1cd0c7d622b743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T18:23:32.439 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:23:32 smithi174 ceph-mds[62541]: -- 172.21.15.174:0/2117438040 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5563743a1a40 con 0x556374375000 2024-04-26T18:23:32.439 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:23:32 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-l[62537]: starting mds.l at 2024-04-26T18:23:32.439 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:23:32 smithi174 bash[62521]: e004f7cad9de9d5f9353b50f5c20d286b70783aec9123ba63d1cd0c7d622b743 2024-04-26T18:23:32.439 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:23:32 smithi174 systemd[1]: Started Ceph mds.l for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:23:33.239 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:33 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.174:6832/280377862,v1:172.21.15.174:6833/280377862] up:boot 2024-04-26T18:23:33.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:33 smithi103 ceph-mon[32727]: fsmap 1 up:standby 2024-04-26T18:23:33.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:23:33.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:33 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/343169912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:33.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:33.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:33.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:33.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:33.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:33.240 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:33 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:33 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.174:6832/280377862,v1:172.21.15.174:6833/280377862] up:boot 2024-04-26T18:23:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:33 smithi084 ceph-mon[29141]: fsmap 1 up:standby 2024-04-26T18:23:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:23:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:33 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/343169912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:33.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:33 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:33.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:33 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.174:6832/280377862,v1:172.21.15.174:6833/280377862] up:boot 2024-04-26T18:23:33.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:33 smithi174 ceph-mon[32705]: fsmap 1 up:standby 2024-04-26T18:23:33.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:23:33.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:33 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/343169912' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:33.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:33.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:33.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:33.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:33.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:33.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:33 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:34.322 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:34 smithi103 ceph-mon[32727]: Deploying daemon mds.k on smithi103 2024-04-26T18:23:34.322 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:34 smithi103 ceph-mon[32727]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:34.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:34 smithi174 ceph-mon[32705]: Deploying daemon mds.k on smithi103 2024-04-26T18:23:34.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:34 smithi174 ceph-mon[32705]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:34.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:34 smithi084 ceph-mon[29141]: Deploying daemon mds.k on smithi103 2024-04-26T18:23:34.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:34 smithi084 ceph-mon[29141]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:34.627 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:23:34 smithi103 systemd[1]: Starting Ceph mds.k for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:23:34.944 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:23:35.076 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:23:34 smithi103 podman[61988]: 2024-04-26 18:23:34.62820578 +0000 UTC m=+0.018674493 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:23:35.076 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:23:34 smithi103 podman[61988]: 2024-04-26 18:23:34.731680922 +0000 UTC m=+0.122149620 container create 4956ca44d39c545c1c1b0f98bc070f025a279995979044e537c849b641055143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T18:23:35.194 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:35.377 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:23:35 smithi103 podman[61988]: 2024-04-26 18:23:35.07692925 +0000 UTC m=+0.467397948 container init 4956ca44d39c545c1c1b0f98bc070f025a279995979044e537c849b641055143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-k, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:23:35.377 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:23:35 smithi103 podman[61988]: 2024-04-26 18:23:35.079735545 +0000 UTC m=+0.470204246 container start 4956ca44d39c545c1c1b0f98bc070f025a279995979044e537c849b641055143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T18:23:35.377 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:23:35 smithi103 ceph-mds[62007]: -- 172.21.15.103:0/116044178 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ea216cda40 con 0x55ea216a1000 2024-04-26T18:23:35.377 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:23:35 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-k[62003]: starting mds.k at 2024-04-26T18:23:35.377 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:23:35 smithi103 bash[61988]: 4956ca44d39c545c1c1b0f98bc070f025a279995979044e537c849b641055143 2024-04-26T18:23:35.377 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:23:35 smithi103 systemd[1]: Started Ceph mds.k for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:23:36.420 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:36 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.103:6832/3711970972,v1:172.21.15.103:6833/3711970972] up:boot 2024-04-26T18:23:36.420 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:36 smithi084 ceph-mon[29141]: fsmap 2 up:standby 2024-04-26T18:23:36.420 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:36 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:23:36.420 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:36 smithi084 ceph-mon[29141]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:36.421 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:36 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:36.421 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:36 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:36.421 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:36 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:36.421 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:36 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:36.421 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:36 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:36.421 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:36 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:36.421 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:36 smithi084 ceph-mon[29141]: Deploying daemon mds.j on smithi084 2024-04-26T18:23:36.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:36 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.103:6832/3711970972,v1:172.21.15.103:6833/3711970972] up:boot 2024-04-26T18:23:36.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:36 smithi174 ceph-mon[32705]: fsmap 2 up:standby 2024-04-26T18:23:36.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:36 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:23:36.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:36 smithi174 ceph-mon[32705]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:36.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:36 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:36.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:36 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:36.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:36 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:36.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:36 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:36.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:36 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:36.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:36 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:36.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:36 smithi174 ceph-mon[32705]: Deploying daemon mds.j on smithi084 2024-04-26T18:23:36.499 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:36 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.103:6832/3711970972,v1:172.21.15.103:6833/3711970972] up:boot 2024-04-26T18:23:36.499 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:36 smithi103 ceph-mon[32727]: fsmap 2 up:standby 2024-04-26T18:23:36.499 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:36 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:23:36.499 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:36 smithi103 ceph-mon[32727]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:36.499 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:36 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:36.499 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:36 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:36.499 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:36 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:36.499 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:36 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:36.500 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:36 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:36.500 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:36 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:36.500 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:36 smithi103 ceph-mon[32727]: Deploying daemon mds.j on smithi084 2024-04-26T18:23:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 -- 172.21.15.84:0/740055585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c8c102460 msgr2=0x7f3c8c10e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 --2- 172.21.15.84:0/740055585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c8c102460 0x7f3c8c10e970 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3c8400b0a0 tx=0x7f3c8400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:23:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 -- 172.21.15.84:0/740055585 shutdown_connections 2024-04-26T18:23:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 --2- 172.21.15.84:0/740055585 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c8c102460 0x7f3c8c10e970 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 --2- 172.21.15.84:0/740055585 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c8c107aa0 0x7f3c8c107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 --2- 172.21.15.84:0/740055585 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8c101aa0 0x7f3c8c101f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 -- 172.21.15.84:0/740055585 >> 172.21.15.84:0/740055585 conn(0x7f3c8c078920 msgr2=0x7f3c8c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 -- 172.21.15.84:0/740055585 shutdown_connections 2024-04-26T18:23:36.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 -- 172.21.15.84:0/740055585 wait complete. 2024-04-26T18:23:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 Processor -- start 2024-04-26T18:23:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 -- start start 2024-04-26T18:23:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c8c101aa0 0x7f3c8c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c8c102460 0x7f3c8c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8c107aa0 0x7f3c8c1a5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c111ce0 con 0x7f3c8c102460 2024-04-26T18:23:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c111b60 con 0x7f3c8c101aa0 2024-04-26T18:23:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.906+0000 7f3c93105700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c8c111e60 con 0x7f3c8c107aa0 2024-04-26T18:23:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c8bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c8c102460 0x7f3c8c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c8bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c8c102460 0x7f3c8c19aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53040/0 (socket says 172.21.15.84:53040) 2024-04-26T18:23:36.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c8bfff700 1 -- 172.21.15.84:0/884932842 learned_addr learned my addr 172.21.15.84:0/884932842 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:23:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c90ea1700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c8c101aa0 0x7f3c8c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c916a2700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8c107aa0 0x7f3c8c1a5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c8bfff700 1 -- 172.21.15.84:0/884932842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8c107aa0 msgr2=0x7f3c8c1a5a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c8bfff700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8c107aa0 0x7f3c8c1a5a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c8bfff700 1 -- 172.21.15.84:0/884932842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c8c101aa0 msgr2=0x7f3c8c19a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c8bfff700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c8c101aa0 0x7f3c8c19a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c8bfff700 1 -- 172.21.15.84:0/884932842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c8c1a5f40 con 0x7f3c8c102460 2024-04-26T18:23:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c916a2700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8c107aa0 0x7f3c8c1a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c90ea1700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c8c101aa0 0x7f3c8c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:23:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c8bfff700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c8c102460 0x7f3c8c19aec0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800b530 tx=0x7f3c7800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c89ffb700 1 -- 172.21.15.84:0/884932842 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78015020 con 0x7f3c8c102460 2024-04-26T18:23:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c93105700 1 -- 172.21.15.84:0/884932842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c8c1a6170 con 0x7f3c8c102460 2024-04-26T18:23:36.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c89ffb700 1 -- 172.21.15.84:0/884932842 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c78011ce0 con 0x7f3c8c102460 2024-04-26T18:23:36.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.907+0000 7f3c89ffb700 1 -- 172.21.15.84:0/884932842 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c78010dd0 con 0x7f3c8c102460 2024-04-26T18:23:36.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.908+0000 7f3c93105700 1 -- 172.21.15.84:0/884932842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c8c1a66b0 con 0x7f3c8c102460 2024-04-26T18:23:36.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.908+0000 7f3c93105700 1 -- 172.21.15.84:0/884932842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c8c06a110 con 0x7f3c8c102460 2024-04-26T18:23:36.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.909+0000 7f3c89ffb700 1 -- 172.21.15.84:0/884932842 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3c78018020 con 0x7f3c8c102460 2024-04-26T18:23:36.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.910+0000 7f3c89ffb700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c7c0a3c40 0x7f3c7c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:36.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.910+0000 7f3c90ea1700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c7c0a3c40 0x7f3c7c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:36.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.910+0000 7f3c89ffb700 1 -- 172.21.15.84:0/884932842 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f3c780cf7f0 con 0x7f3c8c102460 2024-04-26T18:23:36.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.910+0000 7f3c90ea1700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c7c0a3c40 0x7f3c7c0a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3c80005fd0 tx=0x7f3c80005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:36.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:36.915+0000 7f3c89ffb700 1 -- 172.21.15.84:0/884932842 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3c7801f020 con 0x7f3c8c102460 2024-04-26T18:23:37.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.174+0000 7f3c93105700 1 -- 172.21.15.84:0/884932842 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3c8c1a6970 con 0x7f3c8c102460 2024-04-26T18:23:37.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.174+0000 7f3c89ffb700 1 -- 172.21.15.84:0/884932842 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f3c7809c670 con 0x7f3c8c102460 2024-04-26T18:23:37.174 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:23:37.175 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T18:23:37.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.176+0000 7f3c93105700 1 -- 172.21.15.84:0/884932842 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c7c0a3c40 msgr2=0x7f3c7c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:37.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.176+0000 7f3c93105700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c7c0a3c40 0x7f3c7c0a6100 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3c80005fd0 tx=0x7f3c80005ee0 comp rx=0 tx=0).stop 2024-04-26T18:23:37.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.176+0000 7f3c93105700 1 -- 172.21.15.84:0/884932842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c8c102460 msgr2=0x7f3c8c19aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:37.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.176+0000 7f3c93105700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c8c102460 0x7f3c8c19aec0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3c7800b530 tx=0x7f3c7800b8f0 comp rx=0 tx=0).stop 2024-04-26T18:23:37.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.176+0000 7f3c93105700 1 -- 172.21.15.84:0/884932842 shutdown_connections 2024-04-26T18:23:37.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.176+0000 7f3c93105700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c7c0a3c40 0x7f3c7c0a6100 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:37.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.176+0000 7f3c93105700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c8c107aa0 0x7f3c8c1a5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:37.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.176+0000 7f3c93105700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c8c101aa0 0x7f3c8c19a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:37.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.176+0000 7f3c93105700 1 --2- 172.21.15.84:0/884932842 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c8c102460 0x7f3c8c19aec0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:37.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.176+0000 7f3c93105700 1 -- 172.21.15.84:0/884932842 >> 172.21.15.84:0/884932842 conn(0x7f3c8c078920 msgr2=0x7f3c8c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:37.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.176+0000 7f3c93105700 1 -- 172.21.15.84:0/884932842 shutdown_connections 2024-04-26T18:23:37.177 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:37.176+0000 7f3c93105700 1 -- 172.21.15.84:0/884932842 wait complete. 2024-04-26T18:23:37.177 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 3 2024-04-26T18:23:37.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:37 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/884932842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:37.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:37 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/884932842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:37.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:37 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/884932842' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:38.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:38 smithi084 ceph-mon[29141]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:38 smithi103 ceph-mon[32727]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:38.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:38 smithi174 ceph-mon[32705]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:39.595 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:23:39 smithi084 systemd[1]: Starting Ceph mds.j for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:23:39.926 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:23:39 smithi084 podman[69616]: 2024-04-26 18:23:39.665270594 +0000 UTC m=+0.109206823 container create b83ce86a90444b2d6fbd498754d36f8333d83294b82e17add50cbef7accae60d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-j, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:23:39.926 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:23:39 smithi084 podman[69616]: 2024-04-26 18:23:39.574019254 +0000 UTC m=+0.017955496 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:23:39.926 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:23:39 smithi084 podman[69616]: 2024-04-26 18:23:39.900974946 +0000 UTC m=+0.344911177 container init b83ce86a90444b2d6fbd498754d36f8333d83294b82e17add50cbef7accae60d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:23:39.926 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:23:39 smithi084 podman[69616]: 2024-04-26 18:23:39.903505337 +0000 UTC m=+0.347441572 container start b83ce86a90444b2d6fbd498754d36f8333d83294b82e17add50cbef7accae60d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T18:23:40.087 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:23:40.235 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:23:39 smithi084 ceph-mds[69635]: -- 172.21.15.84:0/2263032210 <== mon.0 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56045544da40 con 0x560455420c00 2024-04-26T18:23:40.235 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:23:39 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-j[69631]: starting mds.j at 2024-04-26T18:23:40.235 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:23:40 smithi084 bash[69616]: b83ce86a90444b2d6fbd498754d36f8333d83294b82e17add50cbef7accae60d 2024-04-26T18:23:40.236 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:23:40 smithi084 systemd[1]: Started Ceph mds.j for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:23:40.236 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:40 smithi084 ceph-mon[29141]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:40.236 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:40 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:40.236 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:40 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:40.236 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:40 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:40.236 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:40 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:40.236 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:40 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:40.236 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:40 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:40.339 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:40 smithi103 ceph-mon[32727]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:40 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:40 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:40 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:40 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:40 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:40.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:40 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:40.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:40 smithi174 ceph-mon[32705]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:40 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:40 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:40 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:40 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:40 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:40 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:41.593 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:41 smithi084 ceph-mon[29141]: Deploying daemon mds.d on smithi084 2024-04-26T18:23:41.593 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:41 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.84:6834/1853556606,v1:172.21.15.84:6835/1853556606] up:boot 2024-04-26T18:23:41.593 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:41 smithi084 ceph-mon[29141]: fsmap 3 up:standby 2024-04-26T18:23:41.593 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:41 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:23:41.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:41 smithi103 ceph-mon[32727]: Deploying daemon mds.d on smithi084 2024-04-26T18:23:41.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:41 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.84:6834/1853556606,v1:172.21.15.84:6835/1853556606] up:boot 2024-04-26T18:23:41.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:41 smithi103 ceph-mon[32727]: fsmap 3 up:standby 2024-04-26T18:23:41.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:41 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:23:41.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:41 smithi174 ceph-mon[32705]: Deploying daemon mds.d on smithi084 2024-04-26T18:23:41.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:41 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.84:6834/1853556606,v1:172.21.15.84:6835/1853556606] up:boot 2024-04-26T18:23:41.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:41 smithi174 ceph-mon[32705]: fsmap 3 up:standby 2024-04-26T18:23:41.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:41 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:23:42.533 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:42 smithi084 ceph-mon[29141]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:42.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:42 smithi103 ceph-mon[32727]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:42.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:42 smithi174 ceph-mon[32705]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.206+0000 7faec96f9700 1 -- 172.21.15.84:0/2423256716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec4102960 msgr2=0x7faec4102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.206+0000 7faec96f9700 1 --2- 172.21.15.84:0/2423256716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec4102960 0x7faec4102d40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7faeb80099c0 tx=0x7faeb8009cd0 comp rx=0 tx=0).stop 2024-04-26T18:23:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.206+0000 7faec96f9700 1 -- 172.21.15.84:0/2423256716 shutdown_connections 2024-04-26T18:23:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.206+0000 7faec96f9700 1 --2- 172.21.15.84:0/2423256716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec4102960 0x7faec4102d40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.206+0000 7faec96f9700 1 --2- 172.21.15.84:0/2423256716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faec41076b0 0x7faec410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.206+0000 7faec96f9700 1 --2- 172.21.15.84:0/2423256716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faec4103310 0x7faec4106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.206+0000 7faec96f9700 1 -- 172.21.15.84:0/2423256716 >> 172.21.15.84:0/2423256716 conn(0x7faec40792d0 msgr2=0x7faec40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.206+0000 7faec96f9700 1 -- 172.21.15.84:0/2423256716 shutdown_connections 2024-04-26T18:23:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.206+0000 7faec96f9700 1 -- 172.21.15.84:0/2423256716 wait complete. 2024-04-26T18:23:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.207+0000 7faec96f9700 1 Processor -- start 2024-04-26T18:23:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.207+0000 7faec96f9700 1 -- start start 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.207+0000 7faec96f9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec4102960 0x7faec419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.207+0000 7faec96f9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faec4103310 0x7faec419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.207+0000 7faec96f9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faec41076b0 0x7faec4199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.207+0000 7faec96f9700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faec41125f0 con 0x7faec4102960 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.207+0000 7faec96f9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faec4112470 con 0x7faec4103310 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.207+0000 7faec96f9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faec4112770 con 0x7faec41076b0 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.207+0000 7faec2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec4102960 0x7faec419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.207+0000 7faec2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec4102960 0x7faec419f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53050/0 (socket says 172.21.15.84:53050) 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.207+0000 7faec2ffd700 1 -- 172.21.15.84:0/307621638 learned_addr learned my addr 172.21.15.84:0/307621638 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.207+0000 7faec27fc700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faec4103310 0x7faec419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.208+0000 7faec37fe700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faec41076b0 0x7faec4199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.208+0000 7faec2ffd700 1 -- 172.21.15.84:0/307621638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faec41076b0 msgr2=0x7faec4199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.208+0000 7faec2ffd700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faec41076b0 0x7faec4199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.208+0000 7faec2ffd700 1 -- 172.21.15.84:0/307621638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faec4103310 msgr2=0x7faec419fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.208+0000 7faec2ffd700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faec4103310 0x7faec419fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.208+0000 7faec2ffd700 1 -- 172.21.15.84:0/307621638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec419a1e0 con 0x7faec4102960 2024-04-26T18:23:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.208+0000 7faec37fe700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faec41076b0 0x7faec4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:23:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.208+0000 7faec2ffd700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec4102960 0x7faec419f870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faeb80047b0 tx=0x7faeb8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.208+0000 7faeabfff700 1 -- 172.21.15.84:0/307621638 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb8027070 con 0x7faec4102960 2024-04-26T18:23:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.208+0000 7faeabfff700 1 -- 172.21.15.84:0/307621638 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faeb8004160 con 0x7faec4102960 2024-04-26T18:23:43.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.208+0000 7faec96f9700 1 -- 172.21.15.84:0/307621638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec419a470 con 0x7faec4102960 2024-04-26T18:23:43.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.209+0000 7faeabfff700 1 -- 172.21.15.84:0/307621638 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb800f460 con 0x7faec4102960 2024-04-26T18:23:43.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.209+0000 7faec96f9700 1 -- 172.21.15.84:0/307621638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec41a6740 con 0x7faec4102960 2024-04-26T18:23:43.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.210+0000 7faec96f9700 1 -- 172.21.15.84:0/307621638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec4103a60 con 0x7faec4102960 2024-04-26T18:23:43.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.211+0000 7faeabfff700 1 -- 172.21.15.84:0/307621638 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7faeb802a070 con 0x7faec4102960 2024-04-26T18:23:43.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.212+0000 7faeabfff700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faeac0a3cc0 0x7faeac0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:43.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.212+0000 7faec27fc700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faeac0a3cc0 0x7faeac0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:43.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.212+0000 7faeabfff700 1 -- 172.21.15.84:0/307621638 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7faeb80d0ef0 con 0x7faec4102960 2024-04-26T18:23:43.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.212+0000 7faec27fc700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faeac0a3cc0 0x7faeac0a6180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faec41071f0 tx=0x7faeb4009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:43.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.217+0000 7faeabfff700 1 -- 172.21.15.84:0/307621638 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faeb805eff0 con 0x7faec4102960 2024-04-26T18:23:43.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.492+0000 7faec96f9700 1 -- 172.21.15.84:0/307621638 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faec419af60 con 0x7faec4102960 2024-04-26T18:23:43.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.492+0000 7faeabfff700 1 -- 172.21.15.84:0/307621638 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7faeb8013070 con 0x7faec4102960 2024-04-26T18:23:43.493 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:23:43.493 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-26T18:23:43.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.495+0000 7faec96f9700 1 -- 172.21.15.84:0/307621638 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faeac0a3cc0 msgr2=0x7faeac0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:43.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.495+0000 7faec96f9700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faeac0a3cc0 0x7faeac0a6180 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7faec41071f0 tx=0x7faeb4009190 comp rx=0 tx=0).stop 2024-04-26T18:23:43.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.495+0000 7faec96f9700 1 -- 172.21.15.84:0/307621638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec4102960 msgr2=0x7faec419f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:43.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.495+0000 7faec96f9700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec4102960 0x7faec419f870 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7faeb80047b0 tx=0x7faeb8003680 comp rx=0 tx=0).stop 2024-04-26T18:23:43.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.495+0000 7faec96f9700 1 -- 172.21.15.84:0/307621638 shutdown_connections 2024-04-26T18:23:43.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.495+0000 7faec96f9700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faeac0a3cc0 0x7faeac0a6180 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:43.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.495+0000 7faec96f9700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faec4102960 0x7faec419f870 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:43.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.495+0000 7faec96f9700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faec41076b0 0x7faec4199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:43.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.495+0000 7faec96f9700 1 --2- 172.21.15.84:0/307621638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faec4103310 0x7faec419fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:43.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.495+0000 7faec96f9700 1 -- 172.21.15.84:0/307621638 >> 172.21.15.84:0/307621638 conn(0x7faec40792d0 msgr2=0x7faec406bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:43.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.496+0000 7faec96f9700 1 -- 172.21.15.84:0/307621638 shutdown_connections 2024-04-26T18:23:43.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:43.496+0000 7faec96f9700 1 -- 172.21.15.84:0/307621638 wait complete. 2024-04-26T18:23:43.497 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 4 2024-04-26T18:23:44.436 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:44 smithi084 ceph-mon[29141]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:44.436 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:44 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/307621638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:44.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:44 smithi103 ceph-mon[32727]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:44.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:44 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/307621638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:44.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:44 smithi174 ceph-mon[32705]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:44.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:44 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/307621638' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:44.844 INFO:journalctl@ceph.mds.d.smithi084.stdout:Apr 26 18:23:44 smithi084 systemd[1]: Started Ceph mds.d for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:23:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:45 smithi084 ceph-mon[29141]: Deploying daemon mds.e on smithi103 2024-04-26T18:23:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:45 smithi084 ceph-mon[29141]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:45 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.84:6836/3254529801,v1:172.21.15.84:6837/3254529801] up:boot 2024-04-26T18:23:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:45 smithi084 ceph-mon[29141]: fsmap 4 up:standby 2024-04-26T18:23:46.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:45 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:23:46.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:46.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:46.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:46.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:46.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:46.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:46.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:45 smithi103 ceph-mon[32727]: Deploying daemon mds.e on smithi103 2024-04-26T18:23:46.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:45 smithi103 ceph-mon[32727]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:46.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:45 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.84:6836/3254529801,v1:172.21.15.84:6837/3254529801] up:boot 2024-04-26T18:23:46.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:45 smithi103 ceph-mon[32727]: fsmap 4 up:standby 2024-04-26T18:23:46.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:45 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:23:46.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:46.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:46.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:46.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:46.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:46.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:46.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:45 smithi174 ceph-mon[32705]: Deploying daemon mds.e on smithi103 2024-04-26T18:23:46.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:45 smithi174 ceph-mon[32705]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:46.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:45 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.84:6836/3254529801,v1:172.21.15.84:6837/3254529801] up:boot 2024-04-26T18:23:46.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:45 smithi174 ceph-mon[32705]: fsmap 4 up:standby 2024-04-26T18:23:46.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:45 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:23:46.190 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:23:46.443 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:47.328 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:23:47 smithi103 systemd[1]: Starting Ceph mds.e for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:23:47.591 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:23:47 smithi103 podman[62308]: 2024-04-26 18:23:47.23223236 +0000 UTC m=+0.018439180 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:23:47.592 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:23:47 smithi103 podman[62308]: 2024-04-26 18:23:47.339164369 +0000 UTC m=+0.125371168 container create 3c7b2ab98ba066d990a2bdbadbc134c3c99c06012ed494397a37547652d3cadf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-e, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T18:23:47.592 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:23:47 smithi103 podman[62308]: 2024-04-26 18:23:47.592254258 +0000 UTC m=+0.378461054 container init 3c7b2ab98ba066d990a2bdbadbc134c3c99c06012ed494397a37547652d3cadf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-26T18:23:47.877 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:23:47 smithi103 podman[62308]: 2024-04-26 18:23:47.595814826 +0000 UTC m=+0.382021635 container start 3c7b2ab98ba066d990a2bdbadbc134c3c99c06012ed494397a37547652d3cadf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T18:23:47.877 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:23:47 smithi103 ceph-mds[62328]: -- 172.21.15.103:0/3603421046 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564884781a40 con 0x564884755000 2024-04-26T18:23:47.877 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:23:47 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-e[62324]: starting mds.e at 2024-04-26T18:23:47.877 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:23:47 smithi103 bash[62308]: 3c7b2ab98ba066d990a2bdbadbc134c3c99c06012ed494397a37547652d3cadf 2024-04-26T18:23:47.877 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:23:47 smithi103 systemd[1]: Started Ceph mds.e for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:23:48.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.176+0000 7f97ef9a2700 1 -- 172.21.15.84:0/3061243755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97e8103470 msgr2=0x7f97e81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:48.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.176+0000 7f97ef9a2700 1 --2- 172.21.15.84:0/3061243755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97e8103470 0x7f97e81038f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009990 tx=0x7f97d8009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:48.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.176+0000 7f97ef9a2700 1 -- 172.21.15.84:0/3061243755 shutdown_connections 2024-04-26T18:23:48.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.176+0000 7f97ef9a2700 1 --2- 172.21.15.84:0/3061243755 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8103e30 0x7f97e8110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:48.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.176+0000 7f97ef9a2700 1 --2- 172.21.15.84:0/3061243755 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e81094c0 0x7f97e81098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:48.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.176+0000 7f97ef9a2700 1 --2- 172.21.15.84:0/3061243755 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97e8103470 0x7f97e81038f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:48.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.176+0000 7f97ef9a2700 1 -- 172.21.15.84:0/3061243755 >> 172.21.15.84:0/3061243755 conn(0x7f97e807a580 msgr2=0x7f97e807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.176+0000 7f97ef9a2700 1 -- 172.21.15.84:0/3061243755 shutdown_connections 2024-04-26T18:23:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.176+0000 7f97ef9a2700 1 -- 172.21.15.84:0/3061243755 wait complete. 2024-04-26T18:23:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ef9a2700 1 Processor -- start 2024-04-26T18:23:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ef9a2700 1 -- start start 2024-04-26T18:23:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ef9a2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e8103470 0x7f97e819f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ef9a2700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8103e30 0x7f97e819fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ef9a2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97e81094c0 0x7f97e8199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ef9a2700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f97e81135c0 con 0x7f97e81094c0 2024-04-26T18:23:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ef9a2700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8113440 con 0x7f97e8103e30 2024-04-26T18:23:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ef9a2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97e8113740 con 0x7f97e8103470 2024-04-26T18:23:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ecf3d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8103e30 0x7f97e819fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ecf3d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8103e30 0x7f97e819fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:60636/0 (socket says 172.21.15.84:60636) 2024-04-26T18:23:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ecf3d700 1 -- 172.21.15.84:0/506449342 learned_addr learned my addr 172.21.15.84:0/506449342 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:23:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97edf3f700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97e81094c0 0x7f97e8199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ed73e700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e8103470 0x7f97e819f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ecf3d700 1 -- 172.21.15.84:0/506449342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e8103470 msgr2=0x7f97e819f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ecf3d700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e8103470 0x7f97e819f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ecf3d700 1 -- 172.21.15.84:0/506449342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97e81094c0 msgr2=0x7f97e8199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ecf3d700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97e81094c0 0x7f97e8199940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97ecf3d700 1 -- 172.21.15.84:0/506449342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97e819a0a0 con 0x7f97e8103e30 2024-04-26T18:23:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.177+0000 7f97edf3f700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97e81094c0 0x7f97e8199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:23:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.178+0000 7f97ecf3d700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8103e30 0x7f97e819fcc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f97d80057c0 tx=0x7f97d80057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.178+0000 7f97de7fc700 1 -- 172.21.15.84:0/506449342 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8027070 con 0x7f97e8103e30 2024-04-26T18:23:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.178+0000 7f97ef9a2700 1 -- 172.21.15.84:0/506449342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97e819a330 con 0x7f97e8103e30 2024-04-26T18:23:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.178+0000 7f97de7fc700 1 -- 172.21.15.84:0/506449342 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97d8003f20 con 0x7f97e8103e30 2024-04-26T18:23:48.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.178+0000 7f97de7fc700 1 -- 172.21.15.84:0/506449342 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8022bc0 con 0x7f97e8103e30 2024-04-26T18:23:48.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.178+0000 7f97ef9a2700 1 -- 172.21.15.84:0/506449342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97e81a6650 con 0x7f97e8103e30 2024-04-26T18:23:48.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.179+0000 7f97ef9a2700 1 -- 172.21.15.84:0/506449342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97e806a110 con 0x7f97e8103e30 2024-04-26T18:23:48.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.180+0000 7f97de7fc700 1 -- 172.21.15.84:0/506449342 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f97d802a070 con 0x7f97e8103e30 2024-04-26T18:23:48.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.180+0000 7f97de7fc700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f97d40a3c40 0x7f97d40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:48.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.181+0000 7f97ed73e700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f97d40a3c40 0x7f97d40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:48.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.181+0000 7f97de7fc700 1 -- 172.21.15.84:0/506449342 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f97d80d0ff0 con 0x7f97e8103e30 2024-04-26T18:23:48.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.181+0000 7f97ed73e700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f97d40a3c40 0x7f97d40a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009550 tx=0x7f97e00091b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:48.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.184+0000 7f97de7fc700 1 -- 172.21.15.84:0/506449342 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97d8013070 con 0x7f97e8103e30 2024-04-26T18:23:48.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.434+0000 7f97ef9a2700 1 -- 172.21.15.84:0/506449342 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97e802e490 con 0x7f97e8103e30 2024-04-26T18:23:48.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.435+0000 7f97de7fc700 1 -- 172.21.15.84:0/506449342 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4699 (secure 0 0 0) 0x7f97d809df00 con 0x7f97e8103e30 2024-04-26T18:23:48.435 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:23:48.435 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-26T18:23:48.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.437+0000 7f97ef9a2700 1 -- 172.21.15.84:0/506449342 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f97d40a3c40 msgr2=0x7f97d40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:48.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.437+0000 7f97ef9a2700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f97d40a3c40 0x7f97d40a6100 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f97e0009550 tx=0x7f97e00091b0 comp rx=0 tx=0).stop 2024-04-26T18:23:48.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.437+0000 7f97ef9a2700 1 -- 172.21.15.84:0/506449342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8103e30 msgr2=0x7f97e819fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:48.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.437+0000 7f97ef9a2700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8103e30 0x7f97e819fcc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f97d80057c0 tx=0x7f97d80057f0 comp rx=0 tx=0).stop 2024-04-26T18:23:48.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.437+0000 7f97ef9a2700 1 -- 172.21.15.84:0/506449342 shutdown_connections 2024-04-26T18:23:48.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.437+0000 7f97ef9a2700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f97d40a3c40 0x7f97d40a6100 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:48.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.437+0000 7f97ef9a2700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f97e81094c0 0x7f97e8199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:48.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.437+0000 7f97ef9a2700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f97e8103470 0x7f97e819f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:48.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.437+0000 7f97ef9a2700 1 --2- 172.21.15.84:0/506449342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f97e8103e30 0x7f97e819fcc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:48.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.437+0000 7f97ef9a2700 1 -- 172.21.15.84:0/506449342 >> 172.21.15.84:0/506449342 conn(0x7f97e807a580 msgr2=0x7f97e8102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:48.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.437+0000 7f97ef9a2700 1 -- 172.21.15.84:0/506449342 shutdown_connections 2024-04-26T18:23:48.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:48.438+0000 7f97ef9a2700 1 -- 172.21.15.84:0/506449342 wait complete. 2024-04-26T18:23:48.439 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 6 2024-04-26T18:23:48.526 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:48 smithi174 ceph-mon[32705]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:48.526 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:48 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:48.526 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:48 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:48.526 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:48 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:48.526 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:48 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:48.526 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:48 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:48.527 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:48 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:48.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:48 smithi084 ceph-mon[29141]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:48.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:48 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:48.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:48 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:48.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:48 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:48.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:48 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:48.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:48 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:48.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:48 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:48 smithi103 ceph-mon[32727]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:48 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:48 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:48 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:48 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:48 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:48 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:49.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:49 smithi084 ceph-mon[29141]: Deploying daemon mds.f on smithi174 2024-04-26T18:23:49.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:49 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] up:boot 2024-04-26T18:23:49.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:49 smithi084 ceph-mon[29141]: fsmap 5 up:standby 2024-04-26T18:23:49.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:49 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:23:49.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:49 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/506449342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:49.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:49 smithi103 ceph-mon[32727]: Deploying daemon mds.f on smithi174 2024-04-26T18:23:49.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:49 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] up:boot 2024-04-26T18:23:49.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:49 smithi103 ceph-mon[32727]: fsmap 5 up:standby 2024-04-26T18:23:49.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:49 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:23:49.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:49 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/506449342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:49.656 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:49 smithi174 ceph-mon[32705]: Deploying daemon mds.f on smithi174 2024-04-26T18:23:49.656 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:49 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] up:boot 2024-04-26T18:23:49.656 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:49 smithi174 ceph-mon[32705]: fsmap 5 up:standby 2024-04-26T18:23:49.656 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:49 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:23:49.656 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:49 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/506449342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:49.939 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:23:49 smithi174 systemd[1]: Starting Ceph mds.f for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:23:50.309 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:50 smithi174 ceph-mon[32705]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:50.310 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:23:50 smithi174 podman[62841]: 2024-04-26 18:23:49.95668422 +0000 UTC m=+0.018771706 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:23:50.310 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:23:50 smithi174 podman[62841]: 2024-04-26 18:23:50.065709103 +0000 UTC m=+0.127796575 container create 7321510a55a7f5378f681906036bbb4f64f1a0ba0d304054e023699c86626ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-f, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:23:50.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:50 smithi084 ceph-mon[29141]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:50 smithi103 ceph-mon[32727]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:50.689 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:23:50 smithi174 podman[62841]: 2024-04-26 18:23:50.310442454 +0000 UTC m=+0.372529927 container init 7321510a55a7f5378f681906036bbb4f64f1a0ba0d304054e023699c86626ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-f, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T18:23:50.689 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:23:50 smithi174 podman[62841]: 2024-04-26 18:23:50.31378977 +0000 UTC m=+0.375877244 container start 7321510a55a7f5378f681906036bbb4f64f1a0ba0d304054e023699c86626ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-f, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T18:23:50.689 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:23:50 smithi174 ceph-mds[62860]: -- 172.21.15.174:0/343095740 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5615b1e51a40 con 0x5615b1e25000 2024-04-26T18:23:50.689 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:23:50 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-f[62856]: starting mds.f at 2024-04-26T18:23:50.689 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:23:50 smithi174 bash[62841]: 7321510a55a7f5378f681906036bbb4f64f1a0ba0d304054e023699c86626ccf 2024-04-26T18:23:50.689 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:23:50 smithi174 systemd[1]: Started Ceph mds.f for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:23:51.126 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:23:51.379 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:51 smithi174 ceph-mon[32705]: Deploying daemon mds.c on smithi174 2024-04-26T18:23:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:51 smithi174 ceph-mon[32705]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:51 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.174:6834/1310802824,v1:172.21.15.174:6835/1310802824] up:boot 2024-04-26T18:23:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:51 smithi174 ceph-mon[32705]: fsmap 6 up:standby 2024-04-26T18:23:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:51 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:23:51.750 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:51.750 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:51.750 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:51.750 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:51.750 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:51.750 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:51.750 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:51 smithi084 ceph-mon[29141]: Deploying daemon mds.c on smithi174 2024-04-26T18:23:51.750 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:51 smithi084 ceph-mon[29141]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:51.750 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:51 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.174:6834/1310802824,v1:172.21.15.174:6835/1310802824] up:boot 2024-04-26T18:23:51.751 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:51 smithi084 ceph-mon[29141]: fsmap 6 up:standby 2024-04-26T18:23:51.751 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:51 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:23:51.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:51.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:51.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:51.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:51.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:51.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:51.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:51 smithi103 ceph-mon[32727]: Deploying daemon mds.c on smithi174 2024-04-26T18:23:51.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:51 smithi103 ceph-mon[32727]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:51.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:51 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.174:6834/1310802824,v1:172.21.15.174:6835/1310802824] up:boot 2024-04-26T18:23:51.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:51 smithi103 ceph-mon[32727]: fsmap 6 up:standby 2024-04-26T18:23:51.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:51 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:23:52.689 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:23:52 smithi174 systemd[1]: Starting Ceph mds.c for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:23:53.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.060+0000 7ffa59826700 1 -- 172.21.15.84:0/3796251927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa541094a0 msgr2=0x7ffa54109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:53.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.060+0000 7ffa59826700 1 --2- 172.21.15.84:0/3796251927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa541094a0 0x7ffa54109880 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffa440099c0 tx=0x7ffa44009cd0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.060+0000 7ffa59826700 1 -- 172.21.15.84:0/3796251927 shutdown_connections 2024-04-26T18:23:53.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.060+0000 7ffa59826700 1 --2- 172.21.15.84:0/3796251927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa541094a0 0x7ffa54109880 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.060+0000 7ffa59826700 1 --2- 172.21.15.84:0/3796251927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5406c4b0 0x7ffa541102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.060+0000 7ffa59826700 1 --2- 172.21.15.84:0/3796251927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa5406baf0 0x7ffa5406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.060+0000 7ffa59826700 1 -- 172.21.15.84:0/3796251927 >> 172.21.15.84:0/3796251927 conn(0x7ffa540792d0 msgr2=0x7ffa540796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:53.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.060+0000 7ffa59826700 1 -- 172.21.15.84:0/3796251927 shutdown_connections 2024-04-26T18:23:53.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.060+0000 7ffa59826700 1 -- 172.21.15.84:0/3796251927 wait complete. 2024-04-26T18:23:53.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.060+0000 7ffa59826700 1 Processor -- start 2024-04-26T18:23:53.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.060+0000 7ffa59826700 1 -- start start 2024-04-26T18:23:53.062 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa59826700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5406baf0 0x7ffa5419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:53.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa59826700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa5406c4b0 0x7ffa5419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:53.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa59826700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa541094a0 0x7ffa54199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:53.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa59826700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffa54113650 con 0x7ffa5406c4b0 2024-04-26T18:23:53.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa59826700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541134d0 con 0x7ffa5406baf0 2024-04-26T18:23:53.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa59826700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa541137d0 con 0x7ffa541094a0 2024-04-26T18:23:53.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa527fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa5406c4b0 0x7ffa5419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:53.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa527fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa5406c4b0 0x7ffa5419fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48208/0 (socket says 172.21.15.84:48208) 2024-04-26T18:23:53.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa527fc700 1 -- 172.21.15.84:0/2983946102 learned_addr learned my addr 172.21.15.84:0/2983946102 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:23:53.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa537fe700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa541094a0 0x7ffa54199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:53.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa52ffd700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5406baf0 0x7ffa5419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:53.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa527fc700 1 -- 172.21.15.84:0/2983946102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa541094a0 msgr2=0x7ffa54199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:53.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa527fc700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa541094a0 0x7ffa54199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa527fc700 1 -- 172.21.15.84:0/2983946102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5406baf0 msgr2=0x7ffa5419f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:53.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa527fc700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5406baf0 0x7ffa5419f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa527fc700 1 -- 172.21.15.84:0/2983946102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa5419a090 con 0x7ffa5406c4b0 2024-04-26T18:23:53.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa527fc700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa5406c4b0 0x7ffa5419fcb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800d950 tx=0x7ffa4800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:53.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa58824700 1 -- 172.21.15.84:0/2983946102 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48016070 con 0x7ffa5406c4b0 2024-04-26T18:23:53.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa58824700 1 -- 172.21.15.84:0/2983946102 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa4800fd40 con 0x7ffa5406c4b0 2024-04-26T18:23:53.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa59826700 1 -- 172.21.15.84:0/2983946102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa5419a380 con 0x7ffa5406c4b0 2024-04-26T18:23:53.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.061+0000 7ffa58824700 1 -- 172.21.15.84:0/2983946102 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4800e3b0 con 0x7ffa5406c4b0 2024-04-26T18:23:53.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.062+0000 7ffa59826700 1 -- 172.21.15.84:0/2983946102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa541a6640 con 0x7ffa5406c4b0 2024-04-26T18:23:53.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.062+0000 7ffa59826700 1 -- 172.21.15.84:0/2983946102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa54104b70 con 0x7ffa5406c4b0 2024-04-26T18:23:53.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.064+0000 7ffa58824700 1 -- 172.21.15.84:0/2983946102 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ffa48019070 con 0x7ffa5406c4b0 2024-04-26T18:23:53.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.064+0000 7ffa58824700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ffa3c0a3c40 0x7ffa3c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:53.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.064+0000 7ffa52ffd700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ffa3c0a3c40 0x7ffa3c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:53.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.064+0000 7ffa58824700 1 -- 172.21.15.84:0/2983946102 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7ffa480cf990 con 0x7ffa5406c4b0 2024-04-26T18:23:53.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.065+0000 7ffa52ffd700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ffa3c0a3c40 0x7ffa3c0a6100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffa5419aa70 tx=0x7ffa44005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:53.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.068+0000 7ffa58824700 1 -- 172.21.15.84:0/2983946102 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa4809c810 con 0x7ffa5406c4b0 2024-04-26T18:23:53.105 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:23:52 smithi174 podman[63154]: 2024-04-26 18:23:52.720385698 +0000 UTC m=+0.018315494 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:23:53.105 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:23:52 smithi174 podman[63154]: 2024-04-26 18:23:52.845409999 +0000 UTC m=+0.143339776 container create 8cc2cd012148bb407de99061ab9e65f4c5dbab60803d776a5331f38123f5ba10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:23:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.331+0000 7ffa59826700 1 -- 172.21.15.84:0/2983946102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa5402e490 con 0x7ffa5406c4b0 2024-04-26T18:23:53.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.332+0000 7ffa58824700 1 -- 172.21.15.84:0/2983946102 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6355 (secure 0 0 0) 0x7ffa4809c110 con 0x7ffa5406c4b0 2024-04-26T18:23:53.332 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:23:53.332 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24406,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-26T18:23:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.334+0000 7ffa59826700 1 -- 172.21.15.84:0/2983946102 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ffa3c0a3c40 msgr2=0x7ffa3c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.334+0000 7ffa59826700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ffa3c0a3c40 0x7ffa3c0a6100 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffa5419aa70 tx=0x7ffa44005c40 comp rx=0 tx=0).stop 2024-04-26T18:23:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.334+0000 7ffa59826700 1 -- 172.21.15.84:0/2983946102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa5406c4b0 msgr2=0x7ffa5419fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.334+0000 7ffa59826700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa5406c4b0 0x7ffa5419fcb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800d950 tx=0x7ffa4800dc60 comp rx=0 tx=0).stop 2024-04-26T18:23:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.334+0000 7ffa59826700 1 -- 172.21.15.84:0/2983946102 shutdown_connections 2024-04-26T18:23:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.334+0000 7ffa59826700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ffa3c0a3c40 0x7ffa3c0a6100 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.334+0000 7ffa59826700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa541094a0 0x7ffa54199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.334+0000 7ffa59826700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa5406baf0 0x7ffa5419f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.334+0000 7ffa59826700 1 --2- 172.21.15.84:0/2983946102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa5406c4b0 0x7ffa5419fcb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.334+0000 7ffa59826700 1 -- 172.21.15.84:0/2983946102 >> 172.21.15.84:0/2983946102 conn(0x7ffa540792d0 msgr2=0x7ffa54101530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:53.335 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.334+0000 7ffa59826700 1 -- 172.21.15.84:0/2983946102 shutdown_connections 2024-04-26T18:23:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:53.334+0000 7ffa59826700 1 -- 172.21.15.84:0/2983946102 wait complete. 2024-04-26T18:23:53.336 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 8 2024-04-26T18:23:53.439 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:23:53 smithi174 podman[63154]: 2024-04-26 18:23:53.106693178 +0000 UTC m=+0.404622959 container init 8cc2cd012148bb407de99061ab9e65f4c5dbab60803d776a5331f38123f5ba10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0) 2024-04-26T18:23:53.439 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:23:53 smithi174 podman[63154]: 2024-04-26 18:23:53.109333132 +0000 UTC m=+0.407262912 container start 8cc2cd012148bb407de99061ab9e65f4c5dbab60803d776a5331f38123f5ba10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-c, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T18:23:53.439 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:23:53 smithi174 ceph-mds[63173]: -- 172.21.15.174:0/721103827 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c2f00e7a40 con 0x55c2f00bac00 2024-04-26T18:23:53.439 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:23:53 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-c[63169]: starting mds.c at 2024-04-26T18:23:53.439 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:23:53 smithi174 bash[63154]: 8cc2cd012148bb407de99061ab9e65f4c5dbab60803d776a5331f38123f5ba10 2024-04-26T18:23:53.439 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:23:53 smithi174 systemd[1]: Started Ceph mds.c for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:23:54.334 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:54 smithi084 ceph-mon[29141]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:54.334 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:54 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] up:boot 2024-04-26T18:23:54.334 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:54 smithi084 ceph-mon[29141]: fsmap 7 up:standby 2024-04-26T18:23:54.334 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:54 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:23:54.334 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:54 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:54.334 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:54 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:54.334 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:54 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:54.334 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:54 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:54.334 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:54 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:54.334 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:54 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:54.334 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:54 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2983946102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:54.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:54 smithi103 ceph-mon[32727]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:54.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:54 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] up:boot 2024-04-26T18:23:54.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:54 smithi103 ceph-mon[32727]: fsmap 7 up:standby 2024-04-26T18:23:54.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:54 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:23:54.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:54 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:54.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:54 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:54.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:54 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:54.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:54 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:54.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:54 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:54.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:54 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:54.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:54 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2983946102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:54 smithi174 ceph-mon[32705]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:54 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] up:boot 2024-04-26T18:23:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:54 smithi174 ceph-mon[32705]: fsmap 7 up:standby 2024-04-26T18:23:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:54 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:23:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:54 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:54 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:54 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:54 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:54 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:54 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:54 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2983946102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:55.580 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:55 smithi103 ceph-mon[32727]: Deploying daemon mds.h on smithi103 2024-04-26T18:23:55.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:55 smithi084 ceph-mon[29141]: Deploying daemon mds.h on smithi103 2024-04-26T18:23:55.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:55 smithi174 ceph-mon[32705]: Deploying daemon mds.h on smithi103 2024-04-26T18:23:55.877 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:23:55 smithi103 podman[62622]: 2024-04-26 18:23:55.82150769 +0000 UTC m=+0.137376928 container create 90fc70c84d828cce78dc134db9ed647d6e1aba71ff25f01282908fa3064cbefc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-h, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:23:55.977 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:23:56.198 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:23:56 smithi103 podman[62622]: 2024-04-26 18:23:56.066315296 +0000 UTC m=+0.382184535 container init 90fc70c84d828cce78dc134db9ed647d6e1aba71ff25f01282908fa3064cbefc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-h, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:23:56.198 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:23:56 smithi103 podman[62622]: 2024-04-26 18:23:56.06891869 +0000 UTC m=+0.384787929 container start 90fc70c84d828cce78dc134db9ed647d6e1aba71ff25f01282908fa3064cbefc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-h, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T18:23:56.198 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:23:56 smithi103 ceph-mds[62641]: -- 172.21.15.103:0/3271101867 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564e9db1fa40 con 0x564e9daf2800 2024-04-26T18:23:56.198 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:23:56 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-h[62637]: starting mds.h at 2024-04-26T18:23:56.198 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:23:56 smithi103 bash[62622]: 90fc70c84d828cce78dc134db9ed647d6e1aba71ff25f01282908fa3064cbefc 2024-04-26T18:23:56.198 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:23:56 smithi103 systemd[1]: Started Ceph mds.h for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:23:56.231 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:23:56.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:56 smithi084 ceph-mon[29141]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:56.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:56 smithi103 ceph-mon[32727]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:56.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:56 smithi174 ceph-mon[32705]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:57 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] up:boot 2024-04-26T18:23:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:57 smithi174 ceph-mon[32705]: fsmap 8 up:standby 2024-04-26T18:23:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:23:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:57.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:57 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:57.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:57 smithi174 ceph-mon[32705]: Deploying daemon mds.i on smithi174 2024-04-26T18:23:57.558 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:57 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] up:boot 2024-04-26T18:23:57.558 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:57 smithi084 ceph-mon[29141]: fsmap 8 up:standby 2024-04-26T18:23:57.558 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:23:57.559 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:57.559 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:57.559 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:57.559 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:57.559 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:57.559 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:57 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:57.559 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:57 smithi084 ceph-mon[29141]: Deploying daemon mds.i on smithi174 2024-04-26T18:23:57.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:57 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] up:boot 2024-04-26T18:23:57.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:57 smithi103 ceph-mon[32727]: fsmap 8 up:standby 2024-04-26T18:23:57.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:23:57.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:57.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:57.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:23:57.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:23:57.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:23:57.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:57 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:23:57.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:57 smithi103 ceph-mon[32727]: Deploying daemon mds.i on smithi174 2024-04-26T18:23:57.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.918+0000 7f12a400c700 1 -- 172.21.15.84:0/1379778203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c101ae0 msgr2=0x7f129c101f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.918+0000 7f12a400c700 1 --2- 172.21.15.84:0/1379778203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c101ae0 0x7f129c101f60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f128c009990 tx=0x7f128c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.918+0000 7f12a400c700 1 -- 172.21.15.84:0/1379778203 shutdown_connections 2024-04-26T18:23:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.918+0000 7f12a400c700 1 --2- 172.21.15.84:0/1379778203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f129c1024a0 0x7f129c10e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.918+0000 7f12a400c700 1 --2- 172.21.15.84:0/1379778203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f129c107a90 0x7f129c107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.918+0000 7f12a400c700 1 --2- 172.21.15.84:0/1379778203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c101ae0 0x7f129c101f60 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.918+0000 7f12a400c700 1 -- 172.21.15.84:0/1379778203 >> 172.21.15.84:0/1379778203 conn(0x7f129c078990 msgr2=0x7f129c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.918+0000 7f12a400c700 1 -- 172.21.15.84:0/1379778203 shutdown_connections 2024-04-26T18:23:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.918+0000 7f12a400c700 1 -- 172.21.15.84:0/1379778203 wait complete. 2024-04-26T18:23:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.919+0000 7f12a400c700 1 Processor -- start 2024-04-26T18:23:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.919+0000 7f12a400c700 1 -- start start 2024-04-26T18:23:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.919+0000 7f12a400c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f129c101ae0 0x7f129c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.919+0000 7f12a400c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c1024a0 0x7f129c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.919+0000 7f12a400c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f129c107a90 0x7f129c199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.919+0000 7f12a400c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f129c112530 con 0x7f129c1024a0 2024-04-26T18:23:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.919+0000 7f12a400c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1123b0 con 0x7f129c101ae0 2024-04-26T18:23:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.919+0000 7f12a400c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f129c1126b0 con 0x7f129c107a90 2024-04-26T18:23:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.919+0000 7f12a15a7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c1024a0 0x7f129c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.919+0000 7f12a15a7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c1024a0 0x7f129c19fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42716/0 (socket says 172.21.15.84:42716) 2024-04-26T18:23:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.919+0000 7f12a15a7700 1 -- 172.21.15.84:0/122459086 learned_addr learned my addr 172.21.15.84:0/122459086 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:23:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.919+0000 7f12a25a9700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f129c107a90 0x7f129c199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.919+0000 7f12a1da8700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f129c101ae0 0x7f129c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.920+0000 7f12a15a7700 1 -- 172.21.15.84:0/122459086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f129c107a90 msgr2=0x7f129c199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.920+0000 7f12a15a7700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f129c107a90 0x7f129c199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.920+0000 7f12a15a7700 1 -- 172.21.15.84:0/122459086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f129c101ae0 msgr2=0x7f129c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.920+0000 7f12a15a7700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f129c101ae0 0x7f129c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.920+0000 7f12a15a7700 1 -- 172.21.15.84:0/122459086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f129c19a080 con 0x7f129c1024a0 2024-04-26T18:23:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.920+0000 7f12a15a7700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c1024a0 0x7f129c19fca0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f128c000c00 tx=0x7f128c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.920+0000 7f1292ffd700 1 -- 172.21.15.84:0/122459086 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c027070 con 0x7f129c1024a0 2024-04-26T18:23:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.920+0000 7f1292ffd700 1 -- 172.21.15.84:0/122459086 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f128c003f20 con 0x7f129c1024a0 2024-04-26T18:23:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.920+0000 7f1292ffd700 1 -- 172.21.15.84:0/122459086 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128c022ca0 con 0x7f129c1024a0 2024-04-26T18:23:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.920+0000 7f12a400c700 1 -- 172.21.15.84:0/122459086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f129c19a310 con 0x7f129c1024a0 2024-04-26T18:23:57.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.920+0000 7f12a400c700 1 -- 172.21.15.84:0/122459086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f129c1a6630 con 0x7f129c1024a0 2024-04-26T18:23:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.921+0000 7f12a400c700 1 -- 172.21.15.84:0/122459086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129c1031d0 con 0x7f129c1024a0 2024-04-26T18:23:57.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.922+0000 7f1292ffd700 1 -- 172.21.15.84:0/122459086 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f128c02a070 con 0x7f129c1024a0 2024-04-26T18:23:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.923+0000 7f1292ffd700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12880a3c40 0x7f12880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:23:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.923+0000 7f12a1da8700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12880a3c40 0x7f12880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:23:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.923+0000 7f1292ffd700 1 -- 172.21.15.84:0/122459086 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f128c0d14a0 con 0x7f129c1024a0 2024-04-26T18:23:57.925 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.923+0000 7f12a1da8700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12880a3c40 0x7f12880a6100 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f129c19aa60 tx=0x7f129800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:23:57.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:57.927+0000 7f1292ffd700 1 -- 172.21.15.84:0/122459086 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f128c031070 con 0x7f129c1024a0 2024-04-26T18:23:58.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.178+0000 7f12a400c700 1 -- 172.21.15.84:0/122459086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f129c06a110 con 0x7f129c1024a0 2024-04-26T18:23:58.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.178+0000 7f1292ffd700 1 -- 172.21.15.84:0/122459086 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7183 (secure 0 0 0) 0x7f129c06a110 con 0x7f129c1024a0 2024-04-26T18:23:58.178 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:23:58.179 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24406,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24422,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-26T18:23:58.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.180+0000 7f12a400c700 1 -- 172.21.15.84:0/122459086 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12880a3c40 msgr2=0x7f12880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.180+0000 7f12a400c700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12880a3c40 0x7f12880a6100 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f129c19aa60 tx=0x7f129800b410 comp rx=0 tx=0).stop 2024-04-26T18:23:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.180+0000 7f12a400c700 1 -- 172.21.15.84:0/122459086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c1024a0 msgr2=0x7f129c19fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:23:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.180+0000 7f12a400c700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c1024a0 0x7f129c19fca0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f128c000c00 tx=0x7f128c0047b0 comp rx=0 tx=0).stop 2024-04-26T18:23:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.180+0000 7f12a400c700 1 -- 172.21.15.84:0/122459086 shutdown_connections 2024-04-26T18:23:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.180+0000 7f12a400c700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12880a3c40 0x7f12880a6100 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.180+0000 7f12a400c700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f129c107a90 0x7f129c199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.180+0000 7f12a400c700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f129c101ae0 0x7f129c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.180+0000 7f12a400c700 1 --2- 172.21.15.84:0/122459086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f129c1024a0 0x7f129c19fca0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:23:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.180+0000 7f12a400c700 1 -- 172.21.15.84:0/122459086 >> 172.21.15.84:0/122459086 conn(0x7f129c078990 msgr2=0x7f129c10f600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:23:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.181+0000 7f12a400c700 1 -- 172.21.15.84:0/122459086 shutdown_connections 2024-04-26T18:23:58.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:23:58.181+0000 7f12a400c700 1 -- 172.21.15.84:0/122459086 wait complete. 2024-04-26T18:23:58.182 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 9 2024-04-26T18:23:58.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:58 smithi174 ceph-mon[32705]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:58.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:58 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/122459086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:58.439 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:23:58 smithi174 systemd[1]: Starting Ceph mds.i for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:23:58.554 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:58 smithi084 ceph-mon[29141]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:58.554 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:58 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/122459086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:58 smithi103 ceph-mon[32727]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:23:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:58 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/122459086' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:23:58.836 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:23:58 smithi174 podman[63460]: 2024-04-26 18:23:58.481674873 +0000 UTC m=+0.017977551 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:23:58.836 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:23:58 smithi174 podman[63460]: 2024-04-26 18:23:58.592535593 +0000 UTC m=+0.128838255 container create d8edfcd179c3056e081287296792787080c7e3796bff22d99b284be159eb0ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-i, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T18:23:58.836 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:23:58 smithi174 podman[63460]: 2024-04-26 18:23:58.836620106 +0000 UTC m=+0.372922768 container init d8edfcd179c3056e081287296792787080c7e3796bff22d99b284be159eb0ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T18:23:59.189 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:23:58 smithi174 podman[63460]: 2024-04-26 18:23:58.839297298 +0000 UTC m=+0.375599961 container start d8edfcd179c3056e081287296792787080c7e3796bff22d99b284be159eb0ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-i, org.label-schema.build-date=20240424, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:23:59.189 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:23:58 smithi174 ceph-mds[63479]: -- 172.21.15.174:0/1120650715 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e9c918da40 con 0x55e9c9160c00 2024-04-26T18:23:59.189 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:23:58 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-i[63475]: starting mds.i at 2024-04-26T18:23:59.189 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:23:58 smithi174 bash[63460]: d8edfcd179c3056e081287296792787080c7e3796bff22d99b284be159eb0ea9 2024-04-26T18:23:59.189 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:23:58 smithi174 systemd[1]: Started Ceph mds.i for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:24:00.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:00.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:00.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:00.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:24:00.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:24:00.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:00.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:59 smithi084 ceph-mon[29141]: Deploying daemon mds.g on smithi084 2024-04-26T18:24:00.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:59 smithi084 ceph-mon[29141]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:00.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:59 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] up:boot 2024-04-26T18:24:00.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:59 smithi084 ceph-mon[29141]: fsmap 9 up:standby 2024-04-26T18:24:00.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:23:59 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:24:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:24:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:24:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:59 smithi103 ceph-mon[32727]: Deploying daemon mds.g on smithi084 2024-04-26T18:24:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:59 smithi103 ceph-mon[32727]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:59 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] up:boot 2024-04-26T18:24:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:59 smithi103 ceph-mon[32727]: fsmap 9 up:standby 2024-04-26T18:24:00.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:23:59 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:24:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:24:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:24:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:00.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:59 smithi174 ceph-mon[32705]: Deploying daemon mds.g on smithi084 2024-04-26T18:24:00.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:59 smithi174 ceph-mon[32705]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:00.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:59 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] up:boot 2024-04-26T18:24:00.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:59 smithi174 ceph-mon[32705]: fsmap 9 up:standby 2024-04-26T18:24:00.440 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:23:59 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:24:00.912 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:24:01.300 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:24:01 smithi084 systemd[1]: Starting Ceph mds.g for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:24:01.560 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:24:01 smithi084 podman[70896]: 2024-04-26 18:24:01.203443591 +0000 UTC m=+0.019003433 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:24:01.560 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:24:01 smithi084 podman[70896]: 2024-04-26 18:24:01.334712098 +0000 UTC m=+0.150271955 container create da32549d9cfa71f1624dc86f11ae8f8e15c83faba0e682a8d32010ef3c491c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T18:24:01.560 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:24:01 smithi084 podman[70896]: 2024-04-26 18:24:01.560535208 +0000 UTC m=+0.376095031 container init da32549d9cfa71f1624dc86f11ae8f8e15c83faba0e682a8d32010ef3c491c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:24:01.743 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:01.844 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:24:01 smithi084 podman[70896]: 2024-04-26 18:24:01.563431667 +0000 UTC m=+0.378991493 container start da32549d9cfa71f1624dc86f11ae8f8e15c83faba0e682a8d32010ef3c491c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-g, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T18:24:01.845 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:24:01 smithi084 ceph-mds[70940]: -- 172.21.15.84:0/415877622 <== mon.0 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557ec2f65a40 con 0x557ec2f39000 2024-04-26T18:24:01.845 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:24:01 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-g[70923]: starting mds.g at 2024-04-26T18:24:01.845 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:24:01 smithi084 bash[70896]: da32549d9cfa71f1624dc86f11ae8f8e15c83faba0e682a8d32010ef3c491c53 2024-04-26T18:24:01.845 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:24:01 smithi084 systemd[1]: Started Ceph mds.g for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:24:02.593 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:02 smithi084 ceph-mon[29141]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:02.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:02.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:02.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:02.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:24:02.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:24:02.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:02 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:02.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:02 smithi103 ceph-mon[32727]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:02.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:02.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:02.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:02.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:24:02.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:24:02.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:02 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:02 smithi174 ceph-mon[32705]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:24:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:24:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:02 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:03.438 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:03 smithi084 ceph-mon[29141]: Deploying daemon mds.a on smithi084 2024-04-26T18:24:03.439 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:03 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.84:6838/96876655,v1:172.21.15.84:6839/96876655] up:boot 2024-04-26T18:24:03.439 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:03 smithi084 ceph-mon[29141]: fsmap 10 up:standby 2024-04-26T18:24:03.439 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:03 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:24:03.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:03 smithi103 ceph-mon[32727]: Deploying daemon mds.a on smithi084 2024-04-26T18:24:03.699 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:03 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.84:6838/96876655,v1:172.21.15.84:6839/96876655] up:boot 2024-04-26T18:24:03.700 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:03 smithi103 ceph-mon[32727]: fsmap 10 up:standby 2024-04-26T18:24:03.700 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:03 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:24:03.701 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:03 smithi174 ceph-mon[32705]: Deploying daemon mds.a on smithi084 2024-04-26T18:24:03.701 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:03 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.84:6838/96876655,v1:172.21.15.84:6839/96876655] up:boot 2024-04-26T18:24:03.701 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:03 smithi174 ceph-mon[32705]: fsmap 10 up:standby 2024-04-26T18:24:03.701 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:03 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:24:03.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.733+0000 7f889699d700 1 -- 172.21.15.84:0/26541251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8890107590 msgr2=0x7f889010f160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.733+0000 7f889699d700 1 --2- 172.21.15.84:0/26541251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8890107590 0x7f889010f160 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f888800b0d0 tx=0x7f888800b3e0 comp rx=0 tx=0).stop 2024-04-26T18:24:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.733+0000 7f889699d700 1 -- 172.21.15.84:0/26541251 shutdown_connections 2024-04-26T18:24:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.733+0000 7f889699d700 1 --2- 172.21.15.84:0/26541251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8890107590 0x7f889010f160 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.733+0000 7f889699d700 1 --2- 172.21.15.84:0/26541251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f88901024b0 0x7f8890102890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.733+0000 7f889699d700 1 --2- 172.21.15.84:0/26541251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8890102e60 0x7f8890106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.733+0000 7f889699d700 1 -- 172.21.15.84:0/26541251 >> 172.21.15.84:0/26541251 conn(0x7f88900fdd40 msgr2=0x7f8890100160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.733+0000 7f889699d700 1 -- 172.21.15.84:0/26541251 shutdown_connections 2024-04-26T18:24:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.733+0000 7f889699d700 1 -- 172.21.15.84:0/26541251 wait complete. 2024-04-26T18:24:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.734+0000 7f889699d700 1 Processor -- start 2024-04-26T18:24:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.734+0000 7f889699d700 1 -- start start 2024-04-26T18:24:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.734+0000 7f889699d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88901024b0 0x7f889019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.734+0000 7f889699d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8890102e60 0x7f889019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:03.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.734+0000 7f889699d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8890107590 0x7f88901a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.734+0000 7f889699d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8890112520 con 0x7f8890107590 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.734+0000 7f889699d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88901123a0 con 0x7f8890102e60 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.734+0000 7f889699d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f88901126a0 con 0x7f88901024b0 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.734+0000 7f889619c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8890107590 0x7f88901a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.734+0000 7f889619c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8890107590 0x7f88901a59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42756/0 (socket says 172.21.15.84:42756) 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.734+0000 7f889619c700 1 -- 172.21.15.84:0/2055702409 learned_addr learned my addr 172.21.15.84:0/2055702409 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.734+0000 7f889599b700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88901024b0 0x7f889019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.735+0000 7f889519a700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8890102e60 0x7f889019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.735+0000 7f889619c700 1 -- 172.21.15.84:0/2055702409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88901024b0 msgr2=0x7f889019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.735+0000 7f889619c700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88901024b0 0x7f889019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.735+0000 7f889619c700 1 -- 172.21.15.84:0/2055702409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8890102e60 msgr2=0x7f889019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.735+0000 7f889619c700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8890102e60 0x7f889019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.735+0000 7f889619c700 1 -- 172.21.15.84:0/2055702409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88901a5f00 con 0x7f8890107590 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.735+0000 7f889519a700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8890102e60 0x7f889019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:03.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.735+0000 7f889619c700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8890107590 0x7f88901a59c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8888009fd0 tx=0x7f8888007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:03.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.735+0000 7f8886ffd700 1 -- 172.21.15.84:0/2055702409 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888801f070 con 0x7f8890107590 2024-04-26T18:24:03.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.735+0000 7f889699d700 1 -- 172.21.15.84:0/2055702409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88901a6130 con 0x7f8890107590 2024-04-26T18:24:03.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.735+0000 7f8886ffd700 1 -- 172.21.15.84:0/2055702409 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8888003c00 con 0x7f8890107590 2024-04-26T18:24:03.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.735+0000 7f8886ffd700 1 -- 172.21.15.84:0/2055702409 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f888801b8e0 con 0x7f8890107590 2024-04-26T18:24:03.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.735+0000 7f889699d700 1 -- 172.21.15.84:0/2055702409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88901a65e0 con 0x7f8890107590 2024-04-26T18:24:03.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.736+0000 7f889699d700 1 -- 172.21.15.84:0/2055702409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f889006a110 con 0x7f8890107590 2024-04-26T18:24:03.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.737+0000 7f8886ffd700 1 -- 172.21.15.84:0/2055702409 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8888022070 con 0x7f8890107590 2024-04-26T18:24:03.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.738+0000 7f8886ffd700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f887c0a3cc0 0x7f887c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:03.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.738+0000 7f889599b700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f887c0a3cc0 0x7f887c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:03.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.738+0000 7f8886ffd700 1 -- 172.21.15.84:0/2055702409 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f888800f020 con 0x7f8890107590 2024-04-26T18:24:03.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.738+0000 7f889599b700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f887c0a3cc0 0x7f887c0a6180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f888c006fd0 tx=0x7f888c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:03.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.742+0000 7f8886ffd700 1 -- 172.21.15.84:0/2055702409 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8888029020 con 0x7f8890107590 2024-04-26T18:24:03.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.994+0000 7f889699d700 1 -- 172.21.15.84:0/2055702409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f889002e490 con 0x7f8890107590 2024-04-26T18:24:03.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.994+0000 7f8886ffd700 1 -- 172.21.15.84:0/2055702409 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7f888809e500 con 0x7f8890107590 2024-04-26T18:24:03.995 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:24:03.995 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24406,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24422,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24430,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-26T18:24:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.997+0000 7f889699d700 1 -- 172.21.15.84:0/2055702409 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f887c0a3cc0 msgr2=0x7f887c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.997+0000 7f889699d700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f887c0a3cc0 0x7f887c0a6180 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f888c006fd0 tx=0x7f888c008040 comp rx=0 tx=0).stop 2024-04-26T18:24:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.997+0000 7f889699d700 1 -- 172.21.15.84:0/2055702409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8890107590 msgr2=0x7f88901a59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.997+0000 7f889699d700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8890107590 0x7f88901a59c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f8888009fd0 tx=0x7f8888007ad0 comp rx=0 tx=0).stop 2024-04-26T18:24:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.997+0000 7f889699d700 1 -- 172.21.15.84:0/2055702409 shutdown_connections 2024-04-26T18:24:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.997+0000 7f889699d700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f887c0a3cc0 0x7f887c0a6180 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.997+0000 7f889699d700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8890107590 0x7f88901a59c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.997+0000 7f889699d700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88901024b0 0x7f889019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.997+0000 7f889699d700 1 --2- 172.21.15.84:0/2055702409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8890102e60 0x7f889019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:03.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.997+0000 7f889699d700 1 -- 172.21.15.84:0/2055702409 >> 172.21.15.84:0/2055702409 conn(0x7f88900fdd40 msgr2=0x7f8890100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.997+0000 7f889699d700 1 -- 172.21.15.84:0/2055702409 shutdown_connections 2024-04-26T18:24:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:03.997+0000 7f889699d700 1 -- 172.21.15.84:0/2055702409 wait complete. 2024-04-26T18:24:03.998 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 11 2024-04-26T18:24:04.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:04 smithi084 ceph-mon[29141]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:04.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:04 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2055702409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:04.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:04 smithi103 ceph-mon[32727]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:04.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:04 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2055702409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:04.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:04 smithi174 ceph-mon[32705]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:04.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:04 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2055702409' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:05.094 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:24:04 smithi084 systemd[1]: Starting Ceph mds.a for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:24:05.531 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:24:05 smithi084 podman[71345]: 2024-04-26 18:24:05.338799354 +0000 UTC m=+0.017915312 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:24:05.531 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:24:05 smithi084 podman[71345]: 2024-04-26 18:24:05.438694577 +0000 UTC m=+0.117810503 container create c63b0eed8f432deedcba9d732b8030e62123b46738f6e8be1052a0a5d75a6b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T18:24:06.095 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:24:05 smithi084 podman[71345]: 2024-04-26 18:24:05.658434287 +0000 UTC m=+0.337550213 container init c63b0eed8f432deedcba9d732b8030e62123b46738f6e8be1052a0a5d75a6b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:24:06.095 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:24:05 smithi084 podman[71345]: 2024-04-26 18:24:05.661398592 +0000 UTC m=+0.340514529 container start c63b0eed8f432deedcba9d732b8030e62123b46738f6e8be1052a0a5d75a6b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-a, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T18:24:06.095 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:24:05 smithi084 ceph-mds[71364]: -- 172.21.15.84:0/3748408657 <== mon.0 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55afb8329a40 con 0x55afb82fd000 2024-04-26T18:24:06.095 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:24:05 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-a[71360]: starting mds.a at 2024-04-26T18:24:06.095 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:24:05 smithi084 bash[71345]: c63b0eed8f432deedcba9d732b8030e62123b46738f6e8be1052a0a5d75a6b9f 2024-04-26T18:24:06.095 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:24:05 smithi084 systemd[1]: Started Ceph mds.a for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:24:06.517 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:06 smithi103 ceph-mon[32727]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:06.517 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:06.517 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:06.517 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:06.517 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:24:06.517 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:24:06.517 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:06 smithi084 ceph-mon[29141]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:24:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:24:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:06 smithi174 ceph-mon[32705]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T18:24:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T18:24:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:07.239 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:24:07.494 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:07.520 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:07 smithi103 ceph-mon[32727]: Deploying daemon mds.b on smithi103 2024-04-26T18:24:07.520 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:07 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.84:6840/2504248516,v1:172.21.15.84:6841/2504248516] up:boot 2024-04-26T18:24:07.520 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:07 smithi103 ceph-mon[32727]: fsmap 11 up:standby 2024-04-26T18:24:07.520 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:24:07.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:07 smithi084 ceph-mon[29141]: Deploying daemon mds.b on smithi103 2024-04-26T18:24:07.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:07 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.84:6840/2504248516,v1:172.21.15.84:6841/2504248516] up:boot 2024-04-26T18:24:07.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:07 smithi084 ceph-mon[29141]: fsmap 11 up:standby 2024-04-26T18:24:07.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:24:07.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:07 smithi174 ceph-mon[32705]: Deploying daemon mds.b on smithi103 2024-04-26T18:24:07.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:07 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.84:6840/2504248516,v1:172.21.15.84:6841/2504248516] up:boot 2024-04-26T18:24:07.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:07 smithi174 ceph-mon[32705]: fsmap 11 up:standby 2024-04-26T18:24:07.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:24:08.258 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:24:08 smithi103 systemd[1]: Starting Ceph mds.b for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:24:08.508 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:08 smithi103 ceph-mon[32727]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:08.509 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:24:08 smithi103 podman[62933]: 2024-04-26 18:24:08.26867231 +0000 UTC m=+0.018324194 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:24:08.509 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:24:08 smithi103 podman[62933]: 2024-04-26 18:24:08.387318408 +0000 UTC m=+0.136970325 container create 9008ada52e454062cd95881beb9020cb7f4d83344d5808c58d540a7ce938c682 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T18:24:08.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:08 smithi084 ceph-mon[29141]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:08 smithi174 ceph-mon[32705]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:08.765 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:24:08 smithi103 podman[62933]: 2024-04-26 18:24:08.665615735 +0000 UTC m=+0.415267609 container init 9008ada52e454062cd95881beb9020cb7f4d83344d5808c58d540a7ce938c682 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T18:24:08.765 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:24:08 smithi103 podman[62933]: 2024-04-26 18:24:08.668286585 +0000 UTC m=+0.417938465 container start 9008ada52e454062cd95881beb9020cb7f4d83344d5808c58d540a7ce938c682 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:24:08.765 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:24:08 smithi103 ceph-mds[62952]: -- 172.21.15.103:0/2782286716 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56005b20ba40 con 0x56005b1dec00 2024-04-26T18:24:08.765 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:24:08 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b[62948]: starting mds.b at 2024-04-26T18:24:09.127 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:24:08 smithi103 bash[62933]: 9008ada52e454062cd95881beb9020cb7f4d83344d5808c58d540a7ce938c682 2024-04-26T18:24:09.127 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:24:08 smithi103 systemd[1]: Started Ceph mds.b for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:24:09.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.207+0000 7fdb85391700 1 -- 172.21.15.84:0/268016504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb80102ce0 msgr2=0x7fdb8010f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:09.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.207+0000 7fdb85391700 1 --2- 172.21.15.84:0/268016504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb80102ce0 0x7fdb8010f1d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdb7800b0a0 tx=0x7fdb7800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:24:09.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.207+0000 7fdb85391700 1 -- 172.21.15.84:0/268016504 shutdown_connections 2024-04-26T18:24:09.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.207+0000 7fdb85391700 1 --2- 172.21.15.84:0/268016504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb80102ce0 0x7fdb8010f1d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:09.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.207+0000 7fdb85391700 1 --2- 172.21.15.84:0/268016504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb80108390 0x7fdb80108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:09.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.207+0000 7fdb85391700 1 --2- 172.21.15.84:0/268016504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb80102320 0x7fdb801027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:09.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.207+0000 7fdb85391700 1 -- 172.21.15.84:0/268016504 >> 172.21.15.84:0/268016504 conn(0x7fdb800792d0 msgr2=0x7fdb800796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:09.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.207+0000 7fdb85391700 1 -- 172.21.15.84:0/268016504 shutdown_connections 2024-04-26T18:24:09.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.207+0000 7fdb85391700 1 -- 172.21.15.84:0/268016504 wait complete. 2024-04-26T18:24:09.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.208+0000 7fdb85391700 1 Processor -- start 2024-04-26T18:24:09.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.208+0000 7fdb85391700 1 -- start start 2024-04-26T18:24:09.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.208+0000 7fdb85391700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb80102320 0x7fdb8019e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:09.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.208+0000 7fdb85391700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb80102ce0 0x7fdb8019ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:09.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.208+0000 7fdb85391700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb80108390 0x7fdb80198800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:09.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.208+0000 7fdb85391700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdb801124e0 con 0x7fdb80102320 2024-04-26T18:24:09.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.208+0000 7fdb85391700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80112360 con 0x7fdb80108390 2024-04-26T18:24:09.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.208+0000 7fdb85391700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdb80112660 con 0x7fdb80102ce0 2024-04-26T18:24:09.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.208+0000 7fdb7effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb80102320 0x7fdb8019e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:09.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.208+0000 7fdb7effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb80102320 0x7fdb8019e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60920/0 (socket says 172.21.15.84:60920) 2024-04-26T18:24:09.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.208+0000 7fdb7effd700 1 -- 172.21.15.84:0/4046246534 learned_addr learned my addr 172.21.15.84:0/4046246534 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:09.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.209+0000 7fdb7f7fe700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb80108390 0x7fdb80198800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:09.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.209+0000 7fdb7e7fc700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb80102ce0 0x7fdb8019ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:09.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.209+0000 7fdb7effd700 1 -- 172.21.15.84:0/4046246534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb80102ce0 msgr2=0x7fdb8019ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:09.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.209+0000 7fdb7effd700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb80102ce0 0x7fdb8019ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:09.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.209+0000 7fdb7effd700 1 -- 172.21.15.84:0/4046246534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb80108390 msgr2=0x7fdb80198800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:09.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.209+0000 7fdb7effd700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb80108390 0x7fdb80198800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:09.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.209+0000 7fdb7effd700 1 -- 172.21.15.84:0/4046246534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb80198fe0 con 0x7fdb80102320 2024-04-26T18:24:09.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.209+0000 7fdb7effd700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb80102320 0x7fdb8019e680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000d950 tx=0x7fdb7000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:09.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.209+0000 7fdb67fff700 1 -- 172.21.15.84:0/4046246534 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb70016070 con 0x7fdb80102320 2024-04-26T18:24:09.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.209+0000 7fdb67fff700 1 -- 172.21.15.84:0/4046246534 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7000fd40 con 0x7fdb80102320 2024-04-26T18:24:09.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.209+0000 7fdb85391700 1 -- 172.21.15.84:0/4046246534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb801992d0 con 0x7fdb80102320 2024-04-26T18:24:09.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.209+0000 7fdb67fff700 1 -- 172.21.15.84:0/4046246534 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7000e3b0 con 0x7fdb80102320 2024-04-26T18:24:09.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.209+0000 7fdb85391700 1 -- 172.21.15.84:0/4046246534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8006bb90 con 0x7fdb80102320 2024-04-26T18:24:09.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.210+0000 7fdb85391700 1 -- 172.21.15.84:0/4046246534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb80103a40 con 0x7fdb80102320 2024-04-26T18:24:09.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.211+0000 7fdb67fff700 1 -- 172.21.15.84:0/4046246534 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdb70019070 con 0x7fdb80102320 2024-04-26T18:24:09.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.212+0000 7fdb67fff700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdb680a3c40 0x7fdb680a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:09.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.212+0000 7fdb7e7fc700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdb680a3c40 0x7fdb680a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:09.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.212+0000 7fdb67fff700 1 -- 172.21.15.84:0/4046246534 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fdb700cf930 con 0x7fdb80102320 2024-04-26T18:24:09.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.212+0000 7fdb7e7fc700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdb680a3c40 0x7fdb680a6100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdb800798d0 tx=0x7fdb7400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:09.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.216+0000 7fdb67fff700 1 -- 172.21.15.84:0/4046246534 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb7009c7b0 con 0x7fdb80102320 2024-04-26T18:24:09.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.475+0000 7fdb85391700 1 -- 172.21.15.84:0/4046246534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb8006c680 con 0x7fdb80102320 2024-04-26T18:24:09.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.476+0000 7fdb67fff700 1 -- 172.21.15.84:0/4046246534 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10485 (secure 0 0 0) 0x7fdb7009c0b0 con 0x7fdb80102320 2024-04-26T18:24:09.476 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:24:09.476 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14598,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14610,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24389,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24406,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24407,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24418,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24422,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24430,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24434,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-26T18:24:09.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.478+0000 7fdb85391700 1 -- 172.21.15.84:0/4046246534 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdb680a3c40 msgr2=0x7fdb680a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:09.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.478+0000 7fdb85391700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdb680a3c40 0x7fdb680a6100 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdb800798d0 tx=0x7fdb7400b3c0 comp rx=0 tx=0).stop 2024-04-26T18:24:09.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.478+0000 7fdb85391700 1 -- 172.21.15.84:0/4046246534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb80102320 msgr2=0x7fdb8019e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:09.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.478+0000 7fdb85391700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb80102320 0x7fdb8019e680 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdb7000d950 tx=0x7fdb7000dc60 comp rx=0 tx=0).stop 2024-04-26T18:24:09.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.479+0000 7fdb85391700 1 -- 172.21.15.84:0/4046246534 shutdown_connections 2024-04-26T18:24:09.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.479+0000 7fdb85391700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdb680a3c40 0x7fdb680a6100 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:09.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.479+0000 7fdb85391700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdb80102320 0x7fdb8019e680 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:09.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.479+0000 7fdb85391700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdb80108390 0x7fdb80198800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:09.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.479+0000 7fdb85391700 1 --2- 172.21.15.84:0/4046246534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdb80102ce0 0x7fdb8019ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:09.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.479+0000 7fdb85391700 1 -- 172.21.15.84:0/4046246534 >> 172.21.15.84:0/4046246534 conn(0x7fdb800792d0 msgr2=0x7fdb80100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:09.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.479+0000 7fdb85391700 1 -- 172.21.15.84:0/4046246534 shutdown_connections 2024-04-26T18:24:09.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:09.479+0000 7fdb85391700 1 -- 172.21.15.84:0/4046246534 wait complete. 2024-04-26T18:24:09.480 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 13 2024-04-26T18:24:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:09 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:09 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:09 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:09 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:09 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:09 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:09 smithi084 ceph-mon[29141]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:09 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] up:boot 2024-04-26T18:24:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:09 smithi084 ceph-mon[29141]: fsmap 12 up:standby 2024-04-26T18:24:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:09 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:24:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:09 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/4046246534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:10.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:09 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:09 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:09 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:09 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:09 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:09 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:10.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:09 smithi103 ceph-mon[32727]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:10.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:09 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] up:boot 2024-04-26T18:24:10.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:09 smithi103 ceph-mon[32727]: fsmap 12 up:standby 2024-04-26T18:24:10.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:09 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:24:10.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:09 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/4046246534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:10.150 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-04-26T18:24:10.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:09 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:09 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:09 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:09 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:09 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:10.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:09 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:10.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:09 smithi174 ceph-mon[32705]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:10.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:09 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] up:boot 2024-04-26T18:24:10.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:09 smithi174 ceph-mon[32705]: fsmap 12 up:standby 2024-04-26T18:24:10.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:09 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:24:10.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:09 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/4046246534' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:24:11.006 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:11.381 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.382 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.382 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.382 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.382 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:11 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:11 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:11 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:12.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:12 smithi174 ceph-mon[32705]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:12.447 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:12 smithi084 ceph-mon[29141]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:12.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:12 smithi103 ceph-mon[32727]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:12.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.672+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/1627283472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f78102350 msgr2=0x7f8f781027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.672+0000 7f8f7cdaf700 1 --2- 172.21.15.84:0/1627283472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f78102350 0x7f8f781027d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8f68009990 tx=0x7f8f68009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.673+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/1627283472 shutdown_connections 2024-04-26T18:24:12.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.673+0000 7f8f7cdaf700 1 --2- 172.21.15.84:0/1627283472 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f78102d10 0x7f8f7810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.673+0000 7f8f7cdaf700 1 --2- 172.21.15.84:0/1627283472 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f781083d0 0x7f8f781087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.673+0000 7f8f7cdaf700 1 --2- 172.21.15.84:0/1627283472 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f78102350 0x7f8f781027d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.673+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/1627283472 >> 172.21.15.84:0/1627283472 conn(0x7f8f78078390 msgr2=0x7f8f780787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.673+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/1627283472 shutdown_connections 2024-04-26T18:24:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.673+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/1627283472 wait complete. 2024-04-26T18:24:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.673+0000 7f8f7cdaf700 1 Processor -- start 2024-04-26T18:24:12.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.673+0000 7f8f7cdaf700 1 -- start start 2024-04-26T18:24:12.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f7cdaf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f78102350 0x7f8f78104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:12.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f7cdaf700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f78102d10 0x7f8f78104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:12.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f7cdaf700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f781083d0 0x7f8f78104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:12.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f7cdaf700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8f781124a0 con 0x7f8f781083d0 2024-04-26T18:24:12.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f7cdaf700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78112320 con 0x7f8f78102d10 2024-04-26T18:24:12.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f7cdaf700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8f78112620 con 0x7f8f78102350 2024-04-26T18:24:12.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f76d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f781083d0 0x7f8f78104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:12.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f76d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f781083d0 0x7f8f78104a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60930/0 (socket says 172.21.15.84:60930) 2024-04-26T18:24:12.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f76d9d700 1 -- 172.21.15.84:0/875329396 learned_addr learned my addr 172.21.15.84:0/875329396 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:12.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f7659c700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f78102350 0x7f8f78104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:12.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f75d9b700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f78102d10 0x7f8f78104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:12.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f76d9d700 1 -- 172.21.15.84:0/875329396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f78102350 msgr2=0x7f8f78104000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f76d9d700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f78102350 0x7f8f78104000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f76d9d700 1 -- 172.21.15.84:0/875329396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f78102d10 msgr2=0x7f8f78104540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f76d9d700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f78102d10 0x7f8f78104540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f76d9d700 1 -- 172.21.15.84:0/875329396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f781a7f60 con 0x7f8f781083d0 2024-04-26T18:24:12.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f76d9d700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f781083d0 0x7f8f78104a80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000ada0 tx=0x7f8f7000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:12.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f677fe700 1 -- 172.21.15.84:0/875329396 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70017020 con 0x7f8f781083d0 2024-04-26T18:24:12.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/875329396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f781a8250 con 0x7f8f781083d0 2024-04-26T18:24:12.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f677fe700 1 -- 172.21.15.84:0/875329396 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f7000ce30 con 0x7f8f781083d0 2024-04-26T18:24:12.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f677fe700 1 -- 172.21.15.84:0/875329396 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f70005350 con 0x7f8f781083d0 2024-04-26T18:24:12.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.674+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/875329396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f781a8740 con 0x7f8f781083d0 2024-04-26T18:24:12.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.675+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/875329396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f78103a40 con 0x7f8f781083d0 2024-04-26T18:24:12.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.677+0000 7f8f677fe700 1 -- 172.21.15.84:0/875329396 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8f7000c980 con 0x7f8f781083d0 2024-04-26T18:24:12.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.677+0000 7f8f677fe700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8f600a3c40 0x7f8f600a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:12.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.677+0000 7f8f7659c700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8f600a3c40 0x7f8f600a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:12.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.677+0000 7f8f677fe700 1 -- 172.21.15.84:0/875329396 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7f8f700cf6b0 con 0x7f8f781083d0 2024-04-26T18:24:12.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.678+0000 7f8f7659c700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8f600a3c40 0x7f8f600a6100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00abd0 tx=0x7f8f6c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:12.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.682+0000 7f8f677fe700 1 -- 172.21.15.84:0/875329396 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8f70021020 con 0x7f8f781083d0 2024-04-26T18:24:12.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.930+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/875329396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8f781a89a0 con 0x7f8f781083d0 2024-04-26T18:24:12.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.930+0000 7f8f677fe700 1 -- 172.21.15.84:0/875329396 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f8f7009c5c0 con 0x7f8f781083d0 2024-04-26T18:24:12.931 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:24:12.931 INFO:teuthology.orchestra.run.smithi084.stdout:[ 2024-04-26T18:24:12.931 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:24:12.931 INFO:teuthology.orchestra.run.smithi084.stdout: "poolnum": 1, 2024-04-26T18:24:12.931 INFO:teuthology.orchestra.run.smithi084.stdout: "poolname": ".mgr" 2024-04-26T18:24:12.931 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:24:12.931 INFO:teuthology.orchestra.run.smithi084.stdout:] 2024-04-26T18:24:12.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.933+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/875329396 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8f600a3c40 msgr2=0x7f8f600a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.933+0000 7f8f7cdaf700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8f600a3c40 0x7f8f600a6100 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8f6c00abd0 tx=0x7f8f6c00a380 comp rx=0 tx=0).stop 2024-04-26T18:24:12.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.933+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/875329396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f781083d0 msgr2=0x7f8f78104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:12.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.933+0000 7f8f7cdaf700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f781083d0 0x7f8f78104a80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8f7000ada0 tx=0x7f8f7000c330 comp rx=0 tx=0).stop 2024-04-26T18:24:12.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.933+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/875329396 shutdown_connections 2024-04-26T18:24:12.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.933+0000 7f8f7cdaf700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f8f600a3c40 0x7f8f600a6100 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.933+0000 7f8f7cdaf700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8f781083d0 0x7f8f78104a80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.933+0000 7f8f7cdaf700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8f78102350 0x7f8f78104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.933+0000 7f8f7cdaf700 1 --2- 172.21.15.84:0/875329396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8f78102d10 0x7f8f78104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:12.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.933+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/875329396 >> 172.21.15.84:0/875329396 conn(0x7f8f78078390 msgr2=0x7f8f7806be10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:12.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.933+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/875329396 shutdown_connections 2024-04-26T18:24:12.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:12.933+0000 7f8f7cdaf700 1 -- 172.21.15.84:0/875329396 wait complete. 2024-04-26T18:24:13.635 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-26T18:24:13.635 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-26T18:24:14.183 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:14.183 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:14.183 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:14.183 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:14.183 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:14.183 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:13 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/875329396' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T18:24:14.183 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:13 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:14.183 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:13 smithi174 ceph-mon[32705]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:14.215 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:14.215 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:14.215 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:14.215 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:14.215 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:14.215 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:13 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/875329396' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T18:24:14.216 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:13 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:14.216 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:13 smithi103 ceph-mon[32727]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:14.231 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:14.231 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:14.231 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:14.231 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:14.232 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:14.232 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:13 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/875329396' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-26T18:24:14.232 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:13 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:24:14.232 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:13 smithi084 ceph-mon[29141]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:14.522 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:16.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:15 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:16.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:15 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:16.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:15 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:16.032 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:15 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:16.033 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:15 smithi084 ceph-mon[29141]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:16.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:15 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:16.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:15 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:16.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:15 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:16.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:15 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:16.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:15 smithi103 ceph-mon[32727]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:16.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:15 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:16.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:15 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:16.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:15 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:16.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:15 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:16.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:15 smithi174 ceph-mon[32705]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.290+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3669843305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcba4101de0 msgr2=0x7fcba4102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.290+0000 7fcbab7bd700 1 --2- 172.21.15.84:0/3669843305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcba4101de0 0x7fcba4102260 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fcba0009990 tx=0x7fcba0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.290+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3669843305 shutdown_connections 2024-04-26T18:24:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.290+0000 7fcbab7bd700 1 --2- 172.21.15.84:0/3669843305 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcba41027a0 0x7fcba4110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.290+0000 7fcbab7bd700 1 --2- 172.21.15.84:0/3669843305 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcba4108500 0x7fcba41088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.290+0000 7fcbab7bd700 1 --2- 172.21.15.84:0/3669843305 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcba4101de0 0x7fcba4102260 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.290+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3669843305 >> 172.21.15.84:0/3669843305 conn(0x7fcba4077640 msgr2=0x7fcba4077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.291+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3669843305 shutdown_connections 2024-04-26T18:24:16.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.291+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3669843305 wait complete. 2024-04-26T18:24:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcbab7bd700 1 Processor -- start 2024-04-26T18:24:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcbab7bd700 1 -- start start 2024-04-26T18:24:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcbab7bd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcba4101de0 0x7fcba419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcbab7bd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcba41027a0 0x7fcba419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:16.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcbab7bd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcba4108500 0x7fcba4199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcbab7bd700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4113610 con 0x7fcba4108500 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcbab7bd700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4113490 con 0x7fcba41027a0 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcbab7bd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcba4113790 con 0x7fcba4101de0 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcba9559700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcba4101de0 0x7fcba419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcba9d5a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcba4108500 0x7fcba4199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcba9559700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcba4101de0 0x7fcba419f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.84:53694/0 (socket says 172.21.15.84:53694) 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcba9559700 1 -- 172.21.15.84:0/3367580896 learned_addr learned my addr 172.21.15.84:0/3367580896 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcba8d58700 1 --2- 172.21.15.84:0/3367580896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcba41027a0 0x7fcba419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcba9d5a700 1 -- 172.21.15.84:0/3367580896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcba4101de0 msgr2=0x7fcba419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcba9d5a700 1 --2- 172.21.15.84:0/3367580896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcba4101de0 0x7fcba419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcba9d5a700 1 -- 172.21.15.84:0/3367580896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcba41027a0 msgr2=0x7fcba419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcba9d5a700 1 --2- 172.21.15.84:0/3367580896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcba41027a0 0x7fcba419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.292+0000 7fcba9d5a700 1 -- 172.21.15.84:0/3367580896 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba419a070 con 0x7fcba4108500 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.293+0000 7fcba9d5a700 1 --2- 172.21.15.84:0/3367580896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcba4108500 0x7fcba4199910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c007f10 tx=0x7fcb9c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:16.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.293+0000 7fcb9a7fc700 1 -- 172.21.15.84:0/3367580896 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c004db0 con 0x7fcba4108500 2024-04-26T18:24:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.293+0000 7fcb9a7fc700 1 -- 172.21.15.84:0/3367580896 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb9c009e20 con 0x7fcba4108500 2024-04-26T18:24:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.293+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3367580896 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba419a360 con 0x7fcba4108500 2024-04-26T18:24:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.293+0000 7fcb9a7fc700 1 -- 172.21.15.84:0/3367580896 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c015400 con 0x7fcba4108500 2024-04-26T18:24:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.293+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3367580896 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba41a6620 con 0x7fcba4108500 2024-04-26T18:24:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.294+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3367580896 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb88005330 con 0x7fcba4108500 2024-04-26T18:24:16.296 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.296+0000 7fcb9a7fc700 1 -- 172.21.15.84:0/3367580896 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fcb9c00db20 con 0x7fcba4108500 2024-04-26T18:24:16.297 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.296+0000 7fcb9a7fc700 1 --2- 172.21.15.84:0/3367580896 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fcb900a3c40 0x7fcb900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:16.297 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.296+0000 7fcba9559700 1 --2- 172.21.15.84:0/3367580896 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fcb900a3c40 0x7fcb900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:16.297 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.296+0000 7fcb9a7fc700 1 -- 172.21.15.84:0/3367580896 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7127+0+0 (secure 0 0 0) 0x7fcb9c0cfc80 con 0x7fcba4108500 2024-04-26T18:24:16.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.297+0000 7fcba9559700 1 --2- 172.21.15.84:0/3367580896 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fcb900a3c40 0x7fcb900a6100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcba419aa50 tx=0x7fcb9400b450 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:16.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.300+0000 7fcb9a7fc700 1 -- 172.21.15.84:0/3367580896 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb9c010070 con 0x7fcba4108500 2024-04-26T18:24:16.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.527+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3367580896 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fcb88005620 con 0x7fcba4108500 2024-04-26T18:24:16.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.911+0000 7fcb9a7fc700 1 -- 172.21.15.84:0/3367580896 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7fcb9c09cb00 con 0x7fcba4108500 2024-04-26T18:24:16.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.913+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3367580896 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fcb900a3c40 msgr2=0x7fcb900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:16.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.913+0000 7fcbab7bd700 1 --2- 172.21.15.84:0/3367580896 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fcb900a3c40 0x7fcb900a6100 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fcba419aa50 tx=0x7fcb9400b450 comp rx=0 tx=0).stop 2024-04-26T18:24:16.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.913+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3367580896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcba4108500 msgr2=0x7fcba4199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:16.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.913+0000 7fcbab7bd700 1 --2- 172.21.15.84:0/3367580896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcba4108500 0x7fcba4199910 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c007f10 tx=0x7fcb9c00d3b0 comp rx=0 tx=0).stop 2024-04-26T18:24:16.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.914+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3367580896 shutdown_connections 2024-04-26T18:24:16.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.914+0000 7fcbab7bd700 1 --2- 172.21.15.84:0/3367580896 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fcb900a3c40 0x7fcb900a6100 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:16.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.914+0000 7fcbab7bd700 1 --2- 172.21.15.84:0/3367580896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcba4108500 0x7fcba4199910 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:16.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.914+0000 7fcbab7bd700 1 --2- 172.21.15.84:0/3367580896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcba4101de0 0x7fcba419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:16.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.914+0000 7fcbab7bd700 1 --2- 172.21.15.84:0/3367580896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcba41027a0 0x7fcba419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:16.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.914+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3367580896 >> 172.21.15.84:0/3367580896 conn(0x7fcba4077640 msgr2=0x7fcba4100a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:16.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.914+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3367580896 shutdown_connections 2024-04-26T18:24:16.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:16.914+0000 7fcbab7bd700 1 -- 172.21.15.84:0/3367580896 wait complete. 2024-04-26T18:24:16.916 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' created 2024-04-26T18:24:17.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:16 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3367580896' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T18:24:17.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:17.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:16 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:17.241 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:16 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3367580896' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T18:24:17.241 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:17.241 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:16 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:17.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:16 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3367580896' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-26T18:24:17.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:17.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:16 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:17.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-26T18:24:17.852 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:18.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:17 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3367580896' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T18:24:18.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:17 smithi174 ceph-mon[32705]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T18:24:18.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:17 smithi174 ceph-mon[32705]: pgmap v289: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:18.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:18.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:18.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:17 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:18.227 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:17 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3367580896' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T18:24:18.227 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:17 smithi084 ceph-mon[29141]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T18:24:18.227 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:17 smithi084 ceph-mon[29141]: pgmap v289: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:18.227 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:18.227 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:18.227 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:17 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:18.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:17 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3367580896' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-26T18:24:18.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:17 smithi103 ceph-mon[32727]: osdmap e83: 12 total, 12 up, 12 in 2024-04-26T18:24:18.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:17 smithi103 ceph-mon[32727]: pgmap v289: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:18.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:24:18.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:24:18.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:17 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:24:18.990 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:18 smithi084 ceph-mon[29141]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T18:24:18.990 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:18 smithi084 ceph-mon[29141]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T18:24:19.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:18 smithi174 ceph-mon[32705]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T18:24:19.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:18 smithi174 ceph-mon[32705]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T18:24:19.376 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:18 smithi103 ceph-mon[32727]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-26T18:24:19.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:18 smithi103 ceph-mon[32727]: osdmap e84: 12 total, 12 up, 12 in 2024-04-26T18:24:19.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.533+0000 7f9b387f5700 1 -- 172.21.15.84:0/1809444716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b30102320 msgr2=0x7f9b301027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:19.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.533+0000 7f9b387f5700 1 --2- 172.21.15.84:0/1809444716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b30102320 0x7f9b301027a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009990 tx=0x7f9b20009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:19.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.533+0000 7f9b387f5700 1 -- 172.21.15.84:0/1809444716 shutdown_connections 2024-04-26T18:24:19.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.533+0000 7f9b387f5700 1 --2- 172.21.15.84:0/1809444716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b30102ce0 0x7f9b3010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:19.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.533+0000 7f9b387f5700 1 --2- 172.21.15.84:0/1809444716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b30108390 0x7f9b30108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:19.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.533+0000 7f9b387f5700 1 --2- 172.21.15.84:0/1809444716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b30102320 0x7f9b301027a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:19.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.533+0000 7f9b387f5700 1 -- 172.21.15.84:0/1809444716 >> 172.21.15.84:0/1809444716 conn(0x7f9b300792d0 msgr2=0x7f9b300796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:19.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.533+0000 7f9b387f5700 1 -- 172.21.15.84:0/1809444716 shutdown_connections 2024-04-26T18:24:19.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.533+0000 7f9b387f5700 1 -- 172.21.15.84:0/1809444716 wait complete. 2024-04-26T18:24:19.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b387f5700 1 Processor -- start 2024-04-26T18:24:19.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b387f5700 1 -- start start 2024-04-26T18:24:19.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b387f5700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b30102320 0x7f9b3019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:19.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b387f5700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b30102ce0 0x7f9b3019b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:19.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b387f5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b30108390 0x7f9b301a5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:19.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b387f5700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301125b0 con 0x7f9b30102320 2024-04-26T18:24:19.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b387f5700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b30112430 con 0x7f9b30102ce0 2024-04-26T18:24:19.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b387f5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b30112730 con 0x7f9b30108390 2024-04-26T18:24:19.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b36591700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b30102320 0x7f9b3019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:19.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b36591700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b30102320 0x7f9b3019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52126/0 (socket says 172.21.15.84:52126) 2024-04-26T18:24:19.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b36591700 1 -- 172.21.15.84:0/774091469 learned_addr learned my addr 172.21.15.84:0/774091469 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:19.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b35d90700 1 --2- 172.21.15.84:0/774091469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b30102ce0 0x7f9b3019b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:19.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b36591700 1 -- 172.21.15.84:0/774091469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b30108390 msgr2=0x7f9b301a5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:19.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b36591700 1 --2- 172.21.15.84:0/774091469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b30108390 0x7f9b301a5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:19.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b36591700 1 -- 172.21.15.84:0/774091469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b30102ce0 msgr2=0x7f9b3019b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:19.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b36591700 1 --2- 172.21.15.84:0/774091469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b30102ce0 0x7f9b3019b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:19.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.534+0000 7f9b36591700 1 -- 172.21.15.84:0/774091469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b301a6150 con 0x7f9b30102320 2024-04-26T18:24:19.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.535+0000 7f9b36591700 1 --2- 172.21.15.84:0/774091469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b30102320 0x7f9b3019ab90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00b530 tx=0x7f9b2c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:19.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.535+0000 7f9b277fe700 1 -- 172.21.15.84:0/774091469 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c015020 con 0x7f9b30102320 2024-04-26T18:24:19.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.535+0000 7f9b277fe700 1 -- 172.21.15.84:0/774091469 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b2c011ce0 con 0x7f9b30102320 2024-04-26T18:24:19.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.535+0000 7f9b277fe700 1 -- 172.21.15.84:0/774091469 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2c010d20 con 0x7f9b30102320 2024-04-26T18:24:19.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.535+0000 7f9b387f5700 1 -- 172.21.15.84:0/774091469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b301a6320 con 0x7f9b30102320 2024-04-26T18:24:19.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.535+0000 7f9b387f5700 1 -- 172.21.15.84:0/774091469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b301a67e0 con 0x7f9b30102320 2024-04-26T18:24:19.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.536+0000 7f9b387f5700 1 -- 172.21.15.84:0/774091469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b3010a900 con 0x7f9b30102320 2024-04-26T18:24:19.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.537+0000 7f9b277fe700 1 -- 172.21.15.84:0/774091469 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9b2c018020 con 0x7f9b30102320 2024-04-26T18:24:19.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.537+0000 7f9b277fe700 1 --2- 172.21.15.84:0/774091469 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f9b1c0a3c40 0x7f9b1c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:19.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.537+0000 7f9b35d90700 1 --2- 172.21.15.84:0/774091469 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f9b1c0a3c40 0x7f9b1c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:19.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.537+0000 7f9b277fe700 1 -- 172.21.15.84:0/774091469 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 7497+0+0 (secure 0 0 0) 0x7f9b2c0cfed0 con 0x7f9b30102320 2024-04-26T18:24:19.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.538+0000 7f9b35d90700 1 --2- 172.21.15.84:0/774091469 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f9b1c0a3c40 0x7f9b1c0a6100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9b20005cd0 tx=0x7f9b20005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:19.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.541+0000 7f9b277fe700 1 -- 172.21.15.84:0/774091469 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b2c01f020 con 0x7f9b30102320 2024-04-26T18:24:19.738 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.738+0000 7f9b387f5700 1 -- 172.21.15.84:0/774091469 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f9b3006b6f0 con 0x7f9b30102320 2024-04-26T18:24:19.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.931+0000 7f9b277fe700 1 -- 172.21.15.84:0/774091469 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v86) ==== 170+0+0 (secure 0 0 0) 0x7f9b2c09cbe0 con 0x7f9b30102320 2024-04-26T18:24:19.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.934+0000 7f9b387f5700 1 -- 172.21.15.84:0/774091469 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f9b1c0a3c40 msgr2=0x7f9b1c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:19.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.934+0000 7f9b387f5700 1 --2- 172.21.15.84:0/774091469 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f9b1c0a3c40 0x7f9b1c0a6100 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f9b20005cd0 tx=0x7f9b20005c40 comp rx=0 tx=0).stop 2024-04-26T18:24:19.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.934+0000 7f9b387f5700 1 -- 172.21.15.84:0/774091469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b30102320 msgr2=0x7f9b3019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:19.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.934+0000 7f9b387f5700 1 --2- 172.21.15.84:0/774091469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b30102320 0x7f9b3019ab90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9b2c00b530 tx=0x7f9b2c00b8f0 comp rx=0 tx=0).stop 2024-04-26T18:24:19.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.934+0000 7f9b387f5700 1 -- 172.21.15.84:0/774091469 shutdown_connections 2024-04-26T18:24:19.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.934+0000 7f9b387f5700 1 --2- 172.21.15.84:0/774091469 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f9b1c0a3c40 0x7f9b1c0a6100 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:19.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.934+0000 7f9b387f5700 1 --2- 172.21.15.84:0/774091469 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b30102320 0x7f9b3019ab90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:19.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.934+0000 7f9b387f5700 1 --2- 172.21.15.84:0/774091469 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b30108390 0x7f9b301a5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:19.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.934+0000 7f9b387f5700 1 --2- 172.21.15.84:0/774091469 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b30102ce0 0x7f9b3019b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:19.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.934+0000 7f9b387f5700 1 -- 172.21.15.84:0/774091469 >> 172.21.15.84:0/774091469 conn(0x7f9b300792d0 msgr2=0x7f9b30101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:19.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.934+0000 7f9b387f5700 1 -- 172.21.15.84:0/774091469 shutdown_connections 2024-04-26T18:24:19.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:19.934+0000 7f9b387f5700 1 -- 172.21.15.84:0/774091469 wait complete. 2024-04-26T18:24:19.937 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' created 2024-04-26T18:24:20.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:19 smithi084 ceph-mon[29141]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T18:24:20.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:19 smithi084 ceph-mon[29141]: pgmap v292: 33 pgs: 6 creating+peering, 9 active+clean, 18 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:20.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:19 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/774091469' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T18:24:20.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:19 smithi174 ceph-mon[32705]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T18:24:20.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:19 smithi174 ceph-mon[32705]: pgmap v292: 33 pgs: 6 creating+peering, 9 active+clean, 18 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:20.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:19 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/774091469' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T18:24:20.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:19 smithi103 ceph-mon[32727]: osdmap e85: 12 total, 12 up, 12 in 2024-04-26T18:24:20.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:19 smithi103 ceph-mon[32727]: pgmap v292: 33 pgs: 6 creating+peering, 9 active+clean, 18 unknown; 577 KiB data, 359 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:20.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:19 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/774091469' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-26T18:24:20.627 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-26T18:24:20.866 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:21.218 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:20 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/774091469' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T18:24:21.218 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:20 smithi084 ceph-mon[29141]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T18:24:21.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:20 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/774091469' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T18:24:21.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:20 smithi103 ceph-mon[32727]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T18:24:21.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:20 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/774091469' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-26T18:24:21.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:20 smithi174 ceph-mon[32705]: osdmap e86: 12 total, 12 up, 12 in 2024-04-26T18:24:22.318 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:21 smithi084 ceph-mon[29141]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T18:24:22.318 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:21 smithi084 ceph-mon[29141]: pgmap v295: 97 pgs: 25 creating+peering, 34 active+clean, 38 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:22.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:21 smithi103 ceph-mon[32727]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T18:24:22.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:21 smithi103 ceph-mon[32727]: pgmap v295: 97 pgs: 25 creating+peering, 34 active+clean, 38 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:22.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:21 smithi174 ceph-mon[32705]: osdmap e87: 12 total, 12 up, 12 in 2024-04-26T18:24:22.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:21 smithi174 ceph-mon[32705]: pgmap v295: 97 pgs: 25 creating+peering, 34 active+clean, 38 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:22.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.557+0000 7f65585f3700 1 -- 172.21.15.84:0/750962100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6550102320 msgr2=0x7f65501027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:22.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.557+0000 7f65585f3700 1 --2- 172.21.15.84:0/750962100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6550102320 0x7f65501027a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6540009990 tx=0x7f6540009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 -- 172.21.15.84:0/750962100 shutdown_connections 2024-04-26T18:24:22.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 --2- 172.21.15.84:0/750962100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6550102ce0 0x7f655010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 --2- 172.21.15.84:0/750962100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6550108390 0x7f6550108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 --2- 172.21.15.84:0/750962100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6550102320 0x7f65501027a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 -- 172.21.15.84:0/750962100 >> 172.21.15.84:0/750962100 conn(0x7f65500792d0 msgr2=0x7f65500796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:22.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 -- 172.21.15.84:0/750962100 shutdown_connections 2024-04-26T18:24:22.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 -- 172.21.15.84:0/750962100 wait complete. 2024-04-26T18:24:22.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 Processor -- start 2024-04-26T18:24:22.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 -- start start 2024-04-26T18:24:22.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6550102320 0x7f655019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6550102ce0 0x7f655019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6550108390 0x7f65501a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f65501124b0 con 0x7f6550102ce0 2024-04-26T18:24:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6550112330 con 0x7f6550102320 2024-04-26T18:24:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.558+0000 7f65585f3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6550112630 con 0x7f6550108390 2024-04-26T18:24:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f6555b8e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6550102ce0 0x7f655019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f6555b8e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6550102ce0 0x7f655019aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52134/0 (socket says 172.21.15.84:52134) 2024-04-26T18:24:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f6555b8e700 1 -- 172.21.15.84:0/2421287660 learned_addr learned my addr 172.21.15.84:0/2421287660 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f655638f700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6550102320 0x7f655019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f6556b90700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6550108390 0x7f65501a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f6555b8e700 1 -- 172.21.15.84:0/2421287660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6550108390 msgr2=0x7f65501a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f6555b8e700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6550108390 0x7f65501a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f6555b8e700 1 -- 172.21.15.84:0/2421287660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6550102320 msgr2=0x7f655019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f6555b8e700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6550102320 0x7f655019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f6555b8e700 1 -- 172.21.15.84:0/2421287660 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65501a5f70 con 0x7f6550102ce0 2024-04-26T18:24:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f6556b90700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6550108390 0x7f65501a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f6555b8e700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6550102ce0 0x7f655019aef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6540000c00 tx=0x7f65400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f65477fe700 1 -- 172.21.15.84:0/2421287660 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540027070 con 0x7f6550102ce0 2024-04-26T18:24:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f65585f3700 1 -- 172.21.15.84:0/2421287660 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65501a6140 con 0x7f6550102ce0 2024-04-26T18:24:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f65477fe700 1 -- 172.21.15.84:0/2421287660 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6540003f20 con 0x7f6550102ce0 2024-04-26T18:24:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f65477fe700 1 -- 172.21.15.84:0/2421287660 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6540022cf0 con 0x7f6550102ce0 2024-04-26T18:24:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.559+0000 7f65585f3700 1 -- 172.21.15.84:0/2421287660 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65501a65a0 con 0x7f6550102ce0 2024-04-26T18:24:22.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.560+0000 7f65585f3700 1 -- 172.21.15.84:0/2421287660 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f655006a110 con 0x7f6550102ce0 2024-04-26T18:24:22.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.561+0000 7f65477fe700 1 -- 172.21.15.84:0/2421287660 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f654002a070 con 0x7f6550102ce0 2024-04-26T18:24:22.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.562+0000 7f65477fe700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f653c0a3c40 0x7f653c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:22.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.562+0000 7f655638f700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f653c0a3c40 0x7f653c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:22.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.562+0000 7f65477fe700 1 -- 172.21.15.84:0/2421287660 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 7879+0+0 (secure 0 0 0) 0x7f65400d12e0 con 0x7f6550102ce0 2024-04-26T18:24:22.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.562+0000 7f655638f700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f653c0a3c40 0x7f653c0a6100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f654c005950 tx=0x7f654c00a300 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:22.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.566+0000 7f65477fe700 1 -- 172.21.15.84:0/2421287660 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6540013070 con 0x7f6550102ce0 2024-04-26T18:24:22.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.809+0000 7f65585f3700 1 -- 172.21.15.84:0/2421287660 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f655002e490 con 0x7f6550102ce0 2024-04-26T18:24:22.948 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.948+0000 7f65477fe700 1 -- 172.21.15.84:0/2421287660 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f654009df00 con 0x7f6550102ce0 2024-04-26T18:24:22.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.950+0000 7f65585f3700 1 -- 172.21.15.84:0/2421287660 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f653c0a3c40 msgr2=0x7f653c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:22.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.950+0000 7f65585f3700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f653c0a3c40 0x7f653c0a6100 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f654c005950 tx=0x7f654c00a300 comp rx=0 tx=0).stop 2024-04-26T18:24:22.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.950+0000 7f65585f3700 1 -- 172.21.15.84:0/2421287660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6550102ce0 msgr2=0x7f655019aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:22.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.950+0000 7f65585f3700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6550102ce0 0x7f655019aef0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6540000c00 tx=0x7f65400047b0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.951+0000 7f65585f3700 1 -- 172.21.15.84:0/2421287660 shutdown_connections 2024-04-26T18:24:22.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.951+0000 7f65585f3700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f653c0a3c40 0x7f653c0a6100 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.951+0000 7f65585f3700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6550108390 0x7f65501a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.951+0000 7f65585f3700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6550102320 0x7f655019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.951+0000 7f65585f3700 1 --2- 172.21.15.84:0/2421287660 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6550102ce0 0x7f655019aef0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:22.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.951+0000 7f65585f3700 1 -- 172.21.15.84:0/2421287660 >> 172.21.15.84:0/2421287660 conn(0x7f65500792d0 msgr2=0x7f6550101130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:22.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.951+0000 7f65585f3700 1 -- 172.21.15.84:0/2421287660 shutdown_connections 2024-04-26T18:24:22.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:22.951+0000 7f65585f3700 1 -- 172.21.15.84:0/2421287660 wait complete. 2024-04-26T18:24:22.951 INFO:teuthology.orchestra.run.smithi084.stderr:new fs with metadata pool 2 and data pool 3 2024-04-26T18:24:23.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:22 smithi084 ceph-mon[29141]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T18:24:23.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:22 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2421287660' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T18:24:23.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:22 smithi084 ceph-mon[29141]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T18:24:23.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:22 smithi103 ceph-mon[32727]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T18:24:23.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:22 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2421287660' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T18:24:23.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:22 smithi103 ceph-mon[32727]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T18:24:23.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:22 smithi174 ceph-mon[32705]: osdmap e88: 12 total, 12 up, 12 in 2024-04-26T18:24:23.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:22 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2421287660' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-26T18:24:23.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:22 smithi174 ceph-mon[32705]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-26T18:24:23.641 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:24:23.878 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:24.243 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:23 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2421287660' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T18:24:24.244 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:23 smithi084 ceph-mon[29141]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T18:24:24.244 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:23 smithi084 ceph-mon[29141]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T18:24:24.244 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:23 smithi084 ceph-mon[29141]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T18:24:24.244 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:23 smithi084 ceph-mon[29141]: pgmap v298: 97 pgs: 19 creating+peering, 59 active+clean, 19 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:24.269 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:23 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2421287660' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T18:24:24.270 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:23 smithi103 ceph-mon[32727]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T18:24:24.270 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:23 smithi103 ceph-mon[32727]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T18:24:24.270 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:23 smithi103 ceph-mon[32727]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T18:24:24.270 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:23 smithi103 ceph-mon[32727]: pgmap v298: 97 pgs: 19 creating+peering, 59 active+clean, 19 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:24.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:23 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2421287660' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-26T18:24:24.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:23 smithi174 ceph-mon[32705]: osdmap e89: 12 total, 12 up, 12 in 2024-04-26T18:24:24.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:23 smithi174 ceph-mon[32705]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-26T18:24:24.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:23 smithi174 ceph-mon[32705]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-26T18:24:24.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:23 smithi174 ceph-mon[32705]: pgmap v298: 97 pgs: 19 creating+peering, 59 active+clean, 19 unknown; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:24:24.626 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:24:24 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b[62948]: 2024-04-26T18:24:24.269+0000 7fea53be7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-26T18:24:25.257 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:24 smithi084 ceph-mon[29141]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T18:24:25.257 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:24 smithi084 ceph-mon[29141]: Cluster is now healthy 2024-04-26T18:24:25.257 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:24 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] up:active 2024-04-26T18:24:25.257 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:24 smithi084 ceph-mon[29141]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:24:25.257 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:24 smithi084 ceph-mon[29141]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:24:25.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:24 smithi103 ceph-mon[32727]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T18:24:25.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:24 smithi103 ceph-mon[32727]: Cluster is now healthy 2024-04-26T18:24:25.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:24 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] up:active 2024-04-26T18:24:25.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:24 smithi103 ceph-mon[32727]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:24:25.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:24 smithi103 ceph-mon[32727]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:24:25.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:24 smithi174 ceph-mon[32705]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-04-26T18:24:25.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:24 smithi174 ceph-mon[32705]: Cluster is now healthy 2024-04-26T18:24:25.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:24 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] up:active 2024-04-26T18:24:25.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:24 smithi174 ceph-mon[32705]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:24:25.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:24 smithi174 ceph-mon[32705]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:24:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.626+0000 7fa1d143b700 1 -- 172.21.15.84:0/375852016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc1023b0 msgr2=0x7fa1cc1102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.626+0000 7fa1d143b700 1 --2- 172.21.15.84:0/375852016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc1023b0 0x7fa1cc1102f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fa1c400b0a0 tx=0x7fa1c400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:24:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.626+0000 7fa1d143b700 1 -- 172.21.15.84:0/375852016 shutdown_connections 2024-04-26T18:24:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.626+0000 7fa1d143b700 1 --2- 172.21.15.84:0/375852016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc1023b0 0x7fa1cc1102f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.626+0000 7fa1d143b700 1 --2- 172.21.15.84:0/375852016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1cc1094b0 0x7fa1cc109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.626+0000 7fa1d143b700 1 --2- 172.21.15.84:0/375852016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1cc101a10 0x7fa1cc101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.626+0000 7fa1d143b700 1 -- 172.21.15.84:0/375852016 >> 172.21.15.84:0/375852016 conn(0x7fa1cc078990 msgr2=0x7fa1cc0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.626+0000 7fa1d143b700 1 -- 172.21.15.84:0/375852016 shutdown_connections 2024-04-26T18:24:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.626+0000 7fa1d143b700 1 -- 172.21.15.84:0/375852016 wait complete. 2024-04-26T18:24:25.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1d143b700 1 Processor -- start 2024-04-26T18:24:25.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1d143b700 1 -- start start 2024-04-26T18:24:25.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1d143b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc101a10 0x7fa1cc1a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:25.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1d143b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1cc1023b0 0x7fa1cc1a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:25.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1d143b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1cc1094b0 0x7fa1cc19ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:25.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1d143b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc113660 con 0x7fa1cc101a10 2024-04-26T18:24:25.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1d143b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1134e0 con 0x7fa1cc1023b0 2024-04-26T18:24:25.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1d143b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1137e0 con 0x7fa1cc1094b0 2024-04-26T18:24:25.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1caffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc101a10 0x7fa1cc1a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:25.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1caffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc101a10 0x7fa1cc1a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52142/0 (socket says 172.21.15.84:52142) 2024-04-26T18:24:25.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1caffd700 1 -- 172.21.15.84:0/1914970144 learned_addr learned my addr 172.21.15.84:0/1914970144 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:25.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1c27fc700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1cc1023b0 0x7fa1cc1a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:25.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1caffd700 1 -- 172.21.15.84:0/1914970144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1cc1094b0 msgr2=0x7fa1cc19ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:25.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1caffd700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1cc1094b0 0x7fa1cc19ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:25.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1caffd700 1 -- 172.21.15.84:0/1914970144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1cc1023b0 msgr2=0x7fa1cc1a1e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:25.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1caffd700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1cc1023b0 0x7fa1cc1a1e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:25.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1caffd700 1 -- 172.21.15.84:0/1914970144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1cc19c1f0 con 0x7fa1cc101a10 2024-04-26T18:24:25.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.627+0000 7fa1c27fc700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1cc1023b0 0x7fa1cc1a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:25.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.628+0000 7fa1caffd700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc101a10 0x7fa1cc1a18d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00c9b0 tx=0x7fa1bc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:25.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.628+0000 7fa1c8ff9700 1 -- 172.21.15.84:0/1914970144 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc007b00 con 0x7fa1cc101a10 2024-04-26T18:24:25.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.628+0000 7fa1d143b700 1 -- 172.21.15.84:0/1914970144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1cc19c4e0 con 0x7fa1cc101a10 2024-04-26T18:24:25.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.628+0000 7fa1c8ff9700 1 -- 172.21.15.84:0/1914970144 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1bc007ca0 con 0x7fa1cc101a10 2024-04-26T18:24:25.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.628+0000 7fa1c8ff9700 1 -- 172.21.15.84:0/1914970144 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc018730 con 0x7fa1cc101a10 2024-04-26T18:24:25.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.628+0000 7fa1d143b700 1 -- 172.21.15.84:0/1914970144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1cc1a87a0 con 0x7fa1cc101a10 2024-04-26T18:24:25.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.629+0000 7fa1d143b700 1 -- 172.21.15.84:0/1914970144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1cc104d30 con 0x7fa1cc101a10 2024-04-26T18:24:25.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.630+0000 7fa1c8ff9700 1 -- 172.21.15.84:0/1914970144 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa1bc00f450 con 0x7fa1cc101a10 2024-04-26T18:24:25.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.630+0000 7fa1c8ff9700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa1b80a3c40 0x7fa1b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:25.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.630+0000 7fa1c27fc700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa1b80a3c40 0x7fa1b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:25.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.630+0000 7fa1c8ff9700 1 -- 172.21.15.84:0/1914970144 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fa1bc0d0660 con 0x7fa1cc101a10 2024-04-26T18:24:25.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.631+0000 7fa1c27fc700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa1b80a3c40 0x7fa1b80a6100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc107e20 tx=0x7fa1b4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:25.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.634+0000 7fa1c8ff9700 1 -- 172.21.15.84:0/1914970144 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1bc014030 con 0x7fa1cc101a10 2024-04-26T18:24:25.829 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.829+0000 7fa1d143b700 1 -- 172.21.15.84:0/1914970144 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa1cc1a8ad0 con 0x7fa1cc101a10 2024-04-26T18:24:25.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.831+0000 7fa1c8ff9700 1 -- 172.21.15.84:0/1914970144 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22183 (secure 0 0 0) 0x7fa1bc09d220 con 0x7fa1cc101a10 2024-04-26T18:24:25.831 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:24:25.832 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":89,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:16:06.267694+0000","modified":"2024-04-26T18:24:22.811030+0000","last_up_change":"2024-04-26T18:23:19.906511+0000","last_in_change":"2024-04-26T18:22:59.541869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:19:31.174024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:24:16.533128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:24:19.742656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1076912-986e-4825-b1d1-5fa9d69bc07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6803","nonce":140050011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6805","nonce":140050011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6809","nonce":140050011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6807","nonce":140050011}]},"public_addr":"172.21.15.84:6803/140050011","cluster_addr":"172.21.15.84:6805/140050011","heartbeat_back_addr":"172.21.15.84:6809/140050011","heartbeat_front_addr":"172.21.15.84:6807/140050011","state":["exists","up"]},{"osd":1,"uuid":"fa93293d-9058-4804-ba4b-7e1e674b0fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6801","nonce":2512857657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6803","nonce":2512857657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6807","nonce":2512857657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6805","nonce":2512857657}]},"public_addr":"172.21.15.103:6801/2512857657","cluster_addr":"172.21.15.103:6803/2512857657","heartbeat_back_addr":"172.21.15.103:6807/2512857657","heartbeat_front_addr":"172.21.15.103:6805/2512857657","state":["exists","up"]},{"osd":2,"uuid":"bf78a44f-80a2-4cd7-974f-5abdf771e650","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6801","nonce":76340924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6803","nonce":76340924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6807","nonce":76340924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6805","nonce":76340924}]},"public_addr":"172.21.15.174:6801/76340924","cluster_addr":"172.21.15.174:6803/76340924","heartbeat_back_addr":"172.21.15.174:6807/76340924","heartbeat_front_addr":"172.21.15.174:6805/76340924","state":["exists","up"]},{"osd":3,"uuid":"55fd0035-bc76-4235-b8b5-5cf5ef770139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6811","nonce":2352789256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6813","nonce":2352789256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6817","nonce":2352789256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6815","nonce":2352789256}]},"public_addr":"172.21.15.84:6811/2352789256","cluster_addr":"172.21.15.84:6813/2352789256","heartbeat_back_addr":"172.21.15.84:6817/2352789256","heartbeat_front_addr":"172.21.15.84:6815/2352789256","state":["exists","up"]},{"osd":4,"uuid":"c278a0f3-e436-4d1c-a009-f9165ebf3c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6809","nonce":1619258252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6811","nonce":1619258252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6815","nonce":1619258252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6813","nonce":1619258252}]},"public_addr":"172.21.15.103:6809/1619258252","cluster_addr":"172.21.15.103:6811/1619258252","heartbeat_back_addr":"172.21.15.103:6815/1619258252","heartbeat_front_addr":"172.21.15.103:6813/1619258252","state":["exists","up"]},{"osd":5,"uuid":"6505be9b-c1b0-4d8b-944e-9920f90cb88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6809","nonce":3168291768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6811","nonce":3168291768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6815","nonce":3168291768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6813","nonce":3168291768}]},"public_addr":"172.21.15.174:6809/3168291768","cluster_addr":"172.21.15.174:6811/3168291768","heartbeat_back_addr":"172.21.15.174:6815/3168291768","heartbeat_front_addr":"172.21.15.174:6813/3168291768","state":["exists","up"]},{"osd":6,"uuid":"a037b66a-7488-469d-b73a-14cc0d35cb5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6819","nonce":886635297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6821","nonce":886635297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6825","nonce":886635297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6823","nonce":886635297}]},"public_addr":"172.21.15.84:6819/886635297","cluster_addr":"172.21.15.84:6821/886635297","heartbeat_back_addr":"172.21.15.84:6825/886635297","heartbeat_front_addr":"172.21.15.84:6823/886635297","state":["exists","up"]},{"osd":7,"uuid":"c425a13f-8019-4484-86a0-a34fa4d8de81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6817","nonce":1738278079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6819","nonce":1738278079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6823","nonce":1738278079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6821","nonce":1738278079}]},"public_addr":"172.21.15.103:6817/1738278079","cluster_addr":"172.21.15.103:6819/1738278079","heartbeat_back_addr":"172.21.15.103:6823/1738278079","heartbeat_front_addr":"172.21.15.103:6821/1738278079","state":["exists","up"]},{"osd":8,"uuid":"687e6c08-bfa4-440c-b3bc-90a5748989d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6817","nonce":38410548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6819","nonce":38410548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6823","nonce":38410548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6821","nonce":38410548}]},"public_addr":"172.21.15.174:6817/38410548","cluster_addr":"172.21.15.174:6819/38410548","heartbeat_back_addr":"172.21.15.174:6823/38410548","heartbeat_front_addr":"172.21.15.174:6821/38410548","state":["exists","up"]},{"osd":9,"uuid":"059eb22d-6dfb-4dd0-b192-84ed42ee1fbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6827","nonce":2926393927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6829","nonce":2926393927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6833","nonce":2926393927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6831","nonce":2926393927}]},"public_addr":"172.21.15.84:6827/2926393927","cluster_addr":"172.21.15.84:6829/2926393927","heartbeat_back_addr":"172.21.15.84:6833/2926393927","heartbeat_front_addr":"172.21.15.84:6831/2926393927","state":["exists","up"]},{"osd":10,"uuid":"da31e305-5220-4a87-a131-005b7c7f76b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6825","nonce":2705726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6827","nonce":2705726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6831","nonce":2705726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6829","nonce":2705726915}]},"public_addr":"172.21.15.103:6825/2705726915","cluster_addr":"172.21.15.103:6827/2705726915","heartbeat_back_addr":"172.21.15.103:6831/2705726915","heartbeat_front_addr":"172.21.15.103:6829/2705726915","state":["exists","up"]},{"osd":11,"uuid":"3fde6c8f-ee02-41ee-a34f-73072b4761e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6825","nonce":1668672128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6827","nonce":1668672128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6831","nonce":1668672128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6829","nonce":1668672128}]},"public_addr":"172.21.15.174:6825/1668672128","cluster_addr":"172.21.15.174:6827/1668672128","heartbeat_back_addr":"172.21.15.174:6831/1668672128","heartbeat_front_addr":"172.21.15.174:6829/1668672128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:39.099188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:03.327608+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:27.714108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:52.506015+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:17.631670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:42.702489+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:07.606697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:34.123214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:59.534658+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:25.357720+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:51.543106+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:23:18.208230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1604680842":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6800/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/538363975":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/952052990":"2024-04-27T18:17:05.083192+0000","172.21.15.84:0/1266942705":"2024-04-27T18:16:37.859669+0000","172.21.15.84:6800/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3970542149":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3607125049":"2024-04-27T18:17:05.083192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:24:25.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.834+0000 7fa1d143b700 1 -- 172.21.15.84:0/1914970144 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa1b80a3c40 msgr2=0x7fa1b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:25.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.834+0000 7fa1d143b700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa1b80a3c40 0x7fa1b80a6100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc107e20 tx=0x7fa1b4008040 comp rx=0 tx=0).stop 2024-04-26T18:24:25.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.834+0000 7fa1d143b700 1 -- 172.21.15.84:0/1914970144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc101a10 msgr2=0x7fa1cc1a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:25.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.834+0000 7fa1d143b700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc101a10 0x7fa1cc1a18d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00c9b0 tx=0x7fa1bc00cd70 comp rx=0 tx=0).stop 2024-04-26T18:24:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.834+0000 7fa1d143b700 1 -- 172.21.15.84:0/1914970144 shutdown_connections 2024-04-26T18:24:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.834+0000 7fa1d143b700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa1b80a3c40 0x7fa1b80a6100 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.834+0000 7fa1d143b700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa1cc101a10 0x7fa1cc1a18d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.834+0000 7fa1d143b700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa1cc1094b0 0x7fa1cc19ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.834+0000 7fa1d143b700 1 --2- 172.21.15.84:0/1914970144 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa1cc1023b0 0x7fa1cc1a1e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.834+0000 7fa1d143b700 1 -- 172.21.15.84:0/1914970144 >> 172.21.15.84:0/1914970144 conn(0x7fa1cc078990 msgr2=0x7fa1cc106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.834+0000 7fa1d143b700 1 -- 172.21.15.84:0/1914970144 shutdown_connections 2024-04-26T18:24:25.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:25.834+0000 7fa1d143b700 1 -- 172.21.15.84:0/1914970144 wait complete. 2024-04-26T18:24:26.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:25 smithi084 ceph-mon[29141]: pgmap v299: 97 pgs: 19 creating+peering, 78 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 2 op/s 2024-04-26T18:24:26.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:25 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1914970144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:24:26.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:25 smithi103 ceph-mon[32727]: pgmap v299: 97 pgs: 19 creating+peering, 78 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 2 op/s 2024-04-26T18:24:26.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:25 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1914970144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:24:26.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:25 smithi174 ceph-mon[32705]: pgmap v299: 97 pgs: 19 creating+peering, 78 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 2 op/s 2024-04-26T18:24:26.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:25 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1914970144' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:24:26.526 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:24:26.778 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:28.432 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:28 smithi084 ceph-mon[29141]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-04-26T18:24:28.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:28 smithi103 ceph-mon[32727]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-04-26T18:24:28.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.653+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/657718258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6acc103470 msgr2=0x7f6acc1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:28.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.653+0000 7f6ad1f2b700 1 --2- 172.21.15.84:0/657718258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6acc103470 0x7f6acc1038f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ac0009990 tx=0x7f6ac0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:28.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.653+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/657718258 shutdown_connections 2024-04-26T18:24:28.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.653+0000 7f6ad1f2b700 1 --2- 172.21.15.84:0/657718258 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6acc103e30 0x7f6acc110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:28.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.653+0000 7f6ad1f2b700 1 --2- 172.21.15.84:0/657718258 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6acc1094c0 0x7f6acc1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:28.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.653+0000 7f6ad1f2b700 1 --2- 172.21.15.84:0/657718258 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6acc103470 0x7f6acc1038f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:28.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.653+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/657718258 >> 172.21.15.84:0/657718258 conn(0x7f6acc07a580 msgr2=0x7f6acc07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:28.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.653+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/657718258 shutdown_connections 2024-04-26T18:24:28.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.653+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/657718258 wait complete. 2024-04-26T18:24:28.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.653+0000 7f6ad1f2b700 1 Processor -- start 2024-04-26T18:24:28.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6ad1f2b700 1 -- start start 2024-04-26T18:24:28.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6ad1f2b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6acc103470 0x7f6acc19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:28.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6ad1f2b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6acc103e30 0x7f6acc19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:28.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6ad1f2b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6acc1094c0 0x7f6acc1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:28.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6ad1f2b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6acc113640 con 0x7f6acc1094c0 2024-04-26T18:24:28.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6ad1f2b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6acc1134c0 con 0x7f6acc103470 2024-04-26T18:24:28.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6ad1f2b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6acc1137c0 con 0x7f6acc103e30 2024-04-26T18:24:28.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6acbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6acc1094c0 0x7f6acc1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:28.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6acbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6acc1094c0 0x7f6acc1999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54828/0 (socket says 172.21.15.84:54828) 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6acbfff700 1 -- 172.21.15.84:0/1012794912 learned_addr learned my addr 172.21.15.84:0/1012794912 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6acb7fe700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6acc103470 0x7f6acc19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6acaffd700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6acc103e30 0x7f6acc19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6acbfff700 1 -- 172.21.15.84:0/1012794912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6acc103e30 msgr2=0x7f6acc19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6acbfff700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6acc103e30 0x7f6acc19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6acbfff700 1 -- 172.21.15.84:0/1012794912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6acc103470 msgr2=0x7f6acc19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6acbfff700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6acc103470 0x7f6acc19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6acbfff700 1 -- 172.21.15.84:0/1012794912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6acc19a120 con 0x7f6acc1094c0 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.654+0000 7f6acaffd700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6acc103e30 0x7f6acc19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.655+0000 7f6acbfff700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6acc1094c0 0x7f6acc1999c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400ada0 tx=0x7f6ac400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.655+0000 7f6ac8ff9700 1 -- 172.21.15.84:0/1012794912 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4017020 con 0x7f6acc1094c0 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.655+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/1012794912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6acc19a410 con 0x7f6acc1094c0 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.655+0000 7f6ac8ff9700 1 -- 172.21.15.84:0/1012794912 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ac400ce30 con 0x7f6acc1094c0 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.655+0000 7f6ac8ff9700 1 -- 172.21.15.84:0/1012794912 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ac4005350 con 0x7f6acc1094c0 2024-04-26T18:24:28.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.655+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/1012794912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6acc1a66d0 con 0x7f6acc1094c0 2024-04-26T18:24:28.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.656+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/1012794912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6acc06a110 con 0x7f6acc1094c0 2024-04-26T18:24:28.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.657+0000 7f6ac8ff9700 1 -- 172.21.15.84:0/1012794912 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6ac400c980 con 0x7f6acc1094c0 2024-04-26T18:24:28.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.658+0000 7f6ac8ff9700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6ab40a3c40 0x7f6ab40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:28.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.658+0000 7f6acb7fe700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6ab40a3c40 0x7f6ab40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:28.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.658+0000 7f6ac8ff9700 1 -- 172.21.15.84:0/1012794912 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f6ac40cffe0 con 0x7f6acc1094c0 2024-04-26T18:24:28.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.658+0000 7f6acb7fe700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6ab40a3c40 0x7f6ab40a6100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6acc19ab00 tx=0x7f6abc009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:28.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.661+0000 7f6ac8ff9700 1 -- 172.21.15.84:0/1012794912 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ac4021020 con 0x7f6acc1094c0 2024-04-26T18:24:28.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:28 smithi174 ceph-mon[32705]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 3 op/s 2024-04-26T18:24:28.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.863+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/1012794912 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6acc02e490 con 0x7f6acc1094c0 2024-04-26T18:24:28.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.865+0000 7f6ac8ff9700 1 -- 172.21.15.84:0/1012794912 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22183 (secure 0 0 0) 0x7f6ac401a310 con 0x7f6acc1094c0 2024-04-26T18:24:28.865 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:24:28.865 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":89,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:16:06.267694+0000","modified":"2024-04-26T18:24:22.811030+0000","last_up_change":"2024-04-26T18:23:19.906511+0000","last_in_change":"2024-04-26T18:22:59.541869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:19:31.174024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:24:16.533128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:24:19.742656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1076912-986e-4825-b1d1-5fa9d69bc07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6803","nonce":140050011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6805","nonce":140050011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6809","nonce":140050011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6807","nonce":140050011}]},"public_addr":"172.21.15.84:6803/140050011","cluster_addr":"172.21.15.84:6805/140050011","heartbeat_back_addr":"172.21.15.84:6809/140050011","heartbeat_front_addr":"172.21.15.84:6807/140050011","state":["exists","up"]},{"osd":1,"uuid":"fa93293d-9058-4804-ba4b-7e1e674b0fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6801","nonce":2512857657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6803","nonce":2512857657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6807","nonce":2512857657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6805","nonce":2512857657}]},"public_addr":"172.21.15.103:6801/2512857657","cluster_addr":"172.21.15.103:6803/2512857657","heartbeat_back_addr":"172.21.15.103:6807/2512857657","heartbeat_front_addr":"172.21.15.103:6805/2512857657","state":["exists","up"]},{"osd":2,"uuid":"bf78a44f-80a2-4cd7-974f-5abdf771e650","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6801","nonce":76340924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6803","nonce":76340924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6807","nonce":76340924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6805","nonce":76340924}]},"public_addr":"172.21.15.174:6801/76340924","cluster_addr":"172.21.15.174:6803/76340924","heartbeat_back_addr":"172.21.15.174:6807/76340924","heartbeat_front_addr":"172.21.15.174:6805/76340924","state":["exists","up"]},{"osd":3,"uuid":"55fd0035-bc76-4235-b8b5-5cf5ef770139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6811","nonce":2352789256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6813","nonce":2352789256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6817","nonce":2352789256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6815","nonce":2352789256}]},"public_addr":"172.21.15.84:6811/2352789256","cluster_addr":"172.21.15.84:6813/2352789256","heartbeat_back_addr":"172.21.15.84:6817/2352789256","heartbeat_front_addr":"172.21.15.84:6815/2352789256","state":["exists","up"]},{"osd":4,"uuid":"c278a0f3-e436-4d1c-a009-f9165ebf3c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6809","nonce":1619258252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6811","nonce":1619258252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6815","nonce":1619258252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6813","nonce":1619258252}]},"public_addr":"172.21.15.103:6809/1619258252","cluster_addr":"172.21.15.103:6811/1619258252","heartbeat_back_addr":"172.21.15.103:6815/1619258252","heartbeat_front_addr":"172.21.15.103:6813/1619258252","state":["exists","up"]},{"osd":5,"uuid":"6505be9b-c1b0-4d8b-944e-9920f90cb88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6809","nonce":3168291768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6811","nonce":3168291768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6815","nonce":3168291768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6813","nonce":3168291768}]},"public_addr":"172.21.15.174:6809/3168291768","cluster_addr":"172.21.15.174:6811/3168291768","heartbeat_back_addr":"172.21.15.174:6815/3168291768","heartbeat_front_addr":"172.21.15.174:6813/3168291768","state":["exists","up"]},{"osd":6,"uuid":"a037b66a-7488-469d-b73a-14cc0d35cb5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6819","nonce":886635297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6821","nonce":886635297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6825","nonce":886635297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6823","nonce":886635297}]},"public_addr":"172.21.15.84:6819/886635297","cluster_addr":"172.21.15.84:6821/886635297","heartbeat_back_addr":"172.21.15.84:6825/886635297","heartbeat_front_addr":"172.21.15.84:6823/886635297","state":["exists","up"]},{"osd":7,"uuid":"c425a13f-8019-4484-86a0-a34fa4d8de81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6817","nonce":1738278079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6819","nonce":1738278079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6823","nonce":1738278079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6821","nonce":1738278079}]},"public_addr":"172.21.15.103:6817/1738278079","cluster_addr":"172.21.15.103:6819/1738278079","heartbeat_back_addr":"172.21.15.103:6823/1738278079","heartbeat_front_addr":"172.21.15.103:6821/1738278079","state":["exists","up"]},{"osd":8,"uuid":"687e6c08-bfa4-440c-b3bc-90a5748989d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6817","nonce":38410548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6819","nonce":38410548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6823","nonce":38410548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6821","nonce":38410548}]},"public_addr":"172.21.15.174:6817/38410548","cluster_addr":"172.21.15.174:6819/38410548","heartbeat_back_addr":"172.21.15.174:6823/38410548","heartbeat_front_addr":"172.21.15.174:6821/38410548","state":["exists","up"]},{"osd":9,"uuid":"059eb22d-6dfb-4dd0-b192-84ed42ee1fbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6827","nonce":2926393927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6829","nonce":2926393927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6833","nonce":2926393927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6831","nonce":2926393927}]},"public_addr":"172.21.15.84:6827/2926393927","cluster_addr":"172.21.15.84:6829/2926393927","heartbeat_back_addr":"172.21.15.84:6833/2926393927","heartbeat_front_addr":"172.21.15.84:6831/2926393927","state":["exists","up"]},{"osd":10,"uuid":"da31e305-5220-4a87-a131-005b7c7f76b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6825","nonce":2705726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6827","nonce":2705726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6831","nonce":2705726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6829","nonce":2705726915}]},"public_addr":"172.21.15.103:6825/2705726915","cluster_addr":"172.21.15.103:6827/2705726915","heartbeat_back_addr":"172.21.15.103:6831/2705726915","heartbeat_front_addr":"172.21.15.103:6829/2705726915","state":["exists","up"]},{"osd":11,"uuid":"3fde6c8f-ee02-41ee-a34f-73072b4761e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6825","nonce":1668672128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6827","nonce":1668672128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6831","nonce":1668672128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6829","nonce":1668672128}]},"public_addr":"172.21.15.174:6825/1668672128","cluster_addr":"172.21.15.174:6827/1668672128","heartbeat_back_addr":"172.21.15.174:6831/1668672128","heartbeat_front_addr":"172.21.15.174:6829/1668672128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:39.099188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:03.327608+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:27.714108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:52.506015+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:17.631670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:42.702489+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:07.606697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:34.123214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:59.534658+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:25.357720+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:51.543106+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:23:18.208230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1604680842":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6800/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/538363975":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/952052990":"2024-04-27T18:17:05.083192+0000","172.21.15.84:0/1266942705":"2024-04-27T18:16:37.859669+0000","172.21.15.84:6800/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3970542149":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3607125049":"2024-04-27T18:17:05.083192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:24:28.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.867+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/1012794912 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6ab40a3c40 msgr2=0x7f6ab40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:28.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.867+0000 7f6ad1f2b700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6ab40a3c40 0x7f6ab40a6100 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6acc19ab00 tx=0x7f6abc009190 comp rx=0 tx=0).stop 2024-04-26T18:24:28.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.867+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/1012794912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6acc1094c0 msgr2=0x7f6acc1999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:28.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.867+0000 7f6ad1f2b700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6acc1094c0 0x7f6acc1999c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6ac400ada0 tx=0x7f6ac400c330 comp rx=0 tx=0).stop 2024-04-26T18:24:28.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.867+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/1012794912 shutdown_connections 2024-04-26T18:24:28.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.867+0000 7f6ad1f2b700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6ab40a3c40 0x7f6ab40a6100 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:28.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.867+0000 7f6ad1f2b700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6acc1094c0 0x7f6acc1999c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:28.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.867+0000 7f6ad1f2b700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6acc103470 0x7f6acc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:28.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.867+0000 7f6ad1f2b700 1 --2- 172.21.15.84:0/1012794912 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6acc103e30 0x7f6acc19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:28.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.867+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/1012794912 >> 172.21.15.84:0/1012794912 conn(0x7f6acc07a580 msgr2=0x7f6acc102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:28.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.867+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/1012794912 shutdown_connections 2024-04-26T18:24:28.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:28.867+0000 7f6ad1f2b700 1 -- 172.21.15.84:0/1012794912 wait complete. 2024-04-26T18:24:29.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:29 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1012794912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:24:29.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:29 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1012794912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:24:29.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:29 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1012794912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:24:29.694 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-04-26T18:24:29.934 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:30.319 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:30 smithi084 ceph-mon[29141]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-26T18:24:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:30 smithi103 ceph-mon[32727]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-26T18:24:30.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:30 smithi174 ceph-mon[32705]: pgmap v301: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-04-26T18:24:31.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.724+0000 7fa71cd77700 1 -- 172.21.15.84:0/3663619965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7181076b0 msgr2=0x7fa71810f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.724+0000 7fa71cd77700 1 --2- 172.21.15.84:0/3663619965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7181076b0 0x7fa71810f280 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa71000b0d0 tx=0x7fa71000b3e0 comp rx=0 tx=0).stop 2024-04-26T18:24:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.724+0000 7fa71cd77700 1 -- 172.21.15.84:0/3663619965 shutdown_connections 2024-04-26T18:24:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.724+0000 7fa71cd77700 1 --2- 172.21.15.84:0/3663619965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa7181076b0 0x7fa71810f280 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.724+0000 7fa71cd77700 1 --2- 172.21.15.84:0/3663619965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7181026b0 0x7fa718102a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.724+0000 7fa71cd77700 1 --2- 172.21.15.84:0/3663619965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa718103060 0x7fa718106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.724+0000 7fa71cd77700 1 -- 172.21.15.84:0/3663619965 >> 172.21.15.84:0/3663619965 conn(0x7fa718078230 msgr2=0x7fa718078640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.724+0000 7fa71cd77700 1 -- 172.21.15.84:0/3663619965 shutdown_connections 2024-04-26T18:24:31.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.724+0000 7fa71cd77700 1 -- 172.21.15.84:0/3663619965 wait complete. 2024-04-26T18:24:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.725+0000 7fa71cd77700 1 Processor -- start 2024-04-26T18:24:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.725+0000 7fa71cd77700 1 -- start start 2024-04-26T18:24:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa71cd77700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7181026b0 0x7fa71819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa71cd77700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa718103060 0x7fa71819ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa71cd77700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7181076b0 0x7fa7181989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa71cd77700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa7181125e0 con 0x7fa718103060 2024-04-26T18:24:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa71cd77700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa718112460 con 0x7fa7181076b0 2024-04-26T18:24:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa71cd77700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa718112760 con 0x7fa7181026b0 2024-04-26T18:24:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa715d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa718103060 0x7fa71819ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa715d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa718103060 0x7fa71819ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54840/0 (socket says 172.21.15.84:54840) 2024-04-26T18:24:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa715d9b700 1 -- 172.21.15.84:0/3693460400 learned_addr learned my addr 172.21.15.84:0/3693460400 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:31.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa71659c700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7181026b0 0x7fa71819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa716d9d700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7181076b0 0x7fa7181989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa715d9b700 1 -- 172.21.15.84:0/3693460400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7181026b0 msgr2=0x7fa71819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa715d9b700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7181026b0 0x7fa71819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa715d9b700 1 -- 172.21.15.84:0/3693460400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7181076b0 msgr2=0x7fa7181989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa715d9b700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7181076b0 0x7fa7181989a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa715d9b700 1 -- 172.21.15.84:0/3693460400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa718199100 con 0x7fa718103060 2024-04-26T18:24:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa715d9b700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa718103060 0x7fa71819ecd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa70800c970 tx=0x7fa70800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa7077fe700 1 -- 172.21.15.84:0/3693460400 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa70800ce60 con 0x7fa718103060 2024-04-26T18:24:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa71cd77700 1 -- 172.21.15.84:0/3693460400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7181993f0 con 0x7fa718103060 2024-04-26T18:24:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa7077fe700 1 -- 172.21.15.84:0/3693460400 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa708004830 con 0x7fa718103060 2024-04-26T18:24:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.726+0000 7fa7077fe700 1 -- 172.21.15.84:0/3693460400 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa708005530 con 0x7fa718103060 2024-04-26T18:24:31.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.727+0000 7fa71cd77700 1 -- 172.21.15.84:0/3693460400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7181a5660 con 0x7fa718103060 2024-04-26T18:24:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.727+0000 7fa71cd77700 1 -- 172.21.15.84:0/3693460400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa71806a110 con 0x7fa718103060 2024-04-26T18:24:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.729+0000 7fa7077fe700 1 -- 172.21.15.84:0/3693460400 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa7080049e0 con 0x7fa718103060 2024-04-26T18:24:31.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.729+0000 7fa7077fe700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa7000a3c40 0x7fa7000a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.729+0000 7fa71659c700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa7000a3c40 0x7fa7000a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.729+0000 7fa7077fe700 1 -- 172.21.15.84:0/3693460400 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fa7080d0250 con 0x7fa718103060 2024-04-26T18:24:31.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.730+0000 7fa71659c700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa7000a3c40 0x7fa7000a6100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa7181072f0 tx=0x7fa70c009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:31.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.733+0000 7fa7077fe700 1 -- 172.21.15.84:0/3693460400 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa708014030 con 0x7fa718103060 2024-04-26T18:24:31.976 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:31.976+0000 7fa71cd77700 1 -- 172.21.15.84:0/3693460400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fa71802e490 con 0x7fa718103060 2024-04-26T18:24:32.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:32.232+0000 7fa7077fe700 1 -- 172.21.15.84:0/3693460400 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7fa70809cd80 con 0x7fa718103060 2024-04-26T18:24:32.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:32.234+0000 7fa71cd77700 1 -- 172.21.15.84:0/3693460400 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa7000a3c40 msgr2=0x7fa7000a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:32.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:32.234+0000 7fa71cd77700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa7000a3c40 0x7fa7000a6100 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa7181072f0 tx=0x7fa70c009190 comp rx=0 tx=0).stop 2024-04-26T18:24:32.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:32.234+0000 7fa71cd77700 1 -- 172.21.15.84:0/3693460400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa718103060 msgr2=0x7fa71819ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:32.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:32.234+0000 7fa71cd77700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa718103060 0x7fa71819ecd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa70800c970 tx=0x7fa70800cc80 comp rx=0 tx=0).stop 2024-04-26T18:24:32.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:32.234+0000 7fa71cd77700 1 -- 172.21.15.84:0/3693460400 shutdown_connections 2024-04-26T18:24:32.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:32.234+0000 7fa71cd77700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa7000a3c40 0x7fa7000a6100 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:32.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:32.234+0000 7fa71cd77700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa7181076b0 0x7fa7181989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:32.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:32.234+0000 7fa71cd77700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa7181026b0 0x7fa71819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:32.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:32.234+0000 7fa71cd77700 1 --2- 172.21.15.84:0/3693460400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa718103060 0x7fa71819ecd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:32.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:32.234+0000 7fa71cd77700 1 -- 172.21.15.84:0/3693460400 >> 172.21.15.84:0/3693460400 conn(0x7fa718078230 msgr2=0x7fa718100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:32.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:32.235+0000 7fa71cd77700 1 -- 172.21.15.84:0/3693460400 shutdown_connections 2024-04-26T18:24:32.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:32.235+0000 7fa71cd77700 1 -- 172.21.15.84:0/3693460400 wait complete. 2024-04-26T18:24:32.558 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:32 smithi084 ceph-mon[29141]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-26T18:24:32.558 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:32 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3693460400' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T18:24:32.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:32 smithi103 ceph-mon[32727]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-26T18:24:32.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:32 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3693460400' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T18:24:32.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:32 smithi174 ceph-mon[32705]: pgmap v302: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-04-26T18:24:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:32 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3693460400' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-26T18:24:32.911 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-26T18:24:32.911 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-04-26T18:24:33.150 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:33.495 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:33 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3693460400' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T18:24:33.495 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:33 smithi084 ceph-mon[29141]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:24:33.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:33 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3693460400' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T18:24:33.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:33 smithi103 ceph-mon[32727]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:24:33.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:33 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3693460400' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-26T18:24:33.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:33 smithi174 ceph-mon[32705]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:24:34.512 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:34 smithi084 ceph-mon[29141]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T18:24:34.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:34 smithi103 ceph-mon[32727]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T18:24:34.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:34 smithi174 ceph-mon[32705]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-04-26T18:24:34.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.865+0000 7f307b6dc700 1 -- 172.21.15.84:0/995012184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3074102f60 msgr2=0x7f3074106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:34.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.865+0000 7f307b6dc700 1 --2- 172.21.15.84:0/995012184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3074102f60 0x7f3074106f60 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f30640099c0 tx=0x7f3064009cd0 comp rx=0 tx=0).stop 2024-04-26T18:24:34.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.865+0000 7f307b6dc700 1 -- 172.21.15.84:0/995012184 shutdown_connections 2024-04-26T18:24:34.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.865+0000 7f307b6dc700 1 --2- 172.21.15.84:0/995012184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3074107640 0x7f307410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:34.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.865+0000 7f307b6dc700 1 --2- 172.21.15.84:0/995012184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30741025b0 0x7f3074102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:34.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.865+0000 7f307b6dc700 1 --2- 172.21.15.84:0/995012184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3074102f60 0x7f3074106f60 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:34.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.865+0000 7f307b6dc700 1 -- 172.21.15.84:0/995012184 >> 172.21.15.84:0/995012184 conn(0x7f30740792d0 msgr2=0x7f30740796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:34.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.865+0000 7f307b6dc700 1 -- 172.21.15.84:0/995012184 shutdown_connections 2024-04-26T18:24:34.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.865+0000 7f307b6dc700 1 -- 172.21.15.84:0/995012184 wait complete. 2024-04-26T18:24:34.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.866+0000 7f307b6dc700 1 Processor -- start 2024-04-26T18:24:34.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.866+0000 7f307b6dc700 1 -- start start 2024-04-26T18:24:34.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.866+0000 7f307b6dc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30741025b0 0x7f307419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:34.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.866+0000 7f307b6dc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3074102f60 0x7f307419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:34.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.866+0000 7f307b6dc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3074107640 0x7f30741999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:34.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.866+0000 7f307b6dc700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3074112530 con 0x7f3074102f60 2024-04-26T18:24:34.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.866+0000 7f307b6dc700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f30741123b0 con 0x7f3074107640 2024-04-26T18:24:34.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.866+0000 7f307b6dc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f30741126b0 con 0x7f30741025b0 2024-04-26T18:24:34.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.866+0000 7f3078c77700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3074102f60 0x7f307419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:34.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.866+0000 7f3078c77700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3074102f60 0x7f307419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54864/0 (socket says 172.21.15.84:54864) 2024-04-26T18:24:34.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.866+0000 7f3078c77700 1 -- 172.21.15.84:0/790473789 learned_addr learned my addr 172.21.15.84:0/790473789 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:34.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.866+0000 7f3079478700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30741025b0 0x7f307419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:34.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.866+0000 7f3079c79700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3074107640 0x7f30741999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:34.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.867+0000 7f3078c77700 1 -- 172.21.15.84:0/790473789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30741025b0 msgr2=0x7f307419f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:34.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.867+0000 7f3078c77700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30741025b0 0x7f307419f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:34.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.867+0000 7f3078c77700 1 -- 172.21.15.84:0/790473789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3074107640 msgr2=0x7f30741999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:34.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.867+0000 7f3078c77700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3074107640 0x7f30741999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:34.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.867+0000 7f3078c77700 1 -- 172.21.15.84:0/790473789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307419a100 con 0x7f3074102f60 2024-04-26T18:24:34.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.867+0000 7f3078c77700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3074102f60 0x7f307419fd20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3064006010 tx=0x7f30640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:34.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.867+0000 7f306a7fc700 1 -- 172.21.15.84:0/790473789 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064027070 con 0x7f3074102f60 2024-04-26T18:24:34.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.867+0000 7f306a7fc700 1 -- 172.21.15.84:0/790473789 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3064003f20 con 0x7f3074102f60 2024-04-26T18:24:34.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.867+0000 7f307b6dc700 1 -- 172.21.15.84:0/790473789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f307419a390 con 0x7f3074102f60 2024-04-26T18:24:34.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.867+0000 7f306a7fc700 1 -- 172.21.15.84:0/790473789 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3064022c60 con 0x7f3074102f60 2024-04-26T18:24:34.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.867+0000 7f307b6dc700 1 -- 172.21.15.84:0/790473789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3074103f30 con 0x7f3074102f60 2024-04-26T18:24:34.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.868+0000 7f307b6dc700 1 -- 172.21.15.84:0/790473789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f307406a110 con 0x7f3074102f60 2024-04-26T18:24:34.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.869+0000 7f306a7fc700 1 -- 172.21.15.84:0/790473789 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f306402a070 con 0x7f3074102f60 2024-04-26T18:24:34.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.870+0000 7f306a7fc700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f30600a3cc0 0x7f30600a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:34.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.870+0000 7f3079478700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f30600a3cc0 0x7f30600a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:34.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.870+0000 7f306a7fc700 1 -- 172.21.15.84:0/790473789 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f3064022e00 con 0x7f3074102f60 2024-04-26T18:24:34.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.870+0000 7f3079478700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f30600a3cc0 0x7f30600a6180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f307419aae0 tx=0x7f307000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:34.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:34.873+0000 7f306a7fc700 1 -- 172.21.15.84:0/790473789 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3064031070 con 0x7f3074102f60 2024-04-26T18:24:35.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.121+0000 7f307b6dc700 1 -- 172.21.15.84:0/790473789 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f307419aed0 con 0x7f3074102f60 2024-04-26T18:24:35.253 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.253+0000 7f306a7fc700 1 -- 172.21.15.84:0/790473789 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v18) ==== 105+0+0 (secure 0 0 0) 0x7f3064013070 con 0x7f3074102f60 2024-04-26T18:24:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.255+0000 7f307b6dc700 1 -- 172.21.15.84:0/790473789 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f30600a3cc0 msgr2=0x7f30600a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.255+0000 7f307b6dc700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f30600a3cc0 0x7f30600a6180 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f307419aae0 tx=0x7f307000b410 comp rx=0 tx=0).stop 2024-04-26T18:24:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.255+0000 7f307b6dc700 1 -- 172.21.15.84:0/790473789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3074102f60 msgr2=0x7f307419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.255+0000 7f307b6dc700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3074102f60 0x7f307419fd20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3064006010 tx=0x7f30640047b0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.255+0000 7f307b6dc700 1 -- 172.21.15.84:0/790473789 shutdown_connections 2024-04-26T18:24:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.255+0000 7f307b6dc700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f30600a3cc0 0x7f30600a6180 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.255+0000 7f307b6dc700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3074107640 0x7f30741999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.255+0000 7f307b6dc700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f30741025b0 0x7f307419f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.255+0000 7f307b6dc700 1 --2- 172.21.15.84:0/790473789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3074102f60 0x7f307419fd20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.255+0000 7f307b6dc700 1 -- 172.21.15.84:0/790473789 >> 172.21.15.84:0/790473789 conn(0x7f30740792d0 msgr2=0x7f307406be70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.256+0000 7f307b6dc700 1 -- 172.21.15.84:0/790473789 shutdown_connections 2024-04-26T18:24:35.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:35.256+0000 7f307b6dc700 1 -- 172.21.15.84:0/790473789 wait complete. 2024-04-26T18:24:35.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:35 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/790473789' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-26T18:24:35.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:35 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/790473789' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-26T18:24:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:35 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/790473789' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-26T18:24:35.969 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-04-26T18:24:36.206 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:36.587 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:36 smithi084 ceph-mon[29141]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T18:24:36.587 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:36 smithi084 ceph-mon[29141]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T18:24:36.587 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:36 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/790473789' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-26T18:24:36.587 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:36 smithi084 ceph-mon[29141]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T18:24:36.587 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:36 smithi084 ceph-mon[29141]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:24:36.587 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:36 smithi084 ceph-mon[29141]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-26T18:24:36.587 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:36 smithi084 ceph-mon[29141]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:24:36.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:36 smithi103 ceph-mon[32727]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T18:24:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:36 smithi103 ceph-mon[32727]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T18:24:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:36 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/790473789' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-26T18:24:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:36 smithi103 ceph-mon[32727]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T18:24:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:36 smithi103 ceph-mon[32727]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:24:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:36 smithi103 ceph-mon[32727]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-26T18:24:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:36 smithi103 ceph-mon[32727]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:24:36.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:36 smithi174 ceph-mon[32705]: pgmap v304: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T18:24:36.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:36 smithi174 ceph-mon[32705]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T18:24:36.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:36 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/790473789' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-26T18:24:36.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:36 smithi174 ceph-mon[32705]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T18:24:36.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:36 smithi174 ceph-mon[32705]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-26T18:24:36.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:36 smithi174 ceph-mon[32705]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-26T18:24:36.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:36 smithi174 ceph-mon[32705]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-26T18:24:37.453 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:37 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] up:active 2024-04-26T18:24:37.453 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:37 smithi084 ceph-mon[29141]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T18:24:37.453 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:37 smithi084 ceph-mon[29141]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-26T18:24:37.453 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:37 smithi084 ceph-mon[29141]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-26T18:24:37.453 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:37 smithi084 ceph-mon[29141]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-26T18:24:37.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:37 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] up:active 2024-04-26T18:24:37.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:37 smithi103 ceph-mon[32727]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T18:24:37.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:37 smithi103 ceph-mon[32727]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-26T18:24:37.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:37 smithi103 ceph-mon[32727]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-26T18:24:37.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:37 smithi103 ceph-mon[32727]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-26T18:24:37.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:37 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] up:active 2024-04-26T18:24:37.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:37 smithi174 ceph-mon[32705]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-26T18:24:37.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:37 smithi174 ceph-mon[32705]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-26T18:24:37.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:37 smithi174 ceph-mon[32705]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-26T18:24:37.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:37 smithi174 ceph-mon[32705]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-26T18:24:37.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.993+0000 7f0f440c1700 1 -- 172.21.15.84:0/1047634318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3c102780 msgr2=0x7f0f3c1101f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:37.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.993+0000 7f0f440c1700 1 --2- 172.21.15.84:0/1047634318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3c102780 0x7f0f3c1101f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0f3400b0a0 tx=0x7f0f3400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:24:37.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.993+0000 7f0f440c1700 1 -- 172.21.15.84:0/1047634318 shutdown_connections 2024-04-26T18:24:37.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.993+0000 7f0f440c1700 1 --2- 172.21.15.84:0/1047634318 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3c102780 0x7f0f3c1101f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:37.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.993+0000 7f0f440c1700 1 --2- 172.21.15.84:0/1047634318 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f3c1084e0 0x7f0f3c1088c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:37.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.993+0000 7f0f440c1700 1 --2- 172.21.15.84:0/1047634318 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f3c101dc0 0x7f0f3c102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:37.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.993+0000 7f0f440c1700 1 -- 172.21.15.84:0/1047634318 >> 172.21.15.84:0/1047634318 conn(0x7f0f3c077640 msgr2=0x7f0f3c077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:37.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.993+0000 7f0f440c1700 1 -- 172.21.15.84:0/1047634318 shutdown_connections 2024-04-26T18:24:37.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.993+0000 7f0f440c1700 1 -- 172.21.15.84:0/1047634318 wait complete. 2024-04-26T18:24:37.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.993+0000 7f0f440c1700 1 Processor -- start 2024-04-26T18:24:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.993+0000 7f0f440c1700 1 -- start start 2024-04-26T18:24:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f440c1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3c101dc0 0x7f0f3c102fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f440c1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f3c102780 0x7f0f3c1034e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f440c1700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f3c1084e0 0x7f0f3c103a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f440c1700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c1135f0 con 0x7f0f3c101dc0 2024-04-26T18:24:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f440c1700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c113470 con 0x7f0f3c1084e0 2024-04-26T18:24:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f440c1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0f3c113770 con 0x7f0f3c102780 2024-04-26T18:24:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f41e5d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3c101dc0 0x7f0f3c102fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f41e5d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3c101dc0 0x7f0f3c102fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59020/0 (socket says 172.21.15.84:59020) 2024-04-26T18:24:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f41e5d700 1 -- 172.21.15.84:0/2343698398 learned_addr learned my addr 172.21.15.84:0/2343698398 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f4165c700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f3c102780 0x7f0f3c1034e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:37.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f4265e700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f3c1084e0 0x7f0f3c103a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f41e5d700 1 -- 172.21.15.84:0/2343698398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f3c102780 msgr2=0x7f0f3c1034e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f41e5d700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f3c102780 0x7f0f3c1034e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f41e5d700 1 -- 172.21.15.84:0/2343698398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f3c1084e0 msgr2=0x7f0f3c103a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f41e5d700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f3c1084e0 0x7f0f3c103a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f41e5d700 1 -- 172.21.15.84:0/2343698398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f3c1a7f90 con 0x7f0f3c101dc0 2024-04-26T18:24:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f41e5d700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3c101dc0 0x7f0f3c102fa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800d980 tx=0x7f0f3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f32ffd700 1 -- 172.21.15.84:0/2343698398 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f38017070 con 0x7f0f3c101dc0 2024-04-26T18:24:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f32ffd700 1 -- 172.21.15.84:0/2343698398 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f3800fd40 con 0x7f0f3c101dc0 2024-04-26T18:24:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f440c1700 1 -- 172.21.15.84:0/2343698398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f3c1a8280 con 0x7f0f3c101dc0 2024-04-26T18:24:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f32ffd700 1 -- 172.21.15.84:0/2343698398 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f3800e3b0 con 0x7f0f3c101dc0 2024-04-26T18:24:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.994+0000 7f0f440c1700 1 -- 172.21.15.84:0/2343698398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f3c1a8770 con 0x7f0f3c101dc0 2024-04-26T18:24:37.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.995+0000 7f0f440c1700 1 -- 172.21.15.84:0/2343698398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f3c06a110 con 0x7f0f3c101dc0 2024-04-26T18:24:37.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.997+0000 7f0f32ffd700 1 -- 172.21.15.84:0/2343698398 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f0f3800f890 con 0x7f0f3c101dc0 2024-04-26T18:24:37.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.998+0000 7f0f32ffd700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0f280a3c40 0x7f0f280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:37.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.998+0000 7f0f4165c700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0f280a3c40 0x7f0f280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:37.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.998+0000 7f0f32ffd700 1 -- 172.21.15.84:0/2343698398 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f0f380cfce0 con 0x7f0f3c101dc0 2024-04-26T18:24:37.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:37.998+0000 7f0f4165c700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0f280a3c40 0x7f0f280a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c077c40 tx=0x7f0f2c009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:38.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.001+0000 7f0f32ffd700 1 -- 172.21.15.84:0/2343698398 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f3809c810 con 0x7f0f3c101dc0 2024-04-26T18:24:38.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.244+0000 7f0f440c1700 1 -- 172.21.15.84:0/2343698398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f0f3c02e6b0 con 0x7f0f3c101dc0 2024-04-26T18:24:38.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.285+0000 7f0f32ffd700 1 -- 172.21.15.84:0/2343698398 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v24) ==== 121+0+0 (secure 0 0 0) 0x7f0f3809c110 con 0x7f0f3c101dc0 2024-04-26T18:24:38.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.287+0000 7f0f440c1700 1 -- 172.21.15.84:0/2343698398 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0f280a3c40 msgr2=0x7f0f280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:38.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.287+0000 7f0f440c1700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0f280a3c40 0x7f0f280a6100 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c077c40 tx=0x7f0f2c009260 comp rx=0 tx=0).stop 2024-04-26T18:24:38.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.287+0000 7f0f440c1700 1 -- 172.21.15.84:0/2343698398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3c101dc0 msgr2=0x7f0f3c102fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:38.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.287+0000 7f0f440c1700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3c101dc0 0x7f0f3c102fa0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800d980 tx=0x7f0f3800dc90 comp rx=0 tx=0).stop 2024-04-26T18:24:38.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.287+0000 7f0f440c1700 1 -- 172.21.15.84:0/2343698398 shutdown_connections 2024-04-26T18:24:38.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.287+0000 7f0f440c1700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0f280a3c40 0x7f0f280a6100 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.287+0000 7f0f440c1700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f3c101dc0 0x7f0f3c102fa0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.287+0000 7f0f440c1700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f3c1084e0 0x7f0f3c103a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.287+0000 7f0f440c1700 1 --2- 172.21.15.84:0/2343698398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f3c102780 0x7f0f3c1034e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:38.288 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.287+0000 7f0f440c1700 1 -- 172.21.15.84:0/2343698398 >> 172.21.15.84:0/2343698398 conn(0x7f0f3c077640 msgr2=0x7f0f3c1009b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:38.288 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.287+0000 7f0f440c1700 1 -- 172.21.15.84:0/2343698398 shutdown_connections 2024-04-26T18:24:38.288 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:38.287+0000 7f0f440c1700 1 -- 172.21.15.84:0/2343698398 wait complete. 2024-04-26T18:24:38.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:38 smithi084 ceph-mon[29141]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T18:24:38.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:38 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] up:active 2024-04-26T18:24:38.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:38 smithi084 ceph-mon[29141]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-26T18:24:38.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:38 smithi084 ceph-mon[29141]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-26T18:24:38.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:38 smithi084 ceph-mon[29141]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-04-26T18:24:38.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:38 smithi084 ceph-mon[29141]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-26T18:24:38.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:38 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2343698398' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T18:24:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:38 smithi103 ceph-mon[32727]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T18:24:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:38 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] up:active 2024-04-26T18:24:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:38 smithi103 ceph-mon[32727]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-26T18:24:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:38 smithi103 ceph-mon[32727]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-26T18:24:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:38 smithi103 ceph-mon[32727]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-04-26T18:24:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:38 smithi103 ceph-mon[32727]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-26T18:24:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:38 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2343698398' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T18:24:38.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:38 smithi174 ceph-mon[32705]: pgmap v305: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-04-26T18:24:38.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:38 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] up:active 2024-04-26T18:24:38.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:38 smithi174 ceph-mon[32705]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-26T18:24:38.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:38 smithi174 ceph-mon[32705]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 9 up:standby 2024-04-26T18:24:38.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:38 smithi174 ceph-mon[32705]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:creating} 8 up:standby 2024-04-26T18:24:38.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:38 smithi174 ceph-mon[32705]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-04-26T18:24:38.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:38 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2343698398' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-26T18:24:39.118 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-04-26T18:24:39.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:39 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2343698398' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T18:24:39.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:39 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] up:active 2024-04-26T18:24:39.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:39 smithi084 ceph-mon[29141]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-26T18:24:39.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:39 smithi084 ceph-mon[29141]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T18:24:39.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:39 smithi084 ceph-mon[29141]: Cluster is now healthy 2024-04-26T18:24:39.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:39 smithi084 ceph-mon[29141]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-04-26T18:24:39.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:39 smithi084 ceph-mon[29141]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-26T18:24:39.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:39 smithi084 ceph-mon[29141]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-04-26T18:24:39.362 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:39.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:39 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2343698398' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T18:24:39.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:39 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] up:active 2024-04-26T18:24:39.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:39 smithi103 ceph-mon[32727]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-26T18:24:39.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:39 smithi103 ceph-mon[32727]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T18:24:39.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:39 smithi103 ceph-mon[32727]: Cluster is now healthy 2024-04-26T18:24:39.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:39 smithi103 ceph-mon[32727]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-04-26T18:24:39.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:39 smithi103 ceph-mon[32727]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-26T18:24:39.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:39 smithi103 ceph-mon[32727]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-04-26T18:24:39.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:39 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2343698398' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-26T18:24:39.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:39 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] up:active 2024-04-26T18:24:39.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:39 smithi174 ceph-mon[32705]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-26T18:24:39.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:39 smithi174 ceph-mon[32705]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T18:24:39.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:39 smithi174 ceph-mon[32705]: Cluster is now healthy 2024-04-26T18:24:39.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:39 smithi174 ceph-mon[32705]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=c=up:active} 8 up:standby 2024-04-26T18:24:39.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:39 smithi174 ceph-mon[32705]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-04-26T18:24:39.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:39 smithi174 ceph-mon[32705]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-04-26T18:24:40.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:40 smithi084 ceph-mon[29141]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 8 op/s 2024-04-26T18:24:40.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:40 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] up:active 2024-04-26T18:24:40.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:40 smithi084 ceph-mon[29141]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-26T18:24:40.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:40 smithi084 ceph-mon[29141]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T18:24:40.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:40 smithi103 ceph-mon[32727]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 8 op/s 2024-04-26T18:24:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:40 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] up:active 2024-04-26T18:24:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:40 smithi103 ceph-mon[32727]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-26T18:24:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:40 smithi103 ceph-mon[32727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T18:24:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:40 smithi174 ceph-mon[32705]: pgmap v306: 97 pgs: 97 active+clean; 579 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s wr, 8 op/s 2024-04-26T18:24:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:40 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] up:active 2024-04-26T18:24:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:40 smithi174 ceph-mon[32705]: fsmap cephfs:5 7 up:standby 5 up:active 2024-04-26T18:24:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:40 smithi174 ceph-mon[32705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T18:24:41.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.170+0000 7fe80b8f7700 1 -- 172.21.15.84:0/2883134699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe804102fc0 msgr2=0x7fe80410f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:41.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.170+0000 7fe80b8f7700 1 --2- 172.21.15.84:0/2883134699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe804102fc0 0x7fe80410f1b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc00b0a0 tx=0x7fe7fc00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.170+0000 7fe80b8f7700 1 -- 172.21.15.84:0/2883134699 shutdown_connections 2024-04-26T18:24:41.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.170+0000 7fe80b8f7700 1 --2- 172.21.15.84:0/2883134699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe804102fc0 0x7fe80410f1b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.170+0000 7fe80b8f7700 1 --2- 172.21.15.84:0/2883134699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe804108470 0x7fe804108850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.170+0000 7fe80b8f7700 1 --2- 172.21.15.84:0/2883134699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe804102600 0x7fe804102a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.170+0000 7fe80b8f7700 1 -- 172.21.15.84:0/2883134699 >> 172.21.15.84:0/2883134699 conn(0x7fe804079210 msgr2=0x7fe804079620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:41.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.170+0000 7fe80b8f7700 1 -- 172.21.15.84:0/2883134699 shutdown_connections 2024-04-26T18:24:41.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.170+0000 7fe80b8f7700 1 -- 172.21.15.84:0/2883134699 wait complete. 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.170+0000 7fe80b8f7700 1 Processor -- start 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe80b8f7700 1 -- start start 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe80b8f7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe804102600 0x7fe80419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe80b8f7700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe804102fc0 0x7fe80419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe80b8f7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe804108470 0x7fe8041998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe80b8f7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe8041125c0 con 0x7fe804108470 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe80b8f7700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe804112440 con 0x7fe804102fc0 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe80b8f7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe804112740 con 0x7fe804102600 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe809e94700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe804108470 0x7fe8041998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe809e94700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe804108470 0x7fe8041998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59026/0 (socket says 172.21.15.84:59026) 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe809e94700 1 -- 172.21.15.84:0/3465011942 learned_addr learned my addr 172.21.15.84:0/3465011942 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe809693700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe804102600 0x7fe80419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe808e92700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe804102fc0 0x7fe80419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe809e94700 1 -- 172.21.15.84:0/3465011942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe804102600 msgr2=0x7fe80419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:41.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe809e94700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe804102600 0x7fe80419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe809e94700 1 -- 172.21.15.84:0/3465011942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe804102fc0 msgr2=0x7fe80419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:41.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe809e94700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe804102fc0 0x7fe80419fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:41.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe809e94700 1 -- 172.21.15.84:0/3465011942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe80419a010 con 0x7fe804108470 2024-04-26T18:24:41.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe809693700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe804102600 0x7fe80419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:41.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe808e92700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe804102fc0 0x7fe80419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:41.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.171+0000 7fe809e94700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe804108470 0x7fe8041998b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0062a0 tx=0x7fe7fc004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:41.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.172+0000 7fe7fa7fc700 1 -- 172.21.15.84:0/3465011942 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc021070 con 0x7fe804108470 2024-04-26T18:24:41.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.172+0000 7fe7fa7fc700 1 -- 172.21.15.84:0/3465011942 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7fc007d00 con 0x7fe804108470 2024-04-26T18:24:41.174 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.172+0000 7fe80b8f7700 1 -- 172.21.15.84:0/3465011942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe80419a2a0 con 0x7fe804108470 2024-04-26T18:24:41.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.172+0000 7fe7fa7fc700 1 -- 172.21.15.84:0/3465011942 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7fc01d450 con 0x7fe804108470 2024-04-26T18:24:41.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.172+0000 7fe80b8f7700 1 -- 172.21.15.84:0/3465011942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8041a6570 con 0x7fe804108470 2024-04-26T18:24:41.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.173+0000 7fe80b8f7700 1 -- 172.21.15.84:0/3465011942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe80406a110 con 0x7fe804108470 2024-04-26T18:24:41.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.174+0000 7fe7fa7fc700 1 -- 172.21.15.84:0/3465011942 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe7fc004000 con 0x7fe804108470 2024-04-26T18:24:41.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.175+0000 7fe7fa7fc700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe7f00a3cc0 0x7fe7f00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:41.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.175+0000 7fe809693700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe7f00a3cc0 0x7fe7f00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:41.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.175+0000 7fe7fa7fc700 1 -- 172.21.15.84:0/3465011942 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fe7fc00f020 con 0x7fe804108470 2024-04-26T18:24:41.176 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.175+0000 7fe809693700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe7f00a3cc0 0x7fe7f00a6180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe804100cd0 tx=0x7fe80000a680 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:41.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.179+0000 7fe7fa7fc700 1 -- 172.21.15.84:0/3465011942 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe7fc02a020 con 0x7fe804108470 2024-04-26T18:24:41.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:41.424+0000 7fe80b8f7700 1 -- 172.21.15.84:0/3465011942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fe8041a6740 con 0x7fe804108470 2024-04-26T18:24:42.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:42.295+0000 7fe7fa7fc700 1 -- 172.21.15.84:0/3465011942 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7fe7fc09e940 con 0x7fe804108470 2024-04-26T18:24:42.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:42.298+0000 7fe80b8f7700 1 -- 172.21.15.84:0/3465011942 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe7f00a3cc0 msgr2=0x7fe7f00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:42.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:42.298+0000 7fe80b8f7700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe7f00a3cc0 0x7fe7f00a6180 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe804100cd0 tx=0x7fe80000a680 comp rx=0 tx=0).stop 2024-04-26T18:24:42.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:42.298+0000 7fe80b8f7700 1 -- 172.21.15.84:0/3465011942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe804108470 msgr2=0x7fe8041998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:42.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:42.298+0000 7fe80b8f7700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe804108470 0x7fe8041998b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe7fc0062a0 tx=0x7fe7fc004ed0 comp rx=0 tx=0).stop 2024-04-26T18:24:42.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:42.298+0000 7fe80b8f7700 1 -- 172.21.15.84:0/3465011942 shutdown_connections 2024-04-26T18:24:42.299 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:42.298+0000 7fe80b8f7700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe7f00a3cc0 0x7fe7f00a6180 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:42.299 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:42.298+0000 7fe80b8f7700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe804108470 0x7fe8041998b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:42.299 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:42.298+0000 7fe80b8f7700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe804102600 0x7fe80419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:42.299 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:42.298+0000 7fe80b8f7700 1 --2- 172.21.15.84:0/3465011942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe804102fc0 0x7fe80419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:42.299 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:42.298+0000 7fe80b8f7700 1 -- 172.21.15.84:0/3465011942 >> 172.21.15.84:0/3465011942 conn(0x7fe804079210 msgr2=0x7fe804101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:42.299 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:42.298+0000 7fe80b8f7700 1 -- 172.21.15.84:0/3465011942 shutdown_connections 2024-04-26T18:24:42.299 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:42.298+0000 7fe80b8f7700 1 -- 172.21.15.84:0/3465011942 wait complete. 2024-04-26T18:24:42.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:42 smithi084 ceph-mon[29141]: pgmap v307: 97 pgs: 97 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-04-26T18:24:42.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:42 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3465011942' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T18:24:42.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:42 smithi103 ceph-mon[32727]: pgmap v307: 97 pgs: 97 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-04-26T18:24:42.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:42 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3465011942' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T18:24:42.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:42 smithi174 ceph-mon[32705]: pgmap v307: 97 pgs: 97 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 3.7 KiB/s wr, 14 op/s 2024-04-26T18:24:42.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:42 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3465011942' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-26T18:24:43.173 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-26T18:24:43.174 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-04-26T18:24:43.420 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:43.542 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:43 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3465011942' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T18:24:43.542 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:43 smithi084 ceph-mon[29141]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T18:24:43.542 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:43 smithi084 ceph-mon[29141]: pgmap v308: 97 pgs: 97 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.9 KiB/s wr, 16 op/s 2024-04-26T18:24:43.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:43 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3465011942' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T18:24:43.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:43 smithi103 ceph-mon[32727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T18:24:43.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:43 smithi103 ceph-mon[32727]: pgmap v308: 97 pgs: 97 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.9 KiB/s wr, 16 op/s 2024-04-26T18:24:43.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:43 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3465011942' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-26T18:24:43.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:43 smithi174 ceph-mon[32705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T18:24:43.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:43 smithi174 ceph-mon[32705]: pgmap v308: 97 pgs: 97 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 3.9 KiB/s wr, 16 op/s 2024-04-26T18:24:45.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.207+0000 7f1e723e7700 1 -- 172.21.15.84:0/3493608081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c1094f0 msgr2=0x7f1e6c1098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:45.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.207+0000 7f1e723e7700 1 --2- 172.21.15.84:0/3493608081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c1094f0 0x7f1e6c1098d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c009990 tx=0x7f1e5c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:45.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.207+0000 7f1e723e7700 1 -- 172.21.15.84:0/3493608081 shutdown_connections 2024-04-26T18:24:45.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.207+0000 7f1e723e7700 1 --2- 172.21.15.84:0/3493608081 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c1094f0 0x7f1e6c1098d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:45.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.207+0000 7f1e723e7700 1 --2- 172.21.15.84:0/3493608081 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c103e30 0x7f1e6c1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:45.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.207+0000 7f1e723e7700 1 --2- 172.21.15.84:0/3493608081 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e6c103470 0x7f1e6c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:45.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.207+0000 7f1e723e7700 1 -- 172.21.15.84:0/3493608081 >> 172.21.15.84:0/3493608081 conn(0x7f1e6c07a5d0 msgr2=0x7f1e6c07a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:45.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.207+0000 7f1e723e7700 1 -- 172.21.15.84:0/3493608081 shutdown_connections 2024-04-26T18:24:45.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.207+0000 7f1e723e7700 1 -- 172.21.15.84:0/3493608081 wait complete. 2024-04-26T18:24:45.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e723e7700 1 Processor -- start 2024-04-26T18:24:45.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e723e7700 1 -- start start 2024-04-26T18:24:45.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e723e7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c103470 0x7f1e6c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:45.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e723e7700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c103e30 0x7f1e6c19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:45.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e723e7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e6c1094f0 0x7f1e6c1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:45.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e723e7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c1135f0 con 0x7f1e6c103470 2024-04-26T18:24:45.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e723e7700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c113470 con 0x7f1e6c103e30 2024-04-26T18:24:45.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e723e7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1e6c113770 con 0x7f1e6c1094f0 2024-04-26T18:24:45.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e6bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c103470 0x7f1e6c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:45.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e6bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c103470 0x7f1e6c19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59040/0 (socket says 172.21.15.84:59040) 2024-04-26T18:24:45.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e6bfff700 1 -- 172.21.15.84:0/2959102518 learned_addr learned my addr 172.21.15.84:0/2959102518 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:45.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e63fff700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c103e30 0x7f1e6c19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:45.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e70984700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e6c1094f0 0x7f1e6c1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:45.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e6bfff700 1 -- 172.21.15.84:0/2959102518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e6c1094f0 msgr2=0x7f1e6c1997f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:45.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e6bfff700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e6c1094f0 0x7f1e6c1997f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:45.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e6bfff700 1 -- 172.21.15.84:0/2959102518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c103e30 msgr2=0x7f1e6c19fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:45.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e6bfff700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c103e30 0x7f1e6c19fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:45.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e6bfff700 1 -- 172.21.15.84:0/2959102518 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e6c199fd0 con 0x7f1e6c103470 2024-04-26T18:24:45.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e70984700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e6c1094f0 0x7f1e6c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:45.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.208+0000 7f1e63fff700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c103e30 0x7f1e6c19fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:24:45.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.209+0000 7f1e6bfff700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c103470 0x7f1e6c19f670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0057f0 tx=0x7f1e5c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:45.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.209+0000 7f1e69ffb700 1 -- 172.21.15.84:0/2959102518 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c027070 con 0x7f1e6c103470 2024-04-26T18:24:45.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.209+0000 7f1e723e7700 1 -- 172.21.15.84:0/2959102518 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e6c19a260 con 0x7f1e6c103470 2024-04-26T18:24:45.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.209+0000 7f1e69ffb700 1 -- 172.21.15.84:0/2959102518 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e5c0041c0 con 0x7f1e6c103470 2024-04-26T18:24:45.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.209+0000 7f1e69ffb700 1 -- 172.21.15.84:0/2959102518 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c00f460 con 0x7f1e6c103470 2024-04-26T18:24:45.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.209+0000 7f1e723e7700 1 -- 172.21.15.84:0/2959102518 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e6c1a64d0 con 0x7f1e6c103470 2024-04-26T18:24:45.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.210+0000 7f1e723e7700 1 -- 172.21.15.84:0/2959102518 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e6c104b90 con 0x7f1e6c103470 2024-04-26T18:24:45.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.211+0000 7f1e69ffb700 1 -- 172.21.15.84:0/2959102518 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1e5c02a070 con 0x7f1e6c103470 2024-04-26T18:24:45.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.211+0000 7f1e69ffb700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1e580a3c40 0x7f1e580a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:45.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.212+0000 7f1e63fff700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1e580a3c40 0x7f1e580a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:45.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.212+0000 7f1e69ffb700 1 -- 172.21.15.84:0/2959102518 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f1e5c0d1a30 con 0x7f1e6c103470 2024-04-26T18:24:45.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.212+0000 7f1e63fff700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1e580a3c40 0x7f1e580a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1e54005fd0 tx=0x7f1e54005e30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:45.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.215+0000 7f1e69ffb700 1 -- 172.21.15.84:0/2959102518 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e5c031020 con 0x7f1e6c103470 2024-04-26T18:24:45.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.451+0000 7f1e723e7700 1 -- 172.21.15.84:0/2959102518 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f1e6c063860 con 0x7f1e580a3c40 2024-04-26T18:24:45.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.480+0000 7f1e69ffb700 1 -- 172.21.15.84:0/2959102518 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1e6c063860 con 0x7f1e580a3c40 2024-04-26T18:24:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.482+0000 7f1e723e7700 1 -- 172.21.15.84:0/2959102518 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1e580a3c40 msgr2=0x7f1e580a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.482+0000 7f1e723e7700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1e580a3c40 0x7f1e580a6100 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1e54005fd0 tx=0x7f1e54005e30 comp rx=0 tx=0).stop 2024-04-26T18:24:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.482+0000 7f1e723e7700 1 -- 172.21.15.84:0/2959102518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c103470 msgr2=0x7f1e6c19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.482+0000 7f1e723e7700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c103470 0x7f1e6c19f670 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c0057f0 tx=0x7f1e5c003730 comp rx=0 tx=0).stop 2024-04-26T18:24:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.482+0000 7f1e723e7700 1 -- 172.21.15.84:0/2959102518 shutdown_connections 2024-04-26T18:24:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.482+0000 7f1e723e7700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1e580a3c40 0x7f1e580a6100 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.482+0000 7f1e723e7700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1e6c103470 0x7f1e6c19f670 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.482+0000 7f1e723e7700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1e6c1094f0 0x7f1e6c1997f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.482+0000 7f1e723e7700 1 --2- 172.21.15.84:0/2959102518 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1e6c103e30 0x7f1e6c19fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.482+0000 7f1e723e7700 1 -- 172.21.15.84:0/2959102518 >> 172.21.15.84:0/2959102518 conn(0x7f1e6c07a5d0 msgr2=0x7f1e6c101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.482+0000 7f1e723e7700 1 -- 172.21.15.84:0/2959102518 shutdown_connections 2024-04-26T18:24:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:45.482+0000 7f1e723e7700 1 -- 172.21.15.84:0/2959102518 wait complete. 2024-04-26T18:24:45.844 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:24:45 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:24:45.456+0000 7f905fb4d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:45.844 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:24:45 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:24:45.456+0000 7f905fb4d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:45.845 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:24:45 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:24:45.456+0000 7f905fb4d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:45.845 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:24:45 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:24:45.456+0000 7f905fb4d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:45.845 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:24:45 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:24:45.456+0000 7f905fb4d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:24:46.220 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-26T18:24:46.462 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:46.584 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:46 smithi084 ceph-mon[29141]: pgmap v309: 97 pgs: 97 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-04-26T18:24:46.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:46 smithi103 ceph-mon[32727]: pgmap v309: 97 pgs: 97 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-04-26T18:24:46.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:46 smithi174 ceph-mon[32705]: pgmap v309: 97 pgs: 97 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 4.0 KiB/s wr, 18 op/s 2024-04-26T18:24:47.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:47 smithi084 ceph-mon[29141]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:47.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:47 smithi103 ceph-mon[32727]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:47.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:47 smithi174 ceph-mon[32705]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.273+0000 7f674ded0700 1 -- 172.21.15.84:0/1720214933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67481094f0 msgr2=0x7f67481098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.273+0000 7f674ded0700 1 --2- 172.21.15.84:0/1720214933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67481094f0 0x7f67481098d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6738009990 tx=0x7f6738009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.273+0000 7f674ded0700 1 -- 172.21.15.84:0/1720214933 shutdown_connections 2024-04-26T18:24:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.273+0000 7f674ded0700 1 --2- 172.21.15.84:0/1720214933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67481094f0 0x7f67481098d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.273+0000 7f674ded0700 1 --2- 172.21.15.84:0/1720214933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6748103e90 0x7f67481102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.273+0000 7f674ded0700 1 --2- 172.21.15.84:0/1720214933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67481034d0 0x7f6748103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.273+0000 7f674ded0700 1 -- 172.21.15.84:0/1720214933 >> 172.21.15.84:0/1720214933 conn(0x7f674807a500 msgr2=0x7f674807a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.273+0000 7f674ded0700 1 -- 172.21.15.84:0/1720214933 shutdown_connections 2024-04-26T18:24:48.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.273+0000 7f674ded0700 1 -- 172.21.15.84:0/1720214933 wait complete. 2024-04-26T18:24:48.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.274+0000 7f674ded0700 1 Processor -- start 2024-04-26T18:24:48.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.274+0000 7f674ded0700 1 -- start start 2024-04-26T18:24:48.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.274+0000 7f674ded0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67481034d0 0x7f6748075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.274+0000 7f674ded0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6748103e90 0x7f6748075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.274+0000 7f674ded0700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67481094f0 0x7f67480761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.274+0000 7f674ded0700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6748113620 con 0x7f6748103e90 2024-04-26T18:24:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.274+0000 7f674ded0700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f67481134a0 con 0x7f67481094f0 2024-04-26T18:24:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.274+0000 7f674ded0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f67481137a0 con 0x7f67481034d0 2024-04-26T18:24:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.274+0000 7f6746ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6748103e90 0x7f6748075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.274+0000 7f6746ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6748103e90 0x7f6748075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49172/0 (socket says 172.21.15.84:49172) 2024-04-26T18:24:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f6746ffd700 1 -- 172.21.15.84:0/3017075465 learned_addr learned my addr 172.21.15.84:0/3017075465 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f67477fe700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67481034d0 0x7f6748075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.276 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f6747fff700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67481094f0 0x7f67480761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.277 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:48 smithi084 ceph-mon[29141]: pgmap v310: 97 pgs: 97 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f6746ffd700 1 -- 172.21.15.84:0/3017075465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67481034d0 msgr2=0x7f6748075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f6746ffd700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67481034d0 0x7f6748075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f6746ffd700 1 -- 172.21.15.84:0/3017075465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67481094f0 msgr2=0x7f67480761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f6746ffd700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67481094f0 0x7f67480761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f6746ffd700 1 -- 172.21.15.84:0/3017075465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67481a7f50 con 0x7f6748103e90 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f67477fe700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67481034d0 0x7f6748075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f6746ffd700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6748103e90 0x7f6748075c60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f673c00c9b0 tx=0x7f673c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f6744ff9700 1 -- 172.21.15.84:0/3017075465 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c007b00 con 0x7f6748103e90 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f6744ff9700 1 -- 172.21.15.84:0/3017075465 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f673c00ce50 con 0x7f6748103e90 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f6744ff9700 1 -- 172.21.15.84:0/3017075465 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f673c018660 con 0x7f6748103e90 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.275+0000 7f674ded0700 1 -- 172.21.15.84:0/3017075465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67481a8240 con 0x7f6748103e90 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.276+0000 7f674ded0700 1 -- 172.21.15.84:0/3017075465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67481a8730 con 0x7f6748103e90 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.277+0000 7f674ded0700 1 -- 172.21.15.84:0/3017075465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f674810b960 con 0x7f6748103e90 2024-04-26T18:24:48.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.277+0000 7f6744ff9700 1 -- 172.21.15.84:0/3017075465 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f673c00f450 con 0x7f6748103e90 2024-04-26T18:24:48.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.278+0000 7f6744ff9700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f67300a3d50 0x7f67300a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:48.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.278+0000 7f67477fe700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f67300a3d50 0x7f67300a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:48.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.278+0000 7f6744ff9700 1 -- 172.21.15.84:0/3017075465 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f673c0d0270 con 0x7f6748103e90 2024-04-26T18:24:48.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.279+0000 7f67477fe700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f67300a3d50 0x7f67300a6210 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f673800b5c0 tx=0x7f6738005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:48.282 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.283+0000 7f6744ff9700 1 -- 172.21.15.84:0/3017075465 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f673c014030 con 0x7f6748103e90 2024-04-26T18:24:48.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.515+0000 7f674ded0700 1 -- 172.21.15.84:0/3017075465 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f6748063860 con 0x7f67300a3d50 2024-04-26T18:24:48.542 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.542+0000 7f6744ff9700 1 -- 172.21.15.84:0/3017075465 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6748063860 con 0x7f67300a3d50 2024-04-26T18:24:48.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.544+0000 7f674ded0700 1 -- 172.21.15.84:0/3017075465 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f67300a3d50 msgr2=0x7f67300a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.544+0000 7f674ded0700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f67300a3d50 0x7f67300a6210 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f673800b5c0 tx=0x7f6738005960 comp rx=0 tx=0).stop 2024-04-26T18:24:48.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.544+0000 7f674ded0700 1 -- 172.21.15.84:0/3017075465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6748103e90 msgr2=0x7f6748075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:48.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.544+0000 7f674ded0700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6748103e90 0x7f6748075c60 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f673c00c9b0 tx=0x7f673c00ccc0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.544+0000 7f674ded0700 1 -- 172.21.15.84:0/3017075465 shutdown_connections 2024-04-26T18:24:48.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.544+0000 7f674ded0700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f67300a3d50 0x7f67300a6210 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.544+0000 7f674ded0700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67481094f0 0x7f67480761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.544+0000 7f674ded0700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67481034d0 0x7f6748075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.544+0000 7f674ded0700 1 --2- 172.21.15.84:0/3017075465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6748103e90 0x7f6748075c60 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:48.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.544+0000 7f674ded0700 1 -- 172.21.15.84:0/3017075465 >> 172.21.15.84:0/3017075465 conn(0x7f674807a500 msgr2=0x7f67481020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:48.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.544+0000 7f674ded0700 1 -- 172.21.15.84:0/3017075465 shutdown_connections 2024-04-26T18:24:48.545 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:48.545+0000 7f674ded0700 1 -- 172.21.15.84:0/3017075465 wait complete. 2024-04-26T18:24:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:48 smithi103 ceph-mon[32727]: pgmap v310: 97 pgs: 97 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-04-26T18:24:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:48 smithi174 ceph-mon[32705]: pgmap v310: 97 pgs: 97 active+clean; 584 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 4.0 KiB/s wr, 20 op/s 2024-04-26T18:24:49.313 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-26T18:24:49.553 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:49.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:49 smithi084 ceph-mon[29141]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-26T18:24:49.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:49 smithi103 ceph-mon[32727]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-26T18:24:49.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:49 smithi174 ceph-mon[32705]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-26T18:24:50.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:50 smithi084 ceph-mon[29141]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:50.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:50 smithi084 ceph-mon[29141]: pgmap v311: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.6 KiB/s wr, 22 op/s 2024-04-26T18:24:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:50 smithi103 ceph-mon[32727]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:50 smithi103 ceph-mon[32727]: pgmap v311: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.6 KiB/s wr, 22 op/s 2024-04-26T18:24:50.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:50 smithi174 ceph-mon[32705]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:50.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:50 smithi174 ceph-mon[32705]: pgmap v311: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 5.6 KiB/s wr, 22 op/s 2024-04-26T18:24:51.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.313+0000 7f90b9778700 1 -- 172.21.15.84:0/2123761047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b4108380 msgr2=0x7f90b4108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.313+0000 7f90b9778700 1 --2- 172.21.15.84:0/2123761047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b4108380 0x7f90b4108760 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00b0a0 tx=0x7f90ac00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.313+0000 7f90b9778700 1 -- 172.21.15.84:0/2123761047 shutdown_connections 2024-04-26T18:24:51.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.313+0000 7f90b9778700 1 --2- 172.21.15.84:0/2123761047 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b4108380 0x7f90b4108760 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.313+0000 7f90b9778700 1 --2- 172.21.15.84:0/2123761047 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90b4102cd0 0x7f90b410f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.313+0000 7f90b9778700 1 --2- 172.21.15.84:0/2123761047 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b4102310 0x7f90b4102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.313+0000 7f90b9778700 1 -- 172.21.15.84:0/2123761047 >> 172.21.15.84:0/2123761047 conn(0x7f90b4079290 msgr2=0x7f90b40796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.313+0000 7f90b9778700 1 -- 172.21.15.84:0/2123761047 shutdown_connections 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.313+0000 7f90b9778700 1 -- 172.21.15.84:0/2123761047 wait complete. 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.313+0000 7f90b9778700 1 Processor -- start 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b9778700 1 -- start start 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b9778700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90b4102310 0x7f90b41987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b9778700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b4102cd0 0x7f90b4198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b9778700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b4108380 0x7f90b4199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b9778700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f90b4112520 con 0x7f90b4102cd0 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b9778700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41123a0 con 0x7f90b4102310 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b9778700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f90b41126a0 con 0x7f90b4108380 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b27fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b4102cd0 0x7f90b4198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b27fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b4102cd0 0x7f90b4198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49176/0 (socket says 172.21.15.84:49176) 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b27fc700 1 -- 172.21.15.84:0/4116468758 learned_addr learned my addr 172.21.15.84:0/4116468758 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b2ffd700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90b4102310 0x7f90b41987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b37fe700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b4108380 0x7f90b4199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b27fc700 1 -- 172.21.15.84:0/4116468758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b4108380 msgr2=0x7f90b4199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b27fc700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b4108380 0x7f90b4199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b27fc700 1 -- 172.21.15.84:0/4116468758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90b4102310 msgr2=0x7f90b41987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b27fc700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90b4102310 0x7f90b41987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b27fc700 1 -- 172.21.15.84:0/4116468758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90b41a8040 con 0x7f90b4102cd0 2024-04-26T18:24:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.314+0000 7f90b27fc700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b4102cd0 0x7f90b4198d10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f909c00b530 tx=0x7f909c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.315+0000 7f90abfff700 1 -- 172.21.15.84:0/4116468758 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c015020 con 0x7f90b4102cd0 2024-04-26T18:24:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.315+0000 7f90b9778700 1 -- 172.21.15.84:0/4116468758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90b41a8330 con 0x7f90b4102cd0 2024-04-26T18:24:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.315+0000 7f90b9778700 1 -- 172.21.15.84:0/4116468758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90b41a88c0 con 0x7f90b4102cd0 2024-04-26T18:24:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.315+0000 7f90abfff700 1 -- 172.21.15.84:0/4116468758 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f909c011ce0 con 0x7f90b4102cd0 2024-04-26T18:24:51.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.315+0000 7f90abfff700 1 -- 172.21.15.84:0/4116468758 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f909c01adf0 con 0x7f90b4102cd0 2024-04-26T18:24:51.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.316+0000 7f90b9778700 1 -- 172.21.15.84:0/4116468758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9094005330 con 0x7f90b4102cd0 2024-04-26T18:24:51.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.317+0000 7f90abfff700 1 -- 172.21.15.84:0/4116468758 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f909c018020 con 0x7f90b4102cd0 2024-04-26T18:24:51.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.317+0000 7f90abfff700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f90a00a3d50 0x7f90a00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:51.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.317+0000 7f90b2ffd700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f90a00a3d50 0x7f90a00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:51.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.317+0000 7f90abfff700 1 -- 172.21.15.84:0/4116468758 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f909c0cfeb0 con 0x7f90b4102cd0 2024-04-26T18:24:51.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.318+0000 7f90b2ffd700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f90a00a3d50 0x7f90a00a6210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009fd0 tx=0x7f90ac00b7b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:51.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.321+0000 7f90abfff700 1 -- 172.21.15.84:0/4116468758 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f909c01f020 con 0x7f90b4102cd0 2024-04-26T18:24:51.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.556+0000 7f90b9778700 1 -- 172.21.15.84:0/4116468758 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f9094000bc0 con 0x7f90a00a3d50 2024-04-26T18:24:51.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.562+0000 7f90abfff700 1 -- 172.21.15.84:0/4116468758 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f9094000bc0 con 0x7f90a00a3d50 2024-04-26T18:24:51.562 INFO:teuthology.orchestra.run.smithi084.stdout:/volumes/qa/sv_0/dc237957-be98-4cf4-8f32-1c82eaa7a693 2024-04-26T18:24:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.564+0000 7f90b9778700 1 -- 172.21.15.84:0/4116468758 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f90a00a3d50 msgr2=0x7f90a00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.564+0000 7f90b9778700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f90a00a3d50 0x7f90a00a6210 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f90ac009fd0 tx=0x7f90ac00b7b0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.564+0000 7f90b9778700 1 -- 172.21.15.84:0/4116468758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b4102cd0 msgr2=0x7f90b4198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.564+0000 7f90b9778700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b4102cd0 0x7f90b4198d10 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f909c00b530 tx=0x7f909c00b840 comp rx=0 tx=0).stop 2024-04-26T18:24:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.565+0000 7f90b9778700 1 -- 172.21.15.84:0/4116468758 shutdown_connections 2024-04-26T18:24:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.565+0000 7f90b9778700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f90a00a3d50 0x7f90a00a6210 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.565+0000 7f90b9778700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f90b4108380 0x7f90b4199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.565+0000 7f90b9778700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f90b4102310 0x7f90b41987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.565+0000 7f90b9778700 1 --2- 172.21.15.84:0/4116468758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f90b4102cd0 0x7f90b4198d10 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.565+0000 7f90b9778700 1 -- 172.21.15.84:0/4116468758 >> 172.21.15.84:0/4116468758 conn(0x7f90b4079290 msgr2=0x7f90b41011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.565+0000 7f90b9778700 1 -- 172.21.15.84:0/4116468758 shutdown_connections 2024-04-26T18:24:51.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:51.565+0000 7f90b9778700 1 -- 172.21.15.84:0/4116468758 wait complete. 2024-04-26T18:24:52.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:52 smithi084 ceph-mon[29141]: pgmap v312: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-04-26T18:24:52.389 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-26T18:24:52.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:52 smithi103 ceph-mon[32727]: pgmap v312: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-04-26T18:24:52.628 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:52.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:52 smithi174 ceph-mon[32705]: pgmap v312: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 4.7 KiB/s wr, 19 op/s 2024-04-26T18:24:53.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:53 smithi084 ceph-mon[29141]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:53.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:53 smithi103 ceph-mon[32727]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:53.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:53 smithi174 ceph-mon[32705]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:54.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.372+0000 7facfddba700 1 -- 172.21.15.84:0/1193067932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf806c7a0 msgr2=0x7facf8108060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:54.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.372+0000 7facfddba700 1 --2- 172.21.15.84:0/1193067932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf806c7a0 0x7facf8108060 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7face80099c0 tx=0x7face8009cd0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.372+0000 7facfddba700 1 -- 172.21.15.84:0/1193067932 shutdown_connections 2024-04-26T18:24:54.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.372+0000 7facfddba700 1 --2- 172.21.15.84:0/1193067932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facf8108740 0x7facf8110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.372+0000 7facfddba700 1 --2- 172.21.15.84:0/1193067932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf806bdf0 0x7facf806c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.372+0000 7facfddba700 1 --2- 172.21.15.84:0/1193067932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf806c7a0 0x7facf8108060 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.372+0000 7facfddba700 1 -- 172.21.15.84:0/1193067932 >> 172.21.15.84:0/1193067932 conn(0x7facf8078370 msgr2=0x7facf8078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:54.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.372+0000 7facfddba700 1 -- 172.21.15.84:0/1193067932 shutdown_connections 2024-04-26T18:24:54.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.372+0000 7facfddba700 1 -- 172.21.15.84:0/1193067932 wait complete. 2024-04-26T18:24:54.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.372+0000 7facfddba700 1 Processor -- start 2024-04-26T18:24:54.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facfddba700 1 -- start start 2024-04-26T18:24:54.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facfddba700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf806bdf0 0x7facf8102480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facfddba700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf806c7a0 0x7facf81029c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facfddba700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facf8108740 0x7facf8102f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facfddba700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7facf8113610 con 0x7facf806bdf0 2024-04-26T18:24:54.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facfddba700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7facf8113490 con 0x7facf806c7a0 2024-04-26T18:24:54.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facfddba700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7facf8113790 con 0x7facf8108740 2024-04-26T18:24:54.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf806bdf0 0x7facf8102480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf806bdf0 0x7facf8102480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49192/0 (socket says 172.21.15.84:49192) 2024-04-26T18:24:54.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf77fe700 1 -- 172.21.15.84:0/4252203048 learned_addr learned my addr 172.21.15.84:0/4252203048 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:54.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf7fff700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facf8108740 0x7facf8102f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf6ffd700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf806c7a0 0x7facf81029c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf77fe700 1 -- 172.21.15.84:0/4252203048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facf8108740 msgr2=0x7facf8102f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf77fe700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facf8108740 0x7facf8102f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf77fe700 1 -- 172.21.15.84:0/4252203048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf806c7a0 msgr2=0x7facf81029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf77fe700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf806c7a0 0x7facf81029c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf77fe700 1 -- 172.21.15.84:0/4252203048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf81a8070 con 0x7facf806bdf0 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf6ffd700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf806c7a0 0x7facf81029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf77fe700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf806bdf0 0x7facf8102480 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7facec00d950 tx=0x7facec00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf4ff9700 1 -- 172.21.15.84:0/4252203048 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec016070 con 0x7facf806bdf0 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf4ff9700 1 -- 172.21.15.84:0/4252203048 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facec010460 con 0x7facf806bdf0 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.373+0000 7facf4ff9700 1 -- 172.21.15.84:0/4252203048 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facec00f620 con 0x7facf806bdf0 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.374+0000 7facfddba700 1 -- 172.21.15.84:0/4252203048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf81a8360 con 0x7facf806bdf0 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.374+0000 7facfddba700 1 -- 172.21.15.84:0/4252203048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf81a8850 con 0x7facf806bdf0 2024-04-26T18:24:54.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.375+0000 7facfddba700 1 -- 172.21.15.84:0/4252203048 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf806a110 con 0x7facf806bdf0 2024-04-26T18:24:54.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.376+0000 7facf4ff9700 1 -- 172.21.15.84:0/4252203048 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7facec019070 con 0x7facf806bdf0 2024-04-26T18:24:54.378 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:54 smithi084 ceph-mon[29141]: pgmap v313: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-26T18:24:54.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.376+0000 7facf4ff9700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7face00a3d50 0x7face00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:54.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.377+0000 7facf6ffd700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7face00a3d50 0x7face00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:54.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.377+0000 7facf4ff9700 1 -- 172.21.15.84:0/4252203048 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7facec0cffb0 con 0x7facf806bdf0 2024-04-26T18:24:54.379 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.377+0000 7facf6ffd700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7face00a3d50 0x7face00a6210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7face8000c00 tx=0x7face8009f10 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:54.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.381+0000 7facf4ff9700 1 -- 172.21.15.84:0/4252203048 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7facec09c9e0 con 0x7facf806bdf0 2024-04-26T18:24:54.609 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.609+0000 7facfddba700 1 -- 172.21.15.84:0/4252203048 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7facf8063860 con 0x7face00a3d50 2024-04-26T18:24:54.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:54 smithi103 ceph-mon[32727]: pgmap v313: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-26T18:24:54.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.634+0000 7facf4ff9700 1 -- 172.21.15.84:0/4252203048 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7facf8063860 con 0x7face00a3d50 2024-04-26T18:24:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.636+0000 7facfddba700 1 -- 172.21.15.84:0/4252203048 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7face00a3d50 msgr2=0x7face00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:54.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.636+0000 7facfddba700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7face00a3d50 0x7face00a6210 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7face8000c00 tx=0x7face8009f10 comp rx=0 tx=0).stop 2024-04-26T18:24:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.636+0000 7facfddba700 1 -- 172.21.15.84:0/4252203048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf806bdf0 msgr2=0x7facf8102480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.636+0000 7facfddba700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf806bdf0 0x7facf8102480 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7facec00d950 tx=0x7facec00dd10 comp rx=0 tx=0).stop 2024-04-26T18:24:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.636+0000 7facfddba700 1 -- 172.21.15.84:0/4252203048 shutdown_connections 2024-04-26T18:24:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.636+0000 7facfddba700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7face00a3d50 0x7face00a6210 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.636+0000 7facfddba700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf806bdf0 0x7facf8102480 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.636+0000 7facfddba700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facf8108740 0x7facf8102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.636+0000 7facfddba700 1 --2- 172.21.15.84:0/4252203048 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf806c7a0 0x7facf81029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.636+0000 7facfddba700 1 -- 172.21.15.84:0/4252203048 >> 172.21.15.84:0/4252203048 conn(0x7facf8078370 msgr2=0x7facf8107380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.636+0000 7facfddba700 1 -- 172.21.15.84:0/4252203048 shutdown_connections 2024-04-26T18:24:54.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:54.636+0000 7facfddba700 1 -- 172.21.15.84:0/4252203048 wait complete. 2024-04-26T18:24:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:54 smithi174 ceph-mon[32705]: pgmap v313: 97 pgs: 97 active+clean; 605 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 11 op/s 2024-04-26T18:24:55.416 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-26T18:24:55.655 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:56.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:56 smithi084 ceph-mon[29141]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:56.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:56 smithi084 ceph-mon[29141]: pgmap v314: 97 pgs: 97 active+clean; 610 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-04-26T18:24:56.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:56 smithi103 ceph-mon[32727]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:56.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:56 smithi103 ceph-mon[32727]: pgmap v314: 97 pgs: 97 active+clean; 610 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-04-26T18:24:56.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:56 smithi174 ceph-mon[32705]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:56.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:56 smithi174 ceph-mon[32705]: pgmap v314: 97 pgs: 97 active+clean; 610 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 8.3 KiB/s rd, 2.8 KiB/s wr, 12 op/s 2024-04-26T18:24:57.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.555+0000 7feda2919700 1 -- 172.21.15.84:0/623098934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed9c101a10 msgr2=0x7fed9c101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:57.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.555+0000 7feda2919700 1 --2- 172.21.15.84:0/623098934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed9c101a10 0x7fed9c101e70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fed84009990 tx=0x7fed84009ca0 comp rx=0 tx=0).stop 2024-04-26T18:24:57.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.555+0000 7feda2919700 1 -- 172.21.15.84:0/623098934 shutdown_connections 2024-04-26T18:24:57.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.555+0000 7feda2919700 1 --2- 172.21.15.84:0/623098934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c1023b0 0x7fed9c1102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:57.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.555+0000 7feda2919700 1 --2- 172.21.15.84:0/623098934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed9c1094b0 0x7fed9c109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:57.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.555+0000 7feda2919700 1 --2- 172.21.15.84:0/623098934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed9c101a10 0x7fed9c101e70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:57.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.555+0000 7feda2919700 1 -- 172.21.15.84:0/623098934 >> 172.21.15.84:0/623098934 conn(0x7fed9c078990 msgr2=0x7fed9c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:57.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.555+0000 7feda2919700 1 -- 172.21.15.84:0/623098934 shutdown_connections 2024-04-26T18:24:57.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.555+0000 7feda2919700 1 -- 172.21.15.84:0/623098934 wait complete. 2024-04-26T18:24:57.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7feda2919700 1 Processor -- start 2024-04-26T18:24:57.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7feda2919700 1 -- start start 2024-04-26T18:24:57.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7feda2919700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c101a10 0x7fed9c102bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:57.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7feda2919700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed9c1023b0 0x7fed9c103130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:57.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7feda2919700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed9c1094b0 0x7fed9c103670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:57.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7feda2919700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c1135c0 con 0x7fed9c1023b0 2024-04-26T18:24:57.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7feda2919700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c113440 con 0x7fed9c101a10 2024-04-26T18:24:57.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7feda2919700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed9c113740 con 0x7fed9c1094b0 2024-04-26T18:24:57.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7fed93fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed9c1023b0 0x7fed9c103130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:57.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7fed93fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed9c1023b0 0x7fed9c103130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39414/0 (socket says 172.21.15.84:39414) 2024-04-26T18:24:57.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7fed93fff700 1 -- 172.21.15.84:0/2734264446 learned_addr learned my addr 172.21.15.84:0/2734264446 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:24:57.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7fed9bfff700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c101a10 0x7fed9c102bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:57.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7feda0eb6700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed9c1094b0 0x7fed9c103670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:57.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7fed93fff700 1 -- 172.21.15.84:0/2734264446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed9c1094b0 msgr2=0x7fed9c103670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:57.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7fed93fff700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed9c1094b0 0x7fed9c103670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:57.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7fed93fff700 1 -- 172.21.15.84:0/2734264446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c101a10 msgr2=0x7fed9c102bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:57.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7fed93fff700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c101a10 0x7fed9c102bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:57.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7fed93fff700 1 -- 172.21.15.84:0/2734264446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed9c1a8020 con 0x7fed9c1023b0 2024-04-26T18:24:57.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7fed9bfff700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c101a10 0x7fed9c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:57.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7feda0eb6700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed9c1094b0 0x7fed9c103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:24:57.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.556+0000 7fed93fff700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed9c1023b0 0x7fed9c103130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fed84000c00 tx=0x7fed840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:57.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.557+0000 7fed99ffb700 1 -- 172.21.15.84:0/2734264446 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84027070 con 0x7fed9c1023b0 2024-04-26T18:24:57.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.557+0000 7fed99ffb700 1 -- 172.21.15.84:0/2734264446 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed84003fa0 con 0x7fed9c1023b0 2024-04-26T18:24:57.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.557+0000 7feda2919700 1 -- 172.21.15.84:0/2734264446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed9c1a82b0 con 0x7fed9c1023b0 2024-04-26T18:24:57.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.557+0000 7fed99ffb700 1 -- 172.21.15.84:0/2734264446 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed84022db0 con 0x7fed9c1023b0 2024-04-26T18:24:57.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.557+0000 7feda2919700 1 -- 172.21.15.84:0/2734264446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed9c1a8700 con 0x7fed9c1023b0 2024-04-26T18:24:57.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.558+0000 7feda2919700 1 -- 172.21.15.84:0/2734264446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed9c104d30 con 0x7fed9c1023b0 2024-04-26T18:24:57.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.559+0000 7fed99ffb700 1 -- 172.21.15.84:0/2734264446 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fed8402a070 con 0x7fed9c1023b0 2024-04-26T18:24:57.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.559+0000 7fed99ffb700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fed880a3cd0 0x7fed880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:24:57.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.559+0000 7fed9bfff700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fed880a3cd0 0x7fed880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:24:57.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.559+0000 7fed99ffb700 1 -- 172.21.15.84:0/2734264446 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7fed840d14a0 con 0x7fed9c1023b0 2024-04-26T18:24:57.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.560+0000 7fed9bfff700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fed880a3cd0 0x7fed880a6190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009b50 tx=0x7fed8c009550 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:24:57.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.563+0000 7fed99ffb700 1 -- 172.21.15.84:0/2734264446 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed84013070 con 0x7fed9c1023b0 2024-04-26T18:24:57.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.794+0000 7feda2919700 1 -- 172.21.15.84:0/2734264446 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fed9c063860 con 0x7fed880a3cd0 2024-04-26T18:24:57.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.799+0000 7fed99ffb700 1 -- 172.21.15.84:0/2734264446 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fed9c063860 con 0x7fed880a3cd0 2024-04-26T18:24:57.799 INFO:teuthology.orchestra.run.smithi084.stdout:/volumes/qa/sv_1/44f1e2f4-deda-464d-9481-4db402deadff 2024-04-26T18:24:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.801+0000 7feda2919700 1 -- 172.21.15.84:0/2734264446 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fed880a3cd0 msgr2=0x7fed880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.801+0000 7feda2919700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fed880a3cd0 0x7fed880a6190 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009b50 tx=0x7fed8c009550 comp rx=0 tx=0).stop 2024-04-26T18:24:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.801+0000 7feda2919700 1 -- 172.21.15.84:0/2734264446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed9c1023b0 msgr2=0x7fed9c103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:24:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.801+0000 7feda2919700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed9c1023b0 0x7fed9c103130 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fed84000c00 tx=0x7fed840047b0 comp rx=0 tx=0).stop 2024-04-26T18:24:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.801+0000 7feda2919700 1 -- 172.21.15.84:0/2734264446 shutdown_connections 2024-04-26T18:24:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.801+0000 7feda2919700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fed880a3cd0 0x7fed880a6190 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.801+0000 7feda2919700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed9c1094b0 0x7fed9c103670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.801+0000 7feda2919700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed9c101a10 0x7fed9c102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.801+0000 7feda2919700 1 --2- 172.21.15.84:0/2734264446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed9c1023b0 0x7fed9c103130 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:24:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.801+0000 7feda2919700 1 -- 172.21.15.84:0/2734264446 >> 172.21.15.84:0/2734264446 conn(0x7fed9c078990 msgr2=0x7fed9c106540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:24:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.802+0000 7feda2919700 1 -- 172.21.15.84:0/2734264446 shutdown_connections 2024-04-26T18:24:57.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:24:57.802+0000 7feda2919700 1 -- 172.21.15.84:0/2734264446 wait complete. 2024-04-26T18:24:58.447 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:58 smithi084 ceph-mon[29141]: pgmap v315: 97 pgs: 97 active+clean; 610 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-04-26T18:24:58.475 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:24:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:58 smithi103 ceph-mon[32727]: pgmap v315: 97 pgs: 97 active+clean; 610 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-04-26T18:24:58.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:58 smithi174 ceph-mon[32705]: pgmap v315: 97 pgs: 97 active+clean; 610 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 2.8 KiB/s wr, 11 op/s 2024-04-26T18:24:58.721 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:24:59.572 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:24:59 smithi084 ceph-mon[29141]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:24:59 smithi103 ceph-mon[32727]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:24:59.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:24:59 smithi174 ceph-mon[32705]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:25:00.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.419+0000 7f99e0387700 1 -- 172.21.15.84:0/697739062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99d806c580 msgr2=0x7f99d81102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:00.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.420+0000 7f99e0387700 1 --2- 172.21.15.84:0/697739062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99d806c580 0x7f99d81102e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f99d000b0a0 tx=0x7f99d000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:25:00.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.420+0000 7f99e0387700 1 -- 172.21.15.84:0/697739062 shutdown_connections 2024-04-26T18:25:00.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.420+0000 7f99e0387700 1 --2- 172.21.15.84:0/697739062 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99d806c580 0x7f99d81102e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:00.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.420+0000 7f99e0387700 1 --2- 172.21.15.84:0/697739062 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99d81094a0 0x7f99d8109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:00.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.420+0000 7f99e0387700 1 --2- 172.21.15.84:0/697739062 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99d806bbe0 0x7f99d806c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:00.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.420+0000 7f99e0387700 1 -- 172.21.15.84:0/697739062 >> 172.21.15.84:0/697739062 conn(0x7f99d80792d0 msgr2=0x7f99d80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:00.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.420+0000 7f99e0387700 1 -- 172.21.15.84:0/697739062 shutdown_connections 2024-04-26T18:25:00.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.420+0000 7f99e0387700 1 -- 172.21.15.84:0/697739062 wait complete. 2024-04-26T18:25:00.422 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.420+0000 7f99e0387700 1 Processor -- start 2024-04-26T18:25:00.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.420+0000 7f99e0387700 1 -- start start 2024-04-26T18:25:00.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99e0387700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99d806bbe0 0x7f99d81023f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:00.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99e0387700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99d806c580 0x7f99d8102930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:00.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99e0387700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99d81094a0 0x7f99d8102e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:00.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99e0387700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f99d81135d0 con 0x7f99d806c580 2024-04-26T18:25:00.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99e0387700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8113450 con 0x7f99d806bbe0 2024-04-26T18:25:00.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99e0387700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f99d8113750 con 0x7f99d81094a0 2024-04-26T18:25:00.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99dd922700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99d806c580 0x7f99d8102930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:00.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99dd922700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99d806c580 0x7f99d8102930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39434/0 (socket says 172.21.15.84:39434) 2024-04-26T18:25:00.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99dd922700 1 -- 172.21.15.84:0/3041890041 learned_addr learned my addr 172.21.15.84:0/3041890041 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:00.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99de123700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99d806bbe0 0x7f99d81023f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:00.423 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99de924700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99d81094a0 0x7f99d8102e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:00.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99dd922700 1 -- 172.21.15.84:0/3041890041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99d81094a0 msgr2=0x7f99d8102e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:00.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99dd922700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99d81094a0 0x7f99d8102e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:00.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99dd922700 1 -- 172.21.15.84:0/3041890041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99d806bbe0 msgr2=0x7f99d81023f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:00.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99dd922700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99d806bbe0 0x7f99d81023f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:00.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99dd922700 1 -- 172.21.15.84:0/3041890041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99d81a7f20 con 0x7f99d806c580 2024-04-26T18:25:00.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99dd922700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99d806c580 0x7f99d8102930 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f99c800e9c0 tx=0x7f99c800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:00.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99cf7fe700 1 -- 172.21.15.84:0/3041890041 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c800cbd0 con 0x7f99d806c580 2024-04-26T18:25:00.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99cf7fe700 1 -- 172.21.15.84:0/3041890041 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99c800ee60 con 0x7f99d806c580 2024-04-26T18:25:00.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99e0387700 1 -- 172.21.15.84:0/3041890041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99d81a8210 con 0x7f99d806c580 2024-04-26T18:25:00.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.421+0000 7f99cf7fe700 1 -- 172.21.15.84:0/3041890041 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99c80055c0 con 0x7f99d806c580 2024-04-26T18:25:00.424 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.422+0000 7f99e0387700 1 -- 172.21.15.84:0/3041890041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99d81a8700 con 0x7f99d806c580 2024-04-26T18:25:00.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.423+0000 7f99e0387700 1 -- 172.21.15.84:0/3041890041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99d806a110 con 0x7f99d806c580 2024-04-26T18:25:00.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.423+0000 7f99cf7fe700 1 -- 172.21.15.84:0/3041890041 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f99c8004750 con 0x7f99d806c580 2024-04-26T18:25:00.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.424+0000 7f99cf7fe700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f99c40a3cd0 0x7f99c40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:00.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.424+0000 7f99de123700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f99c40a3cd0 0x7f99c40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:00.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.424+0000 7f99cf7fe700 1 -- 172.21.15.84:0/3041890041 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f99c80d0690 con 0x7f99d806c580 2024-04-26T18:25:00.425 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.424+0000 7f99de123700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f99c40a3cd0 0x7f99c40a6190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f99d400abd0 tx=0x7f99d400a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:00.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.429+0000 7f99cf7fe700 1 -- 172.21.15.84:0/3041890041 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f99c809d0c0 con 0x7f99d806c580 2024-04-26T18:25:00.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:00 smithi084 ceph-mon[29141]: pgmap v316: 97 pgs: 97 active+clean; 628 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-04-26T18:25:00.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:00 smithi103 ceph-mon[32727]: pgmap v316: 97 pgs: 97 active+clean; 628 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-04-26T18:25:00.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.680+0000 7f99e0387700 1 -- 172.21.15.84:0/3041890041 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f99d802e490 con 0x7f99d806c580 2024-04-26T18:25:00.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.681+0000 7f99cf7fe700 1 -- 172.21.15.84:0/3041890041 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11936 (secure 0 0 0) 0x7f99c809c9c0 con 0x7f99d806c580 2024-04-26T18:25:00.681 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:25:00.681 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:24:41.425794+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24407,24418,24422,24430,24434]},"id":1}]} 2024-04-26T18:25:00.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.683+0000 7f99e0387700 1 -- 172.21.15.84:0/3041890041 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f99c40a3cd0 msgr2=0x7f99c40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:00.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.683+0000 7f99e0387700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f99c40a3cd0 0x7f99c40a6190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f99d400abd0 tx=0x7f99d400a380 comp rx=0 tx=0).stop 2024-04-26T18:25:00.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.683+0000 7f99e0387700 1 -- 172.21.15.84:0/3041890041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99d806c580 msgr2=0x7f99d8102930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:00.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.683+0000 7f99e0387700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99d806c580 0x7f99d8102930 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f99c800e9c0 tx=0x7f99c800ecd0 comp rx=0 tx=0).stop 2024-04-26T18:25:00.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.683+0000 7f99e0387700 1 -- 172.21.15.84:0/3041890041 shutdown_connections 2024-04-26T18:25:00.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.683+0000 7f99e0387700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f99c40a3cd0 0x7f99c40a6190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:00.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.683+0000 7f99e0387700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f99d81094a0 0x7f99d8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:00.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.683+0000 7f99e0387700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f99d806bbe0 0x7f99d81023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:00.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.683+0000 7f99e0387700 1 --2- 172.21.15.84:0/3041890041 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f99d806c580 0x7f99d8102930 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:00.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.683+0000 7f99e0387700 1 -- 172.21.15.84:0/3041890041 >> 172.21.15.84:0/3041890041 conn(0x7f99d80792d0 msgr2=0x7f99d81002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:00.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.683+0000 7f99e0387700 1 -- 172.21.15.84:0/3041890041 shutdown_connections 2024-04-26T18:25:00.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:00.683+0000 7f99e0387700 1 -- 172.21.15.84:0/3041890041 wait complete. 2024-04-26T18:25:00.684 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 28 2024-04-26T18:25:00.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:00 smithi174 ceph-mon[32705]: pgmap v316: 97 pgs: 97 active+clean; 628 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 4.6 KiB/s wr, 13 op/s 2024-04-26T18:25:01.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:01 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3041890041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:25:01.421 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:25:01.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:01 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3041890041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:25:01.661 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:01.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:01 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3041890041' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:25:02.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:02 smithi084 ceph-mon[29141]: pgmap v317: 97 pgs: 97 active+clean; 628 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-26T18:25:02.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:02 smithi103 ceph-mon[32727]: pgmap v317: 97 pgs: 97 active+clean; 628 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-26T18:25:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:02 smithi174 ceph-mon[32705]: pgmap v317: 97 pgs: 97 active+clean; 628 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.5 KiB/s wr, 12 op/s 2024-04-26T18:25:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.328+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/3342418118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f64109480 msgr2=0x7f0f64109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.328+0000 7f0f6b9f8700 1 --2- 172.21.15.84:0/3342418118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f64109480 0x7f0f64109860 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0f60009990 tx=0x7f0f60009ca0 comp rx=0 tx=0).stop 2024-04-26T18:25:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.328+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/3342418118 shutdown_connections 2024-04-26T18:25:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.328+0000 7f0f6b9f8700 1 --2- 172.21.15.84:0/3342418118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f64109480 0x7f0f64109860 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.328+0000 7f0f6b9f8700 1 --2- 172.21.15.84:0/3342418118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f64103ea0 0x7f0f641102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:03.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.328+0000 7f0f6b9f8700 1 --2- 172.21.15.84:0/3342418118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f641034e0 0x7f0f64103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.328+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/3342418118 >> 172.21.15.84:0/3342418118 conn(0x7f0f6407a5e0 msgr2=0x7f0f6407a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.328+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/3342418118 shutdown_connections 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.328+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/3342418118 wait complete. 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.329+0000 7f0f6b9f8700 1 Processor -- start 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.329+0000 7f0f6b9f8700 1 -- start start 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.329+0000 7f0f6b9f8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f641034e0 0x7f0f6419f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.329+0000 7f0f6b9f8700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f64103ea0 0x7f0f6419fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.329+0000 7f0f6b9f8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f64109480 0x7f0f64199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.329+0000 7f0f6b9f8700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0f64113690 con 0x7f0f64103ea0 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.329+0000 7f0f6b9f8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0f64113510 con 0x7f0f64109480 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.329+0000 7f0f6b9f8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0f64113810 con 0x7f0f641034e0 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.329+0000 7f0f68f93700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f64103ea0 0x7f0f6419fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.329+0000 7f0f68f93700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f64103ea0 0x7f0f6419fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39456/0 (socket says 172.21.15.84:39456) 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.329+0000 7f0f68f93700 1 -- 172.21.15.84:0/1444546394 learned_addr learned my addr 172.21.15.84:0/1444546394 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:03.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.329+0000 7f0f69f95700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f64109480 0x7f0f64199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.329+0000 7f0f69794700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f641034e0 0x7f0f6419f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.330+0000 7f0f68f93700 1 -- 172.21.15.84:0/1444546394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f641034e0 msgr2=0x7f0f6419f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.330+0000 7f0f68f93700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f641034e0 0x7f0f6419f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.330+0000 7f0f68f93700 1 -- 172.21.15.84:0/1444546394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f64109480 msgr2=0x7f0f64199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.330+0000 7f0f68f93700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f64109480 0x7f0f64199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.330+0000 7f0f68f93700 1 -- 172.21.15.84:0/1444546394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f6419a190 con 0x7f0f64103ea0 2024-04-26T18:25:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.330+0000 7f0f69f95700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f64109480 0x7f0f64199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.330+0000 7f0f68f93700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f64103ea0 0x7f0f6419fdb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400b530 tx=0x7f0f5400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.330+0000 7f0f5a7fc700 1 -- 172.21.15.84:0/1444546394 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54015020 con 0x7f0f64103ea0 2024-04-26T18:25:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.330+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/1444546394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f6419a480 con 0x7f0f64103ea0 2024-04-26T18:25:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.330+0000 7f0f5a7fc700 1 -- 172.21.15.84:0/1444546394 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f54011ce0 con 0x7f0f64103ea0 2024-04-26T18:25:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.330+0000 7f0f5a7fc700 1 -- 172.21.15.84:0/1444546394 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f54010d20 con 0x7f0f64103ea0 2024-04-26T18:25:03.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.330+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/1444546394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f641a6740 con 0x7f0f64103ea0 2024-04-26T18:25:03.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.331+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/1444546394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f64104b30 con 0x7f0f64103ea0 2024-04-26T18:25:03.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.332+0000 7f0f5a7fc700 1 -- 172.21.15.84:0/1444546394 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f0f54018020 con 0x7f0f64103ea0 2024-04-26T18:25:03.334 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.333+0000 7f0f5a7fc700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0f500a3cd0 0x7f0f500a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:03.334 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.333+0000 7f0f69794700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0f500a3cd0 0x7f0f500a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:03.334 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.333+0000 7f0f5a7fc700 1 -- 172.21.15.84:0/1444546394 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 7979+0+0 (secure 0 0 0) 0x7f0f540cfb10 con 0x7f0f64103ea0 2024-04-26T18:25:03.334 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.333+0000 7f0f69794700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0f500a3cd0 0x7f0f500a6190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0f60006010 tx=0x7f0f6000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:03.337 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.337+0000 7f0f5a7fc700 1 -- 172.21.15.84:0/1444546394 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0f5401f020 con 0x7f0f64103ea0 2024-04-26T18:25:03.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.539+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/1444546394 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0f6419af90 con 0x7f0f64103ea0 2024-04-26T18:25:03.541 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.541+0000 7f0f5a7fc700 1 -- 172.21.15.84:0/1444546394 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+22183 (secure 0 0 0) 0x7f0f6419af90 con 0x7f0f64103ea0 2024-04-26T18:25:03.541 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:25:03.542 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":89,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:16:06.267694+0000","modified":"2024-04-26T18:24:22.811030+0000","last_up_change":"2024-04-26T18:23:19.906511+0000","last_in_change":"2024-04-26T18:22:59.541869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:19:31.174024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:24:16.533128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:24:19.742656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1076912-986e-4825-b1d1-5fa9d69bc07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6803","nonce":140050011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6805","nonce":140050011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6809","nonce":140050011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6807","nonce":140050011}]},"public_addr":"172.21.15.84:6803/140050011","cluster_addr":"172.21.15.84:6805/140050011","heartbeat_back_addr":"172.21.15.84:6809/140050011","heartbeat_front_addr":"172.21.15.84:6807/140050011","state":["exists","up"]},{"osd":1,"uuid":"fa93293d-9058-4804-ba4b-7e1e674b0fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6801","nonce":2512857657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6803","nonce":2512857657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6807","nonce":2512857657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6805","nonce":2512857657}]},"public_addr":"172.21.15.103:6801/2512857657","cluster_addr":"172.21.15.103:6803/2512857657","heartbeat_back_addr":"172.21.15.103:6807/2512857657","heartbeat_front_addr":"172.21.15.103:6805/2512857657","state":["exists","up"]},{"osd":2,"uuid":"bf78a44f-80a2-4cd7-974f-5abdf771e650","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6801","nonce":76340924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6803","nonce":76340924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6807","nonce":76340924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6805","nonce":76340924}]},"public_addr":"172.21.15.174:6801/76340924","cluster_addr":"172.21.15.174:6803/76340924","heartbeat_back_addr":"172.21.15.174:6807/76340924","heartbeat_front_addr":"172.21.15.174:6805/76340924","state":["exists","up"]},{"osd":3,"uuid":"55fd0035-bc76-4235-b8b5-5cf5ef770139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6811","nonce":2352789256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6813","nonce":2352789256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6817","nonce":2352789256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6815","nonce":2352789256}]},"public_addr":"172.21.15.84:6811/2352789256","cluster_addr":"172.21.15.84:6813/2352789256","heartbeat_back_addr":"172.21.15.84:6817/2352789256","heartbeat_front_addr":"172.21.15.84:6815/2352789256","state":["exists","up"]},{"osd":4,"uuid":"c278a0f3-e436-4d1c-a009-f9165ebf3c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6809","nonce":1619258252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6811","nonce":1619258252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6815","nonce":1619258252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6813","nonce":1619258252}]},"public_addr":"172.21.15.103:6809/1619258252","cluster_addr":"172.21.15.103:6811/1619258252","heartbeat_back_addr":"172.21.15.103:6815/1619258252","heartbeat_front_addr":"172.21.15.103:6813/1619258252","state":["exists","up"]},{"osd":5,"uuid":"6505be9b-c1b0-4d8b-944e-9920f90cb88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6809","nonce":3168291768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6811","nonce":3168291768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6815","nonce":3168291768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6813","nonce":3168291768}]},"public_addr":"172.21.15.174:6809/3168291768","cluster_addr":"172.21.15.174:6811/3168291768","heartbeat_back_addr":"172.21.15.174:6815/3168291768","heartbeat_front_addr":"172.21.15.174:6813/3168291768","state":["exists","up"]},{"osd":6,"uuid":"a037b66a-7488-469d-b73a-14cc0d35cb5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6819","nonce":886635297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6821","nonce":886635297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6825","nonce":886635297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6823","nonce":886635297}]},"public_addr":"172.21.15.84:6819/886635297","cluster_addr":"172.21.15.84:6821/886635297","heartbeat_back_addr":"172.21.15.84:6825/886635297","heartbeat_front_addr":"172.21.15.84:6823/886635297","state":["exists","up"]},{"osd":7,"uuid":"c425a13f-8019-4484-86a0-a34fa4d8de81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6817","nonce":1738278079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6819","nonce":1738278079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6823","nonce":1738278079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6821","nonce":1738278079}]},"public_addr":"172.21.15.103:6817/1738278079","cluster_addr":"172.21.15.103:6819/1738278079","heartbeat_back_addr":"172.21.15.103:6823/1738278079","heartbeat_front_addr":"172.21.15.103:6821/1738278079","state":["exists","up"]},{"osd":8,"uuid":"687e6c08-bfa4-440c-b3bc-90a5748989d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6817","nonce":38410548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6819","nonce":38410548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6823","nonce":38410548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6821","nonce":38410548}]},"public_addr":"172.21.15.174:6817/38410548","cluster_addr":"172.21.15.174:6819/38410548","heartbeat_back_addr":"172.21.15.174:6823/38410548","heartbeat_front_addr":"172.21.15.174:6821/38410548","state":["exists","up"]},{"osd":9,"uuid":"059eb22d-6dfb-4dd0-b192-84ed42ee1fbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6827","nonce":2926393927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6829","nonce":2926393927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6833","nonce":2926393927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6831","nonce":2926393927}]},"public_addr":"172.21.15.84:6827/2926393927","cluster_addr":"172.21.15.84:6829/2926393927","heartbeat_back_addr":"172.21.15.84:6833/2926393927","heartbeat_front_addr":"172.21.15.84:6831/2926393927","state":["exists","up"]},{"osd":10,"uuid":"da31e305-5220-4a87-a131-005b7c7f76b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6825","nonce":2705726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6827","nonce":2705726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6831","nonce":2705726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6829","nonce":2705726915}]},"public_addr":"172.21.15.103:6825/2705726915","cluster_addr":"172.21.15.103:6827/2705726915","heartbeat_back_addr":"172.21.15.103:6831/2705726915","heartbeat_front_addr":"172.21.15.103:6829/2705726915","state":["exists","up"]},{"osd":11,"uuid":"3fde6c8f-ee02-41ee-a34f-73072b4761e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6825","nonce":1668672128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6827","nonce":1668672128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6831","nonce":1668672128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6829","nonce":1668672128}]},"public_addr":"172.21.15.174:6825/1668672128","cluster_addr":"172.21.15.174:6827/1668672128","heartbeat_back_addr":"172.21.15.174:6831/1668672128","heartbeat_front_addr":"172.21.15.174:6829/1668672128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:39.099188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:03.327608+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:27.714108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:52.506015+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:17.631670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:42.702489+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:07.606697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:34.123214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:59.534658+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:25.357720+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:51.543106+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:23:18.208230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1604680842":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6800/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/538363975":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/952052990":"2024-04-27T18:17:05.083192+0000","172.21.15.84:0/1266942705":"2024-04-27T18:16:37.859669+0000","172.21.15.84:6800/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3970542149":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3607125049":"2024-04-27T18:17:05.083192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:25:03.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.543+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/1444546394 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0f500a3cd0 msgr2=0x7f0f500a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:03.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.543+0000 7f0f6b9f8700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0f500a3cd0 0x7f0f500a6190 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f0f60006010 tx=0x7f0f6000b540 comp rx=0 tx=0).stop 2024-04-26T18:25:03.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.543+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/1444546394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f64103ea0 msgr2=0x7f0f6419fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:03.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.543+0000 7f0f6b9f8700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f64103ea0 0x7f0f6419fdb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400b530 tx=0x7f0f5400b8f0 comp rx=0 tx=0).stop 2024-04-26T18:25:03.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.543+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/1444546394 shutdown_connections 2024-04-26T18:25:03.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.543+0000 7f0f6b9f8700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0f500a3cd0 0x7f0f500a6190 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:03.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.543+0000 7f0f6b9f8700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0f64109480 0x7f0f64199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:03.543 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.543+0000 7f0f6b9f8700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0f641034e0 0x7f0f6419f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:03.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.543+0000 7f0f6b9f8700 1 --2- 172.21.15.84:0/1444546394 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0f64103ea0 0x7f0f6419fdb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:03.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.543+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/1444546394 >> 172.21.15.84:0/1444546394 conn(0x7f0f6407a5e0 msgr2=0x7f0f64101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:03.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.543+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/1444546394 shutdown_connections 2024-04-26T18:25:03.544 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:03.543+0000 7f0f6b9f8700 1 -- 172.21.15.84:0/1444546394 wait complete. 2024-04-26T18:25:04.273 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T18:25:04.273 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:25:04.516 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:04.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:04 smithi084 ceph-mon[29141]: pgmap v318: 97 pgs: 97 active+clean; 628 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-26T18:25:04.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:04 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1444546394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:25:04.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:04 smithi103 ceph-mon[32727]: pgmap v318: 97 pgs: 97 active+clean; 628 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-26T18:25:04.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:04 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1444546394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:25:04.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:04 smithi174 ceph-mon[32705]: pgmap v318: 97 pgs: 97 active+clean; 628 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-26T18:25:04.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:04 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1444546394' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:25:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-26T18:25:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-26T18:25:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-26T18:25:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-04-26T18:25:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]: dispatch 2024-04-26T18:25:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-26T18:25:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-04-26T18:25:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-04-26T18:25:05.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:05 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-04-26T18:25:05.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-26T18:25:05.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-26T18:25:05.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-26T18:25:05.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-04-26T18:25:05.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]: dispatch 2024-04-26T18:25:05.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-26T18:25:05.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-04-26T18:25:05.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-04-26T18:25:05.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:05 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-04-26T18:25:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]: dispatch 2024-04-26T18:25:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]: dispatch 2024-04-26T18:25:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-26T18:25:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]: dispatch 2024-04-26T18:25:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]: dispatch 2024-04-26T18:25:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-04-26T18:25:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]: dispatch 2024-04-26T18:25:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]: dispatch 2024-04-26T18:25:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:05 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]: dispatch 2024-04-26T18:25:06.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.204+0000 7f6afdd2d700 1 -- 172.21.15.84:0/3830325068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af8102610 msgr2=0x7f6af81029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:06.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.204+0000 7f6afdd2d700 1 --2- 172.21.15.84:0/3830325068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af8102610 0x7f6af81029f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6af000b0d0 tx=0x7f6af000b3e0 comp rx=0 tx=0).stop 2024-04-26T18:25:06.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.204+0000 7f6afdd2d700 1 -- 172.21.15.84:0/3830325068 shutdown_connections 2024-04-26T18:25:06.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.204+0000 7f6afdd2d700 1 --2- 172.21.15.84:0/3830325068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af8102610 0x7f6af81029f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:06.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.204+0000 7f6afdd2d700 1 --2- 172.21.15.84:0/3830325068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8107660 0x7f6af810f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:06.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.204+0000 7f6afdd2d700 1 --2- 172.21.15.84:0/3830325068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6af8102fc0 0x7f6af8106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:06.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.204+0000 7f6afdd2d700 1 -- 172.21.15.84:0/3830325068 >> 172.21.15.84:0/3830325068 conn(0x7f6af8078370 msgr2=0x7f6af8078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:06.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.204+0000 7f6afdd2d700 1 -- 172.21.15.84:0/3830325068 shutdown_connections 2024-04-26T18:25:06.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.204+0000 7f6afdd2d700 1 -- 172.21.15.84:0/3830325068 wait complete. 2024-04-26T18:25:06.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.205+0000 7f6afdd2d700 1 Processor -- start 2024-04-26T18:25:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.205+0000 7f6afdd2d700 1 -- start start 2024-04-26T18:25:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.205+0000 7f6afdd2d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6af8102610 0x7f6af819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.205+0000 7f6afdd2d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8102fc0 0x7f6af819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.205+0000 7f6afdd2d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af8107660 0x7f6af8199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.205+0000 7f6afdd2d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6af81124f0 con 0x7f6af8107660 2024-04-26T18:25:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.205+0000 7f6afdd2d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8112370 con 0x7f6af8102610 2024-04-26T18:25:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.205+0000 7f6afdd2d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6af8112670 con 0x7f6af8102fc0 2024-04-26T18:25:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.205+0000 7f6af7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af8107660 0x7f6af8199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.205+0000 7f6af7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af8107660 0x7f6af8199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39478/0 (socket says 172.21.15.84:39478) 2024-04-26T18:25:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.205+0000 7f6af7fff700 1 -- 172.21.15.84:0/2352420745 learned_addr learned my addr 172.21.15.84:0/2352420745 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:06.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.205+0000 7f6af77fe700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6af8102610 0x7f6af819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.206+0000 7f6af6ffd700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8102fc0 0x7f6af819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.206+0000 7f6af7fff700 1 -- 172.21.15.84:0/2352420745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8102fc0 msgr2=0x7f6af819fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.206+0000 7f6af7fff700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8102fc0 0x7f6af819fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.206+0000 7f6af7fff700 1 -- 172.21.15.84:0/2352420745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6af8102610 msgr2=0x7f6af819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.206+0000 7f6af7fff700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6af8102610 0x7f6af819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.206+0000 7f6af7fff700 1 -- 172.21.15.84:0/2352420745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af819a090 con 0x7f6af8107660 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.206+0000 7f6af77fe700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6af8102610 0x7f6af819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.206+0000 7f6af7fff700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af8107660 0x7f6af8199930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800c970 tx=0x7f6ae800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.206+0000 7f6af4ff9700 1 -- 172.21.15.84:0/2352420745 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8007b00 con 0x7f6af8107660 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.206+0000 7f6af4ff9700 1 -- 172.21.15.84:0/2352420745 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae8004ce0 con 0x7f6af8107660 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.206+0000 7f6af4ff9700 1 -- 172.21.15.84:0/2352420745 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae8005530 con 0x7f6af8107660 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.206+0000 7f6afdd2d700 1 -- 172.21.15.84:0/2352420745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af819a380 con 0x7f6af8107660 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.206+0000 7f6afdd2d700 1 -- 172.21.15.84:0/2352420745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af81a6640 con 0x7f6af8107660 2024-04-26T18:25:06.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.207+0000 7f6afdd2d700 1 -- 172.21.15.84:0/2352420745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af806a110 con 0x7f6af8107660 2024-04-26T18:25:06.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.209+0000 7f6af4ff9700 1 -- 172.21.15.84:0/2352420745 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6ae8004750 con 0x7f6af8107660 2024-04-26T18:25:06.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.209+0000 7f6af4ff9700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6ae40a3cd0 0x7f6ae40a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:06.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.210+0000 7f6af77fe700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6ae40a3cd0 0x7f6ae40a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:06.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.210+0000 7f6af4ff9700 1 -- 172.21.15.84:0/2352420745 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8248+0+0 (secure 0 0 0) 0x7f6ae80d05c0 con 0x7f6af8107660 2024-04-26T18:25:06.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.210+0000 7f6af77fe700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6ae40a3cd0 0x7f6ae40a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6af000d010 tx=0x7f6af000b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:06.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.213+0000 7f6af4ff9700 1 -- 172.21.15.84:0/2352420745 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ae8014030 con 0x7f6af8107660 2024-04-26T18:25:06.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.420+0000 7f6afdd2d700 1 -- 172.21.15.84:0/2352420745 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6af8106160 con 0x7f6ae40a3cd0 2024-04-26T18:25:06.428 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.428+0000 7f6af4ff9700 1 -- 172.21.15.84:0/2352420745 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258809 (secure 0 0 0) 0x7f6af8106160 con 0x7f6ae40a3cd0 2024-04-26T18:25:06.432 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:25:06.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.431+0000 7f6afdd2d700 1 -- 172.21.15.84:0/2352420745 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6ae40a3cd0 msgr2=0x7f6ae40a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:06.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.431+0000 7f6afdd2d700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6ae40a3cd0 0x7f6ae40a6190 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f6af000d010 tx=0x7f6af000b040 comp rx=0 tx=0).stop 2024-04-26T18:25:06.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.431+0000 7f6afdd2d700 1 -- 172.21.15.84:0/2352420745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af8107660 msgr2=0x7f6af8199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:06.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.431+0000 7f6afdd2d700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af8107660 0x7f6af8199930 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6ae800c970 tx=0x7f6ae800cd30 comp rx=0 tx=0).stop 2024-04-26T18:25:06.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.431+0000 7f6afdd2d700 1 -- 172.21.15.84:0/2352420745 shutdown_connections 2024-04-26T18:25:06.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.431+0000 7f6afdd2d700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6ae40a3cd0 0x7f6ae40a6190 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:06.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.431+0000 7f6afdd2d700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6af8107660 0x7f6af8199930 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:06.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.431+0000 7f6afdd2d700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6af8102610 0x7f6af819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:06.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.431+0000 7f6afdd2d700 1 --2- 172.21.15.84:0/2352420745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6af8102fc0 0x7f6af819fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:06.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.431+0000 7f6afdd2d700 1 -- 172.21.15.84:0/2352420745 >> 172.21.15.84:0/2352420745 conn(0x7f6af8078370 msgr2=0x7f6af8108830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:06.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.431+0000 7f6afdd2d700 1 -- 172.21.15.84:0/2352420745 shutdown_connections 2024-04-26T18:25:06.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:06.431+0000 7f6afdd2d700 1 -- 172.21.15.84:0/2352420745 wait complete. 2024-04-26T18:25:06.435 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-04-26T18:25:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:06 smithi084 ceph-mon[29141]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-04-26T18:25:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-26T18:25:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-26T18:25:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-26T18:25:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-04-26T18:25:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]': finished 2024-04-26T18:25:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-26T18:25:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-04-26T18:25:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-04-26T18:25:06.596 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-04-26T18:25:06.596 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:06 smithi084 ceph-mon[29141]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T18:25:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:06 smithi103 ceph-mon[32727]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-04-26T18:25:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-26T18:25:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-26T18:25:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-26T18:25:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-04-26T18:25:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]': finished 2024-04-26T18:25:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-26T18:25:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-04-26T18:25:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-04-26T18:25:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-04-26T18:25:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:06 smithi103 ceph-mon[32727]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T18:25:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:06 smithi174 ceph-mon[32705]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.2 KiB/s wr, 12 op/s 2024-04-26T18:25:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [8, 11]}]': finished 2024-04-26T18:25:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.c", "id": [2, 1]}]': finished 2024-04-26T18:25:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-26T18:25:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4]}]': finished 2024-04-26T18:25:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8, 5, 9]}]': finished 2024-04-26T18:25:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-04-26T18:25:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 6]}]': finished 2024-04-26T18:25:06.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.31", "id": [4, 7]}]': finished 2024-04-26T18:25:06.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.39", "id": [5, 6]}]': finished 2024-04-26T18:25:06.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:06 smithi174 ceph-mon[32705]: osdmap e90: 12 total, 12 up, 12 in 2024-04-26T18:25:07.144 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":321,"stamp":"2024-04-26T18:25:06.292639+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652173,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":4,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":428,"num_read_kb":329,"num_write":420,"num_write_kb":3065,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":343,"ondisk_log_size":343,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369424,"kb_used_data":11948,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572400,"statfs":{"total":1151940427776,"available":1151562137600,"internally_reserved":0,"allocated":12234752,"data_stored":7557013,"data_compressed":115187,"data_compressed_allocated":151552,"data_compressed_original":1802240,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27422,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":4,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":75,"num_write":17,"num_write_kb":31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.115482"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.295324+0000","last_change":"2024-04-26T18:24:20.951581+0000","last_active":"2024-04-26T18:25:05.295324+0000","last_peered":"2024-04-26T18:25:05.295324+0000","last_clean":"2024-04-26T18:25:05.295324+0000","last_became_active":"2024-04-26T18:24:20.951260+0000","last_became_peered":"2024-04-26T18:24:20.951260+0000","last_unstale":"2024-04-26T18:25:05.295324+0000","last_undegraded":"2024-04-26T18:25:05.295324+0000","last_fullsized":"2024-04-26T18:25:05.295324+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:54:28.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.963225+0000","last_change":"2024-04-26T18:24:20.952263+0000","last_active":"2024-04-26T18:24:22.963225+0000","last_peered":"2024-04-26T18:24:22.963225+0000","last_clean":"2024-04-26T18:24:22.963225+0000","last_became_active":"2024-04-26T18:24:20.951930+0000","last_became_peered":"2024-04-26T18:24:20.951930+0000","last_unstale":"2024-04-26T18:24:22.963225+0000","last_undegraded":"2024-04-26T18:24:22.963225+0000","last_fullsized":"2024-04-26T18:24:22.963225+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:54.816572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.962859+0000","last_change":"2024-04-26T18:24:20.952280+0000","last_active":"2024-04-26T18:24:22.962859+0000","last_peered":"2024-04-26T18:24:22.962859+0000","last_clean":"2024-04-26T18:24:22.962859+0000","last_became_active":"2024-04-26T18:24:20.951623+0000","last_became_peered":"2024-04-26T18:24:20.951623+0000","last_unstale":"2024-04-26T18:24:22.962859+0000","last_undegraded":"2024-04-26T18:24:22.962859+0000","last_fullsized":"2024-04-26T18:24:22.962859+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:02.502261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.293188+0000","last_change":"2024-04-26T18:24:20.962274+0000","last_active":"2024-04-26T18:25:05.293188+0000","last_peered":"2024-04-26T18:25:05.293188+0000","last_clean":"2024-04-26T18:25:05.293188+0000","last_became_active":"2024-04-26T18:24:20.961920+0000","last_became_peered":"2024-04-26T18:24:20.961920+0000","last_unstale":"2024-04-26T18:25:05.293188+0000","last_undegraded":"2024-04-26T18:25:05.293188+0000","last_fullsized":"2024-04-26T18:25:05.293188+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:49:46.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.958453+0000","last_change":"2024-04-26T18:24:20.949096+0000","last_active":"2024-04-26T18:24:22.958453+0000","last_peered":"2024-04-26T18:24:22.958453+0000","last_clean":"2024-04-26T18:24:22.958453+0000","last_became_active":"2024-04-26T18:24:20.948737+0000","last_became_peered":"2024-04-26T18:24:20.948737+0000","last_unstale":"2024-04-26T18:24:22.958453+0000","last_undegraded":"2024-04-26T18:24:22.958453+0000","last_fullsized":"2024-04-26T18:24:22.958453+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:12:40.371589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959537+0000","last_change":"2024-04-26T18:24:20.962670+0000","last_active":"2024-04-26T18:24:22.959537+0000","last_peered":"2024-04-26T18:24:22.959537+0000","last_clean":"2024-04-26T18:24:22.959537+0000","last_became_active":"2024-04-26T18:24:20.962309+0000","last_became_peered":"2024-04-26T18:24:20.962309+0000","last_unstale":"2024-04-26T18:24:22.959537+0000","last_undegraded":"2024-04-26T18:24:22.959537+0000","last_fullsized":"2024-04-26T18:24:22.959537+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:40.315864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"peering","last_fresh":"2024-04-26T18:24:22.952465+0000","last_change":"2024-04-26T18:25:05.290871+0000","last_active":"2024-04-26T18:25:05.290871+0000","last_peered":"2024-04-26T18:24:22.952465+0000","last_clean":"2024-04-26T18:24:22.952465+0000","last_became_active":"2024-04-26T18:24:20.946249+0000","last_became_peered":"2024-04-26T18:24:20.946249+0000","last_unstale":"2024-04-26T18:24:22.952465+0000","last_undegraded":"2024-04-26T18:24:22.952465+0000","last_fullsized":"2024-04-26T18:24:22.952465+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959797+0000","last_change":"2024-04-26T18:24:20.961096+0000","last_active":"2024-04-26T18:24:22.959797+0000","last_peered":"2024-04-26T18:24:22.959797+0000","last_clean":"2024-04-26T18:24:22.959797+0000","last_became_active":"2024-04-26T18:24:20.960796+0000","last_became_peered":"2024-04-26T18:24:20.960796+0000","last_unstale":"2024-04-26T18:24:22.959797+0000","last_undegraded":"2024-04-26T18:24:22.959797+0000","last_fullsized":"2024-04-26T18:24:22.959797+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:04.538836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.963547+0000","last_change":"2024-04-26T18:24:20.959871+0000","last_active":"2024-04-26T18:24:22.963547+0000","last_peered":"2024-04-26T18:24:22.963547+0000","last_clean":"2024-04-26T18:24:22.963547+0000","last_became_active":"2024-04-26T18:24:20.959404+0000","last_became_peered":"2024-04-26T18:24:20.959404+0000","last_unstale":"2024-04-26T18:24:22.963547+0000","last_undegraded":"2024-04-26T18:24:22.963547+0000","last_fullsized":"2024-04-26T18:24:22.963547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:43.053618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.961334+0000","last_change":"2024-04-26T18:24:20.963815+0000","last_active":"2024-04-26T18:24:22.961334+0000","last_peered":"2024-04-26T18:24:22.961334+0000","last_clean":"2024-04-26T18:24:22.961334+0000","last_became_active":"2024-04-26T18:24:20.963362+0000","last_became_peered":"2024-04-26T18:24:20.963362+0000","last_unstale":"2024-04-26T18:24:22.961334+0000","last_undegraded":"2024-04-26T18:24:22.961334+0000","last_fullsized":"2024-04-26T18:24:22.961334+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:57.907485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290523+0000","last_change":"2024-04-26T18:24:20.952109+0000","last_active":"2024-04-26T18:25:05.290523+0000","last_peered":"2024-04-26T18:25:05.290523+0000","last_clean":"2024-04-26T18:25:05.290523+0000","last_became_active":"2024-04-26T18:24:20.951776+0000","last_became_peered":"2024-04-26T18:24:20.951776+0000","last_unstale":"2024-04-26T18:25:05.290523+0000","last_undegraded":"2024-04-26T18:25:05.290523+0000","last_fullsized":"2024-04-26T18:25:05.290523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:53.220246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.292933+0000","last_change":"2024-04-26T18:24:20.963965+0000","last_active":"2024-04-26T18:25:05.292933+0000","last_peered":"2024-04-26T18:25:05.292933+0000","last_clean":"2024-04-26T18:25:05.292933+0000","last_became_active":"2024-04-26T18:24:20.963648+0000","last_became_peered":"2024-04-26T18:24:20.963648+0000","last_unstale":"2024-04-26T18:25:05.292933+0000","last_undegraded":"2024-04-26T18:25:05.292933+0000","last_fullsized":"2024-04-26T18:25:05.292933+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:04.080813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959580+0000","last_change":"2024-04-26T18:24:20.954186+0000","last_active":"2024-04-26T18:24:22.959580+0000","last_peered":"2024-04-26T18:24:22.959580+0000","last_clean":"2024-04-26T18:24:22.959580+0000","last_became_active":"2024-04-26T18:24:20.953431+0000","last_became_peered":"2024-04-26T18:24:20.953431+0000","last_unstale":"2024-04-26T18:24:22.959580+0000","last_undegraded":"2024-04-26T18:24:22.959580+0000","last_fullsized":"2024-04-26T18:24:22.959580+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:21.498476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.962735+0000","last_change":"2024-04-26T18:24:20.945392+0000","last_active":"2024-04-26T18:24:22.962735+0000","last_peered":"2024-04-26T18:24:22.962735+0000","last_clean":"2024-04-26T18:24:22.962735+0000","last_became_active":"2024-04-26T18:24:20.944971+0000","last_became_peered":"2024-04-26T18:24:20.944971+0000","last_unstale":"2024-04-26T18:24:22.962735+0000","last_undegraded":"2024-04-26T18:24:22.962735+0000","last_fullsized":"2024-04-26T18:24:22.962735+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:06.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.962504+0000","last_change":"2024-04-26T18:24:20.959209+0000","last_active":"2024-04-26T18:24:22.962504+0000","last_peered":"2024-04-26T18:24:22.962504+0000","last_clean":"2024-04-26T18:24:22.962504+0000","last_became_active":"2024-04-26T18:24:20.958951+0000","last_became_peered":"2024-04-26T18:24:20.958951+0000","last_unstale":"2024-04-26T18:24:22.962504+0000","last_undegraded":"2024-04-26T18:24:22.962504+0000","last_fullsized":"2024-04-26T18:24:22.962504+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:44:15.053984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.962604+0000","last_change":"2024-04-26T18:24:20.954610+0000","last_active":"2024-04-26T18:24:22.962604+0000","last_peered":"2024-04-26T18:24:22.962604+0000","last_clean":"2024-04-26T18:24:22.962604+0000","last_became_active":"2024-04-26T18:24:20.954239+0000","last_became_peered":"2024-04-26T18:24:20.954239+0000","last_unstale":"2024-04-26T18:24:22.962604+0000","last_undegraded":"2024-04-26T18:24:22.962604+0000","last_fullsized":"2024-04-26T18:24:22.962604+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:41.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.295271+0000","last_change":"2024-04-26T18:24:20.956932+0000","last_active":"2024-04-26T18:25:05.295271+0000","last_peered":"2024-04-26T18:25:05.295271+0000","last_clean":"2024-04-26T18:25:05.295271+0000","last_became_active":"2024-04-26T18:24:20.956464+0000","last_became_peered":"2024-04-26T18:24:20.956464+0000","last_unstale":"2024-04-26T18:25:05.295271+0000","last_undegraded":"2024-04-26T18:25:05.295271+0000","last_fullsized":"2024-04-26T18:25:05.295271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:01.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.953700+0000","last_change":"2024-04-26T18:24:20.954870+0000","last_active":"2024-04-26T18:24:22.953700+0000","last_peered":"2024-04-26T18:24:22.953700+0000","last_clean":"2024-04-26T18:24:22.953700+0000","last_became_active":"2024-04-26T18:24:20.954542+0000","last_became_peered":"2024-04-26T18:24:20.954542+0000","last_unstale":"2024-04-26T18:24:22.953700+0000","last_undegraded":"2024-04-26T18:24:22.953700+0000","last_fullsized":"2024-04-26T18:24:22.953700+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:13:51.932625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.953299+0000","last_change":"2024-04-26T18:24:20.950994+0000","last_active":"2024-04-26T18:24:22.953299+0000","last_peered":"2024-04-26T18:24:22.953299+0000","last_clean":"2024-04-26T18:24:22.953299+0000","last_became_active":"2024-04-26T18:24:20.950343+0000","last_became_peered":"2024-04-26T18:24:20.950343+0000","last_unstale":"2024-04-26T18:24:22.953299+0000","last_undegraded":"2024-04-26T18:24:22.953299+0000","last_fullsized":"2024-04-26T18:24:22.953299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:25.274049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":60,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:25:01.340514+0000","last_change":"2024-04-26T18:24:17.929731+0000","last_active":"2024-04-26T18:25:01.340514+0000","last_peered":"2024-04-26T18:25:01.340514+0000","last_clean":"2024-04-26T18:25:01.340514+0000","last_became_active":"2024-04-26T18:24:17.929379+0000","last_became_peered":"2024-04-26T18:24:17.929379+0000","last_unstale":"2024-04-26T18:25:01.340514+0000","last_undegraded":"2024-04-26T18:25:01.340514+0000","last_fullsized":"2024-04-26T18:25:01.340514+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:44.821660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.958930+0000","last_change":"2024-04-26T18:24:20.962885+0000","last_active":"2024-04-26T18:24:22.958930+0000","last_peered":"2024-04-26T18:24:22.958930+0000","last_clean":"2024-04-26T18:24:22.958930+0000","last_became_active":"2024-04-26T18:24:20.962674+0000","last_became_peered":"2024-04-26T18:24:20.962674+0000","last_unstale":"2024-04-26T18:24:22.958930+0000","last_undegraded":"2024-04-26T18:24:22.958930+0000","last_fullsized":"2024-04-26T18:24:22.958930+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:32.444889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.966367+0000","last_change":"2024-04-26T18:24:18.141500+0000","last_active":"2024-04-26T18:24:22.966367+0000","last_peered":"2024-04-26T18:24:22.966367+0000","last_clean":"2024-04-26T18:24:22.966367+0000","last_became_active":"2024-04-26T18:24:18.140870+0000","last_became_peered":"2024-04-26T18:24:18.140870+0000","last_unstale":"2024-04-26T18:24:22.966367+0000","last_undegraded":"2024-04-26T18:24:22.966367+0000","last_fullsized":"2024-04-26T18:24:22.966367+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:13.821872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.963352+0000","last_change":"2024-04-26T18:24:20.949183+0000","last_active":"2024-04-26T18:24:22.963352+0000","last_peered":"2024-04-26T18:24:22.963352+0000","last_clean":"2024-04-26T18:24:22.963352+0000","last_became_active":"2024-04-26T18:24:20.948677+0000","last_became_peered":"2024-04-26T18:24:20.948677+0000","last_unstale":"2024-04-26T18:24:22.963352+0000","last_undegraded":"2024-04-26T18:24:22.963352+0000","last_fullsized":"2024-04-26T18:24:22.963352+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:55.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:39.309342+0000","last_change":"2024-04-26T18:24:17.932830+0000","last_active":"2024-04-26T18:24:39.309342+0000","last_peered":"2024-04-26T18:24:39.309342+0000","last_clean":"2024-04-26T18:24:39.309342+0000","last_became_active":"2024-04-26T18:24:17.932522+0000","last_became_peered":"2024-04-26T18:24:17.932522+0000","last_unstale":"2024-04-26T18:24:39.309342+0000","last_undegraded":"2024-04-26T18:24:39.309342+0000","last_fullsized":"2024-04-26T18:24:39.309342+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:15:27.256940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.963329+0000","last_change":"2024-04-26T18:24:20.955555+0000","last_active":"2024-04-26T18:24:22.963329+0000","last_peered":"2024-04-26T18:24:22.963329+0000","last_clean":"2024-04-26T18:24:22.963329+0000","last_became_active":"2024-04-26T18:24:20.955176+0000","last_became_peered":"2024-04-26T18:24:20.955176+0000","last_unstale":"2024-04-26T18:24:22.963329+0000","last_undegraded":"2024-04-26T18:24:22.963329+0000","last_fullsized":"2024-04-26T18:24:22.963329+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:02:30.368971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:38.300894+0000","last_change":"2024-04-26T18:24:17.930919+0000","last_active":"2024-04-26T18:24:38.300894+0000","last_peered":"2024-04-26T18:24:38.300894+0000","last_clean":"2024-04-26T18:24:38.300894+0000","last_became_active":"2024-04-26T18:24:17.930596+0000","last_became_peered":"2024-04-26T18:24:17.930596+0000","last_unstale":"2024-04-26T18:24:38.300894+0000","last_undegraded":"2024-04-26T18:24:38.300894+0000","last_fullsized":"2024-04-26T18:24:38.300894+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:57.476027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.960323+0000","last_change":"2024-04-26T18:24:20.955796+0000","last_active":"2024-04-26T18:24:22.960323+0000","last_peered":"2024-04-26T18:24:22.960323+0000","last_clean":"2024-04-26T18:24:22.960323+0000","last_became_active":"2024-04-26T18:24:20.955432+0000","last_became_peered":"2024-04-26T18:24:20.955432+0000","last_unstale":"2024-04-26T18:24:22.960323+0000","last_undegraded":"2024-04-26T18:24:22.960323+0000","last_fullsized":"2024-04-26T18:24:22.960323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:25.161797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:25:03.344282+0000","last_change":"2024-04-26T18:24:17.929505+0000","last_active":"2024-04-26T18:25:03.344282+0000","last_peered":"2024-04-26T18:25:03.344282+0000","last_clean":"2024-04-26T18:25:03.344282+0000","last_became_active":"2024-04-26T18:24:17.929205+0000","last_became_peered":"2024-04-26T18:24:17.929205+0000","last_unstale":"2024-04-26T18:25:03.344282+0000","last_undegraded":"2024-04-26T18:25:03.344282+0000","last_fullsized":"2024-04-26T18:25:03.344282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:42.460063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.957764+0000","last_change":"2024-04-26T18:24:20.952134+0000","last_active":"2024-04-26T18:24:22.957764+0000","last_peered":"2024-04-26T18:24:22.957764+0000","last_clean":"2024-04-26T18:24:22.957764+0000","last_became_active":"2024-04-26T18:24:20.951758+0000","last_became_peered":"2024-04-26T18:24:20.951758+0000","last_unstale":"2024-04-26T18:24:22.957764+0000","last_undegraded":"2024-04-26T18:24:22.957764+0000","last_fullsized":"2024-04-26T18:24:22.957764+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:25.389221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:37.296052+0000","last_change":"2024-04-26T18:24:17.928704+0000","last_active":"2024-04-26T18:24:37.296052+0000","last_peered":"2024-04-26T18:24:37.296052+0000","last_clean":"2024-04-26T18:24:37.296052+0000","last_became_active":"2024-04-26T18:24:17.928343+0000","last_became_peered":"2024-04-26T18:24:17.928343+0000","last_unstale":"2024-04-26T18:24:37.296052+0000","last_undegraded":"2024-04-26T18:24:37.296052+0000","last_fullsized":"2024-04-26T18:24:37.296052+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:49.400481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.960263+0000","last_change":"2024-04-26T18:24:20.956808+0000","last_active":"2024-04-26T18:24:22.960263+0000","last_peered":"2024-04-26T18:24:22.960263+0000","last_clean":"2024-04-26T18:24:22.960263+0000","last_became_active":"2024-04-26T18:24:20.956466+0000","last_became_peered":"2024-04-26T18:24:20.956466+0000","last_unstale":"2024-04-26T18:24:22.960263+0000","last_undegraded":"2024-04-26T18:24:22.960263+0000","last_fullsized":"2024-04-26T18:24:22.960263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:31.150670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:36.282580+0000","last_change":"2024-04-26T18:24:17.929134+0000","last_active":"2024-04-26T18:24:36.282580+0000","last_peered":"2024-04-26T18:24:36.282580+0000","last_clean":"2024-04-26T18:24:36.282580+0000","last_became_active":"2024-04-26T18:24:17.928798+0000","last_became_peered":"2024-04-26T18:24:17.928798+0000","last_unstale":"2024-04-26T18:24:36.282580+0000","last_undegraded":"2024-04-26T18:24:36.282580+0000","last_fullsized":"2024-04-26T18:24:36.282580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:10.450825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959549+0000","last_change":"2024-04-26T18:24:20.951594+0000","last_active":"2024-04-26T18:24:22.959549+0000","last_peered":"2024-04-26T18:24:22.959549+0000","last_clean":"2024-04-26T18:24:22.959549+0000","last_became_active":"2024-04-26T18:24:20.951148+0000","last_became_peered":"2024-04-26T18:24:20.951148+0000","last_unstale":"2024-04-26T18:24:22.959549+0000","last_undegraded":"2024-04-26T18:24:22.959549+0000","last_fullsized":"2024-04-26T18:24:22.959549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:18.703292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:38.303074+0000","last_change":"2024-04-26T18:24:17.931101+0000","last_active":"2024-04-26T18:24:38.303074+0000","last_peered":"2024-04-26T18:24:38.303074+0000","last_clean":"2024-04-26T18:24:38.303074+0000","last_became_active":"2024-04-26T18:24:17.930819+0000","last_became_peered":"2024-04-26T18:24:17.930819+0000","last_unstale":"2024-04-26T18:24:38.303074+0000","last_undegraded":"2024-04-26T18:24:38.303074+0000","last_fullsized":"2024-04-26T18:24:38.303074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.953454+0000","last_change":"2024-04-26T18:24:20.948866+0000","last_active":"2024-04-26T18:24:22.953454+0000","last_peered":"2024-04-26T18:24:22.953454+0000","last_clean":"2024-04-26T18:24:22.953454+0000","last_became_active":"2024-04-26T18:24:20.948562+0000","last_became_peered":"2024-04-26T18:24:20.948562+0000","last_unstale":"2024-04-26T18:24:22.953454+0000","last_undegraded":"2024-04-26T18:24:22.953454+0000","last_fullsized":"2024-04-26T18:24:22.953454+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:33:19.616660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":76,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:25:03.343542+0000","last_change":"2024-04-26T18:24:17.926384+0000","last_active":"2024-04-26T18:25:03.343542+0000","last_peered":"2024-04-26T18:25:03.343542+0000","last_clean":"2024-04-26T18:25:03.343542+0000","last_became_active":"2024-04-26T18:24:17.926037+0000","last_became_peered":"2024-04-26T18:24:17.926037+0000","last_unstale":"2024-04-26T18:25:03.343542+0000","last_undegraded":"2024-04-26T18:25:03.343542+0000","last_fullsized":"2024-04-26T18:25:03.343542+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:05.283755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.963496+0000","last_change":"2024-04-26T18:24:20.954257+0000","last_active":"2024-04-26T18:24:22.963496+0000","last_peered":"2024-04-26T18:24:22.963496+0000","last_clean":"2024-04-26T18:24:22.963496+0000","last_became_active":"2024-04-26T18:24:20.953913+0000","last_became_peered":"2024-04-26T18:24:20.953913+0000","last_unstale":"2024-04-26T18:24:22.963496+0000","last_undegraded":"2024-04-26T18:24:22.963496+0000","last_fullsized":"2024-04-26T18:24:22.963496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:06.814156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290317+0000","last_change":"2024-04-26T18:24:17.932843+0000","last_active":"2024-04-26T18:25:05.290317+0000","last_peered":"2024-04-26T18:25:05.290317+0000","last_clean":"2024-04-26T18:25:05.290317+0000","last_became_active":"2024-04-26T18:24:17.932502+0000","last_became_peered":"2024-04-26T18:24:17.932502+0000","last_unstale":"2024-04-26T18:25:05.290317+0000","last_undegraded":"2024-04-26T18:25:05.290317+0000","last_fullsized":"2024-04-26T18:25:05.290317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:49.727094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.963295+0000","last_change":"2024-04-26T18:24:20.959881+0000","last_active":"2024-04-26T18:24:22.963295+0000","last_peered":"2024-04-26T18:24:22.963295+0000","last_clean":"2024-04-26T18:24:22.963295+0000","last_became_active":"2024-04-26T18:24:20.959541+0000","last_became_peered":"2024-04-26T18:24:20.959541+0000","last_unstale":"2024-04-26T18:24:22.963295+0000","last_undegraded":"2024-04-26T18:24:22.963295+0000","last_fullsized":"2024-04-26T18:24:22.963295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:23.393521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:37.294338+0000","last_change":"2024-04-26T18:24:18.142146+0000","last_active":"2024-04-26T18:24:37.294338+0000","last_peered":"2024-04-26T18:24:37.294338+0000","last_clean":"2024-04-26T18:24:37.294338+0000","last_became_active":"2024-04-26T18:24:18.141829+0000","last_became_peered":"2024-04-26T18:24:18.141829+0000","last_unstale":"2024-04-26T18:24:37.294338+0000","last_undegraded":"2024-04-26T18:24:37.294338+0000","last_fullsized":"2024-04-26T18:24:37.294338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:49.341638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.963635+0000","last_change":"2024-04-26T18:24:20.957993+0000","last_active":"2024-04-26T18:24:22.963635+0000","last_peered":"2024-04-26T18:24:22.963635+0000","last_clean":"2024-04-26T18:24:22.963635+0000","last_became_active":"2024-04-26T18:24:20.957443+0000","last_became_peered":"2024-04-26T18:24:20.957443+0000","last_unstale":"2024-04-26T18:24:22.963635+0000","last_undegraded":"2024-04-26T18:24:22.963635+0000","last_fullsized":"2024-04-26T18:24:22.963635+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:06:32.960546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'27","reported_seq":106,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:25:03.344083+0000","last_change":"2024-04-26T18:24:17.931283+0000","last_active":"2024-04-26T18:25:03.344083+0000","last_peered":"2024-04-26T18:25:03.344083+0000","last_clean":"2024-04-26T18:25:03.344083+0000","last_became_active":"2024-04-26T18:24:17.930943+0000","last_became_peered":"2024-04-26T18:24:17.930943+0000","last_unstale":"2024-04-26T18:25:03.344083+0000","last_undegraded":"2024-04-26T18:25:03.344083+0000","last_fullsized":"2024-04-26T18:25:03.344083+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:50.511677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":54367,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":57,"num_read_kb":81,"num_write":29,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959506+0000","last_change":"2024-04-26T18:24:20.947509+0000","last_active":"2024-04-26T18:24:22.959506+0000","last_peered":"2024-04-26T18:24:22.959506+0000","last_clean":"2024-04-26T18:24:22.959506+0000","last_became_active":"2024-04-26T18:24:20.947181+0000","last_became_peered":"2024-04-26T18:24:20.947181+0000","last_unstale":"2024-04-26T18:24:22.959506+0000","last_undegraded":"2024-04-26T18:24:22.959506+0000","last_fullsized":"2024-04-26T18:24:22.959506+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.262312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:39.309250+0000","last_change":"2024-04-26T18:24:17.934042+0000","last_active":"2024-04-26T18:24:39.309250+0000","last_peered":"2024-04-26T18:24:39.309250+0000","last_clean":"2024-04-26T18:24:39.309250+0000","last_became_active":"2024-04-26T18:24:17.933537+0000","last_became_peered":"2024-04-26T18:24:17.933537+0000","last_unstale":"2024-04-26T18:24:39.309250+0000","last_undegraded":"2024-04-26T18:24:39.309250+0000","last_fullsized":"2024-04-26T18:24:39.309250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:35:06.904160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959689+0000","last_change":"2024-04-26T18:24:20.957672+0000","last_active":"2024-04-26T18:24:22.959689+0000","last_peered":"2024-04-26T18:24:22.959689+0000","last_clean":"2024-04-26T18:24:22.959689+0000","last_became_active":"2024-04-26T18:24:20.957214+0000","last_became_peered":"2024-04-26T18:24:20.957214+0000","last_unstale":"2024-04-26T18:24:22.959689+0000","last_undegraded":"2024-04-26T18:24:22.959689+0000","last_fullsized":"2024-04-26T18:24:22.959689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:42:35.341858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:38.303247+0000","last_change":"2024-04-26T18:24:17.930324+0000","last_active":"2024-04-26T18:24:38.303247+0000","last_peered":"2024-04-26T18:24:38.303247+0000","last_clean":"2024-04-26T18:24:38.303247+0000","last_became_active":"2024-04-26T18:24:17.930022+0000","last_became_peered":"2024-04-26T18:24:17.930022+0000","last_unstale":"2024-04-26T18:24:38.303247+0000","last_undegraded":"2024-04-26T18:24:38.303247+0000","last_fullsized":"2024-04-26T18:24:38.303247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:36.864816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"peering","last_fresh":"2024-04-26T18:24:22.953034+0000","last_change":"2024-04-26T18:25:05.290798+0000","last_active":"2024-04-26T18:25:05.290798+0000","last_peered":"2024-04-26T18:24:22.953034+0000","last_clean":"2024-04-26T18:24:22.953034+0000","last_became_active":"2024-04-26T18:24:20.958961+0000","last_became_peered":"2024-04-26T18:24:20.958961+0000","last_unstale":"2024-04-26T18:24:22.953034+0000","last_undegraded":"2024-04-26T18:24:22.953034+0000","last_fullsized":"2024-04-26T18:24:22.953034+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.292879+0000","last_change":"2024-04-26T18:24:17.934676+0000","last_active":"2024-04-26T18:25:05.292879+0000","last_peered":"2024-04-26T18:25:05.292879+0000","last_clean":"2024-04-26T18:25:05.292879+0000","last_became_active":"2024-04-26T18:24:17.934307+0000","last_became_peered":"2024-04-26T18:24:17.934307+0000","last_unstale":"2024-04-26T18:25:05.292879+0000","last_undegraded":"2024-04-26T18:25:05.292879+0000","last_fullsized":"2024-04-26T18:25:05.292879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:27.312248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290136+0000","last_change":"2024-04-26T18:24:20.947371+0000","last_active":"2024-04-26T18:25:05.290136+0000","last_peered":"2024-04-26T18:25:05.290136+0000","last_clean":"2024-04-26T18:25:05.290136+0000","last_became_active":"2024-04-26T18:24:20.947048+0000","last_became_peered":"2024-04-26T18:24:20.947048+0000","last_unstale":"2024-04-26T18:25:05.290136+0000","last_undegraded":"2024-04-26T18:25:05.290136+0000","last_fullsized":"2024-04-26T18:25:05.290136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:55.038074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":40,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:38.301758+0000","last_change":"2024-04-26T18:24:18.142680+0000","last_active":"2024-04-26T18:24:38.301758+0000","last_peered":"2024-04-26T18:24:38.301758+0000","last_clean":"2024-04-26T18:24:38.301758+0000","last_became_active":"2024-04-26T18:24:18.142392+0000","last_became_peered":"2024-04-26T18:24:18.142392+0000","last_unstale":"2024-04-26T18:24:38.301758+0000","last_undegraded":"2024-04-26T18:24:38.301758+0000","last_fullsized":"2024-04-26T18:24:38.301758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:52.654813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959476+0000","last_change":"2024-04-26T18:24:20.958131+0000","last_active":"2024-04-26T18:24:22.959476+0000","last_peered":"2024-04-26T18:24:22.959476+0000","last_clean":"2024-04-26T18:24:22.959476+0000","last_became_active":"2024-04-26T18:24:20.957701+0000","last_became_peered":"2024-04-26T18:24:20.957701+0000","last_unstale":"2024-04-26T18:24:22.959476+0000","last_undegraded":"2024-04-26T18:24:22.959476+0000","last_fullsized":"2024-04-26T18:24:22.959476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:02.018158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":2,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-26T18:24:22.966802+0000","last_change":"2024-04-26T18:25:05.511258+0000","last_active":"2024-04-26T18:25:05.511258+0000","last_peered":"2024-04-26T18:24:22.966802+0000","last_clean":"2024-04-26T18:24:22.966802+0000","last_became_active":"2024-04-26T18:24:17.928840+0000","last_became_peered":"2024-04-26T18:24:17.928840+0000","last_unstale":"2024-04-26T18:24:22.966802+0000","last_undegraded":"2024-04-26T18:24:22.966802+0000","last_fullsized":"2024-04-26T18:24:22.966802+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.295124+0000","last_change":"2024-04-26T18:24:20.961230+0000","last_active":"2024-04-26T18:25:05.295124+0000","last_peered":"2024-04-26T18:25:05.295124+0000","last_clean":"2024-04-26T18:25:05.295124+0000","last_became_active":"2024-04-26T18:24:20.960807+0000","last_became_peered":"2024-04-26T18:24:20.960807+0000","last_unstale":"2024-04-26T18:25:05.295124+0000","last_undegraded":"2024-04-26T18:25:05.295124+0000","last_fullsized":"2024-04-26T18:25:05.295124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:39.535100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.953583+0000","last_change":"2024-04-26T18:24:18.142082+0000","last_active":"2024-04-26T18:24:22.953583+0000","last_peered":"2024-04-26T18:24:22.953583+0000","last_clean":"2024-04-26T18:24:22.953583+0000","last_became_active":"2024-04-26T18:24:18.141765+0000","last_became_peered":"2024-04-26T18:24:18.141765+0000","last_unstale":"2024-04-26T18:24:22.953583+0000","last_undegraded":"2024-04-26T18:24:22.953583+0000","last_fullsized":"2024-04-26T18:24:22.953583+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:02.140027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":17,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:23.583795+0000","last_change":"2024-04-26T18:24:20.952867+0000","last_active":"2024-04-26T18:24:23.583795+0000","last_peered":"2024-04-26T18:24:23.583795+0000","last_clean":"2024-04-26T18:24:23.583795+0000","last_became_active":"2024-04-26T18:24:20.952525+0000","last_became_peered":"2024-04-26T18:24:20.952525+0000","last_unstale":"2024-04-26T18:24:23.583795+0000","last_undegraded":"2024-04-26T18:24:23.583795+0000","last_fullsized":"2024-04-26T18:24:23.583795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:52.804312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290664+0000","last_change":"2024-04-26T18:24:17.932270+0000","last_active":"2024-04-26T18:25:05.290664+0000","last_peered":"2024-04-26T18:25:05.290664+0000","last_clean":"2024-04-26T18:25:05.290664+0000","last_became_active":"2024-04-26T18:24:17.931901+0000","last_became_peered":"2024-04-26T18:24:17.931901+0000","last_unstale":"2024-04-26T18:25:05.290664+0000","last_undegraded":"2024-04-26T18:25:05.290664+0000","last_fullsized":"2024-04-26T18:25:05.290664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:13.524174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.958693+0000","last_change":"2024-04-26T18:24:20.960612+0000","last_active":"2024-04-26T18:24:22.958693+0000","last_peered":"2024-04-26T18:24:22.958693+0000","last_clean":"2024-04-26T18:24:22.958693+0000","last_became_active":"2024-04-26T18:24:20.960280+0000","last_became_peered":"2024-04-26T18:24:20.960280+0000","last_unstale":"2024-04-26T18:24:22.958693+0000","last_undegraded":"2024-04-26T18:24:22.958693+0000","last_fullsized":"2024-04-26T18:24:22.958693+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:53.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.960242+0000","last_change":"2024-04-26T18:24:20.960048+0000","last_active":"2024-04-26T18:24:22.960242+0000","last_peered":"2024-04-26T18:24:22.960242+0000","last_clean":"2024-04-26T18:24:22.960242+0000","last_became_active":"2024-04-26T18:24:20.959715+0000","last_became_peered":"2024-04-26T18:24:20.959715+0000","last_unstale":"2024-04-26T18:24:22.960242+0000","last_undegraded":"2024-04-26T18:24:22.960242+0000","last_fullsized":"2024-04-26T18:24:22.960242+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:30.223161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:25:03.343364+0000","last_change":"2024-04-26T18:24:18.141181+0000","last_active":"2024-04-26T18:25:03.343364+0000","last_peered":"2024-04-26T18:25:03.343364+0000","last_clean":"2024-04-26T18:25:03.343364+0000","last_became_active":"2024-04-26T18:24:18.140854+0000","last_became_peered":"2024-04-26T18:24:18.140854+0000","last_unstale":"2024-04-26T18:25:03.343364+0000","last_undegraded":"2024-04-26T18:25:03.343364+0000","last_fullsized":"2024-04-26T18:25:03.343364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:27.900701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.958994+0000","last_change":"2024-04-26T18:24:20.961653+0000","last_active":"2024-04-26T18:24:22.958994+0000","last_peered":"2024-04-26T18:24:22.958994+0000","last_clean":"2024-04-26T18:24:22.958994+0000","last_became_active":"2024-04-26T18:24:20.961330+0000","last_became_peered":"2024-04-26T18:24:20.961330+0000","last_unstale":"2024-04-26T18:24:22.958994+0000","last_undegraded":"2024-04-26T18:24:22.958994+0000","last_fullsized":"2024-04-26T18:24:22.958994+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:19.033932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.293582+0000","last_change":"2024-04-26T18:24:17.933613+0000","last_active":"2024-04-26T18:25:05.293582+0000","last_peered":"2024-04-26T18:25:05.293582+0000","last_clean":"2024-04-26T18:25:05.293582+0000","last_became_active":"2024-04-26T18:24:17.933219+0000","last_became_peered":"2024-04-26T18:24:17.933219+0000","last_unstale":"2024-04-26T18:25:05.293582+0000","last_undegraded":"2024-04-26T18:25:05.293582+0000","last_fullsized":"2024-04-26T18:25:05.293582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:04.215821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290245+0000","last_change":"2024-04-26T18:22:57.585634+0000","last_active":"2024-04-26T18:25:05.290245+0000","last_peered":"2024-04-26T18:25:05.290245+0000","last_clean":"2024-04-26T18:25:05.290245+0000","last_became_active":"2024-04-26T18:22:57.585198+0000","last_became_peered":"2024-04-26T18:22:57.585198+0000","last_unstale":"2024-04-26T18:25:05.290245+0000","last_undegraded":"2024-04-26T18:25:05.290245+0000","last_fullsized":"2024-04-26T18:25:05.290245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_clean_scrub_stamp":"2024-04-26T18:19:31.594684+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:56:11.861930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959033+0000","last_change":"2024-04-26T18:24:20.950303+0000","last_active":"2024-04-26T18:24:22.959033+0000","last_peered":"2024-04-26T18:24:22.959033+0000","last_clean":"2024-04-26T18:24:22.959033+0000","last_became_active":"2024-04-26T18:24:20.950002+0000","last_became_peered":"2024-04-26T18:24:20.950002+0000","last_unstale":"2024-04-26T18:24:22.959033+0000","last_undegraded":"2024-04-26T18:24:22.959033+0000","last_fullsized":"2024-04-26T18:24:22.959033+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:00.912752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290381+0000","last_change":"2024-04-26T18:24:17.934924+0000","last_active":"2024-04-26T18:25:05.290381+0000","last_peered":"2024-04-26T18:25:05.290381+0000","last_clean":"2024-04-26T18:25:05.290381+0000","last_became_active":"2024-04-26T18:24:17.934598+0000","last_became_peered":"2024-04-26T18:24:17.934598+0000","last_unstale":"2024-04-26T18:25:05.290381+0000","last_undegraded":"2024-04-26T18:25:05.290381+0000","last_fullsized":"2024-04-26T18:25:05.290381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:28.690043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:38.301419+0000","last_change":"2024-04-26T18:24:18.142560+0000","last_active":"2024-04-26T18:24:38.301419+0000","last_peered":"2024-04-26T18:24:38.301419+0000","last_clean":"2024-04-26T18:24:38.301419+0000","last_became_active":"2024-04-26T18:24:18.142184+0000","last_became_peered":"2024-04-26T18:24:18.142184+0000","last_unstale":"2024-04-26T18:24:38.301419+0000","last_undegraded":"2024-04-26T18:24:38.301419+0000","last_fullsized":"2024-04-26T18:24:38.301419+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:26.656113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959122+0000","last_change":"2024-04-26T18:24:20.952073+0000","last_active":"2024-04-26T18:24:22.959122+0000","last_peered":"2024-04-26T18:24:22.959122+0000","last_clean":"2024-04-26T18:24:22.959122+0000","last_became_active":"2024-04-26T18:24:20.951761+0000","last_became_peered":"2024-04-26T18:24:20.951761+0000","last_unstale":"2024-04-26T18:24:22.959122+0000","last_undegraded":"2024-04-26T18:24:22.959122+0000","last_fullsized":"2024-04-26T18:24:22.959122+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:30.244060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'13","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:58.961897+0000","last_change":"2024-04-26T18:24:17.927171+0000","last_active":"2024-04-26T18:24:58.961897+0000","last_peered":"2024-04-26T18:24:58.961897+0000","last_clean":"2024-04-26T18:24:58.961897+0000","last_became_active":"2024-04-26T18:24:17.926816+0000","last_became_peered":"2024-04-26T18:24:17.926816+0000","last_unstale":"2024-04-26T18:24:58.961897+0000","last_undegraded":"2024-04-26T18:24:58.961897+0000","last_fullsized":"2024-04-26T18:24:58.961897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:18.249363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":23,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959469+0000","last_change":"2024-04-26T18:24:20.951684+0000","last_active":"2024-04-26T18:24:22.959469+0000","last_peered":"2024-04-26T18:24:22.959469+0000","last_clean":"2024-04-26T18:24:22.959469+0000","last_became_active":"2024-04-26T18:24:20.951364+0000","last_became_peered":"2024-04-26T18:24:20.951364+0000","last_unstale":"2024-04-26T18:24:22.959469+0000","last_undegraded":"2024-04-26T18:24:22.959469+0000","last_fullsized":"2024-04-26T18:24:22.959469+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:28.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.292911+0000","last_change":"2024-04-26T18:24:18.229770+0000","last_active":"2024-04-26T18:25:05.292911+0000","last_peered":"2024-04-26T18:25:05.292911+0000","last_clean":"2024-04-26T18:25:05.292911+0000","last_became_active":"2024-04-26T18:24:18.229446+0000","last_became_peered":"2024-04-26T18:24:18.229446+0000","last_unstale":"2024-04-26T18:25:05.292911+0000","last_undegraded":"2024-04-26T18:25:05.292911+0000","last_fullsized":"2024-04-26T18:25:05.292911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:14.794797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.963208+0000","last_change":"2024-04-26T18:24:20.960513+0000","last_active":"2024-04-26T18:24:22.963208+0000","last_peered":"2024-04-26T18:24:22.963208+0000","last_clean":"2024-04-26T18:24:22.963208+0000","last_became_active":"2024-04-26T18:24:20.959549+0000","last_became_peered":"2024-04-26T18:24:20.959549+0000","last_unstale":"2024-04-26T18:24:22.963208+0000","last_undegraded":"2024-04-26T18:24:22.963208+0000","last_fullsized":"2024-04-26T18:24:22.963208+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:18:37.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:35.271025+0000","last_change":"2024-04-26T18:24:18.141549+0000","last_active":"2024-04-26T18:24:35.271025+0000","last_peered":"2024-04-26T18:24:35.271025+0000","last_clean":"2024-04-26T18:24:35.271025+0000","last_became_active":"2024-04-26T18:24:18.141172+0000","last_became_peered":"2024-04-26T18:24:18.141172+0000","last_unstale":"2024-04-26T18:24:35.271025+0000","last_undegraded":"2024-04-26T18:24:35.271025+0000","last_fullsized":"2024-04-26T18:24:35.271025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:15.878130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.957670+0000","last_change":"2024-04-26T18:24:20.948121+0000","last_active":"2024-04-26T18:24:22.957670+0000","last_peered":"2024-04-26T18:24:22.957670+0000","last_clean":"2024-04-26T18:24:22.957670+0000","last_became_active":"2024-04-26T18:24:20.947738+0000","last_became_peered":"2024-04-26T18:24:20.947738+0000","last_unstale":"2024-04-26T18:24:22.957670+0000","last_undegraded":"2024-04-26T18:24:22.957670+0000","last_fullsized":"2024-04-26T18:24:22.957670+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:04.237302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":29,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:38.300352+0000","last_change":"2024-04-26T18:24:17.926631+0000","last_active":"2024-04-26T18:24:38.300352+0000","last_peered":"2024-04-26T18:24:38.300352+0000","last_clean":"2024-04-26T18:24:38.300352+0000","last_became_active":"2024-04-26T18:24:17.926259+0000","last_became_peered":"2024-04-26T18:24:17.926259+0000","last_unstale":"2024-04-26T18:24:38.300352+0000","last_undegraded":"2024-04-26T18:24:38.300352+0000","last_fullsized":"2024-04-26T18:24:38.300352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:47.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.294419+0000","last_change":"2024-04-26T18:24:20.947283+0000","last_active":"2024-04-26T18:25:05.294419+0000","last_peered":"2024-04-26T18:25:05.294419+0000","last_clean":"2024-04-26T18:25:05.294419+0000","last_became_active":"2024-04-26T18:24:20.946812+0000","last_became_peered":"2024-04-26T18:24:20.946812+0000","last_unstale":"2024-04-26T18:25:05.294419+0000","last_undegraded":"2024-04-26T18:25:05.294419+0000","last_fullsized":"2024-04-26T18:25:05.294419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:33.730698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:37.294726+0000","last_change":"2024-04-26T18:24:17.931843+0000","last_active":"2024-04-26T18:24:37.294726+0000","last_peered":"2024-04-26T18:24:37.294726+0000","last_clean":"2024-04-26T18:24:37.294726+0000","last_became_active":"2024-04-26T18:24:17.931442+0000","last_became_peered":"2024-04-26T18:24:17.931442+0000","last_unstale":"2024-04-26T18:24:37.294726+0000","last_undegraded":"2024-04-26T18:24:37.294726+0000","last_fullsized":"2024-04-26T18:24:37.294726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:09.649435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.958787+0000","last_change":"2024-04-26T18:24:20.955661+0000","last_active":"2024-04-26T18:24:22.958787+0000","last_peered":"2024-04-26T18:24:22.958787+0000","last_clean":"2024-04-26T18:24:22.958787+0000","last_became_active":"2024-04-26T18:24:20.955344+0000","last_became_peered":"2024-04-26T18:24:20.955344+0000","last_unstale":"2024-04-26T18:24:22.958787+0000","last_undegraded":"2024-04-26T18:24:22.958787+0000","last_fullsized":"2024-04-26T18:24:22.958787+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:44.274381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":2,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-26T18:24:22.968300+0000","last_change":"2024-04-26T18:25:05.511234+0000","last_active":"2024-04-26T18:25:05.511234+0000","last_peered":"2024-04-26T18:24:22.968300+0000","last_clean":"2024-04-26T18:24:22.968300+0000","last_became_active":"2024-04-26T18:24:17.932354+0000","last_became_peered":"2024-04-26T18:24:17.932354+0000","last_unstale":"2024-04-26T18:24:22.968300+0000","last_undegraded":"2024-04-26T18:24:22.968300+0000","last_fullsized":"2024-04-26T18:24:22.968300+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290753+0000","last_change":"2024-04-26T18:24:20.955746+0000","last_active":"2024-04-26T18:25:05.290753+0000","last_peered":"2024-04-26T18:25:05.290753+0000","last_clean":"2024-04-26T18:25:05.290753+0000","last_became_active":"2024-04-26T18:24:20.953652+0000","last_became_peered":"2024-04-26T18:24:20.953652+0000","last_unstale":"2024-04-26T18:25:05.290753+0000","last_undegraded":"2024-04-26T18:25:05.290753+0000","last_fullsized":"2024-04-26T18:25:05.290753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:31.414185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:25:03.344092+0000","last_change":"2024-04-26T18:24:18.229386+0000","last_active":"2024-04-26T18:25:03.344092+0000","last_peered":"2024-04-26T18:25:03.344092+0000","last_clean":"2024-04-26T18:25:03.344092+0000","last_became_active":"2024-04-26T18:24:18.229082+0000","last_became_peered":"2024-04-26T18:24:18.229082+0000","last_unstale":"2024-04-26T18:25:03.344092+0000","last_undegraded":"2024-04-26T18:25:03.344092+0000","last_fullsized":"2024-04-26T18:25:03.344092+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:13:19.755625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290882+0000","last_change":"2024-04-26T18:24:20.954238+0000","last_active":"2024-04-26T18:25:05.290882+0000","last_peered":"2024-04-26T18:25:05.290882+0000","last_clean":"2024-04-26T18:25:05.290882+0000","last_became_active":"2024-04-26T18:24:20.953883+0000","last_became_peered":"2024-04-26T18:24:20.953883+0000","last_unstale":"2024-04-26T18:25:05.290882+0000","last_undegraded":"2024-04-26T18:25:05.290882+0000","last_fullsized":"2024-04-26T18:25:05.290882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:10.580835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:38.301373+0000","last_change":"2024-04-26T18:24:17.926901+0000","last_active":"2024-04-26T18:24:38.301373+0000","last_peered":"2024-04-26T18:24:38.301373+0000","last_clean":"2024-04-26T18:24:38.301373+0000","last_became_active":"2024-04-26T18:24:17.926291+0000","last_became_peered":"2024-04-26T18:24:17.926291+0000","last_unstale":"2024-04-26T18:24:38.301373+0000","last_undegraded":"2024-04-26T18:24:38.301373+0000","last_fullsized":"2024-04-26T18:24:38.301373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:17.695198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290270+0000","last_change":"2024-04-26T18:24:20.950578+0000","last_active":"2024-04-26T18:25:05.290270+0000","last_peered":"2024-04-26T18:25:05.290270+0000","last_clean":"2024-04-26T18:25:05.290270+0000","last_became_active":"2024-04-26T18:24:20.950174+0000","last_became_peered":"2024-04-26T18:24:20.950174+0000","last_unstale":"2024-04-26T18:25:05.290270+0000","last_undegraded":"2024-04-26T18:25:05.290270+0000","last_fullsized":"2024-04-26T18:25:05.290270+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:38.301172+0000","last_change":"2024-04-26T18:24:17.928054+0000","last_active":"2024-04-26T18:24:38.301172+0000","last_peered":"2024-04-26T18:24:38.301172+0000","last_clean":"2024-04-26T18:24:38.301172+0000","last_became_active":"2024-04-26T18:24:17.927715+0000","last_became_peered":"2024-04-26T18:24:17.927715+0000","last_unstale":"2024-04-26T18:24:38.301172+0000","last_undegraded":"2024-04-26T18:24:38.301172+0000","last_fullsized":"2024-04-26T18:24:38.301172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.155436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290636+0000","last_change":"2024-04-26T18:24:20.944512+0000","last_active":"2024-04-26T18:25:05.290636+0000","last_peered":"2024-04-26T18:25:05.290636+0000","last_clean":"2024-04-26T18:25:05.290636+0000","last_became_active":"2024-04-26T18:24:20.944123+0000","last_became_peered":"2024-04-26T18:24:20.944123+0000","last_unstale":"2024-04-26T18:25:05.290636+0000","last_undegraded":"2024-04-26T18:25:05.290636+0000","last_fullsized":"2024-04-26T18:25:05.290636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.240206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.294617+0000","last_change":"2024-04-26T18:24:20.959138+0000","last_active":"2024-04-26T18:25:05.294617+0000","last_peered":"2024-04-26T18:25:05.294617+0000","last_clean":"2024-04-26T18:25:05.294617+0000","last_became_active":"2024-04-26T18:24:20.958675+0000","last_became_peered":"2024-04-26T18:24:20.958675+0000","last_unstale":"2024-04-26T18:25:05.294617+0000","last_undegraded":"2024-04-26T18:25:05.294617+0000","last_fullsized":"2024-04-26T18:25:05.294617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:34:32.103380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.952807+0000","last_change":"2024-04-26T18:24:20.961689+0000","last_active":"2024-04-26T18:24:22.952807+0000","last_peered":"2024-04-26T18:24:22.952807+0000","last_clean":"2024-04-26T18:24:22.952807+0000","last_became_active":"2024-04-26T18:24:20.961374+0000","last_became_peered":"2024-04-26T18:24:20.961374+0000","last_unstale":"2024-04-26T18:24:22.952807+0000","last_undegraded":"2024-04-26T18:24:22.952807+0000","last_fullsized":"2024-04-26T18:24:22.952807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:55.243887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.952751+0000","last_change":"2024-04-26T18:24:20.950711+0000","last_active":"2024-04-26T18:24:22.952751+0000","last_peered":"2024-04-26T18:24:22.952751+0000","last_clean":"2024-04-26T18:24:22.952751+0000","last_became_active":"2024-04-26T18:24:20.950371+0000","last_became_peered":"2024-04-26T18:24:20.950371+0000","last_unstale":"2024-04-26T18:24:22.952751+0000","last_undegraded":"2024-04-26T18:24:22.952751+0000","last_fullsized":"2024-04-26T18:24:22.952751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:35.307921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.962993+0000","last_change":"2024-04-26T18:24:20.959990+0000","last_active":"2024-04-26T18:24:22.962993+0000","last_peered":"2024-04-26T18:24:22.962993+0000","last_clean":"2024-04-26T18:24:22.962993+0000","last_became_active":"2024-04-26T18:24:20.959514+0000","last_became_peered":"2024-04-26T18:24:20.959514+0000","last_unstale":"2024-04-26T18:24:22.962993+0000","last_undegraded":"2024-04-26T18:24:22.962993+0000","last_fullsized":"2024-04-26T18:24:22.962993+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:00.184448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959261+0000","last_change":"2024-04-26T18:24:20.957338+0000","last_active":"2024-04-26T18:24:22.959261+0000","last_peered":"2024-04-26T18:24:22.959261+0000","last_clean":"2024-04-26T18:24:22.959261+0000","last_became_active":"2024-04-26T18:24:20.956999+0000","last_became_peered":"2024-04-26T18:24:20.956999+0000","last_unstale":"2024-04-26T18:24:22.959261+0000","last_undegraded":"2024-04-26T18:24:22.959261+0000","last_fullsized":"2024-04-26T18:24:22.959261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.066902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.294887+0000","last_change":"2024-04-26T18:24:20.953634+0000","last_active":"2024-04-26T18:25:05.294887+0000","last_peered":"2024-04-26T18:25:05.294887+0000","last_clean":"2024-04-26T18:25:05.294887+0000","last_became_active":"2024-04-26T18:24:20.953259+0000","last_became_peered":"2024-04-26T18:24:20.953259+0000","last_unstale":"2024-04-26T18:25:05.294887+0000","last_undegraded":"2024-04-26T18:25:05.294887+0000","last_fullsized":"2024-04-26T18:25:05.294887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:00.095344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.953399+0000","last_change":"2024-04-26T18:24:20.954997+0000","last_active":"2024-04-26T18:24:22.953399+0000","last_peered":"2024-04-26T18:24:22.953399+0000","last_clean":"2024-04-26T18:24:22.953399+0000","last_became_active":"2024-04-26T18:24:20.954223+0000","last_became_peered":"2024-04-26T18:24:20.954223+0000","last_unstale":"2024-04-26T18:24:22.953399+0000","last_undegraded":"2024-04-26T18:24:22.953399+0000","last_fullsized":"2024-04-26T18:24:22.953399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:14.092455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959255+0000","last_change":"2024-04-26T18:24:20.957056+0000","last_active":"2024-04-26T18:24:22.959255+0000","last_peered":"2024-04-26T18:24:22.959255+0000","last_clean":"2024-04-26T18:24:22.959255+0000","last_became_active":"2024-04-26T18:24:20.956737+0000","last_became_peered":"2024-04-26T18:24:20.956737+0000","last_unstale":"2024-04-26T18:24:22.959255+0000","last_undegraded":"2024-04-26T18:24:22.959255+0000","last_fullsized":"2024-04-26T18:24:22.959255+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:48.358862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959844+0000","last_change":"2024-04-26T18:24:20.957731+0000","last_active":"2024-04-26T18:24:22.959844+0000","last_peered":"2024-04-26T18:24:22.959844+0000","last_clean":"2024-04-26T18:24:22.959844+0000","last_became_active":"2024-04-26T18:24:20.957405+0000","last_became_peered":"2024-04-26T18:24:20.957405+0000","last_unstale":"2024-04-26T18:24:22.959844+0000","last_undegraded":"2024-04-26T18:24:22.959844+0000","last_fullsized":"2024-04-26T18:24:22.959844+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:12:47.483476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959461+0000","last_change":"2024-04-26T18:24:20.956000+0000","last_active":"2024-04-26T18:24:22.959461+0000","last_peered":"2024-04-26T18:24:22.959461+0000","last_clean":"2024-04-26T18:24:22.959461+0000","last_became_active":"2024-04-26T18:24:20.955677+0000","last_became_peered":"2024-04-26T18:24:20.955677+0000","last_unstale":"2024-04-26T18:24:22.959461+0000","last_undegraded":"2024-04-26T18:24:22.959461+0000","last_fullsized":"2024-04-26T18:24:22.959461+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:20.895674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.963031+0000","last_change":"2024-04-26T18:24:20.959317+0000","last_active":"2024-04-26T18:24:22.963031+0000","last_peered":"2024-04-26T18:24:22.963031+0000","last_clean":"2024-04-26T18:24:22.963031+0000","last_became_active":"2024-04-26T18:24:20.958963+0000","last_became_peered":"2024-04-26T18:24:20.958963+0000","last_unstale":"2024-04-26T18:24:22.963031+0000","last_undegraded":"2024-04-26T18:24:22.963031+0000","last_fullsized":"2024-04-26T18:24:22.963031+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:08.231089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959429+0000","last_change":"2024-04-26T18:24:20.954445+0000","last_active":"2024-04-26T18:24:22.959429+0000","last_peered":"2024-04-26T18:24:22.959429+0000","last_clean":"2024-04-26T18:24:22.959429+0000","last_became_active":"2024-04-26T18:24:20.953852+0000","last_became_peered":"2024-04-26T18:24:20.953852+0000","last_unstale":"2024-04-26T18:24:22.959429+0000","last_undegraded":"2024-04-26T18:24:22.959429+0000","last_fullsized":"2024-04-26T18:24:22.959429+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:05.636897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.294705+0000","last_change":"2024-04-26T18:24:20.958023+0000","last_active":"2024-04-26T18:25:05.294705+0000","last_peered":"2024-04-26T18:25:05.294705+0000","last_clean":"2024-04-26T18:25:05.294705+0000","last_became_active":"2024-04-26T18:24:20.957642+0000","last_became_peered":"2024-04-26T18:24:20.957642+0000","last_unstale":"2024-04-26T18:25:05.294705+0000","last_undegraded":"2024-04-26T18:25:05.294705+0000","last_fullsized":"2024-04-26T18:25:05.294705+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:46.332710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244217,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61571,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":4,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":292,"num_read_kb":210,"num_write":237,"num_write_kb":125,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":548864,"data_stored":193418,"data_compressed":5529,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":204,"ondisk_log_size":204,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351000,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":475556,"data_compressed":2885,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612636,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32268,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712884,"statfs":{"total":95995035648,"available":95961993216,"internally_reserved":0,"allocated":1118208,"data_stored":1111844,"data_compressed":37434,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776162,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":995328,"data_stored":466378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037799,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":466756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":1059274,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462962,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":466095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574922,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1032192,"data_stored":521375,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738447,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":467825,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":531082,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149760,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":995328,"data_stored":465920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888124,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":465956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724597,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":1058952,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":56119,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65826,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":56103,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:25:07.148 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:25:07.388 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:07.502 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:07 smithi084 ceph-mon[29141]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T18:25:07.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:07 smithi103 ceph-mon[32727]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T18:25:07.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:07 smithi174 ceph-mon[32705]: osdmap e91: 12 total, 12 up, 12 in 2024-04-26T18:25:08.526 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:08 smithi084 ceph-mon[29141]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:08.526 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:08 smithi084 ceph-mon[29141]: pgmap v322: 97 pgs: 4 peering, 93 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 195 B/s, 0 objects/s recovering 2024-04-26T18:25:08.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:08 smithi103 ceph-mon[32727]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:08.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:08 smithi103 ceph-mon[32727]: pgmap v322: 97 pgs: 4 peering, 93 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 195 B/s, 0 objects/s recovering 2024-04-26T18:25:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:08 smithi174 ceph-mon[32705]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:08 smithi174 ceph-mon[32705]: pgmap v322: 97 pgs: 4 peering, 93 active+clean; 637 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.2 KiB/s wr, 6 op/s; 195 B/s, 0 objects/s recovering 2024-04-26T18:25:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.063+0000 7f21a6a14700 1 -- 172.21.15.84:0/3123077594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0102d80 msgr2=0x7f21a0103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.063+0000 7f21a6a14700 1 --2- 172.21.15.84:0/3123077594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0102d80 0x7f21a0103160 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f21940099c0 tx=0x7f2194009cd0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.063+0000 7f21a6a14700 1 -- 172.21.15.84:0/3123077594 shutdown_connections 2024-04-26T18:25:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.063+0000 7f21a6a14700 1 --2- 172.21.15.84:0/3123077594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0102d80 0x7f21a0103160 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.063+0000 7f21a6a14700 1 --2- 172.21.15.84:0/3123077594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a01076a0 0x7f21a010f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.063+0000 7f21a6a14700 1 --2- 172.21.15.84:0/3123077594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a0103730 0x7f21a0106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.063+0000 7f21a6a14700 1 -- 172.21.15.84:0/3123077594 >> 172.21.15.84:0/3123077594 conn(0x7f21a0078990 msgr2=0x7f21a00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:09.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.063+0000 7f21a6a14700 1 -- 172.21.15.84:0/3123077594 shutdown_connections 2024-04-26T18:25:09.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.063+0000 7f21a6a14700 1 -- 172.21.15.84:0/3123077594 wait complete. 2024-04-26T18:25:09.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.064+0000 7f21a6a14700 1 Processor -- start 2024-04-26T18:25:09.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.064+0000 7f21a6a14700 1 -- start start 2024-04-26T18:25:09.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f21a6a14700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0102d80 0x7f21a019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f21a6a14700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a0103730 0x7f21a019afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f21a6a14700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a01076a0 0x7f21a01a5af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f21a6a14700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f21a01125f0 con 0x7f21a0102d80 2024-04-26T18:25:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f21a6a14700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0112470 con 0x7f21a0103730 2024-04-26T18:25:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f21a6a14700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f21a0112770 con 0x7f21a01076a0 2024-04-26T18:25:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0102d80 0x7f21a019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0102d80 0x7f21a019aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39664/0 (socket says 172.21.15.84:39664) 2024-04-26T18:25:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219ffff700 1 -- 172.21.15.84:0/2883642396 learned_addr learned my addr 172.21.15.84:0/2883642396 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f21a4fb1700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a01076a0 0x7f21a01a5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219f7fe700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a0103730 0x7f21a019afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219ffff700 1 -- 172.21.15.84:0/2883642396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a01076a0 msgr2=0x7f21a01a5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219ffff700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a01076a0 0x7f21a01a5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219ffff700 1 -- 172.21.15.84:0/2883642396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a0103730 msgr2=0x7f21a019afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:09.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219ffff700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a0103730 0x7f21a019afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219ffff700 1 -- 172.21.15.84:0/2883642396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a01a6030 con 0x7f21a0102d80 2024-04-26T18:25:09.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219f7fe700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a0103730 0x7f21a019afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:25:09.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219ffff700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0102d80 0x7f21a019aa70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f219400b580 tx=0x7f21940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:09.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219d7fa700 1 -- 172.21.15.84:0/2883642396 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2194027070 con 0x7f21a0102d80 2024-04-26T18:25:09.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f21a6a14700 1 -- 172.21.15.84:0/2883642396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a01a6200 con 0x7f21a0102d80 2024-04-26T18:25:09.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219d7fa700 1 -- 172.21.15.84:0/2883642396 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21940040a0 con 0x7f21a0102d80 2024-04-26T18:25:09.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.065+0000 7f219d7fa700 1 -- 172.21.15.84:0/2883642396 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f219400f460 con 0x7f21a0102d80 2024-04-26T18:25:09.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.066+0000 7f21a6a14700 1 -- 172.21.15.84:0/2883642396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a01a66b0 con 0x7f21a0102d80 2024-04-26T18:25:09.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.066+0000 7f21a6a14700 1 -- 172.21.15.84:0/2883642396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21a010a930 con 0x7f21a0102d80 2024-04-26T18:25:09.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.067+0000 7f219d7fa700 1 -- 172.21.15.84:0/2883642396 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f219402a070 con 0x7f21a0102d80 2024-04-26T18:25:09.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.068+0000 7f219d7fa700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f21880a3cd0 0x7f21880a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:09.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.068+0000 7f219f7fe700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f21880a3cd0 0x7f21880a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:09.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.068+0000 7f219d7fa700 1 -- 172.21.15.84:0/2883642396 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f21940d15e0 con 0x7f21a0102d80 2024-04-26T18:25:09.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.068+0000 7f219f7fe700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f21880a3cd0 0x7f21880a6190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f21a0104e60 tx=0x7f2190008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:09.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.073+0000 7f219d7fa700 1 -- 172.21.15.84:0/2883642396 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f219405eff0 con 0x7f21a0102d80 2024-04-26T18:25:09.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.270+0000 7f21a6a14700 1 -- 172.21.15.84:0/2883642396 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21a0102000 con 0x7f21880a3cd0 2024-04-26T18:25:09.277 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.277+0000 7f219d7fa700 1 -- 172.21.15.84:0/2883642396 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258867 (secure 0 0 0) 0x7f21a0102000 con 0x7f21880a3cd0 2024-04-26T18:25:09.280 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:25:09.282 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.280+0000 7f21a6a14700 1 -- 172.21.15.84:0/2883642396 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f21880a3cd0 msgr2=0x7f21880a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:09.282 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.280+0000 7f21a6a14700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f21880a3cd0 0x7f21880a6190 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f21a0104e60 tx=0x7f2190008040 comp rx=0 tx=0).stop 2024-04-26T18:25:09.282 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.280+0000 7f21a6a14700 1 -- 172.21.15.84:0/2883642396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0102d80 msgr2=0x7f21a019aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.280+0000 7f21a6a14700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0102d80 0x7f21a019aa70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f219400b580 tx=0x7f21940047b0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.280+0000 7f21a6a14700 1 -- 172.21.15.84:0/2883642396 shutdown_connections 2024-04-26T18:25:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.280+0000 7f21a6a14700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f21880a3cd0 0x7f21880a6190 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.280+0000 7f21a6a14700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f21a0102d80 0x7f21a019aa70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.280+0000 7f21a6a14700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f21a01076a0 0x7f21a01a5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.280+0000 7f21a6a14700 1 --2- 172.21.15.84:0/2883642396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f21a0103730 0x7f21a019afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.280+0000 7f21a6a14700 1 -- 172.21.15.84:0/2883642396 >> 172.21.15.84:0/2883642396 conn(0x7f21a0078990 msgr2=0x7f21a00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.280+0000 7f21a6a14700 1 -- 172.21.15.84:0/2883642396 shutdown_connections 2024-04-26T18:25:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:09.280+0000 7f21a6a14700 1 -- 172.21.15.84:0/2883642396 wait complete. 2024-04-26T18:25:09.283 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-04-26T18:25:09.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:09 smithi084 ceph-mon[29141]: pgmap v323: 97 pgs: 4 peering, 93 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s; 195 B/s, 1 keys/s, 1 objects/s recovering 2024-04-26T18:25:09.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:09 smithi084 ceph-mon[29141]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:09.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:09 smithi103 ceph-mon[32727]: pgmap v323: 97 pgs: 4 peering, 93 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s; 195 B/s, 1 keys/s, 1 objects/s recovering 2024-04-26T18:25:09.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:09 smithi103 ceph-mon[32727]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:09.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:09 smithi174 ceph-mon[32705]: pgmap v323: 97 pgs: 4 peering, 93 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 2.7 KiB/s wr, 7 op/s; 195 B/s, 1 keys/s, 1 objects/s recovering 2024-04-26T18:25:09.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:09 smithi174 ceph-mon[32705]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:09.985 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":323,"stamp":"2024-04-26T18:25:09.180250+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660878,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":4,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":430,"num_read_kb":356,"num_write":427,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1182297,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369588,"kb_used_data":12080,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572236,"statfs":{"total":1151940427776,"available":1151561969664,"internally_reserved":0,"allocated":12369920,"data_stored":7660719,"data_compressed":144065,"data_compressed_allocated":192512,"data_compressed_original":1884160,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17561,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":4,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":58,"num_write":12,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1561,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001936"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.295324+0000","last_change":"2024-04-26T18:24:20.951581+0000","last_active":"2024-04-26T18:25:05.295324+0000","last_peered":"2024-04-26T18:25:05.295324+0000","last_clean":"2024-04-26T18:25:05.295324+0000","last_became_active":"2024-04-26T18:24:20.951260+0000","last_became_peered":"2024-04-26T18:24:20.951260+0000","last_unstale":"2024-04-26T18:25:05.295324+0000","last_undegraded":"2024-04-26T18:25:05.295324+0000","last_fullsized":"2024-04-26T18:25:05.295324+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:54:28.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296510+0000","last_change":"2024-04-26T18:24:20.952263+0000","last_active":"2024-04-26T18:25:06.296510+0000","last_peered":"2024-04-26T18:25:06.296510+0000","last_clean":"2024-04-26T18:25:06.296510+0000","last_became_active":"2024-04-26T18:24:20.951930+0000","last_became_peered":"2024-04-26T18:24:20.951930+0000","last_unstale":"2024-04-26T18:25:06.296510+0000","last_undegraded":"2024-04-26T18:25:06.296510+0000","last_fullsized":"2024-04-26T18:25:06.296510+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:54.816572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296658+0000","last_change":"2024-04-26T18:24:20.952280+0000","last_active":"2024-04-26T18:25:06.296658+0000","last_peered":"2024-04-26T18:25:06.296658+0000","last_clean":"2024-04-26T18:25:06.296658+0000","last_became_active":"2024-04-26T18:24:20.951623+0000","last_became_peered":"2024-04-26T18:24:20.951623+0000","last_unstale":"2024-04-26T18:25:06.296658+0000","last_undegraded":"2024-04-26T18:25:06.296658+0000","last_fullsized":"2024-04-26T18:25:06.296658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:02.502261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.293188+0000","last_change":"2024-04-26T18:24:20.962274+0000","last_active":"2024-04-26T18:25:05.293188+0000","last_peered":"2024-04-26T18:25:05.293188+0000","last_clean":"2024-04-26T18:25:05.293188+0000","last_became_active":"2024-04-26T18:24:20.961920+0000","last_became_peered":"2024-04-26T18:24:20.961920+0000","last_unstale":"2024-04-26T18:25:05.293188+0000","last_undegraded":"2024-04-26T18:25:05.293188+0000","last_fullsized":"2024-04-26T18:25:05.293188+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:49:46.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297438+0000","last_change":"2024-04-26T18:24:20.949096+0000","last_active":"2024-04-26T18:25:06.297438+0000","last_peered":"2024-04-26T18:25:06.297438+0000","last_clean":"2024-04-26T18:25:06.297438+0000","last_became_active":"2024-04-26T18:24:20.948737+0000","last_became_peered":"2024-04-26T18:24:20.948737+0000","last_unstale":"2024-04-26T18:25:06.297438+0000","last_undegraded":"2024-04-26T18:25:06.297438+0000","last_fullsized":"2024-04-26T18:25:06.297438+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:12:40.371589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351373+0000","last_change":"2024-04-26T18:24:20.962670+0000","last_active":"2024-04-26T18:25:06.351373+0000","last_peered":"2024-04-26T18:25:06.351373+0000","last_clean":"2024-04-26T18:25:06.351373+0000","last_became_active":"2024-04-26T18:24:20.962309+0000","last_became_peered":"2024-04-26T18:24:20.962309+0000","last_unstale":"2024-04-26T18:25:06.351373+0000","last_undegraded":"2024-04-26T18:25:06.351373+0000","last_fullsized":"2024-04-26T18:25:06.351373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:40.315864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"peering","last_fresh":"2024-04-26T18:24:22.952465+0000","last_change":"2024-04-26T18:25:05.290871+0000","last_active":"2024-04-26T18:25:05.290871+0000","last_peered":"2024-04-26T18:24:22.952465+0000","last_clean":"2024-04-26T18:24:22.952465+0000","last_became_active":"2024-04-26T18:24:20.946249+0000","last_became_peered":"2024-04-26T18:24:20.946249+0000","last_unstale":"2024-04-26T18:24:22.952465+0000","last_undegraded":"2024-04-26T18:24:22.952465+0000","last_fullsized":"2024-04-26T18:24:22.952465+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5,6,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754918+0000","last_change":"2024-04-26T18:24:20.961096+0000","last_active":"2024-04-26T18:25:06.754918+0000","last_peered":"2024-04-26T18:25:06.754918+0000","last_clean":"2024-04-26T18:25:06.754918+0000","last_became_active":"2024-04-26T18:24:20.960796+0000","last_became_peered":"2024-04-26T18:24:20.960796+0000","last_unstale":"2024-04-26T18:25:06.754918+0000","last_undegraded":"2024-04-26T18:25:06.754918+0000","last_fullsized":"2024-04-26T18:25:06.754918+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:04.538836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298266+0000","last_change":"2024-04-26T18:24:20.959871+0000","last_active":"2024-04-26T18:25:06.298266+0000","last_peered":"2024-04-26T18:25:06.298266+0000","last_clean":"2024-04-26T18:25:06.298266+0000","last_became_active":"2024-04-26T18:24:20.959404+0000","last_became_peered":"2024-04-26T18:24:20.959404+0000","last_unstale":"2024-04-26T18:25:06.298266+0000","last_undegraded":"2024-04-26T18:25:06.298266+0000","last_fullsized":"2024-04-26T18:25:06.298266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:43.053618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351694+0000","last_change":"2024-04-26T18:24:20.963815+0000","last_active":"2024-04-26T18:25:06.351694+0000","last_peered":"2024-04-26T18:25:06.351694+0000","last_clean":"2024-04-26T18:25:06.351694+0000","last_became_active":"2024-04-26T18:24:20.963362+0000","last_became_peered":"2024-04-26T18:24:20.963362+0000","last_unstale":"2024-04-26T18:25:06.351694+0000","last_undegraded":"2024-04-26T18:25:06.351694+0000","last_fullsized":"2024-04-26T18:25:06.351694+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:57.907485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290523+0000","last_change":"2024-04-26T18:24:20.952109+0000","last_active":"2024-04-26T18:25:05.290523+0000","last_peered":"2024-04-26T18:25:05.290523+0000","last_clean":"2024-04-26T18:25:05.290523+0000","last_became_active":"2024-04-26T18:24:20.951776+0000","last_became_peered":"2024-04-26T18:24:20.951776+0000","last_unstale":"2024-04-26T18:25:05.290523+0000","last_undegraded":"2024-04-26T18:25:05.290523+0000","last_fullsized":"2024-04-26T18:25:05.290523+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:53.220246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.292933+0000","last_change":"2024-04-26T18:24:20.963965+0000","last_active":"2024-04-26T18:25:05.292933+0000","last_peered":"2024-04-26T18:25:05.292933+0000","last_clean":"2024-04-26T18:25:05.292933+0000","last_became_active":"2024-04-26T18:24:20.963648+0000","last_became_peered":"2024-04-26T18:24:20.963648+0000","last_unstale":"2024-04-26T18:25:05.292933+0000","last_undegraded":"2024-04-26T18:25:05.292933+0000","last_fullsized":"2024-04-26T18:25:05.292933+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:04.080813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959580+0000","last_change":"2024-04-26T18:24:20.954186+0000","last_active":"2024-04-26T18:24:22.959580+0000","last_peered":"2024-04-26T18:24:22.959580+0000","last_clean":"2024-04-26T18:24:22.959580+0000","last_became_active":"2024-04-26T18:24:20.953431+0000","last_became_peered":"2024-04-26T18:24:20.953431+0000","last_unstale":"2024-04-26T18:24:22.959580+0000","last_undegraded":"2024-04-26T18:24:22.959580+0000","last_fullsized":"2024-04-26T18:24:22.959580+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:21.498476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295940+0000","last_change":"2024-04-26T18:24:20.945392+0000","last_active":"2024-04-26T18:25:06.295940+0000","last_peered":"2024-04-26T18:25:06.295940+0000","last_clean":"2024-04-26T18:25:06.295940+0000","last_became_active":"2024-04-26T18:24:20.944971+0000","last_became_peered":"2024-04-26T18:24:20.944971+0000","last_unstale":"2024-04-26T18:25:06.295940+0000","last_undegraded":"2024-04-26T18:25:06.295940+0000","last_fullsized":"2024-04-26T18:25:06.295940+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:06.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299351+0000","last_change":"2024-04-26T18:25:06.299351+0000","last_active":"2024-04-26T18:25:06.299351+0000","last_peered":"2024-04-26T18:25:06.299351+0000","last_clean":"2024-04-26T18:25:06.299351+0000","last_became_active":"2024-04-26T18:25:06.298895+0000","last_became_peered":"2024-04-26T18:25:06.298895+0000","last_unstale":"2024-04-26T18:25:06.299351+0000","last_undegraded":"2024-04-26T18:25:06.299351+0000","last_fullsized":"2024-04-26T18:25:06.299351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:37.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297807+0000","last_change":"2024-04-26T18:24:20.954610+0000","last_active":"2024-04-26T18:25:06.297807+0000","last_peered":"2024-04-26T18:25:06.297807+0000","last_clean":"2024-04-26T18:25:06.297807+0000","last_became_active":"2024-04-26T18:24:20.954239+0000","last_became_peered":"2024-04-26T18:24:20.954239+0000","last_unstale":"2024-04-26T18:25:06.297807+0000","last_undegraded":"2024-04-26T18:25:06.297807+0000","last_fullsized":"2024-04-26T18:25:06.297807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:41.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.295271+0000","last_change":"2024-04-26T18:24:20.956932+0000","last_active":"2024-04-26T18:25:05.295271+0000","last_peered":"2024-04-26T18:25:05.295271+0000","last_clean":"2024-04-26T18:25:05.295271+0000","last_became_active":"2024-04-26T18:24:20.956464+0000","last_became_peered":"2024-04-26T18:24:20.956464+0000","last_unstale":"2024-04-26T18:25:05.295271+0000","last_undegraded":"2024-04-26T18:25:05.295271+0000","last_fullsized":"2024-04-26T18:25:05.295271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:01.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294148+0000","last_change":"2024-04-26T18:24:20.954870+0000","last_active":"2024-04-26T18:25:06.294148+0000","last_peered":"2024-04-26T18:25:06.294148+0000","last_clean":"2024-04-26T18:25:06.294148+0000","last_became_active":"2024-04-26T18:24:20.954542+0000","last_became_peered":"2024-04-26T18:24:20.954542+0000","last_unstale":"2024-04-26T18:25:06.294148+0000","last_undegraded":"2024-04-26T18:25:06.294148+0000","last_fullsized":"2024-04-26T18:25:06.294148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:13:51.932625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.953299+0000","last_change":"2024-04-26T18:24:20.950994+0000","last_active":"2024-04-26T18:24:22.953299+0000","last_peered":"2024-04-26T18:24:22.953299+0000","last_clean":"2024-04-26T18:24:22.953299+0000","last_became_active":"2024-04-26T18:24:20.950343+0000","last_became_peered":"2024-04-26T18:24:20.950343+0000","last_unstale":"2024-04-26T18:24:22.953299+0000","last_undegraded":"2024-04-26T18:24:22.953299+0000","last_fullsized":"2024-04-26T18:24:22.953299+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:25.274049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.346730+0000","last_change":"2024-04-26T18:24:17.929731+0000","last_active":"2024-04-26T18:25:06.346730+0000","last_peered":"2024-04-26T18:25:06.346730+0000","last_clean":"2024-04-26T18:25:06.346730+0000","last_became_active":"2024-04-26T18:24:17.929379+0000","last_became_peered":"2024-04-26T18:24:17.929379+0000","last_unstale":"2024-04-26T18:25:06.346730+0000","last_undegraded":"2024-04-26T18:25:06.346730+0000","last_fullsized":"2024-04-26T18:25:06.346730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:44.821660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754917+0000","last_change":"2024-04-26T18:24:20.962885+0000","last_active":"2024-04-26T18:25:06.754917+0000","last_peered":"2024-04-26T18:25:06.754917+0000","last_clean":"2024-04-26T18:25:06.754917+0000","last_became_active":"2024-04-26T18:24:20.962674+0000","last_became_peered":"2024-04-26T18:24:20.962674+0000","last_unstale":"2024-04-26T18:25:06.754917+0000","last_undegraded":"2024-04-26T18:25:06.754917+0000","last_fullsized":"2024-04-26T18:25:06.754917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:32.444889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.755048+0000","last_change":"2024-04-26T18:24:18.141500+0000","last_active":"2024-04-26T18:25:06.755048+0000","last_peered":"2024-04-26T18:25:06.755048+0000","last_clean":"2024-04-26T18:25:06.755048+0000","last_became_active":"2024-04-26T18:24:18.140870+0000","last_became_peered":"2024-04-26T18:24:18.140870+0000","last_unstale":"2024-04-26T18:25:06.755048+0000","last_undegraded":"2024-04-26T18:25:06.755048+0000","last_fullsized":"2024-04-26T18:25:06.755048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:13.821872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296790+0000","last_change":"2024-04-26T18:24:20.949183+0000","last_active":"2024-04-26T18:25:06.296790+0000","last_peered":"2024-04-26T18:25:06.296790+0000","last_clean":"2024-04-26T18:25:06.296790+0000","last_became_active":"2024-04-26T18:24:20.948677+0000","last_became_peered":"2024-04-26T18:24:20.948677+0000","last_unstale":"2024-04-26T18:25:06.296790+0000","last_undegraded":"2024-04-26T18:25:06.296790+0000","last_fullsized":"2024-04-26T18:25:06.296790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:55.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.312239+0000","last_change":"2024-04-26T18:25:06.312239+0000","last_active":"2024-04-26T18:25:06.312239+0000","last_peered":"2024-04-26T18:25:06.312239+0000","last_clean":"2024-04-26T18:25:06.312239+0000","last_became_active":"2024-04-26T18:25:06.301304+0000","last_became_peered":"2024-04-26T18:25:06.301304+0000","last_unstale":"2024-04-26T18:25:06.312239+0000","last_undegraded":"2024-04-26T18:25:06.312239+0000","last_fullsized":"2024-04-26T18:25:06.312239+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:36.159348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298667+0000","last_change":"2024-04-26T18:24:20.955555+0000","last_active":"2024-04-26T18:25:06.298667+0000","last_peered":"2024-04-26T18:25:06.298667+0000","last_clean":"2024-04-26T18:25:06.298667+0000","last_became_active":"2024-04-26T18:24:20.955176+0000","last_became_peered":"2024-04-26T18:24:20.955176+0000","last_unstale":"2024-04-26T18:25:06.298667+0000","last_undegraded":"2024-04-26T18:25:06.298667+0000","last_fullsized":"2024-04-26T18:25:06.298667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:02:30.368971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298703+0000","last_change":"2024-04-26T18:24:17.930919+0000","last_active":"2024-04-26T18:25:06.298703+0000","last_peered":"2024-04-26T18:25:06.298703+0000","last_clean":"2024-04-26T18:25:06.298703+0000","last_became_active":"2024-04-26T18:24:17.930596+0000","last_became_peered":"2024-04-26T18:24:17.930596+0000","last_unstale":"2024-04-26T18:25:06.298703+0000","last_undegraded":"2024-04-26T18:25:06.298703+0000","last_fullsized":"2024-04-26T18:25:06.298703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:57.476027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.960323+0000","last_change":"2024-04-26T18:24:20.955796+0000","last_active":"2024-04-26T18:24:22.960323+0000","last_peered":"2024-04-26T18:24:22.960323+0000","last_clean":"2024-04-26T18:24:22.960323+0000","last_became_active":"2024-04-26T18:24:20.955432+0000","last_became_peered":"2024-04-26T18:24:20.955432+0000","last_unstale":"2024-04-26T18:24:22.960323+0000","last_undegraded":"2024-04-26T18:24:22.960323+0000","last_fullsized":"2024-04-26T18:24:22.960323+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:25.161797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":57,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:25:03.344282+0000","last_change":"2024-04-26T18:24:17.929505+0000","last_active":"2024-04-26T18:25:03.344282+0000","last_peered":"2024-04-26T18:25:03.344282+0000","last_clean":"2024-04-26T18:25:03.344282+0000","last_became_active":"2024-04-26T18:24:17.929205+0000","last_became_peered":"2024-04-26T18:24:17.929205+0000","last_unstale":"2024-04-26T18:25:03.344282+0000","last_undegraded":"2024-04-26T18:25:03.344282+0000","last_fullsized":"2024-04-26T18:25:03.344282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:42.460063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297815+0000","last_change":"2024-04-26T18:24:20.952134+0000","last_active":"2024-04-26T18:25:06.297815+0000","last_peered":"2024-04-26T18:25:06.297815+0000","last_clean":"2024-04-26T18:25:06.297815+0000","last_became_active":"2024-04-26T18:24:20.951758+0000","last_became_peered":"2024-04-26T18:24:20.951758+0000","last_unstale":"2024-04-26T18:25:06.297815+0000","last_undegraded":"2024-04-26T18:25:06.297815+0000","last_fullsized":"2024-04-26T18:25:06.297815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:25.389221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351236+0000","last_change":"2024-04-26T18:24:17.928704+0000","last_active":"2024-04-26T18:25:06.351236+0000","last_peered":"2024-04-26T18:25:06.351236+0000","last_clean":"2024-04-26T18:25:06.351236+0000","last_became_active":"2024-04-26T18:24:17.928343+0000","last_became_peered":"2024-04-26T18:24:17.928343+0000","last_unstale":"2024-04-26T18:25:06.351236+0000","last_undegraded":"2024-04-26T18:25:06.351236+0000","last_fullsized":"2024-04-26T18:25:06.351236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:49.400481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.960263+0000","last_change":"2024-04-26T18:24:20.956808+0000","last_active":"2024-04-26T18:24:22.960263+0000","last_peered":"2024-04-26T18:24:22.960263+0000","last_clean":"2024-04-26T18:24:22.960263+0000","last_became_active":"2024-04-26T18:24:20.956466+0000","last_became_peered":"2024-04-26T18:24:20.956466+0000","last_unstale":"2024-04-26T18:24:22.960263+0000","last_undegraded":"2024-04-26T18:24:22.960263+0000","last_fullsized":"2024-04-26T18:24:22.960263+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:31.150670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:36.282580+0000","last_change":"2024-04-26T18:24:17.929134+0000","last_active":"2024-04-26T18:24:36.282580+0000","last_peered":"2024-04-26T18:24:36.282580+0000","last_clean":"2024-04-26T18:24:36.282580+0000","last_became_active":"2024-04-26T18:24:17.928798+0000","last_became_peered":"2024-04-26T18:24:17.928798+0000","last_unstale":"2024-04-26T18:24:36.282580+0000","last_undegraded":"2024-04-26T18:24:36.282580+0000","last_fullsized":"2024-04-26T18:24:36.282580+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:10.450825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754806+0000","last_change":"2024-04-26T18:24:20.951594+0000","last_active":"2024-04-26T18:25:06.754806+0000","last_peered":"2024-04-26T18:25:06.754806+0000","last_clean":"2024-04-26T18:25:06.754806+0000","last_became_active":"2024-04-26T18:24:20.951148+0000","last_became_peered":"2024-04-26T18:24:20.951148+0000","last_unstale":"2024-04-26T18:25:06.754806+0000","last_undegraded":"2024-04-26T18:25:06.754806+0000","last_fullsized":"2024-04-26T18:25:06.754806+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:18.703292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:38.303074+0000","last_change":"2024-04-26T18:24:17.931101+0000","last_active":"2024-04-26T18:24:38.303074+0000","last_peered":"2024-04-26T18:24:38.303074+0000","last_clean":"2024-04-26T18:24:38.303074+0000","last_became_active":"2024-04-26T18:24:17.930819+0000","last_became_peered":"2024-04-26T18:24:17.930819+0000","last_unstale":"2024-04-26T18:24:38.303074+0000","last_undegraded":"2024-04-26T18:24:38.303074+0000","last_fullsized":"2024-04-26T18:24:38.303074+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.293645+0000","last_change":"2024-04-26T18:24:20.948866+0000","last_active":"2024-04-26T18:25:06.293645+0000","last_peered":"2024-04-26T18:25:06.293645+0000","last_clean":"2024-04-26T18:25:06.293645+0000","last_became_active":"2024-04-26T18:24:20.948562+0000","last_became_peered":"2024-04-26T18:24:20.948562+0000","last_unstale":"2024-04-26T18:25:06.293645+0000","last_undegraded":"2024-04-26T18:25:06.293645+0000","last_fullsized":"2024-04-26T18:25:06.293645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:33:19.616660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":89,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:08.310046+0000","last_change":"2024-04-26T18:24:17.926384+0000","last_active":"2024-04-26T18:25:08.310046+0000","last_peered":"2024-04-26T18:25:08.310046+0000","last_clean":"2024-04-26T18:25:08.310046+0000","last_became_active":"2024-04-26T18:24:17.926037+0000","last_became_peered":"2024-04-26T18:24:17.926037+0000","last_unstale":"2024-04-26T18:25:08.310046+0000","last_undegraded":"2024-04-26T18:25:08.310046+0000","last_fullsized":"2024-04-26T18:25:08.310046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:05.283755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298313+0000","last_change":"2024-04-26T18:24:20.954257+0000","last_active":"2024-04-26T18:25:06.298313+0000","last_peered":"2024-04-26T18:25:06.298313+0000","last_clean":"2024-04-26T18:25:06.298313+0000","last_became_active":"2024-04-26T18:24:20.953913+0000","last_became_peered":"2024-04-26T18:24:20.953913+0000","last_unstale":"2024-04-26T18:25:06.298313+0000","last_undegraded":"2024-04-26T18:25:06.298313+0000","last_fullsized":"2024-04-26T18:25:06.298313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:06.814156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290317+0000","last_change":"2024-04-26T18:24:17.932843+0000","last_active":"2024-04-26T18:25:05.290317+0000","last_peered":"2024-04-26T18:25:05.290317+0000","last_clean":"2024-04-26T18:25:05.290317+0000","last_became_active":"2024-04-26T18:24:17.932502+0000","last_became_peered":"2024-04-26T18:24:17.932502+0000","last_unstale":"2024-04-26T18:25:05.290317+0000","last_undegraded":"2024-04-26T18:25:05.290317+0000","last_fullsized":"2024-04-26T18:25:05.290317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:49.727094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296758+0000","last_change":"2024-04-26T18:24:20.959881+0000","last_active":"2024-04-26T18:25:06.296758+0000","last_peered":"2024-04-26T18:25:06.296758+0000","last_clean":"2024-04-26T18:25:06.296758+0000","last_became_active":"2024-04-26T18:24:20.959541+0000","last_became_peered":"2024-04-26T18:24:20.959541+0000","last_unstale":"2024-04-26T18:25:06.296758+0000","last_undegraded":"2024-04-26T18:25:06.296758+0000","last_fullsized":"2024-04-26T18:25:06.296758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:23.393521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:37.294338+0000","last_change":"2024-04-26T18:24:18.142146+0000","last_active":"2024-04-26T18:24:37.294338+0000","last_peered":"2024-04-26T18:24:37.294338+0000","last_clean":"2024-04-26T18:24:37.294338+0000","last_became_active":"2024-04-26T18:24:18.141829+0000","last_became_peered":"2024-04-26T18:24:18.141829+0000","last_unstale":"2024-04-26T18:24:37.294338+0000","last_undegraded":"2024-04-26T18:24:37.294338+0000","last_fullsized":"2024-04-26T18:24:37.294338+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:49.341638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298792+0000","last_change":"2024-04-26T18:24:20.957993+0000","last_active":"2024-04-26T18:25:06.298792+0000","last_peered":"2024-04-26T18:25:06.298792+0000","last_clean":"2024-04-26T18:25:06.298792+0000","last_became_active":"2024-04-26T18:24:20.957443+0000","last_became_peered":"2024-04-26T18:24:20.957443+0000","last_unstale":"2024-04-26T18:25:06.298792+0000","last_undegraded":"2024-04-26T18:25:06.298792+0000","last_fullsized":"2024-04-26T18:25:06.298792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:06:32.960546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'32","reported_seq":126,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:08.354501+0000","last_change":"2024-04-26T18:24:17.931283+0000","last_active":"2024-04-26T18:25:08.354501+0000","last_peered":"2024-04-26T18:25:08.354501+0000","last_clean":"2024-04-26T18:25:08.354501+0000","last_became_active":"2024-04-26T18:24:17.930943+0000","last_became_peered":"2024-04-26T18:24:17.930943+0000","last_unstale":"2024-04-26T18:25:08.354501+0000","last_undegraded":"2024-04-26T18:25:08.354501+0000","last_fullsized":"2024-04-26T18:25:08.354501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:50.511677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63072,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":68,"num_read_kb":95,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298440+0000","last_change":"2024-04-26T18:24:20.947509+0000","last_active":"2024-04-26T18:25:06.298440+0000","last_peered":"2024-04-26T18:25:06.298440+0000","last_clean":"2024-04-26T18:25:06.298440+0000","last_became_active":"2024-04-26T18:24:20.947181+0000","last_became_peered":"2024-04-26T18:24:20.947181+0000","last_unstale":"2024-04-26T18:25:06.298440+0000","last_undegraded":"2024-04-26T18:25:06.298440+0000","last_fullsized":"2024-04-26T18:25:06.298440+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.262312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:39.309250+0000","last_change":"2024-04-26T18:24:17.934042+0000","last_active":"2024-04-26T18:24:39.309250+0000","last_peered":"2024-04-26T18:24:39.309250+0000","last_clean":"2024-04-26T18:24:39.309250+0000","last_became_active":"2024-04-26T18:24:17.933537+0000","last_became_peered":"2024-04-26T18:24:17.933537+0000","last_unstale":"2024-04-26T18:24:39.309250+0000","last_undegraded":"2024-04-26T18:24:39.309250+0000","last_fullsized":"2024-04-26T18:24:39.309250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:35:06.904160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959689+0000","last_change":"2024-04-26T18:24:20.957672+0000","last_active":"2024-04-26T18:24:22.959689+0000","last_peered":"2024-04-26T18:24:22.959689+0000","last_clean":"2024-04-26T18:24:22.959689+0000","last_became_active":"2024-04-26T18:24:20.957214+0000","last_became_peered":"2024-04-26T18:24:20.957214+0000","last_unstale":"2024-04-26T18:24:22.959689+0000","last_undegraded":"2024-04-26T18:24:22.959689+0000","last_fullsized":"2024-04-26T18:24:22.959689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:42:35.341858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:38.303247+0000","last_change":"2024-04-26T18:24:17.930324+0000","last_active":"2024-04-26T18:24:38.303247+0000","last_peered":"2024-04-26T18:24:38.303247+0000","last_clean":"2024-04-26T18:24:38.303247+0000","last_became_active":"2024-04-26T18:24:17.930022+0000","last_became_peered":"2024-04-26T18:24:17.930022+0000","last_unstale":"2024-04-26T18:24:38.303247+0000","last_undegraded":"2024-04-26T18:24:38.303247+0000","last_fullsized":"2024-04-26T18:24:38.303247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:36.864816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":17,"reported_epoch":89,"state":"peering","last_fresh":"2024-04-26T18:24:22.953034+0000","last_change":"2024-04-26T18:25:05.290798+0000","last_active":"2024-04-26T18:25:05.290798+0000","last_peered":"2024-04-26T18:24:22.953034+0000","last_clean":"2024-04-26T18:24:22.953034+0000","last_became_active":"2024-04-26T18:24:20.958961+0000","last_became_peered":"2024-04-26T18:24:20.958961+0000","last_unstale":"2024-04-26T18:24:22.953034+0000","last_undegraded":"2024-04-26T18:24:22.953034+0000","last_fullsized":"2024-04-26T18:24:22.953034+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4,6,9],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":58,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.292879+0000","last_change":"2024-04-26T18:24:17.934676+0000","last_active":"2024-04-26T18:25:05.292879+0000","last_peered":"2024-04-26T18:25:05.292879+0000","last_clean":"2024-04-26T18:25:05.292879+0000","last_became_active":"2024-04-26T18:24:17.934307+0000","last_became_peered":"2024-04-26T18:24:17.934307+0000","last_unstale":"2024-04-26T18:25:05.292879+0000","last_undegraded":"2024-04-26T18:25:05.292879+0000","last_fullsized":"2024-04-26T18:25:05.292879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:27.312248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290136+0000","last_change":"2024-04-26T18:24:20.947371+0000","last_active":"2024-04-26T18:25:05.290136+0000","last_peered":"2024-04-26T18:25:05.290136+0000","last_clean":"2024-04-26T18:25:05.290136+0000","last_became_active":"2024-04-26T18:24:20.947048+0000","last_became_peered":"2024-04-26T18:24:20.947048+0000","last_unstale":"2024-04-26T18:25:05.290136+0000","last_undegraded":"2024-04-26T18:25:05.290136+0000","last_fullsized":"2024-04-26T18:25:05.290136+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:55.038074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297413+0000","last_change":"2024-04-26T18:24:18.142680+0000","last_active":"2024-04-26T18:25:06.297413+0000","last_peered":"2024-04-26T18:25:06.297413+0000","last_clean":"2024-04-26T18:25:06.297413+0000","last_became_active":"2024-04-26T18:24:18.142392+0000","last_became_peered":"2024-04-26T18:24:18.142392+0000","last_unstale":"2024-04-26T18:25:06.297413+0000","last_undegraded":"2024-04-26T18:25:06.297413+0000","last_fullsized":"2024-04-26T18:25:06.297413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:52.654813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959476+0000","last_change":"2024-04-26T18:24:20.958131+0000","last_active":"2024-04-26T18:24:22.959476+0000","last_peered":"2024-04-26T18:24:22.959476+0000","last_clean":"2024-04-26T18:24:22.959476+0000","last_became_active":"2024-04-26T18:24:20.957701+0000","last_became_peered":"2024-04-26T18:24:20.957701+0000","last_unstale":"2024-04-26T18:24:22.959476+0000","last_undegraded":"2024-04-26T18:24:22.959476+0000","last_fullsized":"2024-04-26T18:24:22.959476+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:02.018158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":2,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-26T18:24:22.966802+0000","last_change":"2024-04-26T18:25:05.511258+0000","last_active":"2024-04-26T18:25:05.511258+0000","last_peered":"2024-04-26T18:24:22.966802+0000","last_clean":"2024-04-26T18:24:22.966802+0000","last_became_active":"2024-04-26T18:24:17.928840+0000","last_became_peered":"2024-04-26T18:24:17.928840+0000","last_unstale":"2024-04-26T18:24:22.966802+0000","last_undegraded":"2024-04-26T18:24:22.966802+0000","last_fullsized":"2024-04-26T18:24:22.966802+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.295124+0000","last_change":"2024-04-26T18:24:20.961230+0000","last_active":"2024-04-26T18:25:05.295124+0000","last_peered":"2024-04-26T18:25:05.295124+0000","last_clean":"2024-04-26T18:25:05.295124+0000","last_became_active":"2024-04-26T18:24:20.960807+0000","last_became_peered":"2024-04-26T18:24:20.960807+0000","last_unstale":"2024-04-26T18:25:05.295124+0000","last_undegraded":"2024-04-26T18:25:05.295124+0000","last_fullsized":"2024-04-26T18:25:05.295124+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:39.535100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.293995+0000","last_change":"2024-04-26T18:24:18.142082+0000","last_active":"2024-04-26T18:25:06.293995+0000","last_peered":"2024-04-26T18:25:06.293995+0000","last_clean":"2024-04-26T18:25:06.293995+0000","last_became_active":"2024-04-26T18:24:18.141765+0000","last_became_peered":"2024-04-26T18:24:18.141765+0000","last_unstale":"2024-04-26T18:25:06.293995+0000","last_undegraded":"2024-04-26T18:25:06.293995+0000","last_fullsized":"2024-04-26T18:25:06.293995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:02.140027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351535+0000","last_change":"2024-04-26T18:24:20.952867+0000","last_active":"2024-04-26T18:25:06.351535+0000","last_peered":"2024-04-26T18:25:06.351535+0000","last_clean":"2024-04-26T18:25:06.351535+0000","last_became_active":"2024-04-26T18:24:20.952525+0000","last_became_peered":"2024-04-26T18:24:20.952525+0000","last_unstale":"2024-04-26T18:25:06.351535+0000","last_undegraded":"2024-04-26T18:25:06.351535+0000","last_fullsized":"2024-04-26T18:25:06.351535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:52.804312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":57,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290664+0000","last_change":"2024-04-26T18:24:17.932270+0000","last_active":"2024-04-26T18:25:05.290664+0000","last_peered":"2024-04-26T18:25:05.290664+0000","last_clean":"2024-04-26T18:25:05.290664+0000","last_became_active":"2024-04-26T18:24:17.931901+0000","last_became_peered":"2024-04-26T18:24:17.931901+0000","last_unstale":"2024-04-26T18:25:05.290664+0000","last_undegraded":"2024-04-26T18:25:05.290664+0000","last_fullsized":"2024-04-26T18:25:05.290664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:13.524174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351380+0000","last_change":"2024-04-26T18:24:20.960612+0000","last_active":"2024-04-26T18:25:06.351380+0000","last_peered":"2024-04-26T18:25:06.351380+0000","last_clean":"2024-04-26T18:25:06.351380+0000","last_became_active":"2024-04-26T18:24:20.960280+0000","last_became_peered":"2024-04-26T18:24:20.960280+0000","last_unstale":"2024-04-26T18:25:06.351380+0000","last_undegraded":"2024-04-26T18:25:06.351380+0000","last_fullsized":"2024-04-26T18:25:06.351380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:53.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.300141+0000","last_change":"2024-04-26T18:25:06.299736+0000","last_active":"2024-04-26T18:25:06.300141+0000","last_peered":"2024-04-26T18:25:06.300141+0000","last_clean":"2024-04-26T18:25:06.300141+0000","last_became_active":"2024-04-26T18:25:06.299265+0000","last_became_peered":"2024-04-26T18:25:06.299265+0000","last_unstale":"2024-04-26T18:25:06.300141+0000","last_undegraded":"2024-04-26T18:25:06.300141+0000","last_fullsized":"2024-04-26T18:25:06.300141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:32.427551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":54,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:25:03.343364+0000","last_change":"2024-04-26T18:24:18.141181+0000","last_active":"2024-04-26T18:25:03.343364+0000","last_peered":"2024-04-26T18:25:03.343364+0000","last_clean":"2024-04-26T18:25:03.343364+0000","last_became_active":"2024-04-26T18:24:18.140854+0000","last_became_peered":"2024-04-26T18:24:18.140854+0000","last_unstale":"2024-04-26T18:25:03.343364+0000","last_undegraded":"2024-04-26T18:25:03.343364+0000","last_fullsized":"2024-04-26T18:25:03.343364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:27.900701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754882+0000","last_change":"2024-04-26T18:24:20.961653+0000","last_active":"2024-04-26T18:25:06.754882+0000","last_peered":"2024-04-26T18:25:06.754882+0000","last_clean":"2024-04-26T18:25:06.754882+0000","last_became_active":"2024-04-26T18:24:20.961330+0000","last_became_peered":"2024-04-26T18:24:20.961330+0000","last_unstale":"2024-04-26T18:25:06.754882+0000","last_undegraded":"2024-04-26T18:25:06.754882+0000","last_fullsized":"2024-04-26T18:25:06.754882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:19.033932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":33,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.293582+0000","last_change":"2024-04-26T18:24:17.933613+0000","last_active":"2024-04-26T18:25:05.293582+0000","last_peered":"2024-04-26T18:25:05.293582+0000","last_clean":"2024-04-26T18:25:05.293582+0000","last_became_active":"2024-04-26T18:24:17.933219+0000","last_became_peered":"2024-04-26T18:24:17.933219+0000","last_unstale":"2024-04-26T18:25:05.293582+0000","last_undegraded":"2024-04-26T18:25:05.293582+0000","last_fullsized":"2024-04-26T18:25:05.293582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:04.215821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":356,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290245+0000","last_change":"2024-04-26T18:22:57.585634+0000","last_active":"2024-04-26T18:25:05.290245+0000","last_peered":"2024-04-26T18:25:05.290245+0000","last_clean":"2024-04-26T18:25:05.290245+0000","last_became_active":"2024-04-26T18:22:57.585198+0000","last_became_peered":"2024-04-26T18:22:57.585198+0000","last_unstale":"2024-04-26T18:25:05.290245+0000","last_undegraded":"2024-04-26T18:25:05.290245+0000","last_fullsized":"2024-04-26T18:25:05.290245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_clean_scrub_stamp":"2024-04-26T18:19:31.594684+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:56:11.861930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959033+0000","last_change":"2024-04-26T18:24:20.950303+0000","last_active":"2024-04-26T18:24:22.959033+0000","last_peered":"2024-04-26T18:24:22.959033+0000","last_clean":"2024-04-26T18:24:22.959033+0000","last_became_active":"2024-04-26T18:24:20.950002+0000","last_became_peered":"2024-04-26T18:24:20.950002+0000","last_unstale":"2024-04-26T18:24:22.959033+0000","last_undegraded":"2024-04-26T18:24:22.959033+0000","last_fullsized":"2024-04-26T18:24:22.959033+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:00.912752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290381+0000","last_change":"2024-04-26T18:24:17.934924+0000","last_active":"2024-04-26T18:25:05.290381+0000","last_peered":"2024-04-26T18:25:05.290381+0000","last_clean":"2024-04-26T18:25:05.290381+0000","last_became_active":"2024-04-26T18:24:17.934598+0000","last_became_peered":"2024-04-26T18:24:17.934598+0000","last_unstale":"2024-04-26T18:25:05.290381+0000","last_undegraded":"2024-04-26T18:25:05.290381+0000","last_fullsized":"2024-04-26T18:25:05.290381+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:28.690043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298555+0000","last_change":"2024-04-26T18:24:18.142560+0000","last_active":"2024-04-26T18:25:06.298555+0000","last_peered":"2024-04-26T18:25:06.298555+0000","last_clean":"2024-04-26T18:25:06.298555+0000","last_became_active":"2024-04-26T18:24:18.142184+0000","last_became_peered":"2024-04-26T18:24:18.142184+0000","last_unstale":"2024-04-26T18:25:06.298555+0000","last_undegraded":"2024-04-26T18:25:06.298555+0000","last_fullsized":"2024-04-26T18:25:06.298555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:26.656113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959122+0000","last_change":"2024-04-26T18:24:20.952073+0000","last_active":"2024-04-26T18:24:22.959122+0000","last_peered":"2024-04-26T18:24:22.959122+0000","last_clean":"2024-04-26T18:24:22.959122+0000","last_became_active":"2024-04-26T18:24:20.951761+0000","last_became_peered":"2024-04-26T18:24:20.951761+0000","last_unstale":"2024-04-26T18:24:22.959122+0000","last_undegraded":"2024-04-26T18:24:22.959122+0000","last_fullsized":"2024-04-26T18:24:22.959122+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:30.244060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'14","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294095+0000","last_change":"2024-04-26T18:24:17.927171+0000","last_active":"2024-04-26T18:25:06.294095+0000","last_peered":"2024-04-26T18:25:06.294095+0000","last_clean":"2024-04-26T18:25:06.294095+0000","last_became_active":"2024-04-26T18:24:17.926816+0000","last_became_peered":"2024-04-26T18:24:17.926816+0000","last_unstale":"2024-04-26T18:25:06.294095+0000","last_undegraded":"2024-04-26T18:25:06.294095+0000","last_fullsized":"2024-04-26T18:25:06.294095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:18.249363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.755057+0000","last_change":"2024-04-26T18:24:20.951684+0000","last_active":"2024-04-26T18:25:06.755057+0000","last_peered":"2024-04-26T18:25:06.755057+0000","last_clean":"2024-04-26T18:25:06.755057+0000","last_became_active":"2024-04-26T18:24:20.951364+0000","last_became_peered":"2024-04-26T18:24:20.951364+0000","last_unstale":"2024-04-26T18:25:06.755057+0000","last_undegraded":"2024-04-26T18:25:06.755057+0000","last_fullsized":"2024-04-26T18:25:06.755057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:28.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":32,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.292911+0000","last_change":"2024-04-26T18:24:18.229770+0000","last_active":"2024-04-26T18:25:05.292911+0000","last_peered":"2024-04-26T18:25:05.292911+0000","last_clean":"2024-04-26T18:25:05.292911+0000","last_became_active":"2024-04-26T18:24:18.229446+0000","last_became_peered":"2024-04-26T18:24:18.229446+0000","last_unstale":"2024-04-26T18:25:05.292911+0000","last_undegraded":"2024-04-26T18:25:05.292911+0000","last_fullsized":"2024-04-26T18:25:05.292911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:14.794797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296547+0000","last_change":"2024-04-26T18:24:20.960513+0000","last_active":"2024-04-26T18:25:06.296547+0000","last_peered":"2024-04-26T18:25:06.296547+0000","last_clean":"2024-04-26T18:25:06.296547+0000","last_became_active":"2024-04-26T18:24:20.959549+0000","last_became_peered":"2024-04-26T18:24:20.959549+0000","last_unstale":"2024-04-26T18:25:06.296547+0000","last_undegraded":"2024-04-26T18:25:06.296547+0000","last_fullsized":"2024-04-26T18:25:06.296547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:18:37.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297969+0000","last_change":"2024-04-26T18:24:18.141549+0000","last_active":"2024-04-26T18:25:06.297969+0000","last_peered":"2024-04-26T18:25:06.297969+0000","last_clean":"2024-04-26T18:25:06.297969+0000","last_became_active":"2024-04-26T18:24:18.141172+0000","last_became_peered":"2024-04-26T18:24:18.141172+0000","last_unstale":"2024-04-26T18:25:06.297969+0000","last_undegraded":"2024-04-26T18:25:06.297969+0000","last_fullsized":"2024-04-26T18:25:06.297969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:15.878130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297295+0000","last_change":"2024-04-26T18:24:20.948121+0000","last_active":"2024-04-26T18:25:06.297295+0000","last_peered":"2024-04-26T18:25:06.297295+0000","last_clean":"2024-04-26T18:25:06.297295+0000","last_became_active":"2024-04-26T18:24:20.947738+0000","last_became_peered":"2024-04-26T18:24:20.947738+0000","last_unstale":"2024-04-26T18:25:06.297295+0000","last_undegraded":"2024-04-26T18:25:06.297295+0000","last_fullsized":"2024-04-26T18:25:06.297295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:04.237302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754809+0000","last_change":"2024-04-26T18:24:17.926631+0000","last_active":"2024-04-26T18:25:06.754809+0000","last_peered":"2024-04-26T18:25:06.754809+0000","last_clean":"2024-04-26T18:25:06.754809+0000","last_became_active":"2024-04-26T18:24:17.926259+0000","last_became_peered":"2024-04-26T18:24:17.926259+0000","last_unstale":"2024-04-26T18:25:06.754809+0000","last_undegraded":"2024-04-26T18:25:06.754809+0000","last_fullsized":"2024-04-26T18:25:06.754809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:47.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.294419+0000","last_change":"2024-04-26T18:24:20.947283+0000","last_active":"2024-04-26T18:25:05.294419+0000","last_peered":"2024-04-26T18:25:05.294419+0000","last_clean":"2024-04-26T18:25:05.294419+0000","last_became_active":"2024-04-26T18:24:20.946812+0000","last_became_peered":"2024-04-26T18:24:20.946812+0000","last_unstale":"2024-04-26T18:25:05.294419+0000","last_undegraded":"2024-04-26T18:25:05.294419+0000","last_fullsized":"2024-04-26T18:25:05.294419+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:33.730698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:37.294726+0000","last_change":"2024-04-26T18:24:17.931843+0000","last_active":"2024-04-26T18:24:37.294726+0000","last_peered":"2024-04-26T18:24:37.294726+0000","last_clean":"2024-04-26T18:24:37.294726+0000","last_became_active":"2024-04-26T18:24:17.931442+0000","last_became_peered":"2024-04-26T18:24:17.931442+0000","last_unstale":"2024-04-26T18:24:37.294726+0000","last_undegraded":"2024-04-26T18:24:37.294726+0000","last_fullsized":"2024-04-26T18:24:37.294726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:09.649435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.755046+0000","last_change":"2024-04-26T18:24:20.955661+0000","last_active":"2024-04-26T18:25:06.755046+0000","last_peered":"2024-04-26T18:25:06.755046+0000","last_clean":"2024-04-26T18:25:06.755046+0000","last_became_active":"2024-04-26T18:24:20.955344+0000","last_became_peered":"2024-04-26T18:24:20.955344+0000","last_unstale":"2024-04-26T18:25:06.755046+0000","last_undegraded":"2024-04-26T18:25:06.755046+0000","last_fullsized":"2024-04-26T18:25:06.755046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:44.274381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":2,"reported_epoch":90,"state":"peering","last_fresh":"2024-04-26T18:24:22.968300+0000","last_change":"2024-04-26T18:25:05.511234+0000","last_active":"2024-04-26T18:25:05.511234+0000","last_peered":"2024-04-26T18:24:22.968300+0000","last_clean":"2024-04-26T18:24:22.968300+0000","last_became_active":"2024-04-26T18:24:17.932354+0000","last_became_peered":"2024-04-26T18:24:17.932354+0000","last_unstale":"2024-04-26T18:24:22.968300+0000","last_undegraded":"2024-04-26T18:24:22.968300+0000","last_fullsized":"2024-04-26T18:24:22.968300+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":2,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290753+0000","last_change":"2024-04-26T18:24:20.955746+0000","last_active":"2024-04-26T18:25:05.290753+0000","last_peered":"2024-04-26T18:25:05.290753+0000","last_clean":"2024-04-26T18:25:05.290753+0000","last_became_active":"2024-04-26T18:24:20.953652+0000","last_became_peered":"2024-04-26T18:24:20.953652+0000","last_unstale":"2024-04-26T18:25:05.290753+0000","last_undegraded":"2024-04-26T18:25:05.290753+0000","last_fullsized":"2024-04-26T18:25:05.290753+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:31.414185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.309800+0000","last_change":"2024-04-26T18:25:06.309393+0000","last_active":"2024-04-26T18:25:06.309800+0000","last_peered":"2024-04-26T18:25:06.309800+0000","last_clean":"2024-04-26T18:25:06.309800+0000","last_became_active":"2024-04-26T18:25:06.302434+0000","last_became_peered":"2024-04-26T18:25:06.302434+0000","last_unstale":"2024-04-26T18:25:06.309800+0000","last_undegraded":"2024-04-26T18:25:06.309800+0000","last_fullsized":"2024-04-26T18:25:06.309800+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:54.730295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290882+0000","last_change":"2024-04-26T18:24:20.954238+0000","last_active":"2024-04-26T18:25:05.290882+0000","last_peered":"2024-04-26T18:25:05.290882+0000","last_clean":"2024-04-26T18:25:05.290882+0000","last_became_active":"2024-04-26T18:24:20.953883+0000","last_became_peered":"2024-04-26T18:24:20.953883+0000","last_unstale":"2024-04-26T18:25:05.290882+0000","last_undegraded":"2024-04-26T18:25:05.290882+0000","last_fullsized":"2024-04-26T18:25:05.290882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:10.580835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:38.301373+0000","last_change":"2024-04-26T18:24:17.926901+0000","last_active":"2024-04-26T18:24:38.301373+0000","last_peered":"2024-04-26T18:24:38.301373+0000","last_clean":"2024-04-26T18:24:38.301373+0000","last_became_active":"2024-04-26T18:24:17.926291+0000","last_became_peered":"2024-04-26T18:24:17.926291+0000","last_unstale":"2024-04-26T18:24:38.301373+0000","last_undegraded":"2024-04-26T18:24:38.301373+0000","last_fullsized":"2024-04-26T18:24:38.301373+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:17.695198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":19,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290270+0000","last_change":"2024-04-26T18:24:20.950578+0000","last_active":"2024-04-26T18:25:05.290270+0000","last_peered":"2024-04-26T18:25:05.290270+0000","last_clean":"2024-04-26T18:25:05.290270+0000","last_became_active":"2024-04-26T18:24:20.950174+0000","last_became_peered":"2024-04-26T18:24:20.950174+0000","last_unstale":"2024-04-26T18:25:05.290270+0000","last_undegraded":"2024-04-26T18:25:05.290270+0000","last_fullsized":"2024-04-26T18:25:05.290270+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:38.301172+0000","last_change":"2024-04-26T18:24:17.928054+0000","last_active":"2024-04-26T18:24:38.301172+0000","last_peered":"2024-04-26T18:24:38.301172+0000","last_clean":"2024-04-26T18:24:38.301172+0000","last_became_active":"2024-04-26T18:24:17.927715+0000","last_became_peered":"2024-04-26T18:24:17.927715+0000","last_unstale":"2024-04-26T18:24:38.301172+0000","last_undegraded":"2024-04-26T18:24:38.301172+0000","last_fullsized":"2024-04-26T18:24:38.301172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.155436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.290636+0000","last_change":"2024-04-26T18:24:20.944512+0000","last_active":"2024-04-26T18:25:05.290636+0000","last_peered":"2024-04-26T18:25:05.290636+0000","last_clean":"2024-04-26T18:25:05.290636+0000","last_became_active":"2024-04-26T18:24:20.944123+0000","last_became_peered":"2024-04-26T18:24:20.944123+0000","last_unstale":"2024-04-26T18:25:05.290636+0000","last_undegraded":"2024-04-26T18:25:05.290636+0000","last_fullsized":"2024-04-26T18:25:05.290636+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.240206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.294617+0000","last_change":"2024-04-26T18:24:20.959138+0000","last_active":"2024-04-26T18:25:05.294617+0000","last_peered":"2024-04-26T18:25:05.294617+0000","last_clean":"2024-04-26T18:25:05.294617+0000","last_became_active":"2024-04-26T18:24:20.958675+0000","last_became_peered":"2024-04-26T18:24:20.958675+0000","last_unstale":"2024-04-26T18:25:05.294617+0000","last_undegraded":"2024-04-26T18:25:05.294617+0000","last_fullsized":"2024-04-26T18:25:05.294617+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:34:32.103380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.952807+0000","last_change":"2024-04-26T18:24:20.961689+0000","last_active":"2024-04-26T18:24:22.952807+0000","last_peered":"2024-04-26T18:24:22.952807+0000","last_clean":"2024-04-26T18:24:22.952807+0000","last_became_active":"2024-04-26T18:24:20.961374+0000","last_became_peered":"2024-04-26T18:24:20.961374+0000","last_unstale":"2024-04-26T18:24:22.952807+0000","last_undegraded":"2024-04-26T18:24:22.952807+0000","last_fullsized":"2024-04-26T18:24:22.952807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:55.243887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.952751+0000","last_change":"2024-04-26T18:24:20.950711+0000","last_active":"2024-04-26T18:24:22.952751+0000","last_peered":"2024-04-26T18:24:22.952751+0000","last_clean":"2024-04-26T18:24:22.952751+0000","last_became_active":"2024-04-26T18:24:20.950371+0000","last_became_peered":"2024-04-26T18:24:20.950371+0000","last_unstale":"2024-04-26T18:24:22.952751+0000","last_undegraded":"2024-04-26T18:24:22.952751+0000","last_fullsized":"2024-04-26T18:24:22.952751+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:35.307921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295964+0000","last_change":"2024-04-26T18:24:20.959990+0000","last_active":"2024-04-26T18:25:06.295964+0000","last_peered":"2024-04-26T18:25:06.295964+0000","last_clean":"2024-04-26T18:25:06.295964+0000","last_became_active":"2024-04-26T18:24:20.959514+0000","last_became_peered":"2024-04-26T18:24:20.959514+0000","last_unstale":"2024-04-26T18:25:06.295964+0000","last_undegraded":"2024-04-26T18:25:06.295964+0000","last_fullsized":"2024-04-26T18:25:06.295964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:00.184448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959261+0000","last_change":"2024-04-26T18:24:20.957338+0000","last_active":"2024-04-26T18:24:22.959261+0000","last_peered":"2024-04-26T18:24:22.959261+0000","last_clean":"2024-04-26T18:24:22.959261+0000","last_became_active":"2024-04-26T18:24:20.956999+0000","last_became_peered":"2024-04-26T18:24:20.956999+0000","last_unstale":"2024-04-26T18:24:22.959261+0000","last_undegraded":"2024-04-26T18:24:22.959261+0000","last_fullsized":"2024-04-26T18:24:22.959261+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.066902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.294887+0000","last_change":"2024-04-26T18:24:20.953634+0000","last_active":"2024-04-26T18:25:05.294887+0000","last_peered":"2024-04-26T18:25:05.294887+0000","last_clean":"2024-04-26T18:25:05.294887+0000","last_became_active":"2024-04-26T18:24:20.953259+0000","last_became_peered":"2024-04-26T18:24:20.953259+0000","last_unstale":"2024-04-26T18:25:05.294887+0000","last_undegraded":"2024-04-26T18:25:05.294887+0000","last_fullsized":"2024-04-26T18:25:05.294887+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:00.095344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.953399+0000","last_change":"2024-04-26T18:24:20.954997+0000","last_active":"2024-04-26T18:24:22.953399+0000","last_peered":"2024-04-26T18:24:22.953399+0000","last_clean":"2024-04-26T18:24:22.953399+0000","last_became_active":"2024-04-26T18:24:20.954223+0000","last_became_peered":"2024-04-26T18:24:20.954223+0000","last_unstale":"2024-04-26T18:24:22.953399+0000","last_undegraded":"2024-04-26T18:24:22.953399+0000","last_fullsized":"2024-04-26T18:24:22.953399+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:14.092455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754906+0000","last_change":"2024-04-26T18:24:20.957056+0000","last_active":"2024-04-26T18:25:06.754906+0000","last_peered":"2024-04-26T18:25:06.754906+0000","last_clean":"2024-04-26T18:25:06.754906+0000","last_became_active":"2024-04-26T18:24:20.956737+0000","last_became_peered":"2024-04-26T18:24:20.956737+0000","last_unstale":"2024-04-26T18:25:06.754906+0000","last_undegraded":"2024-04-26T18:25:06.754906+0000","last_fullsized":"2024-04-26T18:25:06.754906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:48.358862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959844+0000","last_change":"2024-04-26T18:24:20.957731+0000","last_active":"2024-04-26T18:24:22.959844+0000","last_peered":"2024-04-26T18:24:22.959844+0000","last_clean":"2024-04-26T18:24:22.959844+0000","last_became_active":"2024-04-26T18:24:20.957405+0000","last_became_peered":"2024-04-26T18:24:20.957405+0000","last_unstale":"2024-04-26T18:24:22.959844+0000","last_undegraded":"2024-04-26T18:24:22.959844+0000","last_fullsized":"2024-04-26T18:24:22.959844+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:12:47.483476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754750+0000","last_change":"2024-04-26T18:24:20.956000+0000","last_active":"2024-04-26T18:25:06.754750+0000","last_peered":"2024-04-26T18:25:06.754750+0000","last_clean":"2024-04-26T18:25:06.754750+0000","last_became_active":"2024-04-26T18:24:20.955677+0000","last_became_peered":"2024-04-26T18:24:20.955677+0000","last_unstale":"2024-04-26T18:25:06.754750+0000","last_undegraded":"2024-04-26T18:25:06.754750+0000","last_fullsized":"2024-04-26T18:25:06.754750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:20.895674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298152+0000","last_change":"2024-04-26T18:24:20.959317+0000","last_active":"2024-04-26T18:25:06.298152+0000","last_peered":"2024-04-26T18:25:06.298152+0000","last_clean":"2024-04-26T18:25:06.298152+0000","last_became_active":"2024-04-26T18:24:20.958963+0000","last_became_peered":"2024-04-26T18:24:20.958963+0000","last_unstale":"2024-04-26T18:25:06.298152+0000","last_undegraded":"2024-04-26T18:25:06.298152+0000","last_fullsized":"2024-04-26T18:25:06.298152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:08.231089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-26T18:24:22.959429+0000","last_change":"2024-04-26T18:24:20.954445+0000","last_active":"2024-04-26T18:24:22.959429+0000","last_peered":"2024-04-26T18:24:22.959429+0000","last_clean":"2024-04-26T18:24:22.959429+0000","last_became_active":"2024-04-26T18:24:20.953852+0000","last_became_peered":"2024-04-26T18:24:20.953852+0000","last_unstale":"2024-04-26T18:24:22.959429+0000","last_undegraded":"2024-04-26T18:24:22.959429+0000","last_fullsized":"2024-04-26T18:24:22.959429+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:05.636897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-26T18:25:05.294705+0000","last_change":"2024-04-26T18:24:20.958023+0000","last_active":"2024-04-26T18:25:05.294705+0000","last_peered":"2024-04-26T18:25:05.294705+0000","last_clean":"2024-04-26T18:25:05.294705+0000","last_became_active":"2024-04-26T18:24:20.957642+0000","last_became_peered":"2024-04-26T18:24:20.957642+0000","last_unstale":"2024-04-26T18:25:05.294705+0000","last_undegraded":"2024-04-26T18:25:05.294705+0000","last_fullsized":"2024-04-26T18:25:05.294705+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:46.332710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244217,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70276,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":4,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":294,"num_read_kb":237,"num_write":244,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1561,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":569344,"data_stored":212367,"data_compressed":5529,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351000,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":983040,"data_stored":475556,"data_compressed":2885,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612637,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1150976,"data_stored":1137770,"data_compressed":43211,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776163,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":483599,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037800,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":483955,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299436,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32196,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712956,"statfs":{"total":95995035648,"available":95962066944,"internally_reserved":0,"allocated":1044480,"data_stored":1059274,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462962,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":466095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":538797,"data_compressed":4728,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":476542,"data_compressed":2885,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1060864,"data_stored":531082,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":483141,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888124,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":978944,"data_stored":465956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724597,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":1058952,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66385,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":65826,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64808,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1583,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:25:09.990 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:25:10.229 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.904+0000 7fd6138f8700 1 -- 172.21.15.84:0/3569580789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c102f60 msgr2=0x7fd60c106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.904+0000 7fd6138f8700 1 --2- 172.21.15.84:0/3569580789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c102f60 0x7fd60c106f60 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fd6080099c0 tx=0x7fd608009cd0 comp rx=0 tx=0).stop 2024-04-26T18:25:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.904+0000 7fd6138f8700 1 -- 172.21.15.84:0/3569580789 shutdown_connections 2024-04-26T18:25:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.904+0000 7fd6138f8700 1 --2- 172.21.15.84:0/3569580789 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd60c107640 0x7fd60c10f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.904+0000 7fd6138f8700 1 --2- 172.21.15.84:0/3569580789 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd60c1025b0 0x7fd60c102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.904+0000 7fd6138f8700 1 --2- 172.21.15.84:0/3569580789 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c102f60 0x7fd60c106f60 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.904+0000 7fd6138f8700 1 -- 172.21.15.84:0/3569580789 >> 172.21.15.84:0/3569580789 conn(0x7fd60c0792f0 msgr2=0x7fd60c079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.904+0000 7fd6138f8700 1 -- 172.21.15.84:0/3569580789 shutdown_connections 2024-04-26T18:25:11.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.904+0000 7fd6138f8700 1 -- 172.21.15.84:0/3569580789 wait complete. 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.905+0000 7fd6138f8700 1 Processor -- start 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.905+0000 7fd6138f8700 1 -- start start 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.905+0000 7fd6138f8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd60c1025b0 0x7fd60c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.905+0000 7fd6138f8700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c102f60 0x7fd60c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.905+0000 7fd6138f8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd60c107640 0x7fd60c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.905+0000 7fd6138f8700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c112530 con 0x7fd60c102f60 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.905+0000 7fd6138f8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1123b0 con 0x7fd60c1025b0 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.905+0000 7fd6138f8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd60c1126b0 con 0x7fd60c107640 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.905+0000 7fd610e93700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c102f60 0x7fd60c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.905+0000 7fd610e93700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c102f60 0x7fd60c19fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39682/0 (socket says 172.21.15.84:39682) 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.905+0000 7fd610e93700 1 -- 172.21.15.84:0/3998372698 learned_addr learned my addr 172.21.15.84:0/3998372698 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.905+0000 7fd611e95700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd60c107640 0x7fd60c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.906+0000 7fd611694700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd60c1025b0 0x7fd60c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:11.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.906+0000 7fd610e93700 1 -- 172.21.15.84:0/3998372698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd60c107640 msgr2=0x7fd60c1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.906+0000 7fd610e93700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd60c107640 0x7fd60c1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.906+0000 7fd610e93700 1 -- 172.21.15.84:0/3998372698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd60c1025b0 msgr2=0x7fd60c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.906+0000 7fd610e93700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd60c1025b0 0x7fd60c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.906+0000 7fd610e93700 1 -- 172.21.15.84:0/3998372698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd60c19a100 con 0x7fd60c102f60 2024-04-26T18:25:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.906+0000 7fd610e93700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c102f60 0x7fd60c19fd20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd608006010 tx=0x7fd6080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.906+0000 7fd6027fc700 1 -- 172.21.15.84:0/3998372698 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608027070 con 0x7fd60c102f60 2024-04-26T18:25:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.906+0000 7fd6027fc700 1 -- 172.21.15.84:0/3998372698 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd608003f20 con 0x7fd60c102f60 2024-04-26T18:25:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.906+0000 7fd6138f8700 1 -- 172.21.15.84:0/3998372698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd60c19a390 con 0x7fd60c102f60 2024-04-26T18:25:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.906+0000 7fd6027fc700 1 -- 172.21.15.84:0/3998372698 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd608022cd0 con 0x7fd60c102f60 2024-04-26T18:25:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.906+0000 7fd6138f8700 1 -- 172.21.15.84:0/3998372698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd60c1a66b0 con 0x7fd60c102f60 2024-04-26T18:25:11.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.907+0000 7fd6138f8700 1 -- 172.21.15.84:0/3998372698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd60c103a40 con 0x7fd60c102f60 2024-04-26T18:25:11.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.908+0000 7fd6027fc700 1 -- 172.21.15.84:0/3998372698 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd60802a070 con 0x7fd60c102f60 2024-04-26T18:25:11.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.909+0000 7fd6027fc700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd5f80a3cd0 0x7fd5f80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:11.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.909+0000 7fd611694700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd5f80a3cd0 0x7fd5f80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:11.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.909+0000 7fd6027fc700 1 -- 172.21.15.84:0/3998372698 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fd6080d1410 con 0x7fd60c102f60 2024-04-26T18:25:11.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.909+0000 7fd611694700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd5f80a3cd0 0x7fd5f80a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc005950 tx=0x7fd5fc0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:11.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:11.913+0000 7fd6027fc700 1 -- 172.21.15.84:0/3998372698 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd60805eff0 con 0x7fd60c102f60 2024-04-26T18:25:12.107 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.107+0000 7fd6138f8700 1 -- 172.21.15.84:0/3998372698 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd60c19aed0 con 0x7fd5f80a3cd0 2024-04-26T18:25:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.114+0000 7fd6027fc700 1 -- 172.21.15.84:0/3998372698 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259052 (secure 0 0 0) 0x7fd60c19aed0 con 0x7fd5f80a3cd0 2024-04-26T18:25:12.117 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:25:12.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.117+0000 7fd6138f8700 1 -- 172.21.15.84:0/3998372698 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd5f80a3cd0 msgr2=0x7fd5f80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:12.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.117+0000 7fd6138f8700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd5f80a3cd0 0x7fd5f80a6190 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc005950 tx=0x7fd5fc0058e0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.117+0000 7fd6138f8700 1 -- 172.21.15.84:0/3998372698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c102f60 msgr2=0x7fd60c19fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:12.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.117+0000 7fd6138f8700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c102f60 0x7fd60c19fd20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd608006010 tx=0x7fd6080047b0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.117+0000 7fd6138f8700 1 -- 172.21.15.84:0/3998372698 shutdown_connections 2024-04-26T18:25:12.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.117+0000 7fd6138f8700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fd5f80a3cd0 0x7fd5f80a6190 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.117+0000 7fd6138f8700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd60c107640 0x7fd60c1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.117+0000 7fd6138f8700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd60c1025b0 0x7fd60c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.117+0000 7fd6138f8700 1 --2- 172.21.15.84:0/3998372698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd60c102f60 0x7fd60c19fd20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:12.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.117+0000 7fd6138f8700 1 -- 172.21.15.84:0/3998372698 >> 172.21.15.84:0/3998372698 conn(0x7fd60c0792f0 msgr2=0x7fd60c06be90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:12.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.117+0000 7fd6138f8700 1 -- 172.21.15.84:0/3998372698 shutdown_connections 2024-04-26T18:25:12.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:12.117+0000 7fd6138f8700 1 -- 172.21.15.84:0/3998372698 wait complete. 2024-04-26T18:25:12.120 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-04-26T18:25:12.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:12 smithi084 ceph-mon[29141]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.7 KiB/s wr, 10 op/s; 197 B/s, 1 keys/s, 1 objects/s recovering 2024-04-26T18:25:12.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:12 smithi103 ceph-mon[32727]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.7 KiB/s wr, 10 op/s; 197 B/s, 1 keys/s, 1 objects/s recovering 2024-04-26T18:25:12.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:12 smithi174 ceph-mon[32705]: pgmap v324: 97 pgs: 97 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 2.7 KiB/s wr, 10 op/s; 197 B/s, 1 keys/s, 1 objects/s recovering 2024-04-26T18:25:12.797 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":324,"stamp":"2024-04-26T18:25:11.180755+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660878,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":449,"num_read_kb":361,"num_write":427,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182319,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":293,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369748,"kb_used_data":12208,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572076,"statfs":{"total":1151940427776,"available":1151561805824,"internally_reserved":0,"allocated":12500992,"data_stored":7753789,"data_compressed":175842,"data_compressed_allocated":237568,"data_compressed_original":1974272,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":17561,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":63,"num_write":12,"num_write_kb":22,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1583,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001882"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299065+0000","last_change":"2024-04-26T18:24:20.951581+0000","last_active":"2024-04-26T18:25:06.299065+0000","last_peered":"2024-04-26T18:25:06.299065+0000","last_clean":"2024-04-26T18:25:06.299065+0000","last_became_active":"2024-04-26T18:24:20.951260+0000","last_became_peered":"2024-04-26T18:24:20.951260+0000","last_unstale":"2024-04-26T18:25:06.299065+0000","last_undegraded":"2024-04-26T18:25:06.299065+0000","last_fullsized":"2024-04-26T18:25:06.299065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:54:28.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296510+0000","last_change":"2024-04-26T18:24:20.952263+0000","last_active":"2024-04-26T18:25:06.296510+0000","last_peered":"2024-04-26T18:25:06.296510+0000","last_clean":"2024-04-26T18:25:06.296510+0000","last_became_active":"2024-04-26T18:24:20.951930+0000","last_became_peered":"2024-04-26T18:24:20.951930+0000","last_unstale":"2024-04-26T18:25:06.296510+0000","last_undegraded":"2024-04-26T18:25:06.296510+0000","last_fullsized":"2024-04-26T18:25:06.296510+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:54.816572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296658+0000","last_change":"2024-04-26T18:24:20.952280+0000","last_active":"2024-04-26T18:25:06.296658+0000","last_peered":"2024-04-26T18:25:06.296658+0000","last_clean":"2024-04-26T18:25:06.296658+0000","last_became_active":"2024-04-26T18:24:20.951623+0000","last_became_peered":"2024-04-26T18:24:20.951623+0000","last_unstale":"2024-04-26T18:25:06.296658+0000","last_undegraded":"2024-04-26T18:25:06.296658+0000","last_fullsized":"2024-04-26T18:25:06.296658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:02.502261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295410+0000","last_change":"2024-04-26T18:24:20.962274+0000","last_active":"2024-04-26T18:25:06.295410+0000","last_peered":"2024-04-26T18:25:06.295410+0000","last_clean":"2024-04-26T18:25:06.295410+0000","last_became_active":"2024-04-26T18:24:20.961920+0000","last_became_peered":"2024-04-26T18:24:20.961920+0000","last_unstale":"2024-04-26T18:25:06.295410+0000","last_undegraded":"2024-04-26T18:25:06.295410+0000","last_fullsized":"2024-04-26T18:25:06.295410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:49:46.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297438+0000","last_change":"2024-04-26T18:24:20.949096+0000","last_active":"2024-04-26T18:25:06.297438+0000","last_peered":"2024-04-26T18:25:06.297438+0000","last_clean":"2024-04-26T18:25:06.297438+0000","last_became_active":"2024-04-26T18:24:20.948737+0000","last_became_peered":"2024-04-26T18:24:20.948737+0000","last_unstale":"2024-04-26T18:25:06.297438+0000","last_undegraded":"2024-04-26T18:25:06.297438+0000","last_fullsized":"2024-04-26T18:25:06.297438+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:12:40.371589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351373+0000","last_change":"2024-04-26T18:24:20.962670+0000","last_active":"2024-04-26T18:25:06.351373+0000","last_peered":"2024-04-26T18:25:06.351373+0000","last_clean":"2024-04-26T18:25:06.351373+0000","last_became_active":"2024-04-26T18:24:20.962309+0000","last_became_peered":"2024-04-26T18:24:20.962309+0000","last_unstale":"2024-04-26T18:25:06.351373+0000","last_undegraded":"2024-04-26T18:25:06.351373+0000","last_fullsized":"2024-04-26T18:25:06.351373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:40.315864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.350579+0000","last_change":"2024-04-26T18:25:06.350579+0000","last_active":"2024-04-26T18:25:06.350579+0000","last_peered":"2024-04-26T18:25:06.350579+0000","last_clean":"2024-04-26T18:25:06.350579+0000","last_became_active":"2024-04-26T18:25:06.350104+0000","last_became_peered":"2024-04-26T18:25:06.350104+0000","last_unstale":"2024-04-26T18:25:06.350579+0000","last_undegraded":"2024-04-26T18:25:06.350579+0000","last_fullsized":"2024-04-26T18:25:06.350579+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:00.950883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754918+0000","last_change":"2024-04-26T18:24:20.961096+0000","last_active":"2024-04-26T18:25:06.754918+0000","last_peered":"2024-04-26T18:25:06.754918+0000","last_clean":"2024-04-26T18:25:06.754918+0000","last_became_active":"2024-04-26T18:24:20.960796+0000","last_became_peered":"2024-04-26T18:24:20.960796+0000","last_unstale":"2024-04-26T18:25:06.754918+0000","last_undegraded":"2024-04-26T18:25:06.754918+0000","last_fullsized":"2024-04-26T18:25:06.754918+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:04.538836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298266+0000","last_change":"2024-04-26T18:24:20.959871+0000","last_active":"2024-04-26T18:25:06.298266+0000","last_peered":"2024-04-26T18:25:06.298266+0000","last_clean":"2024-04-26T18:25:06.298266+0000","last_became_active":"2024-04-26T18:24:20.959404+0000","last_became_peered":"2024-04-26T18:24:20.959404+0000","last_unstale":"2024-04-26T18:25:06.298266+0000","last_undegraded":"2024-04-26T18:25:06.298266+0000","last_fullsized":"2024-04-26T18:25:06.298266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:43.053618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351694+0000","last_change":"2024-04-26T18:24:20.963815+0000","last_active":"2024-04-26T18:25:06.351694+0000","last_peered":"2024-04-26T18:25:06.351694+0000","last_clean":"2024-04-26T18:25:06.351694+0000","last_became_active":"2024-04-26T18:24:20.963362+0000","last_became_peered":"2024-04-26T18:24:20.963362+0000","last_unstale":"2024-04-26T18:25:06.351694+0000","last_undegraded":"2024-04-26T18:25:06.351694+0000","last_fullsized":"2024-04-26T18:25:06.351694+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:57.907485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296283+0000","last_change":"2024-04-26T18:24:20.952109+0000","last_active":"2024-04-26T18:25:06.296283+0000","last_peered":"2024-04-26T18:25:06.296283+0000","last_clean":"2024-04-26T18:25:06.296283+0000","last_became_active":"2024-04-26T18:24:20.951776+0000","last_became_peered":"2024-04-26T18:24:20.951776+0000","last_unstale":"2024-04-26T18:25:06.296283+0000","last_undegraded":"2024-04-26T18:25:06.296283+0000","last_fullsized":"2024-04-26T18:25:06.296283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:53.220246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295295+0000","last_change":"2024-04-26T18:24:20.963965+0000","last_active":"2024-04-26T18:25:06.295295+0000","last_peered":"2024-04-26T18:25:06.295295+0000","last_clean":"2024-04-26T18:25:06.295295+0000","last_became_active":"2024-04-26T18:24:20.963648+0000","last_became_peered":"2024-04-26T18:24:20.963648+0000","last_unstale":"2024-04-26T18:25:06.295295+0000","last_undegraded":"2024-04-26T18:25:06.295295+0000","last_fullsized":"2024-04-26T18:25:06.295295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:04.080813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349258+0000","last_change":"2024-04-26T18:24:20.954186+0000","last_active":"2024-04-26T18:25:06.349258+0000","last_peered":"2024-04-26T18:25:06.349258+0000","last_clean":"2024-04-26T18:25:06.349258+0000","last_became_active":"2024-04-26T18:24:20.953431+0000","last_became_peered":"2024-04-26T18:24:20.953431+0000","last_unstale":"2024-04-26T18:25:06.349258+0000","last_undegraded":"2024-04-26T18:25:06.349258+0000","last_fullsized":"2024-04-26T18:25:06.349258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:21.498476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295940+0000","last_change":"2024-04-26T18:24:20.945392+0000","last_active":"2024-04-26T18:25:06.295940+0000","last_peered":"2024-04-26T18:25:06.295940+0000","last_clean":"2024-04-26T18:25:06.295940+0000","last_became_active":"2024-04-26T18:24:20.944971+0000","last_became_peered":"2024-04-26T18:24:20.944971+0000","last_unstale":"2024-04-26T18:25:06.295940+0000","last_undegraded":"2024-04-26T18:25:06.295940+0000","last_fullsized":"2024-04-26T18:25:06.295940+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:06.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299351+0000","last_change":"2024-04-26T18:25:06.299351+0000","last_active":"2024-04-26T18:25:06.299351+0000","last_peered":"2024-04-26T18:25:06.299351+0000","last_clean":"2024-04-26T18:25:06.299351+0000","last_became_active":"2024-04-26T18:25:06.298895+0000","last_became_peered":"2024-04-26T18:25:06.298895+0000","last_unstale":"2024-04-26T18:25:06.299351+0000","last_undegraded":"2024-04-26T18:25:06.299351+0000","last_fullsized":"2024-04-26T18:25:06.299351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:37.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297807+0000","last_change":"2024-04-26T18:24:20.954610+0000","last_active":"2024-04-26T18:25:06.297807+0000","last_peered":"2024-04-26T18:25:06.297807+0000","last_clean":"2024-04-26T18:25:06.297807+0000","last_became_active":"2024-04-26T18:24:20.954239+0000","last_became_peered":"2024-04-26T18:24:20.954239+0000","last_unstale":"2024-04-26T18:25:06.297807+0000","last_undegraded":"2024-04-26T18:25:06.297807+0000","last_fullsized":"2024-04-26T18:25:06.297807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:41.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298741+0000","last_change":"2024-04-26T18:24:20.956932+0000","last_active":"2024-04-26T18:25:06.298741+0000","last_peered":"2024-04-26T18:25:06.298741+0000","last_clean":"2024-04-26T18:25:06.298741+0000","last_became_active":"2024-04-26T18:24:20.956464+0000","last_became_peered":"2024-04-26T18:24:20.956464+0000","last_unstale":"2024-04-26T18:25:06.298741+0000","last_undegraded":"2024-04-26T18:25:06.298741+0000","last_fullsized":"2024-04-26T18:25:06.298741+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:01.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294148+0000","last_change":"2024-04-26T18:24:20.954870+0000","last_active":"2024-04-26T18:25:06.294148+0000","last_peered":"2024-04-26T18:25:06.294148+0000","last_clean":"2024-04-26T18:25:06.294148+0000","last_became_active":"2024-04-26T18:24:20.954542+0000","last_became_peered":"2024-04-26T18:24:20.954542+0000","last_unstale":"2024-04-26T18:25:06.294148+0000","last_undegraded":"2024-04-26T18:25:06.294148+0000","last_fullsized":"2024-04-26T18:25:06.294148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:13:51.932625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295319+0000","last_change":"2024-04-26T18:24:20.950994+0000","last_active":"2024-04-26T18:25:06.295319+0000","last_peered":"2024-04-26T18:25:06.295319+0000","last_clean":"2024-04-26T18:25:06.295319+0000","last_became_active":"2024-04-26T18:24:20.950343+0000","last_became_peered":"2024-04-26T18:24:20.950343+0000","last_unstale":"2024-04-26T18:25:06.295319+0000","last_undegraded":"2024-04-26T18:25:06.295319+0000","last_fullsized":"2024-04-26T18:25:06.295319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:25.274049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.346730+0000","last_change":"2024-04-26T18:24:17.929731+0000","last_active":"2024-04-26T18:25:06.346730+0000","last_peered":"2024-04-26T18:25:06.346730+0000","last_clean":"2024-04-26T18:25:06.346730+0000","last_became_active":"2024-04-26T18:24:17.929379+0000","last_became_peered":"2024-04-26T18:24:17.929379+0000","last_unstale":"2024-04-26T18:25:06.346730+0000","last_undegraded":"2024-04-26T18:25:06.346730+0000","last_fullsized":"2024-04-26T18:25:06.346730+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:44.821660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754917+0000","last_change":"2024-04-26T18:24:20.962885+0000","last_active":"2024-04-26T18:25:06.754917+0000","last_peered":"2024-04-26T18:25:06.754917+0000","last_clean":"2024-04-26T18:25:06.754917+0000","last_became_active":"2024-04-26T18:24:20.962674+0000","last_became_peered":"2024-04-26T18:24:20.962674+0000","last_unstale":"2024-04-26T18:25:06.754917+0000","last_undegraded":"2024-04-26T18:25:06.754917+0000","last_fullsized":"2024-04-26T18:25:06.754917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:32.444889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.755048+0000","last_change":"2024-04-26T18:24:18.141500+0000","last_active":"2024-04-26T18:25:06.755048+0000","last_peered":"2024-04-26T18:25:06.755048+0000","last_clean":"2024-04-26T18:25:06.755048+0000","last_became_active":"2024-04-26T18:24:18.140870+0000","last_became_peered":"2024-04-26T18:24:18.140870+0000","last_unstale":"2024-04-26T18:25:06.755048+0000","last_undegraded":"2024-04-26T18:25:06.755048+0000","last_fullsized":"2024-04-26T18:25:06.755048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:13.821872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296790+0000","last_change":"2024-04-26T18:24:20.949183+0000","last_active":"2024-04-26T18:25:06.296790+0000","last_peered":"2024-04-26T18:25:06.296790+0000","last_clean":"2024-04-26T18:25:06.296790+0000","last_became_active":"2024-04-26T18:24:20.948677+0000","last_became_peered":"2024-04-26T18:24:20.948677+0000","last_unstale":"2024-04-26T18:25:06.296790+0000","last_undegraded":"2024-04-26T18:25:06.296790+0000","last_fullsized":"2024-04-26T18:25:06.296790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:55.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.312239+0000","last_change":"2024-04-26T18:25:06.312239+0000","last_active":"2024-04-26T18:25:06.312239+0000","last_peered":"2024-04-26T18:25:06.312239+0000","last_clean":"2024-04-26T18:25:06.312239+0000","last_became_active":"2024-04-26T18:25:06.301304+0000","last_became_peered":"2024-04-26T18:25:06.301304+0000","last_unstale":"2024-04-26T18:25:06.312239+0000","last_undegraded":"2024-04-26T18:25:06.312239+0000","last_fullsized":"2024-04-26T18:25:06.312239+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:36.159348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298667+0000","last_change":"2024-04-26T18:24:20.955555+0000","last_active":"2024-04-26T18:25:06.298667+0000","last_peered":"2024-04-26T18:25:06.298667+0000","last_clean":"2024-04-26T18:25:06.298667+0000","last_became_active":"2024-04-26T18:24:20.955176+0000","last_became_peered":"2024-04-26T18:24:20.955176+0000","last_unstale":"2024-04-26T18:25:06.298667+0000","last_undegraded":"2024-04-26T18:25:06.298667+0000","last_fullsized":"2024-04-26T18:25:06.298667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:02:30.368971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298703+0000","last_change":"2024-04-26T18:24:17.930919+0000","last_active":"2024-04-26T18:25:06.298703+0000","last_peered":"2024-04-26T18:25:06.298703+0000","last_clean":"2024-04-26T18:25:06.298703+0000","last_became_active":"2024-04-26T18:24:17.930596+0000","last_became_peered":"2024-04-26T18:24:17.930596+0000","last_unstale":"2024-04-26T18:25:06.298703+0000","last_undegraded":"2024-04-26T18:25:06.298703+0000","last_fullsized":"2024-04-26T18:25:06.298703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:57.476027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296942+0000","last_change":"2024-04-26T18:24:20.955796+0000","last_active":"2024-04-26T18:25:06.296942+0000","last_peered":"2024-04-26T18:25:06.296942+0000","last_clean":"2024-04-26T18:25:06.296942+0000","last_became_active":"2024-04-26T18:24:20.955432+0000","last_became_peered":"2024-04-26T18:24:20.955432+0000","last_unstale":"2024-04-26T18:25:06.296942+0000","last_undegraded":"2024-04-26T18:25:06.296942+0000","last_fullsized":"2024-04-26T18:25:06.296942+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:25.161797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":66,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:08.355085+0000","last_change":"2024-04-26T18:24:17.929505+0000","last_active":"2024-04-26T18:25:08.355085+0000","last_peered":"2024-04-26T18:25:08.355085+0000","last_clean":"2024-04-26T18:25:08.355085+0000","last_became_active":"2024-04-26T18:24:17.929205+0000","last_became_peered":"2024-04-26T18:24:17.929205+0000","last_unstale":"2024-04-26T18:25:08.355085+0000","last_undegraded":"2024-04-26T18:25:08.355085+0000","last_fullsized":"2024-04-26T18:25:08.355085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:42.460063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297815+0000","last_change":"2024-04-26T18:24:20.952134+0000","last_active":"2024-04-26T18:25:06.297815+0000","last_peered":"2024-04-26T18:25:06.297815+0000","last_clean":"2024-04-26T18:25:06.297815+0000","last_became_active":"2024-04-26T18:24:20.951758+0000","last_became_peered":"2024-04-26T18:24:20.951758+0000","last_unstale":"2024-04-26T18:25:06.297815+0000","last_undegraded":"2024-04-26T18:25:06.297815+0000","last_fullsized":"2024-04-26T18:25:06.297815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:25.389221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351236+0000","last_change":"2024-04-26T18:24:17.928704+0000","last_active":"2024-04-26T18:25:06.351236+0000","last_peered":"2024-04-26T18:25:06.351236+0000","last_clean":"2024-04-26T18:25:06.351236+0000","last_became_active":"2024-04-26T18:24:17.928343+0000","last_became_peered":"2024-04-26T18:24:17.928343+0000","last_unstale":"2024-04-26T18:25:06.351236+0000","last_undegraded":"2024-04-26T18:25:06.351236+0000","last_fullsized":"2024-04-26T18:25:06.351236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:49.400481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296463+0000","last_change":"2024-04-26T18:24:20.956808+0000","last_active":"2024-04-26T18:25:06.296463+0000","last_peered":"2024-04-26T18:25:06.296463+0000","last_clean":"2024-04-26T18:25:06.296463+0000","last_became_active":"2024-04-26T18:24:20.956466+0000","last_became_peered":"2024-04-26T18:24:20.956466+0000","last_unstale":"2024-04-26T18:25:06.296463+0000","last_undegraded":"2024-04-26T18:25:06.296463+0000","last_fullsized":"2024-04-26T18:25:06.296463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:31.150670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349100+0000","last_change":"2024-04-26T18:24:17.929134+0000","last_active":"2024-04-26T18:25:06.349100+0000","last_peered":"2024-04-26T18:25:06.349100+0000","last_clean":"2024-04-26T18:25:06.349100+0000","last_became_active":"2024-04-26T18:24:17.928798+0000","last_became_peered":"2024-04-26T18:24:17.928798+0000","last_unstale":"2024-04-26T18:25:06.349100+0000","last_undegraded":"2024-04-26T18:25:06.349100+0000","last_fullsized":"2024-04-26T18:25:06.349100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:10.450825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754806+0000","last_change":"2024-04-26T18:24:20.951594+0000","last_active":"2024-04-26T18:25:06.754806+0000","last_peered":"2024-04-26T18:25:06.754806+0000","last_clean":"2024-04-26T18:25:06.754806+0000","last_became_active":"2024-04-26T18:24:20.951148+0000","last_became_peered":"2024-04-26T18:24:20.951148+0000","last_unstale":"2024-04-26T18:25:06.754806+0000","last_undegraded":"2024-04-26T18:25:06.754806+0000","last_fullsized":"2024-04-26T18:25:06.754806+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:18.703292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296335+0000","last_change":"2024-04-26T18:24:17.931101+0000","last_active":"2024-04-26T18:25:06.296335+0000","last_peered":"2024-04-26T18:25:06.296335+0000","last_clean":"2024-04-26T18:25:06.296335+0000","last_became_active":"2024-04-26T18:24:17.930819+0000","last_became_peered":"2024-04-26T18:24:17.930819+0000","last_unstale":"2024-04-26T18:25:06.296335+0000","last_undegraded":"2024-04-26T18:25:06.296335+0000","last_fullsized":"2024-04-26T18:25:06.296335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.293645+0000","last_change":"2024-04-26T18:24:20.948866+0000","last_active":"2024-04-26T18:25:06.293645+0000","last_peered":"2024-04-26T18:25:06.293645+0000","last_clean":"2024-04-26T18:25:06.293645+0000","last_became_active":"2024-04-26T18:24:20.948562+0000","last_became_peered":"2024-04-26T18:24:20.948562+0000","last_unstale":"2024-04-26T18:25:06.293645+0000","last_undegraded":"2024-04-26T18:25:06.293645+0000","last_fullsized":"2024-04-26T18:25:06.293645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:33:19.616660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":89,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:08.310046+0000","last_change":"2024-04-26T18:24:17.926384+0000","last_active":"2024-04-26T18:25:08.310046+0000","last_peered":"2024-04-26T18:25:08.310046+0000","last_clean":"2024-04-26T18:25:08.310046+0000","last_became_active":"2024-04-26T18:24:17.926037+0000","last_became_peered":"2024-04-26T18:24:17.926037+0000","last_unstale":"2024-04-26T18:25:08.310046+0000","last_undegraded":"2024-04-26T18:25:08.310046+0000","last_fullsized":"2024-04-26T18:25:08.310046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:05.283755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":59,"num_read_kb":59,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298313+0000","last_change":"2024-04-26T18:24:20.954257+0000","last_active":"2024-04-26T18:25:06.298313+0000","last_peered":"2024-04-26T18:25:06.298313+0000","last_clean":"2024-04-26T18:25:06.298313+0000","last_became_active":"2024-04-26T18:24:20.953913+0000","last_became_peered":"2024-04-26T18:24:20.953913+0000","last_unstale":"2024-04-26T18:25:06.298313+0000","last_undegraded":"2024-04-26T18:25:06.298313+0000","last_fullsized":"2024-04-26T18:25:06.298313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:06.814156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296415+0000","last_change":"2024-04-26T18:24:17.932843+0000","last_active":"2024-04-26T18:25:06.296415+0000","last_peered":"2024-04-26T18:25:06.296415+0000","last_clean":"2024-04-26T18:25:06.296415+0000","last_became_active":"2024-04-26T18:24:17.932502+0000","last_became_peered":"2024-04-26T18:24:17.932502+0000","last_unstale":"2024-04-26T18:25:06.296415+0000","last_undegraded":"2024-04-26T18:25:06.296415+0000","last_fullsized":"2024-04-26T18:25:06.296415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:49.727094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296758+0000","last_change":"2024-04-26T18:24:20.959881+0000","last_active":"2024-04-26T18:25:06.296758+0000","last_peered":"2024-04-26T18:25:06.296758+0000","last_clean":"2024-04-26T18:25:06.296758+0000","last_became_active":"2024-04-26T18:24:20.959541+0000","last_became_peered":"2024-04-26T18:24:20.959541+0000","last_unstale":"2024-04-26T18:25:06.296758+0000","last_undegraded":"2024-04-26T18:25:06.296758+0000","last_fullsized":"2024-04-26T18:25:06.296758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:23.393521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296973+0000","last_change":"2024-04-26T18:24:18.142146+0000","last_active":"2024-04-26T18:25:06.296973+0000","last_peered":"2024-04-26T18:25:06.296973+0000","last_clean":"2024-04-26T18:25:06.296973+0000","last_became_active":"2024-04-26T18:24:18.141829+0000","last_became_peered":"2024-04-26T18:24:18.141829+0000","last_unstale":"2024-04-26T18:25:06.296973+0000","last_undegraded":"2024-04-26T18:25:06.296973+0000","last_fullsized":"2024-04-26T18:25:06.296973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:49.341638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298792+0000","last_change":"2024-04-26T18:24:20.957993+0000","last_active":"2024-04-26T18:25:06.298792+0000","last_peered":"2024-04-26T18:25:06.298792+0000","last_clean":"2024-04-26T18:25:06.298792+0000","last_became_active":"2024-04-26T18:24:20.957443+0000","last_became_peered":"2024-04-26T18:24:20.957443+0000","last_unstale":"2024-04-26T18:25:06.298792+0000","last_undegraded":"2024-04-26T18:25:06.298792+0000","last_fullsized":"2024-04-26T18:25:06.298792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:06:32.960546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'32","reported_seq":126,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:08.354501+0000","last_change":"2024-04-26T18:24:17.931283+0000","last_active":"2024-04-26T18:25:08.354501+0000","last_peered":"2024-04-26T18:25:08.354501+0000","last_clean":"2024-04-26T18:25:08.354501+0000","last_became_active":"2024-04-26T18:24:17.930943+0000","last_became_peered":"2024-04-26T18:24:17.930943+0000","last_unstale":"2024-04-26T18:25:08.354501+0000","last_undegraded":"2024-04-26T18:25:08.354501+0000","last_fullsized":"2024-04-26T18:25:08.354501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:50.511677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63072,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":68,"num_read_kb":95,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298440+0000","last_change":"2024-04-26T18:24:20.947509+0000","last_active":"2024-04-26T18:25:06.298440+0000","last_peered":"2024-04-26T18:25:06.298440+0000","last_clean":"2024-04-26T18:25:06.298440+0000","last_became_active":"2024-04-26T18:24:20.947181+0000","last_became_peered":"2024-04-26T18:24:20.947181+0000","last_unstale":"2024-04-26T18:25:06.298440+0000","last_undegraded":"2024-04-26T18:25:06.298440+0000","last_fullsized":"2024-04-26T18:25:06.298440+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.262312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295054+0000","last_change":"2024-04-26T18:24:17.934042+0000","last_active":"2024-04-26T18:25:06.295054+0000","last_peered":"2024-04-26T18:25:06.295054+0000","last_clean":"2024-04-26T18:25:06.295054+0000","last_became_active":"2024-04-26T18:24:17.933537+0000","last_became_peered":"2024-04-26T18:24:17.933537+0000","last_unstale":"2024-04-26T18:25:06.295054+0000","last_undegraded":"2024-04-26T18:25:06.295054+0000","last_fullsized":"2024-04-26T18:25:06.295054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:35:06.904160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.348985+0000","last_change":"2024-04-26T18:24:20.957672+0000","last_active":"2024-04-26T18:25:06.348985+0000","last_peered":"2024-04-26T18:25:06.348985+0000","last_clean":"2024-04-26T18:25:06.348985+0000","last_became_active":"2024-04-26T18:24:20.957214+0000","last_became_peered":"2024-04-26T18:24:20.957214+0000","last_unstale":"2024-04-26T18:25:06.348985+0000","last_undegraded":"2024-04-26T18:25:06.348985+0000","last_fullsized":"2024-04-26T18:25:06.348985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:42:35.341858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295760+0000","last_change":"2024-04-26T18:24:17.930324+0000","last_active":"2024-04-26T18:25:06.295760+0000","last_peered":"2024-04-26T18:25:06.295760+0000","last_clean":"2024-04-26T18:25:06.295760+0000","last_became_active":"2024-04-26T18:24:17.930022+0000","last_became_peered":"2024-04-26T18:24:17.930022+0000","last_unstale":"2024-04-26T18:25:06.295760+0000","last_undegraded":"2024-04-26T18:25:06.295760+0000","last_fullsized":"2024-04-26T18:25:06.295760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:36.864816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.350388+0000","last_change":"2024-04-26T18:25:06.350388+0000","last_active":"2024-04-26T18:25:06.350388+0000","last_peered":"2024-04-26T18:25:06.350388+0000","last_clean":"2024-04-26T18:25:06.350388+0000","last_became_active":"2024-04-26T18:25:06.349836+0000","last_became_peered":"2024-04-26T18:25:06.349836+0000","last_unstale":"2024-04-26T18:25:06.350388+0000","last_undegraded":"2024-04-26T18:25:06.350388+0000","last_fullsized":"2024-04-26T18:25:06.350388+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:57.762115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:08.353497+0000","last_change":"2024-04-26T18:24:17.934676+0000","last_active":"2024-04-26T18:25:08.353497+0000","last_peered":"2024-04-26T18:25:08.353497+0000","last_clean":"2024-04-26T18:25:08.353497+0000","last_became_active":"2024-04-26T18:24:17.934307+0000","last_became_peered":"2024-04-26T18:24:17.934307+0000","last_unstale":"2024-04-26T18:25:08.353497+0000","last_undegraded":"2024-04-26T18:25:08.353497+0000","last_fullsized":"2024-04-26T18:25:08.353497+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:27.312248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":34,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296271+0000","last_change":"2024-04-26T18:24:20.947371+0000","last_active":"2024-04-26T18:25:06.296271+0000","last_peered":"2024-04-26T18:25:06.296271+0000","last_clean":"2024-04-26T18:25:06.296271+0000","last_became_active":"2024-04-26T18:24:20.947048+0000","last_became_peered":"2024-04-26T18:24:20.947048+0000","last_unstale":"2024-04-26T18:25:06.296271+0000","last_undegraded":"2024-04-26T18:25:06.296271+0000","last_fullsized":"2024-04-26T18:25:06.296271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:55.038074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297413+0000","last_change":"2024-04-26T18:24:18.142680+0000","last_active":"2024-04-26T18:25:06.297413+0000","last_peered":"2024-04-26T18:25:06.297413+0000","last_clean":"2024-04-26T18:25:06.297413+0000","last_became_active":"2024-04-26T18:24:18.142392+0000","last_became_peered":"2024-04-26T18:24:18.142392+0000","last_unstale":"2024-04-26T18:25:06.297413+0000","last_undegraded":"2024-04-26T18:25:06.297413+0000","last_fullsized":"2024-04-26T18:25:06.297413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:52.654813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295840+0000","last_change":"2024-04-26T18:24:20.958131+0000","last_active":"2024-04-26T18:25:06.295840+0000","last_peered":"2024-04-26T18:25:06.295840+0000","last_clean":"2024-04-26T18:25:06.295840+0000","last_became_active":"2024-04-26T18:24:20.957701+0000","last_became_peered":"2024-04-26T18:24:20.957701+0000","last_unstale":"2024-04-26T18:25:06.295840+0000","last_undegraded":"2024-04-26T18:25:06.295840+0000","last_fullsized":"2024-04-26T18:25:06.295840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:02.018158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.319173+0000","last_change":"2024-04-26T18:25:06.319173+0000","last_active":"2024-04-26T18:25:06.319173+0000","last_peered":"2024-04-26T18:25:06.319173+0000","last_clean":"2024-04-26T18:25:06.319173+0000","last_became_active":"2024-04-26T18:25:06.302935+0000","last_became_peered":"2024-04-26T18:25:06.302935+0000","last_unstale":"2024-04-26T18:25:06.319173+0000","last_undegraded":"2024-04-26T18:25:06.319173+0000","last_fullsized":"2024-04-26T18:25:06.319173+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:40.198105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299067+0000","last_change":"2024-04-26T18:24:20.961230+0000","last_active":"2024-04-26T18:25:06.299067+0000","last_peered":"2024-04-26T18:25:06.299067+0000","last_clean":"2024-04-26T18:25:06.299067+0000","last_became_active":"2024-04-26T18:24:20.960807+0000","last_became_peered":"2024-04-26T18:24:20.960807+0000","last_unstale":"2024-04-26T18:25:06.299067+0000","last_undegraded":"2024-04-26T18:25:06.299067+0000","last_fullsized":"2024-04-26T18:25:06.299067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:39.535100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.293995+0000","last_change":"2024-04-26T18:24:18.142082+0000","last_active":"2024-04-26T18:25:06.293995+0000","last_peered":"2024-04-26T18:25:06.293995+0000","last_clean":"2024-04-26T18:25:06.293995+0000","last_became_active":"2024-04-26T18:24:18.141765+0000","last_became_peered":"2024-04-26T18:24:18.141765+0000","last_unstale":"2024-04-26T18:25:06.293995+0000","last_undegraded":"2024-04-26T18:25:06.293995+0000","last_fullsized":"2024-04-26T18:25:06.293995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:02.140027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351535+0000","last_change":"2024-04-26T18:24:20.952867+0000","last_active":"2024-04-26T18:25:06.351535+0000","last_peered":"2024-04-26T18:25:06.351535+0000","last_clean":"2024-04-26T18:25:06.351535+0000","last_became_active":"2024-04-26T18:24:20.952525+0000","last_became_peered":"2024-04-26T18:24:20.952525+0000","last_unstale":"2024-04-26T18:25:06.351535+0000","last_undegraded":"2024-04-26T18:25:06.351535+0000","last_fullsized":"2024-04-26T18:25:06.351535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:52.804312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:09.351239+0000","last_change":"2024-04-26T18:24:17.932270+0000","last_active":"2024-04-26T18:25:09.351239+0000","last_peered":"2024-04-26T18:25:09.351239+0000","last_clean":"2024-04-26T18:25:09.351239+0000","last_became_active":"2024-04-26T18:24:17.931901+0000","last_became_peered":"2024-04-26T18:24:17.931901+0000","last_unstale":"2024-04-26T18:25:09.351239+0000","last_undegraded":"2024-04-26T18:25:09.351239+0000","last_fullsized":"2024-04-26T18:25:09.351239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:13.524174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351380+0000","last_change":"2024-04-26T18:24:20.960612+0000","last_active":"2024-04-26T18:25:06.351380+0000","last_peered":"2024-04-26T18:25:06.351380+0000","last_clean":"2024-04-26T18:25:06.351380+0000","last_became_active":"2024-04-26T18:24:20.960280+0000","last_became_peered":"2024-04-26T18:24:20.960280+0000","last_unstale":"2024-04-26T18:25:06.351380+0000","last_undegraded":"2024-04-26T18:25:06.351380+0000","last_fullsized":"2024-04-26T18:25:06.351380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:53.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.300141+0000","last_change":"2024-04-26T18:25:06.299736+0000","last_active":"2024-04-26T18:25:06.300141+0000","last_peered":"2024-04-26T18:25:06.300141+0000","last_clean":"2024-04-26T18:25:06.300141+0000","last_became_active":"2024-04-26T18:25:06.299265+0000","last_became_peered":"2024-04-26T18:25:06.299265+0000","last_unstale":"2024-04-26T18:25:06.300141+0000","last_undegraded":"2024-04-26T18:25:06.300141+0000","last_fullsized":"2024-04-26T18:25:06.300141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:32.427551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":63,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:08.352661+0000","last_change":"2024-04-26T18:24:18.141181+0000","last_active":"2024-04-26T18:25:08.352661+0000","last_peered":"2024-04-26T18:25:08.352661+0000","last_clean":"2024-04-26T18:25:08.352661+0000","last_became_active":"2024-04-26T18:24:18.140854+0000","last_became_peered":"2024-04-26T18:24:18.140854+0000","last_unstale":"2024-04-26T18:25:08.352661+0000","last_undegraded":"2024-04-26T18:25:08.352661+0000","last_fullsized":"2024-04-26T18:25:08.352661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:27.900701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":32,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754882+0000","last_change":"2024-04-26T18:24:20.961653+0000","last_active":"2024-04-26T18:25:06.754882+0000","last_peered":"2024-04-26T18:25:06.754882+0000","last_clean":"2024-04-26T18:25:06.754882+0000","last_became_active":"2024-04-26T18:24:20.961330+0000","last_became_peered":"2024-04-26T18:24:20.961330+0000","last_unstale":"2024-04-26T18:25:06.754882+0000","last_undegraded":"2024-04-26T18:25:06.754882+0000","last_fullsized":"2024-04-26T18:25:06.754882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:19.033932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295928+0000","last_change":"2024-04-26T18:24:17.933613+0000","last_active":"2024-04-26T18:25:06.295928+0000","last_peered":"2024-04-26T18:25:06.295928+0000","last_clean":"2024-04-26T18:25:06.295928+0000","last_became_active":"2024-04-26T18:24:17.933219+0000","last_became_peered":"2024-04-26T18:24:17.933219+0000","last_unstale":"2024-04-26T18:25:06.295928+0000","last_undegraded":"2024-04-26T18:25:06.295928+0000","last_fullsized":"2024-04-26T18:25:06.295928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:04.215821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296210+0000","last_change":"2024-04-26T18:22:57.585634+0000","last_active":"2024-04-26T18:25:06.296210+0000","last_peered":"2024-04-26T18:25:06.296210+0000","last_clean":"2024-04-26T18:25:06.296210+0000","last_became_active":"2024-04-26T18:22:57.585198+0000","last_became_peered":"2024-04-26T18:22:57.585198+0000","last_unstale":"2024-04-26T18:25:06.296210+0000","last_undegraded":"2024-04-26T18:25:06.296210+0000","last_fullsized":"2024-04-26T18:25:06.296210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_clean_scrub_stamp":"2024-04-26T18:19:31.594684+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:56:11.861930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.348978+0000","last_change":"2024-04-26T18:24:20.950303+0000","last_active":"2024-04-26T18:25:06.348978+0000","last_peered":"2024-04-26T18:25:06.348978+0000","last_clean":"2024-04-26T18:25:06.348978+0000","last_became_active":"2024-04-26T18:24:20.950002+0000","last_became_peered":"2024-04-26T18:24:20.950002+0000","last_unstale":"2024-04-26T18:25:06.348978+0000","last_undegraded":"2024-04-26T18:25:06.348978+0000","last_fullsized":"2024-04-26T18:25:06.348978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:00.912752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296910+0000","last_change":"2024-04-26T18:24:17.934924+0000","last_active":"2024-04-26T18:25:06.296910+0000","last_peered":"2024-04-26T18:25:06.296910+0000","last_clean":"2024-04-26T18:25:06.296910+0000","last_became_active":"2024-04-26T18:24:17.934598+0000","last_became_peered":"2024-04-26T18:24:17.934598+0000","last_unstale":"2024-04-26T18:25:06.296910+0000","last_undegraded":"2024-04-26T18:25:06.296910+0000","last_fullsized":"2024-04-26T18:25:06.296910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:28.690043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298555+0000","last_change":"2024-04-26T18:24:18.142560+0000","last_active":"2024-04-26T18:25:06.298555+0000","last_peered":"2024-04-26T18:25:06.298555+0000","last_clean":"2024-04-26T18:25:06.298555+0000","last_became_active":"2024-04-26T18:24:18.142184+0000","last_became_peered":"2024-04-26T18:24:18.142184+0000","last_unstale":"2024-04-26T18:25:06.298555+0000","last_undegraded":"2024-04-26T18:25:06.298555+0000","last_fullsized":"2024-04-26T18:25:06.298555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:26.656113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349333+0000","last_change":"2024-04-26T18:24:20.952073+0000","last_active":"2024-04-26T18:25:06.349333+0000","last_peered":"2024-04-26T18:25:06.349333+0000","last_clean":"2024-04-26T18:25:06.349333+0000","last_became_active":"2024-04-26T18:24:20.951761+0000","last_became_peered":"2024-04-26T18:24:20.951761+0000","last_unstale":"2024-04-26T18:25:06.349333+0000","last_undegraded":"2024-04-26T18:25:06.349333+0000","last_fullsized":"2024-04-26T18:25:06.349333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:30.244060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'14","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294095+0000","last_change":"2024-04-26T18:24:17.927171+0000","last_active":"2024-04-26T18:25:06.294095+0000","last_peered":"2024-04-26T18:25:06.294095+0000","last_clean":"2024-04-26T18:25:06.294095+0000","last_became_active":"2024-04-26T18:24:17.926816+0000","last_became_peered":"2024-04-26T18:24:17.926816+0000","last_unstale":"2024-04-26T18:25:06.294095+0000","last_undegraded":"2024-04-26T18:25:06.294095+0000","last_fullsized":"2024-04-26T18:25:06.294095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:18.249363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.755057+0000","last_change":"2024-04-26T18:24:20.951684+0000","last_active":"2024-04-26T18:25:06.755057+0000","last_peered":"2024-04-26T18:25:06.755057+0000","last_clean":"2024-04-26T18:25:06.755057+0000","last_became_active":"2024-04-26T18:24:20.951364+0000","last_became_peered":"2024-04-26T18:24:20.951364+0000","last_unstale":"2024-04-26T18:25:06.755057+0000","last_undegraded":"2024-04-26T18:25:06.755057+0000","last_fullsized":"2024-04-26T18:25:06.755057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:28.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295402+0000","last_change":"2024-04-26T18:24:18.229770+0000","last_active":"2024-04-26T18:25:06.295402+0000","last_peered":"2024-04-26T18:25:06.295402+0000","last_clean":"2024-04-26T18:25:06.295402+0000","last_became_active":"2024-04-26T18:24:18.229446+0000","last_became_peered":"2024-04-26T18:24:18.229446+0000","last_unstale":"2024-04-26T18:25:06.295402+0000","last_undegraded":"2024-04-26T18:25:06.295402+0000","last_fullsized":"2024-04-26T18:25:06.295402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:14.794797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296547+0000","last_change":"2024-04-26T18:24:20.960513+0000","last_active":"2024-04-26T18:25:06.296547+0000","last_peered":"2024-04-26T18:25:06.296547+0000","last_clean":"2024-04-26T18:25:06.296547+0000","last_became_active":"2024-04-26T18:24:20.959549+0000","last_became_peered":"2024-04-26T18:24:20.959549+0000","last_unstale":"2024-04-26T18:25:06.296547+0000","last_undegraded":"2024-04-26T18:25:06.296547+0000","last_fullsized":"2024-04-26T18:25:06.296547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:18:37.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297969+0000","last_change":"2024-04-26T18:24:18.141549+0000","last_active":"2024-04-26T18:25:06.297969+0000","last_peered":"2024-04-26T18:25:06.297969+0000","last_clean":"2024-04-26T18:25:06.297969+0000","last_became_active":"2024-04-26T18:24:18.141172+0000","last_became_peered":"2024-04-26T18:24:18.141172+0000","last_unstale":"2024-04-26T18:25:06.297969+0000","last_undegraded":"2024-04-26T18:25:06.297969+0000","last_fullsized":"2024-04-26T18:25:06.297969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:15.878130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297295+0000","last_change":"2024-04-26T18:24:20.948121+0000","last_active":"2024-04-26T18:25:06.297295+0000","last_peered":"2024-04-26T18:25:06.297295+0000","last_clean":"2024-04-26T18:25:06.297295+0000","last_became_active":"2024-04-26T18:24:20.947738+0000","last_became_peered":"2024-04-26T18:24:20.947738+0000","last_unstale":"2024-04-26T18:25:06.297295+0000","last_undegraded":"2024-04-26T18:25:06.297295+0000","last_fullsized":"2024-04-26T18:25:06.297295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:04.237302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754809+0000","last_change":"2024-04-26T18:24:17.926631+0000","last_active":"2024-04-26T18:25:06.754809+0000","last_peered":"2024-04-26T18:25:06.754809+0000","last_clean":"2024-04-26T18:25:06.754809+0000","last_became_active":"2024-04-26T18:24:17.926259+0000","last_became_peered":"2024-04-26T18:24:17.926259+0000","last_unstale":"2024-04-26T18:25:06.754809+0000","last_undegraded":"2024-04-26T18:25:06.754809+0000","last_fullsized":"2024-04-26T18:25:06.754809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:47.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298685+0000","last_change":"2024-04-26T18:24:20.947283+0000","last_active":"2024-04-26T18:25:06.298685+0000","last_peered":"2024-04-26T18:25:06.298685+0000","last_clean":"2024-04-26T18:25:06.298685+0000","last_became_active":"2024-04-26T18:24:20.946812+0000","last_became_peered":"2024-04-26T18:24:20.946812+0000","last_unstale":"2024-04-26T18:25:06.298685+0000","last_undegraded":"2024-04-26T18:25:06.298685+0000","last_fullsized":"2024-04-26T18:25:06.298685+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:33.730698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294878+0000","last_change":"2024-04-26T18:24:17.931843+0000","last_active":"2024-04-26T18:25:06.294878+0000","last_peered":"2024-04-26T18:25:06.294878+0000","last_clean":"2024-04-26T18:25:06.294878+0000","last_became_active":"2024-04-26T18:24:17.931442+0000","last_became_peered":"2024-04-26T18:24:17.931442+0000","last_unstale":"2024-04-26T18:25:06.294878+0000","last_undegraded":"2024-04-26T18:25:06.294878+0000","last_fullsized":"2024-04-26T18:25:06.294878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:09.649435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.755046+0000","last_change":"2024-04-26T18:24:20.955661+0000","last_active":"2024-04-26T18:25:06.755046+0000","last_peered":"2024-04-26T18:25:06.755046+0000","last_clean":"2024-04-26T18:25:06.755046+0000","last_became_active":"2024-04-26T18:24:20.955344+0000","last_became_peered":"2024-04-26T18:24:20.955344+0000","last_unstale":"2024-04-26T18:25:06.755046+0000","last_undegraded":"2024-04-26T18:25:06.755046+0000","last_fullsized":"2024-04-26T18:25:06.755046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:44.274381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.315466+0000","last_change":"2024-04-26T18:25:06.315466+0000","last_active":"2024-04-26T18:25:06.315466+0000","last_peered":"2024-04-26T18:25:06.315466+0000","last_clean":"2024-04-26T18:25:06.315466+0000","last_became_active":"2024-04-26T18:25:06.301788+0000","last_became_peered":"2024-04-26T18:25:06.301788+0000","last_unstale":"2024-04-26T18:25:06.315466+0000","last_undegraded":"2024-04-26T18:25:06.315466+0000","last_fullsized":"2024-04-26T18:25:06.315466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:12.998583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295745+0000","last_change":"2024-04-26T18:24:20.955746+0000","last_active":"2024-04-26T18:25:06.295745+0000","last_peered":"2024-04-26T18:25:06.295745+0000","last_clean":"2024-04-26T18:25:06.295745+0000","last_became_active":"2024-04-26T18:24:20.953652+0000","last_became_peered":"2024-04-26T18:24:20.953652+0000","last_unstale":"2024-04-26T18:25:06.295745+0000","last_undegraded":"2024-04-26T18:25:06.295745+0000","last_fullsized":"2024-04-26T18:25:06.295745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:31.414185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.309800+0000","last_change":"2024-04-26T18:25:06.309393+0000","last_active":"2024-04-26T18:25:06.309800+0000","last_peered":"2024-04-26T18:25:06.309800+0000","last_clean":"2024-04-26T18:25:06.309800+0000","last_became_active":"2024-04-26T18:25:06.302434+0000","last_became_peered":"2024-04-26T18:25:06.302434+0000","last_unstale":"2024-04-26T18:25:06.309800+0000","last_undegraded":"2024-04-26T18:25:06.309800+0000","last_fullsized":"2024-04-26T18:25:06.309800+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:54.730295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":6,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296883+0000","last_change":"2024-04-26T18:24:20.954238+0000","last_active":"2024-04-26T18:25:06.296883+0000","last_peered":"2024-04-26T18:25:06.296883+0000","last_clean":"2024-04-26T18:25:06.296883+0000","last_became_active":"2024-04-26T18:24:20.953883+0000","last_became_peered":"2024-04-26T18:24:20.953883+0000","last_unstale":"2024-04-26T18:25:06.296883+0000","last_undegraded":"2024-04-26T18:25:06.296883+0000","last_fullsized":"2024-04-26T18:25:06.296883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:10.580835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296855+0000","last_change":"2024-04-26T18:24:17.926901+0000","last_active":"2024-04-26T18:25:06.296855+0000","last_peered":"2024-04-26T18:25:06.296855+0000","last_clean":"2024-04-26T18:25:06.296855+0000","last_became_active":"2024-04-26T18:24:17.926291+0000","last_became_peered":"2024-04-26T18:24:17.926291+0000","last_unstale":"2024-04-26T18:25:06.296855+0000","last_undegraded":"2024-04-26T18:25:06.296855+0000","last_fullsized":"2024-04-26T18:25:06.296855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:17.695198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296092+0000","last_change":"2024-04-26T18:24:20.950578+0000","last_active":"2024-04-26T18:25:06.296092+0000","last_peered":"2024-04-26T18:25:06.296092+0000","last_clean":"2024-04-26T18:25:06.296092+0000","last_became_active":"2024-04-26T18:24:20.950174+0000","last_became_peered":"2024-04-26T18:24:20.950174+0000","last_unstale":"2024-04-26T18:25:06.296092+0000","last_undegraded":"2024-04-26T18:25:06.296092+0000","last_fullsized":"2024-04-26T18:25:06.296092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349265+0000","last_change":"2024-04-26T18:24:17.928054+0000","last_active":"2024-04-26T18:25:06.349265+0000","last_peered":"2024-04-26T18:25:06.349265+0000","last_clean":"2024-04-26T18:25:06.349265+0000","last_became_active":"2024-04-26T18:24:17.927715+0000","last_became_peered":"2024-04-26T18:24:17.927715+0000","last_unstale":"2024-04-26T18:25:06.349265+0000","last_undegraded":"2024-04-26T18:25:06.349265+0000","last_fullsized":"2024-04-26T18:25:06.349265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.155436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296772+0000","last_change":"2024-04-26T18:24:20.944512+0000","last_active":"2024-04-26T18:25:06.296772+0000","last_peered":"2024-04-26T18:25:06.296772+0000","last_clean":"2024-04-26T18:25:06.296772+0000","last_became_active":"2024-04-26T18:24:20.944123+0000","last_became_peered":"2024-04-26T18:24:20.944123+0000","last_unstale":"2024-04-26T18:25:06.296772+0000","last_undegraded":"2024-04-26T18:25:06.296772+0000","last_fullsized":"2024-04-26T18:25:06.296772+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.240206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298795+0000","last_change":"2024-04-26T18:24:20.959138+0000","last_active":"2024-04-26T18:25:06.298795+0000","last_peered":"2024-04-26T18:25:06.298795+0000","last_clean":"2024-04-26T18:25:06.298795+0000","last_became_active":"2024-04-26T18:24:20.958675+0000","last_became_peered":"2024-04-26T18:24:20.958675+0000","last_unstale":"2024-04-26T18:25:06.298795+0000","last_undegraded":"2024-04-26T18:25:06.298795+0000","last_fullsized":"2024-04-26T18:25:06.298795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:34:32.103380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294547+0000","last_change":"2024-04-26T18:24:20.961689+0000","last_active":"2024-04-26T18:25:06.294547+0000","last_peered":"2024-04-26T18:25:06.294547+0000","last_clean":"2024-04-26T18:25:06.294547+0000","last_became_active":"2024-04-26T18:24:20.961374+0000","last_became_peered":"2024-04-26T18:24:20.961374+0000","last_unstale":"2024-04-26T18:25:06.294547+0000","last_undegraded":"2024-04-26T18:25:06.294547+0000","last_fullsized":"2024-04-26T18:25:06.294547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:55.243887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294515+0000","last_change":"2024-04-26T18:24:20.950711+0000","last_active":"2024-04-26T18:25:06.294515+0000","last_peered":"2024-04-26T18:25:06.294515+0000","last_clean":"2024-04-26T18:25:06.294515+0000","last_became_active":"2024-04-26T18:24:20.950371+0000","last_became_peered":"2024-04-26T18:24:20.950371+0000","last_unstale":"2024-04-26T18:25:06.294515+0000","last_undegraded":"2024-04-26T18:25:06.294515+0000","last_fullsized":"2024-04-26T18:25:06.294515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:35.307921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295964+0000","last_change":"2024-04-26T18:24:20.959990+0000","last_active":"2024-04-26T18:25:06.295964+0000","last_peered":"2024-04-26T18:25:06.295964+0000","last_clean":"2024-04-26T18:25:06.295964+0000","last_became_active":"2024-04-26T18:24:20.959514+0000","last_became_peered":"2024-04-26T18:24:20.959514+0000","last_unstale":"2024-04-26T18:25:06.295964+0000","last_undegraded":"2024-04-26T18:25:06.295964+0000","last_fullsized":"2024-04-26T18:25:06.295964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:00.184448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349163+0000","last_change":"2024-04-26T18:24:20.957338+0000","last_active":"2024-04-26T18:25:06.349163+0000","last_peered":"2024-04-26T18:25:06.349163+0000","last_clean":"2024-04-26T18:25:06.349163+0000","last_became_active":"2024-04-26T18:24:20.956999+0000","last_became_peered":"2024-04-26T18:24:20.956999+0000","last_unstale":"2024-04-26T18:25:06.349163+0000","last_undegraded":"2024-04-26T18:25:06.349163+0000","last_fullsized":"2024-04-26T18:25:06.349163+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.066902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299283+0000","last_change":"2024-04-26T18:24:20.953634+0000","last_active":"2024-04-26T18:25:06.299283+0000","last_peered":"2024-04-26T18:25:06.299283+0000","last_clean":"2024-04-26T18:25:06.299283+0000","last_became_active":"2024-04-26T18:24:20.953259+0000","last_became_peered":"2024-04-26T18:24:20.953259+0000","last_unstale":"2024-04-26T18:25:06.299283+0000","last_undegraded":"2024-04-26T18:25:06.299283+0000","last_fullsized":"2024-04-26T18:25:06.299283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:00.095344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295478+0000","last_change":"2024-04-26T18:24:20.954997+0000","last_active":"2024-04-26T18:25:06.295478+0000","last_peered":"2024-04-26T18:25:06.295478+0000","last_clean":"2024-04-26T18:25:06.295478+0000","last_became_active":"2024-04-26T18:24:20.954223+0000","last_became_peered":"2024-04-26T18:24:20.954223+0000","last_unstale":"2024-04-26T18:25:06.295478+0000","last_undegraded":"2024-04-26T18:25:06.295478+0000","last_fullsized":"2024-04-26T18:25:06.295478+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:14.092455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754906+0000","last_change":"2024-04-26T18:24:20.957056+0000","last_active":"2024-04-26T18:25:06.754906+0000","last_peered":"2024-04-26T18:25:06.754906+0000","last_clean":"2024-04-26T18:25:06.754906+0000","last_became_active":"2024-04-26T18:24:20.956737+0000","last_became_peered":"2024-04-26T18:24:20.956737+0000","last_unstale":"2024-04-26T18:25:06.754906+0000","last_undegraded":"2024-04-26T18:25:06.754906+0000","last_fullsized":"2024-04-26T18:25:06.754906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:48.358862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.350206+0000","last_change":"2024-04-26T18:25:06.350206+0000","last_active":"2024-04-26T18:25:06.350206+0000","last_peered":"2024-04-26T18:25:06.350206+0000","last_clean":"2024-04-26T18:25:06.350206+0000","last_became_active":"2024-04-26T18:25:06.349866+0000","last_became_peered":"2024-04-26T18:25:06.349866+0000","last_unstale":"2024-04-26T18:25:06.350206+0000","last_undegraded":"2024-04-26T18:25:06.350206+0000","last_fullsized":"2024-04-26T18:25:06.350206+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:41.909056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754750+0000","last_change":"2024-04-26T18:24:20.956000+0000","last_active":"2024-04-26T18:25:06.754750+0000","last_peered":"2024-04-26T18:25:06.754750+0000","last_clean":"2024-04-26T18:25:06.754750+0000","last_became_active":"2024-04-26T18:24:20.955677+0000","last_became_peered":"2024-04-26T18:24:20.955677+0000","last_unstale":"2024-04-26T18:25:06.754750+0000","last_undegraded":"2024-04-26T18:25:06.754750+0000","last_fullsized":"2024-04-26T18:25:06.754750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:20.895674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298152+0000","last_change":"2024-04-26T18:24:20.959317+0000","last_active":"2024-04-26T18:25:06.298152+0000","last_peered":"2024-04-26T18:25:06.298152+0000","last_clean":"2024-04-26T18:25:06.298152+0000","last_became_active":"2024-04-26T18:24:20.958963+0000","last_became_peered":"2024-04-26T18:24:20.958963+0000","last_unstale":"2024-04-26T18:25:06.298152+0000","last_undegraded":"2024-04-26T18:25:06.298152+0000","last_fullsized":"2024-04-26T18:25:06.298152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:08.231089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349143+0000","last_change":"2024-04-26T18:24:20.954445+0000","last_active":"2024-04-26T18:25:06.349143+0000","last_peered":"2024-04-26T18:25:06.349143+0000","last_clean":"2024-04-26T18:25:06.349143+0000","last_became_active":"2024-04-26T18:24:20.953852+0000","last_became_peered":"2024-04-26T18:24:20.953852+0000","last_unstale":"2024-04-26T18:25:06.349143+0000","last_undegraded":"2024-04-26T18:25:06.349143+0000","last_fullsized":"2024-04-26T18:25:06.349143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:05.636897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299528+0000","last_change":"2024-04-26T18:24:20.958023+0000","last_active":"2024-04-26T18:25:06.299528+0000","last_peered":"2024-04-26T18:25:06.299528+0000","last_clean":"2024-04-26T18:25:06.299528+0000","last_became_active":"2024-04-26T18:24:20.957642+0000","last_became_peered":"2024-04-26T18:24:20.957642+0000","last_unstale":"2024-04-26T18:25:06.299528+0000","last_undegraded":"2024-04-26T18:25:06.299528+0000","last_fullsized":"2024-04-26T18:25:06.299528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:46.332710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244217,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70276,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":313,"num_read_kb":242,"num_write":244,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1583,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":210828,"data_compressed":5529,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351001,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":484082,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612637,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1150976,"data_stored":1137770,"data_compressed":43211,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776163,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":483599,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037800,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":483955,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299437,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":1076495,"data_compressed":41368,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462963,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":483316,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1052672,"data_stored":538797,"data_compressed":4728,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1011712,"data_stored":476542,"data_compressed":2885,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":546742,"data_compressed":7620,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":483141,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888125,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":483177,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724598,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":1076173,"data_compressed":41368,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":66385,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64265,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64808,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:25:12.804 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph status --format=json 2024-04-26T18:25:13.042 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:13.432 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:13 smithi084 ceph-mon[29141]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:13.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:13 smithi103 ceph-mon[32727]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:13 smithi174 ceph-mon[32705]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:14.481 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:14 smithi084 ceph-mon[29141]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s; 197 B/s, 1 keys/s, 1 objects/s recovering 2024-04-26T18:25:14.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:14 smithi103 ceph-mon[32727]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s; 197 B/s, 1 keys/s, 1 objects/s recovering 2024-04-26T18:25:14.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:14 smithi174 ceph-mon[32705]: pgmap v325: 97 pgs: 97 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.5 KiB/s wr, 6 op/s; 197 B/s, 1 keys/s, 1 objects/s recovering 2024-04-26T18:25:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.714+0000 7fe998ea0700 1 -- 172.21.15.84:0/178118746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9941094c0 msgr2=0x7fe9941098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.714+0000 7fe998ea0700 1 --2- 172.21.15.84:0/178118746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9941094c0 0x7fe9941098a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fe988009990 tx=0x7fe988009ca0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.714+0000 7fe998ea0700 1 -- 172.21.15.84:0/178118746 shutdown_connections 2024-04-26T18:25:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.714+0000 7fe998ea0700 1 --2- 172.21.15.84:0/178118746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe9941094c0 0x7fe9941098a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.714+0000 7fe998ea0700 1 --2- 172.21.15.84:0/178118746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe994103e30 0x7fe994110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.714+0000 7fe998ea0700 1 --2- 172.21.15.84:0/178118746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe994103470 0x7fe9941038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.714+0000 7fe998ea0700 1 -- 172.21.15.84:0/178118746 >> 172.21.15.84:0/178118746 conn(0x7fe99407a580 msgr2=0x7fe99407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.714+0000 7fe998ea0700 1 -- 172.21.15.84:0/178118746 shutdown_connections 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.714+0000 7fe998ea0700 1 -- 172.21.15.84:0/178118746 wait complete. 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe998ea0700 1 Processor -- start 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe998ea0700 1 -- start start 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe998ea0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe994103470 0x7fe994075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe998ea0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe994103e30 0x7fe994075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe998ea0700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9941094c0 0x7fe9940761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe998ea0700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe994113640 con 0x7fe994103e30 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe998ea0700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941134c0 con 0x7fe9941094c0 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe998ea0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe9941137c0 con 0x7fe994103470 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe991d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe994103e30 0x7fe994075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe991d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe994103e30 0x7fe994075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39698/0 (socket says 172.21.15.84:39698) 2024-04-26T18:25:14.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe991d9b700 1 -- 172.21.15.84:0/3660815299 learned_addr learned my addr 172.21.15.84:0/3660815299 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe99259c700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe994103470 0x7fe994075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe992d9d700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9941094c0 0x7fe9940761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe991d9b700 1 -- 172.21.15.84:0/3660815299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe994103470 msgr2=0x7fe994075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe991d9b700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe994103470 0x7fe994075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe991d9b700 1 -- 172.21.15.84:0/3660815299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9941094c0 msgr2=0x7fe9940761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.715+0000 7fe991d9b700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9941094c0 0x7fe9940761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.716+0000 7fe991d9b700 1 -- 172.21.15.84:0/3660815299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9941a8020 con 0x7fe994103e30 2024-04-26T18:25:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.716+0000 7fe991d9b700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe994103e30 0x7fe994075c60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe98400b530 tx=0x7fe98400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.716+0000 7fe9837fe700 1 -- 172.21.15.84:0/3660815299 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984015020 con 0x7fe994103e30 2024-04-26T18:25:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.716+0000 7fe9837fe700 1 -- 172.21.15.84:0/3660815299 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe984011ce0 con 0x7fe994103e30 2024-04-26T18:25:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.716+0000 7fe998ea0700 1 -- 172.21.15.84:0/3660815299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9941a8310 con 0x7fe994103e30 2024-04-26T18:25:14.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.716+0000 7fe9837fe700 1 -- 172.21.15.84:0/3660815299 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe984010dd0 con 0x7fe994103e30 2024-04-26T18:25:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.716+0000 7fe998ea0700 1 -- 172.21.15.84:0/3660815299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9941a8850 con 0x7fe994103e30 2024-04-26T18:25:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.717+0000 7fe998ea0700 1 -- 172.21.15.84:0/3660815299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe99406a110 con 0x7fe994103e30 2024-04-26T18:25:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.718+0000 7fe9837fe700 1 -- 172.21.15.84:0/3660815299 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe984018020 con 0x7fe994103e30 2024-04-26T18:25:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.719+0000 7fe9837fe700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe97c0a3d50 0x7fe97c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.719+0000 7fe99259c700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe97c0a3d50 0x7fe97c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.719+0000 7fe9837fe700 1 -- 172.21.15.84:0/3660815299 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fe9840cfaf0 con 0x7fe994103e30 2024-04-26T18:25:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.719+0000 7fe99259c700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe97c0a3d50 0x7fe97c0a6210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe98800b5c0 tx=0x7fe988005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:14.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:14.724+0000 7fe9837fe700 1 -- 172.21.15.84:0/3660815299 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe98401f020 con 0x7fe994103e30 2024-04-26T18:25:15.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.010+0000 7fe998ea0700 1 -- 172.21.15.84:0/3660815299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fe994076c50 con 0x7fe994103e30 2024-04-26T18:25:15.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.011+0000 7fe9837fe700 1 -- 172.21.15.84:0/3660815299 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+3293 (secure 0 0 0) 0x7fe98409c470 con 0x7fe994103e30 2024-04-26T18:25:15.012 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:25:15.012 INFO:teuthology.orchestra.run.smithi084.stdout:{"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":428,"monmap":{"epoch":3,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":91,"num_osds":12,"num_up_osds":12,"osd_up_since":1714155799,"num_in_osds":12,"osd_in_since":1714155779,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":102,"data_bytes":660878,"bytes_used":378658816,"bytes_avail":1151561768960,"bytes_total":1151940427776,"recovering_objects_per_sec":1,"recovering_bytes_per_sec":197,"recovering_keys_per_sec":1,"num_objects_recovered":14,"num_bytes_recovered":1583,"num_keys_recovered":10,"read_bytes_sec":4350,"write_bytes_sec":1535,"read_op_per_sec":5,"write_op_per_sec":0},"fsmap":{"epoch":28,"id":1,"up":5,"in":5,"max":5,"by_rank":[{"filesystem_id":1,"rank":4,"name":"g","status":"up:standby-replay","gid":14598},{"filesystem_id":1,"rank":3,"name":"a","status":"up:standby-replay","gid":14610},{"filesystem_id":1,"rank":2,"name":"l","status":"up:standby-replay","gid":24385},{"filesystem_id":1,"rank":1,"name":"k","status":"up:standby-replay","gid":24389},{"filesystem_id":1,"rank":0,"name":"f","status":"up:standby-replay","gid":24406},{"filesystem_id":1,"rank":4,"name":"e","status":"up:active","gid":24407},{"filesystem_id":1,"rank":3,"name":"c","status":"up:active","gid":24418},{"filesystem_id":1,"rank":2,"name":"h","status":"up:active","gid":24422},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24430},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24434}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.84:8443/"}},"servicemap":{"epoch":22,"modified":"2024-04-26T18:24:47.174573+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"g":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"k":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"l":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-26T18:25:15.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.013+0000 7fe998ea0700 1 -- 172.21.15.84:0/3660815299 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe97c0a3d50 msgr2=0x7fe97c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:15.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.013+0000 7fe998ea0700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe97c0a3d50 0x7fe97c0a6210 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe98800b5c0 tx=0x7fe988005960 comp rx=0 tx=0).stop 2024-04-26T18:25:15.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.013+0000 7fe998ea0700 1 -- 172.21.15.84:0/3660815299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe994103e30 msgr2=0x7fe994075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:15.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.013+0000 7fe998ea0700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe994103e30 0x7fe994075c60 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe98400b530 tx=0x7fe98400b840 comp rx=0 tx=0).stop 2024-04-26T18:25:15.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.013+0000 7fe998ea0700 1 -- 172.21.15.84:0/3660815299 shutdown_connections 2024-04-26T18:25:15.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.013+0000 7fe998ea0700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe97c0a3d50 0x7fe97c0a6210 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.013+0000 7fe998ea0700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe9941094c0 0x7fe9940761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.013+0000 7fe998ea0700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe994103470 0x7fe994075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.013+0000 7fe998ea0700 1 --2- 172.21.15.84:0/3660815299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe994103e30 0x7fe994075c60 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:15.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.013+0000 7fe998ea0700 1 -- 172.21.15.84:0/3660815299 >> 172.21.15.84:0/3660815299 conn(0x7fe99407a580 msgr2=0x7fe994102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:15.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.013+0000 7fe998ea0700 1 -- 172.21.15.84:0/3660815299 shutdown_connections 2024-04-26T18:25:15.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:15.013+0000 7fe998ea0700 1 -- 172.21.15.84:0/3660815299 wait complete. 2024-04-26T18:25:15.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:15 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3660815299' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-26T18:25:15.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:15 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3660815299' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-26T18:25:15.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:15 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3660815299' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2024-04-26T18:25:15.745 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2024-04-26T18:25:15.746 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:25:15.984 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:16.366 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:16 smithi084 ceph-mon[29141]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s; 159 B/s, 1 keys/s, 1 objects/s recovering 2024-04-26T18:25:16.528 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:16 smithi103 ceph-mon[32727]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s; 159 B/s, 1 keys/s, 1 objects/s recovering 2024-04-26T18:25:16.590 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:16 smithi174 ceph-mon[32705]: pgmap v326: 97 pgs: 97 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.2 KiB/s wr, 7 op/s; 159 B/s, 1 keys/s, 1 objects/s recovering 2024-04-26T18:25:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.657+0000 7f4f272c8700 1 -- 172.21.15.84:0/2464061577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f20108390 msgr2=0x7f4f20108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.657+0000 7f4f272c8700 1 --2- 172.21.15.84:0/2464061577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f20108390 0x7f4f20108770 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009990 tx=0x7f4f10009ca0 comp rx=0 tx=0).stop 2024-04-26T18:25:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.657+0000 7f4f272c8700 1 -- 172.21.15.84:0/2464061577 shutdown_connections 2024-04-26T18:25:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.657+0000 7f4f272c8700 1 --2- 172.21.15.84:0/2464061577 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f20108390 0x7f4f20108770 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.657+0000 7f4f272c8700 1 --2- 172.21.15.84:0/2464061577 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f20102ce0 0x7f4f2010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.657+0000 7f4f272c8700 1 --2- 172.21.15.84:0/2464061577 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f20102320 0x7f4f201027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:17.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.657+0000 7f4f272c8700 1 -- 172.21.15.84:0/2464061577 >> 172.21.15.84:0/2464061577 conn(0x7f4f200792d0 msgr2=0x7f4f200796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.657+0000 7f4f272c8700 1 -- 172.21.15.84:0/2464061577 shutdown_connections 2024-04-26T18:25:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.657+0000 7f4f272c8700 1 -- 172.21.15.84:0/2464061577 wait complete. 2024-04-26T18:25:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f272c8700 1 Processor -- start 2024-04-26T18:25:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f272c8700 1 -- start start 2024-04-26T18:25:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f272c8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f20102320 0x7f4f2019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f272c8700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f20102ce0 0x7f4f2019afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f272c8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f20108390 0x7f4f201a5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f272c8700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4f201125b0 con 0x7f4f20102ce0 2024-04-26T18:25:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f272c8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4f20112430 con 0x7f4f20102320 2024-04-26T18:25:17.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f272c8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f20112730 con 0x7f4f20108390 2024-04-26T18:25:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f24863700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f20102ce0 0x7f4f2019afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f24863700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f20102ce0 0x7f4f2019afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48908/0 (socket says 172.21.15.84:48908) 2024-04-26T18:25:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f24863700 1 -- 172.21.15.84:0/3287090326 learned_addr learned my addr 172.21.15.84:0/3287090326 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f25064700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f20102320 0x7f4f2019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f25865700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f20108390 0x7f4f201a5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f24863700 1 -- 172.21.15.84:0/3287090326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f20108390 msgr2=0x7f4f201a5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f24863700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f20108390 0x7f4f201a5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f24863700 1 -- 172.21.15.84:0/3287090326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f20102320 msgr2=0x7f4f2019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f24863700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f20102320 0x7f4f2019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f24863700 1 -- 172.21.15.84:0/3287090326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f201a6050 con 0x7f4f20102ce0 2024-04-26T18:25:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f25865700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f20108390 0x7f4f201a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:25:17.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f25064700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f20102320 0x7f4f2019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:17.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.658+0000 7f4f24863700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f20102ce0 0x7f4f2019afd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00b530 tx=0x7f4f1c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:17.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.659+0000 7f4f167fc700 1 -- 172.21.15.84:0/3287090326 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c015020 con 0x7f4f20102ce0 2024-04-26T18:25:17.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.659+0000 7f4f167fc700 1 -- 172.21.15.84:0/3287090326 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f1c011ce0 con 0x7f4f20102ce0 2024-04-26T18:25:17.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.659+0000 7f4f272c8700 1 -- 172.21.15.84:0/3287090326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f201a6280 con 0x7f4f20102ce0 2024-04-26T18:25:17.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.659+0000 7f4f167fc700 1 -- 172.21.15.84:0/3287090326 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c010d20 con 0x7f4f20102ce0 2024-04-26T18:25:17.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.659+0000 7f4f272c8700 1 -- 172.21.15.84:0/3287090326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f201a6790 con 0x7f4f20102ce0 2024-04-26T18:25:17.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.660+0000 7f4f272c8700 1 -- 172.21.15.84:0/3287090326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f2006a110 con 0x7f4f20102ce0 2024-04-26T18:25:17.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.661+0000 7f4f167fc700 1 -- 172.21.15.84:0/3287090326 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4f1c018020 con 0x7f4f20102ce0 2024-04-26T18:25:17.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.661+0000 7f4f167fc700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4f0c0a3cd0 0x7f4f0c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:17.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.661+0000 7f4f25064700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4f0c0a3cd0 0x7f4f0c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:17.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.661+0000 7f4f167fc700 1 -- 172.21.15.84:0/3287090326 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f4f1c0cfcc0 con 0x7f4f20102ce0 2024-04-26T18:25:17.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.662+0000 7f4f25064700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4f0c0a3cd0 0x7f4f0c0a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000b5c0 tx=0x7f4f10005e80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:17.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.666+0000 7f4f167fc700 1 -- 172.21.15.84:0/3287090326 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f1c01f020 con 0x7f4f20102ce0 2024-04-26T18:25:17.860 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.859+0000 7f4f272c8700 1 -- 172.21.15.84:0/3287090326 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4f20063860 con 0x7f4f0c0a3cd0 2024-04-26T18:25:17.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.866+0000 7f4f167fc700 1 -- 172.21.15.84:0/3287090326 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259059 (secure 0 0 0) 0x7f4f20063860 con 0x7f4f0c0a3cd0 2024-04-26T18:25:17.872 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:25:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.869+0000 7f4f272c8700 1 -- 172.21.15.84:0/3287090326 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4f0c0a3cd0 msgr2=0x7f4f0c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.869+0000 7f4f272c8700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4f0c0a3cd0 0x7f4f0c0a6190 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000b5c0 tx=0x7f4f10005e80 comp rx=0 tx=0).stop 2024-04-26T18:25:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.869+0000 7f4f272c8700 1 -- 172.21.15.84:0/3287090326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f20102ce0 msgr2=0x7f4f2019afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.869+0000 7f4f272c8700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f20102ce0 0x7f4f2019afd0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c00b530 tx=0x7f4f1c00b8f0 comp rx=0 tx=0).stop 2024-04-26T18:25:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.869+0000 7f4f272c8700 1 -- 172.21.15.84:0/3287090326 shutdown_connections 2024-04-26T18:25:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.869+0000 7f4f272c8700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f4f0c0a3cd0 0x7f4f0c0a6190 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.869+0000 7f4f272c8700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f20108390 0x7f4f201a5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:17.874 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.869+0000 7f4f272c8700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f20102320 0x7f4f2019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.869+0000 7f4f272c8700 1 --2- 172.21.15.84:0/3287090326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f20102ce0 0x7f4f2019afd0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.869+0000 7f4f272c8700 1 -- 172.21.15.84:0/3287090326 >> 172.21.15.84:0/3287090326 conn(0x7f4f200792d0 msgr2=0x7f4f201011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.869+0000 7f4f272c8700 1 -- 172.21.15.84:0/3287090326 shutdown_connections 2024-04-26T18:25:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:17.869+0000 7f4f272c8700 1 -- 172.21.15.84:0/3287090326 wait complete. 2024-04-26T18:25:17.875 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-04-26T18:25:18.555 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:18 smithi084 ceph-mon[29141]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s; 145 B/s, 0 keys/s, 1 objects/s recovering 2024-04-26T18:25:18.555 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:18 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:25:18.580 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":327,"stamp":"2024-04-26T18:25:17.182566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":660878,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":503,"num_read_kb":386,"num_write":427,"num_write_kb":3077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182319,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":349,"ondisk_log_size":349,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369784,"kb_used_data":12240,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572040,"statfs":{"total":1151940427776,"available":1151561768960,"internally_reserved":0,"allocated":12533760,"data_stored":7775662,"data_compressed":181626,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8705,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":-4,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":75,"num_read_kb":57,"num_write":7,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1583,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.889926"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299065+0000","last_change":"2024-04-26T18:24:20.951581+0000","last_active":"2024-04-26T18:25:06.299065+0000","last_peered":"2024-04-26T18:25:06.299065+0000","last_clean":"2024-04-26T18:25:06.299065+0000","last_became_active":"2024-04-26T18:24:20.951260+0000","last_became_peered":"2024-04-26T18:24:20.951260+0000","last_unstale":"2024-04-26T18:25:06.299065+0000","last_undegraded":"2024-04-26T18:25:06.299065+0000","last_fullsized":"2024-04-26T18:25:06.299065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:54:28.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296510+0000","last_change":"2024-04-26T18:24:20.952263+0000","last_active":"2024-04-26T18:25:06.296510+0000","last_peered":"2024-04-26T18:25:06.296510+0000","last_clean":"2024-04-26T18:25:06.296510+0000","last_became_active":"2024-04-26T18:24:20.951930+0000","last_became_peered":"2024-04-26T18:24:20.951930+0000","last_unstale":"2024-04-26T18:25:06.296510+0000","last_undegraded":"2024-04-26T18:25:06.296510+0000","last_fullsized":"2024-04-26T18:25:06.296510+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:54.816572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296658+0000","last_change":"2024-04-26T18:24:20.952280+0000","last_active":"2024-04-26T18:25:06.296658+0000","last_peered":"2024-04-26T18:25:06.296658+0000","last_clean":"2024-04-26T18:25:06.296658+0000","last_became_active":"2024-04-26T18:24:20.951623+0000","last_became_peered":"2024-04-26T18:24:20.951623+0000","last_unstale":"2024-04-26T18:25:06.296658+0000","last_undegraded":"2024-04-26T18:25:06.296658+0000","last_fullsized":"2024-04-26T18:25:06.296658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:02.502261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295410+0000","last_change":"2024-04-26T18:24:20.962274+0000","last_active":"2024-04-26T18:25:06.295410+0000","last_peered":"2024-04-26T18:25:06.295410+0000","last_clean":"2024-04-26T18:25:06.295410+0000","last_became_active":"2024-04-26T18:24:20.961920+0000","last_became_peered":"2024-04-26T18:24:20.961920+0000","last_unstale":"2024-04-26T18:25:06.295410+0000","last_undegraded":"2024-04-26T18:25:06.295410+0000","last_fullsized":"2024-04-26T18:25:06.295410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:49:46.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297438+0000","last_change":"2024-04-26T18:24:20.949096+0000","last_active":"2024-04-26T18:25:06.297438+0000","last_peered":"2024-04-26T18:25:06.297438+0000","last_clean":"2024-04-26T18:25:06.297438+0000","last_became_active":"2024-04-26T18:24:20.948737+0000","last_became_peered":"2024-04-26T18:24:20.948737+0000","last_unstale":"2024-04-26T18:25:06.297438+0000","last_undegraded":"2024-04-26T18:25:06.297438+0000","last_fullsized":"2024-04-26T18:25:06.297438+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:12:40.371589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351373+0000","last_change":"2024-04-26T18:24:20.962670+0000","last_active":"2024-04-26T18:25:06.351373+0000","last_peered":"2024-04-26T18:25:06.351373+0000","last_clean":"2024-04-26T18:25:06.351373+0000","last_became_active":"2024-04-26T18:24:20.962309+0000","last_became_peered":"2024-04-26T18:24:20.962309+0000","last_unstale":"2024-04-26T18:25:06.351373+0000","last_undegraded":"2024-04-26T18:25:06.351373+0000","last_fullsized":"2024-04-26T18:25:06.351373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:40.315864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.350579+0000","last_change":"2024-04-26T18:25:06.350579+0000","last_active":"2024-04-26T18:25:06.350579+0000","last_peered":"2024-04-26T18:25:06.350579+0000","last_clean":"2024-04-26T18:25:06.350579+0000","last_became_active":"2024-04-26T18:25:06.350104+0000","last_became_peered":"2024-04-26T18:25:06.350104+0000","last_unstale":"2024-04-26T18:25:06.350579+0000","last_undegraded":"2024-04-26T18:25:06.350579+0000","last_fullsized":"2024-04-26T18:25:06.350579+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:00.950883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754918+0000","last_change":"2024-04-26T18:24:20.961096+0000","last_active":"2024-04-26T18:25:06.754918+0000","last_peered":"2024-04-26T18:25:06.754918+0000","last_clean":"2024-04-26T18:25:06.754918+0000","last_became_active":"2024-04-26T18:24:20.960796+0000","last_became_peered":"2024-04-26T18:24:20.960796+0000","last_unstale":"2024-04-26T18:25:06.754918+0000","last_undegraded":"2024-04-26T18:25:06.754918+0000","last_fullsized":"2024-04-26T18:25:06.754918+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:04.538836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298266+0000","last_change":"2024-04-26T18:24:20.959871+0000","last_active":"2024-04-26T18:25:06.298266+0000","last_peered":"2024-04-26T18:25:06.298266+0000","last_clean":"2024-04-26T18:25:06.298266+0000","last_became_active":"2024-04-26T18:24:20.959404+0000","last_became_peered":"2024-04-26T18:24:20.959404+0000","last_unstale":"2024-04-26T18:25:06.298266+0000","last_undegraded":"2024-04-26T18:25:06.298266+0000","last_fullsized":"2024-04-26T18:25:06.298266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:43.053618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351694+0000","last_change":"2024-04-26T18:24:20.963815+0000","last_active":"2024-04-26T18:25:06.351694+0000","last_peered":"2024-04-26T18:25:06.351694+0000","last_clean":"2024-04-26T18:25:06.351694+0000","last_became_active":"2024-04-26T18:24:20.963362+0000","last_became_peered":"2024-04-26T18:24:20.963362+0000","last_unstale":"2024-04-26T18:25:06.351694+0000","last_undegraded":"2024-04-26T18:25:06.351694+0000","last_fullsized":"2024-04-26T18:25:06.351694+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:57.907485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296283+0000","last_change":"2024-04-26T18:24:20.952109+0000","last_active":"2024-04-26T18:25:06.296283+0000","last_peered":"2024-04-26T18:25:06.296283+0000","last_clean":"2024-04-26T18:25:06.296283+0000","last_became_active":"2024-04-26T18:24:20.951776+0000","last_became_peered":"2024-04-26T18:24:20.951776+0000","last_unstale":"2024-04-26T18:25:06.296283+0000","last_undegraded":"2024-04-26T18:25:06.296283+0000","last_fullsized":"2024-04-26T18:25:06.296283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:53.220246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295295+0000","last_change":"2024-04-26T18:24:20.963965+0000","last_active":"2024-04-26T18:25:06.295295+0000","last_peered":"2024-04-26T18:25:06.295295+0000","last_clean":"2024-04-26T18:25:06.295295+0000","last_became_active":"2024-04-26T18:24:20.963648+0000","last_became_peered":"2024-04-26T18:24:20.963648+0000","last_unstale":"2024-04-26T18:25:06.295295+0000","last_undegraded":"2024-04-26T18:25:06.295295+0000","last_fullsized":"2024-04-26T18:25:06.295295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:04.080813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349258+0000","last_change":"2024-04-26T18:24:20.954186+0000","last_active":"2024-04-26T18:25:06.349258+0000","last_peered":"2024-04-26T18:25:06.349258+0000","last_clean":"2024-04-26T18:25:06.349258+0000","last_became_active":"2024-04-26T18:24:20.953431+0000","last_became_peered":"2024-04-26T18:24:20.953431+0000","last_unstale":"2024-04-26T18:25:06.349258+0000","last_undegraded":"2024-04-26T18:25:06.349258+0000","last_fullsized":"2024-04-26T18:25:06.349258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:21.498476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295940+0000","last_change":"2024-04-26T18:24:20.945392+0000","last_active":"2024-04-26T18:25:06.295940+0000","last_peered":"2024-04-26T18:25:06.295940+0000","last_clean":"2024-04-26T18:25:06.295940+0000","last_became_active":"2024-04-26T18:24:20.944971+0000","last_became_peered":"2024-04-26T18:24:20.944971+0000","last_unstale":"2024-04-26T18:25:06.295940+0000","last_undegraded":"2024-04-26T18:25:06.295940+0000","last_fullsized":"2024-04-26T18:25:06.295940+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:06.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299351+0000","last_change":"2024-04-26T18:25:06.299351+0000","last_active":"2024-04-26T18:25:06.299351+0000","last_peered":"2024-04-26T18:25:06.299351+0000","last_clean":"2024-04-26T18:25:06.299351+0000","last_became_active":"2024-04-26T18:25:06.298895+0000","last_became_peered":"2024-04-26T18:25:06.298895+0000","last_unstale":"2024-04-26T18:25:06.299351+0000","last_undegraded":"2024-04-26T18:25:06.299351+0000","last_fullsized":"2024-04-26T18:25:06.299351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:37.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297807+0000","last_change":"2024-04-26T18:24:20.954610+0000","last_active":"2024-04-26T18:25:06.297807+0000","last_peered":"2024-04-26T18:25:06.297807+0000","last_clean":"2024-04-26T18:25:06.297807+0000","last_became_active":"2024-04-26T18:24:20.954239+0000","last_became_peered":"2024-04-26T18:24:20.954239+0000","last_unstale":"2024-04-26T18:25:06.297807+0000","last_undegraded":"2024-04-26T18:25:06.297807+0000","last_fullsized":"2024-04-26T18:25:06.297807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:41.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298741+0000","last_change":"2024-04-26T18:24:20.956932+0000","last_active":"2024-04-26T18:25:06.298741+0000","last_peered":"2024-04-26T18:25:06.298741+0000","last_clean":"2024-04-26T18:25:06.298741+0000","last_became_active":"2024-04-26T18:24:20.956464+0000","last_became_peered":"2024-04-26T18:24:20.956464+0000","last_unstale":"2024-04-26T18:25:06.298741+0000","last_undegraded":"2024-04-26T18:25:06.298741+0000","last_fullsized":"2024-04-26T18:25:06.298741+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:01.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294148+0000","last_change":"2024-04-26T18:24:20.954870+0000","last_active":"2024-04-26T18:25:06.294148+0000","last_peered":"2024-04-26T18:25:06.294148+0000","last_clean":"2024-04-26T18:25:06.294148+0000","last_became_active":"2024-04-26T18:24:20.954542+0000","last_became_peered":"2024-04-26T18:24:20.954542+0000","last_unstale":"2024-04-26T18:25:06.294148+0000","last_undegraded":"2024-04-26T18:25:06.294148+0000","last_fullsized":"2024-04-26T18:25:06.294148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:13:51.932625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295319+0000","last_change":"2024-04-26T18:24:20.950994+0000","last_active":"2024-04-26T18:25:06.295319+0000","last_peered":"2024-04-26T18:25:06.295319+0000","last_clean":"2024-04-26T18:25:06.295319+0000","last_became_active":"2024-04-26T18:24:20.950343+0000","last_became_peered":"2024-04-26T18:24:20.950343+0000","last_unstale":"2024-04-26T18:25:06.295319+0000","last_undegraded":"2024-04-26T18:25:06.295319+0000","last_fullsized":"2024-04-26T18:25:06.295319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:25.274049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:11.353216+0000","last_change":"2024-04-26T18:24:17.929731+0000","last_active":"2024-04-26T18:25:11.353216+0000","last_peered":"2024-04-26T18:25:11.353216+0000","last_clean":"2024-04-26T18:25:11.353216+0000","last_became_active":"2024-04-26T18:24:17.929379+0000","last_became_peered":"2024-04-26T18:24:17.929379+0000","last_unstale":"2024-04-26T18:25:11.353216+0000","last_undegraded":"2024-04-26T18:25:11.353216+0000","last_fullsized":"2024-04-26T18:25:11.353216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:44.821660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754917+0000","last_change":"2024-04-26T18:24:20.962885+0000","last_active":"2024-04-26T18:25:06.754917+0000","last_peered":"2024-04-26T18:25:06.754917+0000","last_clean":"2024-04-26T18:25:06.754917+0000","last_became_active":"2024-04-26T18:24:20.962674+0000","last_became_peered":"2024-04-26T18:24:20.962674+0000","last_unstale":"2024-04-26T18:25:06.754917+0000","last_undegraded":"2024-04-26T18:25:06.754917+0000","last_fullsized":"2024-04-26T18:25:06.754917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:32.444889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.755048+0000","last_change":"2024-04-26T18:24:18.141500+0000","last_active":"2024-04-26T18:25:06.755048+0000","last_peered":"2024-04-26T18:25:06.755048+0000","last_clean":"2024-04-26T18:25:06.755048+0000","last_became_active":"2024-04-26T18:24:18.140870+0000","last_became_peered":"2024-04-26T18:24:18.140870+0000","last_unstale":"2024-04-26T18:25:06.755048+0000","last_undegraded":"2024-04-26T18:25:06.755048+0000","last_fullsized":"2024-04-26T18:25:06.755048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:13.821872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296790+0000","last_change":"2024-04-26T18:24:20.949183+0000","last_active":"2024-04-26T18:25:06.296790+0000","last_peered":"2024-04-26T18:25:06.296790+0000","last_clean":"2024-04-26T18:25:06.296790+0000","last_became_active":"2024-04-26T18:24:20.948677+0000","last_became_peered":"2024-04-26T18:24:20.948677+0000","last_unstale":"2024-04-26T18:25:06.296790+0000","last_undegraded":"2024-04-26T18:25:06.296790+0000","last_fullsized":"2024-04-26T18:25:06.296790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:55.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.312239+0000","last_change":"2024-04-26T18:25:06.312239+0000","last_active":"2024-04-26T18:25:06.312239+0000","last_peered":"2024-04-26T18:25:06.312239+0000","last_clean":"2024-04-26T18:25:06.312239+0000","last_became_active":"2024-04-26T18:25:06.301304+0000","last_became_peered":"2024-04-26T18:25:06.301304+0000","last_unstale":"2024-04-26T18:25:06.312239+0000","last_undegraded":"2024-04-26T18:25:06.312239+0000","last_fullsized":"2024-04-26T18:25:06.312239+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:36.159348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298667+0000","last_change":"2024-04-26T18:24:20.955555+0000","last_active":"2024-04-26T18:25:06.298667+0000","last_peered":"2024-04-26T18:25:06.298667+0000","last_clean":"2024-04-26T18:25:06.298667+0000","last_became_active":"2024-04-26T18:24:20.955176+0000","last_became_peered":"2024-04-26T18:24:20.955176+0000","last_unstale":"2024-04-26T18:25:06.298667+0000","last_undegraded":"2024-04-26T18:25:06.298667+0000","last_fullsized":"2024-04-26T18:25:06.298667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:02:30.368971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298703+0000","last_change":"2024-04-26T18:24:17.930919+0000","last_active":"2024-04-26T18:25:06.298703+0000","last_peered":"2024-04-26T18:25:06.298703+0000","last_clean":"2024-04-26T18:25:06.298703+0000","last_became_active":"2024-04-26T18:24:17.930596+0000","last_became_peered":"2024-04-26T18:24:17.930596+0000","last_unstale":"2024-04-26T18:25:06.298703+0000","last_undegraded":"2024-04-26T18:25:06.298703+0000","last_fullsized":"2024-04-26T18:25:06.298703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:57.476027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296942+0000","last_change":"2024-04-26T18:24:20.955796+0000","last_active":"2024-04-26T18:25:06.296942+0000","last_peered":"2024-04-26T18:25:06.296942+0000","last_clean":"2024-04-26T18:25:06.296942+0000","last_became_active":"2024-04-26T18:24:20.955432+0000","last_became_peered":"2024-04-26T18:24:20.955432+0000","last_unstale":"2024-04-26T18:25:06.296942+0000","last_undegraded":"2024-04-26T18:25:06.296942+0000","last_fullsized":"2024-04-26T18:25:06.296942+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:25.161797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":71,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:13.361487+0000","last_change":"2024-04-26T18:24:17.929505+0000","last_active":"2024-04-26T18:25:13.361487+0000","last_peered":"2024-04-26T18:25:13.361487+0000","last_clean":"2024-04-26T18:25:13.361487+0000","last_became_active":"2024-04-26T18:24:17.929205+0000","last_became_peered":"2024-04-26T18:24:17.929205+0000","last_unstale":"2024-04-26T18:25:13.361487+0000","last_undegraded":"2024-04-26T18:25:13.361487+0000","last_fullsized":"2024-04-26T18:25:13.361487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:42.460063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297815+0000","last_change":"2024-04-26T18:24:20.952134+0000","last_active":"2024-04-26T18:25:06.297815+0000","last_peered":"2024-04-26T18:25:06.297815+0000","last_clean":"2024-04-26T18:25:06.297815+0000","last_became_active":"2024-04-26T18:24:20.951758+0000","last_became_peered":"2024-04-26T18:24:20.951758+0000","last_unstale":"2024-04-26T18:25:06.297815+0000","last_undegraded":"2024-04-26T18:25:06.297815+0000","last_fullsized":"2024-04-26T18:25:06.297815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:25.389221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351236+0000","last_change":"2024-04-26T18:24:17.928704+0000","last_active":"2024-04-26T18:25:06.351236+0000","last_peered":"2024-04-26T18:25:06.351236+0000","last_clean":"2024-04-26T18:25:06.351236+0000","last_became_active":"2024-04-26T18:24:17.928343+0000","last_became_peered":"2024-04-26T18:24:17.928343+0000","last_unstale":"2024-04-26T18:25:06.351236+0000","last_undegraded":"2024-04-26T18:25:06.351236+0000","last_fullsized":"2024-04-26T18:25:06.351236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:49.400481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296463+0000","last_change":"2024-04-26T18:24:20.956808+0000","last_active":"2024-04-26T18:25:06.296463+0000","last_peered":"2024-04-26T18:25:06.296463+0000","last_clean":"2024-04-26T18:25:06.296463+0000","last_became_active":"2024-04-26T18:24:20.956466+0000","last_became_peered":"2024-04-26T18:24:20.956466+0000","last_unstale":"2024-04-26T18:25:06.296463+0000","last_undegraded":"2024-04-26T18:25:06.296463+0000","last_fullsized":"2024-04-26T18:25:06.296463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:31.150670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349100+0000","last_change":"2024-04-26T18:24:17.929134+0000","last_active":"2024-04-26T18:25:06.349100+0000","last_peered":"2024-04-26T18:25:06.349100+0000","last_clean":"2024-04-26T18:25:06.349100+0000","last_became_active":"2024-04-26T18:24:17.928798+0000","last_became_peered":"2024-04-26T18:24:17.928798+0000","last_unstale":"2024-04-26T18:25:06.349100+0000","last_undegraded":"2024-04-26T18:25:06.349100+0000","last_fullsized":"2024-04-26T18:25:06.349100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:10.450825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754806+0000","last_change":"2024-04-26T18:24:20.951594+0000","last_active":"2024-04-26T18:25:06.754806+0000","last_peered":"2024-04-26T18:25:06.754806+0000","last_clean":"2024-04-26T18:25:06.754806+0000","last_became_active":"2024-04-26T18:24:20.951148+0000","last_became_peered":"2024-04-26T18:24:20.951148+0000","last_unstale":"2024-04-26T18:25:06.754806+0000","last_undegraded":"2024-04-26T18:25:06.754806+0000","last_fullsized":"2024-04-26T18:25:06.754806+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:18.703292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296335+0000","last_change":"2024-04-26T18:24:17.931101+0000","last_active":"2024-04-26T18:25:06.296335+0000","last_peered":"2024-04-26T18:25:06.296335+0000","last_clean":"2024-04-26T18:25:06.296335+0000","last_became_active":"2024-04-26T18:24:17.930819+0000","last_became_peered":"2024-04-26T18:24:17.930819+0000","last_unstale":"2024-04-26T18:25:06.296335+0000","last_undegraded":"2024-04-26T18:25:06.296335+0000","last_fullsized":"2024-04-26T18:25:06.296335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.293645+0000","last_change":"2024-04-26T18:24:20.948866+0000","last_active":"2024-04-26T18:25:06.293645+0000","last_peered":"2024-04-26T18:25:06.293645+0000","last_clean":"2024-04-26T18:25:06.293645+0000","last_became_active":"2024-04-26T18:24:20.948562+0000","last_became_peered":"2024-04-26T18:24:20.948562+0000","last_unstale":"2024-04-26T18:25:06.293645+0000","last_undegraded":"2024-04-26T18:25:06.293645+0000","last_fullsized":"2024-04-26T18:25:06.293645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:33:19.616660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":99,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:13.317033+0000","last_change":"2024-04-26T18:24:17.926384+0000","last_active":"2024-04-26T18:25:13.317033+0000","last_peered":"2024-04-26T18:25:13.317033+0000","last_clean":"2024-04-26T18:25:13.317033+0000","last_became_active":"2024-04-26T18:24:17.926037+0000","last_became_peered":"2024-04-26T18:24:17.926037+0000","last_unstale":"2024-04-26T18:25:13.317033+0000","last_undegraded":"2024-04-26T18:25:13.317033+0000","last_fullsized":"2024-04-26T18:25:13.317033+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:05.283755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":69,"num_read_kb":69,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298313+0000","last_change":"2024-04-26T18:24:20.954257+0000","last_active":"2024-04-26T18:25:06.298313+0000","last_peered":"2024-04-26T18:25:06.298313+0000","last_clean":"2024-04-26T18:25:06.298313+0000","last_became_active":"2024-04-26T18:24:20.953913+0000","last_became_peered":"2024-04-26T18:24:20.953913+0000","last_unstale":"2024-04-26T18:25:06.298313+0000","last_undegraded":"2024-04-26T18:25:06.298313+0000","last_fullsized":"2024-04-26T18:25:06.298313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:06.814156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296415+0000","last_change":"2024-04-26T18:24:17.932843+0000","last_active":"2024-04-26T18:25:06.296415+0000","last_peered":"2024-04-26T18:25:06.296415+0000","last_clean":"2024-04-26T18:25:06.296415+0000","last_became_active":"2024-04-26T18:24:17.932502+0000","last_became_peered":"2024-04-26T18:24:17.932502+0000","last_unstale":"2024-04-26T18:25:06.296415+0000","last_undegraded":"2024-04-26T18:25:06.296415+0000","last_fullsized":"2024-04-26T18:25:06.296415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:49.727094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296758+0000","last_change":"2024-04-26T18:24:20.959881+0000","last_active":"2024-04-26T18:25:06.296758+0000","last_peered":"2024-04-26T18:25:06.296758+0000","last_clean":"2024-04-26T18:25:06.296758+0000","last_became_active":"2024-04-26T18:24:20.959541+0000","last_became_peered":"2024-04-26T18:24:20.959541+0000","last_unstale":"2024-04-26T18:25:06.296758+0000","last_undegraded":"2024-04-26T18:25:06.296758+0000","last_fullsized":"2024-04-26T18:25:06.296758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:23.393521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296973+0000","last_change":"2024-04-26T18:24:18.142146+0000","last_active":"2024-04-26T18:25:06.296973+0000","last_peered":"2024-04-26T18:25:06.296973+0000","last_clean":"2024-04-26T18:25:06.296973+0000","last_became_active":"2024-04-26T18:24:18.141829+0000","last_became_peered":"2024-04-26T18:24:18.141829+0000","last_unstale":"2024-04-26T18:25:06.296973+0000","last_undegraded":"2024-04-26T18:25:06.296973+0000","last_fullsized":"2024-04-26T18:25:06.296973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:49.341638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298792+0000","last_change":"2024-04-26T18:24:20.957993+0000","last_active":"2024-04-26T18:25:06.298792+0000","last_peered":"2024-04-26T18:25:06.298792+0000","last_clean":"2024-04-26T18:25:06.298792+0000","last_became_active":"2024-04-26T18:24:20.957443+0000","last_became_peered":"2024-04-26T18:24:20.957443+0000","last_unstale":"2024-04-26T18:25:06.298792+0000","last_undegraded":"2024-04-26T18:25:06.298792+0000","last_fullsized":"2024-04-26T18:25:06.298792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:06:32.960546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"89'32","reported_seq":136,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:13.361144+0000","last_change":"2024-04-26T18:24:17.931283+0000","last_active":"2024-04-26T18:25:13.361144+0000","last_peered":"2024-04-26T18:25:13.361144+0000","last_clean":"2024-04-26T18:25:13.361144+0000","last_became_active":"2024-04-26T18:24:17.930943+0000","last_became_peered":"2024-04-26T18:24:17.930943+0000","last_unstale":"2024-04-26T18:25:13.361144+0000","last_undegraded":"2024-04-26T18:25:13.361144+0000","last_fullsized":"2024-04-26T18:25:13.361144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:50.511677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63072,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":78,"num_read_kb":100,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298440+0000","last_change":"2024-04-26T18:24:20.947509+0000","last_active":"2024-04-26T18:25:06.298440+0000","last_peered":"2024-04-26T18:25:06.298440+0000","last_clean":"2024-04-26T18:25:06.298440+0000","last_became_active":"2024-04-26T18:24:20.947181+0000","last_became_peered":"2024-04-26T18:24:20.947181+0000","last_unstale":"2024-04-26T18:25:06.298440+0000","last_undegraded":"2024-04-26T18:25:06.298440+0000","last_fullsized":"2024-04-26T18:25:06.298440+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.262312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295054+0000","last_change":"2024-04-26T18:24:17.934042+0000","last_active":"2024-04-26T18:25:06.295054+0000","last_peered":"2024-04-26T18:25:06.295054+0000","last_clean":"2024-04-26T18:25:06.295054+0000","last_became_active":"2024-04-26T18:24:17.933537+0000","last_became_peered":"2024-04-26T18:24:17.933537+0000","last_unstale":"2024-04-26T18:25:06.295054+0000","last_undegraded":"2024-04-26T18:25:06.295054+0000","last_fullsized":"2024-04-26T18:25:06.295054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:35:06.904160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.348985+0000","last_change":"2024-04-26T18:24:20.957672+0000","last_active":"2024-04-26T18:25:06.348985+0000","last_peered":"2024-04-26T18:25:06.348985+0000","last_clean":"2024-04-26T18:25:06.348985+0000","last_became_active":"2024-04-26T18:24:20.957214+0000","last_became_peered":"2024-04-26T18:24:20.957214+0000","last_unstale":"2024-04-26T18:25:06.348985+0000","last_undegraded":"2024-04-26T18:25:06.348985+0000","last_fullsized":"2024-04-26T18:25:06.348985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:42:35.341858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295760+0000","last_change":"2024-04-26T18:24:17.930324+0000","last_active":"2024-04-26T18:25:06.295760+0000","last_peered":"2024-04-26T18:25:06.295760+0000","last_clean":"2024-04-26T18:25:06.295760+0000","last_became_active":"2024-04-26T18:24:17.930022+0000","last_became_peered":"2024-04-26T18:24:17.930022+0000","last_unstale":"2024-04-26T18:25:06.295760+0000","last_undegraded":"2024-04-26T18:25:06.295760+0000","last_fullsized":"2024-04-26T18:25:06.295760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:36.864816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.350388+0000","last_change":"2024-04-26T18:25:06.350388+0000","last_active":"2024-04-26T18:25:06.350388+0000","last_peered":"2024-04-26T18:25:06.350388+0000","last_clean":"2024-04-26T18:25:06.350388+0000","last_became_active":"2024-04-26T18:25:06.349836+0000","last_became_peered":"2024-04-26T18:25:06.349836+0000","last_unstale":"2024-04-26T18:25:06.350388+0000","last_undegraded":"2024-04-26T18:25:06.350388+0000","last_fullsized":"2024-04-26T18:25:06.350388+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:57.762115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:13.361443+0000","last_change":"2024-04-26T18:24:17.934676+0000","last_active":"2024-04-26T18:25:13.361443+0000","last_peered":"2024-04-26T18:25:13.361443+0000","last_clean":"2024-04-26T18:25:13.361443+0000","last_became_active":"2024-04-26T18:24:17.934307+0000","last_became_peered":"2024-04-26T18:24:17.934307+0000","last_unstale":"2024-04-26T18:25:13.361443+0000","last_undegraded":"2024-04-26T18:25:13.361443+0000","last_fullsized":"2024-04-26T18:25:13.361443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:27.312248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":39,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296271+0000","last_change":"2024-04-26T18:24:20.947371+0000","last_active":"2024-04-26T18:25:06.296271+0000","last_peered":"2024-04-26T18:25:06.296271+0000","last_clean":"2024-04-26T18:25:06.296271+0000","last_became_active":"2024-04-26T18:24:20.947048+0000","last_became_peered":"2024-04-26T18:24:20.947048+0000","last_unstale":"2024-04-26T18:25:06.296271+0000","last_undegraded":"2024-04-26T18:25:06.296271+0000","last_fullsized":"2024-04-26T18:25:06.296271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:55.038074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297413+0000","last_change":"2024-04-26T18:24:18.142680+0000","last_active":"2024-04-26T18:25:06.297413+0000","last_peered":"2024-04-26T18:25:06.297413+0000","last_clean":"2024-04-26T18:25:06.297413+0000","last_became_active":"2024-04-26T18:24:18.142392+0000","last_became_peered":"2024-04-26T18:24:18.142392+0000","last_unstale":"2024-04-26T18:25:06.297413+0000","last_undegraded":"2024-04-26T18:25:06.297413+0000","last_fullsized":"2024-04-26T18:25:06.297413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:52.654813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295840+0000","last_change":"2024-04-26T18:24:20.958131+0000","last_active":"2024-04-26T18:25:06.295840+0000","last_peered":"2024-04-26T18:25:06.295840+0000","last_clean":"2024-04-26T18:25:06.295840+0000","last_became_active":"2024-04-26T18:24:20.957701+0000","last_became_peered":"2024-04-26T18:24:20.957701+0000","last_unstale":"2024-04-26T18:25:06.295840+0000","last_undegraded":"2024-04-26T18:25:06.295840+0000","last_fullsized":"2024-04-26T18:25:06.295840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:02.018158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.319173+0000","last_change":"2024-04-26T18:25:06.319173+0000","last_active":"2024-04-26T18:25:06.319173+0000","last_peered":"2024-04-26T18:25:06.319173+0000","last_clean":"2024-04-26T18:25:06.319173+0000","last_became_active":"2024-04-26T18:25:06.302935+0000","last_became_peered":"2024-04-26T18:25:06.302935+0000","last_unstale":"2024-04-26T18:25:06.319173+0000","last_undegraded":"2024-04-26T18:25:06.319173+0000","last_fullsized":"2024-04-26T18:25:06.319173+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:40.198105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299067+0000","last_change":"2024-04-26T18:24:20.961230+0000","last_active":"2024-04-26T18:25:06.299067+0000","last_peered":"2024-04-26T18:25:06.299067+0000","last_clean":"2024-04-26T18:25:06.299067+0000","last_became_active":"2024-04-26T18:24:20.960807+0000","last_became_peered":"2024-04-26T18:24:20.960807+0000","last_unstale":"2024-04-26T18:25:06.299067+0000","last_undegraded":"2024-04-26T18:25:06.299067+0000","last_fullsized":"2024-04-26T18:25:06.299067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:39.535100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.293995+0000","last_change":"2024-04-26T18:24:18.142082+0000","last_active":"2024-04-26T18:25:06.293995+0000","last_peered":"2024-04-26T18:25:06.293995+0000","last_clean":"2024-04-26T18:25:06.293995+0000","last_became_active":"2024-04-26T18:24:18.141765+0000","last_became_peered":"2024-04-26T18:24:18.141765+0000","last_unstale":"2024-04-26T18:25:06.293995+0000","last_undegraded":"2024-04-26T18:25:06.293995+0000","last_fullsized":"2024-04-26T18:25:06.293995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:02.140027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351535+0000","last_change":"2024-04-26T18:24:20.952867+0000","last_active":"2024-04-26T18:25:06.351535+0000","last_peered":"2024-04-26T18:25:06.351535+0000","last_clean":"2024-04-26T18:25:06.351535+0000","last_became_active":"2024-04-26T18:24:20.952525+0000","last_became_peered":"2024-04-26T18:24:20.952525+0000","last_unstale":"2024-04-26T18:25:06.351535+0000","last_undegraded":"2024-04-26T18:25:06.351535+0000","last_fullsized":"2024-04-26T18:25:06.351535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:52.804312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:14.358125+0000","last_change":"2024-04-26T18:24:17.932270+0000","last_active":"2024-04-26T18:25:14.358125+0000","last_peered":"2024-04-26T18:25:14.358125+0000","last_clean":"2024-04-26T18:25:14.358125+0000","last_became_active":"2024-04-26T18:24:17.931901+0000","last_became_peered":"2024-04-26T18:24:17.931901+0000","last_unstale":"2024-04-26T18:25:14.358125+0000","last_undegraded":"2024-04-26T18:25:14.358125+0000","last_fullsized":"2024-04-26T18:25:14.358125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:13.524174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":38,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351380+0000","last_change":"2024-04-26T18:24:20.960612+0000","last_active":"2024-04-26T18:25:06.351380+0000","last_peered":"2024-04-26T18:25:06.351380+0000","last_clean":"2024-04-26T18:25:06.351380+0000","last_became_active":"2024-04-26T18:24:20.960280+0000","last_became_peered":"2024-04-26T18:24:20.960280+0000","last_unstale":"2024-04-26T18:25:06.351380+0000","last_undegraded":"2024-04-26T18:25:06.351380+0000","last_fullsized":"2024-04-26T18:25:06.351380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:53.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.300141+0000","last_change":"2024-04-26T18:25:06.299736+0000","last_active":"2024-04-26T18:25:06.300141+0000","last_peered":"2024-04-26T18:25:06.300141+0000","last_clean":"2024-04-26T18:25:06.300141+0000","last_became_active":"2024-04-26T18:25:06.299265+0000","last_became_peered":"2024-04-26T18:25:06.299265+0000","last_unstale":"2024-04-26T18:25:06.300141+0000","last_undegraded":"2024-04-26T18:25:06.300141+0000","last_fullsized":"2024-04-26T18:25:06.300141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:32.427551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":68,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:13.360881+0000","last_change":"2024-04-26T18:24:18.141181+0000","last_active":"2024-04-26T18:25:13.360881+0000","last_peered":"2024-04-26T18:25:13.360881+0000","last_clean":"2024-04-26T18:25:13.360881+0000","last_became_active":"2024-04-26T18:24:18.140854+0000","last_became_peered":"2024-04-26T18:24:18.140854+0000","last_unstale":"2024-04-26T18:25:13.360881+0000","last_undegraded":"2024-04-26T18:25:13.360881+0000","last_fullsized":"2024-04-26T18:25:13.360881+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:27.900701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754882+0000","last_change":"2024-04-26T18:24:20.961653+0000","last_active":"2024-04-26T18:25:06.754882+0000","last_peered":"2024-04-26T18:25:06.754882+0000","last_clean":"2024-04-26T18:25:06.754882+0000","last_became_active":"2024-04-26T18:24:20.961330+0000","last_became_peered":"2024-04-26T18:24:20.961330+0000","last_unstale":"2024-04-26T18:25:06.754882+0000","last_undegraded":"2024-04-26T18:25:06.754882+0000","last_fullsized":"2024-04-26T18:25:06.754882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:19.033932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295928+0000","last_change":"2024-04-26T18:24:17.933613+0000","last_active":"2024-04-26T18:25:06.295928+0000","last_peered":"2024-04-26T18:25:06.295928+0000","last_clean":"2024-04-26T18:25:06.295928+0000","last_became_active":"2024-04-26T18:24:17.933219+0000","last_became_peered":"2024-04-26T18:24:17.933219+0000","last_unstale":"2024-04-26T18:25:06.295928+0000","last_undegraded":"2024-04-26T18:25:06.295928+0000","last_fullsized":"2024-04-26T18:25:06.295928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:04.215821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296210+0000","last_change":"2024-04-26T18:22:57.585634+0000","last_active":"2024-04-26T18:25:06.296210+0000","last_peered":"2024-04-26T18:25:06.296210+0000","last_clean":"2024-04-26T18:25:06.296210+0000","last_became_active":"2024-04-26T18:22:57.585198+0000","last_became_peered":"2024-04-26T18:22:57.585198+0000","last_unstale":"2024-04-26T18:25:06.296210+0000","last_undegraded":"2024-04-26T18:25:06.296210+0000","last_fullsized":"2024-04-26T18:25:06.296210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_clean_scrub_stamp":"2024-04-26T18:19:31.594684+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:56:11.861930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.348978+0000","last_change":"2024-04-26T18:24:20.950303+0000","last_active":"2024-04-26T18:25:06.348978+0000","last_peered":"2024-04-26T18:25:06.348978+0000","last_clean":"2024-04-26T18:25:06.348978+0000","last_became_active":"2024-04-26T18:24:20.950002+0000","last_became_peered":"2024-04-26T18:24:20.950002+0000","last_unstale":"2024-04-26T18:25:06.348978+0000","last_undegraded":"2024-04-26T18:25:06.348978+0000","last_fullsized":"2024-04-26T18:25:06.348978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:00.912752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296910+0000","last_change":"2024-04-26T18:24:17.934924+0000","last_active":"2024-04-26T18:25:06.296910+0000","last_peered":"2024-04-26T18:25:06.296910+0000","last_clean":"2024-04-26T18:25:06.296910+0000","last_became_active":"2024-04-26T18:24:17.934598+0000","last_became_peered":"2024-04-26T18:24:17.934598+0000","last_unstale":"2024-04-26T18:25:06.296910+0000","last_undegraded":"2024-04-26T18:25:06.296910+0000","last_fullsized":"2024-04-26T18:25:06.296910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:28.690043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298555+0000","last_change":"2024-04-26T18:24:18.142560+0000","last_active":"2024-04-26T18:25:06.298555+0000","last_peered":"2024-04-26T18:25:06.298555+0000","last_clean":"2024-04-26T18:25:06.298555+0000","last_became_active":"2024-04-26T18:24:18.142184+0000","last_became_peered":"2024-04-26T18:24:18.142184+0000","last_unstale":"2024-04-26T18:25:06.298555+0000","last_undegraded":"2024-04-26T18:25:06.298555+0000","last_fullsized":"2024-04-26T18:25:06.298555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:26.656113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349333+0000","last_change":"2024-04-26T18:24:20.952073+0000","last_active":"2024-04-26T18:25:06.349333+0000","last_peered":"2024-04-26T18:25:06.349333+0000","last_clean":"2024-04-26T18:25:06.349333+0000","last_became_active":"2024-04-26T18:24:20.951761+0000","last_became_peered":"2024-04-26T18:24:20.951761+0000","last_unstale":"2024-04-26T18:25:06.349333+0000","last_undegraded":"2024-04-26T18:25:06.349333+0000","last_fullsized":"2024-04-26T18:25:06.349333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:30.244060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'14","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294095+0000","last_change":"2024-04-26T18:24:17.927171+0000","last_active":"2024-04-26T18:25:06.294095+0000","last_peered":"2024-04-26T18:25:06.294095+0000","last_clean":"2024-04-26T18:25:06.294095+0000","last_became_active":"2024-04-26T18:24:17.926816+0000","last_became_peered":"2024-04-26T18:24:17.926816+0000","last_unstale":"2024-04-26T18:25:06.294095+0000","last_undegraded":"2024-04-26T18:25:06.294095+0000","last_fullsized":"2024-04-26T18:25:06.294095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:18.249363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.755057+0000","last_change":"2024-04-26T18:24:20.951684+0000","last_active":"2024-04-26T18:25:06.755057+0000","last_peered":"2024-04-26T18:25:06.755057+0000","last_clean":"2024-04-26T18:25:06.755057+0000","last_became_active":"2024-04-26T18:24:20.951364+0000","last_became_peered":"2024-04-26T18:24:20.951364+0000","last_unstale":"2024-04-26T18:25:06.755057+0000","last_undegraded":"2024-04-26T18:25:06.755057+0000","last_fullsized":"2024-04-26T18:25:06.755057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:28.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295402+0000","last_change":"2024-04-26T18:24:18.229770+0000","last_active":"2024-04-26T18:25:06.295402+0000","last_peered":"2024-04-26T18:25:06.295402+0000","last_clean":"2024-04-26T18:25:06.295402+0000","last_became_active":"2024-04-26T18:24:18.229446+0000","last_became_peered":"2024-04-26T18:24:18.229446+0000","last_unstale":"2024-04-26T18:25:06.295402+0000","last_undegraded":"2024-04-26T18:25:06.295402+0000","last_fullsized":"2024-04-26T18:25:06.295402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:14.794797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296547+0000","last_change":"2024-04-26T18:24:20.960513+0000","last_active":"2024-04-26T18:25:06.296547+0000","last_peered":"2024-04-26T18:25:06.296547+0000","last_clean":"2024-04-26T18:25:06.296547+0000","last_became_active":"2024-04-26T18:24:20.959549+0000","last_became_peered":"2024-04-26T18:24:20.959549+0000","last_unstale":"2024-04-26T18:25:06.296547+0000","last_undegraded":"2024-04-26T18:25:06.296547+0000","last_fullsized":"2024-04-26T18:25:06.296547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:18:37.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297969+0000","last_change":"2024-04-26T18:24:18.141549+0000","last_active":"2024-04-26T18:25:06.297969+0000","last_peered":"2024-04-26T18:25:06.297969+0000","last_clean":"2024-04-26T18:25:06.297969+0000","last_became_active":"2024-04-26T18:24:18.141172+0000","last_became_peered":"2024-04-26T18:24:18.141172+0000","last_unstale":"2024-04-26T18:25:06.297969+0000","last_undegraded":"2024-04-26T18:25:06.297969+0000","last_fullsized":"2024-04-26T18:25:06.297969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:15.878130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297295+0000","last_change":"2024-04-26T18:24:20.948121+0000","last_active":"2024-04-26T18:25:06.297295+0000","last_peered":"2024-04-26T18:25:06.297295+0000","last_clean":"2024-04-26T18:25:06.297295+0000","last_became_active":"2024-04-26T18:24:20.947738+0000","last_became_peered":"2024-04-26T18:24:20.947738+0000","last_unstale":"2024-04-26T18:25:06.297295+0000","last_undegraded":"2024-04-26T18:25:06.297295+0000","last_fullsized":"2024-04-26T18:25:06.297295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:04.237302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754809+0000","last_change":"2024-04-26T18:24:17.926631+0000","last_active":"2024-04-26T18:25:06.754809+0000","last_peered":"2024-04-26T18:25:06.754809+0000","last_clean":"2024-04-26T18:25:06.754809+0000","last_became_active":"2024-04-26T18:24:17.926259+0000","last_became_peered":"2024-04-26T18:24:17.926259+0000","last_unstale":"2024-04-26T18:25:06.754809+0000","last_undegraded":"2024-04-26T18:25:06.754809+0000","last_fullsized":"2024-04-26T18:25:06.754809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:47.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298685+0000","last_change":"2024-04-26T18:24:20.947283+0000","last_active":"2024-04-26T18:25:06.298685+0000","last_peered":"2024-04-26T18:25:06.298685+0000","last_clean":"2024-04-26T18:25:06.298685+0000","last_became_active":"2024-04-26T18:24:20.946812+0000","last_became_peered":"2024-04-26T18:24:20.946812+0000","last_unstale":"2024-04-26T18:25:06.298685+0000","last_undegraded":"2024-04-26T18:25:06.298685+0000","last_fullsized":"2024-04-26T18:25:06.298685+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:33.730698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294878+0000","last_change":"2024-04-26T18:24:17.931843+0000","last_active":"2024-04-26T18:25:06.294878+0000","last_peered":"2024-04-26T18:25:06.294878+0000","last_clean":"2024-04-26T18:25:06.294878+0000","last_became_active":"2024-04-26T18:24:17.931442+0000","last_became_peered":"2024-04-26T18:24:17.931442+0000","last_unstale":"2024-04-26T18:25:06.294878+0000","last_undegraded":"2024-04-26T18:25:06.294878+0000","last_fullsized":"2024-04-26T18:25:06.294878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:09.649435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.755046+0000","last_change":"2024-04-26T18:24:20.955661+0000","last_active":"2024-04-26T18:25:06.755046+0000","last_peered":"2024-04-26T18:25:06.755046+0000","last_clean":"2024-04-26T18:25:06.755046+0000","last_became_active":"2024-04-26T18:24:20.955344+0000","last_became_peered":"2024-04-26T18:24:20.955344+0000","last_unstale":"2024-04-26T18:25:06.755046+0000","last_undegraded":"2024-04-26T18:25:06.755046+0000","last_fullsized":"2024-04-26T18:25:06.755046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:44.274381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.315466+0000","last_change":"2024-04-26T18:25:06.315466+0000","last_active":"2024-04-26T18:25:06.315466+0000","last_peered":"2024-04-26T18:25:06.315466+0000","last_clean":"2024-04-26T18:25:06.315466+0000","last_became_active":"2024-04-26T18:25:06.301788+0000","last_became_peered":"2024-04-26T18:25:06.301788+0000","last_unstale":"2024-04-26T18:25:06.315466+0000","last_undegraded":"2024-04-26T18:25:06.315466+0000","last_fullsized":"2024-04-26T18:25:06.315466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:12.998583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295745+0000","last_change":"2024-04-26T18:24:20.955746+0000","last_active":"2024-04-26T18:25:06.295745+0000","last_peered":"2024-04-26T18:25:06.295745+0000","last_clean":"2024-04-26T18:25:06.295745+0000","last_became_active":"2024-04-26T18:24:20.953652+0000","last_became_peered":"2024-04-26T18:24:20.953652+0000","last_unstale":"2024-04-26T18:25:06.295745+0000","last_undegraded":"2024-04-26T18:25:06.295745+0000","last_fullsized":"2024-04-26T18:25:06.295745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:31.414185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:14.318918+0000","last_change":"2024-04-26T18:25:06.309393+0000","last_active":"2024-04-26T18:25:14.318918+0000","last_peered":"2024-04-26T18:25:14.318918+0000","last_clean":"2024-04-26T18:25:14.318918+0000","last_became_active":"2024-04-26T18:25:06.302434+0000","last_became_peered":"2024-04-26T18:25:06.302434+0000","last_unstale":"2024-04-26T18:25:14.318918+0000","last_undegraded":"2024-04-26T18:25:14.318918+0000","last_fullsized":"2024-04-26T18:25:14.318918+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:54.730295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296883+0000","last_change":"2024-04-26T18:24:20.954238+0000","last_active":"2024-04-26T18:25:06.296883+0000","last_peered":"2024-04-26T18:25:06.296883+0000","last_clean":"2024-04-26T18:25:06.296883+0000","last_became_active":"2024-04-26T18:24:20.953883+0000","last_became_peered":"2024-04-26T18:24:20.953883+0000","last_unstale":"2024-04-26T18:25:06.296883+0000","last_undegraded":"2024-04-26T18:25:06.296883+0000","last_fullsized":"2024-04-26T18:25:06.296883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:10.580835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296855+0000","last_change":"2024-04-26T18:24:17.926901+0000","last_active":"2024-04-26T18:25:06.296855+0000","last_peered":"2024-04-26T18:25:06.296855+0000","last_clean":"2024-04-26T18:25:06.296855+0000","last_became_active":"2024-04-26T18:24:17.926291+0000","last_became_peered":"2024-04-26T18:24:17.926291+0000","last_unstale":"2024-04-26T18:25:06.296855+0000","last_undegraded":"2024-04-26T18:25:06.296855+0000","last_fullsized":"2024-04-26T18:25:06.296855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:17.695198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296092+0000","last_change":"2024-04-26T18:24:20.950578+0000","last_active":"2024-04-26T18:25:06.296092+0000","last_peered":"2024-04-26T18:25:06.296092+0000","last_clean":"2024-04-26T18:25:06.296092+0000","last_became_active":"2024-04-26T18:24:20.950174+0000","last_became_peered":"2024-04-26T18:24:20.950174+0000","last_unstale":"2024-04-26T18:25:06.296092+0000","last_undegraded":"2024-04-26T18:25:06.296092+0000","last_fullsized":"2024-04-26T18:25:06.296092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349265+0000","last_change":"2024-04-26T18:24:17.928054+0000","last_active":"2024-04-26T18:25:06.349265+0000","last_peered":"2024-04-26T18:25:06.349265+0000","last_clean":"2024-04-26T18:25:06.349265+0000","last_became_active":"2024-04-26T18:24:17.927715+0000","last_became_peered":"2024-04-26T18:24:17.927715+0000","last_unstale":"2024-04-26T18:25:06.349265+0000","last_undegraded":"2024-04-26T18:25:06.349265+0000","last_fullsized":"2024-04-26T18:25:06.349265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.155436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296772+0000","last_change":"2024-04-26T18:24:20.944512+0000","last_active":"2024-04-26T18:25:06.296772+0000","last_peered":"2024-04-26T18:25:06.296772+0000","last_clean":"2024-04-26T18:25:06.296772+0000","last_became_active":"2024-04-26T18:24:20.944123+0000","last_became_peered":"2024-04-26T18:24:20.944123+0000","last_unstale":"2024-04-26T18:25:06.296772+0000","last_undegraded":"2024-04-26T18:25:06.296772+0000","last_fullsized":"2024-04-26T18:25:06.296772+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.240206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298795+0000","last_change":"2024-04-26T18:24:20.959138+0000","last_active":"2024-04-26T18:25:06.298795+0000","last_peered":"2024-04-26T18:25:06.298795+0000","last_clean":"2024-04-26T18:25:06.298795+0000","last_became_active":"2024-04-26T18:24:20.958675+0000","last_became_peered":"2024-04-26T18:24:20.958675+0000","last_unstale":"2024-04-26T18:25:06.298795+0000","last_undegraded":"2024-04-26T18:25:06.298795+0000","last_fullsized":"2024-04-26T18:25:06.298795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:34:32.103380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294547+0000","last_change":"2024-04-26T18:24:20.961689+0000","last_active":"2024-04-26T18:25:06.294547+0000","last_peered":"2024-04-26T18:25:06.294547+0000","last_clean":"2024-04-26T18:25:06.294547+0000","last_became_active":"2024-04-26T18:24:20.961374+0000","last_became_peered":"2024-04-26T18:24:20.961374+0000","last_unstale":"2024-04-26T18:25:06.294547+0000","last_undegraded":"2024-04-26T18:25:06.294547+0000","last_fullsized":"2024-04-26T18:25:06.294547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:55.243887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294515+0000","last_change":"2024-04-26T18:24:20.950711+0000","last_active":"2024-04-26T18:25:06.294515+0000","last_peered":"2024-04-26T18:25:06.294515+0000","last_clean":"2024-04-26T18:25:06.294515+0000","last_became_active":"2024-04-26T18:24:20.950371+0000","last_became_peered":"2024-04-26T18:24:20.950371+0000","last_unstale":"2024-04-26T18:25:06.294515+0000","last_undegraded":"2024-04-26T18:25:06.294515+0000","last_fullsized":"2024-04-26T18:25:06.294515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:35.307921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295964+0000","last_change":"2024-04-26T18:24:20.959990+0000","last_active":"2024-04-26T18:25:06.295964+0000","last_peered":"2024-04-26T18:25:06.295964+0000","last_clean":"2024-04-26T18:25:06.295964+0000","last_became_active":"2024-04-26T18:24:20.959514+0000","last_became_peered":"2024-04-26T18:24:20.959514+0000","last_unstale":"2024-04-26T18:25:06.295964+0000","last_undegraded":"2024-04-26T18:25:06.295964+0000","last_fullsized":"2024-04-26T18:25:06.295964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:00.184448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349163+0000","last_change":"2024-04-26T18:24:20.957338+0000","last_active":"2024-04-26T18:25:06.349163+0000","last_peered":"2024-04-26T18:25:06.349163+0000","last_clean":"2024-04-26T18:25:06.349163+0000","last_became_active":"2024-04-26T18:24:20.956999+0000","last_became_peered":"2024-04-26T18:24:20.956999+0000","last_unstale":"2024-04-26T18:25:06.349163+0000","last_undegraded":"2024-04-26T18:25:06.349163+0000","last_fullsized":"2024-04-26T18:25:06.349163+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.066902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299283+0000","last_change":"2024-04-26T18:24:20.953634+0000","last_active":"2024-04-26T18:25:06.299283+0000","last_peered":"2024-04-26T18:25:06.299283+0000","last_clean":"2024-04-26T18:25:06.299283+0000","last_became_active":"2024-04-26T18:24:20.953259+0000","last_became_peered":"2024-04-26T18:24:20.953259+0000","last_unstale":"2024-04-26T18:25:06.299283+0000","last_undegraded":"2024-04-26T18:25:06.299283+0000","last_fullsized":"2024-04-26T18:25:06.299283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:00.095344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295478+0000","last_change":"2024-04-26T18:24:20.954997+0000","last_active":"2024-04-26T18:25:06.295478+0000","last_peered":"2024-04-26T18:25:06.295478+0000","last_clean":"2024-04-26T18:25:06.295478+0000","last_became_active":"2024-04-26T18:24:20.954223+0000","last_became_peered":"2024-04-26T18:24:20.954223+0000","last_unstale":"2024-04-26T18:25:06.295478+0000","last_undegraded":"2024-04-26T18:25:06.295478+0000","last_fullsized":"2024-04-26T18:25:06.295478+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:14.092455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754906+0000","last_change":"2024-04-26T18:24:20.957056+0000","last_active":"2024-04-26T18:25:06.754906+0000","last_peered":"2024-04-26T18:25:06.754906+0000","last_clean":"2024-04-26T18:25:06.754906+0000","last_became_active":"2024-04-26T18:24:20.956737+0000","last_became_peered":"2024-04-26T18:24:20.956737+0000","last_unstale":"2024-04-26T18:25:06.754906+0000","last_undegraded":"2024-04-26T18:25:06.754906+0000","last_fullsized":"2024-04-26T18:25:06.754906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:48.358862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.350206+0000","last_change":"2024-04-26T18:25:06.350206+0000","last_active":"2024-04-26T18:25:06.350206+0000","last_peered":"2024-04-26T18:25:06.350206+0000","last_clean":"2024-04-26T18:25:06.350206+0000","last_became_active":"2024-04-26T18:25:06.349866+0000","last_became_peered":"2024-04-26T18:25:06.349866+0000","last_unstale":"2024-04-26T18:25:06.350206+0000","last_undegraded":"2024-04-26T18:25:06.350206+0000","last_fullsized":"2024-04-26T18:25:06.350206+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:41.909056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754750+0000","last_change":"2024-04-26T18:24:20.956000+0000","last_active":"2024-04-26T18:25:06.754750+0000","last_peered":"2024-04-26T18:25:06.754750+0000","last_clean":"2024-04-26T18:25:06.754750+0000","last_became_active":"2024-04-26T18:24:20.955677+0000","last_became_peered":"2024-04-26T18:24:20.955677+0000","last_unstale":"2024-04-26T18:25:06.754750+0000","last_undegraded":"2024-04-26T18:25:06.754750+0000","last_fullsized":"2024-04-26T18:25:06.754750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:20.895674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298152+0000","last_change":"2024-04-26T18:24:20.959317+0000","last_active":"2024-04-26T18:25:06.298152+0000","last_peered":"2024-04-26T18:25:06.298152+0000","last_clean":"2024-04-26T18:25:06.298152+0000","last_became_active":"2024-04-26T18:24:20.958963+0000","last_became_peered":"2024-04-26T18:24:20.958963+0000","last_unstale":"2024-04-26T18:25:06.298152+0000","last_undegraded":"2024-04-26T18:25:06.298152+0000","last_fullsized":"2024-04-26T18:25:06.298152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:08.231089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349143+0000","last_change":"2024-04-26T18:24:20.954445+0000","last_active":"2024-04-26T18:25:06.349143+0000","last_peered":"2024-04-26T18:25:06.349143+0000","last_clean":"2024-04-26T18:25:06.349143+0000","last_became_active":"2024-04-26T18:24:20.953852+0000","last_became_peered":"2024-04-26T18:24:20.953852+0000","last_unstale":"2024-04-26T18:25:06.349143+0000","last_undegraded":"2024-04-26T18:25:06.349143+0000","last_fullsized":"2024-04-26T18:25:06.349143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:05.636897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299528+0000","last_change":"2024-04-26T18:24:20.958023+0000","last_active":"2024-04-26T18:25:06.299528+0000","last_peered":"2024-04-26T18:25:06.299528+0000","last_clean":"2024-04-26T18:25:06.299528+0000","last_became_active":"2024-04-26T18:24:20.957642+0000","last_became_peered":"2024-04-26T18:24:20.957642+0000","last_unstale":"2024-04-26T18:25:06.299528+0000","last_undegraded":"2024-04-26T18:25:06.299528+0000","last_fullsized":"2024-04-26T18:25:06.299528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:46.332710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244217,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":70276,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":367,"num_read_kb":267,"num_write":244,"num_write_kb":137,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1583,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":214132,"data_compressed":5529,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":210,"ondisk_log_size":210,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351002,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":484082,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612638,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1150976,"data_stored":1137770,"data_compressed":43211,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776164,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":483599,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037801,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":483955,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299438,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":1076495,"data_compressed":41368,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462964,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":483316,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":550514,"data_compressed":7620,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":485046,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":548394,"data_compressed":7620,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149762,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":483141,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888126,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":483177,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724599,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":1076173,"data_compressed":41368,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68037,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65917,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":64808,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:25:18.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:18 smithi103 ceph-mon[32727]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s; 145 B/s, 0 keys/s, 1 objects/s recovering 2024-04-26T18:25:18.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:18 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:25:18.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:18 smithi174 ceph-mon[32705]: pgmap v327: 97 pgs: 97 active+clean; 645 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 7 op/s; 145 B/s, 0 keys/s, 1 objects/s recovering 2024-04-26T18:25:18.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:18 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:25:19.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:19 smithi174 ceph-mon[32705]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:19.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:19 smithi084 ceph-mon[29141]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:19.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:19 smithi103 ceph-mon[32727]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:20.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:20 smithi084 ceph-mon[29141]: pgmap v328: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T18:25:20.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:20 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:20 smithi103 ceph-mon[32727]: pgmap v328: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T18:25:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:20 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:20 smithi174 ceph-mon[32705]: pgmap v328: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s; 1 B/s, 0 keys/s, 0 objects/s recovering 2024-04-26T18:25:20.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:20.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:20 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:21.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:25:21.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:25:21.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:25:21.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:21.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:25:21.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:25:21.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:21.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:25:21.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:25:21.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:25:21.836 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:22.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:22 smithi084 ceph-mon[29141]: pgmap v329: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s; 1 B/s, 0 objects/s recovering 2024-04-26T18:25:22.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:22 smithi103 ceph-mon[32727]: pgmap v329: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s; 1 B/s, 0 objects/s recovering 2024-04-26T18:25:22.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:22 smithi174 ceph-mon[32705]: pgmap v329: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 9 op/s; 1 B/s, 0 objects/s recovering 2024-04-26T18:25:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.493+0000 7fc693eeb700 1 -- 172.21.15.84:0/270251532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc68c101dc0 msgr2=0x7fc68c102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.493+0000 7fc693eeb700 1 --2- 172.21.15.84:0/270251532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc68c101dc0 0x7fc68c102220 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc67c0099c0 tx=0x7fc67c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:25:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.493+0000 7fc693eeb700 1 -- 172.21.15.84:0/270251532 shutdown_connections 2024-04-26T18:25:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.493+0000 7fc693eeb700 1 --2- 172.21.15.84:0/270251532 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc68c102760 0x7fc68c112550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.493+0000 7fc693eeb700 1 --2- 172.21.15.84:0/270251532 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c104270 0x7fc68c104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:23.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.493+0000 7fc693eeb700 1 --2- 172.21.15.84:0/270251532 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc68c101dc0 0x7fc68c102220 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.493+0000 7fc693eeb700 1 -- 172.21.15.84:0/270251532 >> 172.21.15.84:0/270251532 conn(0x7fc68c077640 msgr2=0x7fc68c077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.493+0000 7fc693eeb700 1 -- 172.21.15.84:0/270251532 shutdown_connections 2024-04-26T18:25:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.493+0000 7fc693eeb700 1 -- 172.21.15.84:0/270251532 wait complete. 2024-04-26T18:25:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.493+0000 7fc693eeb700 1 Processor -- start 2024-04-26T18:25:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.493+0000 7fc693eeb700 1 -- start start 2024-04-26T18:25:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc693eeb700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc68c101dc0 0x7fc68c19be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc693eeb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc68c102760 0x7fc68c19c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc693eeb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c104270 0x7fc68c1a0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc693eeb700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c115980 con 0x7fc68c102760 2024-04-26T18:25:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc693eeb700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c115800 con 0x7fc68c101dc0 2024-04-26T18:25:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc693eeb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc68c115b00 con 0x7fc68c104270 2024-04-26T18:25:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc691486700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc68c102760 0x7fc68c19c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:23.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc691486700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc68c102760 0x7fc68c19c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48928/0 (socket says 172.21.15.84:48928) 2024-04-26T18:25:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc691486700 1 -- 172.21.15.84:0/761300377 learned_addr learned my addr 172.21.15.84:0/761300377 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc692488700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c104270 0x7fc68c1a0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc691c87700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc68c101dc0 0x7fc68c19be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc691486700 1 -- 172.21.15.84:0/761300377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c104270 msgr2=0x7fc68c1a0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc691486700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c104270 0x7fc68c1a0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc691486700 1 -- 172.21.15.84:0/761300377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc68c101dc0 msgr2=0x7fc68c19be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc691486700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc68c101dc0 0x7fc68c19be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc691486700 1 -- 172.21.15.84:0/761300377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc68c1a0e80 con 0x7fc68c102760 2024-04-26T18:25:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc691486700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc68c102760 0x7fc68c19c3b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00b580 tx=0x7fc67c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc682ffd700 1 -- 172.21.15.84:0/761300377 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c027070 con 0x7fc68c102760 2024-04-26T18:25:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc693eeb700 1 -- 172.21.15.84:0/761300377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc68c1a1110 con 0x7fc68c102760 2024-04-26T18:25:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc682ffd700 1 -- 172.21.15.84:0/761300377 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc67c003f20 con 0x7fc68c102760 2024-04-26T18:25:23.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc682ffd700 1 -- 172.21.15.84:0/761300377 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc67c022d10 con 0x7fc68c102760 2024-04-26T18:25:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.494+0000 7fc693eeb700 1 -- 172.21.15.84:0/761300377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc68c1a89f0 con 0x7fc68c102760 2024-04-26T18:25:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.495+0000 7fc693eeb700 1 -- 172.21.15.84:0/761300377 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc68c06a110 con 0x7fc68c102760 2024-04-26T18:25:23.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.496+0000 7fc682ffd700 1 -- 172.21.15.84:0/761300377 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc67c022470 con 0x7fc68c102760 2024-04-26T18:25:23.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.497+0000 7fc682ffd700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc6780a3cd0 0x7fc6780a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:23.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.497+0000 7fc691c87700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc6780a3cd0 0x7fc6780a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:23.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.497+0000 7fc682ffd700 1 -- 172.21.15.84:0/761300377 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fc67c0d1170 con 0x7fc68c102760 2024-04-26T18:25:23.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.497+0000 7fc691c87700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc6780a3cd0 0x7fc6780a6190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc68c078c80 tx=0x7fc688009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:23.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.502+0000 7fc682ffd700 1 -- 172.21.15.84:0/761300377 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc67c013070 con 0x7fc68c102760 2024-04-26T18:25:23.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.702+0000 7fc693eeb700 1 -- 172.21.15.84:0/761300377 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc68c19d1d0 con 0x7fc6780a3cd0 2024-04-26T18:25:23.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.709+0000 7fc682ffd700 1 -- 172.21.15.84:0/761300377 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259054 (secure 0 0 0) 0x7fc68c19d1d0 con 0x7fc6780a3cd0 2024-04-26T18:25:23.713 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:25:23.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.712+0000 7fc693eeb700 1 -- 172.21.15.84:0/761300377 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc6780a3cd0 msgr2=0x7fc6780a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:23.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.712+0000 7fc693eeb700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc6780a3cd0 0x7fc6780a6190 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fc68c078c80 tx=0x7fc688009190 comp rx=0 tx=0).stop 2024-04-26T18:25:23.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.712+0000 7fc693eeb700 1 -- 172.21.15.84:0/761300377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc68c102760 msgr2=0x7fc68c19c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:23.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.712+0000 7fc693eeb700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc68c102760 0x7fc68c19c3b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc67c00b580 tx=0x7fc67c0047b0 comp rx=0 tx=0).stop 2024-04-26T18:25:23.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.712+0000 7fc693eeb700 1 -- 172.21.15.84:0/761300377 shutdown_connections 2024-04-26T18:25:23.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.712+0000 7fc693eeb700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc6780a3cd0 0x7fc6780a6190 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:23.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.712+0000 7fc693eeb700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc68c104270 0x7fc68c1a0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:23.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.712+0000 7fc693eeb700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc68c101dc0 0x7fc68c19be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:23.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.712+0000 7fc693eeb700 1 --2- 172.21.15.84:0/761300377 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc68c102760 0x7fc68c19c3b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:23.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.712+0000 7fc693eeb700 1 -- 172.21.15.84:0/761300377 >> 172.21.15.84:0/761300377 conn(0x7fc68c077640 msgr2=0x7fc68c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:23.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.712+0000 7fc693eeb700 1 -- 172.21.15.84:0/761300377 shutdown_connections 2024-04-26T18:25:23.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:23.712+0000 7fc693eeb700 1 -- 172.21.15.84:0/761300377 wait complete. 2024-04-26T18:25:23.715 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-04-26T18:25:24.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:24 smithi084 ceph-mon[29141]: pgmap v330: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T18:25:24.442 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-04-26T18:25:23.184194+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662530,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":554,"num_read_kb":413,"num_write":428,"num_write_kb":3079,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182319,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":350,"ondisk_log_size":350,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":369784,"kb_used_data":12240,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572040,"statfs":{"total":1151940427776,"available":1151561768960,"internally_reserved":0,"allocated":12533760,"data_stored":7777314,"data_compressed":181626,"data_compressed_allocated":245760,"data_compressed_original":1990656,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":52,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003439"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299065+0000","last_change":"2024-04-26T18:24:20.951581+0000","last_active":"2024-04-26T18:25:06.299065+0000","last_peered":"2024-04-26T18:25:06.299065+0000","last_clean":"2024-04-26T18:25:06.299065+0000","last_became_active":"2024-04-26T18:24:20.951260+0000","last_became_peered":"2024-04-26T18:24:20.951260+0000","last_unstale":"2024-04-26T18:25:06.299065+0000","last_undegraded":"2024-04-26T18:25:06.299065+0000","last_fullsized":"2024-04-26T18:25:06.299065+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:54:28.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296510+0000","last_change":"2024-04-26T18:24:20.952263+0000","last_active":"2024-04-26T18:25:06.296510+0000","last_peered":"2024-04-26T18:25:06.296510+0000","last_clean":"2024-04-26T18:25:06.296510+0000","last_became_active":"2024-04-26T18:24:20.951930+0000","last_became_peered":"2024-04-26T18:24:20.951930+0000","last_unstale":"2024-04-26T18:25:06.296510+0000","last_undegraded":"2024-04-26T18:25:06.296510+0000","last_fullsized":"2024-04-26T18:25:06.296510+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:54.816572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296658+0000","last_change":"2024-04-26T18:24:20.952280+0000","last_active":"2024-04-26T18:25:06.296658+0000","last_peered":"2024-04-26T18:25:06.296658+0000","last_clean":"2024-04-26T18:25:06.296658+0000","last_became_active":"2024-04-26T18:24:20.951623+0000","last_became_peered":"2024-04-26T18:24:20.951623+0000","last_unstale":"2024-04-26T18:25:06.296658+0000","last_undegraded":"2024-04-26T18:25:06.296658+0000","last_fullsized":"2024-04-26T18:25:06.296658+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:02.502261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295410+0000","last_change":"2024-04-26T18:24:20.962274+0000","last_active":"2024-04-26T18:25:06.295410+0000","last_peered":"2024-04-26T18:25:06.295410+0000","last_clean":"2024-04-26T18:25:06.295410+0000","last_became_active":"2024-04-26T18:24:20.961920+0000","last_became_peered":"2024-04-26T18:24:20.961920+0000","last_unstale":"2024-04-26T18:25:06.295410+0000","last_undegraded":"2024-04-26T18:25:06.295410+0000","last_fullsized":"2024-04-26T18:25:06.295410+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:49:46.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297438+0000","last_change":"2024-04-26T18:24:20.949096+0000","last_active":"2024-04-26T18:25:06.297438+0000","last_peered":"2024-04-26T18:25:06.297438+0000","last_clean":"2024-04-26T18:25:06.297438+0000","last_became_active":"2024-04-26T18:24:20.948737+0000","last_became_peered":"2024-04-26T18:24:20.948737+0000","last_unstale":"2024-04-26T18:25:06.297438+0000","last_undegraded":"2024-04-26T18:25:06.297438+0000","last_fullsized":"2024-04-26T18:25:06.297438+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:12:40.371589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351373+0000","last_change":"2024-04-26T18:24:20.962670+0000","last_active":"2024-04-26T18:25:06.351373+0000","last_peered":"2024-04-26T18:25:06.351373+0000","last_clean":"2024-04-26T18:25:06.351373+0000","last_became_active":"2024-04-26T18:24:20.962309+0000","last_became_peered":"2024-04-26T18:24:20.962309+0000","last_unstale":"2024-04-26T18:25:06.351373+0000","last_undegraded":"2024-04-26T18:25:06.351373+0000","last_fullsized":"2024-04-26T18:25:06.351373+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:40.315864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.350579+0000","last_change":"2024-04-26T18:25:06.350579+0000","last_active":"2024-04-26T18:25:06.350579+0000","last_peered":"2024-04-26T18:25:06.350579+0000","last_clean":"2024-04-26T18:25:06.350579+0000","last_became_active":"2024-04-26T18:25:06.350104+0000","last_became_peered":"2024-04-26T18:25:06.350104+0000","last_unstale":"2024-04-26T18:25:06.350579+0000","last_undegraded":"2024-04-26T18:25:06.350579+0000","last_fullsized":"2024-04-26T18:25:06.350579+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:00.950883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754918+0000","last_change":"2024-04-26T18:24:20.961096+0000","last_active":"2024-04-26T18:25:06.754918+0000","last_peered":"2024-04-26T18:25:06.754918+0000","last_clean":"2024-04-26T18:25:06.754918+0000","last_became_active":"2024-04-26T18:24:20.960796+0000","last_became_peered":"2024-04-26T18:24:20.960796+0000","last_unstale":"2024-04-26T18:25:06.754918+0000","last_undegraded":"2024-04-26T18:25:06.754918+0000","last_fullsized":"2024-04-26T18:25:06.754918+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:04.538836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298266+0000","last_change":"2024-04-26T18:24:20.959871+0000","last_active":"2024-04-26T18:25:06.298266+0000","last_peered":"2024-04-26T18:25:06.298266+0000","last_clean":"2024-04-26T18:25:06.298266+0000","last_became_active":"2024-04-26T18:24:20.959404+0000","last_became_peered":"2024-04-26T18:24:20.959404+0000","last_unstale":"2024-04-26T18:25:06.298266+0000","last_undegraded":"2024-04-26T18:25:06.298266+0000","last_fullsized":"2024-04-26T18:25:06.298266+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:43.053618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351694+0000","last_change":"2024-04-26T18:24:20.963815+0000","last_active":"2024-04-26T18:25:06.351694+0000","last_peered":"2024-04-26T18:25:06.351694+0000","last_clean":"2024-04-26T18:25:06.351694+0000","last_became_active":"2024-04-26T18:24:20.963362+0000","last_became_peered":"2024-04-26T18:24:20.963362+0000","last_unstale":"2024-04-26T18:25:06.351694+0000","last_undegraded":"2024-04-26T18:25:06.351694+0000","last_fullsized":"2024-04-26T18:25:06.351694+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:06:57.907485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296283+0000","last_change":"2024-04-26T18:24:20.952109+0000","last_active":"2024-04-26T18:25:06.296283+0000","last_peered":"2024-04-26T18:25:06.296283+0000","last_clean":"2024-04-26T18:25:06.296283+0000","last_became_active":"2024-04-26T18:24:20.951776+0000","last_became_peered":"2024-04-26T18:24:20.951776+0000","last_unstale":"2024-04-26T18:25:06.296283+0000","last_undegraded":"2024-04-26T18:25:06.296283+0000","last_fullsized":"2024-04-26T18:25:06.296283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:53.220246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295295+0000","last_change":"2024-04-26T18:24:20.963965+0000","last_active":"2024-04-26T18:25:06.295295+0000","last_peered":"2024-04-26T18:25:06.295295+0000","last_clean":"2024-04-26T18:25:06.295295+0000","last_became_active":"2024-04-26T18:24:20.963648+0000","last_became_peered":"2024-04-26T18:24:20.963648+0000","last_unstale":"2024-04-26T18:25:06.295295+0000","last_undegraded":"2024-04-26T18:25:06.295295+0000","last_fullsized":"2024-04-26T18:25:06.295295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:04.080813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349258+0000","last_change":"2024-04-26T18:24:20.954186+0000","last_active":"2024-04-26T18:25:06.349258+0000","last_peered":"2024-04-26T18:25:06.349258+0000","last_clean":"2024-04-26T18:25:06.349258+0000","last_became_active":"2024-04-26T18:24:20.953431+0000","last_became_peered":"2024-04-26T18:24:20.953431+0000","last_unstale":"2024-04-26T18:25:06.349258+0000","last_undegraded":"2024-04-26T18:25:06.349258+0000","last_fullsized":"2024-04-26T18:25:06.349258+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:21.498476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295940+0000","last_change":"2024-04-26T18:24:20.945392+0000","last_active":"2024-04-26T18:25:06.295940+0000","last_peered":"2024-04-26T18:25:06.295940+0000","last_clean":"2024-04-26T18:25:06.295940+0000","last_became_active":"2024-04-26T18:24:20.944971+0000","last_became_peered":"2024-04-26T18:24:20.944971+0000","last_unstale":"2024-04-26T18:25:06.295940+0000","last_undegraded":"2024-04-26T18:25:06.295940+0000","last_fullsized":"2024-04-26T18:25:06.295940+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:06.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299351+0000","last_change":"2024-04-26T18:25:06.299351+0000","last_active":"2024-04-26T18:25:06.299351+0000","last_peered":"2024-04-26T18:25:06.299351+0000","last_clean":"2024-04-26T18:25:06.299351+0000","last_became_active":"2024-04-26T18:25:06.298895+0000","last_became_peered":"2024-04-26T18:25:06.298895+0000","last_unstale":"2024-04-26T18:25:06.299351+0000","last_undegraded":"2024-04-26T18:25:06.299351+0000","last_fullsized":"2024-04-26T18:25:06.299351+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:37.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297807+0000","last_change":"2024-04-26T18:24:20.954610+0000","last_active":"2024-04-26T18:25:06.297807+0000","last_peered":"2024-04-26T18:25:06.297807+0000","last_clean":"2024-04-26T18:25:06.297807+0000","last_became_active":"2024-04-26T18:24:20.954239+0000","last_became_peered":"2024-04-26T18:24:20.954239+0000","last_unstale":"2024-04-26T18:25:06.297807+0000","last_undegraded":"2024-04-26T18:25:06.297807+0000","last_fullsized":"2024-04-26T18:25:06.297807+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:41.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298741+0000","last_change":"2024-04-26T18:24:20.956932+0000","last_active":"2024-04-26T18:25:06.298741+0000","last_peered":"2024-04-26T18:25:06.298741+0000","last_clean":"2024-04-26T18:25:06.298741+0000","last_became_active":"2024-04-26T18:24:20.956464+0000","last_became_peered":"2024-04-26T18:24:20.956464+0000","last_unstale":"2024-04-26T18:25:06.298741+0000","last_undegraded":"2024-04-26T18:25:06.298741+0000","last_fullsized":"2024-04-26T18:25:06.298741+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:01.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294148+0000","last_change":"2024-04-26T18:24:20.954870+0000","last_active":"2024-04-26T18:25:06.294148+0000","last_peered":"2024-04-26T18:25:06.294148+0000","last_clean":"2024-04-26T18:25:06.294148+0000","last_became_active":"2024-04-26T18:24:20.954542+0000","last_became_peered":"2024-04-26T18:24:20.954542+0000","last_unstale":"2024-04-26T18:25:06.294148+0000","last_undegraded":"2024-04-26T18:25:06.294148+0000","last_fullsized":"2024-04-26T18:25:06.294148+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:13:51.932625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295319+0000","last_change":"2024-04-26T18:24:20.950994+0000","last_active":"2024-04-26T18:25:06.295319+0000","last_peered":"2024-04-26T18:25:06.295319+0000","last_clean":"2024-04-26T18:25:06.295319+0000","last_became_active":"2024-04-26T18:24:20.950343+0000","last_became_peered":"2024-04-26T18:24:20.950343+0000","last_unstale":"2024-04-26T18:25:06.295319+0000","last_undegraded":"2024-04-26T18:25:06.295319+0000","last_fullsized":"2024-04-26T18:25:06.295319+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:25.274049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":79,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:16.360043+0000","last_change":"2024-04-26T18:24:17.929731+0000","last_active":"2024-04-26T18:25:16.360043+0000","last_peered":"2024-04-26T18:25:16.360043+0000","last_clean":"2024-04-26T18:25:16.360043+0000","last_became_active":"2024-04-26T18:24:17.929379+0000","last_became_peered":"2024-04-26T18:24:17.929379+0000","last_unstale":"2024-04-26T18:25:16.360043+0000","last_undegraded":"2024-04-26T18:25:16.360043+0000","last_fullsized":"2024-04-26T18:25:16.360043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:44.821660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754917+0000","last_change":"2024-04-26T18:24:20.962885+0000","last_active":"2024-04-26T18:25:06.754917+0000","last_peered":"2024-04-26T18:25:06.754917+0000","last_clean":"2024-04-26T18:25:06.754917+0000","last_became_active":"2024-04-26T18:24:20.962674+0000","last_became_peered":"2024-04-26T18:24:20.962674+0000","last_unstale":"2024-04-26T18:25:06.754917+0000","last_undegraded":"2024-04-26T18:25:06.754917+0000","last_fullsized":"2024-04-26T18:25:06.754917+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:32.444889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.755048+0000","last_change":"2024-04-26T18:24:18.141500+0000","last_active":"2024-04-26T18:25:06.755048+0000","last_peered":"2024-04-26T18:25:06.755048+0000","last_clean":"2024-04-26T18:25:06.755048+0000","last_became_active":"2024-04-26T18:24:18.140870+0000","last_became_peered":"2024-04-26T18:24:18.140870+0000","last_unstale":"2024-04-26T18:25:06.755048+0000","last_undegraded":"2024-04-26T18:25:06.755048+0000","last_fullsized":"2024-04-26T18:25:06.755048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:13.821872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296790+0000","last_change":"2024-04-26T18:24:20.949183+0000","last_active":"2024-04-26T18:25:06.296790+0000","last_peered":"2024-04-26T18:25:06.296790+0000","last_clean":"2024-04-26T18:25:06.296790+0000","last_became_active":"2024-04-26T18:24:20.948677+0000","last_became_peered":"2024-04-26T18:24:20.948677+0000","last_unstale":"2024-04-26T18:25:06.296790+0000","last_undegraded":"2024-04-26T18:25:06.296790+0000","last_fullsized":"2024-04-26T18:25:06.296790+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:55.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":61,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.312239+0000","last_change":"2024-04-26T18:25:06.312239+0000","last_active":"2024-04-26T18:25:06.312239+0000","last_peered":"2024-04-26T18:25:06.312239+0000","last_clean":"2024-04-26T18:25:06.312239+0000","last_became_active":"2024-04-26T18:25:06.301304+0000","last_became_peered":"2024-04-26T18:25:06.301304+0000","last_unstale":"2024-04-26T18:25:06.312239+0000","last_undegraded":"2024-04-26T18:25:06.312239+0000","last_fullsized":"2024-04-26T18:25:06.312239+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:36.159348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298667+0000","last_change":"2024-04-26T18:24:20.955555+0000","last_active":"2024-04-26T18:25:06.298667+0000","last_peered":"2024-04-26T18:25:06.298667+0000","last_clean":"2024-04-26T18:25:06.298667+0000","last_became_active":"2024-04-26T18:24:20.955176+0000","last_became_peered":"2024-04-26T18:24:20.955176+0000","last_unstale":"2024-04-26T18:25:06.298667+0000","last_undegraded":"2024-04-26T18:25:06.298667+0000","last_fullsized":"2024-04-26T18:25:06.298667+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:02:30.368971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298703+0000","last_change":"2024-04-26T18:24:17.930919+0000","last_active":"2024-04-26T18:25:06.298703+0000","last_peered":"2024-04-26T18:25:06.298703+0000","last_clean":"2024-04-26T18:25:06.298703+0000","last_became_active":"2024-04-26T18:24:17.930596+0000","last_became_peered":"2024-04-26T18:24:17.930596+0000","last_unstale":"2024-04-26T18:25:06.298703+0000","last_undegraded":"2024-04-26T18:25:06.298703+0000","last_fullsized":"2024-04-26T18:25:06.298703+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:57.476027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296942+0000","last_change":"2024-04-26T18:24:20.955796+0000","last_active":"2024-04-26T18:25:06.296942+0000","last_peered":"2024-04-26T18:25:06.296942+0000","last_clean":"2024-04-26T18:25:06.296942+0000","last_became_active":"2024-04-26T18:24:20.955432+0000","last_became_peered":"2024-04-26T18:24:20.955432+0000","last_unstale":"2024-04-26T18:25:06.296942+0000","last_undegraded":"2024-04-26T18:25:06.296942+0000","last_fullsized":"2024-04-26T18:25:06.296942+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:25.161797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":76,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:18.368434+0000","last_change":"2024-04-26T18:24:17.929505+0000","last_active":"2024-04-26T18:25:18.368434+0000","last_peered":"2024-04-26T18:25:18.368434+0000","last_clean":"2024-04-26T18:25:18.368434+0000","last_became_active":"2024-04-26T18:24:17.929205+0000","last_became_peered":"2024-04-26T18:24:17.929205+0000","last_unstale":"2024-04-26T18:25:18.368434+0000","last_undegraded":"2024-04-26T18:25:18.368434+0000","last_fullsized":"2024-04-26T18:25:18.368434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:42.460063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":44,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297815+0000","last_change":"2024-04-26T18:24:20.952134+0000","last_active":"2024-04-26T18:25:06.297815+0000","last_peered":"2024-04-26T18:25:06.297815+0000","last_clean":"2024-04-26T18:25:06.297815+0000","last_became_active":"2024-04-26T18:24:20.951758+0000","last_became_peered":"2024-04-26T18:24:20.951758+0000","last_unstale":"2024-04-26T18:25:06.297815+0000","last_undegraded":"2024-04-26T18:25:06.297815+0000","last_fullsized":"2024-04-26T18:25:06.297815+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:25.389221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351236+0000","last_change":"2024-04-26T18:24:17.928704+0000","last_active":"2024-04-26T18:25:06.351236+0000","last_peered":"2024-04-26T18:25:06.351236+0000","last_clean":"2024-04-26T18:25:06.351236+0000","last_became_active":"2024-04-26T18:24:17.928343+0000","last_became_peered":"2024-04-26T18:24:17.928343+0000","last_unstale":"2024-04-26T18:25:06.351236+0000","last_undegraded":"2024-04-26T18:25:06.351236+0000","last_fullsized":"2024-04-26T18:25:06.351236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:49.400481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296463+0000","last_change":"2024-04-26T18:24:20.956808+0000","last_active":"2024-04-26T18:25:06.296463+0000","last_peered":"2024-04-26T18:25:06.296463+0000","last_clean":"2024-04-26T18:25:06.296463+0000","last_became_active":"2024-04-26T18:24:20.956466+0000","last_became_peered":"2024-04-26T18:24:20.956466+0000","last_unstale":"2024-04-26T18:25:06.296463+0000","last_undegraded":"2024-04-26T18:25:06.296463+0000","last_fullsized":"2024-04-26T18:25:06.296463+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:31.150670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349100+0000","last_change":"2024-04-26T18:24:17.929134+0000","last_active":"2024-04-26T18:25:06.349100+0000","last_peered":"2024-04-26T18:25:06.349100+0000","last_clean":"2024-04-26T18:25:06.349100+0000","last_became_active":"2024-04-26T18:24:17.928798+0000","last_became_peered":"2024-04-26T18:24:17.928798+0000","last_unstale":"2024-04-26T18:25:06.349100+0000","last_undegraded":"2024-04-26T18:25:06.349100+0000","last_fullsized":"2024-04-26T18:25:06.349100+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:10.450825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754806+0000","last_change":"2024-04-26T18:24:20.951594+0000","last_active":"2024-04-26T18:25:06.754806+0000","last_peered":"2024-04-26T18:25:06.754806+0000","last_clean":"2024-04-26T18:25:06.754806+0000","last_became_active":"2024-04-26T18:24:20.951148+0000","last_became_peered":"2024-04-26T18:24:20.951148+0000","last_unstale":"2024-04-26T18:25:06.754806+0000","last_undegraded":"2024-04-26T18:25:06.754806+0000","last_fullsized":"2024-04-26T18:25:06.754806+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:18.703292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296335+0000","last_change":"2024-04-26T18:24:17.931101+0000","last_active":"2024-04-26T18:25:06.296335+0000","last_peered":"2024-04-26T18:25:06.296335+0000","last_clean":"2024-04-26T18:25:06.296335+0000","last_became_active":"2024-04-26T18:24:17.930819+0000","last_became_peered":"2024-04-26T18:24:17.930819+0000","last_unstale":"2024-04-26T18:25:06.296335+0000","last_undegraded":"2024-04-26T18:25:06.296335+0000","last_fullsized":"2024-04-26T18:25:06.296335+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.293645+0000","last_change":"2024-04-26T18:24:20.948866+0000","last_active":"2024-04-26T18:25:06.293645+0000","last_peered":"2024-04-26T18:25:06.293645+0000","last_clean":"2024-04-26T18:25:06.293645+0000","last_became_active":"2024-04-26T18:24:20.948562+0000","last_became_peered":"2024-04-26T18:24:20.948562+0000","last_unstale":"2024-04-26T18:25:06.293645+0000","last_undegraded":"2024-04-26T18:25:06.293645+0000","last_fullsized":"2024-04-26T18:25:06.293645+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:33:19.616660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"89'4","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:18.324515+0000","last_change":"2024-04-26T18:24:17.926384+0000","last_active":"2024-04-26T18:25:18.324515+0000","last_peered":"2024-04-26T18:25:18.324515+0000","last_clean":"2024-04-26T18:25:18.324515+0000","last_became_active":"2024-04-26T18:24:17.926037+0000","last_became_peered":"2024-04-26T18:24:17.926037+0000","last_unstale":"2024-04-26T18:25:18.324515+0000","last_undegraded":"2024-04-26T18:25:18.324515+0000","last_fullsized":"2024-04-26T18:25:18.324515+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:05.283755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298313+0000","last_change":"2024-04-26T18:24:20.954257+0000","last_active":"2024-04-26T18:25:06.298313+0000","last_peered":"2024-04-26T18:25:06.298313+0000","last_clean":"2024-04-26T18:25:06.298313+0000","last_became_active":"2024-04-26T18:24:20.953913+0000","last_became_peered":"2024-04-26T18:24:20.953913+0000","last_unstale":"2024-04-26T18:25:06.298313+0000","last_undegraded":"2024-04-26T18:25:06.298313+0000","last_fullsized":"2024-04-26T18:25:06.298313+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:06.814156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296415+0000","last_change":"2024-04-26T18:24:17.932843+0000","last_active":"2024-04-26T18:25:06.296415+0000","last_peered":"2024-04-26T18:25:06.296415+0000","last_clean":"2024-04-26T18:25:06.296415+0000","last_became_active":"2024-04-26T18:24:17.932502+0000","last_became_peered":"2024-04-26T18:24:17.932502+0000","last_unstale":"2024-04-26T18:25:06.296415+0000","last_undegraded":"2024-04-26T18:25:06.296415+0000","last_fullsized":"2024-04-26T18:25:06.296415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:49.727094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296758+0000","last_change":"2024-04-26T18:24:20.959881+0000","last_active":"2024-04-26T18:25:06.296758+0000","last_peered":"2024-04-26T18:25:06.296758+0000","last_clean":"2024-04-26T18:25:06.296758+0000","last_became_active":"2024-04-26T18:24:20.959541+0000","last_became_peered":"2024-04-26T18:24:20.959541+0000","last_unstale":"2024-04-26T18:25:06.296758+0000","last_undegraded":"2024-04-26T18:25:06.296758+0000","last_fullsized":"2024-04-26T18:25:06.296758+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:23.393521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296973+0000","last_change":"2024-04-26T18:24:18.142146+0000","last_active":"2024-04-26T18:25:06.296973+0000","last_peered":"2024-04-26T18:25:06.296973+0000","last_clean":"2024-04-26T18:25:06.296973+0000","last_became_active":"2024-04-26T18:24:18.141829+0000","last_became_peered":"2024-04-26T18:24:18.141829+0000","last_unstale":"2024-04-26T18:25:06.296973+0000","last_undegraded":"2024-04-26T18:25:06.296973+0000","last_fullsized":"2024-04-26T18:25:06.296973+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:49.341638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298792+0000","last_change":"2024-04-26T18:24:20.957993+0000","last_active":"2024-04-26T18:25:06.298792+0000","last_peered":"2024-04-26T18:25:06.298792+0000","last_clean":"2024-04-26T18:25:06.298792+0000","last_became_active":"2024-04-26T18:24:20.957443+0000","last_became_peered":"2024-04-26T18:24:20.957443+0000","last_unstale":"2024-04-26T18:25:06.298792+0000","last_undegraded":"2024-04-26T18:25:06.298792+0000","last_fullsized":"2024-04-26T18:25:06.298792+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:06:32.960546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'33","reported_seq":148,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:18.368091+0000","last_change":"2024-04-26T18:24:17.931283+0000","last_active":"2024-04-26T18:25:18.368091+0000","last_peered":"2024-04-26T18:25:18.368091+0000","last_clean":"2024-04-26T18:25:18.368091+0000","last_became_active":"2024-04-26T18:24:17.930943+0000","last_became_peered":"2024-04-26T18:24:17.930943+0000","last_unstale":"2024-04-26T18:25:18.368091+0000","last_undegraded":"2024-04-26T18:25:18.368091+0000","last_fullsized":"2024-04-26T18:25:18.368091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:50.511677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64724,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":89,"num_read_kb":107,"num_write":35,"num_write_kb":74,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298440+0000","last_change":"2024-04-26T18:24:20.947509+0000","last_active":"2024-04-26T18:25:06.298440+0000","last_peered":"2024-04-26T18:25:06.298440+0000","last_clean":"2024-04-26T18:25:06.298440+0000","last_became_active":"2024-04-26T18:24:20.947181+0000","last_became_peered":"2024-04-26T18:24:20.947181+0000","last_unstale":"2024-04-26T18:25:06.298440+0000","last_undegraded":"2024-04-26T18:25:06.298440+0000","last_fullsized":"2024-04-26T18:25:06.298440+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.262312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295054+0000","last_change":"2024-04-26T18:24:17.934042+0000","last_active":"2024-04-26T18:25:06.295054+0000","last_peered":"2024-04-26T18:25:06.295054+0000","last_clean":"2024-04-26T18:25:06.295054+0000","last_became_active":"2024-04-26T18:24:17.933537+0000","last_became_peered":"2024-04-26T18:24:17.933537+0000","last_unstale":"2024-04-26T18:25:06.295054+0000","last_undegraded":"2024-04-26T18:25:06.295054+0000","last_fullsized":"2024-04-26T18:25:06.295054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:35:06.904160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.348985+0000","last_change":"2024-04-26T18:24:20.957672+0000","last_active":"2024-04-26T18:25:06.348985+0000","last_peered":"2024-04-26T18:25:06.348985+0000","last_clean":"2024-04-26T18:25:06.348985+0000","last_became_active":"2024-04-26T18:24:20.957214+0000","last_became_peered":"2024-04-26T18:24:20.957214+0000","last_unstale":"2024-04-26T18:25:06.348985+0000","last_undegraded":"2024-04-26T18:25:06.348985+0000","last_fullsized":"2024-04-26T18:25:06.348985+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:42:35.341858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295760+0000","last_change":"2024-04-26T18:24:17.930324+0000","last_active":"2024-04-26T18:25:06.295760+0000","last_peered":"2024-04-26T18:25:06.295760+0000","last_clean":"2024-04-26T18:25:06.295760+0000","last_became_active":"2024-04-26T18:24:17.930022+0000","last_became_peered":"2024-04-26T18:24:17.930022+0000","last_unstale":"2024-04-26T18:25:06.295760+0000","last_undegraded":"2024-04-26T18:25:06.295760+0000","last_fullsized":"2024-04-26T18:25:06.295760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:36.864816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.350388+0000","last_change":"2024-04-26T18:25:06.350388+0000","last_active":"2024-04-26T18:25:06.350388+0000","last_peered":"2024-04-26T18:25:06.350388+0000","last_clean":"2024-04-26T18:25:06.350388+0000","last_became_active":"2024-04-26T18:25:06.349836+0000","last_became_peered":"2024-04-26T18:25:06.349836+0000","last_unstale":"2024-04-26T18:25:06.350388+0000","last_undegraded":"2024-04-26T18:25:06.350388+0000","last_fullsized":"2024-04-26T18:25:06.350388+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:57.762115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:18.368604+0000","last_change":"2024-04-26T18:24:17.934676+0000","last_active":"2024-04-26T18:25:18.368604+0000","last_peered":"2024-04-26T18:25:18.368604+0000","last_clean":"2024-04-26T18:25:18.368604+0000","last_became_active":"2024-04-26T18:24:17.934307+0000","last_became_peered":"2024-04-26T18:24:17.934307+0000","last_unstale":"2024-04-26T18:25:18.368604+0000","last_undegraded":"2024-04-26T18:25:18.368604+0000","last_fullsized":"2024-04-26T18:25:18.368604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:27.312248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":44,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296271+0000","last_change":"2024-04-26T18:24:20.947371+0000","last_active":"2024-04-26T18:25:06.296271+0000","last_peered":"2024-04-26T18:25:06.296271+0000","last_clean":"2024-04-26T18:25:06.296271+0000","last_became_active":"2024-04-26T18:24:20.947048+0000","last_became_peered":"2024-04-26T18:24:20.947048+0000","last_unstale":"2024-04-26T18:25:06.296271+0000","last_undegraded":"2024-04-26T18:25:06.296271+0000","last_fullsized":"2024-04-26T18:25:06.296271+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:55.038074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297413+0000","last_change":"2024-04-26T18:24:18.142680+0000","last_active":"2024-04-26T18:25:06.297413+0000","last_peered":"2024-04-26T18:25:06.297413+0000","last_clean":"2024-04-26T18:25:06.297413+0000","last_became_active":"2024-04-26T18:24:18.142392+0000","last_became_peered":"2024-04-26T18:24:18.142392+0000","last_unstale":"2024-04-26T18:25:06.297413+0000","last_undegraded":"2024-04-26T18:25:06.297413+0000","last_fullsized":"2024-04-26T18:25:06.297413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:52.654813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295840+0000","last_change":"2024-04-26T18:24:20.958131+0000","last_active":"2024-04-26T18:25:06.295840+0000","last_peered":"2024-04-26T18:25:06.295840+0000","last_clean":"2024-04-26T18:25:06.295840+0000","last_became_active":"2024-04-26T18:24:20.957701+0000","last_became_peered":"2024-04-26T18:24:20.957701+0000","last_unstale":"2024-04-26T18:25:06.295840+0000","last_undegraded":"2024-04-26T18:25:06.295840+0000","last_fullsized":"2024-04-26T18:25:06.295840+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:02.018158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.319173+0000","last_change":"2024-04-26T18:25:06.319173+0000","last_active":"2024-04-26T18:25:06.319173+0000","last_peered":"2024-04-26T18:25:06.319173+0000","last_clean":"2024-04-26T18:25:06.319173+0000","last_became_active":"2024-04-26T18:25:06.302935+0000","last_became_peered":"2024-04-26T18:25:06.302935+0000","last_unstale":"2024-04-26T18:25:06.319173+0000","last_undegraded":"2024-04-26T18:25:06.319173+0000","last_fullsized":"2024-04-26T18:25:06.319173+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:40.198105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299067+0000","last_change":"2024-04-26T18:24:20.961230+0000","last_active":"2024-04-26T18:25:06.299067+0000","last_peered":"2024-04-26T18:25:06.299067+0000","last_clean":"2024-04-26T18:25:06.299067+0000","last_became_active":"2024-04-26T18:24:20.960807+0000","last_became_peered":"2024-04-26T18:24:20.960807+0000","last_unstale":"2024-04-26T18:25:06.299067+0000","last_undegraded":"2024-04-26T18:25:06.299067+0000","last_fullsized":"2024-04-26T18:25:06.299067+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:39.535100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.293995+0000","last_change":"2024-04-26T18:24:18.142082+0000","last_active":"2024-04-26T18:25:06.293995+0000","last_peered":"2024-04-26T18:25:06.293995+0000","last_clean":"2024-04-26T18:25:06.293995+0000","last_became_active":"2024-04-26T18:24:18.141765+0000","last_became_peered":"2024-04-26T18:24:18.141765+0000","last_unstale":"2024-04-26T18:25:06.293995+0000","last_undegraded":"2024-04-26T18:25:06.293995+0000","last_fullsized":"2024-04-26T18:25:06.293995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:02.140027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351535+0000","last_change":"2024-04-26T18:24:20.952867+0000","last_active":"2024-04-26T18:25:06.351535+0000","last_peered":"2024-04-26T18:25:06.351535+0000","last_clean":"2024-04-26T18:25:06.351535+0000","last_became_active":"2024-04-26T18:24:20.952525+0000","last_became_peered":"2024-04-26T18:24:20.952525+0000","last_unstale":"2024-04-26T18:25:06.351535+0000","last_undegraded":"2024-04-26T18:25:06.351535+0000","last_fullsized":"2024-04-26T18:25:06.351535+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:52.804312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":74,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:19.364066+0000","last_change":"2024-04-26T18:24:17.932270+0000","last_active":"2024-04-26T18:25:19.364066+0000","last_peered":"2024-04-26T18:25:19.364066+0000","last_clean":"2024-04-26T18:25:19.364066+0000","last_became_active":"2024-04-26T18:24:17.931901+0000","last_became_peered":"2024-04-26T18:24:17.931901+0000","last_unstale":"2024-04-26T18:25:19.364066+0000","last_undegraded":"2024-04-26T18:25:19.364066+0000","last_fullsized":"2024-04-26T18:25:19.364066+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:13.524174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.351380+0000","last_change":"2024-04-26T18:24:20.960612+0000","last_active":"2024-04-26T18:25:06.351380+0000","last_peered":"2024-04-26T18:25:06.351380+0000","last_clean":"2024-04-26T18:25:06.351380+0000","last_became_active":"2024-04-26T18:24:20.960280+0000","last_became_peered":"2024-04-26T18:24:20.960280+0000","last_unstale":"2024-04-26T18:25:06.351380+0000","last_undegraded":"2024-04-26T18:25:06.351380+0000","last_fullsized":"2024-04-26T18:25:06.351380+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:53.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.300141+0000","last_change":"2024-04-26T18:25:06.299736+0000","last_active":"2024-04-26T18:25:06.300141+0000","last_peered":"2024-04-26T18:25:06.300141+0000","last_clean":"2024-04-26T18:25:06.300141+0000","last_became_active":"2024-04-26T18:25:06.299265+0000","last_became_peered":"2024-04-26T18:25:06.299265+0000","last_unstale":"2024-04-26T18:25:06.300141+0000","last_undegraded":"2024-04-26T18:25:06.300141+0000","last_fullsized":"2024-04-26T18:25:06.300141+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:32.427551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":73,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:18.368121+0000","last_change":"2024-04-26T18:24:18.141181+0000","last_active":"2024-04-26T18:25:18.368121+0000","last_peered":"2024-04-26T18:25:18.368121+0000","last_clean":"2024-04-26T18:25:18.368121+0000","last_became_active":"2024-04-26T18:24:18.140854+0000","last_became_peered":"2024-04-26T18:24:18.140854+0000","last_unstale":"2024-04-26T18:25:18.368121+0000","last_undegraded":"2024-04-26T18:25:18.368121+0000","last_fullsized":"2024-04-26T18:25:18.368121+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:27.900701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":42,"num_read_kb":42,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754882+0000","last_change":"2024-04-26T18:24:20.961653+0000","last_active":"2024-04-26T18:25:06.754882+0000","last_peered":"2024-04-26T18:25:06.754882+0000","last_clean":"2024-04-26T18:25:06.754882+0000","last_became_active":"2024-04-26T18:24:20.961330+0000","last_became_peered":"2024-04-26T18:24:20.961330+0000","last_unstale":"2024-04-26T18:25:06.754882+0000","last_undegraded":"2024-04-26T18:25:06.754882+0000","last_fullsized":"2024-04-26T18:25:06.754882+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:19.033932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295928+0000","last_change":"2024-04-26T18:24:17.933613+0000","last_active":"2024-04-26T18:25:06.295928+0000","last_peered":"2024-04-26T18:25:06.295928+0000","last_clean":"2024-04-26T18:25:06.295928+0000","last_became_active":"2024-04-26T18:24:17.933219+0000","last_became_peered":"2024-04-26T18:24:17.933219+0000","last_unstale":"2024-04-26T18:25:06.295928+0000","last_undegraded":"2024-04-26T18:25:06.295928+0000","last_fullsized":"2024-04-26T18:25:06.295928+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:04.215821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":358,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296210+0000","last_change":"2024-04-26T18:22:57.585634+0000","last_active":"2024-04-26T18:25:06.296210+0000","last_peered":"2024-04-26T18:25:06.296210+0000","last_clean":"2024-04-26T18:25:06.296210+0000","last_became_active":"2024-04-26T18:22:57.585198+0000","last_became_peered":"2024-04-26T18:22:57.585198+0000","last_unstale":"2024-04-26T18:25:06.296210+0000","last_undegraded":"2024-04-26T18:25:06.296210+0000","last_fullsized":"2024-04-26T18:25:06.296210+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_clean_scrub_stamp":"2024-04-26T18:19:31.594684+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:56:11.861930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.348978+0000","last_change":"2024-04-26T18:24:20.950303+0000","last_active":"2024-04-26T18:25:06.348978+0000","last_peered":"2024-04-26T18:25:06.348978+0000","last_clean":"2024-04-26T18:25:06.348978+0000","last_became_active":"2024-04-26T18:24:20.950002+0000","last_became_peered":"2024-04-26T18:24:20.950002+0000","last_unstale":"2024-04-26T18:25:06.348978+0000","last_undegraded":"2024-04-26T18:25:06.348978+0000","last_fullsized":"2024-04-26T18:25:06.348978+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:00.912752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296910+0000","last_change":"2024-04-26T18:24:17.934924+0000","last_active":"2024-04-26T18:25:06.296910+0000","last_peered":"2024-04-26T18:25:06.296910+0000","last_clean":"2024-04-26T18:25:06.296910+0000","last_became_active":"2024-04-26T18:24:17.934598+0000","last_became_peered":"2024-04-26T18:24:17.934598+0000","last_unstale":"2024-04-26T18:25:06.296910+0000","last_undegraded":"2024-04-26T18:25:06.296910+0000","last_fullsized":"2024-04-26T18:25:06.296910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:28.690043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298555+0000","last_change":"2024-04-26T18:24:18.142560+0000","last_active":"2024-04-26T18:25:06.298555+0000","last_peered":"2024-04-26T18:25:06.298555+0000","last_clean":"2024-04-26T18:25:06.298555+0000","last_became_active":"2024-04-26T18:24:18.142184+0000","last_became_peered":"2024-04-26T18:24:18.142184+0000","last_unstale":"2024-04-26T18:25:06.298555+0000","last_undegraded":"2024-04-26T18:25:06.298555+0000","last_fullsized":"2024-04-26T18:25:06.298555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:26.656113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349333+0000","last_change":"2024-04-26T18:24:20.952073+0000","last_active":"2024-04-26T18:25:06.349333+0000","last_peered":"2024-04-26T18:25:06.349333+0000","last_clean":"2024-04-26T18:25:06.349333+0000","last_became_active":"2024-04-26T18:24:20.951761+0000","last_became_peered":"2024-04-26T18:24:20.951761+0000","last_unstale":"2024-04-26T18:25:06.349333+0000","last_undegraded":"2024-04-26T18:25:06.349333+0000","last_fullsized":"2024-04-26T18:25:06.349333+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:30.244060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"89'14","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294095+0000","last_change":"2024-04-26T18:24:17.927171+0000","last_active":"2024-04-26T18:25:06.294095+0000","last_peered":"2024-04-26T18:25:06.294095+0000","last_clean":"2024-04-26T18:25:06.294095+0000","last_became_active":"2024-04-26T18:24:17.926816+0000","last_became_peered":"2024-04-26T18:24:17.926816+0000","last_unstale":"2024-04-26T18:25:06.294095+0000","last_undegraded":"2024-04-26T18:25:06.294095+0000","last_fullsized":"2024-04-26T18:25:06.294095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:18.249363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":25,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.755057+0000","last_change":"2024-04-26T18:24:20.951684+0000","last_active":"2024-04-26T18:25:06.755057+0000","last_peered":"2024-04-26T18:25:06.755057+0000","last_clean":"2024-04-26T18:25:06.755057+0000","last_became_active":"2024-04-26T18:24:20.951364+0000","last_became_peered":"2024-04-26T18:24:20.951364+0000","last_unstale":"2024-04-26T18:25:06.755057+0000","last_undegraded":"2024-04-26T18:25:06.755057+0000","last_fullsized":"2024-04-26T18:25:06.755057+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:28.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295402+0000","last_change":"2024-04-26T18:24:18.229770+0000","last_active":"2024-04-26T18:25:06.295402+0000","last_peered":"2024-04-26T18:25:06.295402+0000","last_clean":"2024-04-26T18:25:06.295402+0000","last_became_active":"2024-04-26T18:24:18.229446+0000","last_became_peered":"2024-04-26T18:24:18.229446+0000","last_unstale":"2024-04-26T18:25:06.295402+0000","last_undegraded":"2024-04-26T18:25:06.295402+0000","last_fullsized":"2024-04-26T18:25:06.295402+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:14.794797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296547+0000","last_change":"2024-04-26T18:24:20.960513+0000","last_active":"2024-04-26T18:25:06.296547+0000","last_peered":"2024-04-26T18:25:06.296547+0000","last_clean":"2024-04-26T18:25:06.296547+0000","last_became_active":"2024-04-26T18:24:20.959549+0000","last_became_peered":"2024-04-26T18:24:20.959549+0000","last_unstale":"2024-04-26T18:25:06.296547+0000","last_undegraded":"2024-04-26T18:25:06.296547+0000","last_fullsized":"2024-04-26T18:25:06.296547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:18:37.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297969+0000","last_change":"2024-04-26T18:24:18.141549+0000","last_active":"2024-04-26T18:25:06.297969+0000","last_peered":"2024-04-26T18:25:06.297969+0000","last_clean":"2024-04-26T18:25:06.297969+0000","last_became_active":"2024-04-26T18:24:18.141172+0000","last_became_peered":"2024-04-26T18:24:18.141172+0000","last_unstale":"2024-04-26T18:25:06.297969+0000","last_undegraded":"2024-04-26T18:25:06.297969+0000","last_fullsized":"2024-04-26T18:25:06.297969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:15.878130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.297295+0000","last_change":"2024-04-26T18:24:20.948121+0000","last_active":"2024-04-26T18:25:06.297295+0000","last_peered":"2024-04-26T18:25:06.297295+0000","last_clean":"2024-04-26T18:25:06.297295+0000","last_became_active":"2024-04-26T18:24:20.947738+0000","last_became_peered":"2024-04-26T18:24:20.947738+0000","last_unstale":"2024-04-26T18:25:06.297295+0000","last_undegraded":"2024-04-26T18:25:06.297295+0000","last_fullsized":"2024-04-26T18:25:06.297295+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:04.237302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754809+0000","last_change":"2024-04-26T18:24:17.926631+0000","last_active":"2024-04-26T18:25:06.754809+0000","last_peered":"2024-04-26T18:25:06.754809+0000","last_clean":"2024-04-26T18:25:06.754809+0000","last_became_active":"2024-04-26T18:24:17.926259+0000","last_became_peered":"2024-04-26T18:24:17.926259+0000","last_unstale":"2024-04-26T18:25:06.754809+0000","last_undegraded":"2024-04-26T18:25:06.754809+0000","last_fullsized":"2024-04-26T18:25:06.754809+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:47.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298685+0000","last_change":"2024-04-26T18:24:20.947283+0000","last_active":"2024-04-26T18:25:06.298685+0000","last_peered":"2024-04-26T18:25:06.298685+0000","last_clean":"2024-04-26T18:25:06.298685+0000","last_became_active":"2024-04-26T18:24:20.946812+0000","last_became_peered":"2024-04-26T18:24:20.946812+0000","last_unstale":"2024-04-26T18:25:06.298685+0000","last_undegraded":"2024-04-26T18:25:06.298685+0000","last_fullsized":"2024-04-26T18:25:06.298685+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:33.730698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294878+0000","last_change":"2024-04-26T18:24:17.931843+0000","last_active":"2024-04-26T18:25:06.294878+0000","last_peered":"2024-04-26T18:25:06.294878+0000","last_clean":"2024-04-26T18:25:06.294878+0000","last_became_active":"2024-04-26T18:24:17.931442+0000","last_became_peered":"2024-04-26T18:24:17.931442+0000","last_unstale":"2024-04-26T18:25:06.294878+0000","last_undegraded":"2024-04-26T18:25:06.294878+0000","last_fullsized":"2024-04-26T18:25:06.294878+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:09.649435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.755046+0000","last_change":"2024-04-26T18:24:20.955661+0000","last_active":"2024-04-26T18:25:06.755046+0000","last_peered":"2024-04-26T18:25:06.755046+0000","last_clean":"2024-04-26T18:25:06.755046+0000","last_became_active":"2024-04-26T18:24:20.955344+0000","last_became_peered":"2024-04-26T18:24:20.955344+0000","last_unstale":"2024-04-26T18:25:06.755046+0000","last_undegraded":"2024-04-26T18:25:06.755046+0000","last_fullsized":"2024-04-26T18:25:06.755046+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:44.274381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.315466+0000","last_change":"2024-04-26T18:25:06.315466+0000","last_active":"2024-04-26T18:25:06.315466+0000","last_peered":"2024-04-26T18:25:06.315466+0000","last_clean":"2024-04-26T18:25:06.315466+0000","last_became_active":"2024-04-26T18:25:06.301788+0000","last_became_peered":"2024-04-26T18:25:06.301788+0000","last_unstale":"2024-04-26T18:25:06.315466+0000","last_undegraded":"2024-04-26T18:25:06.315466+0000","last_fullsized":"2024-04-26T18:25:06.315466+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:12.998583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295745+0000","last_change":"2024-04-26T18:24:20.955746+0000","last_active":"2024-04-26T18:25:06.295745+0000","last_peered":"2024-04-26T18:25:06.295745+0000","last_clean":"2024-04-26T18:25:06.295745+0000","last_became_active":"2024-04-26T18:24:20.953652+0000","last_became_peered":"2024-04-26T18:24:20.953652+0000","last_unstale":"2024-04-26T18:25:06.295745+0000","last_undegraded":"2024-04-26T18:25:06.295745+0000","last_fullsized":"2024-04-26T18:25:06.295745+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:31.414185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"89'6","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:19.326139+0000","last_change":"2024-04-26T18:25:06.309393+0000","last_active":"2024-04-26T18:25:19.326139+0000","last_peered":"2024-04-26T18:25:19.326139+0000","last_clean":"2024-04-26T18:25:19.326139+0000","last_became_active":"2024-04-26T18:25:06.302434+0000","last_became_peered":"2024-04-26T18:25:06.302434+0000","last_unstale":"2024-04-26T18:25:19.326139+0000","last_undegraded":"2024-04-26T18:25:19.326139+0000","last_fullsized":"2024-04-26T18:25:19.326139+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:54.730295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":2,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296883+0000","last_change":"2024-04-26T18:24:20.954238+0000","last_active":"2024-04-26T18:25:06.296883+0000","last_peered":"2024-04-26T18:25:06.296883+0000","last_clean":"2024-04-26T18:25:06.296883+0000","last_became_active":"2024-04-26T18:24:20.953883+0000","last_became_peered":"2024-04-26T18:24:20.953883+0000","last_unstale":"2024-04-26T18:25:06.296883+0000","last_undegraded":"2024-04-26T18:25:06.296883+0000","last_fullsized":"2024-04-26T18:25:06.296883+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:10.580835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296855+0000","last_change":"2024-04-26T18:24:17.926901+0000","last_active":"2024-04-26T18:25:06.296855+0000","last_peered":"2024-04-26T18:25:06.296855+0000","last_clean":"2024-04-26T18:25:06.296855+0000","last_became_active":"2024-04-26T18:24:17.926291+0000","last_became_peered":"2024-04-26T18:24:17.926291+0000","last_unstale":"2024-04-26T18:25:06.296855+0000","last_undegraded":"2024-04-26T18:25:06.296855+0000","last_fullsized":"2024-04-26T18:25:06.296855+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:17.695198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"89'1","reported_seq":21,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296092+0000","last_change":"2024-04-26T18:24:20.950578+0000","last_active":"2024-04-26T18:25:06.296092+0000","last_peered":"2024-04-26T18:25:06.296092+0000","last_clean":"2024-04-26T18:25:06.296092+0000","last_became_active":"2024-04-26T18:24:20.950174+0000","last_became_peered":"2024-04-26T18:24:20.950174+0000","last_unstale":"2024-04-26T18:25:06.296092+0000","last_undegraded":"2024-04-26T18:25:06.296092+0000","last_fullsized":"2024-04-26T18:25:06.296092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349265+0000","last_change":"2024-04-26T18:24:17.928054+0000","last_active":"2024-04-26T18:25:06.349265+0000","last_peered":"2024-04-26T18:25:06.349265+0000","last_clean":"2024-04-26T18:25:06.349265+0000","last_became_active":"2024-04-26T18:24:17.927715+0000","last_became_peered":"2024-04-26T18:24:17.927715+0000","last_unstale":"2024-04-26T18:25:06.349265+0000","last_undegraded":"2024-04-26T18:25:06.349265+0000","last_fullsized":"2024-04-26T18:25:06.349265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.155436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.296772+0000","last_change":"2024-04-26T18:24:20.944512+0000","last_active":"2024-04-26T18:25:06.296772+0000","last_peered":"2024-04-26T18:25:06.296772+0000","last_clean":"2024-04-26T18:25:06.296772+0000","last_became_active":"2024-04-26T18:24:20.944123+0000","last_became_peered":"2024-04-26T18:24:20.944123+0000","last_unstale":"2024-04-26T18:25:06.296772+0000","last_undegraded":"2024-04-26T18:25:06.296772+0000","last_fullsized":"2024-04-26T18:25:06.296772+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.240206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298795+0000","last_change":"2024-04-26T18:24:20.959138+0000","last_active":"2024-04-26T18:25:06.298795+0000","last_peered":"2024-04-26T18:25:06.298795+0000","last_clean":"2024-04-26T18:25:06.298795+0000","last_became_active":"2024-04-26T18:24:20.958675+0000","last_became_peered":"2024-04-26T18:24:20.958675+0000","last_unstale":"2024-04-26T18:25:06.298795+0000","last_undegraded":"2024-04-26T18:25:06.298795+0000","last_fullsized":"2024-04-26T18:25:06.298795+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:34:32.103380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294547+0000","last_change":"2024-04-26T18:24:20.961689+0000","last_active":"2024-04-26T18:25:06.294547+0000","last_peered":"2024-04-26T18:25:06.294547+0000","last_clean":"2024-04-26T18:25:06.294547+0000","last_became_active":"2024-04-26T18:24:20.961374+0000","last_became_peered":"2024-04-26T18:24:20.961374+0000","last_unstale":"2024-04-26T18:25:06.294547+0000","last_undegraded":"2024-04-26T18:25:06.294547+0000","last_fullsized":"2024-04-26T18:25:06.294547+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:55.243887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.294515+0000","last_change":"2024-04-26T18:24:20.950711+0000","last_active":"2024-04-26T18:25:06.294515+0000","last_peered":"2024-04-26T18:25:06.294515+0000","last_clean":"2024-04-26T18:25:06.294515+0000","last_became_active":"2024-04-26T18:24:20.950371+0000","last_became_peered":"2024-04-26T18:24:20.950371+0000","last_unstale":"2024-04-26T18:25:06.294515+0000","last_undegraded":"2024-04-26T18:25:06.294515+0000","last_fullsized":"2024-04-26T18:25:06.294515+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:35.307921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295964+0000","last_change":"2024-04-26T18:24:20.959990+0000","last_active":"2024-04-26T18:25:06.295964+0000","last_peered":"2024-04-26T18:25:06.295964+0000","last_clean":"2024-04-26T18:25:06.295964+0000","last_became_active":"2024-04-26T18:24:20.959514+0000","last_became_peered":"2024-04-26T18:24:20.959514+0000","last_unstale":"2024-04-26T18:25:06.295964+0000","last_undegraded":"2024-04-26T18:25:06.295964+0000","last_fullsized":"2024-04-26T18:25:06.295964+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:00.184448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349163+0000","last_change":"2024-04-26T18:24:20.957338+0000","last_active":"2024-04-26T18:25:06.349163+0000","last_peered":"2024-04-26T18:25:06.349163+0000","last_clean":"2024-04-26T18:25:06.349163+0000","last_became_active":"2024-04-26T18:24:20.956999+0000","last_became_peered":"2024-04-26T18:24:20.956999+0000","last_unstale":"2024-04-26T18:25:06.349163+0000","last_undegraded":"2024-04-26T18:25:06.349163+0000","last_fullsized":"2024-04-26T18:25:06.349163+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.066902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299283+0000","last_change":"2024-04-26T18:24:20.953634+0000","last_active":"2024-04-26T18:25:06.299283+0000","last_peered":"2024-04-26T18:25:06.299283+0000","last_clean":"2024-04-26T18:25:06.299283+0000","last_became_active":"2024-04-26T18:24:20.953259+0000","last_became_peered":"2024-04-26T18:24:20.953259+0000","last_unstale":"2024-04-26T18:25:06.299283+0000","last_undegraded":"2024-04-26T18:25:06.299283+0000","last_fullsized":"2024-04-26T18:25:06.299283+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:00.095344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.295478+0000","last_change":"2024-04-26T18:24:20.954997+0000","last_active":"2024-04-26T18:25:06.295478+0000","last_peered":"2024-04-26T18:25:06.295478+0000","last_clean":"2024-04-26T18:25:06.295478+0000","last_became_active":"2024-04-26T18:24:20.954223+0000","last_became_peered":"2024-04-26T18:24:20.954223+0000","last_unstale":"2024-04-26T18:25:06.295478+0000","last_undegraded":"2024-04-26T18:25:06.295478+0000","last_fullsized":"2024-04-26T18:25:06.295478+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:14.092455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754906+0000","last_change":"2024-04-26T18:24:20.957056+0000","last_active":"2024-04-26T18:25:06.754906+0000","last_peered":"2024-04-26T18:25:06.754906+0000","last_clean":"2024-04-26T18:25:06.754906+0000","last_became_active":"2024-04-26T18:24:20.956737+0000","last_became_peered":"2024-04-26T18:24:20.956737+0000","last_unstale":"2024-04-26T18:25:06.754906+0000","last_undegraded":"2024-04-26T18:25:06.754906+0000","last_fullsized":"2024-04-26T18:25:06.754906+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:48.358862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":26,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.350206+0000","last_change":"2024-04-26T18:25:06.350206+0000","last_active":"2024-04-26T18:25:06.350206+0000","last_peered":"2024-04-26T18:25:06.350206+0000","last_clean":"2024-04-26T18:25:06.350206+0000","last_became_active":"2024-04-26T18:25:06.349866+0000","last_became_peered":"2024-04-26T18:25:06.349866+0000","last_unstale":"2024-04-26T18:25:06.350206+0000","last_undegraded":"2024-04-26T18:25:06.350206+0000","last_fullsized":"2024-04-26T18:25:06.350206+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:41.909056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.754750+0000","last_change":"2024-04-26T18:24:20.956000+0000","last_active":"2024-04-26T18:25:06.754750+0000","last_peered":"2024-04-26T18:25:06.754750+0000","last_clean":"2024-04-26T18:25:06.754750+0000","last_became_active":"2024-04-26T18:24:20.955677+0000","last_became_peered":"2024-04-26T18:24:20.955677+0000","last_unstale":"2024-04-26T18:25:06.754750+0000","last_undegraded":"2024-04-26T18:25:06.754750+0000","last_fullsized":"2024-04-26T18:25:06.754750+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:20.895674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.298152+0000","last_change":"2024-04-26T18:24:20.959317+0000","last_active":"2024-04-26T18:25:06.298152+0000","last_peered":"2024-04-26T18:25:06.298152+0000","last_clean":"2024-04-26T18:25:06.298152+0000","last_became_active":"2024-04-26T18:24:20.958963+0000","last_became_peered":"2024-04-26T18:24:20.958963+0000","last_unstale":"2024-04-26T18:25:06.298152+0000","last_undegraded":"2024-04-26T18:25:06.298152+0000","last_fullsized":"2024-04-26T18:25:06.298152+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:08.231089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.349143+0000","last_change":"2024-04-26T18:24:20.954445+0000","last_active":"2024-04-26T18:25:06.349143+0000","last_peered":"2024-04-26T18:25:06.349143+0000","last_clean":"2024-04-26T18:25:06.349143+0000","last_became_active":"2024-04-26T18:24:20.953852+0000","last_became_peered":"2024-04-26T18:24:20.953852+0000","last_unstale":"2024-04-26T18:25:06.349143+0000","last_undegraded":"2024-04-26T18:25:06.349143+0000","last_fullsized":"2024-04-26T18:25:06.349143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:05.636897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-26T18:25:06.299528+0000","last_change":"2024-04-26T18:24:20.958023+0000","last_active":"2024-04-26T18:25:06.299528+0000","last_peered":"2024-04-26T18:25:06.299528+0000","last_clean":"2024-04-26T18:25:06.299528+0000","last_became_active":"2024-04-26T18:24:20.957642+0000","last_became_peered":"2024-04-26T18:24:20.957642+0000","last_unstale":"2024-04-26T18:25:06.299528+0000","last_undegraded":"2024-04-26T18:25:06.299528+0000","last_fullsized":"2024-04-26T18:25:06.299528+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:46.332710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244217,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71928,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":418,"num_read_kb":294,"num_write":245,"num_write_kb":139,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1583,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":215784,"data_compressed":5529,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":211,"ondisk_log_size":211,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351003,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":999424,"data_stored":484082,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612639,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32304,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712848,"statfs":{"total":95995035648,"available":95961956352,"internally_reserved":0,"allocated":1150976,"data_stored":1139422,"data_compressed":43211,"data_compressed_allocated":57344,"data_compressed_original":614400,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776165,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32176,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712976,"statfs":{"total":95995035648,"available":95962087424,"internally_reserved":0,"allocated":1019904,"data_stored":483599,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037803,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":483955,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299439,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1069056,"data_stored":1076495,"data_compressed":41368,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462965,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28004,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717148,"statfs":{"total":95995035648,"available":95966359552,"internally_reserved":0,"allocated":1003520,"data_stored":483316,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":550514,"data_compressed":7620,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1024000,"data_stored":485046,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411401,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1052,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1077248,"data_stored":548394,"data_compressed":7620,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149763,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1019904,"data_stored":483141,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888127,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1003520,"data_stored":483177,"data_compressed":5777,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724600,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32236,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712916,"statfs":{"total":95995035648,"available":95962025984,"internally_reserved":0,"allocated":1081344,"data_stored":1076173,"data_compressed":41368,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68037,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65917,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":66460,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:25:24.446 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T18:25:24.447 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T18:25:24.447 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T18:25:24.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:24 smithi103 ceph-mon[32727]: pgmap v330: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T18:25:24.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:24 smithi174 ceph-mon[32705]: pgmap v330: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-26T18:25:24.689 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:25.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:25 smithi084 ceph-mon[29141]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:25.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:25 smithi103 ceph-mon[32727]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:25.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:25 smithi174 ceph-mon[32705]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:25:26.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.394+0000 7f236129c700 1 -- 172.21.15.84:0/3014650058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c1023a0 msgr2=0x7f235c1027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:26.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.394+0000 7f236129c700 1 --2- 172.21.15.84:0/3014650058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c1023a0 0x7f235c1027c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f234c0099c0 tx=0x7f234c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:25:26.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.394+0000 7f236129c700 1 -- 172.21.15.84:0/3014650058 shutdown_connections 2024-04-26T18:25:26.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.394+0000 7f236129c700 1 --2- 172.21.15.84:0/3014650058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c1023a0 0x7f235c1027c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:26.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.394+0000 7f236129c700 1 --2- 172.21.15.84:0/3014650058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c103f40 0x7f235c10a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:26.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.394+0000 7f236129c700 1 --2- 172.21.15.84:0/3014650058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f235c1035a0 0x7f235c103a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:26.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.394+0000 7f236129c700 1 -- 172.21.15.84:0/3014650058 >> 172.21.15.84:0/3014650058 conn(0x7f235c0fd940 msgr2=0x7f235c0ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:26.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.394+0000 7f236129c700 1 -- 172.21.15.84:0/3014650058 shutdown_connections 2024-04-26T18:25:26.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.394+0000 7f236129c700 1 -- 172.21.15.84:0/3014650058 wait complete. 2024-04-26T18:25:26.396 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:26 smithi084 ceph-mon[29141]: pgmap v331: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-26T18:25:26.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.395+0000 7f236129c700 1 Processor -- start 2024-04-26T18:25:26.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.395+0000 7f236129c700 1 -- start start 2024-04-26T18:25:26.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f236129c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f235c1023a0 0x7f235c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:26.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f236129c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c1035a0 0x7f235c19bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:26.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f236129c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c103f40 0x7f235c1a2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:26.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f236129c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f235c10de90 con 0x7f235c1035a0 2024-04-26T18:25:26.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f236129c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f235c10dd10 con 0x7f235c1023a0 2024-04-26T18:25:26.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f236129c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f235c10e010 con 0x7f235c103f40 2024-04-26T18:25:26.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f235a7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c1035a0 0x7f235c19bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:26.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f235a7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c1035a0 0x7f235c19bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48952/0 (socket says 172.21.15.84:48952) 2024-04-26T18:25:26.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f235a7fc700 1 -- 172.21.15.84:0/195696644 learned_addr learned my addr 172.21.15.84:0/195696644 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:26.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f235affd700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f235c1023a0 0x7f235c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:26.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f235b7fe700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c103f40 0x7f235c1a2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:26.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f235a7fc700 1 -- 172.21.15.84:0/195696644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c103f40 msgr2=0x7f235c1a2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:26.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f235a7fc700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c103f40 0x7f235c1a2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:26.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f235a7fc700 1 -- 172.21.15.84:0/195696644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f235c1023a0 msgr2=0x7f235c19b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:26.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f235a7fc700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f235c1023a0 0x7f235c19b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:26.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f235a7fc700 1 -- 172.21.15.84:0/195696644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f235c1a33d0 con 0x7f235c1035a0 2024-04-26T18:25:26.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f235affd700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f235c1023a0 0x7f235c19b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:26.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f235a7fc700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c1035a0 0x7f235c19bcb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f235000c9e0 tx=0x7f235000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:26.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f2343fff700 1 -- 172.21.15.84:0/195696644 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350007ac0 con 0x7f235c1035a0 2024-04-26T18:25:26.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.396+0000 7f236129c700 1 -- 172.21.15.84:0/195696644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f235c1a36c0 con 0x7f235c1035a0 2024-04-26T18:25:26.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.397+0000 7f2343fff700 1 -- 172.21.15.84:0/195696644 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2350007c60 con 0x7f235c1035a0 2024-04-26T18:25:26.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.397+0000 7f236129c700 1 -- 172.21.15.84:0/195696644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f235c1a3c50 con 0x7f235c1035a0 2024-04-26T18:25:26.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.397+0000 7f2343fff700 1 -- 172.21.15.84:0/195696644 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2350018660 con 0x7f235c1035a0 2024-04-26T18:25:26.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.398+0000 7f236129c700 1 -- 172.21.15.84:0/195696644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2348005330 con 0x7f235c1035a0 2024-04-26T18:25:26.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.399+0000 7f2343fff700 1 -- 172.21.15.84:0/195696644 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2350018800 con 0x7f235c1035a0 2024-04-26T18:25:26.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.399+0000 7f2343fff700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f23440a3e00 0x7f23440a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:26.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.400+0000 7f235affd700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f23440a3e00 0x7f23440a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:26.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.400+0000 7f2343fff700 1 -- 172.21.15.84:0/195696644 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f23500d08a0 con 0x7f235c1035a0 2024-04-26T18:25:26.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.400+0000 7f235affd700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f23440a3e00 0x7f23440a62c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f234c00b580 tx=0x7f234c005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:26.403 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.403+0000 7f2343fff700 1 -- 172.21.15.84:0/195696644 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2350014030 con 0x7f235c1035a0 2024-04-26T18:25:26.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:26 smithi103 ceph-mon[32727]: pgmap v331: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-26T18:25:26.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.683+0000 7f236129c700 1 -- 172.21.15.84:0/195696644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f2348005620 con 0x7f235c1035a0 2024-04-26T18:25:26.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.685+0000 7f2343fff700 1 -- 172.21.15.84:0/195696644 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f235009d190 con 0x7f235c1035a0 2024-04-26T18:25:26.686 INFO:teuthology.orchestra.run.smithi084.stdout:[client.0] 2024-04-26T18:25:26.686 INFO:teuthology.orchestra.run.smithi084.stdout: key = AQCW8StmDUbTKBAAQsz7CljtEHQ5uOftYOv6Hg== 2024-04-26T18:25:26.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.687+0000 7f236129c700 1 -- 172.21.15.84:0/195696644 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f23440a3e00 msgr2=0x7f23440a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:26.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.687+0000 7f236129c700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f23440a3e00 0x7f23440a62c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f234c00b580 tx=0x7f234c005fb0 comp rx=0 tx=0).stop 2024-04-26T18:25:26.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.687+0000 7f236129c700 1 -- 172.21.15.84:0/195696644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c1035a0 msgr2=0x7f235c19bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:26.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.687+0000 7f236129c700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c1035a0 0x7f235c19bcb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f235000c9e0 tx=0x7f235000cda0 comp rx=0 tx=0).stop 2024-04-26T18:25:26.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.687+0000 7f236129c700 1 -- 172.21.15.84:0/195696644 shutdown_connections 2024-04-26T18:25:26.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.687+0000 7f236129c700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f23440a3e00 0x7f23440a62c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:26.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.687+0000 7f236129c700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f235c103f40 0x7f235c1a2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:26.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.687+0000 7f236129c700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f235c1023a0 0x7f235c19b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:26.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.687+0000 7f236129c700 1 --2- 172.21.15.84:0/195696644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f235c1035a0 0x7f235c19bcb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:26.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.688+0000 7f236129c700 1 -- 172.21.15.84:0/195696644 >> 172.21.15.84:0/195696644 conn(0x7f235c0fd940 msgr2=0x7f235c06b100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:26.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.688+0000 7f236129c700 1 -- 172.21.15.84:0/195696644 shutdown_connections 2024-04-26T18:25:26.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:26.688+0000 7f236129c700 1 -- 172.21.15.84:0/195696644 wait complete. 2024-04-26T18:25:26.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:26 smithi174 ceph-mon[32705]: pgmap v331: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-26T18:25:27.464 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:27 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/195696644' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T18:25:27.465 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:27 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/195696644' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T18:25:27.499 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:25:27.499 DEBUG:teuthology.orchestra.run.smithi084:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T18:25:27.499 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T18:25:27.538 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T18:25:27.538 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T18:25:27.539 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T18:25:27.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:27 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/195696644' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T18:25:27.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:27 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/195696644' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T18:25:27.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:27 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/195696644' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T18:25:27.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:27 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/195696644' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T18:25:27.819 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:28.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:28 smithi084 ceph-mon[29141]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-26T18:25:28.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:28 smithi103 ceph-mon[32727]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-26T18:25:28.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:28 smithi174 ceph-mon[32705]: pgmap v332: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-26T18:25:29.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.492+0000 7f209b68f700 1 -- 172.21.15.84:0/2515539262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209406bb90 msgr2=0x7f209406c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.492+0000 7f209b68f700 1 --2- 172.21.15.84:0/2515539262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209406bb90 0x7f209406c010 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2090009990 tx=0x7f2090009ca0 comp rx=0 tx=0).stop 2024-04-26T18:25:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.492+0000 7f209b68f700 1 -- 172.21.15.84:0/2515539262 shutdown_connections 2024-04-26T18:25:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.492+0000 7f209b68f700 1 --2- 172.21.15.84:0/2515539262 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f209406c550 0x7f2094110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.492+0000 7f209b68f700 1 --2- 172.21.15.84:0/2515539262 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f20941094c0 0x7f20941098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.492+0000 7f209b68f700 1 --2- 172.21.15.84:0/2515539262 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209406bb90 0x7f209406c010 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.492+0000 7f209b68f700 1 -- 172.21.15.84:0/2515539262 >> 172.21.15.84:0/2515539262 conn(0x7f2094078370 msgr2=0x7f2094078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.492+0000 7f209b68f700 1 -- 172.21.15.84:0/2515539262 shutdown_connections 2024-04-26T18:25:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.492+0000 7f209b68f700 1 -- 172.21.15.84:0/2515539262 wait complete. 2024-04-26T18:25:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.493+0000 7f209b68f700 1 Processor -- start 2024-04-26T18:25:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.493+0000 7f209b68f700 1 -- start start 2024-04-26T18:25:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.493+0000 7f209b68f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209406bb90 0x7f20941a18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:29.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.493+0000 7f209b68f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f209406c550 0x7f20941a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.493+0000 7f209b68f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20941094c0 0x7f209419baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.493+0000 7f209b68f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2094113600 con 0x7f209406bb90 2024-04-26T18:25:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.493+0000 7f209b68f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2094113480 con 0x7f209406c550 2024-04-26T18:25:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.493+0000 7f209b68f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2094113780 con 0x7f20941094c0 2024-04-26T18:25:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.493+0000 7f209942b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209406bb90 0x7f20941a18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.493+0000 7f209942b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209406bb90 0x7f20941a18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37040/0 (socket says 172.21.15.84:37040) 2024-04-26T18:25:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.493+0000 7f209942b700 1 -- 172.21.15.84:0/2724144970 learned_addr learned my addr 172.21.15.84:0/2724144970 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.494+0000 7f2099c2c700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20941094c0 0x7f209419baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.494+0000 7f2098c2a700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f209406c550 0x7f20941a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.494+0000 7f209942b700 1 -- 172.21.15.84:0/2724144970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20941094c0 msgr2=0x7f209419baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.494+0000 7f209942b700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20941094c0 0x7f209419baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.494+0000 7f209942b700 1 -- 172.21.15.84:0/2724144970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f209406c550 msgr2=0x7f20941a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:29.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.494+0000 7f209942b700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f209406c550 0x7f20941a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:29.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.494+0000 7f209942b700 1 -- 172.21.15.84:0/2724144970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f209419c200 con 0x7f209406bb90 2024-04-26T18:25:29.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.494+0000 7f209942b700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209406bb90 0x7f20941a18e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f208400d950 tx=0x7f208400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:29.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.494+0000 7f208a7fc700 1 -- 172.21.15.84:0/2724144970 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2084016070 con 0x7f209406bb90 2024-04-26T18:25:29.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.494+0000 7f209b68f700 1 -- 172.21.15.84:0/2724144970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f209419c4f0 con 0x7f209406bb90 2024-04-26T18:25:29.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.494+0000 7f208a7fc700 1 -- 172.21.15.84:0/2724144970 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f208400fd40 con 0x7f209406bb90 2024-04-26T18:25:29.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.494+0000 7f208a7fc700 1 -- 172.21.15.84:0/2724144970 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f208400e3b0 con 0x7f209406bb90 2024-04-26T18:25:29.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.494+0000 7f209b68f700 1 -- 172.21.15.84:0/2724144970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20941a87b0 con 0x7f209406bb90 2024-04-26T18:25:29.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.495+0000 7f209b68f700 1 -- 172.21.15.84:0/2724144970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f209406a110 con 0x7f209406bb90 2024-04-26T18:25:29.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.496+0000 7f208a7fc700 1 -- 172.21.15.84:0/2724144970 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2084019070 con 0x7f209406bb90 2024-04-26T18:25:29.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.497+0000 7f208a7fc700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f20800a3cd0 0x7f20800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:29.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.497+0000 7f2098c2a700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f20800a3cd0 0x7f20800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:29.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.497+0000 7f208a7fc700 1 -- 172.21.15.84:0/2724144970 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f20840cff80 con 0x7f209406bb90 2024-04-26T18:25:29.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.497+0000 7f2098c2a700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f20800a3cd0 0x7f20800a6190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f209000b5c0 tx=0x7f2090005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:29.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.501+0000 7f208a7fc700 1 -- 172.21.15.84:0/2724144970 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f208409c7f0 con 0x7f209406bb90 2024-04-26T18:25:29.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.762+0000 7f209b68f700 1 -- 172.21.15.84:0/2724144970 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f20941a89e0 con 0x7f209406bb90 2024-04-26T18:25:29.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.767+0000 7f208a7fc700 1 -- 172.21.15.84:0/2724144970 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264995 (secure 0 0 0) 0x7f208409c0f0 con 0x7f209406bb90 2024-04-26T18:25:29.773 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:25:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.771+0000 7f209b68f700 1 -- 172.21.15.84:0/2724144970 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f20800a3cd0 msgr2=0x7f20800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.771+0000 7f209b68f700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f20800a3cd0 0x7f20800a6190 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f209000b5c0 tx=0x7f2090005960 comp rx=0 tx=0).stop 2024-04-26T18:25:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.771+0000 7f209b68f700 1 -- 172.21.15.84:0/2724144970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209406bb90 msgr2=0x7f20941a18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.771+0000 7f209b68f700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209406bb90 0x7f20941a18e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f208400d950 tx=0x7f208400dc60 comp rx=0 tx=0).stop 2024-04-26T18:25:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.771+0000 7f209b68f700 1 -- 172.21.15.84:0/2724144970 shutdown_connections 2024-04-26T18:25:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.771+0000 7f209b68f700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f20800a3cd0 0x7f20800a6190 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.771+0000 7f209b68f700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f209406bb90 0x7f20941a18e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.771+0000 7f209b68f700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f20941094c0 0x7f209419baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.771+0000 7f209b68f700 1 --2- 172.21.15.84:0/2724144970 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f209406c550 0x7f20941a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.771+0000 7f209b68f700 1 -- 172.21.15.84:0/2724144970 >> 172.21.15.84:0/2724144970 conn(0x7f2094078370 msgr2=0x7f20941005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:29.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.771+0000 7f209b68f700 1 -- 172.21.15.84:0/2724144970 shutdown_connections 2024-04-26T18:25:29.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:29.771+0000 7f209b68f700 1 -- 172.21.15.84:0/2724144970 wait complete. 2024-04-26T18:25:30.457 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:30 smithi084 ceph-mon[29141]: pgmap v333: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-26T18:25:30.457 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:30 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2724144970' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T18:25:30.485 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6800","nonce":2059208901},{"type":"v1","addr":"172.21.15.84:6801","nonce":2059208901}]},"active_addr":"172.21.15.84:6801/2059208901","active_change":"2024-04-26T18:17:05.083341+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24113,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.84:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":2142936585}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":3834640430}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":277477867}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":559313476}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.84:0","nonce":506413503}]}]} 2024-04-26T18:25:30.490 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T18:25:30.490 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T18:25:30.490 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:25:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:30 smithi103 ceph-mon[32727]: pgmap v333: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-26T18:25:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:30 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2724144970' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T18:25:30.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:30 smithi174 ceph-mon[32705]: pgmap v333: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-26T18:25:30.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:30 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2724144970' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T18:25:30.735 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:32.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.511+0000 7f1fc858e700 1 -- 172.21.15.84:0/2274788362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fc0102610 msgr2=0x7f1fc01029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:32.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.511+0000 7f1fc858e700 1 --2- 172.21.15.84:0/2274788362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fc0102610 0x7f1fc01029f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1fb800b0d0 tx=0x7f1fb800b3e0 comp rx=0 tx=0).stop 2024-04-26T18:25:32.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.511+0000 7f1fc858e700 1 -- 172.21.15.84:0/2274788362 shutdown_connections 2024-04-26T18:25:32.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.511+0000 7f1fc858e700 1 --2- 172.21.15.84:0/2274788362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fc0102610 0x7f1fc01029f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:32.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.511+0000 7f1fc858e700 1 --2- 172.21.15.84:0/2274788362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fc0107660 0x7f1fc010f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:32.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.511+0000 7f1fc858e700 1 --2- 172.21.15.84:0/2274788362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fc0102fc0 0x7f1fc0106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:32.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.511+0000 7f1fc858e700 1 -- 172.21.15.84:0/2274788362 >> 172.21.15.84:0/2274788362 conn(0x7f1fc0078370 msgr2=0x7f1fc0078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:32.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.511+0000 7f1fc858e700 1 -- 172.21.15.84:0/2274788362 shutdown_connections 2024-04-26T18:25:32.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.512+0000 7f1fc858e700 1 -- 172.21.15.84:0/2274788362 wait complete. 2024-04-26T18:25:32.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.512+0000 7f1fc858e700 1 Processor -- start 2024-04-26T18:25:32.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.512+0000 7f1fc858e700 1 -- start start 2024-04-26T18:25:32.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.512+0000 7f1fc858e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fc0102610 0x7f1fc019aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:32.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.512+0000 7f1fc858e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fc0102fc0 0x7f1fc019b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:32.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.512+0000 7f1fc858e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fc0107660 0x7f1fc01a5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:32.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.512+0000 7f1fc858e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc01125f0 con 0x7f1fc0102fc0 2024-04-26T18:25:32.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.512+0000 7f1fc858e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0112470 con 0x7f1fc0107660 2024-04-26T18:25:32.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.512+0000 7f1fc858e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0112770 con 0x7f1fc0102610 2024-04-26T18:25:32.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fc5b29700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fc0102fc0 0x7f1fc019b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:32.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fc5b29700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fc0102fc0 0x7f1fc019b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37056/0 (socket says 172.21.15.84:37056) 2024-04-26T18:25:32.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fc5b29700 1 -- 172.21.15.84:0/1322951653 learned_addr learned my addr 172.21.15.84:0/1322951653 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:32.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fc632a700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fc0102610 0x7f1fc019aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:32.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fc6b2b700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fc0107660 0x7f1fc01a5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:32.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fc5b29700 1 -- 172.21.15.84:0/1322951653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fc0102610 msgr2=0x7f1fc019aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:32.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fc5b29700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fc0102610 0x7f1fc019aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:32.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fc5b29700 1 -- 172.21.15.84:0/1322951653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fc0107660 msgr2=0x7f1fc01a5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:32.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fc5b29700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fc0107660 0x7f1fc01a5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:32.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fc5b29700 1 -- 172.21.15.84:0/1322951653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc01a60b0 con 0x7f1fc0102fc0 2024-04-26T18:25:32.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fc5b29700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fc0102fc0 0x7f1fc019b030 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400b500 tx=0x7f1fb400b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:32.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fbf7fe700 1 -- 172.21.15.84:0/1322951653 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4015020 con 0x7f1fc0102fc0 2024-04-26T18:25:32.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fbf7fe700 1 -- 172.21.15.84:0/1322951653 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb4011ce0 con 0x7f1fc0102fc0 2024-04-26T18:25:32.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fc858e700 1 -- 172.21.15.84:0/1322951653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc01a62e0 con 0x7f1fc0102fc0 2024-04-26T18:25:32.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fbf7fe700 1 -- 172.21.15.84:0/1322951653 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4010d40 con 0x7f1fc0102fc0 2024-04-26T18:25:32.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.513+0000 7f1fc858e700 1 -- 172.21.15.84:0/1322951653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc01a67f0 con 0x7f1fc0102fc0 2024-04-26T18:25:32.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.514+0000 7f1fc858e700 1 -- 172.21.15.84:0/1322951653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc006a110 con 0x7f1fc0102fc0 2024-04-26T18:25:32.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.516+0000 7f1fbf7fe700 1 -- 172.21.15.84:0/1322951653 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1fb4018020 con 0x7f1fc0102fc0 2024-04-26T18:25:32.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.517+0000 7f1fbf7fe700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1fa80a3cd0 0x7f1fa80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:32.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.517+0000 7f1fc632a700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1fa80a3cd0 0x7f1fa80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:32.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.517+0000 7f1fbf7fe700 1 -- 172.21.15.84:0/1322951653 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f1fb40d0240 con 0x7f1fc0102fc0 2024-04-26T18:25:32.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.517+0000 7f1fc632a700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1fa80a3cd0 0x7f1fa80a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009fd0 tx=0x7f1fb800b040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:32.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.520+0000 7f1fbf7fe700 1 -- 172.21.15.84:0/1322951653 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fb401f020 con 0x7f1fc0102fc0 2024-04-26T18:25:32.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:32 smithi084 ceph-mon[29141]: pgmap v334: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:32.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:32 smithi103 ceph-mon[32727]: pgmap v334: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:32 smithi174 ceph-mon[32705]: pgmap v334: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:32.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.728+0000 7f1fc858e700 1 -- 172.21.15.84:0/1322951653 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1fc002e490 con 0x7f1fc0102fc0 2024-04-26T18:25:32.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.730+0000 7f1fbf7fe700 1 -- 172.21.15.84:0/1322951653 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22680 (secure 0 0 0) 0x7f1fb409cb60 con 0x7f1fc0102fc0 2024-04-26T18:25:32.730 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:25:32.730 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":91,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:16:06.267694+0000","modified":"2024-04-26T18:25:06.286417+0000","last_up_change":"2024-04-26T18:23:19.906511+0000","last_in_change":"2024-04-26T18:22:59.541869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:19:31.174024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:24:16.533128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:24:19.742656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1076912-986e-4825-b1d1-5fa9d69bc07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6803","nonce":140050011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6805","nonce":140050011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6809","nonce":140050011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6807","nonce":140050011}]},"public_addr":"172.21.15.84:6803/140050011","cluster_addr":"172.21.15.84:6805/140050011","heartbeat_back_addr":"172.21.15.84:6809/140050011","heartbeat_front_addr":"172.21.15.84:6807/140050011","state":["exists","up"]},{"osd":1,"uuid":"fa93293d-9058-4804-ba4b-7e1e674b0fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6801","nonce":2512857657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6803","nonce":2512857657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6807","nonce":2512857657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6805","nonce":2512857657}]},"public_addr":"172.21.15.103:6801/2512857657","cluster_addr":"172.21.15.103:6803/2512857657","heartbeat_back_addr":"172.21.15.103:6807/2512857657","heartbeat_front_addr":"172.21.15.103:6805/2512857657","state":["exists","up"]},{"osd":2,"uuid":"bf78a44f-80a2-4cd7-974f-5abdf771e650","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6801","nonce":76340924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6803","nonce":76340924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6807","nonce":76340924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6805","nonce":76340924}]},"public_addr":"172.21.15.174:6801/76340924","cluster_addr":"172.21.15.174:6803/76340924","heartbeat_back_addr":"172.21.15.174:6807/76340924","heartbeat_front_addr":"172.21.15.174:6805/76340924","state":["exists","up"]},{"osd":3,"uuid":"55fd0035-bc76-4235-b8b5-5cf5ef770139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6811","nonce":2352789256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6813","nonce":2352789256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6817","nonce":2352789256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6815","nonce":2352789256}]},"public_addr":"172.21.15.84:6811/2352789256","cluster_addr":"172.21.15.84:6813/2352789256","heartbeat_back_addr":"172.21.15.84:6817/2352789256","heartbeat_front_addr":"172.21.15.84:6815/2352789256","state":["exists","up"]},{"osd":4,"uuid":"c278a0f3-e436-4d1c-a009-f9165ebf3c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6809","nonce":1619258252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6811","nonce":1619258252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6815","nonce":1619258252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6813","nonce":1619258252}]},"public_addr":"172.21.15.103:6809/1619258252","cluster_addr":"172.21.15.103:6811/1619258252","heartbeat_back_addr":"172.21.15.103:6815/1619258252","heartbeat_front_addr":"172.21.15.103:6813/1619258252","state":["exists","up"]},{"osd":5,"uuid":"6505be9b-c1b0-4d8b-944e-9920f90cb88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6809","nonce":3168291768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6811","nonce":3168291768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6815","nonce":3168291768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6813","nonce":3168291768}]},"public_addr":"172.21.15.174:6809/3168291768","cluster_addr":"172.21.15.174:6811/3168291768","heartbeat_back_addr":"172.21.15.174:6815/3168291768","heartbeat_front_addr":"172.21.15.174:6813/3168291768","state":["exists","up"]},{"osd":6,"uuid":"a037b66a-7488-469d-b73a-14cc0d35cb5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6819","nonce":886635297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6821","nonce":886635297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6825","nonce":886635297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6823","nonce":886635297}]},"public_addr":"172.21.15.84:6819/886635297","cluster_addr":"172.21.15.84:6821/886635297","heartbeat_back_addr":"172.21.15.84:6825/886635297","heartbeat_front_addr":"172.21.15.84:6823/886635297","state":["exists","up"]},{"osd":7,"uuid":"c425a13f-8019-4484-86a0-a34fa4d8de81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6817","nonce":1738278079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6819","nonce":1738278079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6823","nonce":1738278079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6821","nonce":1738278079}]},"public_addr":"172.21.15.103:6817/1738278079","cluster_addr":"172.21.15.103:6819/1738278079","heartbeat_back_addr":"172.21.15.103:6823/1738278079","heartbeat_front_addr":"172.21.15.103:6821/1738278079","state":["exists","up"]},{"osd":8,"uuid":"687e6c08-bfa4-440c-b3bc-90a5748989d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6817","nonce":38410548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6819","nonce":38410548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6823","nonce":38410548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6821","nonce":38410548}]},"public_addr":"172.21.15.174:6817/38410548","cluster_addr":"172.21.15.174:6819/38410548","heartbeat_back_addr":"172.21.15.174:6823/38410548","heartbeat_front_addr":"172.21.15.174:6821/38410548","state":["exists","up"]},{"osd":9,"uuid":"059eb22d-6dfb-4dd0-b192-84ed42ee1fbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6827","nonce":2926393927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6829","nonce":2926393927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6833","nonce":2926393927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6831","nonce":2926393927}]},"public_addr":"172.21.15.84:6827/2926393927","cluster_addr":"172.21.15.84:6829/2926393927","heartbeat_back_addr":"172.21.15.84:6833/2926393927","heartbeat_front_addr":"172.21.15.84:6831/2926393927","state":["exists","up"]},{"osd":10,"uuid":"da31e305-5220-4a87-a131-005b7c7f76b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6825","nonce":2705726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6827","nonce":2705726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6831","nonce":2705726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6829","nonce":2705726915}]},"public_addr":"172.21.15.103:6825/2705726915","cluster_addr":"172.21.15.103:6827/2705726915","heartbeat_back_addr":"172.21.15.103:6831/2705726915","heartbeat_front_addr":"172.21.15.103:6829/2705726915","state":["exists","up"]},{"osd":11,"uuid":"3fde6c8f-ee02-41ee-a34f-73072b4761e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6825","nonce":1668672128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6827","nonce":1668672128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6831","nonce":1668672128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6829","nonce":1668672128}]},"public_addr":"172.21.15.174:6825/1668672128","cluster_addr":"172.21.15.174:6827/1668672128","heartbeat_back_addr":"172.21.15.174:6831/1668672128","heartbeat_front_addr":"172.21.15.174:6829/1668672128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:39.099188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:03.327608+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:27.714108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:52.506015+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:17.631670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:42.702489+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:07.606697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:34.123214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:59.534658+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:25.357720+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:51.543106+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:23:18.208230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1604680842":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6800/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/538363975":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/952052990":"2024-04-27T18:17:05.083192+0000","172.21.15.84:0/1266942705":"2024-04-27T18:16:37.859669+0000","172.21.15.84:6800/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3970542149":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3607125049":"2024-04-27T18:17:05.083192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:25:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.732+0000 7f1fc858e700 1 -- 172.21.15.84:0/1322951653 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1fa80a3cd0 msgr2=0x7f1fa80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.732+0000 7f1fc858e700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1fa80a3cd0 0x7f1fa80a6190 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1fb8009fd0 tx=0x7f1fb800b040 comp rx=0 tx=0).stop 2024-04-26T18:25:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.732+0000 7f1fc858e700 1 -- 172.21.15.84:0/1322951653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fc0102fc0 msgr2=0x7f1fc019b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.732+0000 7f1fc858e700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fc0102fc0 0x7f1fc019b030 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1fb400b500 tx=0x7f1fb400b8c0 comp rx=0 tx=0).stop 2024-04-26T18:25:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.732+0000 7f1fc858e700 1 -- 172.21.15.84:0/1322951653 shutdown_connections 2024-04-26T18:25:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.732+0000 7f1fc858e700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1fa80a3cd0 0x7f1fa80a6190 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.732+0000 7f1fc858e700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1fc0107660 0x7f1fc01a5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.732+0000 7f1fc858e700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1fc0102610 0x7f1fc019aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.732+0000 7f1fc858e700 1 --2- 172.21.15.84:0/1322951653 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1fc0102fc0 0x7f1fc019b030 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.732+0000 7f1fc858e700 1 -- 172.21.15.84:0/1322951653 >> 172.21.15.84:0/1322951653 conn(0x7f1fc0078370 msgr2=0x7f1fc01007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:32.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.732+0000 7f1fc858e700 1 -- 172.21.15.84:0/1322951653 shutdown_connections 2024-04-26T18:25:32.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:32.732+0000 7f1fc858e700 1 -- 172.21.15.84:0/1322951653 wait complete. 2024-04-26T18:25:33.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:33 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1322951653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:25:33.418 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T18:25:33.418 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:25:33.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:33 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1322951653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:25:33.657 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:33.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:33 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1322951653' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:25:34.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:34 smithi084 ceph-mon[29141]: pgmap v335: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:25:34.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:34 smithi103 ceph-mon[32727]: pgmap v335: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:25:34.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:34 smithi174 ceph-mon[32705]: pgmap v335: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:25:35.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.354+0000 7fa0da757700 1 -- 172.21.15.84:0/679700602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d41094a0 msgr2=0x7fa0d4109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:35.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.354+0000 7fa0da757700 1 --2- 172.21.15.84:0/679700602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d41094a0 0x7fa0d4109880 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0c4009990 tx=0x7fa0c4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:25:35.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.354+0000 7fa0da757700 1 -- 172.21.15.84:0/679700602 shutdown_connections 2024-04-26T18:25:35.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.354+0000 7fa0da757700 1 --2- 172.21.15.84:0/679700602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d41094a0 0x7fa0d4109880 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:35.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.354+0000 7fa0da757700 1 --2- 172.21.15.84:0/679700602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d406bad0 0x7fa0d41102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:35.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.354+0000 7fa0da757700 1 --2- 172.21.15.84:0/679700602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d406b130 0x7fa0d406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.354+0000 7fa0da757700 1 -- 172.21.15.84:0/679700602 >> 172.21.15.84:0/679700602 conn(0x7fa0d40792d0 msgr2=0x7fa0d40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.354+0000 7fa0da757700 1 -- 172.21.15.84:0/679700602 shutdown_connections 2024-04-26T18:25:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.354+0000 7fa0da757700 1 -- 172.21.15.84:0/679700602 wait complete. 2024-04-26T18:25:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.354+0000 7fa0da757700 1 Processor -- start 2024-04-26T18:25:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0da757700 1 -- start start 2024-04-26T18:25:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0da757700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d406b130 0x7fa0d419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0da757700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d406bad0 0x7fa0d419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0da757700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d41094a0 0x7fa0d4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0da757700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d4113630 con 0x7fa0d41094a0 2024-04-26T18:25:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0da757700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41134b0 con 0x7fa0d406b130 2024-04-26T18:25:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0da757700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa0d41137b0 con 0x7fa0d406bad0 2024-04-26T18:25:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0d8cf4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d41094a0 0x7fa0d4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0d8cf4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d41094a0 0x7fa0d4199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37076/0 (socket says 172.21.15.84:37076) 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0d8cf4700 1 -- 172.21.15.84:0/1845026480 learned_addr learned my addr 172.21.15.84:0/1845026480 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0d3fff700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d406b130 0x7fa0d419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0d37fe700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d406bad0 0x7fa0d419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0d8cf4700 1 -- 172.21.15.84:0/1845026480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d406bad0 msgr2=0x7fa0d419fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0d8cf4700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d406bad0 0x7fa0d419fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0d8cf4700 1 -- 172.21.15.84:0/1845026480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d406b130 msgr2=0x7fa0d419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0d8cf4700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d406b130 0x7fa0d419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0d8cf4700 1 -- 172.21.15.84:0/1845026480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0d419a0f0 con 0x7fa0d41094a0 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0d37fe700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d406bad0 0x7fa0d419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.355+0000 7fa0d8cf4700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d41094a0 0x7fa0d4199990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00b970 tx=0x7fa0cc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.356+0000 7fa0d17fa700 1 -- 172.21.15.84:0/1845026480 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc019070 con 0x7fa0d41094a0 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.356+0000 7fa0da757700 1 -- 172.21.15.84:0/1845026480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0d419a3e0 con 0x7fa0d41094a0 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.356+0000 7fa0d17fa700 1 -- 172.21.15.84:0/1845026480 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0cc00cde0 con 0x7fa0d41094a0 2024-04-26T18:25:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.356+0000 7fa0d17fa700 1 -- 172.21.15.84:0/1845026480 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0cc007370 con 0x7fa0d41094a0 2024-04-26T18:25:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.356+0000 7fa0da757700 1 -- 172.21.15.84:0/1845026480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0d41a66a0 con 0x7fa0d41094a0 2024-04-26T18:25:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.357+0000 7fa0da757700 1 -- 172.21.15.84:0/1845026480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0d406a110 con 0x7fa0d41094a0 2024-04-26T18:25:35.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.358+0000 7fa0d17fa700 1 -- 172.21.15.84:0/1845026480 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa0cc01c070 con 0x7fa0d41094a0 2024-04-26T18:25:35.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.358+0000 7fa0d17fa700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa0bc0a3cd0 0x7fa0bc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:35.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.358+0000 7fa0d3fff700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa0bc0a3cd0 0x7fa0bc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:35.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.358+0000 7fa0d17fa700 1 -- 172.21.15.84:0/1845026480 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fa0cc0d00c0 con 0x7fa0d41094a0 2024-04-26T18:25:35.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.359+0000 7fa0d3fff700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa0bc0a3cd0 0x7fa0bc0a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400b5c0 tx=0x7fa0c4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:35.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.362+0000 7fa0d17fa700 1 -- 172.21.15.84:0/1845026480 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0cc023070 con 0x7fa0d41094a0 2024-04-26T18:25:35.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.565+0000 7fa0da757700 1 -- 172.21.15.84:0/1845026480 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa0d419aec0 con 0x7fa0d41094a0 2024-04-26T18:25:35.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.567+0000 7fa0d17fa700 1 -- 172.21.15.84:0/1845026480 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+22680 (secure 0 0 0) 0x7fa0cc09ca40 con 0x7fa0d41094a0 2024-04-26T18:25:35.567 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:25:35.567 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":91,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:16:06.267694+0000","modified":"2024-04-26T18:25:06.286417+0000","last_up_change":"2024-04-26T18:23:19.906511+0000","last_in_change":"2024-04-26T18:22:59.541869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:19:31.174024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:24:16.533128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:24:19.742656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1076912-986e-4825-b1d1-5fa9d69bc07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6803","nonce":140050011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6805","nonce":140050011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6809","nonce":140050011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6807","nonce":140050011}]},"public_addr":"172.21.15.84:6803/140050011","cluster_addr":"172.21.15.84:6805/140050011","heartbeat_back_addr":"172.21.15.84:6809/140050011","heartbeat_front_addr":"172.21.15.84:6807/140050011","state":["exists","up"]},{"osd":1,"uuid":"fa93293d-9058-4804-ba4b-7e1e674b0fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6801","nonce":2512857657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6803","nonce":2512857657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6807","nonce":2512857657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6805","nonce":2512857657}]},"public_addr":"172.21.15.103:6801/2512857657","cluster_addr":"172.21.15.103:6803/2512857657","heartbeat_back_addr":"172.21.15.103:6807/2512857657","heartbeat_front_addr":"172.21.15.103:6805/2512857657","state":["exists","up"]},{"osd":2,"uuid":"bf78a44f-80a2-4cd7-974f-5abdf771e650","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6801","nonce":76340924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6803","nonce":76340924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6807","nonce":76340924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6805","nonce":76340924}]},"public_addr":"172.21.15.174:6801/76340924","cluster_addr":"172.21.15.174:6803/76340924","heartbeat_back_addr":"172.21.15.174:6807/76340924","heartbeat_front_addr":"172.21.15.174:6805/76340924","state":["exists","up"]},{"osd":3,"uuid":"55fd0035-bc76-4235-b8b5-5cf5ef770139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6811","nonce":2352789256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6813","nonce":2352789256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6817","nonce":2352789256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6815","nonce":2352789256}]},"public_addr":"172.21.15.84:6811/2352789256","cluster_addr":"172.21.15.84:6813/2352789256","heartbeat_back_addr":"172.21.15.84:6817/2352789256","heartbeat_front_addr":"172.21.15.84:6815/2352789256","state":["exists","up"]},{"osd":4,"uuid":"c278a0f3-e436-4d1c-a009-f9165ebf3c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6809","nonce":1619258252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6811","nonce":1619258252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6815","nonce":1619258252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6813","nonce":1619258252}]},"public_addr":"172.21.15.103:6809/1619258252","cluster_addr":"172.21.15.103:6811/1619258252","heartbeat_back_addr":"172.21.15.103:6815/1619258252","heartbeat_front_addr":"172.21.15.103:6813/1619258252","state":["exists","up"]},{"osd":5,"uuid":"6505be9b-c1b0-4d8b-944e-9920f90cb88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6809","nonce":3168291768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6811","nonce":3168291768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6815","nonce":3168291768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6813","nonce":3168291768}]},"public_addr":"172.21.15.174:6809/3168291768","cluster_addr":"172.21.15.174:6811/3168291768","heartbeat_back_addr":"172.21.15.174:6815/3168291768","heartbeat_front_addr":"172.21.15.174:6813/3168291768","state":["exists","up"]},{"osd":6,"uuid":"a037b66a-7488-469d-b73a-14cc0d35cb5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6819","nonce":886635297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6821","nonce":886635297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6825","nonce":886635297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6823","nonce":886635297}]},"public_addr":"172.21.15.84:6819/886635297","cluster_addr":"172.21.15.84:6821/886635297","heartbeat_back_addr":"172.21.15.84:6825/886635297","heartbeat_front_addr":"172.21.15.84:6823/886635297","state":["exists","up"]},{"osd":7,"uuid":"c425a13f-8019-4484-86a0-a34fa4d8de81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6817","nonce":1738278079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6819","nonce":1738278079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6823","nonce":1738278079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6821","nonce":1738278079}]},"public_addr":"172.21.15.103:6817/1738278079","cluster_addr":"172.21.15.103:6819/1738278079","heartbeat_back_addr":"172.21.15.103:6823/1738278079","heartbeat_front_addr":"172.21.15.103:6821/1738278079","state":["exists","up"]},{"osd":8,"uuid":"687e6c08-bfa4-440c-b3bc-90a5748989d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6817","nonce":38410548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6819","nonce":38410548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6823","nonce":38410548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6821","nonce":38410548}]},"public_addr":"172.21.15.174:6817/38410548","cluster_addr":"172.21.15.174:6819/38410548","heartbeat_back_addr":"172.21.15.174:6823/38410548","heartbeat_front_addr":"172.21.15.174:6821/38410548","state":["exists","up"]},{"osd":9,"uuid":"059eb22d-6dfb-4dd0-b192-84ed42ee1fbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6827","nonce":2926393927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6829","nonce":2926393927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6833","nonce":2926393927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6831","nonce":2926393927}]},"public_addr":"172.21.15.84:6827/2926393927","cluster_addr":"172.21.15.84:6829/2926393927","heartbeat_back_addr":"172.21.15.84:6833/2926393927","heartbeat_front_addr":"172.21.15.84:6831/2926393927","state":["exists","up"]},{"osd":10,"uuid":"da31e305-5220-4a87-a131-005b7c7f76b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6825","nonce":2705726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6827","nonce":2705726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6831","nonce":2705726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6829","nonce":2705726915}]},"public_addr":"172.21.15.103:6825/2705726915","cluster_addr":"172.21.15.103:6827/2705726915","heartbeat_back_addr":"172.21.15.103:6831/2705726915","heartbeat_front_addr":"172.21.15.103:6829/2705726915","state":["exists","up"]},{"osd":11,"uuid":"3fde6c8f-ee02-41ee-a34f-73072b4761e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6825","nonce":1668672128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6827","nonce":1668672128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6831","nonce":1668672128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6829","nonce":1668672128}]},"public_addr":"172.21.15.174:6825/1668672128","cluster_addr":"172.21.15.174:6827/1668672128","heartbeat_back_addr":"172.21.15.174:6831/1668672128","heartbeat_front_addr":"172.21.15.174:6829/1668672128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:39.099188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:03.327608+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:27.714108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:52.506015+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:17.631670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:42.702489+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:07.606697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:34.123214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:59.534658+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:25.357720+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:51.543106+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:23:18.208230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1604680842":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6800/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/538363975":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/952052990":"2024-04-27T18:17:05.083192+0000","172.21.15.84:0/1266942705":"2024-04-27T18:16:37.859669+0000","172.21.15.84:6800/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3970542149":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3607125049":"2024-04-27T18:17:05.083192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:25:35.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.569+0000 7fa0da757700 1 -- 172.21.15.84:0/1845026480 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa0bc0a3cd0 msgr2=0x7fa0bc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:35.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.569+0000 7fa0da757700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa0bc0a3cd0 0x7fa0bc0a6190 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa0c400b5c0 tx=0x7fa0c4005960 comp rx=0 tx=0).stop 2024-04-26T18:25:35.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.569+0000 7fa0da757700 1 -- 172.21.15.84:0/1845026480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d41094a0 msgr2=0x7fa0d4199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:35.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.569+0000 7fa0da757700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d41094a0 0x7fa0d4199990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc00b970 tx=0x7fa0cc00bc80 comp rx=0 tx=0).stop 2024-04-26T18:25:35.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.570+0000 7fa0da757700 1 -- 172.21.15.84:0/1845026480 shutdown_connections 2024-04-26T18:25:35.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.570+0000 7fa0da757700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa0bc0a3cd0 0x7fa0bc0a6190 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:35.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.570+0000 7fa0da757700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa0d41094a0 0x7fa0d4199990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:35.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.570+0000 7fa0da757700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa0d406b130 0x7fa0d419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:35.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.570+0000 7fa0da757700 1 --2- 172.21.15.84:0/1845026480 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa0d406bad0 0x7fa0d419fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:35.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.570+0000 7fa0da757700 1 -- 172.21.15.84:0/1845026480 >> 172.21.15.84:0/1845026480 conn(0x7fa0d40792d0 msgr2=0x7fa0d4100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:35.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.570+0000 7fa0da757700 1 -- 172.21.15.84:0/1845026480 shutdown_connections 2024-04-26T18:25:35.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:35.570+0000 7fa0da757700 1 -- 172.21.15.84:0/1845026480 wait complete. 2024-04-26T18:25:36.299 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T18:25:36.300 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T18:25:36.300 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T18:25:36.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T18:25:36.301 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T18:25:36.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T18:25:36.302 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-26T18:25:36.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-26T18:25:36.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-26T18:25:36.303 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-26T18:25:36.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-26T18:25:36.304 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-26T18:25:36.596 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:36 smithi084 ceph-mon[29141]: pgmap v336: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:25:36.597 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:36 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1845026480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:25:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:36 smithi103 ceph-mon[32727]: pgmap v336: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:25:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:36 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1845026480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:25:36.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:36 smithi174 ceph-mon[32705]: pgmap v336: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:25:36.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:36 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1845026480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:25:37.010 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:37.176 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:37.179 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:37.183 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:37.185 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:37.186 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:37.192 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:37.197 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:37.197 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:37.197 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:37.200 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:37.201 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:25:38.547 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:38 smithi084 ceph-mon[29141]: pgmap v337: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:38 smithi103 ceph-mon[32727]: pgmap v337: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:38.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:38 smithi174 ceph-mon[32705]: pgmap v337: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:40.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:40 smithi084 ceph-mon[29141]: pgmap v338: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:25:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:40 smithi103 ceph-mon[32727]: pgmap v338: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:25:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:40 smithi174 ceph-mon[32705]: pgmap v338: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:25:41.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.633+0000 7f74ea05b700 1 -- 172.21.15.84:0/2865324914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e4103e90 msgr2=0x7f74e4110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.633+0000 7f74ea05b700 1 --2- 172.21.15.84:0/2865324914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e4103e90 0x7f74e4110300 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f74dc00b0d0 tx=0x7f74dc00b3e0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.633+0000 7f74ea05b700 1 -- 172.21.15.84:0/2865324914 shutdown_connections 2024-04-26T18:25:41.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.633+0000 7f74ea05b700 1 --2- 172.21.15.84:0/2865324914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e4103e90 0x7f74e4110300 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.633+0000 7f74ea05b700 1 --2- 172.21.15.84:0/2865324914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74e41083b0 0x7f74e4108790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.633+0000 7f74ea05b700 1 --2- 172.21.15.84:0/2865324914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74e41034d0 0x7f74e4103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.633+0000 7f74ea05b700 1 -- 172.21.15.84:0/2865324914 >> 172.21.15.84:0/2865324914 conn(0x7f74e407a580 msgr2=0x7f74e407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.633+0000 7f74ea05b700 1 -- 172.21.15.84:0/2865324914 shutdown_connections 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.633+0000 7f74ea05b700 1 -- 172.21.15.84:0/2865324914 wait complete. 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.633+0000 7f74ea05b700 1 Processor -- start 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.634+0000 7f74ea05b700 1 -- start start 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.634+0000 7f74ea05b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74e41034d0 0x7f74e4075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.634+0000 7f74ea05b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e4103e90 0x7f74e4075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.634+0000 7f74ea05b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74e41083b0 0x7f74e40761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.634+0000 7f74ea05b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f74e4113640 con 0x7f74e4103e90 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.634+0000 7f74ea05b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f74e41134c0 con 0x7f74e41083b0 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.634+0000 7f74ea05b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f74e41137c0 con 0x7f74e41034d0 2024-04-26T18:25:41.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.634+0000 7f74daffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e4103e90 0x7f74e4075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.634+0000 7f74daffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e4103e90 0x7f74e4075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50346/0 (socket says 172.21.15.84:50346) 2024-04-26T18:25:41.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.634+0000 7f74daffd700 1 -- 172.21.15.84:0/1126929271 learned_addr learned my addr 172.21.15.84:0/1126929271 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:41.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.634+0000 7f74e37fe700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74e41034d0 0x7f74e4075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.634+0000 7f74e3fff700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74e41083b0 0x7f74e40761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.635+0000 7f74e37fe700 1 -- 172.21.15.84:0/1126929271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74e41083b0 msgr2=0x7f74e40761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.635+0000 7f74e37fe700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f74e41083b0 0x7f74e40761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.635+0000 7f74e37fe700 1 -- 172.21.15.84:0/1126929271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e4103e90 msgr2=0x7f74e4075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.635+0000 7f74e37fe700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f74e4103e90 0x7f74e4075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.635+0000 7f74e37fe700 1 -- 172.21.15.84:0/1126929271 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74e41a7fb0 con 0x7f74e41034d0 2024-04-26T18:25:41.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.635+0000 7f74e37fe700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74e41034d0 0x7f74e4075720 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f74d400e9f0 tx=0x7f74d400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:41.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.635+0000 7f74e17fa700 1 -- 172.21.15.84:0/1126929271 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4004220 con 0x7f74e41034d0 2024-04-26T18:25:41.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.635+0000 7f74e17fa700 1 -- 172.21.15.84:0/1126929271 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74d40043c0 con 0x7f74e41034d0 2024-04-26T18:25:41.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.635+0000 7f74ea05b700 1 -- 172.21.15.84:0/1126929271 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74e41a82a0 con 0x7f74e41034d0 2024-04-26T18:25:41.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.635+0000 7f74e17fa700 1 -- 172.21.15.84:0/1126929271 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74d4005020 con 0x7f74e41034d0 2024-04-26T18:25:41.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.635+0000 7f74ea05b700 1 -- 172.21.15.84:0/1126929271 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74e41a8790 con 0x7f74e41034d0 2024-04-26T18:25:41.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.636+0000 7f74ea05b700 1 -- 172.21.15.84:0/1126929271 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f74c4001030 con 0x7f74e41034d0 2024-04-26T18:25:41.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.637+0000 7f74e17fa700 1 -- 172.21.15.84:0/1126929271 <== mon.1 v2:172.21.15.174:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f74d4004830 con 0x7f74e41034d0 2024-04-26T18:25:41.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.638+0000 7f74e17fa700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f74d00a3cd0 0x7f74d00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.638+0000 7f74daffd700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f74d00a3cd0 0x7f74d00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.638+0000 7f74e17fa700 1 -- 172.21.15.84:0/1126929271 <== mon.1 v2:172.21.15.174:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f74d40d07f0 con 0x7f74e41034d0 2024-04-26T18:25:41.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.638+0000 7f74e17fa700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297] conn(0x7f74d00ad890 0x7f74d00afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.638+0000 7f74e17fa700 1 -- 172.21.15.84:0/1126929271 --> [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f74d00b0400 con 0x7f74d00ad890 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.638+0000 7f74e17fa700 1 -- 172.21.15.84:0/1126929271 <== mon.1 v2:172.21.15.174:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f74d40d0ba0 con 0x7f74e41034d0 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.638+0000 7f74daffd700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f74d00a3cd0 0x7f74d00a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f74e407aa80 tx=0x7f74cc00b500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.638+0000 7f74e3fff700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297] conn(0x7f74d00ad890 0x7f74d00afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.639+0000 7f74e3fff700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297] conn(0x7f74d00ad890 0x7f74d00afd30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.639+0000 7f74e17fa700 1 -- 172.21.15.84:0/1126929271 <== osd.6 v2:172.21.15.84:6818/886635297 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f74d00b0400 con 0x7f74d00ad890 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.656+0000 7f74ea05b700 1 -- 172.21.15.84:0/1126929271 --> [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f74c4002e60 con 0x7f74d00ad890 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.656+0000 7f74e17fa700 1 -- 172.21.15.84:0/1126929271 <== osd.6 v2:172.21.15.84:6818/886635297 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f74c4002e60 con 0x7f74d00ad890 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.656+0000 7f74ea05b700 1 -- 172.21.15.84:0/1126929271 >> [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297] conn(0x7f74d00ad890 msgr2=0x7f74d00afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.656+0000 7f74ea05b700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297] conn(0x7f74d00ad890 0x7f74d00afd30 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.657+0000 7f74ea05b700 1 -- 172.21.15.84:0/1126929271 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f74d00a3cd0 msgr2=0x7f74d00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.657+0000 7f74ea05b700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f74d00a3cd0 0x7f74d00a6190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f74e407aa80 tx=0x7f74cc00b500 comp rx=0 tx=0).stop 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.657+0000 7f74ea05b700 1 -- 172.21.15.84:0/1126929271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74e41034d0 msgr2=0x7f74e4075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:41.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.657+0000 7f74ea05b700 1 --2- 172.21.15.84:0/1126929271 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f74e41034d0 0x7f74e4075720 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f74d400e9f0 tx=0x7f74d400ed00 comp rx=0 tx=0).stop 2024-04-26T18:25:41.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.657+0000 7f74e3fff700 1 -- 172.21.15.84:0/1126929271 reap_dead start 2024-04-26T18:25:41.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.657+0000 7f74ea05b700 1 -- 172.21.15.84:0/1126929271 shutdown_connections 2024-04-26T18:25:41.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.657+0000 7f74ea05b700 1 -- 172.21.15.84:0/1126929271 >> 172.21.15.84:0/1126929271 conn(0x7f74e407a580 msgr2=0x7f74e4101520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:41.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.657+0000 7f74ea05b700 1 -- 172.21.15.84:0/1126929271 shutdown_connections 2024-04-26T18:25:41.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:41.657+0000 7f74ea05b700 1 -- 172.21.15.84:0/1126929271 wait complete. 2024-04-26T18:25:42.549 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:42 smithi084 ceph-mon[29141]: pgmap v339: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:42.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:42 smithi103 ceph-mon[32727]: pgmap v339: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:42.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:42 smithi174 ceph-mon[32705]: pgmap v339: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:43.623 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:43 smithi084 ceph-mon[29141]: pgmap v340: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T18:25:43.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:43 smithi103 ceph-mon[32727]: pgmap v340: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T18:25:43.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:43 smithi174 ceph-mon[32705]: pgmap v340: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 8 op/s 2024-04-26T18:25:46.568 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:46 smithi084 ceph-mon[29141]: pgmap v341: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-26T18:25:46.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:46 smithi103 ceph-mon[32727]: pgmap v341: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-26T18:25:46.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:46 smithi174 ceph-mon[32705]: pgmap v341: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 10 op/s 2024-04-26T18:25:48.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:48 smithi084 ceph-mon[29141]: pgmap v342: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:48 smithi103 ceph-mon[32727]: pgmap v342: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:48 smithi174 ceph-mon[32705]: pgmap v342: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:50.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:50 smithi084 ceph-mon[29141]: pgmap v343: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:25:50.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 -- 172.21.15.84:0/2645908484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe93c103ea0 msgr2=0x7fe93c1102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:50.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 --2- 172.21.15.84:0/2645908484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe93c103ea0 0x7fe93c1102c0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe93400b0a0 tx=0x7fe93400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:25:50.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 -- 172.21.15.84:0/2645908484 shutdown_connections 2024-04-26T18:25:50.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 --2- 172.21.15.84:0/2645908484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe93c103ea0 0x7fe93c1102c0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:50.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 --2- 172.21.15.84:0/2645908484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe93c109480 0x7fe93c109860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:50.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 --2- 172.21.15.84:0/2645908484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe93c1034e0 0x7fe93c103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:50.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 -- 172.21.15.84:0/2645908484 >> 172.21.15.84:0/2645908484 conn(0x7fe93c07a5e0 msgr2=0x7fe93c07a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:50.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 -- 172.21.15.84:0/2645908484 shutdown_connections 2024-04-26T18:25:50.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 -- 172.21.15.84:0/2645908484 wait complete. 2024-04-26T18:25:50.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 Processor -- start 2024-04-26T18:25:50.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 -- start start 2024-04-26T18:25:50.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe93c1034e0 0x7fe93c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:50.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe93c103ea0 0x7fe93c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:50.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe93c109480 0x7fe93c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:50.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c1135c0 con 0x7fe93c103ea0 2024-04-26T18:25:50.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c113440 con 0x7fe93c1034e0 2024-04-26T18:25:50.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.595+0000 7fe942e49700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe93c113740 con 0x7fe93c109480 2024-04-26T18:25:50.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe93bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe93c103ea0 0x7fe93c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:50.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe93bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe93c103ea0 0x7fe93c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48282/0 (socket says 172.21.15.84:48282) 2024-04-26T18:25:50.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe93bfff700 1 -- 172.21.15.84:0/4284957232 learned_addr learned my addr 172.21.15.84:0/4284957232 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:50.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe940be5700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe93c1034e0 0x7fe93c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:50.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe9413e6700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe93c109480 0x7fe93c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:50.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe93bfff700 1 -- 172.21.15.84:0/4284957232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe93c109480 msgr2=0x7fe93c199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:50.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe93bfff700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe93c109480 0x7fe93c199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:50.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe93bfff700 1 -- 172.21.15.84:0/4284957232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe93c1034e0 msgr2=0x7fe93c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:50.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe93bfff700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe93c1034e0 0x7fe93c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:50.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe93bfff700 1 -- 172.21.15.84:0/4284957232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93c19a0e0 con 0x7fe93c103ea0 2024-04-26T18:25:50.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe93bfff700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe93c103ea0 0x7fe93c19fd00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe92800c970 tx=0x7fe92800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:50.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe939ffb700 1 -- 172.21.15.84:0/4284957232 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9280078d0 con 0x7fe93c103ea0 2024-04-26T18:25:50.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe939ffb700 1 -- 172.21.15.84:0/4284957232 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe928018480 con 0x7fe93c103ea0 2024-04-26T18:25:50.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe942e49700 1 -- 172.21.15.84:0/4284957232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe93c19a3d0 con 0x7fe93c103ea0 2024-04-26T18:25:50.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe939ffb700 1 -- 172.21.15.84:0/4284957232 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92800f6b0 con 0x7fe93c103ea0 2024-04-26T18:25:50.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.596+0000 7fe942e49700 1 -- 172.21.15.84:0/4284957232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe93c1a6690 con 0x7fe93c103ea0 2024-04-26T18:25:50.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.597+0000 7fe942e49700 1 -- 172.21.15.84:0/4284957232 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe93c063ab0 con 0x7fe93c103ea0 2024-04-26T18:25:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.599+0000 7fe939ffb700 1 -- 172.21.15.84:0/4284957232 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe928018930 con 0x7fe93c103ea0 2024-04-26T18:25:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.599+0000 7fe939ffb700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe92c0a3cd0 0x7fe92c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.599+0000 7fe940be5700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe92c0a3cd0 0x7fe92c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.599+0000 7fe939ffb700 1 -- 172.21.15.84:0/4284957232 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fe9280d0ad0 con 0x7fe93c103ea0 2024-04-26T18:25:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.599+0000 7fe939ffb700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657] conn(0x7fe92c0ad890 0x7fe92c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.599+0000 7fe939ffb700 1 -- 172.21.15.84:0/4284957232 --> [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe92c0b0400 con 0x7fe92c0ad890 2024-04-26T18:25:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.599+0000 7fe939ffb700 1 -- 172.21.15.84:0/4284957232 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe9280d0f20 con 0x7fe93c103ea0 2024-04-26T18:25:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.600+0000 7fe940be5700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe92c0a3cd0 0x7fe92c0a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe93c19aac0 tx=0x7fe93000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.600+0000 7fe9413e6700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657] conn(0x7fe92c0ad890 0x7fe92c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.600+0000 7fe9413e6700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657] conn(0x7fe92c0ad890 0x7fe92c0afd30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:50.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.601+0000 7fe939ffb700 1 -- 172.21.15.84:0/4284957232 <== osd.1 v2:172.21.15.103:6800/2512857657 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe92c0b0400 con 0x7fe92c0ad890 2024-04-26T18:25:50.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.618+0000 7fe942e49700 1 -- 172.21.15.84:0/4284957232 --> [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe93c063eb0 con 0x7fe92c0ad890 2024-04-26T18:25:50.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.618+0000 7fe939ffb700 1 -- 172.21.15.84:0/4284957232 <== osd.1 v2:172.21.15.103:6800/2512857657 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe93c063eb0 con 0x7fe92c0ad890 2024-04-26T18:25:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.619+0000 7fe942e49700 1 -- 172.21.15.84:0/4284957232 >> [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657] conn(0x7fe92c0ad890 msgr2=0x7fe92c0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.619+0000 7fe942e49700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657] conn(0x7fe92c0ad890 0x7fe92c0afd30 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.619+0000 7fe942e49700 1 -- 172.21.15.84:0/4284957232 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe92c0a3cd0 msgr2=0x7fe92c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.619+0000 7fe942e49700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe92c0a3cd0 0x7fe92c0a6190 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe93c19aac0 tx=0x7fe93000b410 comp rx=0 tx=0).stop 2024-04-26T18:25:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.619+0000 7fe942e49700 1 -- 172.21.15.84:0/4284957232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe93c103ea0 msgr2=0x7fe93c19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.619+0000 7fe942e49700 1 --2- 172.21.15.84:0/4284957232 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe93c103ea0 0x7fe93c19fd00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe92800c970 tx=0x7fe92800cc80 comp rx=0 tx=0).stop 2024-04-26T18:25:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.619+0000 7fe9413e6700 1 -- 172.21.15.84:0/4284957232 reap_dead start 2024-04-26T18:25:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.619+0000 7fe942e49700 1 -- 172.21.15.84:0/4284957232 shutdown_connections 2024-04-26T18:25:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.619+0000 7fe942e49700 1 -- 172.21.15.84:0/4284957232 >> 172.21.15.84:0/4284957232 conn(0x7fe93c07a5e0 msgr2=0x7fe93c101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.619+0000 7fe942e49700 1 -- 172.21.15.84:0/4284957232 shutdown_connections 2024-04-26T18:25:50.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.619+0000 7fe942e49700 1 -- 172.21.15.84:0/4284957232 wait complete. 2024-04-26T18:25:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:50 smithi103 ceph-mon[32727]: pgmap v343: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:25:50.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:50 smithi174 ceph-mon[32705]: pgmap v343: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:25:51.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.672+0000 7ff11f322700 1 -- 172.21.15.84:0/3678718951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1181094f0 msgr2=0x7ff1181098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.672+0000 7ff11f322700 1 --2- 172.21.15.84:0/3678718951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1181094f0 0x7ff1181098d0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ff108009990 tx=0x7ff108009ca0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.672+0000 7ff11f322700 1 -- 172.21.15.84:0/3678718951 shutdown_connections 2024-04-26T18:25:51.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.672+0000 7ff11f322700 1 --2- 172.21.15.84:0/3678718951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff1181094f0 0x7ff1181098d0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.672+0000 7ff11f322700 1 --2- 172.21.15.84:0/3678718951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff118103e90 0x7ff1181102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.672+0000 7ff11f322700 1 --2- 172.21.15.84:0/3678718951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1181034d0 0x7ff118103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.672+0000 7ff11f322700 1 -- 172.21.15.84:0/3678718951 >> 172.21.15.84:0/3678718951 conn(0x7ff11807a500 msgr2=0x7ff11807a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:51.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.672+0000 7ff11f322700 1 -- 172.21.15.84:0/3678718951 shutdown_connections 2024-04-26T18:25:51.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.672+0000 7ff11f322700 1 -- 172.21.15.84:0/3678718951 wait complete. 2024-04-26T18:25:51.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.672+0000 7ff11f322700 1 Processor -- start 2024-04-26T18:25:51.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11f322700 1 -- start start 2024-04-26T18:25:51.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11f322700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1181034d0 0x7ff11819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11f322700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff118103e90 0x7ff11819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11f322700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1181094f0 0x7ff118199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11f322700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff1181136f0 con 0x7ff118103e90 2024-04-26T18:25:51.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11f322700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff118113570 con 0x7ff1181034d0 2024-04-26T18:25:51.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11f322700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff118113870 con 0x7ff1181094f0 2024-04-26T18:25:51.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11c8bd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff118103e90 0x7ff11819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11c8bd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff118103e90 0x7ff11819fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48298/0 (socket says 172.21.15.84:48298) 2024-04-26T18:25:51.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11c8bd700 1 -- 172.21.15.84:0/2556551534 learned_addr learned my addr 172.21.15.84:0/2556551534 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:51.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11d8bf700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1181094f0 0x7ff118199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11d0be700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1181034d0 0x7ff11819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11c8bd700 1 -- 172.21.15.84:0/2556551534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1181094f0 msgr2=0x7ff118199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11c8bd700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1181094f0 0x7ff118199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11c8bd700 1 -- 172.21.15.84:0/2556551534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1181034d0 msgr2=0x7ff11819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11c8bd700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff1181034d0 0x7ff11819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11c8bd700 1 -- 172.21.15.84:0/2556551534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff11819a1b0 con 0x7ff118103e90 2024-04-26T18:25:51.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11d8bf700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff1181094f0 0x7ff118199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:51.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11c8bd700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff118103e90 0x7ff11819fdd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff11400b530 tx=0x7ff11400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:51.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff10e7fc700 1 -- 172.21.15.84:0/2556551534 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114015020 con 0x7ff118103e90 2024-04-26T18:25:51.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff11f322700 1 -- 172.21.15.84:0/2556551534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff11819a4a0 con 0x7ff118103e90 2024-04-26T18:25:51.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff10e7fc700 1 -- 172.21.15.84:0/2556551534 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff114011ce0 con 0x7ff118103e90 2024-04-26T18:25:51.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.673+0000 7ff10e7fc700 1 -- 172.21.15.84:0/2556551534 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff114010d20 con 0x7ff118103e90 2024-04-26T18:25:51.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.674+0000 7ff11f322700 1 -- 172.21.15.84:0/2556551534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1181a6760 con 0x7ff118103e90 2024-04-26T18:25:51.677 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.674+0000 7ff11f322700 1 -- 172.21.15.84:0/2556551534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff118063bb0 con 0x7ff118103e90 2024-04-26T18:25:51.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.676+0000 7ff10e7fc700 1 -- 172.21.15.84:0/2556551534 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ff114018020 con 0x7ff118103e90 2024-04-26T18:25:51.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.676+0000 7ff10e7fc700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff1040a3cd0 0x7ff1040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.676+0000 7ff11d0be700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff1040a3cd0 0x7ff1040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.676+0000 7ff10e7fc700 1 -- 172.21.15.84:0/2556551534 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7ff1140d0420 con 0x7ff118103e90 2024-04-26T18:25:51.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.676+0000 7ff10e7fc700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011] conn(0x7ff1040ad890 0x7ff1040afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:51.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.676+0000 7ff10e7fc700 1 -- 172.21.15.84:0/2556551534 --> [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff114010ec0 con 0x7ff1040ad890 2024-04-26T18:25:51.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.676+0000 7ff10e7fc700 1 -- 172.21.15.84:0/2556551534 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7ff1140d0870 con 0x7ff118103e90 2024-04-26T18:25:51.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.677+0000 7ff11d8bf700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011] conn(0x7ff1040ad890 0x7ff1040afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:51.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.677+0000 7ff11d0be700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff1040a3cd0 0x7ff1040a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff108006010 tx=0x7ff10801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:51.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.677+0000 7ff11d8bf700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011] conn(0x7ff1040ad890 0x7ff1040afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:51.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.678+0000 7ff10e7fc700 1 -- 172.21.15.84:0/2556551534 <== osd.0 v2:172.21.15.84:6802/140050011 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff114010ec0 con 0x7ff1040ad890 2024-04-26T18:25:51.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.694+0000 7ff11f322700 1 -- 172.21.15.84:0/2556551534 --> [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff118063fb0 con 0x7ff1040ad890 2024-04-26T18:25:51.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.695+0000 7ff10e7fc700 1 -- 172.21.15.84:0/2556551534 <== osd.0 v2:172.21.15.84:6802/140050011 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff118063fb0 con 0x7ff1040ad890 2024-04-26T18:25:51.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.695+0000 7ff11f322700 1 -- 172.21.15.84:0/2556551534 >> [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011] conn(0x7ff1040ad890 msgr2=0x7ff1040afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.695+0000 7ff11f322700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011] conn(0x7ff1040ad890 0x7ff1040afd30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.695+0000 7ff11f322700 1 -- 172.21.15.84:0/2556551534 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff1040a3cd0 msgr2=0x7ff1040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.695+0000 7ff11f322700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff1040a3cd0 0x7ff1040a6190 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7ff108006010 tx=0x7ff10801a040 comp rx=0 tx=0).stop 2024-04-26T18:25:51.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.695+0000 7ff11f322700 1 -- 172.21.15.84:0/2556551534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff118103e90 msgr2=0x7ff11819fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:51.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.695+0000 7ff11f322700 1 --2- 172.21.15.84:0/2556551534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff118103e90 0x7ff11819fdd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff11400b530 tx=0x7ff11400b8f0 comp rx=0 tx=0).stop 2024-04-26T18:25:51.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.695+0000 7ff11d8bf700 1 -- 172.21.15.84:0/2556551534 reap_dead start 2024-04-26T18:25:51.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.695+0000 7ff11f322700 1 -- 172.21.15.84:0/2556551534 shutdown_connections 2024-04-26T18:25:51.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.695+0000 7ff11f322700 1 -- 172.21.15.84:0/2556551534 >> 172.21.15.84:0/2556551534 conn(0x7ff11807a500 msgr2=0x7ff1181023e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:51.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.695+0000 7ff11f322700 1 -- 172.21.15.84:0/2556551534 shutdown_connections 2024-04-26T18:25:51.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:51.695+0000 7ff11f322700 1 -- 172.21.15.84:0/2556551534 wait complete. 2024-04-26T18:25:52.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:52 smithi084 ceph-mon[29141]: pgmap v344: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-26T18:25:52.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:52 smithi103 ceph-mon[32727]: pgmap v344: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-26T18:25:52.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:52 smithi174 ceph-mon[32705]: pgmap v344: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-04-26T18:25:53.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.122+0000 7fe337066700 1 -- 172.21.15.84:0/3377535742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe330102610 msgr2=0x7fe3301029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.122+0000 7fe337066700 1 --2- 172.21.15.84:0/3377535742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe330102610 0x7fe3301029f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fe3240099c0 tx=0x7fe324009cd0 comp rx=0 tx=0).stop 2024-04-26T18:25:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.122+0000 7fe337066700 1 -- 172.21.15.84:0/3377535742 shutdown_connections 2024-04-26T18:25:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.122+0000 7fe337066700 1 --2- 172.21.15.84:0/3377535742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe330102610 0x7fe3301029f0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.122+0000 7fe337066700 1 --2- 172.21.15.84:0/3377535742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe330107660 0x7fe33010f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.122+0000 7fe337066700 1 --2- 172.21.15.84:0/3377535742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330102fc0 0x7fe330106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.122+0000 7fe337066700 1 -- 172.21.15.84:0/3377535742 >> 172.21.15.84:0/3377535742 conn(0x7fe330078370 msgr2=0x7fe330078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:53.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.122+0000 7fe337066700 1 -- 172.21.15.84:0/3377535742 shutdown_connections 2024-04-26T18:25:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.123+0000 7fe337066700 1 -- 172.21.15.84:0/3377535742 wait complete. 2024-04-26T18:25:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe337066700 1 Processor -- start 2024-04-26T18:25:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe337066700 1 -- start start 2024-04-26T18:25:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe337066700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe330102610 0x7fe33019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe337066700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330102fc0 0x7fe33019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe337066700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe330107660 0x7fe330199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe337066700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe3301124f0 con 0x7fe330102610 2024-04-26T18:25:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe337066700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe330112370 con 0x7fe330102fc0 2024-04-26T18:25:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe337066700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe330112670 con 0x7fe330107660 2024-04-26T18:25:53.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe334e02700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe330102610 0x7fe33019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe334e02700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe330102610 0x7fe33019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48336/0 (socket says 172.21.15.84:48336) 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe334e02700 1 -- 172.21.15.84:0/2151618276 learned_addr learned my addr 172.21.15.84:0/2151618276 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe335603700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe330107660 0x7fe330199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe32ffff700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330102fc0 0x7fe33019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe334e02700 1 -- 172.21.15.84:0/2151618276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe330107660 msgr2=0x7fe330199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe334e02700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe330107660 0x7fe330199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe334e02700 1 -- 172.21.15.84:0/2151618276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330102fc0 msgr2=0x7fe33019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe334e02700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe330102fc0 0x7fe33019fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe334e02700 1 -- 172.21.15.84:0/2151618276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33019a090 con 0x7fe330102610 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe334e02700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe330102610 0x7fe33019f770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe324006010 tx=0x7fe3240045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.124+0000 7fe32dffb700 1 -- 172.21.15.84:0/2151618276 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324026070 con 0x7fe330102610 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.125+0000 7fe32dffb700 1 -- 172.21.15.84:0/2151618276 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe32400f460 con 0x7fe330102610 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.125+0000 7fe337066700 1 -- 172.21.15.84:0/2151618276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33019a320 con 0x7fe330102610 2024-04-26T18:25:53.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.125+0000 7fe32dffb700 1 -- 172.21.15.84:0/2151618276 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe324009e60 con 0x7fe330102610 2024-04-26T18:25:53.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.125+0000 7fe337066700 1 -- 172.21.15.84:0/2151618276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3301a6640 con 0x7fe330102610 2024-04-26T18:25:53.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.126+0000 7fe337066700 1 -- 172.21.15.84:0/2151618276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe330063bb0 con 0x7fe330102610 2024-04-26T18:25:53.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.127+0000 7fe32dffb700 1 -- 172.21.15.84:0/2151618276 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fe324029070 con 0x7fe330102610 2024-04-26T18:25:53.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.127+0000 7fe32dffb700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe3200a3cd0 0x7fe3200a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:53.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.128+0000 7fe32ffff700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe3200a3cd0 0x7fe3200a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:53.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.128+0000 7fe32dffb700 1 -- 172.21.15.84:0/2151618276 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fe3240d12d0 con 0x7fe330102610 2024-04-26T18:25:53.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.128+0000 7fe32dffb700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924] conn(0x7fe3200ad7c0 0x7fe3200afc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:53.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.128+0000 7fe32dffb700 1 -- 172.21.15.84:0/2151618276 --> [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe324023ec0 con 0x7fe3200ad7c0 2024-04-26T18:25:53.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.128+0000 7fe32dffb700 1 -- 172.21.15.84:0/2151618276 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fe3240d1720 con 0x7fe330102610 2024-04-26T18:25:53.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.128+0000 7fe32ffff700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe3200a3cd0 0x7fe3200a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005950 tx=0x7fe31c0058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:53.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.128+0000 7fe335603700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924] conn(0x7fe3200ad7c0 0x7fe3200afc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:53.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.128+0000 7fe335603700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924] conn(0x7fe3200ad7c0 0x7fe3200afc60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:53.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.129+0000 7fe32dffb700 1 -- 172.21.15.84:0/2151618276 <== osd.2 v2:172.21.15.174:6800/76340924 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe324023ec0 con 0x7fe3200ad7c0 2024-04-26T18:25:53.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.146+0000 7fe337066700 1 -- 172.21.15.84:0/2151618276 --> [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe330064030 con 0x7fe3200ad7c0 2024-04-26T18:25:53.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.146+0000 7fe32dffb700 1 -- 172.21.15.84:0/2151618276 <== osd.2 v2:172.21.15.174:6800/76340924 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fe330064030 con 0x7fe3200ad7c0 2024-04-26T18:25:53.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.147+0000 7fe337066700 1 -- 172.21.15.84:0/2151618276 >> [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924] conn(0x7fe3200ad7c0 msgr2=0x7fe3200afc60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:53.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.147+0000 7fe337066700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924] conn(0x7fe3200ad7c0 0x7fe3200afc60 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:53.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.147+0000 7fe337066700 1 -- 172.21.15.84:0/2151618276 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe3200a3cd0 msgr2=0x7fe3200a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:53.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.147+0000 7fe337066700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe3200a3cd0 0x7fe3200a6190 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe31c005950 tx=0x7fe31c0058e0 comp rx=0 tx=0).stop 2024-04-26T18:25:53.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.147+0000 7fe337066700 1 -- 172.21.15.84:0/2151618276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe330102610 msgr2=0x7fe33019f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:53.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.147+0000 7fe337066700 1 --2- 172.21.15.84:0/2151618276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe330102610 0x7fe33019f770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe324006010 tx=0x7fe3240045e0 comp rx=0 tx=0).stop 2024-04-26T18:25:53.147 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.147+0000 7fe335603700 1 -- 172.21.15.84:0/2151618276 reap_dead start 2024-04-26T18:25:53.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.147+0000 7fe337066700 1 -- 172.21.15.84:0/2151618276 shutdown_connections 2024-04-26T18:25:53.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.147+0000 7fe337066700 1 -- 172.21.15.84:0/2151618276 >> 172.21.15.84:0/2151618276 conn(0x7fe330078370 msgr2=0x7fe33006bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:53.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.147+0000 7fe337066700 1 -- 172.21.15.84:0/2151618276 shutdown_connections 2024-04-26T18:25:53.148 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.147+0000 7fe337066700 1 -- 172.21.15.84:0/2151618276 wait complete. 2024-04-26T18:25:54.188 INFO:teuthology.orchestra.run.smithi084.stdout:34359738457 2024-04-26T18:25:54.188 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T18:25:54.442 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:54 smithi084 ceph-mon[29141]: pgmap v345: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:54.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:54 smithi103 ceph-mon[32727]: pgmap v345: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:54 smithi174 ceph-mon[32705]: pgmap v345: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:25:54.966 INFO:teuthology.orchestra.run.smithi084.stdout:201863462970 2024-04-26T18:25:54.966 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-26T18:25:55.349 INFO:teuthology.orchestra.run.smithi084.stdout:77309411408 2024-04-26T18:25:55.349 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T18:25:55.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.632+0000 7f141fb35700 1 -- 172.21.15.84:0/2055636084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1418108390 msgr2=0x7f1418108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.632+0000 7f141fb35700 1 --2- 172.21.15.84:0/2055636084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1418108390 0x7f1418108770 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1414009990 tx=0x7f1414009ca0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.632+0000 7f141fb35700 1 -- 172.21.15.84:0/2055636084 shutdown_connections 2024-04-26T18:25:55.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.632+0000 7f141fb35700 1 --2- 172.21.15.84:0/2055636084 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1418108390 0x7f1418108770 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.632+0000 7f141fb35700 1 --2- 172.21.15.84:0/2055636084 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1418102ce0 0x7f141810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.632+0000 7f141fb35700 1 --2- 172.21.15.84:0/2055636084 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418102320 0x7f14181027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.632+0000 7f141fb35700 1 -- 172.21.15.84:0/2055636084 >> 172.21.15.84:0/2055636084 conn(0x7f14180792d0 msgr2=0x7f14180796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:55.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.632+0000 7f141fb35700 1 -- 172.21.15.84:0/2055636084 shutdown_connections 2024-04-26T18:25:55.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.632+0000 7f141fb35700 1 -- 172.21.15.84:0/2055636084 wait complete. 2024-04-26T18:25:55.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141fb35700 1 Processor -- start 2024-04-26T18:25:55.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141fb35700 1 -- start start 2024-04-26T18:25:55.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141fb35700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418102320 0x7f141819f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141fb35700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1418102ce0 0x7f141819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141fb35700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1418108390 0x7f1418199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141fb35700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f14181124e0 con 0x7f1418108390 2024-04-26T18:25:55.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141fb35700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1418112360 con 0x7f1418102ce0 2024-04-26T18:25:55.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141fb35700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1418112660 con 0x7f1418102320 2024-04-26T18:25:55.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141e0d2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1418108390 0x7f1418199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141d8d1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418102320 0x7f141819f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141e0d2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1418108390 0x7f1418199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48360/0 (socket says 172.21.15.84:48360) 2024-04-26T18:25:55.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141e0d2700 1 -- 172.21.15.84:0/2170039038 learned_addr learned my addr 172.21.15.84:0/2170039038 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:55.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141d0d0700 1 --2- 172.21.15.84:0/2170039038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1418102ce0 0x7f141819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141e0d2700 1 -- 172.21.15.84:0/2170039038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418102320 msgr2=0x7f141819f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141e0d2700 1 --2- 172.21.15.84:0/2170039038 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1418102320 0x7f141819f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141e0d2700 1 -- 172.21.15.84:0/2170039038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1418102ce0 msgr2=0x7f141819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141e0d2700 1 --2- 172.21.15.84:0/2170039038 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1418102ce0 0x7f141819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.633+0000 7f141e0d2700 1 -- 172.21.15.84:0/2170039038 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f141819a060 con 0x7f1418108390 2024-04-26T18:25:55.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.634+0000 7f141e0d2700 1 --2- 172.21.15.84:0/2170039038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1418108390 0x7f1418199900 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1410007d70 tx=0x7f141000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:55.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.634+0000 7f140effd700 1 -- 172.21.15.84:0/2170039038 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1410016020 con 0x7f1418108390 2024-04-26T18:25:55.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.634+0000 7f141fb35700 1 -- 172.21.15.84:0/2170039038 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f141819a350 con 0x7f1418108390 2024-04-26T18:25:55.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.634+0000 7f140effd700 1 -- 172.21.15.84:0/2170039038 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f141000ac90 con 0x7f1418108390 2024-04-26T18:25:55.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.635+0000 7f140effd700 1 -- 172.21.15.84:0/2170039038 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14100115e0 con 0x7f1418108390 2024-04-26T18:25:55.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.635+0000 7f141fb35700 1 -- 172.21.15.84:0/2170039038 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14181a65c0 con 0x7f1418108390 2024-04-26T18:25:55.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.636+0000 7f141fb35700 1 -- 172.21.15.84:0/2170039038 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1418106450 con 0x7f1418108390 2024-04-26T18:25:55.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.636+0000 7f140effd700 1 -- 172.21.15.84:0/2170039038 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1410019020 con 0x7f1418108390 2024-04-26T18:25:55.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.637+0000 7f140effd700 1 --2- 172.21.15.84:0/2170039038 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14040a3bb0 0x7f14040a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.637+0000 7f141d8d1700 1 --2- 172.21.15.84:0/2170039038 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14040a3bb0 0x7f14040a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.637+0000 7f140effd700 1 -- 172.21.15.84:0/2170039038 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f14100d83a0 con 0x7f1418108390 2024-04-26T18:25:55.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.637+0000 7f140effd700 1 --2- 172.21.15.84:0/2170039038 >> [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128] conn(0x7f14040ad770 0x7f14040afc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:55.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.637+0000 7f140effd700 1 -- 172.21.15.84:0/2170039038 --> [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f141000ac90 con 0x7f14040ad770 2024-04-26T18:25:55.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.637+0000 7f140effd700 1 -- 172.21.15.84:0/2170039038 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f14100d4060 con 0x7f1418108390 2024-04-26T18:25:55.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.637+0000 7f141d8d1700 1 --2- 172.21.15.84:0/2170039038 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14040a3bb0 0x7f14040a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f141400b5c0 tx=0x7f1414005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:55.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.637+0000 7f141d0d0700 1 --2- 172.21.15.84:0/2170039038 >> [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128] conn(0x7f14040ad770 0x7f14040afc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:55.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.638+0000 7f141d0d0700 1 --2- 172.21.15.84:0/2170039038 >> [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128] conn(0x7f14040ad770 0x7f14040afc10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:55.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.639+0000 7f140effd700 1 -- 172.21.15.84:0/2170039038 <== osd.11 v2:172.21.15.174:6824/1668672128 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f141000ac90 con 0x7f14040ad770 2024-04-26T18:25:55.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.656+0000 7f141fb35700 1 -- 172.21.15.84:0/2170039038 --> [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1418103a40 con 0x7f14040ad770 2024-04-26T18:25:55.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.657+0000 7f140effd700 1 -- 172.21.15.84:0/2170039038 <== osd.11 v2:172.21.15.174:6824/1668672128 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1418103a40 con 0x7f14040ad770 2024-04-26T18:25:55.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.657+0000 7f141fb35700 1 -- 172.21.15.84:0/2170039038 >> [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128] conn(0x7f14040ad770 msgr2=0x7f14040afc10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.657+0000 7f141fb35700 1 --2- 172.21.15.84:0/2170039038 >> [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128] conn(0x7f14040ad770 0x7f14040afc10 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:55.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.657+0000 7f141fb35700 1 -- 172.21.15.84:0/2170039038 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14040a3bb0 msgr2=0x7f14040a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.657+0000 7f141fb35700 1 --2- 172.21.15.84:0/2170039038 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14040a3bb0 0x7f14040a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f141400b5c0 tx=0x7f1414005960 comp rx=0 tx=0).stop 2024-04-26T18:25:55.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.657+0000 7f141fb35700 1 -- 172.21.15.84:0/2170039038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1418108390 msgr2=0x7f1418199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:55.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.657+0000 7f141fb35700 1 --2- 172.21.15.84:0/2170039038 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1418108390 0x7f1418199900 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1410007d70 tx=0x7f141000a340 comp rx=0 tx=0).stop 2024-04-26T18:25:55.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.657+0000 7f141e0d2700 1 -- 172.21.15.84:0/2170039038 reap_dead start 2024-04-26T18:25:55.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.657+0000 7f141fb35700 1 -- 172.21.15.84:0/2170039038 shutdown_connections 2024-04-26T18:25:55.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.657+0000 7f141fb35700 1 -- 172.21.15.84:0/2170039038 >> 172.21.15.84:0/2170039038 conn(0x7f14180792d0 msgr2=0x7f14181016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:55.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.657+0000 7f141fb35700 1 -- 172.21.15.84:0/2170039038 shutdown_connections 2024-04-26T18:25:55.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:55.657+0000 7f141fb35700 1 -- 172.21.15.84:0/2170039038 wait complete. 2024-04-26T18:25:56.533 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:56 smithi084 ceph-mon[29141]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-26T18:25:56.534 INFO:teuthology.orchestra.run.smithi084.stdout:347892351011 2024-04-26T18:25:56.535 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-26T18:25:56.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:56 smithi103 ceph-mon[32727]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-26T18:25:56.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:56 smithi174 ceph-mon[32705]: pgmap v346: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-26T18:25:57.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.010+0000 7f6548570700 1 -- 172.21.15.84:0/1152577684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6540102cf0 msgr2=0x7f654010f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.010+0000 7f6548570700 1 --2- 172.21.15.84:0/1152577684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6540102cf0 0x7f654010f1d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f653800b0a0 tx=0x7f653800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.010+0000 7f6548570700 1 -- 172.21.15.84:0/1152577684 shutdown_connections 2024-04-26T18:25:57.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.010+0000 7f6548570700 1 --2- 172.21.15.84:0/1152577684 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6540102cf0 0x7f654010f1d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.010+0000 7f6548570700 1 --2- 172.21.15.84:0/1152577684 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6540108390 0x7f6540108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.010+0000 7f6548570700 1 --2- 172.21.15.84:0/1152577684 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6540102350 0x7f65401027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.010+0000 7f6548570700 1 -- 172.21.15.84:0/1152577684 >> 172.21.15.84:0/1152577684 conn(0x7f65400792c0 msgr2=0x7f65400796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.010+0000 7f6548570700 1 -- 172.21.15.84:0/1152577684 shutdown_connections 2024-04-26T18:25:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.010+0000 7f6548570700 1 -- 172.21.15.84:0/1152577684 wait complete. 2024-04-26T18:25:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.011+0000 7f6548570700 1 Processor -- start 2024-04-26T18:25:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.011+0000 7f6548570700 1 -- start start 2024-04-26T18:25:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.011+0000 7f6548570700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6540102350 0x7f654019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.011+0000 7f6548570700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6540102cf0 0x7f654019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.011+0000 7f6548570700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6540108390 0x7f6540199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.011+0000 7f6548570700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f65401124c0 con 0x7f6540102cf0 2024-04-26T18:25:57.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.011+0000 7f6548570700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6540112340 con 0x7f6540102350 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.011+0000 7f6548570700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6540112640 con 0x7f6540108390 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.011+0000 7f6545b0b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6540102cf0 0x7f654019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.011+0000 7f6545b0b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6540102cf0 0x7f654019fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60970/0 (socket says 172.21.15.84:60970) 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.011+0000 7f6545b0b700 1 -- 172.21.15.84:0/3640308376 learned_addr learned my addr 172.21.15.84:0/3640308376 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.011+0000 7f654630c700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6540102350 0x7f654019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.011+0000 7f6546b0d700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6540108390 0x7f6540199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.012+0000 7f6545b0b700 1 -- 172.21.15.84:0/3640308376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6540108390 msgr2=0x7f6540199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.012+0000 7f6545b0b700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6540108390 0x7f6540199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.012+0000 7f6545b0b700 1 -- 172.21.15.84:0/3640308376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6540102350 msgr2=0x7f654019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.012+0000 7f6545b0b700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6540102350 0x7f654019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.012+0000 7f6545b0b700 1 -- 172.21.15.84:0/3640308376 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654019a000 con 0x7f6540102cf0 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.012+0000 7f6546b0d700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6540108390 0x7f6540199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.012+0000 7f6545b0b700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6540102cf0 0x7f654019fbe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f653000d980 tx=0x7f653000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.012+0000 7f65377fe700 1 -- 172.21.15.84:0/3640308376 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530017070 con 0x7f6540102cf0 2024-04-26T18:25:57.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.012+0000 7f65377fe700 1 -- 172.21.15.84:0/3640308376 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6530005020 con 0x7f6540102cf0 2024-04-26T18:25:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.012+0000 7f6548570700 1 -- 172.21.15.84:0/3640308376 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654019a2f0 con 0x7f6540102cf0 2024-04-26T18:25:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.012+0000 7f65377fe700 1 -- 172.21.15.84:0/3640308376 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530009b50 con 0x7f6540102cf0 2024-04-26T18:25:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.012+0000 7f6548570700 1 -- 172.21.15.84:0/3640308376 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65401a6500 con 0x7f6540102cf0 2024-04-26T18:25:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.013+0000 7f6548570700 1 -- 172.21.15.84:0/3640308376 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6540106490 con 0x7f6540102cf0 2024-04-26T18:25:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.014+0000 7f65377fe700 1 -- 172.21.15.84:0/3640308376 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f65300051d0 con 0x7f6540102cf0 2024-04-26T18:25:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.015+0000 7f65377fe700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f652c0a3cd0 0x7f652c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.015+0000 7f654630c700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f652c0a3cd0 0x7f652c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.015+0000 7f65377fe700 1 -- 172.21.15.84:0/3640308376 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f65300d0a10 con 0x7f6540102cf0 2024-04-26T18:25:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.015+0000 7f65377fe700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548] conn(0x7f652c0ad890 0x7f652c0afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.015+0000 7f65377fe700 1 -- 172.21.15.84:0/3640308376 --> [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f652c0b0400 con 0x7f652c0ad890 2024-04-26T18:25:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.015+0000 7f65377fe700 1 -- 172.21.15.84:0/3640308376 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f65300d0e60 con 0x7f6540102cf0 2024-04-26T18:25:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.015+0000 7f654630c700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f652c0a3cd0 0x7f652c0a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f653c00abd0 tx=0x7f653c00a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:57.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.015+0000 7f6546b0d700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548] conn(0x7f652c0ad890 0x7f652c0afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.016+0000 7f6546b0d700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548] conn(0x7f652c0ad890 0x7f652c0afd30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:57.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.017+0000 7f65377fe700 1 -- 172.21.15.84:0/3640308376 <== osd.8 v2:172.21.15.174:6816/38410548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f652c0b0400 con 0x7f652c0ad890 2024-04-26T18:25:57.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.036+0000 7f6548570700 1 -- 172.21.15.84:0/3640308376 --> [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6540103a40 con 0x7f652c0ad890 2024-04-26T18:25:57.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.036+0000 7f65377fe700 1 -- 172.21.15.84:0/3640308376 <== osd.8 v2:172.21.15.174:6816/38410548 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6540103a40 con 0x7f652c0ad890 2024-04-26T18:25:57.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.037+0000 7f6548570700 1 -- 172.21.15.84:0/3640308376 >> [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548] conn(0x7f652c0ad890 msgr2=0x7f652c0afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.037+0000 7f6548570700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548] conn(0x7f652c0ad890 0x7f652c0afd30 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.037+0000 7f6548570700 1 -- 172.21.15.84:0/3640308376 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f652c0a3cd0 msgr2=0x7f652c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.037+0000 7f6548570700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f652c0a3cd0 0x7f652c0a6190 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f653c00abd0 tx=0x7f653c00a380 comp rx=0 tx=0).stop 2024-04-26T18:25:57.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.037+0000 7f6548570700 1 -- 172.21.15.84:0/3640308376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6540102cf0 msgr2=0x7f654019fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.037+0000 7f6548570700 1 --2- 172.21.15.84:0/3640308376 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6540102cf0 0x7f654019fbe0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f653000d980 tx=0x7f653000dc90 comp rx=0 tx=0).stop 2024-04-26T18:25:57.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.037+0000 7f6546b0d700 1 -- 172.21.15.84:0/3640308376 reap_dead start 2024-04-26T18:25:57.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.037+0000 7f6548570700 1 -- 172.21.15.84:0/3640308376 shutdown_connections 2024-04-26T18:25:57.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.037+0000 7f6548570700 1 -- 172.21.15.84:0/3640308376 >> 172.21.15.84:0/3640308376 conn(0x7f65400792c0 msgr2=0x7f6540101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:57.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.037+0000 7f6548570700 1 -- 172.21.15.84:0/3640308376 shutdown_connections 2024-04-26T18:25:57.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.037+0000 7f6548570700 1 -- 172.21.15.84:0/3640308376 wait complete. 2024-04-26T18:25:57.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.208+0000 7f61a264b700 1 -- 172.21.15.84:0/786045242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f619c1023b0 msgr2=0x7f619c1102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.208+0000 7f61a264b700 1 --2- 172.21.15.84:0/786045242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f619c1023b0 0x7f619c1102f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f619400b0a0 tx=0x7f619400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.208+0000 7f61a264b700 1 -- 172.21.15.84:0/786045242 shutdown_connections 2024-04-26T18:25:57.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.208+0000 7f61a264b700 1 --2- 172.21.15.84:0/786045242 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f619c1023b0 0x7f619c1102f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.208+0000 7f61a264b700 1 --2- 172.21.15.84:0/786045242 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f619c1094b0 0x7f619c109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.208+0000 7f61a264b700 1 --2- 172.21.15.84:0/786045242 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f619c101a10 0x7f619c101e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.208+0000 7f61a264b700 1 -- 172.21.15.84:0/786045242 >> 172.21.15.84:0/786045242 conn(0x7f619c078990 msgr2=0x7f619c0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:57.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.208+0000 7f61a264b700 1 -- 172.21.15.84:0/786045242 shutdown_connections 2024-04-26T18:25:57.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.208+0000 7f61a264b700 1 -- 172.21.15.84:0/786045242 wait complete. 2024-04-26T18:25:57.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.209+0000 7f61a264b700 1 Processor -- start 2024-04-26T18:25:57.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.209+0000 7f61a264b700 1 -- start start 2024-04-26T18:25:57.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.209+0000 7f61a264b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f619c101a10 0x7f619c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.209+0000 7f61a264b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f619c1023b0 0x7f619c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.209+0000 7f61a264b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f619c1094b0 0x7f619c1999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.209+0000 7f61a264b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f619c113630 con 0x7f619c101a10 2024-04-26T18:25:57.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.209+0000 7f61a264b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f619c1134b0 con 0x7f619c1094b0 2024-04-26T18:25:57.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.209+0000 7f61a264b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f619c1137b0 con 0x7f619c1023b0 2024-04-26T18:25:57.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.209+0000 7f619bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f619c101a10 0x7f619c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.209+0000 7f619bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f619c101a10 0x7f619c19f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60978/0 (socket says 172.21.15.84:60978) 2024-04-26T18:25:57.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.209+0000 7f619bfff700 1 -- 172.21.15.84:0/3308917897 learned_addr learned my addr 172.21.15.84:0/3308917897 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:57.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f619b7fe700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f619c1023b0 0x7f619c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f619bfff700 1 -- 172.21.15.84:0/3308917897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f619c1023b0 msgr2=0x7f619c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f619bfff700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f619c1023b0 0x7f619c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f61a0be8700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f619c1094b0 0x7f619c1999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f619bfff700 1 -- 172.21.15.84:0/3308917897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f619c1094b0 msgr2=0x7f619c1999b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f619bfff700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f619c1094b0 0x7f619c1999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f619bfff700 1 -- 172.21.15.84:0/3308917897 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f619c19a110 con 0x7f619c101a10 2024-04-26T18:25:57.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f619b7fe700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f619c1023b0 0x7f619c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:25:57.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f619bfff700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f619c101a10 0x7f619c19f750 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f618c00b530 tx=0x7f618c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:57.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f61997fa700 1 -- 172.21.15.84:0/3308917897 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c015020 con 0x7f619c101a10 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f61997fa700 1 -- 172.21.15.84:0/3308917897 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f618c011ce0 con 0x7f619c101a10 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f61997fa700 1 -- 172.21.15.84:0/3308917897 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f618c010d20 con 0x7f619c101a10 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f61a264b700 1 -- 172.21.15.84:0/3308917897 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f619c19a400 con 0x7f619c101a10 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.210+0000 7f61a264b700 1 -- 172.21.15.84:0/3308917897 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f619c1a6620 con 0x7f619c101a10 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.211+0000 7f61a264b700 1 -- 172.21.15.84:0/3308917897 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f619c063b70 con 0x7f619c101a10 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.212+0000 7f61997fa700 1 -- 172.21.15.84:0/3308917897 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f618c018020 con 0x7f619c101a10 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.212+0000 7f61997fa700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f61840a3d50 0x7f61840a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.213+0000 7f619b7fe700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f61840a3d50 0x7f61840a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.213+0000 7f61997fa700 1 -- 172.21.15.84:0/3308917897 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f618c0d0350 con 0x7f619c101a10 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.213+0000 7f61997fa700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768] conn(0x7f61840ad910 0x7f61840afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.213+0000 7f61997fa700 1 -- 172.21.15.84:0/3308917897 --> [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f61840b0480 con 0x7f61840ad910 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.213+0000 7f61997fa700 1 -- 172.21.15.84:0/3308917897 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f618c0d07a0 con 0x7f619c101a10 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.213+0000 7f619b7fe700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f61840a3d50 0x7f61840a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6190005fd0 tx=0x7f6190005ee0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:57.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.213+0000 7f61a0be8700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768] conn(0x7f61840ad910 0x7f61840afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.214+0000 7f61a0be8700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768] conn(0x7f61840ad910 0x7f61840afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:57.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.215+0000 7f61997fa700 1 -- 172.21.15.84:0/3308917897 <== osd.5 v2:172.21.15.174:6808/3168291768 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f61840b0480 con 0x7f61840ad910 2024-04-26T18:25:57.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.231+0000 7f61a264b700 1 -- 172.21.15.84:0/3308917897 --> [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f619c063f70 con 0x7f61840ad910 2024-04-26T18:25:57.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.232+0000 7f61997fa700 1 -- 172.21.15.84:0/3308917897 <== osd.5 v2:172.21.15.174:6808/3168291768 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f619c063f70 con 0x7f61840ad910 2024-04-26T18:25:57.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.232+0000 7f61a264b700 1 -- 172.21.15.84:0/3308917897 >> [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768] conn(0x7f61840ad910 msgr2=0x7f61840afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.232+0000 7f61a264b700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768] conn(0x7f61840ad910 0x7f61840afdb0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.232+0000 7f61a264b700 1 -- 172.21.15.84:0/3308917897 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f61840a3d50 msgr2=0x7f61840a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.232+0000 7f61a264b700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f61840a3d50 0x7f61840a6210 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6190005fd0 tx=0x7f6190005ee0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.232+0000 7f61a264b700 1 -- 172.21.15.84:0/3308917897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f619c101a10 msgr2=0x7f619c19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.232+0000 7f61a264b700 1 --2- 172.21.15.84:0/3308917897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f619c101a10 0x7f619c19f750 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f618c00b530 tx=0x7f618c00b8f0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.232+0000 7f61a0be8700 1 -- 172.21.15.84:0/3308917897 reap_dead start 2024-04-26T18:25:57.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.232+0000 7f61a264b700 1 -- 172.21.15.84:0/3308917897 shutdown_connections 2024-04-26T18:25:57.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.232+0000 7f61a264b700 1 -- 172.21.15.84:0/3308917897 >> 172.21.15.84:0/3308917897 conn(0x7f619c078990 msgr2=0x7f619c106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:57.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.233+0000 7f61a264b700 1 -- 172.21.15.84:0/3308917897 shutdown_connections 2024-04-26T18:25:57.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:57.233+0000 7f61a264b700 1 -- 172.21.15.84:0/3308917897 wait complete. 2024-04-26T18:25:57.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.478+0000 7f388d5a3700 1 -- 172.21.15.84:0/3271198040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3888102ce0 msgr2=0x7f388810f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.478+0000 7f388d5a3700 1 --2- 172.21.15.84:0/3271198040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3888102ce0 0x7f388810f1d0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f388000b0a0 tx=0x7f388000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.478+0000 7f388d5a3700 1 -- 172.21.15.84:0/3271198040 shutdown_connections 2024-04-26T18:25:57.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.478+0000 7f388d5a3700 1 --2- 172.21.15.84:0/3271198040 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3888102ce0 0x7f388810f1d0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.478+0000 7f388d5a3700 1 --2- 172.21.15.84:0/3271198040 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3888108390 0x7f3888108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.478+0000 7f388d5a3700 1 --2- 172.21.15.84:0/3271198040 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3888102320 0x7f38881027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.478+0000 7f388d5a3700 1 -- 172.21.15.84:0/3271198040 >> 172.21.15.84:0/3271198040 conn(0x7f38880792d0 msgr2=0x7f38880796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:57.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.478+0000 7f388d5a3700 1 -- 172.21.15.84:0/3271198040 shutdown_connections 2024-04-26T18:25:57.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.478+0000 7f388d5a3700 1 -- 172.21.15.84:0/3271198040 wait complete. 2024-04-26T18:25:57.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.479+0000 7f388d5a3700 1 Processor -- start 2024-04-26T18:25:57.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.479+0000 7f388d5a3700 1 -- start start 2024-04-26T18:25:57.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.479+0000 7f388d5a3700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3888102320 0x7f388819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.479+0000 7f388d5a3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3888102ce0 0x7f388819ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.479+0000 7f388d5a3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3888108390 0x7f3888198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.479+0000 7f388d5a3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f38881124e0 con 0x7f3888102ce0 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.479+0000 7f388d5a3700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3888112360 con 0x7f3888102320 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.479+0000 7f388d5a3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3888112660 con 0x7f3888108390 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f38867fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3888102ce0 0x7f388819ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f38867fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3888102ce0 0x7f388819ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48256/0 (socket says 172.21.15.84:48256) 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f38867fc700 1 -- 172.21.15.84:0/3626507049 learned_addr learned my addr 172.21.15.84:0/3626507049 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f3886ffd700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3888102320 0x7f388819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f38877fe700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3888108390 0x7f3888198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f38867fc700 1 -- 172.21.15.84:0/3626507049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3888108390 msgr2=0x7f3888198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f38867fc700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3888108390 0x7f3888198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f38867fc700 1 -- 172.21.15.84:0/3626507049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3888102320 msgr2=0x7f388819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f38867fc700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3888102320 0x7f388819e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f38867fc700 1 -- 172.21.15.84:0/3626507049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3888199020 con 0x7f3888102ce0 2024-04-26T18:25:57.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f3886ffd700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3888102320 0x7f388819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:57.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f38867fc700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3888102ce0 0x7f388819ec00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f387c00d980 tx=0x7f387c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:57.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f386ffff700 1 -- 172.21.15.84:0/3626507049 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c017070 con 0x7f3888102ce0 2024-04-26T18:25:57.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f388d5a3700 1 -- 172.21.15.84:0/3626507049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3888199310 con 0x7f3888102ce0 2024-04-26T18:25:57.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f386ffff700 1 -- 172.21.15.84:0/3626507049 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f387c004830 con 0x7f3888102ce0 2024-04-26T18:25:57.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f386ffff700 1 -- 172.21.15.84:0/3626507049 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f387c005440 con 0x7f3888102ce0 2024-04-26T18:25:57.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.480+0000 7f388d5a3700 1 -- 172.21.15.84:0/3626507049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38881a5520 con 0x7f3888102ce0 2024-04-26T18:25:57.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.481+0000 7f388d5a3700 1 -- 172.21.15.84:0/3626507049 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3888063af0 con 0x7f3888102ce0 2024-04-26T18:25:57.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.482+0000 7f386ffff700 1 -- 172.21.15.84:0/3626507049 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f387c0055e0 con 0x7f3888102ce0 2024-04-26T18:25:57.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.483+0000 7f386ffff700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f38700a3cd0 0x7f38700a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.483+0000 7f3886ffd700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f38700a3cd0 0x7f38700a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.483+0000 7f386ffff700 1 -- 172.21.15.84:0/3626507049 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f387c0d0b20 con 0x7f3888102ce0 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.483+0000 7f386ffff700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252] conn(0x7f38700ad890 0x7f38700afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.483+0000 7f386ffff700 1 -- 172.21.15.84:0/3626507049 --> [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f38700b0400 con 0x7f38700ad890 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.483+0000 7f386ffff700 1 -- 172.21.15.84:0/3626507049 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f387c0d0f70 con 0x7f3888102ce0 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.483+0000 7f3886ffd700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f38700a3cd0 0x7f38700a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3878005e10 tx=0x7f3878005d60 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.483+0000 7f38877fe700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252] conn(0x7f38700ad890 0x7f38700afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.484+0000 7f38877fe700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252] conn(0x7f38700ad890 0x7f38700afd30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.485+0000 7f386ffff700 1 -- 172.21.15.84:0/3626507049 <== osd.4 v2:172.21.15.103:6808/1619258252 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f38700b0400 con 0x7f38700ad890 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.501+0000 7f388d5a3700 1 -- 172.21.15.84:0/3626507049 --> [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3888063f70 con 0x7f38700ad890 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.502+0000 7f386ffff700 1 -- 172.21.15.84:0/3626507049 <== osd.4 v2:172.21.15.103:6808/1619258252 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3888063f70 con 0x7f38700ad890 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.502+0000 7f388d5a3700 1 -- 172.21.15.84:0/3626507049 >> [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252] conn(0x7f38700ad890 msgr2=0x7f38700afd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.502+0000 7f388d5a3700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252] conn(0x7f38700ad890 0x7f38700afd30 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.502+0000 7f388d5a3700 1 -- 172.21.15.84:0/3626507049 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f38700a3cd0 msgr2=0x7f38700a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.502+0000 7f388d5a3700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f38700a3cd0 0x7f38700a6190 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3878005e10 tx=0x7f3878005d60 comp rx=0 tx=0).stop 2024-04-26T18:25:57.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.502+0000 7f388d5a3700 1 -- 172.21.15.84:0/3626507049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3888102ce0 msgr2=0x7f388819ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:57.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.502+0000 7f388d5a3700 1 --2- 172.21.15.84:0/3626507049 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3888102ce0 0x7f388819ec00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f387c00d980 tx=0x7f387c00dd40 comp rx=0 tx=0).stop 2024-04-26T18:25:57.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.502+0000 7f38877fe700 1 -- 172.21.15.84:0/3626507049 reap_dead start 2024-04-26T18:25:57.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.502+0000 7f388d5a3700 1 -- 172.21.15.84:0/3626507049 shutdown_connections 2024-04-26T18:25:57.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.502+0000 7f388d5a3700 1 -- 172.21.15.84:0/3626507049 >> 172.21.15.84:0/3626507049 conn(0x7f38880792d0 msgr2=0x7f3888100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:57.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.503+0000 7f388d5a3700 1 -- 172.21.15.84:0/3626507049 shutdown_connections 2024-04-26T18:25:57.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:50.503+0000 7f388d5a3700 1 -- 172.21.15.84:0/3626507049 wait complete. 2024-04-26T18:25:58.454 INFO:teuthology.orchestra.run.smithi084.stdout:146028888134 2024-04-26T18:25:58.454 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T18:25:58.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:25:58 smithi084 ceph-mon[29141]: pgmap v347: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T18:25:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:25:58 smithi103 ceph-mon[32727]: pgmap v347: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T18:25:58.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:25:58 smithi174 ceph-mon[32705]: pgmap v347: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 9 op/s 2024-04-26T18:25:59.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.155+0000 7f700671b700 1 -- 172.21.15.84:0/2763774112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7000102d10 msgr2=0x7f700010f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.155+0000 7f700671b700 1 --2- 172.21.15.84:0/2763774112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7000102d10 0x7f700010f200 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800b0a0 tx=0x7f6ff800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.155+0000 7f700671b700 1 -- 172.21.15.84:0/2763774112 shutdown_connections 2024-04-26T18:25:59.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.155+0000 7f700671b700 1 --2- 172.21.15.84:0/2763774112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7000102d10 0x7f700010f200 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.155+0000 7f700671b700 1 --2- 172.21.15.84:0/2763774112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f70001083c0 0x7f70001087a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.155+0000 7f700671b700 1 --2- 172.21.15.84:0/2763774112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7000102350 0x7f70001027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.155+0000 7f700671b700 1 -- 172.21.15.84:0/2763774112 >> 172.21.15.84:0/2763774112 conn(0x7f7000079360 msgr2=0x7f7000079770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:59.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.155+0000 7f700671b700 1 -- 172.21.15.84:0/2763774112 shutdown_connections 2024-04-26T18:25:59.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.155+0000 7f700671b700 1 -- 172.21.15.84:0/2763774112 wait complete. 2024-04-26T18:25:59.156 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.156+0000 7f700671b700 1 Processor -- start 2024-04-26T18:25:59.157 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.156+0000 7f700671b700 1 -- start start 2024-04-26T18:25:59.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.156+0000 7f700671b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7000102350 0x7f70001a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:59.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.156+0000 7f700671b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7000102d10 0x7f70001a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:59.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.156+0000 7f700671b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70001083c0 0x7f700019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:59.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.156+0000 7f700671b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f70001113d0 con 0x7f70001083c0 2024-04-26T18:25:59.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.156+0000 7f700671b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7000111250 con 0x7f7000102d10 2024-04-26T18:25:59.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.156+0000 7f700671b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7000111550 con 0x7f7000102350 2024-04-26T18:25:59.158 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.156+0000 7f7004cb8700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70001083c0 0x7f700019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.156+0000 7f7004cb8700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70001083c0 0x7f700019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60988/0 (socket says 172.21.15.84:60988) 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f7004cb8700 1 -- 172.21.15.84:0/3947976707 learned_addr learned my addr 172.21.15.84:0/3947976707 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f6fff7fe700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7000102d10 0x7f70001a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f6ffffff700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7000102350 0x7f70001a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f7004cb8700 1 -- 172.21.15.84:0/3947976707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7000102350 msgr2=0x7f70001a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f7004cb8700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7000102350 0x7f70001a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f7004cb8700 1 -- 172.21.15.84:0/3947976707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7000102d10 msgr2=0x7f70001a1f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f7004cb8700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7000102d10 0x7f70001a1f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f7004cb8700 1 -- 172.21.15.84:0/3947976707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f700019c590 con 0x7f70001083c0 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f7004cb8700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70001083c0 0x7f700019bd10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800b830 tx=0x7f6ff8004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f6ffd7fa700 1 -- 172.21.15.84:0/3947976707 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff8021070 con 0x7f70001083c0 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f700671b700 1 -- 172.21.15.84:0/3947976707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f700019c820 con 0x7f70001083c0 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f6ffd7fa700 1 -- 172.21.15.84:0/3947976707 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff8007e30 con 0x7f70001083c0 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f6ffd7fa700 1 -- 172.21.15.84:0/3947976707 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff80047d0 con 0x7f70001083c0 2024-04-26T18:25:59.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.157+0000 7f700671b700 1 -- 172.21.15.84:0/3947976707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70001a89c0 con 0x7f70001083c0 2024-04-26T18:25:59.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.158+0000 7f700671b700 1 -- 172.21.15.84:0/3947976707 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7000107210 con 0x7f70001083c0 2024-04-26T18:25:59.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.160+0000 7f6ffd7fa700 1 -- 172.21.15.84:0/3947976707 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6ff8007980 con 0x7f70001083c0 2024-04-26T18:25:59.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.160+0000 7f6ffd7fa700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6fe80a3cd0 0x7f6fe80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:59.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.160+0000 7f6ffffff700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6fe80a3cd0 0x7f6fe80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:59.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.160+0000 7f6ffd7fa700 1 -- 172.21.15.84:0/3947976707 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f6ff800f020 con 0x7f70001083c0 2024-04-26T18:25:59.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.160+0000 7f6ffd7fa700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915] conn(0x7f6fe80ad8f0 0x7f6fe80afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:59.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.160+0000 7f6ffd7fa700 1 -- 172.21.15.84:0/3947976707 --> [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6ff801fe80 con 0x7f6fe80ad8f0 2024-04-26T18:25:59.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.160+0000 7f6ffd7fa700 1 -- 172.21.15.84:0/3947976707 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f6ff80d62c0 con 0x7f70001083c0 2024-04-26T18:25:59.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.161+0000 7f6ffffff700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6fe80a3cd0 0x7f6fe80a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0005fd0 tx=0x7f6ff0005db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:59.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.161+0000 7f6fff7fe700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915] conn(0x7f6fe80ad8f0 0x7f6fe80afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:59.162 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.161+0000 7f6fff7fe700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915] conn(0x7f6fe80ad8f0 0x7f6fe80afd90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:59.162 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.162+0000 7f6ffd7fa700 1 -- 172.21.15.84:0/3947976707 <== osd.10 v2:172.21.15.103:6824/2705726915 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6ff801fe80 con 0x7f6fe80ad8f0 2024-04-26T18:25:59.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.178+0000 7f700671b700 1 -- 172.21.15.84:0/3947976707 --> [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7000103a70 con 0x7f6fe80ad8f0 2024-04-26T18:25:59.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.179+0000 7f6ffd7fa700 1 -- 172.21.15.84:0/3947976707 <== osd.10 v2:172.21.15.103:6824/2705726915 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7000103a70 con 0x7f6fe80ad8f0 2024-04-26T18:25:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.179+0000 7f700671b700 1 -- 172.21.15.84:0/3947976707 >> [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915] conn(0x7f6fe80ad8f0 msgr2=0x7f6fe80afd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.179+0000 7f700671b700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915] conn(0x7f6fe80ad8f0 0x7f6fe80afd90 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.179+0000 7f700671b700 1 -- 172.21.15.84:0/3947976707 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6fe80a3cd0 msgr2=0x7f6fe80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.179+0000 7f700671b700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6fe80a3cd0 0x7f6fe80a6190 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0005fd0 tx=0x7f6ff0005db0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.179+0000 7f700671b700 1 -- 172.21.15.84:0/3947976707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70001083c0 msgr2=0x7f700019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.179+0000 7f700671b700 1 --2- 172.21.15.84:0/3947976707 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f70001083c0 0x7f700019bd10 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f6ff800b830 tx=0x7f6ff8004ce0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.179+0000 7f7004cb8700 1 -- 172.21.15.84:0/3947976707 reap_dead start 2024-04-26T18:25:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.179+0000 7f700671b700 1 -- 172.21.15.84:0/3947976707 shutdown_connections 2024-04-26T18:25:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.180+0000 7f700671b700 1 -- 172.21.15.84:0/3947976707 >> 172.21.15.84:0/3947976707 conn(0x7f7000079360 msgr2=0x7f7000100360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.180+0000 7f700671b700 1 -- 172.21.15.84:0/3947976707 shutdown_connections 2024-04-26T18:25:59.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:59.180+0000 7f700671b700 1 -- 172.21.15.84:0/3947976707 wait complete. 2024-04-26T18:25:59.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.025+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/330803963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c34108390 msgr2=0x7f3c34108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.025+0000 7f3c3a3e5700 1 --2- 172.21.15.84:0/330803963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c34108390 0x7f3c34108770 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f3c28009990 tx=0x7f3c28009ca0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.025+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/330803963 shutdown_connections 2024-04-26T18:25:59.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.025+0000 7f3c3a3e5700 1 --2- 172.21.15.84:0/330803963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c34108390 0x7f3c34108770 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.025+0000 7f3c3a3e5700 1 --2- 172.21.15.84:0/330803963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c34102ce0 0x7f3c3410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.025+0000 7f3c3a3e5700 1 --2- 172.21.15.84:0/330803963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c34102320 0x7f3c341027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.025+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/330803963 >> 172.21.15.84:0/330803963 conn(0x7f3c340792d0 msgr2=0x7f3c340796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:59.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.025+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/330803963 shutdown_connections 2024-04-26T18:25:59.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.025+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/330803963 wait complete. 2024-04-26T18:25:59.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c3a3e5700 1 Processor -- start 2024-04-26T18:25:59.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c3a3e5700 1 -- start start 2024-04-26T18:25:59.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c3a3e5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c34102320 0x7f3c3419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:59.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c3a3e5700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c34102ce0 0x7f3c3419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c3a3e5700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c34108390 0x7f3c341999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c3a3e5700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3c341124e0 con 0x7f3c34108390 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c3a3e5700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34112360 con 0x7f3c34102ce0 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c3a3e5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3c34112660 con 0x7f3c34102320 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c38982700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c34108390 0x7f3c341999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c38982700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c34108390 0x7f3c341999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48316/0 (socket says 172.21.15.84:48316) 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c38982700 1 -- 172.21.15.84:0/3772638279 learned_addr learned my addr 172.21.15.84:0/3772638279 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c33fff700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c34102320 0x7f3c3419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c337fe700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c34102ce0 0x7f3c3419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c38982700 1 -- 172.21.15.84:0/3772638279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c34102320 msgr2=0x7f3c3419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c38982700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c34102320 0x7f3c3419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c38982700 1 -- 172.21.15.84:0/3772638279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c34102ce0 msgr2=0x7f3c3419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c38982700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3c34102ce0 0x7f3c3419fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c38982700 1 -- 172.21.15.84:0/3772638279 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c3419a140 con 0x7f3c34108390 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.026+0000 7f3c33fff700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3c34102320 0x7f3c3419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:25:59.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.027+0000 7f3c38982700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c34108390 0x7f3c341999e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c007ee0 tx=0x7f3c2c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:59.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.027+0000 7f3c317fa700 1 -- 172.21.15.84:0/3772638279 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c00dbd0 con 0x7f3c34108390 2024-04-26T18:25:59.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.027+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/3772638279 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c3419a430 con 0x7f3c34108390 2024-04-26T18:25:59.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.027+0000 7f3c317fa700 1 -- 172.21.15.84:0/3772638279 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c2c00dd70 con 0x7f3c34108390 2024-04-26T18:25:59.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.027+0000 7f3c317fa700 1 -- 172.21.15.84:0/3772638279 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c2c005990 con 0x7f3c34108390 2024-04-26T18:25:59.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.027+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/3772638279 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c341a6650 con 0x7f3c34108390 2024-04-26T18:25:59.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.028+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/3772638279 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3c34107460 con 0x7f3c34108390 2024-04-26T18:25:59.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.030+0000 7f3c317fa700 1 -- 172.21.15.84:0/3772638279 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f3c2c01a020 con 0x7f3c34108390 2024-04-26T18:25:59.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.030+0000 7f3c317fa700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c1c0a3d50 0x7f3c1c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:59.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.030+0000 7f3c33fff700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c1c0a3d50 0x7f3c1c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.030+0000 7f3c317fa700 1 -- 172.21.15.84:0/3772638279 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7f3c2c0d0740 con 0x7f3c34108390 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.030+0000 7f3c317fa700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927] conn(0x7f3c1c0ad910 0x7f3c1c0afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.030+0000 7f3c317fa700 1 -- 172.21.15.84:0/3772638279 --> [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3c2c01de50 con 0x7f3c1c0ad910 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.030+0000 7f3c317fa700 1 -- 172.21.15.84:0/3772638279 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f3c2c0d0b90 con 0x7f3c34108390 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.031+0000 7f3c337fe700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927] conn(0x7f3c1c0ad910 0x7f3c1c0afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.031+0000 7f3c33fff700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c1c0a3d50 0x7f3c1c0a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3c28006010 tx=0x7f3c2800b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.031+0000 7f3c337fe700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927] conn(0x7f3c1c0ad910 0x7f3c1c0afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.032+0000 7f3c317fa700 1 -- 172.21.15.84:0/3772638279 <== osd.9 v2:172.21.15.84:6826/2926393927 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3c2c01de50 con 0x7f3c1c0ad910 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.048+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/3772638279 --> [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3c34103a40 con 0x7f3c1c0ad910 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.049+0000 7f3c317fa700 1 -- 172.21.15.84:0/3772638279 <== osd.9 v2:172.21.15.84:6826/2926393927 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3c34103a40 con 0x7f3c1c0ad910 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.049+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/3772638279 >> [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927] conn(0x7f3c1c0ad910 msgr2=0x7f3c1c0afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.049+0000 7f3c3a3e5700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927] conn(0x7f3c1c0ad910 0x7f3c1c0afdb0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.049+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/3772638279 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c1c0a3d50 msgr2=0x7f3c1c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.049+0000 7f3c3a3e5700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3c1c0a3d50 0x7f3c1c0a6210 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3c28006010 tx=0x7f3c2800b540 comp rx=0 tx=0).stop 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.049+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/3772638279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c34108390 msgr2=0x7f3c341999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:25:59.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.049+0000 7f3c3a3e5700 1 --2- 172.21.15.84:0/3772638279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3c34108390 0x7f3c341999e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3c2c007ee0 tx=0x7f3c2c00d3b0 comp rx=0 tx=0).stop 2024-04-26T18:25:59.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.049+0000 7f3c38982700 1 -- 172.21.15.84:0/3772638279 reap_dead start 2024-04-26T18:25:59.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.049+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/3772638279 shutdown_connections 2024-04-26T18:25:59.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.049+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/3772638279 >> 172.21.15.84:0/3772638279 conn(0x7f3c340792d0 msgr2=0x7f3c3406b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:25:59.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.049+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/3772638279 shutdown_connections 2024-04-26T18:25:59.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:25:53.049+0000 7f3c3a3e5700 1 -- 172.21.15.84:0/3772638279 wait complete. 2024-04-26T18:26:00.107 INFO:teuthology.orchestra.run.smithi084.stdout:292057776172 2024-04-26T18:26:00.107 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-26T18:26:00.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:00 smithi084 ceph-mon[29141]: pgmap v348: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:00.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:00 smithi103 ceph-mon[32727]: pgmap v348: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:00.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:00 smithi174 ceph-mon[32705]: pgmap v348: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:01.177 INFO:teuthology.orchestra.run.smithi084.stdout:313532612648 2024-04-26T18:26:01.178 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-26T18:26:01.907 INFO:teuthology.orchestra.run.smithi084.stdout:167503724608 2024-04-26T18:26:01.907 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T18:26:02.347 INFO:teuthology.orchestra.run.smithi084.stdout:55834574932 2024-04-26T18:26:02.348 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T18:26:02.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:02 smithi084 ceph-mon[29141]: pgmap v349: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:02.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:02 smithi103 ceph-mon[32727]: pgmap v349: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:02 smithi174 ceph-mon[32705]: pgmap v349: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:02.756 INFO:teuthology.orchestra.run.smithi084.stdout:257698037810 2024-04-26T18:26:02.757 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-26T18:26:02.807 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:02.809 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:02.823 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:04.512 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:04 smithi084 ceph-mon[29141]: pgmap v350: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.615+0000 7fb493b6b700 1 -- 172.21.15.84:0/3662774132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c101de0 msgr2=0x7fb48c102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.615+0000 7fb493b6b700 1 --2- 172.21.15.84:0/3662774132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c101de0 0x7fb48c102260 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fb488009990 tx=0x7fb488009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.615+0000 7fb493b6b700 1 -- 172.21.15.84:0/3662774132 shutdown_connections 2024-04-26T18:26:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.615+0000 7fb493b6b700 1 --2- 172.21.15.84:0/3662774132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb48c1027a0 0x7fb48c110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.615+0000 7fb493b6b700 1 --2- 172.21.15.84:0/3662774132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb48c108500 0x7fb48c1088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.615+0000 7fb493b6b700 1 --2- 172.21.15.84:0/3662774132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c101de0 0x7fb48c102260 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.615+0000 7fb493b6b700 1 -- 172.21.15.84:0/3662774132 >> 172.21.15.84:0/3662774132 conn(0x7fb48c077640 msgr2=0x7fb48c077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.616+0000 7fb493b6b700 1 -- 172.21.15.84:0/3662774132 shutdown_connections 2024-04-26T18:26:04.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.616+0000 7fb493b6b700 1 -- 172.21.15.84:0/3662774132 wait complete. 2024-04-26T18:26:04.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.616+0000 7fb493b6b700 1 Processor -- start 2024-04-26T18:26:04.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.616+0000 7fb493b6b700 1 -- start start 2024-04-26T18:26:04.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.616+0000 7fb493b6b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c101de0 0x7fb48c19f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:04.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.616+0000 7fb493b6b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb48c1027a0 0x7fb48c19fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:04.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.616+0000 7fb493b6b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb48c108500 0x7fb48c199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:04.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.616+0000 7fb493b6b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c113710 con 0x7fb48c101de0 2024-04-26T18:26:04.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.616+0000 7fb493b6b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c113590 con 0x7fb48c108500 2024-04-26T18:26:04.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.616+0000 7fb493b6b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb48c113890 con 0x7fb48c1027a0 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.616+0000 7fb491907700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c101de0 0x7fb48c19f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.616+0000 7fb491907700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c101de0 0x7fb48c19f8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:32780/0 (socket says 172.21.15.84:32780) 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb491907700 1 -- 172.21.15.84:0/2250150566 learned_addr learned my addr 172.21.15.84:0/2250150566 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb492108700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb48c108500 0x7fb48c199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb491106700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb48c1027a0 0x7fb48c19fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb491907700 1 -- 172.21.15.84:0/2250150566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb48c1027a0 msgr2=0x7fb48c19fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb491907700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb48c1027a0 0x7fb48c19fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb491907700 1 -- 172.21.15.84:0/2250150566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb48c108500 msgr2=0x7fb48c199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb491907700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb48c108500 0x7fb48c199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb491907700 1 -- 172.21.15.84:0/2250150566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb48c19a210 con 0x7fb48c101de0 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb491907700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c101de0 0x7fb48c19f8a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00b500 tx=0x7fb47c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb482ffd700 1 -- 172.21.15.84:0/2250150566 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c015020 con 0x7fb48c101de0 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb493b6b700 1 -- 172.21.15.84:0/2250150566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb48c19a500 con 0x7fb48c101de0 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb482ffd700 1 -- 172.21.15.84:0/2250150566 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb47c011ce0 con 0x7fb48c101de0 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb493b6b700 1 -- 172.21.15.84:0/2250150566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb48c102ef0 con 0x7fb48c101de0 2024-04-26T18:26:04.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.617+0000 7fb482ffd700 1 -- 172.21.15.84:0/2250150566 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb47c010d20 con 0x7fb48c101de0 2024-04-26T18:26:04.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.619+0000 7fb493b6b700 1 -- 172.21.15.84:0/2250150566 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb470001030 con 0x7fb48c101de0 2024-04-26T18:26:04.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.620+0000 7fb482ffd700 1 -- 172.21.15.84:0/2250150566 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb47c018020 con 0x7fb48c101de0 2024-04-26T18:26:04.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.621+0000 7fb482ffd700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb4780a3d50 0x7fb4780a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:04.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.621+0000 7fb491106700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb4780a3d50 0x7fb4780a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:04.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.621+0000 7fb482ffd700 1 -- 172.21.15.84:0/2250150566 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8248+0+0 (secure 0 0 0) 0x7fb47c0cfaa0 con 0x7fb48c101de0 2024-04-26T18:26:04.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.622+0000 7fb482ffd700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079] conn(0x7fb4780ad910 0x7fb4780afdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:04.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.622+0000 7fb482ffd700 1 -- 172.21.15.84:0/2250150566 --> [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb4780b0480 con 0x7fb4780ad910 2024-04-26T18:26:04.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.622+0000 7fb482ffd700 1 -- 172.21.15.84:0/2250150566 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fb47c0d42c0 con 0x7fb48c101de0 2024-04-26T18:26:04.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.622+0000 7fb491106700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb4780a3d50 0x7fb4780a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb488006010 tx=0x7fb48801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:04.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.622+0000 7fb492108700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079] conn(0x7fb4780ad910 0x7fb4780afdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:04.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.622+0000 7fb492108700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079] conn(0x7fb4780ad910 0x7fb4780afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:04.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.623+0000 7fb482ffd700 1 -- 172.21.15.84:0/2250150566 <== osd.7 v2:172.21.15.103:6816/1738278079 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb4780b0480 con 0x7fb4780ad910 2024-04-26T18:26:04.624 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:04.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:04 smithi103 ceph-mon[32727]: pgmap v350: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:04.627 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:04.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.643+0000 7fb493b6b700 1 -- 172.21.15.84:0/2250150566 --> [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb470002e60 con 0x7fb4780ad910 2024-04-26T18:26:04.643 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.643+0000 7fb482ffd700 1 -- 172.21.15.84:0/2250150566 <== osd.7 v2:172.21.15.103:6816/1738278079 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb470002e60 con 0x7fb4780ad910 2024-04-26T18:26:04.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.644+0000 7fb493b6b700 1 -- 172.21.15.84:0/2250150566 >> [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079] conn(0x7fb4780ad910 msgr2=0x7fb4780afdb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:04.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.644+0000 7fb493b6b700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079] conn(0x7fb4780ad910 0x7fb4780afdb0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:04.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.644+0000 7fb493b6b700 1 -- 172.21.15.84:0/2250150566 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb4780a3d50 msgr2=0x7fb4780a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:04.644 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.644+0000 7fb493b6b700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb4780a3d50 0x7fb4780a6210 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fb488006010 tx=0x7fb48801a040 comp rx=0 tx=0).stop 2024-04-26T18:26:04.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.644+0000 7fb493b6b700 1 -- 172.21.15.84:0/2250150566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c101de0 msgr2=0x7fb48c19f8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:04.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.644+0000 7fb493b6b700 1 --2- 172.21.15.84:0/2250150566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb48c101de0 0x7fb48c19f8a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fb47c00b500 tx=0x7fb47c00b810 comp rx=0 tx=0).stop 2024-04-26T18:26:04.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.644+0000 7fb492108700 1 -- 172.21.15.84:0/2250150566 reap_dead start 2024-04-26T18:26:04.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.644+0000 7fb493b6b700 1 -- 172.21.15.84:0/2250150566 shutdown_connections 2024-04-26T18:26:04.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.644+0000 7fb493b6b700 1 -- 172.21.15.84:0/2250150566 >> 172.21.15.84:0/2250150566 conn(0x7fb48c077640 msgr2=0x7fb48c100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:04.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.644+0000 7fb493b6b700 1 -- 172.21.15.84:0/2250150566 shutdown_connections 2024-04-26T18:26:04.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:04.644+0000 7fb493b6b700 1 -- 172.21.15.84:0/2250150566 wait complete. 2024-04-26T18:26:04.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:04 smithi174 ceph-mon[32705]: pgmap v350: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:04.967 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:05.099 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:06.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:06 smithi084 ceph-mon[29141]: pgmap v351: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T18:26:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:06 smithi084 ceph-mon[29141]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-26T18:26:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:06 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-26T18:26:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:06 smithi103 ceph-mon[32727]: pgmap v351: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T18:26:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:06 smithi103 ceph-mon[32727]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-26T18:26:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:06 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-26T18:26:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:06 smithi174 ceph-mon[32705]: pgmap v351: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 10 op/s 2024-04-26T18:26:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:06 smithi174 ceph-mon[32705]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-04-26T18:26:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:06 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-26T18:26:07.322 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.322+0000 7f68ce8e7700 1 -- 172.21.15.84:0/198006476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c81084e0 msgr2=0x7f68c81088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.322+0000 7f68ce8e7700 1 --2- 172.21.15.84:0/198006476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c81084e0 0x7f68c81088c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009990 tx=0x7f68b8009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.322+0000 7f68ce8e7700 1 -- 172.21.15.84:0/198006476 shutdown_connections 2024-04-26T18:26:07.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.322+0000 7f68ce8e7700 1 --2- 172.21.15.84:0/198006476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c81084e0 0x7f68c81088c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.322+0000 7f68ce8e7700 1 --2- 172.21.15.84:0/198006476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68c8102780 0x7f68c81101f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.322+0000 7f68ce8e7700 1 --2- 172.21.15.84:0/198006476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c8101dc0 0x7f68c8102240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.322+0000 7f68ce8e7700 1 -- 172.21.15.84:0/198006476 >> 172.21.15.84:0/198006476 conn(0x7f68c8077640 msgr2=0x7f68c8077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:07.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.322+0000 7f68ce8e7700 1 -- 172.21.15.84:0/198006476 shutdown_connections 2024-04-26T18:26:07.323 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.323+0000 7f68ce8e7700 1 -- 172.21.15.84:0/198006476 wait complete. 2024-04-26T18:26:07.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.323+0000 7f68ce8e7700 1 Processor -- start 2024-04-26T18:26:07.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.323+0000 7f68ce8e7700 1 -- start start 2024-04-26T18:26:07.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68ce8e7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c8101dc0 0x7f68c819f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68ce8e7700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68c8102780 0x7f68c819fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68ce8e7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c81084e0 0x7f68c8199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68ce8e7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81136d0 con 0x7f68c81084e0 2024-04-26T18:26:07.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68ce8e7700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f68c8113550 con 0x7f68c8102780 2024-04-26T18:26:07.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68ce8e7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f68c8113850 con 0x7f68c8101dc0 2024-04-26T18:26:07.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68cce84700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c81084e0 0x7f68c8199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68cce84700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c81084e0 0x7f68c8199ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39866/0 (socket says 172.21.15.84:39866) 2024-04-26T18:26:07.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68cce84700 1 -- 172.21.15.84:0/3256033196 learned_addr learned my addr 172.21.15.84:0/3256033196 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:07.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68c7fff700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c8101dc0 0x7f68c819f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68c77fe700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68c8102780 0x7f68c819fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68cce84700 1 -- 172.21.15.84:0/3256033196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c8101dc0 msgr2=0x7f68c819f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68cce84700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c8101dc0 0x7f68c819f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68cce84700 1 -- 172.21.15.84:0/3256033196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68c8102780 msgr2=0x7f68c819fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68cce84700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68c8102780 0x7f68c819fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68cce84700 1 -- 172.21.15.84:0/3256033196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c819a230 con 0x7f68c81084e0 2024-04-26T18:26:07.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68cce84700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c81084e0 0x7f68c8199ad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f68c000b970 tx=0x7f68c000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:07.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.324+0000 7f68c57fa700 1 -- 172.21.15.84:0/3256033196 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0019070 con 0x7f68c81084e0 2024-04-26T18:26:07.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.325+0000 7f68c57fa700 1 -- 172.21.15.84:0/3256033196 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c000cde0 con 0x7f68c81084e0 2024-04-26T18:26:07.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.325+0000 7f68ce8e7700 1 -- 172.21.15.84:0/3256033196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c819a520 con 0x7f68c81084e0 2024-04-26T18:26:07.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.325+0000 7f68c57fa700 1 -- 172.21.15.84:0/3256033196 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0007370 con 0x7f68c81084e0 2024-04-26T18:26:07.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.325+0000 7f68ce8e7700 1 -- 172.21.15.84:0/3256033196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c81a67e0 con 0x7f68c81084e0 2024-04-26T18:26:07.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.326+0000 7f68ce8e7700 1 -- 172.21.15.84:0/3256033196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68c806a110 con 0x7f68c81084e0 2024-04-26T18:26:07.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.327+0000 7f68c57fa700 1 -- 172.21.15.84:0/3256033196 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f68c001c070 con 0x7f68c81084e0 2024-04-26T18:26:07.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.327+0000 7f68c57fa700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f68b00a3c40 0x7f68b00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:07.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.327+0000 7f68c7fff700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f68b00a3c40 0x7f68b00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:07.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.327+0000 7f68c57fa700 1 -- 172.21.15.84:0/3256033196 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f68c00cfff0 con 0x7f68c81084e0 2024-04-26T18:26:07.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.328+0000 7f68c7fff700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f68b00a3c40 0x7f68b00a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f68b8006010 tx=0x7f68b8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:07.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.331+0000 7f68c57fa700 1 -- 172.21.15.84:0/3256033196 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f68c0023070 con 0x7f68c81084e0 2024-04-26T18:26:07.537 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:07 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-26T18:26:07.537 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:07 smithi084 ceph-mon[29141]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T18:26:07.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.546+0000 7f68ce8e7700 1 -- 172.21.15.84:0/3256033196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f68c819afc0 con 0x7f68c81084e0 2024-04-26T18:26:07.546 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.546+0000 7f68c57fa700 1 -- 172.21.15.84:0/3256033196 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f68c009c9a0 con 0x7f68c81084e0 2024-04-26T18:26:07.546 INFO:teuthology.orchestra.run.smithi084.stdout:347892351013 2024-04-26T18:26:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.548+0000 7f68ce8e7700 1 -- 172.21.15.84:0/3256033196 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f68b00a3c40 msgr2=0x7f68b00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.548+0000 7f68ce8e7700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f68b00a3c40 0x7f68b00a6100 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f68b8006010 tx=0x7f68b8005960 comp rx=0 tx=0).stop 2024-04-26T18:26:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.548+0000 7f68ce8e7700 1 -- 172.21.15.84:0/3256033196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c81084e0 msgr2=0x7f68c8199ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.548+0000 7f68ce8e7700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c81084e0 0x7f68c8199ad0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f68c000b970 tx=0x7f68c000bc80 comp rx=0 tx=0).stop 2024-04-26T18:26:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.549+0000 7f68ce8e7700 1 -- 172.21.15.84:0/3256033196 shutdown_connections 2024-04-26T18:26:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.549+0000 7f68ce8e7700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f68b00a3c40 0x7f68b00a6100 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.549+0000 7f68ce8e7700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f68c81084e0 0x7f68c8199ad0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.549+0000 7f68ce8e7700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f68c8101dc0 0x7f68c819f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.549+0000 7f68ce8e7700 1 --2- 172.21.15.84:0/3256033196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f68c8102780 0x7f68c819fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.549+0000 7f68ce8e7700 1 -- 172.21.15.84:0/3256033196 >> 172.21.15.84:0/3256033196 conn(0x7f68c8077640 msgr2=0x7f68c8100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.549+0000 7f68ce8e7700 1 -- 172.21.15.84:0/3256033196 shutdown_connections 2024-04-26T18:26:07.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:07.549+0000 7f68ce8e7700 1 -- 172.21.15.84:0/3256033196 wait complete. 2024-04-26T18:26:07.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:07 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-26T18:26:07.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:07 smithi103 ceph-mon[32727]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T18:26:07.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:07 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-26T18:26:07.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:07 smithi174 ceph-mon[32705]: osdmap e92: 12 total, 12 up, 12 in 2024-04-26T18:26:08.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:08 smithi084 ceph-mon[29141]: pgmap v353: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T18:26:08.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:08 smithi084 ceph-mon[29141]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T18:26:08.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:08 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3256033196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T18:26:08.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:08 smithi103 ceph-mon[32727]: pgmap v353: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T18:26:08.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:08 smithi103 ceph-mon[32727]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T18:26:08.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:08 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3256033196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T18:26:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:08 smithi174 ceph-mon[32705]: pgmap v353: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T18:26:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:08 smithi174 ceph-mon[32705]: osdmap e93: 12 total, 12 up, 12 in 2024-04-26T18:26:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:08 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3256033196' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-26T18:26:09.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.158+0000 7fe4d878b700 1 -- 172.21.15.84:0/2226613647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4d01094c0 msgr2=0x7fe4d01098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.158+0000 7fe4d878b700 1 --2- 172.21.15.84:0/2226613647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4d01094c0 0x7fe4d01098a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe4c0009990 tx=0x7fe4c0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.158+0000 7fe4d878b700 1 -- 172.21.15.84:0/2226613647 shutdown_connections 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.158+0000 7fe4d878b700 1 --2- 172.21.15.84:0/2226613647 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4d01094c0 0x7fe4d01098a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.158+0000 7fe4d878b700 1 --2- 172.21.15.84:0/2226613647 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4d0103e30 0x7fe4d0110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.158+0000 7fe4d878b700 1 --2- 172.21.15.84:0/2226613647 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4d0103470 0x7fe4d01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.158+0000 7fe4d878b700 1 -- 172.21.15.84:0/2226613647 >> 172.21.15.84:0/2226613647 conn(0x7fe4d007a580 msgr2=0x7fe4d007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.158+0000 7fe4d878b700 1 -- 172.21.15.84:0/2226613647 shutdown_connections 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.158+0000 7fe4d878b700 1 -- 172.21.15.84:0/2226613647 wait complete. 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d878b700 1 Processor -- start 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d878b700 1 -- start start 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d878b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4d0103470 0x7fe4d019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d878b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4d0103e30 0x7fe4d019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d878b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4d01094c0 0x7fe4d0199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:09.166 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d878b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d0113640 con 0x7fe4d0103e30 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d878b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01134c0 con 0x7fe4d01094c0 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d878b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe4d01137c0 con 0x7fe4d0103470 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d5d26700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4d0103e30 0x7fe4d019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d5d26700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4d0103e30 0x7fe4d019fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39890/0 (socket says 172.21.15.84:39890) 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d5d26700 1 -- 172.21.15.84:0/1361681459 learned_addr learned my addr 172.21.15.84:0/1361681459 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d6527700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4d0103470 0x7fe4d019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d5d26700 1 -- 172.21.15.84:0/1361681459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4d0103470 msgr2=0x7fe4d019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d5d26700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe4d0103470 0x7fe4d019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d6d28700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4d01094c0 0x7fe4d0199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d5d26700 1 -- 172.21.15.84:0/1361681459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4d01094c0 msgr2=0x7fe4d0199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d5d26700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe4d01094c0 0x7fe4d0199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d5d26700 1 -- 172.21.15.84:0/1361681459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4d019a0c0 con 0x7fe4d0103e30 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.159+0000 7fe4d5d26700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4d0103e30 0x7fe4d019fce0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c9b0 tx=0x7fe4cc00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:09.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.160+0000 7fe4c77fe700 1 -- 172.21.15.84:0/1361681459 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc007b00 con 0x7fe4d0103e30 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.160+0000 7fe4d878b700 1 -- 172.21.15.84:0/1361681459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4d019a3b0 con 0x7fe4d0103e30 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.160+0000 7fe4c77fe700 1 -- 172.21.15.84:0/1361681459 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4cc007ca0 con 0x7fe4d0103e30 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.160+0000 7fe4c77fe700 1 -- 172.21.15.84:0/1361681459 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc018730 con 0x7fe4d0103e30 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.160+0000 7fe4d878b700 1 -- 172.21.15.84:0/1361681459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4d01a6670 con 0x7fe4d0103e30 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.160+0000 7fe4d878b700 1 -- 172.21.15.84:0/1361681459 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe4d0063b30 con 0x7fe4d0103e30 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.162+0000 7fe4c77fe700 1 -- 172.21.15.84:0/1361681459 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe4cc00f450 con 0x7fe4d0103e30 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.162+0000 7fe4c77fe700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe4bc0a3c40 0x7fe4bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.162+0000 7fe4d6527700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe4bc0a3c40 0x7fe4bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.162+0000 7fe4c77fe700 1 -- 172.21.15.84:0/1361681459 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fe4cc0d0940 con 0x7fe4d0103e30 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.162+0000 7fe4c77fe700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256] conn(0x7fe4bc0ad880 0x7fe4bc0afd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.162+0000 7fe4c77fe700 1 -- 172.21.15.84:0/1361681459 --> [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe4cc004500 con 0x7fe4bc0ad880 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.162+0000 7fe4c77fe700 1 -- 172.21.15.84:0/1361681459 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_get_version_reply(handle=1 version=93) ==== 24+0+0 (secure 0 0 0) 0x7fe4cc0d0d90 con 0x7fe4d0103e30 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.163+0000 7fe4d6d28700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256] conn(0x7fe4bc0ad880 0x7fe4bc0afd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.163+0000 7fe4d6527700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe4bc0a3c40 0x7fe4bc0a6100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000b5c0 tx=0x7fe4c0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.163+0000 7fe4d6d28700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256] conn(0x7fe4bc0ad880 0x7fe4bc0afd20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:09.168 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.163+0000 7fe4c77fe700 1 -- 172.21.15.84:0/1361681459 <== osd.3 v2:172.21.15.84:6810/2352789256 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe4cc004500 con 0x7fe4bc0ad880 2024-04-26T18:26:09.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.181+0000 7fe4d878b700 1 -- 172.21.15.84:0/1361681459 --> [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe4d0063f30 con 0x7fe4bc0ad880 2024-04-26T18:26:09.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.181+0000 7fe4c77fe700 1 -- 172.21.15.84:0/1361681459 <== osd.3 v2:172.21.15.84:6810/2352789256 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe4d0063f30 con 0x7fe4bc0ad880 2024-04-26T18:26:09.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.181+0000 7fe4d878b700 1 -- 172.21.15.84:0/1361681459 >> [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256] conn(0x7fe4bc0ad880 msgr2=0x7fe4bc0afd20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:09.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.181+0000 7fe4d878b700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256] conn(0x7fe4bc0ad880 0x7fe4bc0afd20 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:09.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.181+0000 7fe4d878b700 1 -- 172.21.15.84:0/1361681459 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe4bc0a3c40 msgr2=0x7fe4bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:09.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.181+0000 7fe4d878b700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe4bc0a3c40 0x7fe4bc0a6100 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000b5c0 tx=0x7fe4c0005fb0 comp rx=0 tx=0).stop 2024-04-26T18:26:09.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.181+0000 7fe4d878b700 1 -- 172.21.15.84:0/1361681459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4d0103e30 msgr2=0x7fe4d019fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:09.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.181+0000 7fe4d878b700 1 --2- 172.21.15.84:0/1361681459 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe4d0103e30 0x7fe4d019fce0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc00c9b0 tx=0x7fe4cc00cd70 comp rx=0 tx=0).stop 2024-04-26T18:26:09.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.181+0000 7fe4d6d28700 1 -- 172.21.15.84:0/1361681459 reap_dead start 2024-04-26T18:26:09.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.181+0000 7fe4d878b700 1 -- 172.21.15.84:0/1361681459 shutdown_connections 2024-04-26T18:26:09.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.181+0000 7fe4d878b700 1 -- 172.21.15.84:0/1361681459 >> 172.21.15.84:0/1361681459 conn(0x7fe4d007a580 msgr2=0x7fe4d01021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:09.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.182+0000 7fe4d878b700 1 -- 172.21.15.84:0/1361681459 shutdown_connections 2024-04-26T18:26:09.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:09.182+0000 7fe4d878b700 1 -- 172.21.15.84:0/1361681459 wait complete. 2024-04-26T18:26:09.705 INFO:teuthology.orchestra.run.smithi084.stdout:223338299448 2024-04-26T18:26:09.705 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-26T18:26:10.462 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892351011 got 347892351013 for osd.11 2024-04-26T18:26:10.462 DEBUG:teuthology.parallel:result is None 2024-04-26T18:26:10.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:10 smithi084 ceph-mon[29141]: pgmap v355: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-26T18:26:10.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:10 smithi103 ceph-mon[32727]: pgmap v355: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-26T18:26:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:10 smithi174 ceph-mon[32705]: pgmap v355: 97 pgs: 97 active+clean; 647 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-26T18:26:11.597 INFO:teuthology.orchestra.run.smithi084.stdout:111669149774 2024-04-26T18:26:11.597 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T18:26:12.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.048+0000 7faa4f6fb700 1 -- 172.21.15.84:0/4247772032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa48102f60 msgr2=0x7faa48106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.048+0000 7faa4f6fb700 1 --2- 172.21.15.84:0/4247772032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa48102f60 0x7faa48106f60 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faa380099c0 tx=0x7faa38009cd0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.048+0000 7faa4f6fb700 1 -- 172.21.15.84:0/4247772032 shutdown_connections 2024-04-26T18:26:12.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.048+0000 7faa4f6fb700 1 --2- 172.21.15.84:0/4247772032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa48107640 0x7faa4810f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.048+0000 7faa4f6fb700 1 --2- 172.21.15.84:0/4247772032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa481025b0 0x7faa48102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.048+0000 7faa4f6fb700 1 --2- 172.21.15.84:0/4247772032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa48102f60 0x7faa48106f60 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.048+0000 7faa4f6fb700 1 -- 172.21.15.84:0/4247772032 >> 172.21.15.84:0/4247772032 conn(0x7faa480792d0 msgr2=0x7faa480796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:12.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.049+0000 7faa4f6fb700 1 -- 172.21.15.84:0/4247772032 shutdown_connections 2024-04-26T18:26:12.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.049+0000 7faa4f6fb700 1 -- 172.21.15.84:0/4247772032 wait complete. 2024-04-26T18:26:12.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.049+0000 7faa4f6fb700 1 Processor -- start 2024-04-26T18:26:12.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.049+0000 7faa4f6fb700 1 -- start start 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.049+0000 7faa4f6fb700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa481025b0 0x7faa4819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.049+0000 7faa4f6fb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa48102f60 0x7faa4819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.049+0000 7faa4f6fb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa48107640 0x7faa48199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.049+0000 7faa4f6fb700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faa481125b0 con 0x7faa48102f60 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.049+0000 7faa4f6fb700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faa48112430 con 0x7faa481025b0 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.049+0000 7faa4f6fb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faa48112730 con 0x7faa48107640 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.049+0000 7faa4cc96700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa48102f60 0x7faa4819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.049+0000 7faa4cc96700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa48102f60 0x7faa4819fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39904/0 (socket says 172.21.15.84:39904) 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.049+0000 7faa4cc96700 1 -- 172.21.15.84:0/3925577997 learned_addr learned my addr 172.21.15.84:0/3925577997 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa4d497700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa481025b0 0x7faa4819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa4dc98700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa48107640 0x7faa48199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa4cc96700 1 -- 172.21.15.84:0/3925577997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa48107640 msgr2=0x7faa48199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa4cc96700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa48107640 0x7faa48199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa4cc96700 1 -- 172.21.15.84:0/3925577997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa481025b0 msgr2=0x7faa4819f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa4cc96700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa481025b0 0x7faa4819f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa4cc96700 1 -- 172.21.15.84:0/3925577997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4819a180 con 0x7faa48102f60 2024-04-26T18:26:12.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa4d497700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa481025b0 0x7faa4819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:12.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa4dc98700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa48107640 0x7faa48199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:12.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa4cc96700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa48102f60 0x7faa4819fda0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faa38009f50 tx=0x7faa380037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:12.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa3e7fc700 1 -- 172.21.15.84:0/3925577997 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa38027070 con 0x7faa48102f60 2024-04-26T18:26:12.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa3e7fc700 1 -- 172.21.15.84:0/3925577997 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa38004140 con 0x7faa48102f60 2024-04-26T18:26:12.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa4f6fb700 1 -- 172.21.15.84:0/3925577997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4819a410 con 0x7faa48102f60 2024-04-26T18:26:12.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa3e7fc700 1 -- 172.21.15.84:0/3925577997 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3800f460 con 0x7faa48102f60 2024-04-26T18:26:12.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.050+0000 7faa4f6fb700 1 -- 172.21.15.84:0/3925577997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa481a6730 con 0x7faa48102f60 2024-04-26T18:26:12.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.051+0000 7faa4f6fb700 1 -- 172.21.15.84:0/3925577997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa48103a40 con 0x7faa48102f60 2024-04-26T18:26:12.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.052+0000 7faa3e7fc700 1 -- 172.21.15.84:0/3925577997 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faa3802a070 con 0x7faa48102f60 2024-04-26T18:26:12.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.052+0000 7faa3e7fc700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faa340a3c40 0x7faa340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:12.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.053+0000 7faa4d497700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faa340a3c40 0x7faa340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:12.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.053+0000 7faa3e7fc700 1 -- 172.21.15.84:0/3925577997 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7faa380d13b0 con 0x7faa48102f60 2024-04-26T18:26:12.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.053+0000 7faa4d497700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faa340a3c40 0x7faa340a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa4819ab60 tx=0x7faa44008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:12.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.057+0000 7faa3e7fc700 1 -- 172.21.15.84:0/3925577997 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa38013070 con 0x7faa48102f60 2024-04-26T18:26:12.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.265+0000 7faa4f6fb700 1 -- 172.21.15.84:0/3925577997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7faa4819af50 con 0x7faa48102f60 2024-04-26T18:26:12.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.265+0000 7faa3e7fc700 1 -- 172.21.15.84:0/3925577997 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faa3809ddf0 con 0x7faa48102f60 2024-04-26T18:26:12.266 INFO:teuthology.orchestra.run.smithi084.stdout:201863462976 2024-04-26T18:26:12.268 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.267+0000 7faa4f6fb700 1 -- 172.21.15.84:0/3925577997 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faa340a3c40 msgr2=0x7faa340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.267+0000 7faa4f6fb700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faa340a3c40 0x7faa340a6100 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7faa4819ab60 tx=0x7faa44008040 comp rx=0 tx=0).stop 2024-04-26T18:26:12.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.267+0000 7faa4f6fb700 1 -- 172.21.15.84:0/3925577997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa48102f60 msgr2=0x7faa4819fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:12.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.267+0000 7faa4f6fb700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa48102f60 0x7faa4819fda0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7faa38009f50 tx=0x7faa380037c0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.267+0000 7faa4f6fb700 1 -- 172.21.15.84:0/3925577997 shutdown_connections 2024-04-26T18:26:12.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.267+0000 7faa4f6fb700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7faa340a3c40 0x7faa340a6100 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.267+0000 7faa4f6fb700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faa48107640 0x7faa48199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.267+0000 7faa4f6fb700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faa481025b0 0x7faa4819f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.267+0000 7faa4f6fb700 1 --2- 172.21.15.84:0/3925577997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faa48102f60 0x7faa4819fda0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:12.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.268+0000 7faa4f6fb700 1 -- 172.21.15.84:0/3925577997 >> 172.21.15.84:0/3925577997 conn(0x7faa480792d0 msgr2=0x7faa4806bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:12.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.268+0000 7faa4f6fb700 1 -- 172.21.15.84:0/3925577997 shutdown_connections 2024-04-26T18:26:12.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:12.268+0000 7faa4f6fb700 1 -- 172.21.15.84:0/3925577997 wait complete. 2024-04-26T18:26:12.552 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:12 smithi084 ceph-mon[29141]: pgmap v356: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2024-04-26T18:26:12.552 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:12 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3925577997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T18:26:12.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:12 smithi103 ceph-mon[32727]: pgmap v356: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2024-04-26T18:26:12.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:12 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3925577997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T18:26:12.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:12 smithi174 ceph-mon[32705]: pgmap v356: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 895 B/s wr, 13 op/s 2024-04-26T18:26:12.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:12 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3925577997' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-26T18:26:13.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:13 smithi103 ceph-mon[32727]: pgmap v357: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 9 op/s 2024-04-26T18:26:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:13 smithi174 ceph-mon[32705]: pgmap v357: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 9 op/s 2024-04-26T18:26:13.717 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:13 smithi084 ceph-mon[29141]: pgmap v357: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 895 B/s wr, 9 op/s 2024-04-26T18:26:15.175 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:15.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.724+0000 7ff62cf83700 1 -- 172.21.15.84:0/166374438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628102ce0 msgr2=0x7ff62810f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:15.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.724+0000 7ff62cf83700 1 --2- 172.21.15.84:0/166374438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628102ce0 0x7ff62810f1d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff62000b0a0 tx=0x7ff62000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.724+0000 7ff62cf83700 1 -- 172.21.15.84:0/166374438 shutdown_connections 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.724+0000 7ff62cf83700 1 --2- 172.21.15.84:0/166374438 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628102ce0 0x7ff62810f1d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.724+0000 7ff62cf83700 1 --2- 172.21.15.84:0/166374438 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff628108390 0x7ff628108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.724+0000 7ff62cf83700 1 --2- 172.21.15.84:0/166374438 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff628102320 0x7ff6281027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.724+0000 7ff62cf83700 1 -- 172.21.15.84:0/166374438 >> 172.21.15.84:0/166374438 conn(0x7ff6280792d0 msgr2=0x7ff6280796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.724+0000 7ff62cf83700 1 -- 172.21.15.84:0/166374438 shutdown_connections 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.724+0000 7ff62cf83700 1 -- 172.21.15.84:0/166374438 wait complete. 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff62cf83700 1 Processor -- start 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff62cf83700 1 -- start start 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff62cf83700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff628102320 0x7ff62819a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff62cf83700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff628102ce0 0x7ff62819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff62cf83700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628108390 0x7ff6281a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff62cf83700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff6281124e0 con 0x7ff628108390 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff62cf83700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff628112360 con 0x7ff628102ce0 2024-04-26T18:26:15.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff62cf83700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff628112660 con 0x7ff628102320 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff626d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628108390 0x7ff6281a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff626d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628108390 0x7ff6281a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39948/0 (socket says 172.21.15.84:39948) 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff626d9d700 1 -- 172.21.15.84:0/1382394838 learned_addr learned my addr 172.21.15.84:0/1382394838 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff62659c700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff628102320 0x7ff62819a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff625d9b700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff628102ce0 0x7ff62819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff626d9d700 1 -- 172.21.15.84:0/1382394838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff628102320 msgr2=0x7ff62819a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff626d9d700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff628102320 0x7ff62819a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff626d9d700 1 -- 172.21.15.84:0/1382394838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff628102ce0 msgr2=0x7ff62819ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff626d9d700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff628102ce0 0x7ff62819ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff626d9d700 1 -- 172.21.15.84:0/1382394838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6281a5e80 con 0x7ff628108390 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.725+0000 7ff625d9b700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff628102ce0 0x7ff62819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.726+0000 7ff626d9d700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628108390 0x7ff6281a5940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6200062a0 tx=0x7ff620004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.726+0000 7ff6177fe700 1 -- 172.21.15.84:0/1382394838 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620021070 con 0x7ff628108390 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.726+0000 7ff62cf83700 1 -- 172.21.15.84:0/1382394838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6281a6050 con 0x7ff628108390 2024-04-26T18:26:15.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.726+0000 7ff6177fe700 1 -- 172.21.15.84:0/1382394838 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff620003c00 con 0x7ff628108390 2024-04-26T18:26:15.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.726+0000 7ff6177fe700 1 -- 172.21.15.84:0/1382394838 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff620004830 con 0x7ff628108390 2024-04-26T18:26:15.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.726+0000 7ff62cf83700 1 -- 172.21.15.84:0/1382394838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6281a6500 con 0x7ff628108390 2024-04-26T18:26:15.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.727+0000 7ff62cf83700 1 -- 172.21.15.84:0/1382394838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff62806a110 con 0x7ff628108390 2024-04-26T18:26:15.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.729+0000 7ff6177fe700 1 -- 172.21.15.84:0/1382394838 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff620007a00 con 0x7ff628108390 2024-04-26T18:26:15.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.729+0000 7ff6177fe700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff6100a3c40 0x7ff6100a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:15.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.729+0000 7ff62659c700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff6100a3c40 0x7ff6100a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:15.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.729+0000 7ff6177fe700 1 -- 172.21.15.84:0/1382394838 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7ff6200d1490 con 0x7ff628108390 2024-04-26T18:26:15.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.730+0000 7ff62659c700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff6100a3c40 0x7ff6100a6100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff618005950 tx=0x7ff6180058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:15.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.733+0000 7ff6177fe700 1 -- 172.21.15.84:0/1382394838 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff62009ded0 con 0x7ff628108390 2024-04-26T18:26:15.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.942+0000 7ff62cf83700 1 -- 172.21.15.84:0/1382394838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ff62802e490 con 0x7ff628108390 2024-04-26T18:26:15.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.942+0000 7ff6177fe700 1 -- 172.21.15.84:0/1382394838 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff62009d7d0 con 0x7ff628108390 2024-04-26T18:26:15.944 INFO:teuthology.orchestra.run.smithi084.stdout:146028888138 2024-04-26T18:26:15.945 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.944+0000 7ff62cf83700 1 -- 172.21.15.84:0/1382394838 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff6100a3c40 msgr2=0x7ff6100a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.944+0000 7ff62cf83700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff6100a3c40 0x7ff6100a6100 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff618005950 tx=0x7ff6180058e0 comp rx=0 tx=0).stop 2024-04-26T18:26:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.944+0000 7ff62cf83700 1 -- 172.21.15.84:0/1382394838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628108390 msgr2=0x7ff6281a5940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.944+0000 7ff62cf83700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628108390 0x7ff6281a5940 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff6200062a0 tx=0x7ff620004d30 comp rx=0 tx=0).stop 2024-04-26T18:26:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.944+0000 7ff62cf83700 1 -- 172.21.15.84:0/1382394838 shutdown_connections 2024-04-26T18:26:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.944+0000 7ff62cf83700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff6100a3c40 0x7ff6100a6100 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.944+0000 7ff62cf83700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff628108390 0x7ff6281a5940 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.944+0000 7ff62cf83700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff628102320 0x7ff62819a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.944+0000 7ff62cf83700 1 --2- 172.21.15.84:0/1382394838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff628102ce0 0x7ff62819ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.944+0000 7ff62cf83700 1 -- 172.21.15.84:0/1382394838 >> 172.21.15.84:0/1382394838 conn(0x7ff6280792d0 msgr2=0x7ff628100e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.944+0000 7ff62cf83700 1 -- 172.21.15.84:0/1382394838 shutdown_connections 2024-04-26T18:26:15.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.944+0000 7ff62cf83700 1 -- 172.21.15.84:0/1382394838 wait complete. 2024-04-26T18:26:16.360 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462970 got 201863462976 for osd.6 2024-04-26T18:26:16.361 DEBUG:teuthology.parallel:result is None 2024-04-26T18:26:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.618+0000 7f1b4e86b700 1 -- 172.21.15.84:0/74844236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b48103e90 msgr2=0x7f1b481102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.618+0000 7f1b4e86b700 1 --2- 172.21.15.84:0/74844236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b48103e90 0x7f1b481102c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000b0a0 tx=0x7f1b4000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:26:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.618+0000 7f1b4e86b700 1 -- 172.21.15.84:0/74844236 shutdown_connections 2024-04-26T18:26:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.618+0000 7f1b4e86b700 1 --2- 172.21.15.84:0/74844236 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b48103e90 0x7f1b481102c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.618+0000 7f1b4e86b700 1 --2- 172.21.15.84:0/74844236 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b481094f0 0x7f1b481098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.618+0000 7f1b4e86b700 1 --2- 172.21.15.84:0/74844236 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b481034d0 0x7f1b48103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.618+0000 7f1b4e86b700 1 -- 172.21.15.84:0/74844236 >> 172.21.15.84:0/74844236 conn(0x7f1b4807a4b0 msgr2=0x7f1b4807a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.618+0000 7f1b4e86b700 1 -- 172.21.15.84:0/74844236 shutdown_connections 2024-04-26T18:26:16.451 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.618+0000 7f1b4e86b700 1 -- 172.21.15.84:0/74844236 wait complete. 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.619+0000 7f1b4e86b700 1 Processor -- start 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.619+0000 7f1b4e86b700 1 -- start start 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.619+0000 7f1b4e86b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b481034d0 0x7f1b4819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.619+0000 7f1b4e86b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b48103e90 0x7f1b4819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.619+0000 7f1b4e86b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b481094f0 0x7f1b48199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.619+0000 7f1b4e86b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1b481135f0 con 0x7f1b481034d0 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.619+0000 7f1b4e86b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48113470 con 0x7f1b48103e90 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.619+0000 7f1b4e86b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1b48113770 con 0x7f1b481094f0 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.619+0000 7f1b47fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b481034d0 0x7f1b4819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.619+0000 7f1b47fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b481034d0 0x7f1b4819f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39922/0 (socket says 172.21.15.84:39922) 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.619+0000 7f1b47fff700 1 -- 172.21.15.84:0/2747049263 learned_addr learned my addr 172.21.15.84:0/2747049263 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.620+0000 7f1b47fff700 1 -- 172.21.15.84:0/2747049263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b481094f0 msgr2=0x7f1b48199870 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.620+0000 7f1b47fff700 1 --2- 172.21.15.84:0/2747049263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b481094f0 0x7f1b48199870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:16.452 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.620+0000 7f1b47fff700 1 -- 172.21.15.84:0/2747049263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b48103e90 msgr2=0x7f1b4819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:16.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.620+0000 7f1b47fff700 1 --2- 172.21.15.84:0/2747049263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b48103e90 0x7f1b4819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:16.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.620+0000 7f1b47fff700 1 -- 172.21.15.84:0/2747049263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b48199fd0 con 0x7f1b481034d0 2024-04-26T18:26:16.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.620+0000 7f1b47fff700 1 --2- 172.21.15.84:0/2747049263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b481034d0 0x7f1b4819f6b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c920 tx=0x7f1b3800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:16.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.620+0000 7f1b45ffb700 1 -- 172.21.15.84:0/2747049263 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38007b00 con 0x7f1b481034d0 2024-04-26T18:26:16.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.620+0000 7f1b45ffb700 1 -- 172.21.15.84:0/2747049263 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b38004ce0 con 0x7f1b481034d0 2024-04-26T18:26:16.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.620+0000 7f1b45ffb700 1 -- 172.21.15.84:0/2747049263 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b38005650 con 0x7f1b481034d0 2024-04-26T18:26:16.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.620+0000 7f1b4e86b700 1 -- 172.21.15.84:0/2747049263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b4819a2c0 con 0x7f1b481034d0 2024-04-26T18:26:16.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.620+0000 7f1b4e86b700 1 -- 172.21.15.84:0/2747049263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b481a6580 con 0x7f1b481034d0 2024-04-26T18:26:16.453 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.622+0000 7f1b3effd700 1 -- 172.21.15.84:0/2747049263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b2c005330 con 0x7f1b481034d0 2024-04-26T18:26:16.453 INFO:teuthology.orchestra.run.smithi084.stdout:34359738462 2024-04-26T18:26:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.624+0000 7f1b45ffb700 1 -- 172.21.15.84:0/2747049263 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1b38004750 con 0x7f1b481034d0 2024-04-26T18:26:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.625+0000 7f1b45ffb700 1 --2- 172.21.15.84:0/2747049263 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1b340a3cc0 0x7f1b340a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.625+0000 7f1b3ffff700 1 --2- 172.21.15.84:0/2747049263 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1b340a3cc0 0x7f1b340a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.625+0000 7f1b45ffb700 1 -- 172.21.15.84:0/2747049263 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f1b380d04a0 con 0x7f1b481034d0 2024-04-26T18:26:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.626+0000 7f1b3ffff700 1 --2- 172.21.15.84:0/2747049263 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1b340a3cc0 0x7f1b340a6180 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1b30005950 tx=0x7f1b300058e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.629+0000 7f1b45ffb700 1 -- 172.21.15.84:0/2747049263 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b38014030 con 0x7f1b481034d0 2024-04-26T18:26:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.841+0000 7f1b3effd700 1 -- 172.21.15.84:0/2747049263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f1b2c0058f0 con 0x7f1b481034d0 2024-04-26T18:26:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.842+0000 7f1b45ffb700 1 -- 172.21.15.84:0/2747049263 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1b3809ced0 con 0x7f1b481034d0 2024-04-26T18:26:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.844+0000 7f1b3effd700 1 -- 172.21.15.84:0/2747049263 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1b340a3cc0 msgr2=0x7f1b340a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.844+0000 7f1b3effd700 1 --2- 172.21.15.84:0/2747049263 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1b340a3cc0 0x7f1b340a6180 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f1b30005950 tx=0x7f1b300058e0 comp rx=0 tx=0).stop 2024-04-26T18:26:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.844+0000 7f1b3effd700 1 -- 172.21.15.84:0/2747049263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b481034d0 msgr2=0x7f1b4819f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:16.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.844+0000 7f1b3effd700 1 --2- 172.21.15.84:0/2747049263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b481034d0 0x7f1b4819f6b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b3800c920 tx=0x7f1b3800cce0 comp rx=0 tx=0).stop 2024-04-26T18:26:16.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.844+0000 7f1b3effd700 1 -- 172.21.15.84:0/2747049263 shutdown_connections 2024-04-26T18:26:16.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.844+0000 7f1b3effd700 1 --2- 172.21.15.84:0/2747049263 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f1b340a3cc0 0x7f1b340a6180 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:16.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.844+0000 7f1b3effd700 1 --2- 172.21.15.84:0/2747049263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b481034d0 0x7f1b4819f6b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:16.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.844+0000 7f1b3effd700 1 --2- 172.21.15.84:0/2747049263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b481094f0 0x7f1b48199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:16.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.844+0000 7f1b3effd700 1 --2- 172.21.15.84:0/2747049263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b48103e90 0x7f1b4819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:16.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.844+0000 7f1b3effd700 1 -- 172.21.15.84:0/2747049263 >> 172.21.15.84:0/2747049263 conn(0x7f1b4807a4b0 msgr2=0x7f1b48102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:16.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.844+0000 7f1b3effd700 1 -- 172.21.15.84:0/2747049263 shutdown_connections 2024-04-26T18:26:16.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:15.844+0000 7f1b3effd700 1 -- 172.21.15.84:0/2747049263 wait complete. 2024-04-26T18:26:16.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:16 smithi084 ceph-mon[29141]: pgmap v358: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 803 B/s wr, 10 op/s 2024-04-26T18:26:16.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:16 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2747049263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T18:26:16.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:16 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1382394838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T18:26:16.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:16 smithi103 ceph-mon[32727]: pgmap v358: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 803 B/s wr, 10 op/s 2024-04-26T18:26:16.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:16 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2747049263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T18:26:16.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:16 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1382394838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T18:26:16.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:16 smithi174 ceph-mon[32705]: pgmap v358: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 803 B/s wr, 10 op/s 2024-04-26T18:26:16.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:16 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2747049263' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T18:26:16.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:16 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1382394838' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T18:26:17.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.353+0000 7f47b5ef6700 1 -- 172.21.15.84:0/3548798504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0102350 msgr2=0x7f47b01027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:17.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.353+0000 7f47b5ef6700 1 --2- 172.21.15.84:0/3548798504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0102350 0x7f47b01027d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f47a4009990 tx=0x7f47a4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.353+0000 7f47b5ef6700 1 -- 172.21.15.84:0/3548798504 shutdown_connections 2024-04-26T18:26:17.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.353+0000 7f47b5ef6700 1 --2- 172.21.15.84:0/3548798504 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47b0102d10 0x7f47b010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.353+0000 7f47b5ef6700 1 --2- 172.21.15.84:0/3548798504 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0104270 0x7f47b0104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.353+0000 7f47b5ef6700 1 --2- 172.21.15.84:0/3548798504 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0102350 0x7f47b01027d0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.353+0000 7f47b5ef6700 1 -- 172.21.15.84:0/3548798504 >> 172.21.15.84:0/3548798504 conn(0x7f47b0078390 msgr2=0x7f47b00787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:17.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.353+0000 7f47b5ef6700 1 -- 172.21.15.84:0/3548798504 shutdown_connections 2024-04-26T18:26:17.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.353+0000 7f47b5ef6700 1 -- 172.21.15.84:0/3548798504 wait complete. 2024-04-26T18:26:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.354+0000 7f47b5ef6700 1 Processor -- start 2024-04-26T18:26:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.354+0000 7f47b5ef6700 1 -- start start 2024-04-26T18:26:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.354+0000 7f47b5ef6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0102350 0x7f47b019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.354+0000 7f47b5ef6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0102d10 0x7f47b019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.354+0000 7f47b5ef6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47b0104270 0x7f47b01a5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.354+0000 7f47b5ef6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f47b01125b0 con 0x7f47b0102350 2024-04-26T18:26:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.354+0000 7f47b5ef6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47b0112430 con 0x7f47b0104270 2024-04-26T18:26:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.354+0000 7f47b5ef6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47b0112730 con 0x7f47b0102d10 2024-04-26T18:26:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.354+0000 7f47af7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0102350 0x7f47b019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:17.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.354+0000 7f47af7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0102350 0x7f47b019a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46520/0 (socket says 172.21.15.84:46520) 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.354+0000 7f47af7fe700 1 -- 172.21.15.84:0/251277922 learned_addr learned my addr 172.21.15.84:0/251277922 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.354+0000 7f47affff700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47b0104270 0x7f47b01a5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.355+0000 7f47aeffd700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0102d10 0x7f47b019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.355+0000 7f47af7fe700 1 -- 172.21.15.84:0/251277922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0102d10 msgr2=0x7f47b019af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.355+0000 7f47af7fe700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0102d10 0x7f47b019af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.355+0000 7f47af7fe700 1 -- 172.21.15.84:0/251277922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47b0104270 msgr2=0x7f47b01a5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.355+0000 7f47af7fe700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47b0104270 0x7f47b01a5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.355+0000 7f47af7fe700 1 -- 172.21.15.84:0/251277922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47b01a5f90 con 0x7f47b0102350 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.355+0000 7f47af7fe700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0102350 0x7f47b019a9d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f47a000d950 tx=0x7f47a000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.355+0000 7f47acff9700 1 -- 172.21.15.84:0/251277922 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a0016070 con 0x7f47b0102350 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.355+0000 7f47b5ef6700 1 -- 172.21.15.84:0/251277922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47b01a61c0 con 0x7f47b0102350 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.355+0000 7f47acff9700 1 -- 172.21.15.84:0/251277922 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47a0009df0 con 0x7f47b0102350 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.355+0000 7f47acff9700 1 -- 172.21.15.84:0/251277922 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a000f3c0 con 0x7f47b0102350 2024-04-26T18:26:17.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.355+0000 7f47b5ef6700 1 -- 172.21.15.84:0/251277922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47b01a6720 con 0x7f47b0102350 2024-04-26T18:26:17.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.356+0000 7f47b5ef6700 1 -- 172.21.15.84:0/251277922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47b006a110 con 0x7f47b0102350 2024-04-26T18:26:17.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.357+0000 7f47acff9700 1 -- 172.21.15.84:0/251277922 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f47a0019070 con 0x7f47b0102350 2024-04-26T18:26:17.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.358+0000 7f47acff9700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f47980a3c40 0x7f47980a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:17.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.358+0000 7f47aeffd700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f47980a3c40 0x7f47980a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:17.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.358+0000 7f47acff9700 1 -- 172.21.15.84:0/251277922 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f47a00cff80 con 0x7f47b0102350 2024-04-26T18:26:17.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.358+0000 7f47aeffd700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f47980a3c40 0x7f47980a6100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f47a4006010 tx=0x7f47a401a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:17.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.363+0000 7f47acff9700 1 -- 172.21.15.84:0/251277922 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47a009c930 con 0x7f47b0102350 2024-04-26T18:26:17.395 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:17.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.567+0000 7facfcddd700 1 -- 172.21.15.84:0/1809983365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf810a410 msgr2=0x7facf8116920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:17.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.567+0000 7facfcddd700 1 --2- 172.21.15.84:0/1809983365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf810a410 0x7facf8116920 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7facf000b0a0 tx=0x7facf000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.567+0000 7facfcddd700 1 -- 172.21.15.84:0/1809983365 shutdown_connections 2024-04-26T18:26:17.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.567+0000 7facfcddd700 1 --2- 172.21.15.84:0/1809983365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf810a410 0x7facf8116920 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.567+0000 7facfcddd700 1 --2- 172.21.15.84:0/1809983365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf810fa50 0x7facf810fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.567+0000 7facfcddd700 1 --2- 172.21.15.84:0/1809983365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facf8109a50 0x7facf8109ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.567+0000 7facfcddd700 1 -- 172.21.15.84:0/1809983365 >> 172.21.15.84:0/1809983365 conn(0x7facf8076bf0 msgr2=0x7facf8079010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:17.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.567+0000 7facfcddd700 1 -- 172.21.15.84:0/1809983365 shutdown_connections 2024-04-26T18:26:17.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.567+0000 7facfcddd700 1 -- 172.21.15.84:0/1809983365 wait complete. 2024-04-26T18:26:17.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.568+0000 7facfcddd700 1 Processor -- start 2024-04-26T18:26:17.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.568+0000 7facfcddd700 1 -- start start 2024-04-26T18:26:17.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.568+0000 7facfcddd700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facf8109a50 0x7facf81a00e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:17.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.568+0000 7facfcddd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf810a410 0x7facf81a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:17.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.568+0000 7facfcddd700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf810fa50 0x7facf81a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:17.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.568+0000 7facfcddd700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7facf8119c30 con 0x7facf810a410 2024-04-26T18:26:17.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.568+0000 7facfcddd700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7facf8119ab0 con 0x7facf810fa50 2024-04-26T18:26:17.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.568+0000 7facfcddd700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7facf8119db0 con 0x7facf8109a50 2024-04-26T18:26:17.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facf5d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf810a410 0x7facf81a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:17.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facf5d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf810a410 0x7facf81a0620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46530/0 (socket says 172.21.15.84:46530) 2024-04-26T18:26:17.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facf5d9b700 1 -- 172.21.15.84:0/2887067564 learned_addr learned my addr 172.21.15.84:0/2887067564 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:17.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facf659c700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facf8109a50 0x7facf81a00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:17.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facf6d9d700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf810fa50 0x7facf81a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:17.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facf5d9b700 1 -- 172.21.15.84:0/2887067564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facf8109a50 msgr2=0x7facf81a00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:17.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facf5d9b700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facf8109a50 0x7facf81a00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facf5d9b700 1 -- 172.21.15.84:0/2887067564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf810fa50 msgr2=0x7facf81a49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:17.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facf5d9b700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf810fa50 0x7facf81a49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facf5d9b700 1 -- 172.21.15.84:0/2887067564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facf81a50f0 con 0x7facf810a410 2024-04-26T18:26:17.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facf5d9b700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf810a410 0x7facf81a0620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7face000d950 tx=0x7face000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:17.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facef7fe700 1 -- 172.21.15.84:0/2887067564 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0016070 con 0x7facf810a410 2024-04-26T18:26:17.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facfcddd700 1 -- 172.21.15.84:0/2887067564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facf81a53e0 con 0x7facf810a410 2024-04-26T18:26:17.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facef7fe700 1 -- 172.21.15.84:0/2887067564 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7face000fd40 con 0x7facf810a410 2024-04-26T18:26:17.577 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.569+0000 7facef7fe700 1 -- 172.21.15.84:0/2887067564 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face000e3b0 con 0x7facf810a410 2024-04-26T18:26:17.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.570+0000 7facfcddd700 1 -- 172.21.15.84:0/2887067564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facf81acc60 con 0x7facf810a410 2024-04-26T18:26:17.578 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.570+0000 7facfcddd700 1 -- 172.21.15.84:0/2887067564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facf806a110 con 0x7facf810a410 2024-04-26T18:26:17.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.572+0000 7facef7fe700 1 -- 172.21.15.84:0/2887067564 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7face0019070 con 0x7facf810a410 2024-04-26T18:26:17.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.572+0000 7facef7fe700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7face40a3cc0 0x7face40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:17.579 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.573+0000 7facf659c700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7face40a3cc0 0x7face40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:17.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.573+0000 7facef7fe700 1 -- 172.21.15.84:0/2887067564 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7face00cfe20 con 0x7facf810a410 2024-04-26T18:26:17.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.573+0000 7facf659c700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7face40a3cc0 0x7face40a6180 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7face8005950 tx=0x7face800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:17.580 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.581+0000 7facef7fe700 1 -- 172.21.15.84:0/2887067564 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7face009c7d0 con 0x7facf810a410 2024-04-26T18:26:17.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.593+0000 7f47b5ef6700 1 -- 172.21.15.84:0/251277922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f47b002e490 con 0x7f47b0102350 2024-04-26T18:26:17.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.594+0000 7f47acff9700 1 -- 172.21.15.84:0/251277922 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f47a009c230 con 0x7f47b0102350 2024-04-26T18:26:17.594 INFO:teuthology.orchestra.run.smithi084.stdout:313532612651 2024-04-26T18:26:17.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.596+0000 7f47b5ef6700 1 -- 172.21.15.84:0/251277922 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f47980a3c40 msgr2=0x7f47980a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:17.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.596+0000 7f47b5ef6700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f47980a3c40 0x7f47980a6100 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f47a4006010 tx=0x7f47a401a040 comp rx=0 tx=0).stop 2024-04-26T18:26:17.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.596+0000 7f47b5ef6700 1 -- 172.21.15.84:0/251277922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0102350 msgr2=0x7f47b019a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:17.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.596+0000 7f47b5ef6700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0102350 0x7f47b019a9d0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f47a000d950 tx=0x7f47a000dc60 comp rx=0 tx=0).stop 2024-04-26T18:26:17.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.596+0000 7f47b5ef6700 1 -- 172.21.15.84:0/251277922 shutdown_connections 2024-04-26T18:26:17.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.596+0000 7f47b5ef6700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f47980a3c40 0x7f47980a6100 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.596+0000 7f47b5ef6700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47b0102350 0x7f47b019a9d0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.596+0000 7f47b5ef6700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47b0104270 0x7f47b01a5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.596+0000 7f47b5ef6700 1 --2- 172.21.15.84:0/251277922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47b0102d10 0x7f47b019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.596+0000 7f47b5ef6700 1 -- 172.21.15.84:0/251277922 >> 172.21.15.84:0/251277922 conn(0x7f47b0078390 msgr2=0x7f47b0100400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:17.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.596+0000 7f47b5ef6700 1 -- 172.21.15.84:0/251277922 shutdown_connections 2024-04-26T18:26:17.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.596+0000 7f47b5ef6700 1 -- 172.21.15.84:0/251277922 wait complete. 2024-04-26T18:26:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.796+0000 7facfcddd700 1 -- 172.21.15.84:0/2887067564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7facf81a1440 con 0x7facf810a410 2024-04-26T18:26:17.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.797+0000 7facef7fe700 1 -- 172.21.15.84:0/2887067564 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7face009c0d0 con 0x7facf810a410 2024-04-26T18:26:17.797 INFO:teuthology.orchestra.run.smithi084.stdout:55834574938 2024-04-26T18:26:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.799+0000 7facfcddd700 1 -- 172.21.15.84:0/2887067564 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7face40a3cc0 msgr2=0x7face40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.799+0000 7facfcddd700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7face40a3cc0 0x7face40a6180 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7face8005950 tx=0x7face800b410 comp rx=0 tx=0).stop 2024-04-26T18:26:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.799+0000 7facfcddd700 1 -- 172.21.15.84:0/2887067564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf810a410 msgr2=0x7facf81a0620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.799+0000 7facfcddd700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf810a410 0x7facf81a0620 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7face000d950 tx=0x7face000dc60 comp rx=0 tx=0).stop 2024-04-26T18:26:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.799+0000 7facfcddd700 1 -- 172.21.15.84:0/2887067564 shutdown_connections 2024-04-26T18:26:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.799+0000 7facfcddd700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7face40a3cc0 0x7face40a6180 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.799+0000 7facfcddd700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7facf810fa50 0x7facf81a49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.799+0000 7facfcddd700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7facf8109a50 0x7facf81a00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.799+0000 7facfcddd700 1 --2- 172.21.15.84:0/2887067564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7facf810a410 0x7facf81a0620 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.799+0000 7facfcddd700 1 -- 172.21.15.84:0/2887067564 >> 172.21.15.84:0/2887067564 conn(0x7facf8076bf0 msgr2=0x7facf8077110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.799+0000 7facfcddd700 1 -- 172.21.15.84:0/2887067564 shutdown_connections 2024-04-26T18:26:17.800 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:17.799+0000 7facfcddd700 1 -- 172.21.15.84:0/2887067564 wait complete. 2024-04-26T18:26:18.504 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:18 smithi084 ceph-mon[29141]: pgmap v359: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 10 op/s 2024-04-26T18:26:18.504 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:18 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/251277922' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T18:26:18.505 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:18 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2887067564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T18:26:18.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:18 smithi103 ceph-mon[32727]: pgmap v359: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 10 op/s 2024-04-26T18:26:18.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:18 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/251277922' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T18:26:18.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:18 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2887067564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T18:26:18.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:18 smithi174 ceph-mon[32705]: pgmap v359: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 614 B/s wr, 10 op/s 2024-04-26T18:26:18.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:18 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/251277922' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-26T18:26:18.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:18 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2887067564' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T18:26:18.733 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738457 got 34359738462 for osd.0 2024-04-26T18:26:18.733 DEBUG:teuthology.parallel:result is None 2024-04-26T18:26:19.591 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612648 got 313532612651 for osd.10 2024-04-26T18:26:19.591 DEBUG:teuthology.parallel:result is None 2024-04-26T18:26:20.045 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888134 got 146028888138 for osd.4 2024-04-26T18:26:20.046 DEBUG:teuthology.parallel:result is None 2024-04-26T18:26:20.248 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:20.249 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:20.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:20 smithi084 ceph-mon[29141]: pgmap v360: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 515 B/s wr, 11 op/s 2024-04-26T18:26:20.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:20 smithi103 ceph-mon[32727]: pgmap v360: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 515 B/s wr, 11 op/s 2024-04-26T18:26:20.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:20 smithi174 ceph-mon[32705]: pgmap v360: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 515 B/s wr, 11 op/s 2024-04-26T18:26:20.697 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574932 got 55834574938 for osd.1 2024-04-26T18:26:20.697 DEBUG:teuthology.parallel:result is None 2024-04-26T18:26:21.475 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:21 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:26:21.564 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:21 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:26:21.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:21 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:26:22.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.167+0000 7fc0cf19e700 1 -- 172.21.15.84:0/1934138067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0c8102d80 msgr2=0x7fc0c8103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:22.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.167+0000 7fc0cf19e700 1 --2- 172.21.15.84:0/1934138067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0c8102d80 0x7fc0c8103160 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc0099c0 tx=0x7fc0bc009cd0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.167+0000 7fc0cf19e700 1 -- 172.21.15.84:0/1934138067 shutdown_connections 2024-04-26T18:26:22.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.167+0000 7fc0cf19e700 1 --2- 172.21.15.84:0/1934138067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0c8102d80 0x7fc0c8103160 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.167+0000 7fc0cf19e700 1 --2- 172.21.15.84:0/1934138067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0c81076a0 0x7fc0c810f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.167+0000 7fc0cf19e700 1 --2- 172.21.15.84:0/1934138067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0c8103730 0x7fc0c8106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.167+0000 7fc0cf19e700 1 -- 172.21.15.84:0/1934138067 >> 172.21.15.84:0/1934138067 conn(0x7fc0c8078990 msgr2=0x7fc0c80ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:22.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.167+0000 7fc0cf19e700 1 -- 172.21.15.84:0/1934138067 shutdown_connections 2024-04-26T18:26:22.169 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.167+0000 7fc0cf19e700 1 -- 172.21.15.84:0/1934138067 wait complete. 2024-04-26T18:26:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.167+0000 7fc0cf19e700 1 Processor -- start 2024-04-26T18:26:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.167+0000 7fc0cf19e700 1 -- start start 2024-04-26T18:26:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0cf19e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0c8102d80 0x7fc0c819bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0cf19e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0c8103730 0x7fc0c819c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0cf19e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0c81076a0 0x7fc0c81a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0cf19e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c8112570 con 0x7fc0c8102d80 2024-04-26T18:26:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0cf19e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c81123f0 con 0x7fc0c81076a0 2024-04-26T18:26:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0cf19e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0c81126f0 con 0x7fc0c8103730 2024-04-26T18:26:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0ccf3a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0c8102d80 0x7fc0c819bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0ccf3a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0c8102d80 0x7fc0c819bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46548/0 (socket says 172.21.15.84:46548) 2024-04-26T18:26:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0ccf3a700 1 -- 172.21.15.84:0/2728510555 learned_addr learned my addr 172.21.15.84:0/2728510555 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0c7fff700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0c8103730 0x7fc0c819c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:22.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0cd73b700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0c81076a0 0x7fc0c81a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:22.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0ccf3a700 1 -- 172.21.15.84:0/2728510555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0c8103730 msgr2=0x7fc0c819c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:22.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0ccf3a700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0c8103730 0x7fc0c819c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0ccf3a700 1 -- 172.21.15.84:0/2728510555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0c81076a0 msgr2=0x7fc0c81a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:22.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0ccf3a700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0c81076a0 0x7fc0c81a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0ccf3a700 1 -- 172.21.15.84:0/2728510555 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0c81a0df0 con 0x7fc0c8102d80 2024-04-26T18:26:22.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0ccf3a700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0c8102d80 0x7fc0c819bde0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc006010 tx=0x7fc0bc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:22.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0c5ffb700 1 -- 172.21.15.84:0/2728510555 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc026070 con 0x7fc0c8102d80 2024-04-26T18:26:22.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0cf19e700 1 -- 172.21.15.84:0/2728510555 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0c81a1080 con 0x7fc0c8102d80 2024-04-26T18:26:22.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0c5ffb700 1 -- 172.21.15.84:0/2728510555 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0bc00f460 con 0x7fc0c8102d80 2024-04-26T18:26:22.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.168+0000 7fc0c5ffb700 1 -- 172.21.15.84:0/2728510555 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0bc009e60 con 0x7fc0c8102d80 2024-04-26T18:26:22.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.169+0000 7fc0cf19e700 1 -- 172.21.15.84:0/2728510555 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0c81a8960 con 0x7fc0c8102d80 2024-04-26T18:26:22.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.170+0000 7fc0cf19e700 1 -- 172.21.15.84:0/2728510555 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc0c8103b30 con 0x7fc0c8102d80 2024-04-26T18:26:22.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.171+0000 7fc0c5ffb700 1 -- 172.21.15.84:0/2728510555 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc0bc029070 con 0x7fc0c8102d80 2024-04-26T18:26:22.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.172+0000 7fc0c5ffb700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc0b80a3c40 0x7fc0b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:22.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.172+0000 7fc0c7fff700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc0b80a3c40 0x7fc0b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:22.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.172+0000 7fc0c5ffb700 1 -- 172.21.15.84:0/2728510555 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fc0bc0d0fc0 con 0x7fc0c8102d80 2024-04-26T18:26:22.173 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.172+0000 7fc0c7fff700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc0b80a3c40 0x7fc0b80a6100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc0c81073a0 tx=0x7fc0b400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:22.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.175+0000 7fc0c5ffb700 1 -- 172.21.15.84:0/2728510555 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc0bc030020 con 0x7fc0c8102d80 2024-04-26T18:26:22.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:22 smithi084 ceph-mon[29141]: pgmap v361: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-26T18:26:22.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:22 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:22.346 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:22 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:22.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.392+0000 7fc0cf19e700 1 -- 172.21.15.84:0/2728510555 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc0c806a110 con 0x7fc0c8102d80 2024-04-26T18:26:22.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.392+0000 7fc0c5ffb700 1 -- 172.21.15.84:0/2728510555 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fc0bc013070 con 0x7fc0c8102d80 2024-04-26T18:26:22.392 INFO:teuthology.orchestra.run.smithi084.stdout:167503724613 2024-04-26T18:26:22.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.394+0000 7fc0cf19e700 1 -- 172.21.15.84:0/2728510555 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc0b80a3c40 msgr2=0x7fc0b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.394+0000 7fc0cf19e700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc0b80a3c40 0x7fc0b80a6100 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fc0c81073a0 tx=0x7fc0b400b410 comp rx=0 tx=0).stop 2024-04-26T18:26:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.394+0000 7fc0cf19e700 1 -- 172.21.15.84:0/2728510555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0c8102d80 msgr2=0x7fc0c819bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.394+0000 7fc0cf19e700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0c8102d80 0x7fc0c819bde0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc0bc006010 tx=0x7fc0bc0045e0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.394+0000 7fc0cf19e700 1 -- 172.21.15.84:0/2728510555 shutdown_connections 2024-04-26T18:26:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.394+0000 7fc0cf19e700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc0b80a3c40 0x7fc0b80a6100 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.394+0000 7fc0cf19e700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0c8102d80 0x7fc0c819bde0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.394+0000 7fc0cf19e700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0c81076a0 0x7fc0c81a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.394+0000 7fc0cf19e700 1 --2- 172.21.15.84:0/2728510555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0c8103730 0x7fc0c819c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.394+0000 7fc0cf19e700 1 -- 172.21.15.84:0/2728510555 >> 172.21.15.84:0/2728510555 conn(0x7fc0c8078990 msgr2=0x7fc0c8104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.394+0000 7fc0cf19e700 1 -- 172.21.15.84:0/2728510555 shutdown_connections 2024-04-26T18:26:22.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.394+0000 7fc0cf19e700 1 -- 172.21.15.84:0/2728510555 wait complete. 2024-04-26T18:26:22.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.381+0000 7ff410494700 1 -- 172.21.15.84:0/3256057228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4081034d0 msgr2=0x7ff408103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.381+0000 7ff410494700 1 --2- 172.21.15.84:0/3256057228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4081034d0 0x7ff408103950 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009990 tx=0x7ff3f8009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.381+0000 7ff410494700 1 -- 172.21.15.84:0/3256057228 shutdown_connections 2024-04-26T18:26:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.381+0000 7ff410494700 1 --2- 172.21.15.84:0/3256057228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff408103e90 0x7ff4081102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.381+0000 7ff410494700 1 --2- 172.21.15.84:0/3256057228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4081094f0 0x7ff4081098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.381+0000 7ff410494700 1 --2- 172.21.15.84:0/3256057228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4081034d0 0x7ff408103950 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.381+0000 7ff410494700 1 -- 172.21.15.84:0/3256057228 >> 172.21.15.84:0/3256057228 conn(0x7ff40807a500 msgr2=0x7ff40807a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.381+0000 7ff410494700 1 -- 172.21.15.84:0/3256057228 shutdown_connections 2024-04-26T18:26:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.381+0000 7ff410494700 1 -- 172.21.15.84:0/3256057228 wait complete. 2024-04-26T18:26:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.382+0000 7ff410494700 1 Processor -- start 2024-04-26T18:26:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.382+0000 7ff410494700 1 -- start start 2024-04-26T18:26:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.382+0000 7ff410494700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4081034d0 0x7ff40819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.382+0000 7ff410494700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff408103e90 0x7ff40819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:22.400 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.382+0000 7ff410494700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4081094f0 0x7ff408199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.382+0000 7ff410494700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff408113620 con 0x7ff4081034d0 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.382+0000 7ff410494700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff4081134a0 con 0x7ff408103e90 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.382+0000 7ff410494700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff4081137a0 con 0x7ff4081094f0 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.382+0000 7ff40e230700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4081034d0 0x7ff40819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.382+0000 7ff40e230700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4081034d0 0x7ff40819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46574/0 (socket says 172.21.15.84:46574) 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.382+0000 7ff40e230700 1 -- 172.21.15.84:0/1248956734 learned_addr learned my addr 172.21.15.84:0/1248956734 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff40da2f700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff408103e90 0x7ff40819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff40ea31700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4081094f0 0x7ff408199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff40e230700 1 -- 172.21.15.84:0/1248956734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4081094f0 msgr2=0x7ff408199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff40e230700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4081094f0 0x7ff408199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff40e230700 1 -- 172.21.15.84:0/1248956734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff408103e90 msgr2=0x7ff40819fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff40e230700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff408103e90 0x7ff40819fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff40e230700 1 -- 172.21.15.84:0/1248956734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff40819a0e0 con 0x7ff4081034d0 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff40ea31700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4081094f0 0x7ff408199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:22.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff40e230700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4081034d0 0x7ff40819f7c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff40400e9a0 tx=0x7ff40400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff3ff7fe700 1 -- 172.21.15.84:0/1248956734 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff40400cbd0 con 0x7ff4081034d0 2024-04-26T18:26:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff3ff7fe700 1 -- 172.21.15.84:0/1248956734 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff40400cd70 con 0x7ff4081034d0 2024-04-26T18:26:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff410494700 1 -- 172.21.15.84:0/1248956734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff40819a3d0 con 0x7ff4081034d0 2024-04-26T18:26:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff3ff7fe700 1 -- 172.21.15.84:0/1248956734 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff404018680 con 0x7ff4081034d0 2024-04-26T18:26:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.383+0000 7ff410494700 1 -- 172.21.15.84:0/1248956734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4081a6690 con 0x7ff4081034d0 2024-04-26T18:26:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.384+0000 7ff410494700 1 -- 172.21.15.84:0/1248956734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff40810b960 con 0x7ff4081034d0 2024-04-26T18:26:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.385+0000 7ff3ff7fe700 1 -- 172.21.15.84:0/1248956734 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff404010430 con 0x7ff4081034d0 2024-04-26T18:26:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.386+0000 7ff3ff7fe700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff3f40a3c40 0x7ff3f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.386+0000 7ff40da2f700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff3f40a3c40 0x7ff3f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.386+0000 7ff3ff7fe700 1 -- 172.21.15.84:0/1248956734 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7ff4040d0aa0 con 0x7ff4081034d0 2024-04-26T18:26:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.386+0000 7ff40da2f700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff3f40a3c40 0x7ff3f40a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800b2e0 tx=0x7ff3f8005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:22.402 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.389+0000 7ff3ff7fe700 1 -- 172.21.15.84:0/1248956734 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff40409d450 con 0x7ff4081034d0 2024-04-26T18:26:22.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.597+0000 7ff410494700 1 -- 172.21.15.84:0/1248956734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7ff40819aeb0 con 0x7ff4081034d0 2024-04-26T18:26:22.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.597+0000 7ff3ff7fe700 1 -- 172.21.15.84:0/1248956734 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ff40409cd50 con 0x7ff4081034d0 2024-04-26T18:26:22.597 INFO:teuthology.orchestra.run.smithi084.stdout:257698037815 2024-04-26T18:26:22.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.599+0000 7ff410494700 1 -- 172.21.15.84:0/1248956734 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff3f40a3c40 msgr2=0x7ff3f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:22.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.599+0000 7ff410494700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff3f40a3c40 0x7ff3f40a6100 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800b2e0 tx=0x7ff3f8005960 comp rx=0 tx=0).stop 2024-04-26T18:26:22.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.599+0000 7ff410494700 1 -- 172.21.15.84:0/1248956734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4081034d0 msgr2=0x7ff40819f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:22.599 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.599+0000 7ff410494700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4081034d0 0x7ff40819f7c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ff40400e9a0 tx=0x7ff40400ecb0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.599+0000 7ff410494700 1 -- 172.21.15.84:0/1248956734 shutdown_connections 2024-04-26T18:26:22.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.599+0000 7ff410494700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff3f40a3c40 0x7ff3f40a6100 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.599+0000 7ff410494700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4081034d0 0x7ff40819f7c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.599+0000 7ff410494700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4081094f0 0x7ff408199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.599+0000 7ff410494700 1 --2- 172.21.15.84:0/1248956734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff408103e90 0x7ff40819fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:22.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.599+0000 7ff410494700 1 -- 172.21.15.84:0/1248956734 >> 172.21.15.84:0/1248956734 conn(0x7ff40807a500 msgr2=0x7ff4081022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:22.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.599+0000 7ff410494700 1 -- 172.21.15.84:0/1248956734 shutdown_connections 2024-04-26T18:26:22.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:22.599+0000 7ff410494700 1 -- 172.21.15.84:0/1248956734 wait complete. 2024-04-26T18:26:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:22 smithi103 ceph-mon[32727]: pgmap v361: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-26T18:26:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:22 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:22 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:22.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:22 smithi174 ceph-mon[32705]: pgmap v361: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 85 B/s wr, 10 op/s 2024-04-26T18:26:22.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:22 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:22.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:22 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:23.527 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:23.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:23 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:23.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:23 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:23.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:23 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2728510555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T18:26:23.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:23 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1248956734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T18:26:23.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:23 smithi084 ceph-mon[29141]: pgmap v362: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:23.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:23 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:23.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:23 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:23.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:23 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2728510555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T18:26:23.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:23 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1248956734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T18:26:23.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:23 smithi103 ceph-mon[32727]: pgmap v362: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:23.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:23 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:23.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:23 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:23.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:23 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2728510555' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T18:26:23.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:23 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1248956734' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-26T18:26:23.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:23 smithi174 ceph-mon[32705]: pgmap v362: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:23.972 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037810 got 257698037815 for osd.8 2024-04-26T18:26:23.972 DEBUG:teuthology.parallel:result is None 2024-04-26T18:26:26.290 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:26 smithi084 ceph-mon[29141]: pgmap v363: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.511+0000 7fb13e1dc700 1 -- 172.21.15.84:0/943320949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1381027a0 msgr2=0x7fb138110210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.511+0000 7fb13e1dc700 1 --2- 172.21.15.84:0/943320949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1381027a0 0x7fb138110210 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb13000b0a0 tx=0x7fb13000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:26:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.511+0000 7fb13e1dc700 1 -- 172.21.15.84:0/943320949 shutdown_connections 2024-04-26T18:26:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.511+0000 7fb13e1dc700 1 --2- 172.21.15.84:0/943320949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb1381027a0 0x7fb138110210 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.511+0000 7fb13e1dc700 1 --2- 172.21.15.84:0/943320949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb138108500 0x7fb1381088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.511+0000 7fb13e1dc700 1 --2- 172.21.15.84:0/943320949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb138101de0 0x7fb138102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.511+0000 7fb13e1dc700 1 -- 172.21.15.84:0/943320949 >> 172.21.15.84:0/943320949 conn(0x7fb138077640 msgr2=0x7fb138077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.511+0000 7fb13e1dc700 1 -- 172.21.15.84:0/943320949 shutdown_connections 2024-04-26T18:26:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.511+0000 7fb13e1dc700 1 -- 172.21.15.84:0/943320949 wait complete. 2024-04-26T18:26:26.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.511+0000 7fb13e1dc700 1 Processor -- start 2024-04-26T18:26:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.511+0000 7fb13e1dc700 1 -- start start 2024-04-26T18:26:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb13e1dc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb138101de0 0x7fb13819f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb13e1dc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1381027a0 0x7fb13819fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb13e1dc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb138108500 0x7fb138199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb13e1dc700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb138113710 con 0x7fb138108500 2024-04-26T18:26:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb13e1dc700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb138113590 con 0x7fb1381027a0 2024-04-26T18:26:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb13e1dc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb138113890 con 0x7fb138101de0 2024-04-26T18:26:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb137fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb138108500 0x7fb138199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb137fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb138108500 0x7fb138199ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46588/0 (socket says 172.21.15.84:46588) 2024-04-26T18:26:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb137fff700 1 -- 172.21.15.84:0/147827565 learned_addr learned my addr 172.21.15.84:0/147827565 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb1377fe700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb138101de0 0x7fb13819f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb137fff700 1 -- 172.21.15.84:0/147827565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb138101de0 msgr2=0x7fb13819f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:26.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb137fff700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb138101de0 0x7fb13819f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:26.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb137fff700 1 -- 172.21.15.84:0/147827565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1381027a0 msgr2=0x7fb13819fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:26.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb136ffd700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1381027a0 0x7fb13819fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:26.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb137fff700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1381027a0 0x7fb13819fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:26.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb137fff700 1 -- 172.21.15.84:0/147827565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13819a210 con 0x7fb138108500 2024-04-26T18:26:26.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb1377fe700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb138101de0 0x7fb13819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:26.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb136ffd700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1381027a0 0x7fb13819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:26.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb137fff700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb138108500 0x7fb138199ab0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb1300062a0 tx=0x7fb130004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:26.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb134ff9700 1 -- 172.21.15.84:0/147827565 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130021070 con 0x7fb138108500 2024-04-26T18:26:26.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb134ff9700 1 -- 172.21.15.84:0/147827565 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb130007d00 con 0x7fb138108500 2024-04-26T18:26:26.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb134ff9700 1 -- 172.21.15.84:0/147827565 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13001d450 con 0x7fb138108500 2024-04-26T18:26:26.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.512+0000 7fb13e1dc700 1 -- 172.21.15.84:0/147827565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13819a4a0 con 0x7fb138108500 2024-04-26T18:26:26.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.513+0000 7fb13e1dc700 1 -- 172.21.15.84:0/147827565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1381a6770 con 0x7fb138108500 2024-04-26T18:26:26.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.513+0000 7fb13e1dc700 1 -- 172.21.15.84:0/147827565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb13806a110 con 0x7fb138108500 2024-04-26T18:26:26.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.514+0000 7fb134ff9700 1 -- 172.21.15.84:0/147827565 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb130004020 con 0x7fb138108500 2024-04-26T18:26:26.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.515+0000 7fb134ff9700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb1200a3cc0 0x7fb1200a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:26.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.515+0000 7fb1377fe700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb1200a3cc0 0x7fb1200a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:26.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.515+0000 7fb134ff9700 1 -- 172.21.15.84:0/147827565 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fb13000f020 con 0x7fb138108500 2024-04-26T18:26:26.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.515+0000 7fb1377fe700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb1200a3cc0 0x7fb1200a6180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1381005a0 tx=0x7fb128008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:26.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.519+0000 7fb134ff9700 1 -- 172.21.15.84:0/147827565 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb13002a020 con 0x7fb138108500 2024-04-26T18:26:26.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:26 smithi103 ceph-mon[32727]: pgmap v363: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:26.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:26 smithi174 ceph-mon[32705]: pgmap v363: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:26.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.733+0000 7fb13e1dc700 1 -- 172.21.15.84:0/147827565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fb13819af90 con 0x7fb138108500 2024-04-26T18:26:26.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.734+0000 7fb134ff9700 1 -- 172.21.15.84:0/147827565 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb13009e8f0 con 0x7fb138108500 2024-04-26T18:26:26.734 INFO:teuthology.orchestra.run.smithi084.stdout:77309411414 2024-04-26T18:26:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.736+0000 7fb13e1dc700 1 -- 172.21.15.84:0/147827565 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb1200a3cc0 msgr2=0x7fb1200a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.736+0000 7fb13e1dc700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb1200a3cc0 0x7fb1200a6180 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fb1381005a0 tx=0x7fb128008040 comp rx=0 tx=0).stop 2024-04-26T18:26:26.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.736+0000 7fb13e1dc700 1 -- 172.21.15.84:0/147827565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb138108500 msgr2=0x7fb138199ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.736+0000 7fb13e1dc700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb138108500 0x7fb138199ab0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fb1300062a0 tx=0x7fb130004ed0 comp rx=0 tx=0).stop 2024-04-26T18:26:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.736+0000 7fb13e1dc700 1 -- 172.21.15.84:0/147827565 shutdown_connections 2024-04-26T18:26:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.736+0000 7fb13e1dc700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb1200a3cc0 0x7fb1200a6180 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.736+0000 7fb13e1dc700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb138108500 0x7fb138199ab0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.736+0000 7fb13e1dc700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb138101de0 0x7fb13819f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.736+0000 7fb13e1dc700 1 --2- 172.21.15.84:0/147827565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb1381027a0 0x7fb13819fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.736+0000 7fb13e1dc700 1 -- 172.21.15.84:0/147827565 >> 172.21.15.84:0/147827565 conn(0x7fb138077640 msgr2=0x7fb1381009f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.736+0000 7fb13e1dc700 1 -- 172.21.15.84:0/147827565 shutdown_connections 2024-04-26T18:26:26.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:26.736+0000 7fb13e1dc700 1 -- 172.21.15.84:0/147827565 wait complete. 2024-04-26T18:26:27.475 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:27 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/147827565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T18:26:27.502 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724608 got 167503724613 for osd.5 2024-04-26T18:26:27.502 DEBUG:teuthology.parallel:result is None 2024-04-26T18:26:27.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:27 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/147827565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T18:26:27.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:27 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/147827565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T18:26:28.144 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411408 got 77309411414 for osd.2 2024-04-26T18:26:28.144 DEBUG:teuthology.parallel:result is None 2024-04-26T18:26:28.365 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:28 smithi084 ceph-mon[29141]: pgmap v364: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:28.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:28 smithi103 ceph-mon[32727]: pgmap v364: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:28.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:28 smithi174 ceph-mon[32705]: pgmap v364: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:29.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.001+0000 7f0292df2700 1 -- 172.21.15.84:0/2980714679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f028c1076b0 msgr2=0x7f028c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:29.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.001+0000 7f0292df2700 1 --2- 172.21.15.84:0/2980714679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f028c1076b0 0x7f028c10f250 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f028400b0d0 tx=0x7f028400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:26:29.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.002+0000 7f0292df2700 1 -- 172.21.15.84:0/2980714679 shutdown_connections 2024-04-26T18:26:29.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.002+0000 7f0292df2700 1 --2- 172.21.15.84:0/2980714679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f028c1076b0 0x7f028c10f250 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:29.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.002+0000 7f0292df2700 1 --2- 172.21.15.84:0/2980714679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f028c102960 0x7f028c102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:29.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.002+0000 7f0292df2700 1 --2- 172.21.15.84:0/2980714679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f028c103310 0x7f028c106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:29.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.002+0000 7f0292df2700 1 -- 172.21.15.84:0/2980714679 >> 172.21.15.84:0/2980714679 conn(0x7f028c0792d0 msgr2=0x7f028c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:29.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.002+0000 7f0292df2700 1 -- 172.21.15.84:0/2980714679 shutdown_connections 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.002+0000 7f0292df2700 1 -- 172.21.15.84:0/2980714679 wait complete. 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.002+0000 7f0292df2700 1 Processor -- start 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.002+0000 7f0292df2700 1 -- start start 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f0292df2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f028c102960 0x7f028c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f0292df2700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f028c103310 0x7f028c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f0292df2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f028c1076b0 0x7f028c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f0292df2700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f028c112570 con 0x7f028c1076b0 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f0292df2700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1123f0 con 0x7f028c103310 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f0292df2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f028c1126f0 con 0x7f028c102960 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f029138f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f028c1076b0 0x7f028c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f029138f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f028c1076b0 0x7f028c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43722/0 (socket says 172.21.15.84:43722) 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f029138f700 1 -- 172.21.15.84:0/1806477891 learned_addr learned my addr 172.21.15.84:0/1806477891 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f0290b8e700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f028c102960 0x7f028c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:29.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f028bfff700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f028c103310 0x7f028c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f029138f700 1 -- 172.21.15.84:0/1806477891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f028c102960 msgr2=0x7f028c19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f029138f700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f028c102960 0x7f028c19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f029138f700 1 -- 172.21.15.84:0/1806477891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f028c103310 msgr2=0x7f028c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f029138f700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f028c103310 0x7f028c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f029138f700 1 -- 172.21.15.84:0/1806477891 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028c199010 con 0x7f028c1076b0 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f029138f700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f028c1076b0 0x7f028c1988b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0284009fd0 tx=0x7f0284007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.003+0000 7f0289ffb700 1 -- 172.21.15.84:0/1806477891 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0284021070 con 0x7f028c1076b0 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.004+0000 7f0289ffb700 1 -- 172.21.15.84:0/1806477891 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f028401db00 con 0x7f028c1076b0 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.004+0000 7f0292df2700 1 -- 172.21.15.84:0/1806477891 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f028c1992a0 con 0x7f028c1076b0 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.004+0000 7f0289ffb700 1 -- 172.21.15.84:0/1806477891 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f028401b970 con 0x7f028c1076b0 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.004+0000 7f0292df2700 1 -- 172.21.15.84:0/1806477891 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f028c1a5570 con 0x7f028c1076b0 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.005+0000 7f0292df2700 1 -- 172.21.15.84:0/1806477891 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f028c06b3a0 con 0x7f028c1076b0 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.006+0000 7f0289ffb700 1 -- 172.21.15.84:0/1806477891 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0284024070 con 0x7f028c1076b0 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.006+0000 7f0289ffb700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f027c0a3c40 0x7f027c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:29.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.007+0000 7f0290b8e700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f027c0a3c40 0x7f027c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:29.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.007+0000 7f0289ffb700 1 -- 172.21.15.84:0/1806477891 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f02840d18f0 con 0x7f028c1076b0 2024-04-26T18:26:29.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.007+0000 7f0290b8e700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f027c0a3c40 0x7f027c0a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f028c1071f0 tx=0x7f028000b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:29.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.015+0000 7f0289ffb700 1 -- 172.21.15.84:0/1806477891 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f028409e330 con 0x7f028c1076b0 2024-04-26T18:26:29.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.219+0000 7f0292df2700 1 -- 172.21.15.84:0/1806477891 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f028c103a60 con 0x7f028c1076b0 2024-04-26T18:26:29.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.220+0000 7f0289ffb700 1 -- 172.21.15.84:0/1806477891 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f028409dc30 con 0x7f028c1076b0 2024-04-26T18:26:29.220 INFO:teuthology.orchestra.run.smithi084.stdout:111669149777 2024-04-26T18:26:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.223+0000 7f0292df2700 1 -- 172.21.15.84:0/1806477891 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f027c0a3c40 msgr2=0x7f027c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.223+0000 7f0292df2700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f027c0a3c40 0x7f027c0a6100 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f028c1071f0 tx=0x7f028000b410 comp rx=0 tx=0).stop 2024-04-26T18:26:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.223+0000 7f0292df2700 1 -- 172.21.15.84:0/1806477891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f028c1076b0 msgr2=0x7f028c1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.223+0000 7f0292df2700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f028c1076b0 0x7f028c1988b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0284009fd0 tx=0x7f0284007a20 comp rx=0 tx=0).stop 2024-04-26T18:26:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.223+0000 7f0292df2700 1 -- 172.21.15.84:0/1806477891 shutdown_connections 2024-04-26T18:26:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.223+0000 7f0292df2700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f027c0a3c40 0x7f027c0a6100 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.223+0000 7f0292df2700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f028c1076b0 0x7f028c1988b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.223+0000 7f0292df2700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f028c102960 0x7f028c19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.223+0000 7f0292df2700 1 --2- 172.21.15.84:0/1806477891 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f028c103310 0x7f028c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.223+0000 7f0292df2700 1 -- 172.21.15.84:0/1806477891 >> 172.21.15.84:0/1806477891 conn(0x7f028c0792d0 msgr2=0x7f028c100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:29.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.223+0000 7f0292df2700 1 -- 172.21.15.84:0/1806477891 shutdown_connections 2024-04-26T18:26:29.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:29.223+0000 7f0292df2700 1 -- 172.21.15.84:0/1806477891 wait complete. 2024-04-26T18:26:29.508 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:29 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1806477891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T18:26:29.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:29 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1806477891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T18:26:29.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:29 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1806477891' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T18:26:30.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:30 smithi084 ceph-mon[29141]: pgmap v365: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:30 smithi103 ceph-mon[32727]: pgmap v365: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:30.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:30 smithi174 ceph-mon[32705]: pgmap v365: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:30.922 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149774 got 111669149777 for osd.3 2024-04-26T18:26:30.922 DEBUG:teuthology.parallel:result is None 2024-04-26T18:26:31.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.157+0000 7f15702de700 1 -- 172.21.15.84:0/3838406162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15681024a0 msgr2=0x7f156810e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:31.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.157+0000 7f15702de700 1 --2- 172.21.15.84:0/3838406162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15681024a0 0x7f156810e960 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f156000b0a0 tx=0x7f156000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.157+0000 7f15702de700 1 -- 172.21.15.84:0/3838406162 shutdown_connections 2024-04-26T18:26:31.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.157+0000 7f15702de700 1 --2- 172.21.15.84:0/3838406162 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f15681024a0 0x7f156810e960 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.157+0000 7f15702de700 1 --2- 172.21.15.84:0/3838406162 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1568107a90 0x7f1568107e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.157+0000 7f15702de700 1 --2- 172.21.15.84:0/3838406162 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1568101ae0 0x7f1568101f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.157+0000 7f15702de700 1 -- 172.21.15.84:0/3838406162 >> 172.21.15.84:0/3838406162 conn(0x7f1568078990 msgr2=0x7f15680ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.157+0000 7f15702de700 1 -- 172.21.15.84:0/3838406162 shutdown_connections 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.157+0000 7f15702de700 1 -- 172.21.15.84:0/3838406162 wait complete. 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f15702de700 1 Processor -- start 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f15702de700 1 -- start start 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f15702de700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1568101ae0 0x7f156819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f15702de700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15681024a0 0x7f156819adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f15702de700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1568107a90 0x7f15681a5930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f15702de700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1568112530 con 0x7f1568101ae0 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f15702de700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f15681123b0 con 0x7f1568107a90 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f15702de700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f15681126b0 con 0x7f15681024a0 2024-04-26T18:26:31.160 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f156e07a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1568101ae0 0x7f156819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f156e07a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1568101ae0 0x7f156819a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43736/0 (socket says 172.21.15.84:43736) 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f156e07a700 1 -- 172.21.15.84:0/423822411 learned_addr learned my addr 172.21.15.84:0/423822411 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f156d879700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15681024a0 0x7f156819adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f156e87b700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1568107a90 0x7f15681a5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f156e07a700 1 -- 172.21.15.84:0/423822411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15681024a0 msgr2=0x7f156819adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f156e07a700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15681024a0 0x7f156819adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f156e07a700 1 -- 172.21.15.84:0/423822411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1568107a90 msgr2=0x7f15681a5930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f156e07a700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1568107a90 0x7f15681a5930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f156e07a700 1 -- 172.21.15.84:0/423822411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15681a5e70 con 0x7f1568101ae0 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.158+0000 7f156d879700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15681024a0 0x7f156819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.159+0000 7f156e07a700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1568101ae0 0x7f156819a8b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f156400c9b0 tx=0x7f156400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.159+0000 7f155f7fe700 1 -- 172.21.15.84:0/423822411 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564007b00 con 0x7f1568101ae0 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.159+0000 7f155f7fe700 1 -- 172.21.15.84:0/423822411 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1564007ca0 con 0x7f1568101ae0 2024-04-26T18:26:31.161 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.159+0000 7f155f7fe700 1 -- 172.21.15.84:0/423822411 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1564018730 con 0x7f1568101ae0 2024-04-26T18:26:31.162 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.159+0000 7f15702de700 1 -- 172.21.15.84:0/423822411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15681a60a0 con 0x7f1568101ae0 2024-04-26T18:26:31.162 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.159+0000 7f15702de700 1 -- 172.21.15.84:0/423822411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15681a6560 con 0x7f1568101ae0 2024-04-26T18:26:31.162 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.160+0000 7f15702de700 1 -- 172.21.15.84:0/423822411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15681031d0 con 0x7f1568101ae0 2024-04-26T18:26:31.163 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.161+0000 7f155f7fe700 1 -- 172.21.15.84:0/423822411 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f156400f450 con 0x7f1568101ae0 2024-04-26T18:26:31.163 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.161+0000 7f155f7fe700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f15540a3cc0 0x7f15540a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:31.163 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.162+0000 7f156d879700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f15540a3cc0 0x7f15540a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:31.163 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.162+0000 7f155f7fe700 1 -- 172.21.15.84:0/423822411 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f15640d0740 con 0x7f1568101ae0 2024-04-26T18:26:31.163 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.162+0000 7f156d879700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f15540a3cc0 0x7f15540a6180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f15681060a0 tx=0x7f1558008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:31.165 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.165+0000 7f155f7fe700 1 -- 172.21.15.84:0/423822411 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1564014030 con 0x7f1568101ae0 2024-04-26T18:26:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.371+0000 7f15702de700 1 -- 172.21.15.84:0/423822411 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f156802e490 con 0x7f1568101ae0 2024-04-26T18:26:31.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.372+0000 7f155f7fe700 1 -- 172.21.15.84:0/423822411 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f156409d200 con 0x7f1568101ae0 2024-04-26T18:26:31.372 INFO:teuthology.orchestra.run.smithi084.stdout:223338299454 2024-04-26T18:26:31.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.374+0000 7f15702de700 1 -- 172.21.15.84:0/423822411 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f15540a3cc0 msgr2=0x7f15540a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:31.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.374+0000 7f15702de700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f15540a3cc0 0x7f15540a6180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f15681060a0 tx=0x7f1558008040 comp rx=0 tx=0).stop 2024-04-26T18:26:31.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.374+0000 7f15702de700 1 -- 172.21.15.84:0/423822411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1568101ae0 msgr2=0x7f156819a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:31.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.374+0000 7f15702de700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1568101ae0 0x7f156819a8b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f156400c9b0 tx=0x7f156400cd70 comp rx=0 tx=0).stop 2024-04-26T18:26:31.374 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.374+0000 7f15702de700 1 -- 172.21.15.84:0/423822411 shutdown_connections 2024-04-26T18:26:31.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.374+0000 7f15702de700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f15540a3cc0 0x7f15540a6180 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.374+0000 7f15702de700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1568101ae0 0x7f156819a8b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.374+0000 7f15702de700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1568107a90 0x7f15681a5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.374+0000 7f15702de700 1 --2- 172.21.15.84:0/423822411 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f15681024a0 0x7f156819adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.374+0000 7f15702de700 1 -- 172.21.15.84:0/423822411 >> 172.21.15.84:0/423822411 conn(0x7f1568078990 msgr2=0x7f1568104770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:31.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.374+0000 7f15702de700 1 -- 172.21.15.84:0/423822411 shutdown_connections 2024-04-26T18:26:31.375 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.374+0000 7f15702de700 1 -- 172.21.15.84:0/423822411 wait complete. 2024-04-26T18:26:31.381 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.375+0000 7fa928a19700 1 -- 172.21.15.84:0/2129595608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa924108380 msgr2=0x7fa924108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.375+0000 7fa928a19700 1 --2- 172.21.15.84:0/2129595608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa924108380 0x7fa924108760 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa914009990 tx=0x7fa914009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.375+0000 7fa928a19700 1 -- 172.21.15.84:0/2129595608 shutdown_connections 2024-04-26T18:26:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.375+0000 7fa928a19700 1 --2- 172.21.15.84:0/2129595608 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa924108380 0x7fa924108760 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.376+0000 7fa928a19700 1 --2- 172.21.15.84:0/2129595608 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa924102cd0 0x7fa92410f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.376+0000 7fa928a19700 1 --2- 172.21.15.84:0/2129595608 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa924102310 0x7fa924102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.376+0000 7fa928a19700 1 -- 172.21.15.84:0/2129595608 >> 172.21.15.84:0/2129595608 conn(0x7fa924079290 msgr2=0x7fa9240796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.376+0000 7fa928a19700 1 -- 172.21.15.84:0/2129595608 shutdown_connections 2024-04-26T18:26:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.376+0000 7fa928a19700 1 -- 172.21.15.84:0/2129595608 wait complete. 2024-04-26T18:26:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.376+0000 7fa928a19700 1 Processor -- start 2024-04-26T18:26:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.376+0000 7fa928a19700 1 -- start start 2024-04-26T18:26:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa928a19700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa924102310 0x7fa92419a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:31.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa928a19700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa924102cd0 0x7fa92419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa928a19700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa924108380 0x7fa9241a59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa928a19700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa9241124d0 con 0x7fa924108380 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa928a19700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa924112350 con 0x7fa924102cd0 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa928a19700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa924112650 con 0x7fa924102310 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa922d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa924108380 0x7fa9241a59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa922d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa924108380 0x7fa9241a59b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43766/0 (socket says 172.21.15.84:43766) 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa922d9d700 1 -- 172.21.15.84:0/416370995 learned_addr learned my addr 172.21.15.84:0/416370995 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa92259c700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa924102310 0x7fa92419a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa921d9b700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa924102cd0 0x7fa92419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa922d9d700 1 -- 172.21.15.84:0/416370995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa924102310 msgr2=0x7fa92419a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa922d9d700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa924102310 0x7fa92419a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa922d9d700 1 -- 172.21.15.84:0/416370995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa924102cd0 msgr2=0x7fa92419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa922d9d700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa924102cd0 0x7fa92419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa922d9d700 1 -- 172.21.15.84:0/416370995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9241a5ef0 con 0x7fa924108380 2024-04-26T18:26:31.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa921d9b700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa924102cd0 0x7fa92419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:31.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa922d9d700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa924108380 0x7fa9241a59b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa91c007ee0 tx=0x7fa91c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:31.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa9137fe700 1 -- 172.21.15.84:0/416370995 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c00dbd0 con 0x7fa924108380 2024-04-26T18:26:31.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa928a19700 1 -- 172.21.15.84:0/416370995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9241a6180 con 0x7fa924108380 2024-04-26T18:26:31.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa9137fe700 1 -- 172.21.15.84:0/416370995 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa91c00dd70 con 0x7fa924108380 2024-04-26T18:26:31.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.377+0000 7fa9137fe700 1 -- 172.21.15.84:0/416370995 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa91c005990 con 0x7fa924108380 2024-04-26T18:26:31.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.378+0000 7fa928a19700 1 -- 172.21.15.84:0/416370995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9241a66e0 con 0x7fa924108380 2024-04-26T18:26:31.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.379+0000 7fa928a19700 1 -- 172.21.15.84:0/416370995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa92406a110 con 0x7fa924108380 2024-04-26T18:26:31.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.380+0000 7fa9137fe700 1 -- 172.21.15.84:0/416370995 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa91c01a020 con 0x7fa924108380 2024-04-26T18:26:31.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.381+0000 7fa9137fe700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa90c0a3c40 0x7fa90c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:31.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.381+0000 7fa9137fe700 1 -- 172.21.15.84:0/416370995 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fa91c0d0160 con 0x7fa924108380 2024-04-26T18:26:31.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.381+0000 7fa92259c700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa90c0a3c40 0x7fa90c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:31.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.381+0000 7fa92259c700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa90c0a3c40 0x7fa90c0a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa9140098e0 tx=0x7fa914005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:31.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.387+0000 7fa9137fe700 1 -- 172.21.15.84:0/416370995 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa91c010070 con 0x7fa924108380 2024-04-26T18:26:31.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.602+0000 7fa928a19700 1 -- 172.21.15.84:0/416370995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fa92406bfe0 con 0x7fa924108380 2024-04-26T18:26:31.603 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.603+0000 7fa9137fe700 1 -- 172.21.15.84:0/416370995 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa91c09cb10 con 0x7fa924108380 2024-04-26T18:26:31.603 INFO:teuthology.orchestra.run.smithi084.stdout:292057776180 2024-04-26T18:26:31.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.605+0000 7fa928a19700 1 -- 172.21.15.84:0/416370995 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa90c0a3c40 msgr2=0x7fa90c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:31.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.605+0000 7fa928a19700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa90c0a3c40 0x7fa90c0a6100 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fa9140098e0 tx=0x7fa914005960 comp rx=0 tx=0).stop 2024-04-26T18:26:31.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.605+0000 7fa928a19700 1 -- 172.21.15.84:0/416370995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa924108380 msgr2=0x7fa9241a59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:31.605 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.605+0000 7fa928a19700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa924108380 0x7fa9241a59b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa91c007ee0 tx=0x7fa91c00d3b0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.605+0000 7fa928a19700 1 -- 172.21.15.84:0/416370995 shutdown_connections 2024-04-26T18:26:31.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.605+0000 7fa928a19700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fa90c0a3c40 0x7fa90c0a6100 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.605+0000 7fa928a19700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa924108380 0x7fa9241a59b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.605+0000 7fa928a19700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa924102310 0x7fa92419a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.605+0000 7fa928a19700 1 --2- 172.21.15.84:0/416370995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa924102cd0 0x7fa92419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:31.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.605+0000 7fa928a19700 1 -- 172.21.15.84:0/416370995 >> 172.21.15.84:0/416370995 conn(0x7fa924079290 msgr2=0x7fa924101150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:31.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.605+0000 7fa928a19700 1 -- 172.21.15.84:0/416370995 shutdown_connections 2024-04-26T18:26:31.606 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:31.605+0000 7fa928a19700 1 -- 172.21.15.84:0/416370995 wait complete. 2024-04-26T18:26:32.046 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299448 got 223338299454 for osd.7 2024-04-26T18:26:32.047 DEBUG:teuthology.parallel:result is None 2024-04-26T18:26:32.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:32 smithi084 ceph-mon[29141]: pgmap v366: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:32.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:32 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/423822411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T18:26:32.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:32 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/416370995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T18:26:32.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:32 smithi103 ceph-mon[32727]: pgmap v366: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:32.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:32 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/423822411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T18:26:32.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:32 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/416370995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T18:26:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:32 smithi174 ceph-mon[32705]: pgmap v366: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:32 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/423822411' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-26T18:26:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:32 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/416370995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-26T18:26:32.835 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776172 got 292057776180 for osd.9 2024-04-26T18:26:32.835 DEBUG:teuthology.parallel:result is None 2024-04-26T18:26:32.835 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T18:26:32.836 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:26:33.095 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:34.200 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:34 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:34.200 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:34 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:34.200 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:34 smithi084 ceph-mon[29141]: pgmap v367: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:34.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:34 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:34.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:34 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:34.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:34 smithi103 ceph-mon[32727]: pgmap v367: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:34.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:34 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:34.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:34 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:34.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:34 smithi174 ceph-mon[32705]: pgmap v367: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:34.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.772+0000 7f69bb455700 1 -- 172.21.15.84:0/3099147986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69b41094a0 msgr2=0x7f69b4109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:34.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.772+0000 7f69bb455700 1 --2- 172.21.15.84:0/3099147986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69b41094a0 0x7f69b4109880 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009990 tx=0x7f69b0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.772+0000 7f69bb455700 1 -- 172.21.15.84:0/3099147986 shutdown_connections 2024-04-26T18:26:34.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.772+0000 7f69bb455700 1 --2- 172.21.15.84:0/3099147986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69b41094a0 0x7f69b4109880 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.774 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.772+0000 7f69bb455700 1 --2- 172.21.15.84:0/3099147986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69b406bad0 0x7f69b41102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.772+0000 7f69bb455700 1 --2- 172.21.15.84:0/3099147986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69b406b130 0x7f69b406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.772+0000 7f69bb455700 1 -- 172.21.15.84:0/3099147986 >> 172.21.15.84:0/3099147986 conn(0x7f69b40792d0 msgr2=0x7f69b40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.772+0000 7f69bb455700 1 -- 172.21.15.84:0/3099147986 shutdown_connections 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.772+0000 7f69bb455700 1 -- 172.21.15.84:0/3099147986 wait complete. 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.772+0000 7f69bb455700 1 Processor -- start 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.772+0000 7f69bb455700 1 -- start start 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69bb455700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69b406b130 0x7f69b419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69bb455700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69b406bad0 0x7f69b419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69bb455700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69b41094a0 0x7f69b4199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69bb455700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69b4113610 con 0x7f69b41094a0 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69bb455700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69b4113490 con 0x7f69b406b130 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69bb455700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69b4113790 con 0x7f69b406bad0 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69b99f2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69b41094a0 0x7f69b4199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:34.775 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69b99f2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69b41094a0 0x7f69b4199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43790/0 (socket says 172.21.15.84:43790) 2024-04-26T18:26:34.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69b99f2700 1 -- 172.21.15.84:0/482759402 learned_addr learned my addr 172.21.15.84:0/482759402 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:34.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69b89f0700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69b406bad0 0x7f69b419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:34.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69b91f1700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69b406b130 0x7f69b419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:34.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69b99f2700 1 -- 172.21.15.84:0/482759402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69b406bad0 msgr2=0x7f69b419fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:34.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69b99f2700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69b406bad0 0x7f69b419fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69b99f2700 1 -- 172.21.15.84:0/482759402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69b406b130 msgr2=0x7f69b419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:34.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69b99f2700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69b406b130 0x7f69b419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69b99f2700 1 -- 172.21.15.84:0/482759402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69b419a0d0 con 0x7f69b41094a0 2024-04-26T18:26:34.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69b99f2700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69b41094a0 0x7f69b4199970 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00b970 tx=0x7f69ac00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:34.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69aa7fc700 1 -- 172.21.15.84:0/482759402 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac019070 con 0x7f69b41094a0 2024-04-26T18:26:34.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69bb455700 1 -- 172.21.15.84:0/482759402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69b419a3c0 con 0x7f69b41094a0 2024-04-26T18:26:34.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.773+0000 7f69aa7fc700 1 -- 172.21.15.84:0/482759402 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69ac00cde0 con 0x7f69b41094a0 2024-04-26T18:26:34.776 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.774+0000 7f69aa7fc700 1 -- 172.21.15.84:0/482759402 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69ac007370 con 0x7f69b41094a0 2024-04-26T18:26:34.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.774+0000 7f69bb455700 1 -- 172.21.15.84:0/482759402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69b41a6680 con 0x7f69b41094a0 2024-04-26T18:26:34.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.775+0000 7f69bb455700 1 -- 172.21.15.84:0/482759402 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69b406a110 con 0x7f69b41094a0 2024-04-26T18:26:34.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.776+0000 7f69aa7fc700 1 -- 172.21.15.84:0/482759402 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f69ac01c070 con 0x7f69b41094a0 2024-04-26T18:26:34.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.776+0000 7f69aa7fc700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f69a00a3c40 0x7f69a00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:34.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.777+0000 7f69b91f1700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f69a00a3c40 0x7f69a00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:34.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.777+0000 7f69aa7fc700 1 -- 172.21.15.84:0/482759402 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f69ac0d0060 con 0x7f69b41094a0 2024-04-26T18:26:34.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.777+0000 7f69b91f1700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f69a00a3c40 0x7f69a00a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f69b0006010 tx=0x7f69b000b540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:34.780 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.780+0000 7f69aa7fc700 1 -- 172.21.15.84:0/482759402 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f69ac023070 con 0x7f69b41094a0 2024-04-26T18:26:34.973 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.973+0000 7f69bb455700 1 -- 172.21.15.84:0/482759402 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f69b419aea0 con 0x7f69a00a3c40 2024-04-26T18:26:34.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.980+0000 7f69aa7fc700 1 -- 172.21.15.84:0/482759402 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259197 (secure 0 0 0) 0x7f69b419aea0 con 0x7f69a00a3c40 2024-04-26T18:26:34.984 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:26:34.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.982+0000 7f69bb455700 1 -- 172.21.15.84:0/482759402 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f69a00a3c40 msgr2=0x7f69a00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:34.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.982+0000 7f69bb455700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f69a00a3c40 0x7f69a00a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f69b0006010 tx=0x7f69b000b540 comp rx=0 tx=0).stop 2024-04-26T18:26:34.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.982+0000 7f69bb455700 1 -- 172.21.15.84:0/482759402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69b41094a0 msgr2=0x7f69b4199970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:34.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.982+0000 7f69bb455700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69b41094a0 0x7f69b4199970 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f69ac00b970 tx=0x7f69ac00bc80 comp rx=0 tx=0).stop 2024-04-26T18:26:34.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.983+0000 7f69bb455700 1 -- 172.21.15.84:0/482759402 shutdown_connections 2024-04-26T18:26:34.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.983+0000 7f69bb455700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f69a00a3c40 0x7f69a00a6100 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.983+0000 7f69bb455700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f69b41094a0 0x7f69b4199970 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.983+0000 7f69bb455700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69b406b130 0x7f69b419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.983+0000 7f69bb455700 1 --2- 172.21.15.84:0/482759402 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f69b406bad0 0x7f69b419fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:34.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.983+0000 7f69bb455700 1 -- 172.21.15.84:0/482759402 >> 172.21.15.84:0/482759402 conn(0x7f69b40792d0 msgr2=0x7f69b4100470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:34.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.983+0000 7f69bb455700 1 -- 172.21.15.84:0/482759402 shutdown_connections 2024-04-26T18:26:34.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:34.983+0000 7f69bb455700 1 -- 172.21.15.84:0/482759402 wait complete. 2024-04-26T18:26:34.987 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-04-26T18:26:35.342 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:35 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:26:35.342 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:35 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:26:35.342 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:35 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:35.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:35 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:26:35.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:35 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:26:35.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:35 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:35.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:35 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:26:35.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:35 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:26:35.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:35 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:26:35.662 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-04-26T18:26:33.213802+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663061,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1257,"num_read_kb":766,"num_write":437,"num_write_kb":3086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182319,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":858,"ondisk_log_size":858,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370084,"kb_used_data":12528,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571740,"statfs":{"total":1151940427776,"available":1151561461760,"internally_reserved":0,"allocated":12828672,"data_stored":7982799,"data_compressed":251142,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":105,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003165"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875849+0000","last_change":"2024-04-26T18:24:20.951581+0000","last_active":"2024-04-26T18:26:07.875849+0000","last_peered":"2024-04-26T18:26:07.875849+0000","last_clean":"2024-04-26T18:26:07.875849+0000","last_became_active":"2024-04-26T18:24:20.951260+0000","last_became_peered":"2024-04-26T18:24:20.951260+0000","last_unstale":"2024-04-26T18:26:07.875849+0000","last_undegraded":"2024-04-26T18:26:07.875849+0000","last_fullsized":"2024-04-26T18:26:07.875849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:54:28.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482171+0000","last_change":"2024-04-26T18:24:20.952263+0000","last_active":"2024-04-26T18:26:07.482171+0000","last_peered":"2024-04-26T18:26:07.482171+0000","last_clean":"2024-04-26T18:26:07.482171+0000","last_became_active":"2024-04-26T18:24:20.951930+0000","last_became_peered":"2024-04-26T18:24:20.951930+0000","last_unstale":"2024-04-26T18:26:07.482171+0000","last_undegraded":"2024-04-26T18:26:07.482171+0000","last_fullsized":"2024-04-26T18:26:07.482171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:54.816572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482303+0000","last_change":"2024-04-26T18:24:20.952280+0000","last_active":"2024-04-26T18:26:07.482303+0000","last_peered":"2024-04-26T18:26:07.482303+0000","last_clean":"2024-04-26T18:26:07.482303+0000","last_became_active":"2024-04-26T18:24:20.951623+0000","last_became_peered":"2024-04-26T18:24:20.951623+0000","last_unstale":"2024-04-26T18:26:07.482303+0000","last_undegraded":"2024-04-26T18:26:07.482303+0000","last_fullsized":"2024-04-26T18:26:07.482303+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:02.502261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302417+0000","last_change":"2024-04-26T18:24:20.962274+0000","last_active":"2024-04-26T18:26:07.302417+0000","last_peered":"2024-04-26T18:26:07.302417+0000","last_clean":"2024-04-26T18:26:07.302417+0000","last_became_active":"2024-04-26T18:24:20.961920+0000","last_became_peered":"2024-04-26T18:24:20.961920+0000","last_unstale":"2024-04-26T18:26:07.302417+0000","last_undegraded":"2024-04-26T18:26:07.302417+0000","last_fullsized":"2024-04-26T18:26:07.302417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:49:46.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.333143+0000","last_change":"2024-04-26T18:24:20.949096+0000","last_active":"2024-04-26T18:26:07.333143+0000","last_peered":"2024-04-26T18:26:07.333143+0000","last_clean":"2024-04-26T18:26:07.333143+0000","last_became_active":"2024-04-26T18:24:20.948737+0000","last_became_peered":"2024-04-26T18:24:20.948737+0000","last_unstale":"2024-04-26T18:26:07.333143+0000","last_undegraded":"2024-04-26T18:26:07.333143+0000","last_fullsized":"2024-04-26T18:26:07.333143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:12:40.371589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.303947+0000","last_change":"2024-04-26T18:24:20.962670+0000","last_active":"2024-04-26T18:26:07.303947+0000","last_peered":"2024-04-26T18:26:07.303947+0000","last_clean":"2024-04-26T18:26:07.303947+0000","last_became_active":"2024-04-26T18:24:20.962309+0000","last_became_peered":"2024-04-26T18:24:20.962309+0000","last_unstale":"2024-04-26T18:26:07.303947+0000","last_undegraded":"2024-04-26T18:26:07.303947+0000","last_fullsized":"2024-04-26T18:26:07.303947+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:40.315864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432274+0000","last_change":"2024-04-26T18:25:06.350579+0000","last_active":"2024-04-26T18:26:07.432274+0000","last_peered":"2024-04-26T18:26:07.432274+0000","last_clean":"2024-04-26T18:26:07.432274+0000","last_became_active":"2024-04-26T18:25:06.350104+0000","last_became_peered":"2024-04-26T18:25:06.350104+0000","last_unstale":"2024-04-26T18:26:07.432274+0000","last_undegraded":"2024-04-26T18:26:07.432274+0000","last_fullsized":"2024-04-26T18:26:07.432274+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:00.950883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.481888+0000","last_change":"2024-04-26T18:24:20.961096+0000","last_active":"2024-04-26T18:26:07.481888+0000","last_peered":"2024-04-26T18:26:07.481888+0000","last_clean":"2024-04-26T18:26:07.481888+0000","last_became_active":"2024-04-26T18:24:20.960796+0000","last_became_peered":"2024-04-26T18:24:20.960796+0000","last_unstale":"2024-04-26T18:26:07.481888+0000","last_undegraded":"2024-04-26T18:26:07.481888+0000","last_fullsized":"2024-04-26T18:26:07.481888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:04.538836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.410236+0000","last_change":"2024-04-26T18:24:20.959871+0000","last_active":"2024-04-26T18:26:07.410236+0000","last_peered":"2024-04-26T18:26:07.410236+0000","last_clean":"2024-04-26T18:26:07.410236+0000","last_became_active":"2024-04-26T18:24:20.959404+0000","last_became_peered":"2024-04-26T18:24:20.959404+0000","last_unstale":"2024-04-26T18:26:07.410236+0000","last_undegraded":"2024-04-26T18:26:07.410236+0000","last_fullsized":"2024-04-26T18:26:07.410236+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:43.053618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.431543+0000","last_change":"2024-04-26T18:26:07.431543+0000","last_active":"2024-04-26T18:26:07.431543+0000","last_peered":"2024-04-26T18:26:07.431543+0000","last_clean":"2024-04-26T18:26:07.431543+0000","last_became_active":"2024-04-26T18:26:07.430976+0000","last_became_peered":"2024-04-26T18:26:07.430976+0000","last_unstale":"2024-04-26T18:26:07.431543+0000","last_undegraded":"2024-04-26T18:26:07.431543+0000","last_fullsized":"2024-04-26T18:26:07.431543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:31.430010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432641+0000","last_change":"2024-04-26T18:24:20.952109+0000","last_active":"2024-04-26T18:26:07.432641+0000","last_peered":"2024-04-26T18:26:07.432641+0000","last_clean":"2024-04-26T18:26:07.432641+0000","last_became_active":"2024-04-26T18:24:20.951776+0000","last_became_peered":"2024-04-26T18:24:20.951776+0000","last_unstale":"2024-04-26T18:26:07.432641+0000","last_undegraded":"2024-04-26T18:26:07.432641+0000","last_fullsized":"2024-04-26T18:26:07.432641+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:53.220246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302315+0000","last_change":"2024-04-26T18:24:20.963965+0000","last_active":"2024-04-26T18:26:07.302315+0000","last_peered":"2024-04-26T18:26:07.302315+0000","last_clean":"2024-04-26T18:26:07.302315+0000","last_became_active":"2024-04-26T18:24:20.963648+0000","last_became_peered":"2024-04-26T18:24:20.963648+0000","last_unstale":"2024-04-26T18:26:07.302315+0000","last_undegraded":"2024-04-26T18:26:07.302315+0000","last_fullsized":"2024-04-26T18:26:07.302315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:04.080813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296837+0000","last_change":"2024-04-26T18:24:20.954186+0000","last_active":"2024-04-26T18:26:07.296837+0000","last_peered":"2024-04-26T18:26:07.296837+0000","last_clean":"2024-04-26T18:26:07.296837+0000","last_became_active":"2024-04-26T18:24:20.953431+0000","last_became_peered":"2024-04-26T18:24:20.953431+0000","last_unstale":"2024-04-26T18:26:07.296837+0000","last_undegraded":"2024-04-26T18:26:07.296837+0000","last_fullsized":"2024-04-26T18:26:07.296837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:21.498476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.481889+0000","last_change":"2024-04-26T18:24:20.945392+0000","last_active":"2024-04-26T18:26:07.481889+0000","last_peered":"2024-04-26T18:26:07.481889+0000","last_clean":"2024-04-26T18:26:07.481889+0000","last_became_active":"2024-04-26T18:24:20.944971+0000","last_became_peered":"2024-04-26T18:24:20.944971+0000","last_unstale":"2024-04-26T18:26:07.481889+0000","last_undegraded":"2024-04-26T18:26:07.481889+0000","last_fullsized":"2024-04-26T18:26:07.481889+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:06.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482071+0000","last_change":"2024-04-26T18:25:06.299351+0000","last_active":"2024-04-26T18:26:07.482071+0000","last_peered":"2024-04-26T18:26:07.482071+0000","last_clean":"2024-04-26T18:26:07.482071+0000","last_became_active":"2024-04-26T18:25:06.298895+0000","last_became_peered":"2024-04-26T18:25:06.298895+0000","last_unstale":"2024-04-26T18:26:07.482071+0000","last_undegraded":"2024-04-26T18:26:07.482071+0000","last_fullsized":"2024-04-26T18:26:07.482071+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:37.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.409813+0000","last_change":"2024-04-26T18:24:20.954610+0000","last_active":"2024-04-26T18:26:07.409813+0000","last_peered":"2024-04-26T18:26:07.409813+0000","last_clean":"2024-04-26T18:26:07.409813+0000","last_became_active":"2024-04-26T18:24:20.954239+0000","last_became_peered":"2024-04-26T18:24:20.954239+0000","last_unstale":"2024-04-26T18:26:07.409813+0000","last_undegraded":"2024-04-26T18:26:07.409813+0000","last_fullsized":"2024-04-26T18:26:07.409813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:41.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875627+0000","last_change":"2024-04-26T18:24:20.956932+0000","last_active":"2024-04-26T18:26:07.875627+0000","last_peered":"2024-04-26T18:26:07.875627+0000","last_clean":"2024-04-26T18:26:07.875627+0000","last_became_active":"2024-04-26T18:24:20.956464+0000","last_became_peered":"2024-04-26T18:24:20.956464+0000","last_unstale":"2024-04-26T18:26:07.875627+0000","last_undegraded":"2024-04-26T18:26:07.875627+0000","last_fullsized":"2024-04-26T18:26:07.875627+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:01.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.429951+0000","last_change":"2024-04-26T18:24:20.954870+0000","last_active":"2024-04-26T18:26:07.429951+0000","last_peered":"2024-04-26T18:26:07.429951+0000","last_clean":"2024-04-26T18:26:07.429951+0000","last_became_active":"2024-04-26T18:24:20.954542+0000","last_became_peered":"2024-04-26T18:24:20.954542+0000","last_unstale":"2024-04-26T18:26:07.429951+0000","last_undegraded":"2024-04-26T18:26:07.429951+0000","last_fullsized":"2024-04-26T18:26:07.429951+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:13:51.932625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.306394+0000","last_change":"2024-04-26T18:24:20.950994+0000","last_active":"2024-04-26T18:26:07.306394+0000","last_peered":"2024-04-26T18:26:07.306394+0000","last_clean":"2024-04-26T18:26:07.306394+0000","last_became_active":"2024-04-26T18:24:20.950343+0000","last_became_peered":"2024-04-26T18:24:20.950343+0000","last_unstale":"2024-04-26T18:26:07.306394+0000","last_undegraded":"2024-04-26T18:26:07.306394+0000","last_fullsized":"2024-04-26T18:26:07.306394+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:25.274049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":154,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:26.474195+0000","last_change":"2024-04-26T18:24:17.929731+0000","last_active":"2024-04-26T18:26:26.474195+0000","last_peered":"2024-04-26T18:26:26.474195+0000","last_clean":"2024-04-26T18:26:26.474195+0000","last_became_active":"2024-04-26T18:24:17.929379+0000","last_became_peered":"2024-04-26T18:24:17.929379+0000","last_unstale":"2024-04-26T18:26:26.474195+0000","last_undegraded":"2024-04-26T18:26:26.474195+0000","last_fullsized":"2024-04-26T18:26:26.474195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:44.821660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":110,"num_read_kb":110,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482804+0000","last_change":"2024-04-26T18:24:20.962885+0000","last_active":"2024-04-26T18:26:07.482804+0000","last_peered":"2024-04-26T18:26:07.482804+0000","last_clean":"2024-04-26T18:26:07.482804+0000","last_became_active":"2024-04-26T18:24:20.962674+0000","last_became_peered":"2024-04-26T18:24:20.962674+0000","last_unstale":"2024-04-26T18:26:07.482804+0000","last_undegraded":"2024-04-26T18:26:07.482804+0000","last_fullsized":"2024-04-26T18:26:07.482804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:32.444889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482977+0000","last_change":"2024-04-26T18:24:18.141500+0000","last_active":"2024-04-26T18:26:07.482977+0000","last_peered":"2024-04-26T18:26:07.482977+0000","last_clean":"2024-04-26T18:26:07.482977+0000","last_became_active":"2024-04-26T18:24:18.140870+0000","last_became_peered":"2024-04-26T18:24:18.140870+0000","last_unstale":"2024-04-26T18:26:07.482977+0000","last_undegraded":"2024-04-26T18:26:07.482977+0000","last_fullsized":"2024-04-26T18:26:07.482977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:13.821872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482384+0000","last_change":"2024-04-26T18:24:20.949183+0000","last_active":"2024-04-26T18:26:07.482384+0000","last_peered":"2024-04-26T18:26:07.482384+0000","last_clean":"2024-04-26T18:26:07.482384+0000","last_became_active":"2024-04-26T18:24:20.948677+0000","last_became_peered":"2024-04-26T18:24:20.948677+0000","last_unstale":"2024-04-26T18:26:07.482384+0000","last_undegraded":"2024-04-26T18:26:07.482384+0000","last_fullsized":"2024-04-26T18:26:07.482384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:55.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":65,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.333289+0000","last_change":"2024-04-26T18:25:06.312239+0000","last_active":"2024-04-26T18:26:07.333289+0000","last_peered":"2024-04-26T18:26:07.333289+0000","last_clean":"2024-04-26T18:26:07.333289+0000","last_became_active":"2024-04-26T18:25:06.301304+0000","last_became_peered":"2024-04-26T18:25:06.301304+0000","last_unstale":"2024-04-26T18:26:07.333289+0000","last_undegraded":"2024-04-26T18:26:07.333289+0000","last_fullsized":"2024-04-26T18:26:07.333289+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:36.159348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.409938+0000","last_change":"2024-04-26T18:24:20.955555+0000","last_active":"2024-04-26T18:26:07.409938+0000","last_peered":"2024-04-26T18:26:07.409938+0000","last_clean":"2024-04-26T18:26:07.409938+0000","last_became_active":"2024-04-26T18:24:20.955176+0000","last_became_peered":"2024-04-26T18:24:20.955176+0000","last_unstale":"2024-04-26T18:26:07.409938+0000","last_undegraded":"2024-04-26T18:26:07.409938+0000","last_fullsized":"2024-04-26T18:26:07.409938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:02:30.368971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.333197+0000","last_change":"2024-04-26T18:24:17.930919+0000","last_active":"2024-04-26T18:26:07.333197+0000","last_peered":"2024-04-26T18:26:07.333197+0000","last_clean":"2024-04-26T18:26:07.333197+0000","last_became_active":"2024-04-26T18:24:17.930596+0000","last_became_peered":"2024-04-26T18:24:17.930596+0000","last_unstale":"2024-04-26T18:26:07.333197+0000","last_undegraded":"2024-04-26T18:26:07.333197+0000","last_fullsized":"2024-04-26T18:26:07.333197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:57.476027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302186+0000","last_change":"2024-04-26T18:24:20.955796+0000","last_active":"2024-04-26T18:26:07.302186+0000","last_peered":"2024-04-26T18:26:07.302186+0000","last_clean":"2024-04-26T18:26:07.302186+0000","last_became_active":"2024-04-26T18:24:20.955432+0000","last_became_peered":"2024-04-26T18:24:20.955432+0000","last_unstale":"2024-04-26T18:26:07.302186+0000","last_undegraded":"2024-04-26T18:26:07.302186+0000","last_fullsized":"2024-04-26T18:26:07.302186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:25.161797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":152,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:28.466017+0000","last_change":"2024-04-26T18:24:17.929505+0000","last_active":"2024-04-26T18:26:28.466017+0000","last_peered":"2024-04-26T18:26:28.466017+0000","last_clean":"2024-04-26T18:26:28.466017+0000","last_became_active":"2024-04-26T18:24:17.929205+0000","last_became_peered":"2024-04-26T18:24:17.929205+0000","last_unstale":"2024-04-26T18:26:28.466017+0000","last_undegraded":"2024-04-26T18:26:28.466017+0000","last_fullsized":"2024-04-26T18:26:28.466017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:42.460063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.332934+0000","last_change":"2024-04-26T18:24:20.952134+0000","last_active":"2024-04-26T18:26:07.332934+0000","last_peered":"2024-04-26T18:26:07.332934+0000","last_clean":"2024-04-26T18:26:07.332934+0000","last_became_active":"2024-04-26T18:24:20.951758+0000","last_became_peered":"2024-04-26T18:24:20.951758+0000","last_unstale":"2024-04-26T18:26:07.332934+0000","last_undegraded":"2024-04-26T18:26:07.332934+0000","last_fullsized":"2024-04-26T18:26:07.332934+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:25.389221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.303910+0000","last_change":"2024-04-26T18:24:17.928704+0000","last_active":"2024-04-26T18:26:07.303910+0000","last_peered":"2024-04-26T18:26:07.303910+0000","last_clean":"2024-04-26T18:26:07.303910+0000","last_became_active":"2024-04-26T18:24:17.928343+0000","last_became_peered":"2024-04-26T18:24:17.928343+0000","last_unstale":"2024-04-26T18:26:07.303910+0000","last_undegraded":"2024-04-26T18:26:07.303910+0000","last_fullsized":"2024-04-26T18:26:07.303910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:49.400481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302088+0000","last_change":"2024-04-26T18:24:20.956808+0000","last_active":"2024-04-26T18:26:07.302088+0000","last_peered":"2024-04-26T18:26:07.302088+0000","last_clean":"2024-04-26T18:26:07.302088+0000","last_became_active":"2024-04-26T18:24:20.956466+0000","last_became_peered":"2024-04-26T18:24:20.956466+0000","last_unstale":"2024-04-26T18:26:07.302088+0000","last_undegraded":"2024-04-26T18:26:07.302088+0000","last_fullsized":"2024-04-26T18:26:07.302088+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:31.150670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296535+0000","last_change":"2024-04-26T18:24:17.929134+0000","last_active":"2024-04-26T18:26:07.296535+0000","last_peered":"2024-04-26T18:26:07.296535+0000","last_clean":"2024-04-26T18:26:07.296535+0000","last_became_active":"2024-04-26T18:24:17.928798+0000","last_became_peered":"2024-04-26T18:24:17.928798+0000","last_unstale":"2024-04-26T18:26:07.296535+0000","last_undegraded":"2024-04-26T18:26:07.296535+0000","last_fullsized":"2024-04-26T18:26:07.296535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:10.450825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.481760+0000","last_change":"2024-04-26T18:24:20.951594+0000","last_active":"2024-04-26T18:26:07.481760+0000","last_peered":"2024-04-26T18:26:07.481760+0000","last_clean":"2024-04-26T18:26:07.481760+0000","last_became_active":"2024-04-26T18:24:20.951148+0000","last_became_peered":"2024-04-26T18:24:20.951148+0000","last_unstale":"2024-04-26T18:26:07.481760+0000","last_undegraded":"2024-04-26T18:26:07.481760+0000","last_fullsized":"2024-04-26T18:26:07.481760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:18.703292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.301797+0000","last_change":"2024-04-26T18:24:17.931101+0000","last_active":"2024-04-26T18:26:07.301797+0000","last_peered":"2024-04-26T18:26:07.301797+0000","last_clean":"2024-04-26T18:26:07.301797+0000","last_became_active":"2024-04-26T18:24:17.930819+0000","last_became_peered":"2024-04-26T18:24:17.930819+0000","last_unstale":"2024-04-26T18:26:07.301797+0000","last_undegraded":"2024-04-26T18:26:07.301797+0000","last_fullsized":"2024-04-26T18:26:07.301797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.429381+0000","last_change":"2024-04-26T18:24:20.948866+0000","last_active":"2024-04-26T18:26:07.429381+0000","last_peered":"2024-04-26T18:26:07.429381+0000","last_clean":"2024-04-26T18:26:07.429381+0000","last_became_active":"2024-04-26T18:24:20.948562+0000","last_became_peered":"2024-04-26T18:24:20.948562+0000","last_unstale":"2024-04-26T18:26:07.429381+0000","last_undegraded":"2024-04-26T18:26:07.429381+0000","last_fullsized":"2024-04-26T18:26:07.429381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:33:19.616660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":255,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:28.445176+0000","last_change":"2024-04-26T18:24:17.926384+0000","last_active":"2024-04-26T18:26:28.445176+0000","last_peered":"2024-04-26T18:26:28.445176+0000","last_clean":"2024-04-26T18:26:28.445176+0000","last_became_active":"2024-04-26T18:24:17.926037+0000","last_became_peered":"2024-04-26T18:24:17.926037+0000","last_unstale":"2024-04-26T18:26:28.445176+0000","last_undegraded":"2024-04-26T18:26:28.445176+0000","last_fullsized":"2024-04-26T18:26:28.445176+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:05.283755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":219,"num_read_kb":219,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.410206+0000","last_change":"2024-04-26T18:24:20.954257+0000","last_active":"2024-04-26T18:26:07.410206+0000","last_peered":"2024-04-26T18:26:07.410206+0000","last_clean":"2024-04-26T18:26:07.410206+0000","last_became_active":"2024-04-26T18:24:20.953913+0000","last_became_peered":"2024-04-26T18:24:20.953913+0000","last_unstale":"2024-04-26T18:26:07.410206+0000","last_undegraded":"2024-04-26T18:26:07.410206+0000","last_fullsized":"2024-04-26T18:26:07.410206+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:06.814156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432469+0000","last_change":"2024-04-26T18:24:17.932843+0000","last_active":"2024-04-26T18:26:07.432469+0000","last_peered":"2024-04-26T18:26:07.432469+0000","last_clean":"2024-04-26T18:26:07.432469+0000","last_became_active":"2024-04-26T18:24:17.932502+0000","last_became_peered":"2024-04-26T18:24:17.932502+0000","last_unstale":"2024-04-26T18:26:07.432469+0000","last_undegraded":"2024-04-26T18:26:07.432469+0000","last_fullsized":"2024-04-26T18:26:07.432469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:49.727094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482355+0000","last_change":"2024-04-26T18:24:20.959881+0000","last_active":"2024-04-26T18:26:07.482355+0000","last_peered":"2024-04-26T18:26:07.482355+0000","last_clean":"2024-04-26T18:26:07.482355+0000","last_became_active":"2024-04-26T18:24:20.959541+0000","last_became_peered":"2024-04-26T18:24:20.959541+0000","last_unstale":"2024-04-26T18:26:07.482355+0000","last_undegraded":"2024-04-26T18:26:07.482355+0000","last_fullsized":"2024-04-26T18:26:07.482355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:23.393521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302266+0000","last_change":"2024-04-26T18:24:18.142146+0000","last_active":"2024-04-26T18:26:07.302266+0000","last_peered":"2024-04-26T18:26:07.302266+0000","last_clean":"2024-04-26T18:26:07.302266+0000","last_became_active":"2024-04-26T18:24:18.141829+0000","last_became_peered":"2024-04-26T18:24:18.141829+0000","last_unstale":"2024-04-26T18:26:07.302266+0000","last_undegraded":"2024-04-26T18:26:07.302266+0000","last_fullsized":"2024-04-26T18:26:07.302266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:49.341638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.410114+0000","last_change":"2024-04-26T18:24:20.957993+0000","last_active":"2024-04-26T18:26:07.410114+0000","last_peered":"2024-04-26T18:26:07.410114+0000","last_clean":"2024-04-26T18:26:07.410114+0000","last_became_active":"2024-04-26T18:24:20.957443+0000","last_became_peered":"2024-04-26T18:24:20.957443+0000","last_unstale":"2024-04-26T18:26:07.410114+0000","last_undegraded":"2024-04-26T18:26:07.410114+0000","last_fullsized":"2024-04-26T18:26:07.410114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:06:32.960546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":296,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:28.465532+0000","last_change":"2024-04-26T18:24:17.931283+0000","last_active":"2024-04-26T18:26:28.465532+0000","last_peered":"2024-04-26T18:26:28.465532+0000","last_clean":"2024-04-26T18:26:28.465532+0000","last_became_active":"2024-04-26T18:24:17.930943+0000","last_became_peered":"2024-04-26T18:24:17.930943+0000","last_unstale":"2024-04-26T18:26:28.465532+0000","last_undegraded":"2024-04-26T18:26:28.465532+0000","last_fullsized":"2024-04-26T18:26:28.465532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:50.511677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64973,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":230,"num_read_kb":178,"num_write":38,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.333092+0000","last_change":"2024-04-26T18:24:20.947509+0000","last_active":"2024-04-26T18:26:07.333092+0000","last_peered":"2024-04-26T18:26:07.333092+0000","last_clean":"2024-04-26T18:26:07.333092+0000","last_became_active":"2024-04-26T18:24:20.947181+0000","last_became_peered":"2024-04-26T18:24:20.947181+0000","last_unstale":"2024-04-26T18:26:07.333092+0000","last_undegraded":"2024-04-26T18:26:07.333092+0000","last_fullsized":"2024-04-26T18:26:07.333092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.262312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.306181+0000","last_change":"2024-04-26T18:24:17.934042+0000","last_active":"2024-04-26T18:26:07.306181+0000","last_peered":"2024-04-26T18:26:07.306181+0000","last_clean":"2024-04-26T18:26:07.306181+0000","last_became_active":"2024-04-26T18:24:17.933537+0000","last_became_peered":"2024-04-26T18:24:17.933537+0000","last_unstale":"2024-04-26T18:26:07.306181+0000","last_undegraded":"2024-04-26T18:26:07.306181+0000","last_fullsized":"2024-04-26T18:26:07.306181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:35:06.904160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296455+0000","last_change":"2024-04-26T18:24:20.957672+0000","last_active":"2024-04-26T18:26:07.296455+0000","last_peered":"2024-04-26T18:26:07.296455+0000","last_clean":"2024-04-26T18:26:07.296455+0000","last_became_active":"2024-04-26T18:24:20.957214+0000","last_became_peered":"2024-04-26T18:24:20.957214+0000","last_unstale":"2024-04-26T18:26:07.296455+0000","last_undegraded":"2024-04-26T18:26:07.296455+0000","last_fullsized":"2024-04-26T18:26:07.296455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:42:35.341858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.301840+0000","last_change":"2024-04-26T18:24:17.930324+0000","last_active":"2024-04-26T18:26:07.301840+0000","last_peered":"2024-04-26T18:26:07.301840+0000","last_clean":"2024-04-26T18:26:07.301840+0000","last_became_active":"2024-04-26T18:24:17.930022+0000","last_became_peered":"2024-04-26T18:24:17.930022+0000","last_unstale":"2024-04-26T18:26:07.301840+0000","last_undegraded":"2024-04-26T18:26:07.301840+0000","last_fullsized":"2024-04-26T18:26:07.301840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:36.864816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432366+0000","last_change":"2024-04-26T18:25:06.350388+0000","last_active":"2024-04-26T18:26:07.432366+0000","last_peered":"2024-04-26T18:26:07.432366+0000","last_clean":"2024-04-26T18:26:07.432366+0000","last_became_active":"2024-04-26T18:25:06.349836+0000","last_became_peered":"2024-04-26T18:25:06.349836+0000","last_unstale":"2024-04-26T18:26:07.432366+0000","last_undegraded":"2024-04-26T18:26:07.432366+0000","last_fullsized":"2024-04-26T18:26:07.432366+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:57.762115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:28.501619+0000","last_change":"2024-04-26T18:24:17.934676+0000","last_active":"2024-04-26T18:26:28.501619+0000","last_peered":"2024-04-26T18:26:28.501619+0000","last_clean":"2024-04-26T18:26:28.501619+0000","last_became_active":"2024-04-26T18:24:17.934307+0000","last_became_peered":"2024-04-26T18:24:17.934307+0000","last_unstale":"2024-04-26T18:26:28.501619+0000","last_undegraded":"2024-04-26T18:26:28.501619+0000","last_fullsized":"2024-04-26T18:26:28.501619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:27.312248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":114,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432020+0000","last_change":"2024-04-26T18:24:20.947371+0000","last_active":"2024-04-26T18:26:07.432020+0000","last_peered":"2024-04-26T18:26:07.432020+0000","last_clean":"2024-04-26T18:26:07.432020+0000","last_became_active":"2024-04-26T18:24:20.947048+0000","last_became_peered":"2024-04-26T18:24:20.947048+0000","last_unstale":"2024-04-26T18:26:07.432020+0000","last_undegraded":"2024-04-26T18:26:07.432020+0000","last_fullsized":"2024-04-26T18:26:07.432020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:55.038074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.332861+0000","last_change":"2024-04-26T18:24:18.142680+0000","last_active":"2024-04-26T18:26:07.332861+0000","last_peered":"2024-04-26T18:26:07.332861+0000","last_clean":"2024-04-26T18:26:07.332861+0000","last_became_active":"2024-04-26T18:24:18.142392+0000","last_became_peered":"2024-04-26T18:24:18.142392+0000","last_unstale":"2024-04-26T18:26:07.332861+0000","last_undegraded":"2024-04-26T18:26:07.332861+0000","last_fullsized":"2024-04-26T18:26:07.332861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:52.654813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.301767+0000","last_change":"2024-04-26T18:24:20.958131+0000","last_active":"2024-04-26T18:26:07.301767+0000","last_peered":"2024-04-26T18:26:07.301767+0000","last_clean":"2024-04-26T18:26:07.301767+0000","last_became_active":"2024-04-26T18:24:20.957701+0000","last_became_peered":"2024-04-26T18:24:20.957701+0000","last_unstale":"2024-04-26T18:26:07.301767+0000","last_undegraded":"2024-04-26T18:26:07.301767+0000","last_fullsized":"2024-04-26T18:26:07.301767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:02.018158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875672+0000","last_change":"2024-04-26T18:25:06.319173+0000","last_active":"2024-04-26T18:26:07.875672+0000","last_peered":"2024-04-26T18:26:07.875672+0000","last_clean":"2024-04-26T18:26:07.875672+0000","last_became_active":"2024-04-26T18:25:06.302935+0000","last_became_peered":"2024-04-26T18:25:06.302935+0000","last_unstale":"2024-04-26T18:26:07.875672+0000","last_undegraded":"2024-04-26T18:26:07.875672+0000","last_fullsized":"2024-04-26T18:26:07.875672+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:40.198105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875707+0000","last_change":"2024-04-26T18:24:20.961230+0000","last_active":"2024-04-26T18:26:07.875707+0000","last_peered":"2024-04-26T18:26:07.875707+0000","last_clean":"2024-04-26T18:26:07.875707+0000","last_became_active":"2024-04-26T18:24:20.960807+0000","last_became_peered":"2024-04-26T18:24:20.960807+0000","last_unstale":"2024-04-26T18:26:07.875707+0000","last_undegraded":"2024-04-26T18:26:07.875707+0000","last_fullsized":"2024-04-26T18:26:07.875707+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:39.535100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.429630+0000","last_change":"2024-04-26T18:24:18.142082+0000","last_active":"2024-04-26T18:26:07.429630+0000","last_peered":"2024-04-26T18:26:07.429630+0000","last_clean":"2024-04-26T18:26:07.429630+0000","last_became_active":"2024-04-26T18:24:18.141765+0000","last_became_peered":"2024-04-26T18:24:18.141765+0000","last_unstale":"2024-04-26T18:26:07.429630+0000","last_undegraded":"2024-04-26T18:26:07.429630+0000","last_fullsized":"2024-04-26T18:26:07.429630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:02.140027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.304077+0000","last_change":"2024-04-26T18:24:20.952867+0000","last_active":"2024-04-26T18:26:07.304077+0000","last_peered":"2024-04-26T18:26:07.304077+0000","last_clean":"2024-04-26T18:26:07.304077+0000","last_became_active":"2024-04-26T18:24:20.952525+0000","last_became_peered":"2024-04-26T18:24:20.952525+0000","last_unstale":"2024-04-26T18:26:07.304077+0000","last_undegraded":"2024-04-26T18:26:07.304077+0000","last_fullsized":"2024-04-26T18:26:07.304077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:52.804312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":149,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:29.478868+0000","last_change":"2024-04-26T18:24:17.932270+0000","last_active":"2024-04-26T18:26:29.478868+0000","last_peered":"2024-04-26T18:26:29.478868+0000","last_clean":"2024-04-26T18:26:29.478868+0000","last_became_active":"2024-04-26T18:24:17.931901+0000","last_became_peered":"2024-04-26T18:24:17.931901+0000","last_unstale":"2024-04-26T18:26:29.478868+0000","last_undegraded":"2024-04-26T18:26:29.478868+0000","last_fullsized":"2024-04-26T18:26:29.478868+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:13.524174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.304162+0000","last_change":"2024-04-26T18:24:20.960612+0000","last_active":"2024-04-26T18:26:07.304162+0000","last_peered":"2024-04-26T18:26:07.304162+0000","last_clean":"2024-04-26T18:26:07.304162+0000","last_became_active":"2024-04-26T18:24:20.960280+0000","last_became_peered":"2024-04-26T18:24:20.960280+0000","last_unstale":"2024-04-26T18:26:07.304162+0000","last_undegraded":"2024-04-26T18:26:07.304162+0000","last_fullsized":"2024-04-26T18:26:07.304162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:53.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.429368+0000","last_change":"2024-04-26T18:25:06.299736+0000","last_active":"2024-04-26T18:26:07.429368+0000","last_peered":"2024-04-26T18:26:07.429368+0000","last_clean":"2024-04-26T18:26:07.429368+0000","last_became_active":"2024-04-26T18:25:06.299265+0000","last_became_peered":"2024-04-26T18:25:06.299265+0000","last_unstale":"2024-04-26T18:26:07.429368+0000","last_undegraded":"2024-04-26T18:26:07.429368+0000","last_fullsized":"2024-04-26T18:26:07.429368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:32.427551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":148,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:28.500879+0000","last_change":"2024-04-26T18:24:18.141181+0000","last_active":"2024-04-26T18:26:28.500879+0000","last_peered":"2024-04-26T18:26:28.500879+0000","last_clean":"2024-04-26T18:26:28.500879+0000","last_became_active":"2024-04-26T18:24:18.140854+0000","last_became_peered":"2024-04-26T18:24:18.140854+0000","last_unstale":"2024-04-26T18:26:28.500879+0000","last_undegraded":"2024-04-26T18:26:28.500879+0000","last_fullsized":"2024-04-26T18:26:28.500879+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:27.900701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482931+0000","last_change":"2024-04-26T18:24:20.961653+0000","last_active":"2024-04-26T18:26:07.482931+0000","last_peered":"2024-04-26T18:26:07.482931+0000","last_clean":"2024-04-26T18:26:07.482931+0000","last_became_active":"2024-04-26T18:24:20.961330+0000","last_became_peered":"2024-04-26T18:24:20.961330+0000","last_unstale":"2024-04-26T18:26:07.482931+0000","last_undegraded":"2024-04-26T18:26:07.482931+0000","last_fullsized":"2024-04-26T18:26:07.482931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:19.033932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302130+0000","last_change":"2024-04-26T18:24:17.933613+0000","last_active":"2024-04-26T18:26:07.302130+0000","last_peered":"2024-04-26T18:26:07.302130+0000","last_clean":"2024-04-26T18:26:07.302130+0000","last_became_active":"2024-04-26T18:24:17.933219+0000","last_became_peered":"2024-04-26T18:24:17.933219+0000","last_unstale":"2024-04-26T18:26:07.302130+0000","last_undegraded":"2024-04-26T18:26:07.302130+0000","last_fullsized":"2024-04-26T18:26:07.302130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:04.215821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432119+0000","last_change":"2024-04-26T18:22:57.585634+0000","last_active":"2024-04-26T18:26:07.432119+0000","last_peered":"2024-04-26T18:26:07.432119+0000","last_clean":"2024-04-26T18:26:07.432119+0000","last_became_active":"2024-04-26T18:22:57.585198+0000","last_became_peered":"2024-04-26T18:22:57.585198+0000","last_unstale":"2024-04-26T18:26:07.432119+0000","last_undegraded":"2024-04-26T18:26:07.432119+0000","last_fullsized":"2024-04-26T18:26:07.432119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_clean_scrub_stamp":"2024-04-26T18:19:31.594684+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:56:11.861930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296467+0000","last_change":"2024-04-26T18:24:20.950303+0000","last_active":"2024-04-26T18:26:07.296467+0000","last_peered":"2024-04-26T18:26:07.296467+0000","last_clean":"2024-04-26T18:26:07.296467+0000","last_became_active":"2024-04-26T18:24:20.950002+0000","last_became_peered":"2024-04-26T18:24:20.950002+0000","last_unstale":"2024-04-26T18:26:07.296467+0000","last_undegraded":"2024-04-26T18:26:07.296467+0000","last_fullsized":"2024-04-26T18:26:07.296467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:00.912752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432283+0000","last_change":"2024-04-26T18:24:17.934924+0000","last_active":"2024-04-26T18:26:07.432283+0000","last_peered":"2024-04-26T18:26:07.432283+0000","last_clean":"2024-04-26T18:26:07.432283+0000","last_became_active":"2024-04-26T18:24:17.934598+0000","last_became_peered":"2024-04-26T18:24:17.934598+0000","last_unstale":"2024-04-26T18:26:07.432283+0000","last_undegraded":"2024-04-26T18:26:07.432283+0000","last_fullsized":"2024-04-26T18:26:07.432283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:28.690043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.333242+0000","last_change":"2024-04-26T18:24:18.142560+0000","last_active":"2024-04-26T18:26:07.333242+0000","last_peered":"2024-04-26T18:26:07.333242+0000","last_clean":"2024-04-26T18:26:07.333242+0000","last_became_active":"2024-04-26T18:24:18.142184+0000","last_became_peered":"2024-04-26T18:24:18.142184+0000","last_unstale":"2024-04-26T18:26:07.333242+0000","last_undegraded":"2024-04-26T18:26:07.333242+0000","last_fullsized":"2024-04-26T18:26:07.333242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:26.656113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296718+0000","last_change":"2024-04-26T18:24:20.952073+0000","last_active":"2024-04-26T18:26:07.296718+0000","last_peered":"2024-04-26T18:26:07.296718+0000","last_clean":"2024-04-26T18:26:07.296718+0000","last_became_active":"2024-04-26T18:24:20.951761+0000","last_became_peered":"2024-04-26T18:24:20.951761+0000","last_unstale":"2024-04-26T18:26:07.296718+0000","last_undegraded":"2024-04-26T18:26:07.296718+0000","last_fullsized":"2024-04-26T18:26:07.296718+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:30.244060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'15","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.429895+0000","last_change":"2024-04-26T18:24:17.927171+0000","last_active":"2024-04-26T18:26:07.429895+0000","last_peered":"2024-04-26T18:26:07.429895+0000","last_clean":"2024-04-26T18:26:07.429895+0000","last_became_active":"2024-04-26T18:24:17.926816+0000","last_became_peered":"2024-04-26T18:24:17.926816+0000","last_unstale":"2024-04-26T18:26:07.429895+0000","last_undegraded":"2024-04-26T18:26:07.429895+0000","last_fullsized":"2024-04-26T18:26:07.429895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:18.249363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482630+0000","last_change":"2024-04-26T18:24:20.951684+0000","last_active":"2024-04-26T18:26:07.482630+0000","last_peered":"2024-04-26T18:26:07.482630+0000","last_clean":"2024-04-26T18:26:07.482630+0000","last_became_active":"2024-04-26T18:24:20.951364+0000","last_became_peered":"2024-04-26T18:24:20.951364+0000","last_unstale":"2024-04-26T18:26:07.482630+0000","last_undegraded":"2024-04-26T18:26:07.482630+0000","last_fullsized":"2024-04-26T18:26:07.482630+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:28.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302507+0000","last_change":"2024-04-26T18:24:18.229770+0000","last_active":"2024-04-26T18:26:07.302507+0000","last_peered":"2024-04-26T18:26:07.302507+0000","last_clean":"2024-04-26T18:26:07.302507+0000","last_became_active":"2024-04-26T18:24:18.229446+0000","last_became_peered":"2024-04-26T18:24:18.229446+0000","last_unstale":"2024-04-26T18:26:07.302507+0000","last_undegraded":"2024-04-26T18:26:07.302507+0000","last_fullsized":"2024-04-26T18:26:07.302507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:14.794797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482273+0000","last_change":"2024-04-26T18:24:20.960513+0000","last_active":"2024-04-26T18:26:07.482273+0000","last_peered":"2024-04-26T18:26:07.482273+0000","last_clean":"2024-04-26T18:26:07.482273+0000","last_became_active":"2024-04-26T18:24:20.959549+0000","last_became_peered":"2024-04-26T18:24:20.959549+0000","last_unstale":"2024-04-26T18:26:07.482273+0000","last_undegraded":"2024-04-26T18:26:07.482273+0000","last_fullsized":"2024-04-26T18:26:07.482273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:18:37.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.409900+0000","last_change":"2024-04-26T18:24:18.141549+0000","last_active":"2024-04-26T18:26:07.409900+0000","last_peered":"2024-04-26T18:26:07.409900+0000","last_clean":"2024-04-26T18:26:07.409900+0000","last_became_active":"2024-04-26T18:24:18.141172+0000","last_became_peered":"2024-04-26T18:24:18.141172+0000","last_unstale":"2024-04-26T18:26:07.409900+0000","last_undegraded":"2024-04-26T18:26:07.409900+0000","last_fullsized":"2024-04-26T18:26:07.409900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:15.878130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.332971+0000","last_change":"2024-04-26T18:24:20.948121+0000","last_active":"2024-04-26T18:26:07.332971+0000","last_peered":"2024-04-26T18:26:07.332971+0000","last_clean":"2024-04-26T18:26:07.332971+0000","last_became_active":"2024-04-26T18:24:20.947738+0000","last_became_peered":"2024-04-26T18:24:20.947738+0000","last_unstale":"2024-04-26T18:26:07.332971+0000","last_undegraded":"2024-04-26T18:26:07.332971+0000","last_fullsized":"2024-04-26T18:26:07.332971+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:04.237302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.481968+0000","last_change":"2024-04-26T18:24:17.926631+0000","last_active":"2024-04-26T18:26:07.481968+0000","last_peered":"2024-04-26T18:26:07.481968+0000","last_clean":"2024-04-26T18:26:07.481968+0000","last_became_active":"2024-04-26T18:24:17.926259+0000","last_became_peered":"2024-04-26T18:24:17.926259+0000","last_unstale":"2024-04-26T18:26:07.481968+0000","last_undegraded":"2024-04-26T18:26:07.481968+0000","last_fullsized":"2024-04-26T18:26:07.481968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:47.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875648+0000","last_change":"2024-04-26T18:24:20.947283+0000","last_active":"2024-04-26T18:26:07.875648+0000","last_peered":"2024-04-26T18:26:07.875648+0000","last_clean":"2024-04-26T18:26:07.875648+0000","last_became_active":"2024-04-26T18:24:20.946812+0000","last_became_peered":"2024-04-26T18:24:20.946812+0000","last_unstale":"2024-04-26T18:26:07.875648+0000","last_undegraded":"2024-04-26T18:26:07.875648+0000","last_fullsized":"2024-04-26T18:26:07.875648+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:33.730698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.306172+0000","last_change":"2024-04-26T18:24:17.931843+0000","last_active":"2024-04-26T18:26:07.306172+0000","last_peered":"2024-04-26T18:26:07.306172+0000","last_clean":"2024-04-26T18:26:07.306172+0000","last_became_active":"2024-04-26T18:24:17.931442+0000","last_became_peered":"2024-04-26T18:24:17.931442+0000","last_unstale":"2024-04-26T18:26:07.306172+0000","last_undegraded":"2024-04-26T18:26:07.306172+0000","last_fullsized":"2024-04-26T18:26:07.306172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:09.649435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482129+0000","last_change":"2024-04-26T18:24:20.955661+0000","last_active":"2024-04-26T18:26:07.482129+0000","last_peered":"2024-04-26T18:26:07.482129+0000","last_clean":"2024-04-26T18:26:07.482129+0000","last_became_active":"2024-04-26T18:24:20.955344+0000","last_became_peered":"2024-04-26T18:24:20.955344+0000","last_unstale":"2024-04-26T18:26:07.482129+0000","last_undegraded":"2024-04-26T18:26:07.482129+0000","last_fullsized":"2024-04-26T18:26:07.482129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:44.274381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875461+0000","last_change":"2024-04-26T18:25:06.315466+0000","last_active":"2024-04-26T18:26:07.875461+0000","last_peered":"2024-04-26T18:26:07.875461+0000","last_clean":"2024-04-26T18:26:07.875461+0000","last_became_active":"2024-04-26T18:25:06.301788+0000","last_became_peered":"2024-04-26T18:25:06.301788+0000","last_unstale":"2024-04-26T18:26:07.875461+0000","last_undegraded":"2024-04-26T18:26:07.875461+0000","last_fullsized":"2024-04-26T18:26:07.875461+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:12.998583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432437+0000","last_change":"2024-04-26T18:24:20.955746+0000","last_active":"2024-04-26T18:26:07.432437+0000","last_peered":"2024-04-26T18:26:07.432437+0000","last_clean":"2024-04-26T18:26:07.432437+0000","last_became_active":"2024-04-26T18:24:20.953652+0000","last_became_peered":"2024-04-26T18:24:20.953652+0000","last_unstale":"2024-04-26T18:26:07.432437+0000","last_undegraded":"2024-04-26T18:26:07.432437+0000","last_fullsized":"2024-04-26T18:26:07.432437+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:31.414185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":110,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:29.447067+0000","last_change":"2024-04-26T18:25:06.309393+0000","last_active":"2024-04-26T18:26:29.447067+0000","last_peered":"2024-04-26T18:26:29.447067+0000","last_clean":"2024-04-26T18:26:29.447067+0000","last_became_active":"2024-04-26T18:25:06.302434+0000","last_became_peered":"2024-04-26T18:25:06.302434+0000","last_unstale":"2024-04-26T18:26:29.447067+0000","last_undegraded":"2024-04-26T18:26:29.447067+0000","last_fullsized":"2024-04-26T18:26:29.447067+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:54.730295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":91,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432729+0000","last_change":"2024-04-26T18:24:20.954238+0000","last_active":"2024-04-26T18:26:07.432729+0000","last_peered":"2024-04-26T18:26:07.432729+0000","last_clean":"2024-04-26T18:26:07.432729+0000","last_became_active":"2024-04-26T18:24:20.953883+0000","last_became_peered":"2024-04-26T18:24:20.953883+0000","last_unstale":"2024-04-26T18:26:07.432729+0000","last_undegraded":"2024-04-26T18:26:07.432729+0000","last_fullsized":"2024-04-26T18:26:07.432729+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:10.580835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302140+0000","last_change":"2024-04-26T18:24:17.926901+0000","last_active":"2024-04-26T18:26:07.302140+0000","last_peered":"2024-04-26T18:26:07.302140+0000","last_clean":"2024-04-26T18:26:07.302140+0000","last_became_active":"2024-04-26T18:24:17.926291+0000","last_became_peered":"2024-04-26T18:24:17.926291+0000","last_unstale":"2024-04-26T18:26:07.302140+0000","last_undegraded":"2024-04-26T18:26:07.302140+0000","last_fullsized":"2024-04-26T18:26:07.302140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:17.695198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432356+0000","last_change":"2024-04-26T18:24:20.950578+0000","last_active":"2024-04-26T18:26:07.432356+0000","last_peered":"2024-04-26T18:26:07.432356+0000","last_clean":"2024-04-26T18:26:07.432356+0000","last_became_active":"2024-04-26T18:24:20.950174+0000","last_became_peered":"2024-04-26T18:24:20.950174+0000","last_unstale":"2024-04-26T18:26:07.432356+0000","last_undegraded":"2024-04-26T18:26:07.432356+0000","last_fullsized":"2024-04-26T18:26:07.432356+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296595+0000","last_change":"2024-04-26T18:24:17.928054+0000","last_active":"2024-04-26T18:26:07.296595+0000","last_peered":"2024-04-26T18:26:07.296595+0000","last_clean":"2024-04-26T18:26:07.296595+0000","last_became_active":"2024-04-26T18:24:17.927715+0000","last_became_peered":"2024-04-26T18:24:17.927715+0000","last_unstale":"2024-04-26T18:26:07.296595+0000","last_undegraded":"2024-04-26T18:26:07.296595+0000","last_fullsized":"2024-04-26T18:26:07.296595+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.155436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432614+0000","last_change":"2024-04-26T18:24:20.944512+0000","last_active":"2024-04-26T18:26:07.432614+0000","last_peered":"2024-04-26T18:26:07.432614+0000","last_clean":"2024-04-26T18:26:07.432614+0000","last_became_active":"2024-04-26T18:24:20.944123+0000","last_became_peered":"2024-04-26T18:24:20.944123+0000","last_unstale":"2024-04-26T18:26:07.432614+0000","last_undegraded":"2024-04-26T18:26:07.432614+0000","last_fullsized":"2024-04-26T18:26:07.432614+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.240206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875583+0000","last_change":"2024-04-26T18:24:20.959138+0000","last_active":"2024-04-26T18:26:07.875583+0000","last_peered":"2024-04-26T18:26:07.875583+0000","last_clean":"2024-04-26T18:26:07.875583+0000","last_became_active":"2024-04-26T18:24:20.958675+0000","last_became_peered":"2024-04-26T18:24:20.958675+0000","last_unstale":"2024-04-26T18:26:07.875583+0000","last_undegraded":"2024-04-26T18:26:07.875583+0000","last_fullsized":"2024-04-26T18:26:07.875583+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:34:32.103380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.306074+0000","last_change":"2024-04-26T18:24:20.961689+0000","last_active":"2024-04-26T18:26:07.306074+0000","last_peered":"2024-04-26T18:26:07.306074+0000","last_clean":"2024-04-26T18:26:07.306074+0000","last_became_active":"2024-04-26T18:24:20.961374+0000","last_became_peered":"2024-04-26T18:24:20.961374+0000","last_unstale":"2024-04-26T18:26:07.306074+0000","last_undegraded":"2024-04-26T18:26:07.306074+0000","last_fullsized":"2024-04-26T18:26:07.306074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:55.243887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.306123+0000","last_change":"2024-04-26T18:24:20.950711+0000","last_active":"2024-04-26T18:26:07.306123+0000","last_peered":"2024-04-26T18:26:07.306123+0000","last_clean":"2024-04-26T18:26:07.306123+0000","last_became_active":"2024-04-26T18:24:20.950371+0000","last_became_peered":"2024-04-26T18:24:20.950371+0000","last_unstale":"2024-04-26T18:26:07.306123+0000","last_undegraded":"2024-04-26T18:26:07.306123+0000","last_fullsized":"2024-04-26T18:26:07.306123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:35.307921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482120+0000","last_change":"2024-04-26T18:24:20.959990+0000","last_active":"2024-04-26T18:26:07.482120+0000","last_peered":"2024-04-26T18:26:07.482120+0000","last_clean":"2024-04-26T18:26:07.482120+0000","last_became_active":"2024-04-26T18:24:20.959514+0000","last_became_peered":"2024-04-26T18:24:20.959514+0000","last_unstale":"2024-04-26T18:26:07.482120+0000","last_undegraded":"2024-04-26T18:26:07.482120+0000","last_fullsized":"2024-04-26T18:26:07.482120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:00.184448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296793+0000","last_change":"2024-04-26T18:24:20.957338+0000","last_active":"2024-04-26T18:26:07.296793+0000","last_peered":"2024-04-26T18:26:07.296793+0000","last_clean":"2024-04-26T18:26:07.296793+0000","last_became_active":"2024-04-26T18:24:20.956999+0000","last_became_peered":"2024-04-26T18:24:20.956999+0000","last_unstale":"2024-04-26T18:26:07.296793+0000","last_undegraded":"2024-04-26T18:26:07.296793+0000","last_fullsized":"2024-04-26T18:26:07.296793+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.066902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875773+0000","last_change":"2024-04-26T18:24:20.953634+0000","last_active":"2024-04-26T18:26:07.875773+0000","last_peered":"2024-04-26T18:26:07.875773+0000","last_clean":"2024-04-26T18:26:07.875773+0000","last_became_active":"2024-04-26T18:24:20.953259+0000","last_became_peered":"2024-04-26T18:24:20.953259+0000","last_unstale":"2024-04-26T18:26:07.875773+0000","last_undegraded":"2024-04-26T18:26:07.875773+0000","last_fullsized":"2024-04-26T18:26:07.875773+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:00.095344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.306497+0000","last_change":"2024-04-26T18:24:20.954997+0000","last_active":"2024-04-26T18:26:07.306497+0000","last_peered":"2024-04-26T18:26:07.306497+0000","last_clean":"2024-04-26T18:26:07.306497+0000","last_became_active":"2024-04-26T18:24:20.954223+0000","last_became_peered":"2024-04-26T18:24:20.954223+0000","last_unstale":"2024-04-26T18:26:07.306497+0000","last_undegraded":"2024-04-26T18:26:07.306497+0000","last_fullsized":"2024-04-26T18:26:07.306497+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:14.092455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":26,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482526+0000","last_change":"2024-04-26T18:24:20.957056+0000","last_active":"2024-04-26T18:26:07.482526+0000","last_peered":"2024-04-26T18:26:07.482526+0000","last_clean":"2024-04-26T18:26:07.482526+0000","last_became_active":"2024-04-26T18:24:20.956737+0000","last_became_peered":"2024-04-26T18:24:20.956737+0000","last_unstale":"2024-04-26T18:26:07.482526+0000","last_undegraded":"2024-04-26T18:26:07.482526+0000","last_fullsized":"2024-04-26T18:26:07.482526+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:48.358862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.301887+0000","last_change":"2024-04-26T18:25:06.350206+0000","last_active":"2024-04-26T18:26:07.301887+0000","last_peered":"2024-04-26T18:26:07.301887+0000","last_clean":"2024-04-26T18:26:07.301887+0000","last_became_active":"2024-04-26T18:25:06.349866+0000","last_became_peered":"2024-04-26T18:25:06.349866+0000","last_unstale":"2024-04-26T18:26:07.301887+0000","last_undegraded":"2024-04-26T18:26:07.301887+0000","last_fullsized":"2024-04-26T18:26:07.301887+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:41.909056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.481796+0000","last_change":"2024-04-26T18:24:20.956000+0000","last_active":"2024-04-26T18:26:07.481796+0000","last_peered":"2024-04-26T18:26:07.481796+0000","last_clean":"2024-04-26T18:26:07.481796+0000","last_became_active":"2024-04-26T18:24:20.955677+0000","last_became_peered":"2024-04-26T18:24:20.955677+0000","last_unstale":"2024-04-26T18:26:07.481796+0000","last_undegraded":"2024-04-26T18:26:07.481796+0000","last_fullsized":"2024-04-26T18:26:07.481796+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:20.895674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.409972+0000","last_change":"2024-04-26T18:24:20.959317+0000","last_active":"2024-04-26T18:26:07.409972+0000","last_peered":"2024-04-26T18:26:07.409972+0000","last_clean":"2024-04-26T18:26:07.409972+0000","last_became_active":"2024-04-26T18:24:20.958963+0000","last_became_peered":"2024-04-26T18:24:20.958963+0000","last_unstale":"2024-04-26T18:26:07.409972+0000","last_undegraded":"2024-04-26T18:26:07.409972+0000","last_fullsized":"2024-04-26T18:26:07.409972+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:08.231089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296811+0000","last_change":"2024-04-26T18:24:20.954445+0000","last_active":"2024-04-26T18:26:07.296811+0000","last_peered":"2024-04-26T18:26:07.296811+0000","last_clean":"2024-04-26T18:26:07.296811+0000","last_became_active":"2024-04-26T18:24:20.953852+0000","last_became_peered":"2024-04-26T18:24:20.953852+0000","last_unstale":"2024-04-26T18:26:07.296811+0000","last_undegraded":"2024-04-26T18:26:07.296811+0000","last_fullsized":"2024-04-26T18:26:07.296811+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:05.636897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875522+0000","last_change":"2024-04-26T18:24:20.958023+0000","last_active":"2024-04-26T18:26:07.875522+0000","last_peered":"2024-04-26T18:26:07.875522+0000","last_clean":"2024-04-26T18:26:07.875522+0000","last_became_active":"2024-04-26T18:24:20.957642+0000","last_became_peered":"2024-04-26T18:24:20.957642+0000","last_unstale":"2024-04-26T18:26:07.875522+0000","last_undegraded":"2024-04-26T18:26:07.875522+0000","last_fullsized":"2024-04-26T18:26:07.875522+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:46.332710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244217,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72459,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1121,"num_read_kb":647,"num_write":254,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1583,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":217377,"data_compressed":5529,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351018,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":501214,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612654,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1175552,"data_stored":1156662,"data_compressed":49004,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776180,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":500590,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037818,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":500946,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299454,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1093627,"data_compressed":47161,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462980,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":500307,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1097728,"data_stored":567895,"data_compressed":13413,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":502178,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":565775,"data_compressed":13413,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149778,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":500132,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888142,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":500168,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724615,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":1093305,"data_compressed":47161,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68427,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":66307,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":66709,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:26:35.667 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:26:35.909 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:36.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:36 smithi084 ceph-mon[29141]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:26:36.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:36 smithi084 ceph-mon[29141]: pgmap v368: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:36.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:36 smithi103 ceph-mon[32727]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:26:36.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:36 smithi103 ceph-mon[32727]: pgmap v368: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:36.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:36 smithi174 ceph-mon[32705]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:26:36.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:36 smithi174 ceph-mon[32705]: pgmap v368: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:37.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.548+0000 7feb062d6700 1 -- 172.21.15.84:0/1305304510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0006bbc0 msgr2=0x7feb0006c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:37.548 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.548+0000 7feb062d6700 1 --2- 172.21.15.84:0/1305304510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0006bbc0 0x7feb0006c040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7feaf00099c0 tx=0x7feaf0009cd0 comp rx=0 tx=0).stop 2024-04-26T18:26:37.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.548+0000 7feb062d6700 1 -- 172.21.15.84:0/1305304510 shutdown_connections 2024-04-26T18:26:37.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.548+0000 7feb062d6700 1 --2- 172.21.15.84:0/1305304510 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb0006c580 0x7feb001102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:37.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.548+0000 7feb062d6700 1 --2- 172.21.15.84:0/1305304510 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb00109490 0x7feb00109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:37.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.548+0000 7feb062d6700 1 --2- 172.21.15.84:0/1305304510 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0006bbc0 0x7feb0006c040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:37.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.548+0000 7feb062d6700 1 -- 172.21.15.84:0/1305304510 >> 172.21.15.84:0/1305304510 conn(0x7feb00078390 msgr2=0x7feb000787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:37.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.548+0000 7feb062d6700 1 -- 172.21.15.84:0/1305304510 shutdown_connections 2024-04-26T18:26:37.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.548+0000 7feb062d6700 1 -- 172.21.15.84:0/1305304510 wait complete. 2024-04-26T18:26:37.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.548+0000 7feb062d6700 1 Processor -- start 2024-04-26T18:26:37.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.548+0000 7feb062d6700 1 -- start start 2024-04-26T18:26:37.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feb062d6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0006bbc0 0x7feb0019f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:37.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feb062d6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb0006c580 0x7feb0019fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:37.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feb062d6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb00109490 0x7feb001999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:37.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feb062d6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7feb001136c0 con 0x7feb0006bbc0 2024-04-26T18:26:37.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feb062d6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feb00113540 con 0x7feb0006c580 2024-04-26T18:26:37.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feb062d6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feb00113840 con 0x7feb00109490 2024-04-26T18:26:37.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feafffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0006bbc0 0x7feb0019f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feafffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0006bbc0 0x7feb0019f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50176/0 (socket says 172.21.15.84:50176) 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feafffff700 1 -- 172.21.15.84:0/1522159483 learned_addr learned my addr 172.21.15.84:0/1522159483 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feaff7fe700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb0006c580 0x7feb0019fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feb04873700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb00109490 0x7feb001999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feafffff700 1 -- 172.21.15.84:0/1522159483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb00109490 msgr2=0x7feb001999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feafffff700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb00109490 0x7feb001999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feafffff700 1 -- 172.21.15.84:0/1522159483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb0006c580 msgr2=0x7feb0019fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feafffff700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb0006c580 0x7feb0019fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feafffff700 1 -- 172.21.15.84:0/1522159483 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0019a140 con 0x7feb0006bbc0 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feb04873700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb00109490 0x7feb001999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.549+0000 7feaff7fe700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb0006c580 0x7feb0019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.550+0000 7feafffff700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0006bbc0 0x7feb0019f820 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feaf400b530 tx=0x7feaf400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.550+0000 7feafd7fa700 1 -- 172.21.15.84:0/1522159483 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4015020 con 0x7feb0006bbc0 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.550+0000 7feb062d6700 1 -- 172.21.15.84:0/1522159483 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0019a430 con 0x7feb0006bbc0 2024-04-26T18:26:37.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.550+0000 7feafd7fa700 1 -- 172.21.15.84:0/1522159483 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf4011ce0 con 0x7feb0006bbc0 2024-04-26T18:26:37.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.550+0000 7feafd7fa700 1 -- 172.21.15.84:0/1522159483 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4010d20 con 0x7feb0006bbc0 2024-04-26T18:26:37.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.550+0000 7feb062d6700 1 -- 172.21.15.84:0/1522159483 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb001a66f0 con 0x7feb0006bbc0 2024-04-26T18:26:37.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.551+0000 7feb062d6700 1 -- 172.21.15.84:0/1522159483 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb00104b80 con 0x7feb0006bbc0 2024-04-26T18:26:37.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.552+0000 7feafd7fa700 1 -- 172.21.15.84:0/1522159483 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7feaf4018020 con 0x7feb0006bbc0 2024-04-26T18:26:37.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.552+0000 7feafd7fa700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7feae80a3c40 0x7feae80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:37.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.552+0000 7feaff7fe700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7feae80a3c40 0x7feae80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:37.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.552+0000 7feafd7fa700 1 -- 172.21.15.84:0/1522159483 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7feaf40d0230 con 0x7feb0006bbc0 2024-04-26T18:26:37.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.553+0000 7feaff7fe700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7feae80a3c40 0x7feae80a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb00078ad0 tx=0x7feaf0005fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:37.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.557+0000 7feafd7fa700 1 -- 172.21.15.84:0/1522159483 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feaf401f020 con 0x7feb0006bbc0 2024-04-26T18:26:37.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.754+0000 7feb062d6700 1 -- 172.21.15.84:0/1522159483 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feb00063860 con 0x7feae80a3c40 2024-04-26T18:26:37.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.761+0000 7feafd7fa700 1 -- 172.21.15.84:0/1522159483 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+259197 (secure 0 0 0) 0x7feb00063860 con 0x7feae80a3c40 2024-04-26T18:26:37.764 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:26:37.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.764+0000 7feb062d6700 1 -- 172.21.15.84:0/1522159483 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7feae80a3c40 msgr2=0x7feae80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:37.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.764+0000 7feb062d6700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7feae80a3c40 0x7feae80a6100 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7feb00078ad0 tx=0x7feaf0005fb0 comp rx=0 tx=0).stop 2024-04-26T18:26:37.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.764+0000 7feb062d6700 1 -- 172.21.15.84:0/1522159483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0006bbc0 msgr2=0x7feb0019f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.764+0000 7feb062d6700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0006bbc0 0x7feb0019f820 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7feaf400b530 tx=0x7feaf400b8f0 comp rx=0 tx=0).stop 2024-04-26T18:26:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.764+0000 7feb062d6700 1 -- 172.21.15.84:0/1522159483 shutdown_connections 2024-04-26T18:26:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.764+0000 7feb062d6700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7feae80a3c40 0x7feae80a6100 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.764+0000 7feb062d6700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feb0006bbc0 0x7feb0019f820 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.764+0000 7feb062d6700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feb00109490 0x7feb001999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.764+0000 7feb062d6700 1 --2- 172.21.15.84:0/1522159483 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feb0006c580 0x7feb0019fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.764+0000 7feb062d6700 1 -- 172.21.15.84:0/1522159483 >> 172.21.15.84:0/1522159483 conn(0x7feb00078390 msgr2=0x7feb001011f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.765+0000 7feb062d6700 1 -- 172.21.15.84:0/1522159483 shutdown_connections 2024-04-26T18:26:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:37.765+0000 7feb062d6700 1 -- 172.21.15.84:0/1522159483 wait complete. 2024-04-26T18:26:37.767 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-04-26T18:26:38.456 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:38 smithi084 ceph-mon[29141]: pgmap v369: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:38.457 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-04-26T18:26:37.214784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663061,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1307,"num_read_kb":791,"num_write":437,"num_write_kb":3086,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182319,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":858,"ondisk_log_size":858,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370084,"kb_used_data":12528,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571740,"statfs":{"total":1151940427776,"available":1151561461760,"internally_reserved":0,"allocated":12828672,"data_stored":7982799,"data_compressed":251142,"data_compressed_allocated":344064,"data_compressed_original":2187264,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":120,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002855"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875849+0000","last_change":"2024-04-26T18:24:20.951581+0000","last_active":"2024-04-26T18:26:07.875849+0000","last_peered":"2024-04-26T18:26:07.875849+0000","last_clean":"2024-04-26T18:26:07.875849+0000","last_became_active":"2024-04-26T18:24:20.951260+0000","last_became_peered":"2024-04-26T18:24:20.951260+0000","last_unstale":"2024-04-26T18:26:07.875849+0000","last_undegraded":"2024-04-26T18:26:07.875849+0000","last_fullsized":"2024-04-26T18:26:07.875849+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:54:28.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482171+0000","last_change":"2024-04-26T18:24:20.952263+0000","last_active":"2024-04-26T18:26:07.482171+0000","last_peered":"2024-04-26T18:26:07.482171+0000","last_clean":"2024-04-26T18:26:07.482171+0000","last_became_active":"2024-04-26T18:24:20.951930+0000","last_became_peered":"2024-04-26T18:24:20.951930+0000","last_unstale":"2024-04-26T18:26:07.482171+0000","last_undegraded":"2024-04-26T18:26:07.482171+0000","last_fullsized":"2024-04-26T18:26:07.482171+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:54.816572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482303+0000","last_change":"2024-04-26T18:24:20.952280+0000","last_active":"2024-04-26T18:26:07.482303+0000","last_peered":"2024-04-26T18:26:07.482303+0000","last_clean":"2024-04-26T18:26:07.482303+0000","last_became_active":"2024-04-26T18:24:20.951623+0000","last_became_peered":"2024-04-26T18:24:20.951623+0000","last_unstale":"2024-04-26T18:26:07.482303+0000","last_undegraded":"2024-04-26T18:26:07.482303+0000","last_fullsized":"2024-04-26T18:26:07.482303+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:02.502261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302417+0000","last_change":"2024-04-26T18:24:20.962274+0000","last_active":"2024-04-26T18:26:07.302417+0000","last_peered":"2024-04-26T18:26:07.302417+0000","last_clean":"2024-04-26T18:26:07.302417+0000","last_became_active":"2024-04-26T18:24:20.961920+0000","last_became_peered":"2024-04-26T18:24:20.961920+0000","last_unstale":"2024-04-26T18:26:07.302417+0000","last_undegraded":"2024-04-26T18:26:07.302417+0000","last_fullsized":"2024-04-26T18:26:07.302417+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:49:46.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":28,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.333143+0000","last_change":"2024-04-26T18:24:20.949096+0000","last_active":"2024-04-26T18:26:07.333143+0000","last_peered":"2024-04-26T18:26:07.333143+0000","last_clean":"2024-04-26T18:26:07.333143+0000","last_became_active":"2024-04-26T18:24:20.948737+0000","last_became_peered":"2024-04-26T18:24:20.948737+0000","last_unstale":"2024-04-26T18:26:07.333143+0000","last_undegraded":"2024-04-26T18:26:07.333143+0000","last_fullsized":"2024-04-26T18:26:07.333143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:12:40.371589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.303947+0000","last_change":"2024-04-26T18:24:20.962670+0000","last_active":"2024-04-26T18:26:07.303947+0000","last_peered":"2024-04-26T18:26:07.303947+0000","last_clean":"2024-04-26T18:26:07.303947+0000","last_became_active":"2024-04-26T18:24:20.962309+0000","last_became_peered":"2024-04-26T18:24:20.962309+0000","last_unstale":"2024-04-26T18:26:07.303947+0000","last_undegraded":"2024-04-26T18:26:07.303947+0000","last_fullsized":"2024-04-26T18:26:07.303947+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:40.315864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432274+0000","last_change":"2024-04-26T18:25:06.350579+0000","last_active":"2024-04-26T18:26:07.432274+0000","last_peered":"2024-04-26T18:26:07.432274+0000","last_clean":"2024-04-26T18:26:07.432274+0000","last_became_active":"2024-04-26T18:25:06.350104+0000","last_became_peered":"2024-04-26T18:25:06.350104+0000","last_unstale":"2024-04-26T18:26:07.432274+0000","last_undegraded":"2024-04-26T18:26:07.432274+0000","last_fullsized":"2024-04-26T18:26:07.432274+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:00.950883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.481888+0000","last_change":"2024-04-26T18:24:20.961096+0000","last_active":"2024-04-26T18:26:07.481888+0000","last_peered":"2024-04-26T18:26:07.481888+0000","last_clean":"2024-04-26T18:26:07.481888+0000","last_became_active":"2024-04-26T18:24:20.960796+0000","last_became_peered":"2024-04-26T18:24:20.960796+0000","last_unstale":"2024-04-26T18:26:07.481888+0000","last_undegraded":"2024-04-26T18:26:07.481888+0000","last_fullsized":"2024-04-26T18:26:07.481888+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:04.538836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.410236+0000","last_change":"2024-04-26T18:24:20.959871+0000","last_active":"2024-04-26T18:26:07.410236+0000","last_peered":"2024-04-26T18:26:07.410236+0000","last_clean":"2024-04-26T18:26:07.410236+0000","last_became_active":"2024-04-26T18:24:20.959404+0000","last_became_peered":"2024-04-26T18:24:20.959404+0000","last_unstale":"2024-04-26T18:26:07.410236+0000","last_undegraded":"2024-04-26T18:26:07.410236+0000","last_fullsized":"2024-04-26T18:26:07.410236+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:43.053618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.431543+0000","last_change":"2024-04-26T18:26:07.431543+0000","last_active":"2024-04-26T18:26:07.431543+0000","last_peered":"2024-04-26T18:26:07.431543+0000","last_clean":"2024-04-26T18:26:07.431543+0000","last_became_active":"2024-04-26T18:26:07.430976+0000","last_became_peered":"2024-04-26T18:26:07.430976+0000","last_unstale":"2024-04-26T18:26:07.431543+0000","last_undegraded":"2024-04-26T18:26:07.431543+0000","last_fullsized":"2024-04-26T18:26:07.431543+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:31.430010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432641+0000","last_change":"2024-04-26T18:24:20.952109+0000","last_active":"2024-04-26T18:26:07.432641+0000","last_peered":"2024-04-26T18:26:07.432641+0000","last_clean":"2024-04-26T18:26:07.432641+0000","last_became_active":"2024-04-26T18:24:20.951776+0000","last_became_peered":"2024-04-26T18:24:20.951776+0000","last_unstale":"2024-04-26T18:26:07.432641+0000","last_undegraded":"2024-04-26T18:26:07.432641+0000","last_fullsized":"2024-04-26T18:26:07.432641+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:53.220246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302315+0000","last_change":"2024-04-26T18:24:20.963965+0000","last_active":"2024-04-26T18:26:07.302315+0000","last_peered":"2024-04-26T18:26:07.302315+0000","last_clean":"2024-04-26T18:26:07.302315+0000","last_became_active":"2024-04-26T18:24:20.963648+0000","last_became_peered":"2024-04-26T18:24:20.963648+0000","last_unstale":"2024-04-26T18:26:07.302315+0000","last_undegraded":"2024-04-26T18:26:07.302315+0000","last_fullsized":"2024-04-26T18:26:07.302315+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:04.080813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296837+0000","last_change":"2024-04-26T18:24:20.954186+0000","last_active":"2024-04-26T18:26:07.296837+0000","last_peered":"2024-04-26T18:26:07.296837+0000","last_clean":"2024-04-26T18:26:07.296837+0000","last_became_active":"2024-04-26T18:24:20.953431+0000","last_became_peered":"2024-04-26T18:24:20.953431+0000","last_unstale":"2024-04-26T18:26:07.296837+0000","last_undegraded":"2024-04-26T18:26:07.296837+0000","last_fullsized":"2024-04-26T18:26:07.296837+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:21.498476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.481889+0000","last_change":"2024-04-26T18:24:20.945392+0000","last_active":"2024-04-26T18:26:07.481889+0000","last_peered":"2024-04-26T18:26:07.481889+0000","last_clean":"2024-04-26T18:26:07.481889+0000","last_became_active":"2024-04-26T18:24:20.944971+0000","last_became_peered":"2024-04-26T18:24:20.944971+0000","last_unstale":"2024-04-26T18:26:07.481889+0000","last_undegraded":"2024-04-26T18:26:07.481889+0000","last_fullsized":"2024-04-26T18:26:07.481889+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:06.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482071+0000","last_change":"2024-04-26T18:25:06.299351+0000","last_active":"2024-04-26T18:26:07.482071+0000","last_peered":"2024-04-26T18:26:07.482071+0000","last_clean":"2024-04-26T18:26:07.482071+0000","last_became_active":"2024-04-26T18:25:06.298895+0000","last_became_peered":"2024-04-26T18:25:06.298895+0000","last_unstale":"2024-04-26T18:26:07.482071+0000","last_undegraded":"2024-04-26T18:26:07.482071+0000","last_fullsized":"2024-04-26T18:26:07.482071+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:37.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.409813+0000","last_change":"2024-04-26T18:24:20.954610+0000","last_active":"2024-04-26T18:26:07.409813+0000","last_peered":"2024-04-26T18:26:07.409813+0000","last_clean":"2024-04-26T18:26:07.409813+0000","last_became_active":"2024-04-26T18:24:20.954239+0000","last_became_peered":"2024-04-26T18:24:20.954239+0000","last_unstale":"2024-04-26T18:26:07.409813+0000","last_undegraded":"2024-04-26T18:26:07.409813+0000","last_fullsized":"2024-04-26T18:26:07.409813+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:41.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875627+0000","last_change":"2024-04-26T18:24:20.956932+0000","last_active":"2024-04-26T18:26:07.875627+0000","last_peered":"2024-04-26T18:26:07.875627+0000","last_clean":"2024-04-26T18:26:07.875627+0000","last_became_active":"2024-04-26T18:24:20.956464+0000","last_became_peered":"2024-04-26T18:24:20.956464+0000","last_unstale":"2024-04-26T18:26:07.875627+0000","last_undegraded":"2024-04-26T18:26:07.875627+0000","last_fullsized":"2024-04-26T18:26:07.875627+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:01.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.429951+0000","last_change":"2024-04-26T18:24:20.954870+0000","last_active":"2024-04-26T18:26:07.429951+0000","last_peered":"2024-04-26T18:26:07.429951+0000","last_clean":"2024-04-26T18:26:07.429951+0000","last_became_active":"2024-04-26T18:24:20.954542+0000","last_became_peered":"2024-04-26T18:24:20.954542+0000","last_unstale":"2024-04-26T18:26:07.429951+0000","last_undegraded":"2024-04-26T18:26:07.429951+0000","last_fullsized":"2024-04-26T18:26:07.429951+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:13:51.932625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.306460+0000","last_change":"2024-04-26T18:24:20.950994+0000","last_active":"2024-04-26T18:26:07.306460+0000","last_peered":"2024-04-26T18:26:07.306460+0000","last_clean":"2024-04-26T18:26:07.306460+0000","last_became_active":"2024-04-26T18:24:20.950343+0000","last_became_peered":"2024-04-26T18:24:20.950343+0000","last_unstale":"2024-04-26T18:26:07.306460+0000","last_undegraded":"2024-04-26T18:26:07.306460+0000","last_fullsized":"2024-04-26T18:26:07.306460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:25.274049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":159,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:31.481111+0000","last_change":"2024-04-26T18:24:17.929731+0000","last_active":"2024-04-26T18:26:31.481111+0000","last_peered":"2024-04-26T18:26:31.481111+0000","last_clean":"2024-04-26T18:26:31.481111+0000","last_became_active":"2024-04-26T18:24:17.929379+0000","last_became_peered":"2024-04-26T18:24:17.929379+0000","last_unstale":"2024-04-26T18:26:31.481111+0000","last_undegraded":"2024-04-26T18:26:31.481111+0000","last_fullsized":"2024-04-26T18:26:31.481111+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:44.821660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":115,"num_read_kb":115,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482804+0000","last_change":"2024-04-26T18:24:20.962885+0000","last_active":"2024-04-26T18:26:07.482804+0000","last_peered":"2024-04-26T18:26:07.482804+0000","last_clean":"2024-04-26T18:26:07.482804+0000","last_became_active":"2024-04-26T18:24:20.962674+0000","last_became_peered":"2024-04-26T18:24:20.962674+0000","last_unstale":"2024-04-26T18:26:07.482804+0000","last_undegraded":"2024-04-26T18:26:07.482804+0000","last_fullsized":"2024-04-26T18:26:07.482804+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:32.444889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482977+0000","last_change":"2024-04-26T18:24:18.141500+0000","last_active":"2024-04-26T18:26:07.482977+0000","last_peered":"2024-04-26T18:26:07.482977+0000","last_clean":"2024-04-26T18:26:07.482977+0000","last_became_active":"2024-04-26T18:24:18.140870+0000","last_became_peered":"2024-04-26T18:24:18.140870+0000","last_unstale":"2024-04-26T18:26:07.482977+0000","last_undegraded":"2024-04-26T18:26:07.482977+0000","last_fullsized":"2024-04-26T18:26:07.482977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:13.821872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482384+0000","last_change":"2024-04-26T18:24:20.949183+0000","last_active":"2024-04-26T18:26:07.482384+0000","last_peered":"2024-04-26T18:26:07.482384+0000","last_clean":"2024-04-26T18:26:07.482384+0000","last_became_active":"2024-04-26T18:24:20.948677+0000","last_became_peered":"2024-04-26T18:24:20.948677+0000","last_unstale":"2024-04-26T18:26:07.482384+0000","last_undegraded":"2024-04-26T18:26:07.482384+0000","last_fullsized":"2024-04-26T18:26:07.482384+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:55.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":65,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.333289+0000","last_change":"2024-04-26T18:25:06.312239+0000","last_active":"2024-04-26T18:26:07.333289+0000","last_peered":"2024-04-26T18:26:07.333289+0000","last_clean":"2024-04-26T18:26:07.333289+0000","last_became_active":"2024-04-26T18:25:06.301304+0000","last_became_peered":"2024-04-26T18:25:06.301304+0000","last_unstale":"2024-04-26T18:26:07.333289+0000","last_undegraded":"2024-04-26T18:26:07.333289+0000","last_fullsized":"2024-04-26T18:26:07.333289+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:36.159348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.409938+0000","last_change":"2024-04-26T18:24:20.955555+0000","last_active":"2024-04-26T18:26:07.409938+0000","last_peered":"2024-04-26T18:26:07.409938+0000","last_clean":"2024-04-26T18:26:07.409938+0000","last_became_active":"2024-04-26T18:24:20.955176+0000","last_became_peered":"2024-04-26T18:24:20.955176+0000","last_unstale":"2024-04-26T18:26:07.409938+0000","last_undegraded":"2024-04-26T18:26:07.409938+0000","last_fullsized":"2024-04-26T18:26:07.409938+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:02:30.368971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.333197+0000","last_change":"2024-04-26T18:24:17.930919+0000","last_active":"2024-04-26T18:26:07.333197+0000","last_peered":"2024-04-26T18:26:07.333197+0000","last_clean":"2024-04-26T18:26:07.333197+0000","last_became_active":"2024-04-26T18:24:17.930596+0000","last_became_peered":"2024-04-26T18:24:17.930596+0000","last_unstale":"2024-04-26T18:26:07.333197+0000","last_undegraded":"2024-04-26T18:26:07.333197+0000","last_fullsized":"2024-04-26T18:26:07.333197+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:57.476027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302186+0000","last_change":"2024-04-26T18:24:20.955796+0000","last_active":"2024-04-26T18:26:07.302186+0000","last_peered":"2024-04-26T18:26:07.302186+0000","last_clean":"2024-04-26T18:26:07.302186+0000","last_became_active":"2024-04-26T18:24:20.955432+0000","last_became_peered":"2024-04-26T18:24:20.955432+0000","last_unstale":"2024-04-26T18:26:07.302186+0000","last_undegraded":"2024-04-26T18:26:07.302186+0000","last_fullsized":"2024-04-26T18:26:07.302186+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:25.161797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":157,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:33.473649+0000","last_change":"2024-04-26T18:24:17.929505+0000","last_active":"2024-04-26T18:26:33.473649+0000","last_peered":"2024-04-26T18:26:33.473649+0000","last_clean":"2024-04-26T18:26:33.473649+0000","last_became_active":"2024-04-26T18:24:17.929205+0000","last_became_peered":"2024-04-26T18:24:17.929205+0000","last_unstale":"2024-04-26T18:26:33.473649+0000","last_undegraded":"2024-04-26T18:26:33.473649+0000","last_fullsized":"2024-04-26T18:26:33.473649+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:42.460063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":120,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.332934+0000","last_change":"2024-04-26T18:24:20.952134+0000","last_active":"2024-04-26T18:26:07.332934+0000","last_peered":"2024-04-26T18:26:07.332934+0000","last_clean":"2024-04-26T18:26:07.332934+0000","last_became_active":"2024-04-26T18:24:20.951758+0000","last_became_peered":"2024-04-26T18:24:20.951758+0000","last_unstale":"2024-04-26T18:26:07.332934+0000","last_undegraded":"2024-04-26T18:26:07.332934+0000","last_fullsized":"2024-04-26T18:26:07.332934+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:25.389221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.303910+0000","last_change":"2024-04-26T18:24:17.928704+0000","last_active":"2024-04-26T18:26:07.303910+0000","last_peered":"2024-04-26T18:26:07.303910+0000","last_clean":"2024-04-26T18:26:07.303910+0000","last_became_active":"2024-04-26T18:24:17.928343+0000","last_became_peered":"2024-04-26T18:24:17.928343+0000","last_unstale":"2024-04-26T18:26:07.303910+0000","last_undegraded":"2024-04-26T18:26:07.303910+0000","last_fullsized":"2024-04-26T18:26:07.303910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:49.400481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302088+0000","last_change":"2024-04-26T18:24:20.956808+0000","last_active":"2024-04-26T18:26:07.302088+0000","last_peered":"2024-04-26T18:26:07.302088+0000","last_clean":"2024-04-26T18:26:07.302088+0000","last_became_active":"2024-04-26T18:24:20.956466+0000","last_became_peered":"2024-04-26T18:24:20.956466+0000","last_unstale":"2024-04-26T18:26:07.302088+0000","last_undegraded":"2024-04-26T18:26:07.302088+0000","last_fullsized":"2024-04-26T18:26:07.302088+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:31.150670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296535+0000","last_change":"2024-04-26T18:24:17.929134+0000","last_active":"2024-04-26T18:26:07.296535+0000","last_peered":"2024-04-26T18:26:07.296535+0000","last_clean":"2024-04-26T18:26:07.296535+0000","last_became_active":"2024-04-26T18:24:17.928798+0000","last_became_peered":"2024-04-26T18:24:17.928798+0000","last_unstale":"2024-04-26T18:26:07.296535+0000","last_undegraded":"2024-04-26T18:26:07.296535+0000","last_fullsized":"2024-04-26T18:26:07.296535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:10.450825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.481760+0000","last_change":"2024-04-26T18:24:20.951594+0000","last_active":"2024-04-26T18:26:07.481760+0000","last_peered":"2024-04-26T18:26:07.481760+0000","last_clean":"2024-04-26T18:26:07.481760+0000","last_became_active":"2024-04-26T18:24:20.951148+0000","last_became_peered":"2024-04-26T18:24:20.951148+0000","last_unstale":"2024-04-26T18:26:07.481760+0000","last_undegraded":"2024-04-26T18:26:07.481760+0000","last_fullsized":"2024-04-26T18:26:07.481760+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:18.703292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.301797+0000","last_change":"2024-04-26T18:24:17.931101+0000","last_active":"2024-04-26T18:26:07.301797+0000","last_peered":"2024-04-26T18:26:07.301797+0000","last_clean":"2024-04-26T18:26:07.301797+0000","last_became_active":"2024-04-26T18:24:17.930819+0000","last_became_peered":"2024-04-26T18:24:17.930819+0000","last_unstale":"2024-04-26T18:26:07.301797+0000","last_undegraded":"2024-04-26T18:26:07.301797+0000","last_fullsized":"2024-04-26T18:26:07.301797+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.429381+0000","last_change":"2024-04-26T18:24:20.948866+0000","last_active":"2024-04-26T18:26:07.429381+0000","last_peered":"2024-04-26T18:26:07.429381+0000","last_clean":"2024-04-26T18:26:07.429381+0000","last_became_active":"2024-04-26T18:24:20.948562+0000","last_became_peered":"2024-04-26T18:24:20.948562+0000","last_unstale":"2024-04-26T18:26:07.429381+0000","last_undegraded":"2024-04-26T18:26:07.429381+0000","last_fullsized":"2024-04-26T18:26:07.429381+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:33:19.616660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":265,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:33.453552+0000","last_change":"2024-04-26T18:24:17.926384+0000","last_active":"2024-04-26T18:26:33.453552+0000","last_peered":"2024-04-26T18:26:33.453552+0000","last_clean":"2024-04-26T18:26:33.453552+0000","last_became_active":"2024-04-26T18:24:17.926037+0000","last_became_peered":"2024-04-26T18:24:17.926037+0000","last_unstale":"2024-04-26T18:26:33.453552+0000","last_undegraded":"2024-04-26T18:26:33.453552+0000","last_fullsized":"2024-04-26T18:26:33.453552+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:05.283755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":229,"num_read_kb":229,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.410206+0000","last_change":"2024-04-26T18:24:20.954257+0000","last_active":"2024-04-26T18:26:07.410206+0000","last_peered":"2024-04-26T18:26:07.410206+0000","last_clean":"2024-04-26T18:26:07.410206+0000","last_became_active":"2024-04-26T18:24:20.953913+0000","last_became_peered":"2024-04-26T18:24:20.953913+0000","last_unstale":"2024-04-26T18:26:07.410206+0000","last_undegraded":"2024-04-26T18:26:07.410206+0000","last_fullsized":"2024-04-26T18:26:07.410206+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:06.814156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432469+0000","last_change":"2024-04-26T18:24:17.932843+0000","last_active":"2024-04-26T18:26:07.432469+0000","last_peered":"2024-04-26T18:26:07.432469+0000","last_clean":"2024-04-26T18:26:07.432469+0000","last_became_active":"2024-04-26T18:24:17.932502+0000","last_became_peered":"2024-04-26T18:24:17.932502+0000","last_unstale":"2024-04-26T18:26:07.432469+0000","last_undegraded":"2024-04-26T18:26:07.432469+0000","last_fullsized":"2024-04-26T18:26:07.432469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:49.727094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482355+0000","last_change":"2024-04-26T18:24:20.959881+0000","last_active":"2024-04-26T18:26:07.482355+0000","last_peered":"2024-04-26T18:26:07.482355+0000","last_clean":"2024-04-26T18:26:07.482355+0000","last_became_active":"2024-04-26T18:24:20.959541+0000","last_became_peered":"2024-04-26T18:24:20.959541+0000","last_unstale":"2024-04-26T18:26:07.482355+0000","last_undegraded":"2024-04-26T18:26:07.482355+0000","last_fullsized":"2024-04-26T18:26:07.482355+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:23.393521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302266+0000","last_change":"2024-04-26T18:24:18.142146+0000","last_active":"2024-04-26T18:26:07.302266+0000","last_peered":"2024-04-26T18:26:07.302266+0000","last_clean":"2024-04-26T18:26:07.302266+0000","last_became_active":"2024-04-26T18:24:18.141829+0000","last_became_peered":"2024-04-26T18:24:18.141829+0000","last_unstale":"2024-04-26T18:26:07.302266+0000","last_undegraded":"2024-04-26T18:26:07.302266+0000","last_fullsized":"2024-04-26T18:26:07.302266+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:49.341638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.410114+0000","last_change":"2024-04-26T18:24:20.957993+0000","last_active":"2024-04-26T18:26:07.410114+0000","last_peered":"2024-04-26T18:26:07.410114+0000","last_clean":"2024-04-26T18:26:07.410114+0000","last_became_active":"2024-04-26T18:24:20.957443+0000","last_became_peered":"2024-04-26T18:24:20.957443+0000","last_unstale":"2024-04-26T18:26:07.410114+0000","last_undegraded":"2024-04-26T18:26:07.410114+0000","last_fullsized":"2024-04-26T18:26:07.410114+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:06:32.960546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"91'36","reported_seq":306,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:33.473212+0000","last_change":"2024-04-26T18:24:17.931283+0000","last_active":"2024-04-26T18:26:33.473212+0000","last_peered":"2024-04-26T18:26:33.473212+0000","last_clean":"2024-04-26T18:26:33.473212+0000","last_became_active":"2024-04-26T18:24:17.930943+0000","last_became_peered":"2024-04-26T18:24:17.930943+0000","last_unstale":"2024-04-26T18:26:33.473212+0000","last_undegraded":"2024-04-26T18:26:33.473212+0000","last_fullsized":"2024-04-26T18:26:33.473212+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:50.511677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64973,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":240,"num_read_kb":183,"num_write":38,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.333092+0000","last_change":"2024-04-26T18:24:20.947509+0000","last_active":"2024-04-26T18:26:07.333092+0000","last_peered":"2024-04-26T18:26:07.333092+0000","last_clean":"2024-04-26T18:26:07.333092+0000","last_became_active":"2024-04-26T18:24:20.947181+0000","last_became_peered":"2024-04-26T18:24:20.947181+0000","last_unstale":"2024-04-26T18:26:07.333092+0000","last_undegraded":"2024-04-26T18:26:07.333092+0000","last_fullsized":"2024-04-26T18:26:07.333092+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.262312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":41,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.306181+0000","last_change":"2024-04-26T18:24:17.934042+0000","last_active":"2024-04-26T18:26:07.306181+0000","last_peered":"2024-04-26T18:26:07.306181+0000","last_clean":"2024-04-26T18:26:07.306181+0000","last_became_active":"2024-04-26T18:24:17.933537+0000","last_became_peered":"2024-04-26T18:24:17.933537+0000","last_unstale":"2024-04-26T18:26:07.306181+0000","last_undegraded":"2024-04-26T18:26:07.306181+0000","last_fullsized":"2024-04-26T18:26:07.306181+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:35:06.904160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296455+0000","last_change":"2024-04-26T18:24:20.957672+0000","last_active":"2024-04-26T18:26:07.296455+0000","last_peered":"2024-04-26T18:26:07.296455+0000","last_clean":"2024-04-26T18:26:07.296455+0000","last_became_active":"2024-04-26T18:24:20.957214+0000","last_became_peered":"2024-04-26T18:24:20.957214+0000","last_unstale":"2024-04-26T18:26:07.296455+0000","last_undegraded":"2024-04-26T18:26:07.296455+0000","last_fullsized":"2024-04-26T18:26:07.296455+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:42:35.341858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.301840+0000","last_change":"2024-04-26T18:24:17.930324+0000","last_active":"2024-04-26T18:26:07.301840+0000","last_peered":"2024-04-26T18:26:07.301840+0000","last_clean":"2024-04-26T18:26:07.301840+0000","last_became_active":"2024-04-26T18:24:17.930022+0000","last_became_peered":"2024-04-26T18:24:17.930022+0000","last_unstale":"2024-04-26T18:26:07.301840+0000","last_undegraded":"2024-04-26T18:26:07.301840+0000","last_fullsized":"2024-04-26T18:26:07.301840+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:36.864816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":45,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432366+0000","last_change":"2024-04-26T18:25:06.350388+0000","last_active":"2024-04-26T18:26:07.432366+0000","last_peered":"2024-04-26T18:26:07.432366+0000","last_clean":"2024-04-26T18:26:07.432366+0000","last_became_active":"2024-04-26T18:25:06.349836+0000","last_became_peered":"2024-04-26T18:25:06.349836+0000","last_unstale":"2024-04-26T18:26:07.432366+0000","last_undegraded":"2024-04-26T18:26:07.432366+0000","last_fullsized":"2024-04-26T18:26:07.432366+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:57.762115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"89'5","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:33.510655+0000","last_change":"2024-04-26T18:24:17.934676+0000","last_active":"2024-04-26T18:26:33.510655+0000","last_peered":"2024-04-26T18:26:33.510655+0000","last_clean":"2024-04-26T18:26:33.510655+0000","last_became_active":"2024-04-26T18:24:17.934307+0000","last_became_peered":"2024-04-26T18:24:17.934307+0000","last_unstale":"2024-04-26T18:26:33.510655+0000","last_undegraded":"2024-04-26T18:26:33.510655+0000","last_fullsized":"2024-04-26T18:26:33.510655+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:27.312248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":119,"num_read_kb":3,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432020+0000","last_change":"2024-04-26T18:24:20.947371+0000","last_active":"2024-04-26T18:26:07.432020+0000","last_peered":"2024-04-26T18:26:07.432020+0000","last_clean":"2024-04-26T18:26:07.432020+0000","last_became_active":"2024-04-26T18:24:20.947048+0000","last_became_peered":"2024-04-26T18:24:20.947048+0000","last_unstale":"2024-04-26T18:26:07.432020+0000","last_undegraded":"2024-04-26T18:26:07.432020+0000","last_fullsized":"2024-04-26T18:26:07.432020+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:55.038074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":48,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.332861+0000","last_change":"2024-04-26T18:24:18.142680+0000","last_active":"2024-04-26T18:26:07.332861+0000","last_peered":"2024-04-26T18:26:07.332861+0000","last_clean":"2024-04-26T18:26:07.332861+0000","last_became_active":"2024-04-26T18:24:18.142392+0000","last_became_peered":"2024-04-26T18:24:18.142392+0000","last_unstale":"2024-04-26T18:26:07.332861+0000","last_undegraded":"2024-04-26T18:26:07.332861+0000","last_fullsized":"2024-04-26T18:26:07.332861+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:52.654813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.301767+0000","last_change":"2024-04-26T18:24:20.958131+0000","last_active":"2024-04-26T18:26:07.301767+0000","last_peered":"2024-04-26T18:26:07.301767+0000","last_clean":"2024-04-26T18:26:07.301767+0000","last_became_active":"2024-04-26T18:24:20.957701+0000","last_became_peered":"2024-04-26T18:24:20.957701+0000","last_unstale":"2024-04-26T18:26:07.301767+0000","last_undegraded":"2024-04-26T18:26:07.301767+0000","last_fullsized":"2024-04-26T18:26:07.301767+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:02.018158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875672+0000","last_change":"2024-04-26T18:25:06.319173+0000","last_active":"2024-04-26T18:26:07.875672+0000","last_peered":"2024-04-26T18:26:07.875672+0000","last_clean":"2024-04-26T18:26:07.875672+0000","last_became_active":"2024-04-26T18:25:06.302935+0000","last_became_peered":"2024-04-26T18:25:06.302935+0000","last_unstale":"2024-04-26T18:26:07.875672+0000","last_undegraded":"2024-04-26T18:26:07.875672+0000","last_fullsized":"2024-04-26T18:26:07.875672+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:40.198105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875765+0000","last_change":"2024-04-26T18:24:20.961230+0000","last_active":"2024-04-26T18:26:07.875765+0000","last_peered":"2024-04-26T18:26:07.875765+0000","last_clean":"2024-04-26T18:26:07.875765+0000","last_became_active":"2024-04-26T18:24:20.960807+0000","last_became_peered":"2024-04-26T18:24:20.960807+0000","last_unstale":"2024-04-26T18:26:07.875765+0000","last_undegraded":"2024-04-26T18:26:07.875765+0000","last_fullsized":"2024-04-26T18:26:07.875765+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:39.535100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.429630+0000","last_change":"2024-04-26T18:24:18.142082+0000","last_active":"2024-04-26T18:26:07.429630+0000","last_peered":"2024-04-26T18:26:07.429630+0000","last_clean":"2024-04-26T18:26:07.429630+0000","last_became_active":"2024-04-26T18:24:18.141765+0000","last_became_peered":"2024-04-26T18:24:18.141765+0000","last_unstale":"2024-04-26T18:26:07.429630+0000","last_undegraded":"2024-04-26T18:26:07.429630+0000","last_fullsized":"2024-04-26T18:26:07.429630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:02.140027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.304077+0000","last_change":"2024-04-26T18:24:20.952867+0000","last_active":"2024-04-26T18:26:07.304077+0000","last_peered":"2024-04-26T18:26:07.304077+0000","last_clean":"2024-04-26T18:26:07.304077+0000","last_became_active":"2024-04-26T18:24:20.952525+0000","last_became_peered":"2024-04-26T18:24:20.952525+0000","last_unstale":"2024-04-26T18:26:07.304077+0000","last_undegraded":"2024-04-26T18:26:07.304077+0000","last_fullsized":"2024-04-26T18:26:07.304077+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:52.804312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":154,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:34.485704+0000","last_change":"2024-04-26T18:24:17.932270+0000","last_active":"2024-04-26T18:26:34.485704+0000","last_peered":"2024-04-26T18:26:34.485704+0000","last_clean":"2024-04-26T18:26:34.485704+0000","last_became_active":"2024-04-26T18:24:17.931901+0000","last_became_peered":"2024-04-26T18:24:17.931901+0000","last_unstale":"2024-04-26T18:26:34.485704+0000","last_undegraded":"2024-04-26T18:26:34.485704+0000","last_fullsized":"2024-04-26T18:26:34.485704+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:13.524174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":118,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.304162+0000","last_change":"2024-04-26T18:24:20.960612+0000","last_active":"2024-04-26T18:26:07.304162+0000","last_peered":"2024-04-26T18:26:07.304162+0000","last_clean":"2024-04-26T18:26:07.304162+0000","last_became_active":"2024-04-26T18:24:20.960280+0000","last_became_peered":"2024-04-26T18:24:20.960280+0000","last_unstale":"2024-04-26T18:26:07.304162+0000","last_undegraded":"2024-04-26T18:26:07.304162+0000","last_fullsized":"2024-04-26T18:26:07.304162+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:53.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":23,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.429368+0000","last_change":"2024-04-26T18:25:06.299736+0000","last_active":"2024-04-26T18:26:07.429368+0000","last_peered":"2024-04-26T18:26:07.429368+0000","last_clean":"2024-04-26T18:26:07.429368+0000","last_became_active":"2024-04-26T18:25:06.299265+0000","last_became_peered":"2024-04-26T18:25:06.299265+0000","last_unstale":"2024-04-26T18:26:07.429368+0000","last_undegraded":"2024-04-26T18:26:07.429368+0000","last_fullsized":"2024-04-26T18:26:07.429368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:32.427551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":153,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:33.509711+0000","last_change":"2024-04-26T18:24:18.141181+0000","last_active":"2024-04-26T18:26:33.509711+0000","last_peered":"2024-04-26T18:26:33.509711+0000","last_clean":"2024-04-26T18:26:33.509711+0000","last_became_active":"2024-04-26T18:24:18.140854+0000","last_became_peered":"2024-04-26T18:24:18.140854+0000","last_unstale":"2024-04-26T18:26:33.509711+0000","last_undegraded":"2024-04-26T18:26:33.509711+0000","last_fullsized":"2024-04-26T18:26:33.509711+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:27.900701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":117,"num_read_kb":117,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482931+0000","last_change":"2024-04-26T18:24:20.961653+0000","last_active":"2024-04-26T18:26:07.482931+0000","last_peered":"2024-04-26T18:26:07.482931+0000","last_clean":"2024-04-26T18:26:07.482931+0000","last_became_active":"2024-04-26T18:24:20.961330+0000","last_became_peered":"2024-04-26T18:24:20.961330+0000","last_unstale":"2024-04-26T18:26:07.482931+0000","last_undegraded":"2024-04-26T18:26:07.482931+0000","last_fullsized":"2024-04-26T18:26:07.482931+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:19.033932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302130+0000","last_change":"2024-04-26T18:24:17.933613+0000","last_active":"2024-04-26T18:26:07.302130+0000","last_peered":"2024-04-26T18:26:07.302130+0000","last_clean":"2024-04-26T18:26:07.302130+0000","last_became_active":"2024-04-26T18:24:17.933219+0000","last_became_peered":"2024-04-26T18:24:17.933219+0000","last_unstale":"2024-04-26T18:26:07.302130+0000","last_undegraded":"2024-04-26T18:26:07.302130+0000","last_fullsized":"2024-04-26T18:26:07.302130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:04.215821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":363,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432119+0000","last_change":"2024-04-26T18:22:57.585634+0000","last_active":"2024-04-26T18:26:07.432119+0000","last_peered":"2024-04-26T18:26:07.432119+0000","last_clean":"2024-04-26T18:26:07.432119+0000","last_became_active":"2024-04-26T18:22:57.585198+0000","last_became_peered":"2024-04-26T18:22:57.585198+0000","last_unstale":"2024-04-26T18:26:07.432119+0000","last_undegraded":"2024-04-26T18:26:07.432119+0000","last_fullsized":"2024-04-26T18:26:07.432119+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_clean_scrub_stamp":"2024-04-26T18:19:31.594684+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:56:11.861930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296467+0000","last_change":"2024-04-26T18:24:20.950303+0000","last_active":"2024-04-26T18:26:07.296467+0000","last_peered":"2024-04-26T18:26:07.296467+0000","last_clean":"2024-04-26T18:26:07.296467+0000","last_became_active":"2024-04-26T18:24:20.950002+0000","last_became_peered":"2024-04-26T18:24:20.950002+0000","last_unstale":"2024-04-26T18:26:07.296467+0000","last_undegraded":"2024-04-26T18:26:07.296467+0000","last_fullsized":"2024-04-26T18:26:07.296467+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:00.912752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432283+0000","last_change":"2024-04-26T18:24:17.934924+0000","last_active":"2024-04-26T18:26:07.432283+0000","last_peered":"2024-04-26T18:26:07.432283+0000","last_clean":"2024-04-26T18:26:07.432283+0000","last_became_active":"2024-04-26T18:24:17.934598+0000","last_became_peered":"2024-04-26T18:24:17.934598+0000","last_unstale":"2024-04-26T18:26:07.432283+0000","last_undegraded":"2024-04-26T18:26:07.432283+0000","last_fullsized":"2024-04-26T18:26:07.432283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:28.690043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.333242+0000","last_change":"2024-04-26T18:24:18.142560+0000","last_active":"2024-04-26T18:26:07.333242+0000","last_peered":"2024-04-26T18:26:07.333242+0000","last_clean":"2024-04-26T18:26:07.333242+0000","last_became_active":"2024-04-26T18:24:18.142184+0000","last_became_peered":"2024-04-26T18:24:18.142184+0000","last_unstale":"2024-04-26T18:26:07.333242+0000","last_undegraded":"2024-04-26T18:26:07.333242+0000","last_fullsized":"2024-04-26T18:26:07.333242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:26.656113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296718+0000","last_change":"2024-04-26T18:24:20.952073+0000","last_active":"2024-04-26T18:26:07.296718+0000","last_peered":"2024-04-26T18:26:07.296718+0000","last_clean":"2024-04-26T18:26:07.296718+0000","last_became_active":"2024-04-26T18:24:20.951761+0000","last_became_peered":"2024-04-26T18:24:20.951761+0000","last_unstale":"2024-04-26T18:26:07.296718+0000","last_undegraded":"2024-04-26T18:26:07.296718+0000","last_fullsized":"2024-04-26T18:26:07.296718+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:30.244060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"91'15","reported_seq":46,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.429895+0000","last_change":"2024-04-26T18:24:17.927171+0000","last_active":"2024-04-26T18:26:07.429895+0000","last_peered":"2024-04-26T18:26:07.429895+0000","last_clean":"2024-04-26T18:26:07.429895+0000","last_became_active":"2024-04-26T18:24:17.926816+0000","last_became_peered":"2024-04-26T18:24:17.926816+0000","last_unstale":"2024-04-26T18:26:07.429895+0000","last_undegraded":"2024-04-26T18:26:07.429895+0000","last_fullsized":"2024-04-26T18:26:07.429895+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:18.249363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482678+0000","last_change":"2024-04-26T18:24:20.951684+0000","last_active":"2024-04-26T18:26:07.482678+0000","last_peered":"2024-04-26T18:26:07.482678+0000","last_clean":"2024-04-26T18:26:07.482678+0000","last_became_active":"2024-04-26T18:24:20.951364+0000","last_became_peered":"2024-04-26T18:24:20.951364+0000","last_unstale":"2024-04-26T18:26:07.482678+0000","last_undegraded":"2024-04-26T18:26:07.482678+0000","last_fullsized":"2024-04-26T18:26:07.482678+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:28.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":39,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302507+0000","last_change":"2024-04-26T18:24:18.229770+0000","last_active":"2024-04-26T18:26:07.302507+0000","last_peered":"2024-04-26T18:26:07.302507+0000","last_clean":"2024-04-26T18:26:07.302507+0000","last_became_active":"2024-04-26T18:24:18.229446+0000","last_became_peered":"2024-04-26T18:24:18.229446+0000","last_unstale":"2024-04-26T18:26:07.302507+0000","last_undegraded":"2024-04-26T18:26:07.302507+0000","last_fullsized":"2024-04-26T18:26:07.302507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:14.794797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482273+0000","last_change":"2024-04-26T18:24:20.960513+0000","last_active":"2024-04-26T18:26:07.482273+0000","last_peered":"2024-04-26T18:26:07.482273+0000","last_clean":"2024-04-26T18:26:07.482273+0000","last_became_active":"2024-04-26T18:24:20.959549+0000","last_became_peered":"2024-04-26T18:24:20.959549+0000","last_unstale":"2024-04-26T18:26:07.482273+0000","last_undegraded":"2024-04-26T18:26:07.482273+0000","last_fullsized":"2024-04-26T18:26:07.482273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:18:37.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.409900+0000","last_change":"2024-04-26T18:24:18.141549+0000","last_active":"2024-04-26T18:26:07.409900+0000","last_peered":"2024-04-26T18:26:07.409900+0000","last_clean":"2024-04-26T18:26:07.409900+0000","last_became_active":"2024-04-26T18:24:18.141172+0000","last_became_peered":"2024-04-26T18:24:18.141172+0000","last_unstale":"2024-04-26T18:26:07.409900+0000","last_undegraded":"2024-04-26T18:26:07.409900+0000","last_fullsized":"2024-04-26T18:26:07.409900+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:15.878130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.332971+0000","last_change":"2024-04-26T18:24:20.948121+0000","last_active":"2024-04-26T18:26:07.332971+0000","last_peered":"2024-04-26T18:26:07.332971+0000","last_clean":"2024-04-26T18:26:07.332971+0000","last_became_active":"2024-04-26T18:24:20.947738+0000","last_became_peered":"2024-04-26T18:24:20.947738+0000","last_unstale":"2024-04-26T18:26:07.332971+0000","last_undegraded":"2024-04-26T18:26:07.332971+0000","last_fullsized":"2024-04-26T18:26:07.332971+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:04.237302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":38,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.481968+0000","last_change":"2024-04-26T18:24:17.926631+0000","last_active":"2024-04-26T18:26:07.481968+0000","last_peered":"2024-04-26T18:26:07.481968+0000","last_clean":"2024-04-26T18:26:07.481968+0000","last_became_active":"2024-04-26T18:24:17.926259+0000","last_became_peered":"2024-04-26T18:24:17.926259+0000","last_unstale":"2024-04-26T18:26:07.481968+0000","last_undegraded":"2024-04-26T18:26:07.481968+0000","last_fullsized":"2024-04-26T18:26:07.481968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:47.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875648+0000","last_change":"2024-04-26T18:24:20.947283+0000","last_active":"2024-04-26T18:26:07.875648+0000","last_peered":"2024-04-26T18:26:07.875648+0000","last_clean":"2024-04-26T18:26:07.875648+0000","last_became_active":"2024-04-26T18:24:20.946812+0000","last_became_peered":"2024-04-26T18:24:20.946812+0000","last_unstale":"2024-04-26T18:26:07.875648+0000","last_undegraded":"2024-04-26T18:26:07.875648+0000","last_fullsized":"2024-04-26T18:26:07.875648+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:33.730698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.306172+0000","last_change":"2024-04-26T18:24:17.931843+0000","last_active":"2024-04-26T18:26:07.306172+0000","last_peered":"2024-04-26T18:26:07.306172+0000","last_clean":"2024-04-26T18:26:07.306172+0000","last_became_active":"2024-04-26T18:24:17.931442+0000","last_became_peered":"2024-04-26T18:24:17.931442+0000","last_unstale":"2024-04-26T18:26:07.306172+0000","last_undegraded":"2024-04-26T18:26:07.306172+0000","last_fullsized":"2024-04-26T18:26:07.306172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:09.649435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482129+0000","last_change":"2024-04-26T18:24:20.955661+0000","last_active":"2024-04-26T18:26:07.482129+0000","last_peered":"2024-04-26T18:26:07.482129+0000","last_clean":"2024-04-26T18:26:07.482129+0000","last_became_active":"2024-04-26T18:24:20.955344+0000","last_became_peered":"2024-04-26T18:24:20.955344+0000","last_unstale":"2024-04-26T18:26:07.482129+0000","last_undegraded":"2024-04-26T18:26:07.482129+0000","last_fullsized":"2024-04-26T18:26:07.482129+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:44.274381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":21,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875461+0000","last_change":"2024-04-26T18:25:06.315466+0000","last_active":"2024-04-26T18:26:07.875461+0000","last_peered":"2024-04-26T18:26:07.875461+0000","last_clean":"2024-04-26T18:26:07.875461+0000","last_became_active":"2024-04-26T18:25:06.301788+0000","last_became_peered":"2024-04-26T18:25:06.301788+0000","last_unstale":"2024-04-26T18:26:07.875461+0000","last_undegraded":"2024-04-26T18:26:07.875461+0000","last_fullsized":"2024-04-26T18:26:07.875461+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:12.998583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":30,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432437+0000","last_change":"2024-04-26T18:24:20.955746+0000","last_active":"2024-04-26T18:26:07.432437+0000","last_peered":"2024-04-26T18:26:07.432437+0000","last_clean":"2024-04-26T18:26:07.432437+0000","last_became_active":"2024-04-26T18:24:20.953652+0000","last_became_peered":"2024-04-26T18:24:20.953652+0000","last_unstale":"2024-04-26T18:26:07.432437+0000","last_undegraded":"2024-04-26T18:26:07.432437+0000","last_fullsized":"2024-04-26T18:26:07.432437+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:31.414185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":115,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:34.455457+0000","last_change":"2024-04-26T18:25:06.309393+0000","last_active":"2024-04-26T18:26:34.455457+0000","last_peered":"2024-04-26T18:26:34.455457+0000","last_clean":"2024-04-26T18:26:34.455457+0000","last_became_active":"2024-04-26T18:25:06.302434+0000","last_became_peered":"2024-04-26T18:25:06.302434+0000","last_unstale":"2024-04-26T18:26:34.455457+0000","last_undegraded":"2024-04-26T18:26:34.455457+0000","last_fullsized":"2024-04-26T18:26:34.455457+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:54.730295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":96,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432757+0000","last_change":"2024-04-26T18:24:20.954238+0000","last_active":"2024-04-26T18:26:07.432757+0000","last_peered":"2024-04-26T18:26:07.432757+0000","last_clean":"2024-04-26T18:26:07.432757+0000","last_became_active":"2024-04-26T18:24:20.953883+0000","last_became_peered":"2024-04-26T18:24:20.953883+0000","last_unstale":"2024-04-26T18:26:07.432757+0000","last_undegraded":"2024-04-26T18:26:07.432757+0000","last_fullsized":"2024-04-26T18:26:07.432757+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:10.580835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.302194+0000","last_change":"2024-04-26T18:24:17.926901+0000","last_active":"2024-04-26T18:26:07.302194+0000","last_peered":"2024-04-26T18:26:07.302194+0000","last_clean":"2024-04-26T18:26:07.302194+0000","last_became_active":"2024-04-26T18:24:17.926291+0000","last_became_peered":"2024-04-26T18:24:17.926291+0000","last_unstale":"2024-04-26T18:26:07.302194+0000","last_undegraded":"2024-04-26T18:26:07.302194+0000","last_fullsized":"2024-04-26T18:26:07.302194+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:17.695198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432411+0000","last_change":"2024-04-26T18:24:20.950578+0000","last_active":"2024-04-26T18:26:07.432411+0000","last_peered":"2024-04-26T18:26:07.432411+0000","last_clean":"2024-04-26T18:26:07.432411+0000","last_became_active":"2024-04-26T18:24:20.950174+0000","last_became_peered":"2024-04-26T18:24:20.950174+0000","last_unstale":"2024-04-26T18:26:07.432411+0000","last_undegraded":"2024-04-26T18:26:07.432411+0000","last_fullsized":"2024-04-26T18:26:07.432411+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296654+0000","last_change":"2024-04-26T18:24:17.928054+0000","last_active":"2024-04-26T18:26:07.296654+0000","last_peered":"2024-04-26T18:26:07.296654+0000","last_clean":"2024-04-26T18:26:07.296654+0000","last_became_active":"2024-04-26T18:24:17.927715+0000","last_became_peered":"2024-04-26T18:24:17.927715+0000","last_unstale":"2024-04-26T18:26:07.296654+0000","last_undegraded":"2024-04-26T18:26:07.296654+0000","last_fullsized":"2024-04-26T18:26:07.296654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.155436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.432673+0000","last_change":"2024-04-26T18:24:20.944512+0000","last_active":"2024-04-26T18:26:07.432673+0000","last_peered":"2024-04-26T18:26:07.432673+0000","last_clean":"2024-04-26T18:26:07.432673+0000","last_became_active":"2024-04-26T18:24:20.944123+0000","last_became_peered":"2024-04-26T18:24:20.944123+0000","last_unstale":"2024-04-26T18:26:07.432673+0000","last_undegraded":"2024-04-26T18:26:07.432673+0000","last_fullsized":"2024-04-26T18:26:07.432673+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.240206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875583+0000","last_change":"2024-04-26T18:24:20.959138+0000","last_active":"2024-04-26T18:26:07.875583+0000","last_peered":"2024-04-26T18:26:07.875583+0000","last_clean":"2024-04-26T18:26:07.875583+0000","last_became_active":"2024-04-26T18:24:20.958675+0000","last_became_peered":"2024-04-26T18:24:20.958675+0000","last_unstale":"2024-04-26T18:26:07.875583+0000","last_undegraded":"2024-04-26T18:26:07.875583+0000","last_fullsized":"2024-04-26T18:26:07.875583+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:34:32.103380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.306074+0000","last_change":"2024-04-26T18:24:20.961689+0000","last_active":"2024-04-26T18:26:07.306074+0000","last_peered":"2024-04-26T18:26:07.306074+0000","last_clean":"2024-04-26T18:26:07.306074+0000","last_became_active":"2024-04-26T18:24:20.961374+0000","last_became_peered":"2024-04-26T18:24:20.961374+0000","last_unstale":"2024-04-26T18:26:07.306074+0000","last_undegraded":"2024-04-26T18:26:07.306074+0000","last_fullsized":"2024-04-26T18:26:07.306074+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:55.243887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.306123+0000","last_change":"2024-04-26T18:24:20.950711+0000","last_active":"2024-04-26T18:26:07.306123+0000","last_peered":"2024-04-26T18:26:07.306123+0000","last_clean":"2024-04-26T18:26:07.306123+0000","last_became_active":"2024-04-26T18:24:20.950371+0000","last_became_peered":"2024-04-26T18:24:20.950371+0000","last_unstale":"2024-04-26T18:26:07.306123+0000","last_undegraded":"2024-04-26T18:26:07.306123+0000","last_fullsized":"2024-04-26T18:26:07.306123+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:35.307921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482120+0000","last_change":"2024-04-26T18:24:20.959990+0000","last_active":"2024-04-26T18:26:07.482120+0000","last_peered":"2024-04-26T18:26:07.482120+0000","last_clean":"2024-04-26T18:26:07.482120+0000","last_became_active":"2024-04-26T18:24:20.959514+0000","last_became_peered":"2024-04-26T18:24:20.959514+0000","last_unstale":"2024-04-26T18:26:07.482120+0000","last_undegraded":"2024-04-26T18:26:07.482120+0000","last_fullsized":"2024-04-26T18:26:07.482120+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:00.184448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296793+0000","last_change":"2024-04-26T18:24:20.957338+0000","last_active":"2024-04-26T18:26:07.296793+0000","last_peered":"2024-04-26T18:26:07.296793+0000","last_clean":"2024-04-26T18:26:07.296793+0000","last_became_active":"2024-04-26T18:24:20.956999+0000","last_became_peered":"2024-04-26T18:24:20.956999+0000","last_unstale":"2024-04-26T18:26:07.296793+0000","last_undegraded":"2024-04-26T18:26:07.296793+0000","last_fullsized":"2024-04-26T18:26:07.296793+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.066902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875773+0000","last_change":"2024-04-26T18:24:20.953634+0000","last_active":"2024-04-26T18:26:07.875773+0000","last_peered":"2024-04-26T18:26:07.875773+0000","last_clean":"2024-04-26T18:26:07.875773+0000","last_became_active":"2024-04-26T18:24:20.953259+0000","last_became_peered":"2024-04-26T18:24:20.953259+0000","last_unstale":"2024-04-26T18:26:07.875773+0000","last_undegraded":"2024-04-26T18:26:07.875773+0000","last_fullsized":"2024-04-26T18:26:07.875773+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:00.095344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.306497+0000","last_change":"2024-04-26T18:24:20.954997+0000","last_active":"2024-04-26T18:26:07.306497+0000","last_peered":"2024-04-26T18:26:07.306497+0000","last_clean":"2024-04-26T18:26:07.306497+0000","last_became_active":"2024-04-26T18:24:20.954223+0000","last_became_peered":"2024-04-26T18:24:20.954223+0000","last_unstale":"2024-04-26T18:26:07.306497+0000","last_undegraded":"2024-04-26T18:26:07.306497+0000","last_fullsized":"2024-04-26T18:26:07.306497+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:14.092455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.482575+0000","last_change":"2024-04-26T18:24:20.957056+0000","last_active":"2024-04-26T18:26:07.482575+0000","last_peered":"2024-04-26T18:26:07.482575+0000","last_clean":"2024-04-26T18:26:07.482575+0000","last_became_active":"2024-04-26T18:24:20.956737+0000","last_became_peered":"2024-04-26T18:24:20.956737+0000","last_unstale":"2024-04-26T18:26:07.482575+0000","last_undegraded":"2024-04-26T18:26:07.482575+0000","last_fullsized":"2024-04-26T18:26:07.482575+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:48.358862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":37,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.301943+0000","last_change":"2024-04-26T18:25:06.350206+0000","last_active":"2024-04-26T18:26:07.301943+0000","last_peered":"2024-04-26T18:26:07.301943+0000","last_clean":"2024-04-26T18:26:07.301943+0000","last_became_active":"2024-04-26T18:25:06.349866+0000","last_became_peered":"2024-04-26T18:25:06.349866+0000","last_unstale":"2024-04-26T18:26:07.301943+0000","last_undegraded":"2024-04-26T18:26:07.301943+0000","last_fullsized":"2024-04-26T18:26:07.301943+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:41.909056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":32,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.481796+0000","last_change":"2024-04-26T18:24:20.956000+0000","last_active":"2024-04-26T18:26:07.481796+0000","last_peered":"2024-04-26T18:26:07.481796+0000","last_clean":"2024-04-26T18:26:07.481796+0000","last_became_active":"2024-04-26T18:24:20.955677+0000","last_became_peered":"2024-04-26T18:24:20.955677+0000","last_unstale":"2024-04-26T18:26:07.481796+0000","last_undegraded":"2024-04-26T18:26:07.481796+0000","last_fullsized":"2024-04-26T18:26:07.481796+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:20.895674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.410027+0000","last_change":"2024-04-26T18:24:20.959317+0000","last_active":"2024-04-26T18:26:07.410027+0000","last_peered":"2024-04-26T18:26:07.410027+0000","last_clean":"2024-04-26T18:26:07.410027+0000","last_became_active":"2024-04-26T18:24:20.958963+0000","last_became_peered":"2024-04-26T18:24:20.958963+0000","last_unstale":"2024-04-26T18:26:07.410027+0000","last_undegraded":"2024-04-26T18:26:07.410027+0000","last_fullsized":"2024-04-26T18:26:07.410027+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:08.231089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.296857+0000","last_change":"2024-04-26T18:24:20.954445+0000","last_active":"2024-04-26T18:26:07.296857+0000","last_peered":"2024-04-26T18:26:07.296857+0000","last_clean":"2024-04-26T18:26:07.296857+0000","last_became_active":"2024-04-26T18:24:20.953852+0000","last_became_peered":"2024-04-26T18:24:20.953852+0000","last_unstale":"2024-04-26T18:26:07.296857+0000","last_undegraded":"2024-04-26T18:26:07.296857+0000","last_fullsized":"2024-04-26T18:26:07.296857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:05.636897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":93,"state":"active+clean","last_fresh":"2024-04-26T18:26:07.875625+0000","last_change":"2024-04-26T18:24:20.958023+0000","last_active":"2024-04-26T18:26:07.875625+0000","last_peered":"2024-04-26T18:26:07.875625+0000","last_clean":"2024-04-26T18:26:07.875625+0000","last_became_active":"2024-04-26T18:24:20.957642+0000","last_became_peered":"2024-04-26T18:24:20.957642+0000","last_unstale":"2024-04-26T18:26:07.875625+0000","last_undegraded":"2024-04-26T18:26:07.875625+0000","last_fullsized":"2024-04-26T18:26:07.875625+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:46.332710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":393216,"data_stored":4132576,"data_compressed":244217,"data_compressed_allocated":315392,"data_compressed_original":4079616,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72459,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1171,"num_read_kb":672,"num_write":254,"num_write_kb":146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1583,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":217377,"data_compressed":5529,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":219,"ondisk_log_size":219,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351019,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":501214,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612655,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32332,"kb_used_data":1148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712820,"statfs":{"total":95995035648,"available":95961927680,"internally_reserved":0,"allocated":1175552,"data_stored":1156662,"data_compressed":49004,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776181,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1044480,"data_stored":500590,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037818,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":500946,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299455,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1093627,"data_compressed":47161,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462981,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":500307,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1097728,"data_stored":567895,"data_compressed":13413,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":502178,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":565775,"data_compressed":13413,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149779,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28044,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717108,"statfs":{"total":95995035648,"available":95966318592,"internally_reserved":0,"allocated":1044480,"data_stored":500132,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888143,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":500168,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724616,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1105920,"data_stored":1093305,"data_compressed":47161,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":33951,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":68427,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":66307,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":66709,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:26:38.461 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T18:26:38.461 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T18:26:38.462 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T18:26:38.462 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T18:26:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:38 smithi103 ceph-mon[32727]: pgmap v369: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:38.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:38 smithi174 ceph-mon[32705]: pgmap v369: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:38.702 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:39.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:39 smithi084 ceph-mon[29141]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:26:39.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:39 smithi103 ceph-mon[32727]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:26:39.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:39 smithi174 ceph-mon[32705]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:26:40.389 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:40 smithi084 ceph-mon[29141]: pgmap v370: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:40.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.389+0000 7f2bd4c89700 1 -- 172.21.15.84:0/2432688612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd0102320 msgr2=0x7f2bd01027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:40.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.389+0000 7f2bd4c89700 1 --2- 172.21.15.84:0/2432688612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd0102320 0x7f2bd01027a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4009990 tx=0x7f2bc4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:40.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.389+0000 7f2bd4c89700 1 -- 172.21.15.84:0/2432688612 shutdown_connections 2024-04-26T18:26:40.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.389+0000 7f2bd4c89700 1 --2- 172.21.15.84:0/2432688612 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd0102ce0 0x7f2bd010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:40.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.389+0000 7f2bd4c89700 1 --2- 172.21.15.84:0/2432688612 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bd0108390 0x7f2bd0108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:40.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.389+0000 7f2bd4c89700 1 --2- 172.21.15.84:0/2432688612 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd0102320 0x7f2bd01027a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:40.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.389+0000 7f2bd4c89700 1 -- 172.21.15.84:0/2432688612 >> 172.21.15.84:0/2432688612 conn(0x7f2bd00792d0 msgr2=0x7f2bd00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:40.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.389+0000 7f2bd4c89700 1 -- 172.21.15.84:0/2432688612 shutdown_connections 2024-04-26T18:26:40.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.389+0000 7f2bd4c89700 1 -- 172.21.15.84:0/2432688612 wait complete. 2024-04-26T18:26:40.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.390+0000 7f2bd4c89700 1 Processor -- start 2024-04-26T18:26:40.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.390+0000 7f2bd4c89700 1 -- start start 2024-04-26T18:26:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.390+0000 7f2bd4c89700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd0102320 0x7f2bd019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.390+0000 7f2bd4c89700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bd0102ce0 0x7f2bd019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.390+0000 7f2bd4c89700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd0108390 0x7f2bd01998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.390+0000 7f2bd4c89700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd01124e0 con 0x7f2bd0102320 2024-04-26T18:26:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.390+0000 7f2bd4c89700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0112360 con 0x7f2bd0102ce0 2024-04-26T18:26:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.390+0000 7f2bd4c89700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd0112660 con 0x7f2bd0108390 2024-04-26T18:26:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bce59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd0102320 0x7f2bd019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bce59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd0102320 0x7f2bd019f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50200/0 (socket says 172.21.15.84:50200) 2024-04-26T18:26:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bce59c700 1 -- 172.21.15.84:0/4159281217 learned_addr learned my addr 172.21.15.84:0/4159281217 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bcdd9b700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bd0102ce0 0x7f2bd019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bced9d700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd0108390 0x7f2bd01998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bce59c700 1 -- 172.21.15.84:0/4159281217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd0108390 msgr2=0x7f2bd01998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:40.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bce59c700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd0108390 0x7f2bd01998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bce59c700 1 -- 172.21.15.84:0/4159281217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bd0102ce0 msgr2=0x7f2bd019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bce59c700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bd0102ce0 0x7f2bd019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bce59c700 1 -- 172.21.15.84:0/4159281217 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd019a030 con 0x7f2bd0102320 2024-04-26T18:26:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bced9d700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd0108390 0x7f2bd01998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bce59c700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd0102320 0x7f2bd019f710 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8007f10 tx=0x7f2bc800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bbf7fe700 1 -- 172.21.15.84:0/4159281217 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800ddd0 con 0x7f2bd0102320 2024-04-26T18:26:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bbf7fe700 1 -- 172.21.15.84:0/4159281217 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bc8004bb0 con 0x7f2bd0102320 2024-04-26T18:26:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bd4c89700 1 -- 172.21.15.84:0/4159281217 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd019a320 con 0x7f2bd0102320 2024-04-26T18:26:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bbf7fe700 1 -- 172.21.15.84:0/4159281217 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80046a0 con 0x7f2bd0102320 2024-04-26T18:26:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.391+0000 7f2bd4c89700 1 -- 172.21.15.84:0/4159281217 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd01a65e0 con 0x7f2bd0102320 2024-04-26T18:26:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.392+0000 7f2bd4c89700 1 -- 172.21.15.84:0/4159281217 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd006a110 con 0x7f2bd0102320 2024-04-26T18:26:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.393+0000 7f2bbf7fe700 1 -- 172.21.15.84:0/4159281217 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2bc801c020 con 0x7f2bd0102320 2024-04-26T18:26:40.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.394+0000 7f2bbf7fe700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f2bb80a3c40 0x7f2bb80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:40.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.394+0000 7f2bcdd9b700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f2bb80a3c40 0x7f2bb80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:40.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.394+0000 7f2bbf7fe700 1 -- 172.21.15.84:0/4159281217 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f2bc80d06b0 con 0x7f2bd0102320 2024-04-26T18:26:40.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.394+0000 7f2bcdd9b700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f2bb80a3c40 0x7f2bb80a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400b5c0 tx=0x7f2bc4005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:40.411 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.398+0000 7f2bbf7fe700 1 -- 172.21.15.84:0/4159281217 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bc8023030 con 0x7f2bd0102320 2024-04-26T18:26:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:40 smithi103 ceph-mon[32727]: pgmap v370: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:40.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.644+0000 7f2bd4c89700 1 -- 172.21.15.84:0/4159281217 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f2bd01a67b0 con 0x7f2bd0102320 2024-04-26T18:26:40.645 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.645+0000 7f2bbf7fe700 1 -- 172.21.15.84:0/4159281217 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f2bc8010070 con 0x7f2bd0102320 2024-04-26T18:26:40.645 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:26:40.645 INFO:teuthology.orchestra.run.smithi084.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T18:26:40.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.647+0000 7f2bd4c89700 1 -- 172.21.15.84:0/4159281217 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f2bb80a3c40 msgr2=0x7f2bb80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:40.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.647+0000 7f2bd4c89700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f2bb80a3c40 0x7f2bb80a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400b5c0 tx=0x7f2bc4005960 comp rx=0 tx=0).stop 2024-04-26T18:26:40.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.647+0000 7f2bd4c89700 1 -- 172.21.15.84:0/4159281217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd0102320 msgr2=0x7f2bd019f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:40.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.647+0000 7f2bd4c89700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd0102320 0x7f2bd019f710 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8007f10 tx=0x7f2bc800d3b0 comp rx=0 tx=0).stop 2024-04-26T18:26:40.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.647+0000 7f2bd4c89700 1 -- 172.21.15.84:0/4159281217 shutdown_connections 2024-04-26T18:26:40.647 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.647+0000 7f2bd4c89700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f2bb80a3c40 0x7f2bb80a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:40.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.647+0000 7f2bd4c89700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2bd0102320 0x7f2bd019f710 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:40.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.647+0000 7f2bd4c89700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2bd0108390 0x7f2bd01998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:40.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.647+0000 7f2bd4c89700 1 --2- 172.21.15.84:0/4159281217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2bd0102ce0 0x7f2bd019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:40.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.647+0000 7f2bd4c89700 1 -- 172.21.15.84:0/4159281217 >> 172.21.15.84:0/4159281217 conn(0x7f2bd00792d0 msgr2=0x7f2bd0101630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:40.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.647+0000 7f2bd4c89700 1 -- 172.21.15.84:0/4159281217 shutdown_connections 2024-04-26T18:26:40.648 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:40.647+0000 7f2bd4c89700 1 -- 172.21.15.84:0/4159281217 wait complete. 2024-04-26T18:26:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:40 smithi174 ceph-mon[32705]: pgmap v370: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:41.304 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T18:26:41.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:26:41.548 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:41.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:41 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/4159281217' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T18:26:41.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:41 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/4159281217' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T18:26:41.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:41 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/4159281217' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T18:26:42.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:42 smithi084 ceph-mon[29141]: pgmap v371: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:42.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:42 smithi103 ceph-mon[32727]: pgmap v371: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:42.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:42 smithi174 ceph-mon[32705]: pgmap v371: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:43.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.106+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/2237991113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dbc109480 msgr2=0x7f7dbc109860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:43.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.106+0000 7f7dc1f7e700 1 --2- 172.21.15.84:0/2237991113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dbc109480 0x7f7dbc109860 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7dac009990 tx=0x7f7dac009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:43.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.106+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/2237991113 shutdown_connections 2024-04-26T18:26:43.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.106+0000 7f7dc1f7e700 1 --2- 172.21.15.84:0/2237991113 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dbc109480 0x7f7dbc109860 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:43.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.106+0000 7f7dc1f7e700 1 --2- 172.21.15.84:0/2237991113 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc103ea0 0x7f7dbc1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:43.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.106+0000 7f7dc1f7e700 1 --2- 172.21.15.84:0/2237991113 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dbc1034e0 0x7f7dbc103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:43.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.106+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/2237991113 >> 172.21.15.84:0/2237991113 conn(0x7f7dbc07a5e0 msgr2=0x7f7dbc07a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:43.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.106+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/2237991113 shutdown_connections 2024-04-26T18:26:43.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.106+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/2237991113 wait complete. 2024-04-26T18:26:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.107+0000 7f7dc1f7e700 1 Processor -- start 2024-04-26T18:26:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.107+0000 7f7dc1f7e700 1 -- start start 2024-04-26T18:26:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.107+0000 7f7dc1f7e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dbc1034e0 0x7f7dbc19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.107+0000 7f7dc1f7e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc103ea0 0x7f7dbc19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.107+0000 7f7dc1f7e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dbc109480 0x7f7dbc199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.107+0000 7f7dc1f7e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc113610 con 0x7f7dbc109480 2024-04-26T18:26:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.107+0000 7f7dc1f7e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc113490 con 0x7f7dbc1034e0 2024-04-26T18:26:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.107+0000 7f7dc1f7e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7dbc113790 con 0x7f7dbc103ea0 2024-04-26T18:26:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.107+0000 7f7dbbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dbc109480 0x7f7dbc199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.107+0000 7f7dbbfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dbc109480 0x7f7dbc199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50224/0 (socket says 172.21.15.84:50224) 2024-04-26T18:26:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.107+0000 7f7dbbfff700 1 -- 172.21.15.84:0/3118001934 learned_addr learned my addr 172.21.15.84:0/3118001934 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:43.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.108+0000 7f7dbb7fe700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dbc1034e0 0x7f7dbc19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.108+0000 7f7dbaffd700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc103ea0 0x7f7dbc19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.108+0000 7f7dbbfff700 1 -- 172.21.15.84:0/3118001934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc103ea0 msgr2=0x7f7dbc19fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.108+0000 7f7dbbfff700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc103ea0 0x7f7dbc19fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.108+0000 7f7dbbfff700 1 -- 172.21.15.84:0/3118001934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dbc1034e0 msgr2=0x7f7dbc19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.108+0000 7f7dbbfff700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dbc1034e0 0x7f7dbc19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.108+0000 7f7dbbfff700 1 -- 172.21.15.84:0/3118001934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dbc19a090 con 0x7f7dbc109480 2024-04-26T18:26:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.108+0000 7f7dbbfff700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dbc109480 0x7f7dbc199930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7db400b970 tx=0x7f7db400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.108+0000 7f7db8ff9700 1 -- 172.21.15.84:0/3118001934 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4019070 con 0x7f7dbc109480 2024-04-26T18:26:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.108+0000 7f7db8ff9700 1 -- 172.21.15.84:0/3118001934 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7db400cde0 con 0x7f7dbc109480 2024-04-26T18:26:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.108+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/3118001934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dbc19a380 con 0x7f7dbc109480 2024-04-26T18:26:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.108+0000 7f7db8ff9700 1 -- 172.21.15.84:0/3118001934 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7db4007370 con 0x7f7dbc109480 2024-04-26T18:26:43.111 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.108+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/3118001934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dbc1a6640 con 0x7f7dbc109480 2024-04-26T18:26:43.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.109+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/3118001934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dbc104b30 con 0x7f7dbc109480 2024-04-26T18:26:43.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.110+0000 7f7db8ff9700 1 -- 172.21.15.84:0/3118001934 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7db401c070 con 0x7f7dbc109480 2024-04-26T18:26:43.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.111+0000 7f7db8ff9700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f7da40a3c40 0x7f7da40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:43.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.111+0000 7f7dbb7fe700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f7da40a3c40 0x7f7da40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:43.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.111+0000 7f7db8ff9700 1 -- 172.21.15.84:0/3118001934 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f7db40cf8f0 con 0x7f7dbc109480 2024-04-26T18:26:43.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.111+0000 7f7dbb7fe700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f7da40a3c40 0x7f7da40a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00b5c0 tx=0x7f7dac005960 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:43.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.115+0000 7f7db8ff9700 1 -- 172.21.15.84:0/3118001934 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7db409c2a0 con 0x7f7dbc109480 2024-04-26T18:26:43.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.362+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/3118001934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7dbc19ae60 con 0x7f7dbc109480 2024-04-26T18:26:43.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.362+0000 7f7db8ff9700 1 -- 172.21.15.84:0/3118001934 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11936 (secure 0 0 0) 0x7f7db409bba0 con 0x7f7dbc109480 2024-04-26T18:26:43.362 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:26:43.362 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:24:41.425794+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24407,24418,24422,24430,24434]},"id":1}]} 2024-04-26T18:26:43.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.364+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/3118001934 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f7da40a3c40 msgr2=0x7f7da40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:43.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.364+0000 7f7dc1f7e700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f7da40a3c40 0x7f7da40a6100 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7dac00b5c0 tx=0x7f7dac005960 comp rx=0 tx=0).stop 2024-04-26T18:26:43.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.364+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/3118001934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dbc109480 msgr2=0x7f7dbc199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:43.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.364+0000 7f7dc1f7e700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dbc109480 0x7f7dbc199930 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7db400b970 tx=0x7f7db400bc80 comp rx=0 tx=0).stop 2024-04-26T18:26:43.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.364+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/3118001934 shutdown_connections 2024-04-26T18:26:43.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.364+0000 7f7dc1f7e700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f7da40a3c40 0x7f7da40a6100 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:43.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.364+0000 7f7dc1f7e700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7dbc109480 0x7f7dbc199930 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:43.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.364+0000 7f7dc1f7e700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7dbc1034e0 0x7f7dbc19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:43.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.364+0000 7f7dc1f7e700 1 --2- 172.21.15.84:0/3118001934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7dbc103ea0 0x7f7dbc19fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:43.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.364+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/3118001934 >> 172.21.15.84:0/3118001934 conn(0x7f7dbc07a5e0 msgr2=0x7f7dbc101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:43.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.365+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/3118001934 shutdown_connections 2024-04-26T18:26:43.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:43.365+0000 7f7dc1f7e700 1 -- 172.21.15.84:0/3118001934 wait complete. 2024-04-26T18:26:43.366 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 28 2024-04-26T18:26:44.050 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:26:44.291 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:44.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:44 smithi084 ceph-mon[29141]: pgmap v372: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:44.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:44 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3118001934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:26:44.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:44 smithi103 ceph-mon[32727]: pgmap v372: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:44.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:44 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3118001934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:26:44.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:44 smithi174 ceph-mon[32705]: pgmap v372: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:44.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:44 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3118001934' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:26:45.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.993+0000 7f144c5ec700 1 -- 172.21.15.84:0/1013572260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14441025b0 msgr2=0x7f1444102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:45.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.993+0000 7f144c5ec700 1 --2- 172.21.15.84:0/1013572260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14441025b0 0x7f1444102990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f14400099c0 tx=0x7f1440009cd0 comp rx=0 tx=0).stop 2024-04-26T18:26:45.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.993+0000 7f144c5ec700 1 -- 172.21.15.84:0/1013572260 shutdown_connections 2024-04-26T18:26:45.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.993+0000 7f144c5ec700 1 --2- 172.21.15.84:0/1013572260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14441025b0 0x7f1444102990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:45.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.993+0000 7f144c5ec700 1 --2- 172.21.15.84:0/1013572260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1444107640 0x7f144410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:45.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.993+0000 7f144c5ec700 1 --2- 172.21.15.84:0/1013572260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1444102f60 0x7f1444106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:45.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.993+0000 7f144c5ec700 1 -- 172.21.15.84:0/1013572260 >> 172.21.15.84:0/1013572260 conn(0x7f14440792f0 msgr2=0x7f1444079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:45.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.993+0000 7f144c5ec700 1 -- 172.21.15.84:0/1013572260 shutdown_connections 2024-04-26T18:26:45.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.993+0000 7f144c5ec700 1 -- 172.21.15.84:0/1013572260 wait complete. 2024-04-26T18:26:45.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.993+0000 7f144c5ec700 1 Processor -- start 2024-04-26T18:26:45.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.993+0000 7f144c5ec700 1 -- start start 2024-04-26T18:26:45.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144c5ec700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14441025b0 0x7f144419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:45.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144c5ec700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1444102f60 0x7f144419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:45.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144c5ec700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1444107640 0x7f14441999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:45.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144c5ec700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1444112550 con 0x7f1444107640 2024-04-26T18:26:45.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144c5ec700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14441123d0 con 0x7f1444102f60 2024-04-26T18:26:45.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144c5ec700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14441126d0 con 0x7f14441025b0 2024-04-26T18:26:45.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144ab89700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1444107640 0x7f14441999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:45.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144ab89700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1444107640 0x7f14441999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50248/0 (socket says 172.21.15.84:50248) 2024-04-26T18:26:45.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144ab89700 1 -- 172.21.15.84:0/3808754333 learned_addr learned my addr 172.21.15.84:0/3808754333 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:45.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f1449b87700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1444102f60 0x7f144419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:45.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144a388700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14441025b0 0x7f144419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:45.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144ab89700 1 -- 172.21.15.84:0/3808754333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14441025b0 msgr2=0x7f144419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:46.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144ab89700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14441025b0 0x7f144419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:46.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144ab89700 1 -- 172.21.15.84:0/3808754333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1444102f60 msgr2=0x7f144419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:46.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144ab89700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1444102f60 0x7f144419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:46.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144ab89700 1 -- 172.21.15.84:0/3808754333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144419a140 con 0x7f1444107640 2024-04-26T18:26:46.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144ab89700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1444107640 0x7f14441999e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b970 tx=0x7f143c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:46.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f143b7fe700 1 -- 172.21.15.84:0/3808754333 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c00d860 con 0x7f1444107640 2024-04-26T18:26:46.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f143b7fe700 1 -- 172.21.15.84:0/3808754333 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f143c00e430 con 0x7f1444107640 2024-04-26T18:26:46.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144c5ec700 1 -- 172.21.15.84:0/3808754333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144419a430 con 0x7f1444107640 2024-04-26T18:26:46.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f143b7fe700 1 -- 172.21.15.84:0/3808754333 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f143c017660 con 0x7f1444107640 2024-04-26T18:26:46.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.994+0000 7f144c5ec700 1 -- 172.21.15.84:0/3808754333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14441a66f0 con 0x7f1444107640 2024-04-26T18:26:46.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.995+0000 7f144c5ec700 1 -- 172.21.15.84:0/3808754333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f144406a110 con 0x7f1444107640 2024-04-26T18:26:46.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.996+0000 7f143b7fe700 1 -- 172.21.15.84:0/3808754333 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f143c00da00 con 0x7f1444107640 2024-04-26T18:26:46.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.997+0000 7f143b7fe700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14300a3c40 0x7f14300a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:46.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.997+0000 7f144a388700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14300a3c40 0x7f14300a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:46.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.997+0000 7f143b7fe700 1 -- 172.21.15.84:0/3808754333 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f143c0cfff0 con 0x7f1444107640 2024-04-26T18:26:46.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:45.997+0000 7f144a388700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14300a3c40 0x7f14300a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f144419ab20 tx=0x7f1440005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:46.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.001+0000 7f143b7fe700 1 -- 172.21.15.84:0/3808754333 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f143c013030 con 0x7f1444107640 2024-04-26T18:26:46.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.247+0000 7f144c5ec700 1 -- 172.21.15.84:0/3808754333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f144402e490 con 0x7f1444107640 2024-04-26T18:26:46.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.248+0000 7f143b7fe700 1 -- 172.21.15.84:0/3808754333 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11936 (secure 0 0 0) 0x7f143c09c9a0 con 0x7f1444107640 2024-04-26T18:26:46.248 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:26:46.248 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:24:41.425794+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24407,24418,24422,24430,24434]},"id":1}]} 2024-04-26T18:26:46.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.250+0000 7f144c5ec700 1 -- 172.21.15.84:0/3808754333 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14300a3c40 msgr2=0x7f14300a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:46.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.250+0000 7f144c5ec700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14300a3c40 0x7f14300a6100 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f144419ab20 tx=0x7f1440005c40 comp rx=0 tx=0).stop 2024-04-26T18:26:46.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.250+0000 7f144c5ec700 1 -- 172.21.15.84:0/3808754333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1444107640 msgr2=0x7f14441999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:46.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.250+0000 7f144c5ec700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1444107640 0x7f14441999e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f143c00b970 tx=0x7f143c00bc80 comp rx=0 tx=0).stop 2024-04-26T18:26:46.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.250+0000 7f144c5ec700 1 -- 172.21.15.84:0/3808754333 shutdown_connections 2024-04-26T18:26:46.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.250+0000 7f144c5ec700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14300a3c40 0x7f14300a6100 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:46.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.250+0000 7f144c5ec700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1444107640 0x7f14441999e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:46.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.250+0000 7f144c5ec700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14441025b0 0x7f144419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:46.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.250+0000 7f144c5ec700 1 --2- 172.21.15.84:0/3808754333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1444102f60 0x7f144419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:46.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.250+0000 7f144c5ec700 1 -- 172.21.15.84:0/3808754333 >> 172.21.15.84:0/3808754333 conn(0x7f14440792f0 msgr2=0x7f144406bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:46.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.250+0000 7f144c5ec700 1 -- 172.21.15.84:0/3808754333 shutdown_connections 2024-04-26T18:26:46.251 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:46.250+0000 7f144c5ec700 1 -- 172.21.15.84:0/3808754333 wait complete. 2024-04-26T18:26:46.251 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 28 2024-04-26T18:26:46.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:46 smithi084 ceph-mon[29141]: pgmap v373: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:46.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:46 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3808754333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:26:46.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:46 smithi103 ceph-mon[32727]: pgmap v373: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:46.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:46 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3808754333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:26:46.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:46 smithi174 ceph-mon[32705]: pgmap v373: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:46.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:46 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3808754333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:26:46.949 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:26:47.188 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:48.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:48 smithi084 ceph-mon[29141]: pgmap v374: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:48 smithi103 ceph-mon[32727]: pgmap v374: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:48 smithi174 ceph-mon[32705]: pgmap v374: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.842+0000 7fe6990b0700 1 -- 172.21.15.84:0/80437811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6941094c0 msgr2=0x7fe6941098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.842+0000 7fe6990b0700 1 --2- 172.21.15.84:0/80437811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6941094c0 0x7fe6941098a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fe688009990 tx=0x7fe688009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.842+0000 7fe6990b0700 1 -- 172.21.15.84:0/80437811 shutdown_connections 2024-04-26T18:26:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.842+0000 7fe6990b0700 1 --2- 172.21.15.84:0/80437811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe6941094c0 0x7fe6941098a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.842+0000 7fe6990b0700 1 --2- 172.21.15.84:0/80437811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe694103e30 0x7fe694110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.842+0000 7fe6990b0700 1 --2- 172.21.15.84:0/80437811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe694103470 0x7fe6941038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.842+0000 7fe6990b0700 1 -- 172.21.15.84:0/80437811 >> 172.21.15.84:0/80437811 conn(0x7fe69407a580 msgr2=0x7fe69407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.842+0000 7fe6990b0700 1 -- 172.21.15.84:0/80437811 shutdown_connections 2024-04-26T18:26:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.842+0000 7fe6990b0700 1 -- 172.21.15.84:0/80437811 wait complete. 2024-04-26T18:26:48.843 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.843+0000 7fe6990b0700 1 Processor -- start 2024-04-26T18:26:48.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.843+0000 7fe6990b0700 1 -- start start 2024-04-26T18:26:48.845 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.843+0000 7fe6990b0700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe694103470 0x7fe69419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.843+0000 7fe6990b0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe694103e30 0x7fe69419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.843+0000 7fe6990b0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6941094c0 0x7fe694199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.843+0000 7fe6990b0700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe6941135c0 con 0x7fe694103e30 2024-04-26T18:26:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.843+0000 7fe6990b0700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe694113440 con 0x7fe694103470 2024-04-26T18:26:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.843+0000 7fe6990b0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe694113740 con 0x7fe6941094c0 2024-04-26T18:26:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.843+0000 7fe69259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe694103e30 0x7fe69419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.843+0000 7fe69259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe694103e30 0x7fe69419fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46616/0 (socket says 172.21.15.84:46616) 2024-04-26T18:26:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.843+0000 7fe69259c700 1 -- 172.21.15.84:0/3273252182 learned_addr learned my addr 172.21.15.84:0/3273252182 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe69359e700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6941094c0 0x7fe694199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe692d9d700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe694103470 0x7fe69419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe69259c700 1 -- 172.21.15.84:0/3273252182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6941094c0 msgr2=0x7fe694199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:48.846 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe69259c700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6941094c0 0x7fe694199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:48.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe69259c700 1 -- 172.21.15.84:0/3273252182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe694103470 msgr2=0x7fe69419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:48.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe69259c700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe694103470 0x7fe69419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:48.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe69259c700 1 -- 172.21.15.84:0/3273252182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69419a060 con 0x7fe694103e30 2024-04-26T18:26:48.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe69359e700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6941094c0 0x7fe694199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:26:48.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe69259c700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe694103e30 0x7fe69419fc80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe68400e9a0 tx=0x7fe68400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:48.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe67bfff700 1 -- 172.21.15.84:0/3273252182 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68400ca10 con 0x7fe694103e30 2024-04-26T18:26:48.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe6990b0700 1 -- 172.21.15.84:0/3273252182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69419a350 con 0x7fe694103e30 2024-04-26T18:26:48.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe67bfff700 1 -- 172.21.15.84:0/3273252182 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe68400cbb0 con 0x7fe694103e30 2024-04-26T18:26:48.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe67bfff700 1 -- 172.21.15.84:0/3273252182 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe684018680 con 0x7fe694103e30 2024-04-26T18:26:48.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.844+0000 7fe6990b0700 1 -- 172.21.15.84:0/3273252182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6941a6610 con 0x7fe694103e30 2024-04-26T18:26:48.847 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.845+0000 7fe6990b0700 1 -- 172.21.15.84:0/3273252182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe69406a110 con 0x7fe694103e30 2024-04-26T18:26:48.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.847+0000 7fe67bfff700 1 -- 172.21.15.84:0/3273252182 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe684010430 con 0x7fe694103e30 2024-04-26T18:26:48.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.848+0000 7fe67bfff700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe67c0a3c40 0x7fe67c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:48.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.848+0000 7fe692d9d700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe67c0a3c40 0x7fe67c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:48.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.848+0000 7fe67bfff700 1 -- 172.21.15.84:0/3273252182 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fe6840d0880 con 0x7fe694103e30 2024-04-26T18:26:48.848 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.848+0000 7fe692d9d700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe67c0a3c40 0x7fe67c0a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe68800b2e0 tx=0x7fe68801a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:48.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:48.851+0000 7fe67bfff700 1 -- 172.21.15.84:0/3273252182 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe68409d230 con 0x7fe694103e30 2024-04-26T18:26:49.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.049+0000 7fe6990b0700 1 -- 172.21.15.84:0/3273252182 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe69419ae30 con 0x7fe694103e30 2024-04-26T18:26:49.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.052+0000 7fe67bfff700 1 -- 172.21.15.84:0/3273252182 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22727 (secure 0 0 0) 0x7fe68409cb30 con 0x7fe694103e30 2024-04-26T18:26:49.052 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:26:49.052 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":93,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:16:06.267694+0000","modified":"2024-04-26T18:26:07.290168+0000","last_up_change":"2024-04-26T18:23:19.906511+0000","last_in_change":"2024-04-26T18:22:59.541869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:19:31.174024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:24:16.533128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:24:19.742656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1076912-986e-4825-b1d1-5fa9d69bc07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6803","nonce":140050011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6805","nonce":140050011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6809","nonce":140050011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6807","nonce":140050011}]},"public_addr":"172.21.15.84:6803/140050011","cluster_addr":"172.21.15.84:6805/140050011","heartbeat_back_addr":"172.21.15.84:6809/140050011","heartbeat_front_addr":"172.21.15.84:6807/140050011","state":["exists","up"]},{"osd":1,"uuid":"fa93293d-9058-4804-ba4b-7e1e674b0fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6801","nonce":2512857657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6803","nonce":2512857657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6807","nonce":2512857657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6805","nonce":2512857657}]},"public_addr":"172.21.15.103:6801/2512857657","cluster_addr":"172.21.15.103:6803/2512857657","heartbeat_back_addr":"172.21.15.103:6807/2512857657","heartbeat_front_addr":"172.21.15.103:6805/2512857657","state":["exists","up"]},{"osd":2,"uuid":"bf78a44f-80a2-4cd7-974f-5abdf771e650","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6801","nonce":76340924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6803","nonce":76340924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6807","nonce":76340924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6805","nonce":76340924}]},"public_addr":"172.21.15.174:6801/76340924","cluster_addr":"172.21.15.174:6803/76340924","heartbeat_back_addr":"172.21.15.174:6807/76340924","heartbeat_front_addr":"172.21.15.174:6805/76340924","state":["exists","up"]},{"osd":3,"uuid":"55fd0035-bc76-4235-b8b5-5cf5ef770139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6811","nonce":2352789256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6813","nonce":2352789256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6817","nonce":2352789256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6815","nonce":2352789256}]},"public_addr":"172.21.15.84:6811/2352789256","cluster_addr":"172.21.15.84:6813/2352789256","heartbeat_back_addr":"172.21.15.84:6817/2352789256","heartbeat_front_addr":"172.21.15.84:6815/2352789256","state":["exists","up"]},{"osd":4,"uuid":"c278a0f3-e436-4d1c-a009-f9165ebf3c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6809","nonce":1619258252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6811","nonce":1619258252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6815","nonce":1619258252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6813","nonce":1619258252}]},"public_addr":"172.21.15.103:6809/1619258252","cluster_addr":"172.21.15.103:6811/1619258252","heartbeat_back_addr":"172.21.15.103:6815/1619258252","heartbeat_front_addr":"172.21.15.103:6813/1619258252","state":["exists","up"]},{"osd":5,"uuid":"6505be9b-c1b0-4d8b-944e-9920f90cb88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6809","nonce":3168291768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6811","nonce":3168291768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6815","nonce":3168291768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6813","nonce":3168291768}]},"public_addr":"172.21.15.174:6809/3168291768","cluster_addr":"172.21.15.174:6811/3168291768","heartbeat_back_addr":"172.21.15.174:6815/3168291768","heartbeat_front_addr":"172.21.15.174:6813/3168291768","state":["exists","up"]},{"osd":6,"uuid":"a037b66a-7488-469d-b73a-14cc0d35cb5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6819","nonce":886635297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6821","nonce":886635297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6825","nonce":886635297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6823","nonce":886635297}]},"public_addr":"172.21.15.84:6819/886635297","cluster_addr":"172.21.15.84:6821/886635297","heartbeat_back_addr":"172.21.15.84:6825/886635297","heartbeat_front_addr":"172.21.15.84:6823/886635297","state":["exists","up"]},{"osd":7,"uuid":"c425a13f-8019-4484-86a0-a34fa4d8de81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6817","nonce":1738278079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6819","nonce":1738278079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6823","nonce":1738278079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6821","nonce":1738278079}]},"public_addr":"172.21.15.103:6817/1738278079","cluster_addr":"172.21.15.103:6819/1738278079","heartbeat_back_addr":"172.21.15.103:6823/1738278079","heartbeat_front_addr":"172.21.15.103:6821/1738278079","state":["exists","up"]},{"osd":8,"uuid":"687e6c08-bfa4-440c-b3bc-90a5748989d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6817","nonce":38410548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6819","nonce":38410548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6823","nonce":38410548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6821","nonce":38410548}]},"public_addr":"172.21.15.174:6817/38410548","cluster_addr":"172.21.15.174:6819/38410548","heartbeat_back_addr":"172.21.15.174:6823/38410548","heartbeat_front_addr":"172.21.15.174:6821/38410548","state":["exists","up"]},{"osd":9,"uuid":"059eb22d-6dfb-4dd0-b192-84ed42ee1fbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6827","nonce":2926393927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6829","nonce":2926393927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6833","nonce":2926393927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6831","nonce":2926393927}]},"public_addr":"172.21.15.84:6827/2926393927","cluster_addr":"172.21.15.84:6829/2926393927","heartbeat_back_addr":"172.21.15.84:6833/2926393927","heartbeat_front_addr":"172.21.15.84:6831/2926393927","state":["exists","up"]},{"osd":10,"uuid":"da31e305-5220-4a87-a131-005b7c7f76b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6825","nonce":2705726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6827","nonce":2705726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6831","nonce":2705726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6829","nonce":2705726915}]},"public_addr":"172.21.15.103:6825/2705726915","cluster_addr":"172.21.15.103:6827/2705726915","heartbeat_back_addr":"172.21.15.103:6831/2705726915","heartbeat_front_addr":"172.21.15.103:6829/2705726915","state":["exists","up"]},{"osd":11,"uuid":"3fde6c8f-ee02-41ee-a34f-73072b4761e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6825","nonce":1668672128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6827","nonce":1668672128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6831","nonce":1668672128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6829","nonce":1668672128}]},"public_addr":"172.21.15.174:6825/1668672128","cluster_addr":"172.21.15.174:6827/1668672128","heartbeat_back_addr":"172.21.15.174:6831/1668672128","heartbeat_front_addr":"172.21.15.174:6829/1668672128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:39.099188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:03.327608+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:27.714108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:52.506015+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:17.631670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:42.702489+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:07.606697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:34.123214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:59.534658+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:25.357720+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:51.543106+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:23:18.208230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1604680842":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6800/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/538363975":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/952052990":"2024-04-27T18:17:05.083192+0000","172.21.15.84:0/1266942705":"2024-04-27T18:16:37.859669+0000","172.21.15.84:6800/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3970542149":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3607125049":"2024-04-27T18:17:05.083192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:26:49.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.054+0000 7fe6990b0700 1 -- 172.21.15.84:0/3273252182 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe67c0a3c40 msgr2=0x7fe67c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:49.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.054+0000 7fe6990b0700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe67c0a3c40 0x7fe67c0a6100 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe68800b2e0 tx=0x7fe68801a040 comp rx=0 tx=0).stop 2024-04-26T18:26:49.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.054+0000 7fe6990b0700 1 -- 172.21.15.84:0/3273252182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe694103e30 msgr2=0x7fe69419fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:49.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.054+0000 7fe6990b0700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe694103e30 0x7fe69419fc80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fe68400e9a0 tx=0x7fe68400ecb0 comp rx=0 tx=0).stop 2024-04-26T18:26:49.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.054+0000 7fe6990b0700 1 -- 172.21.15.84:0/3273252182 shutdown_connections 2024-04-26T18:26:49.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.054+0000 7fe6990b0700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe67c0a3c40 0x7fe67c0a6100 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:49.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.054+0000 7fe6990b0700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe6941094c0 0x7fe694199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:49.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.054+0000 7fe6990b0700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe694103470 0x7fe69419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:49.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.054+0000 7fe6990b0700 1 --2- 172.21.15.84:0/3273252182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe694103e30 0x7fe69419fc80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:49.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.054+0000 7fe6990b0700 1 -- 172.21.15.84:0/3273252182 >> 172.21.15.84:0/3273252182 conn(0x7fe69407a580 msgr2=0x7fe6941022b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:49.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.054+0000 7fe6990b0700 1 -- 172.21.15.84:0/3273252182 shutdown_connections 2024-04-26T18:26:49.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:49.054+0000 7fe6990b0700 1 -- 172.21.15.84:0/3273252182 wait complete. 2024-04-26T18:26:49.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:49 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3273252182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:26:49.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:49 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3273252182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:26:49.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:49 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3273252182' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:26:49.708 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:26:49.946 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:50.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:50 smithi084 ceph-mon[29141]: pgmap v375: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:50 smithi103 ceph-mon[32727]: pgmap v375: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:50.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:50 smithi174 ceph-mon[32705]: pgmap v375: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:51.615 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.611+0000 7f78d64c2700 1 -- 172.21.15.84:0/3451353627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78d0075650 msgr2=0x7f78d0075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:51.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.611+0000 7f78d64c2700 1 --2- 172.21.15.84:0/3451353627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78d0075650 0x7f78d0075a30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f78c0009990 tx=0x7f78c0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:51.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.611+0000 7f78d64c2700 1 -- 172.21.15.84:0/3451353627 shutdown_connections 2024-04-26T18:26:51.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.611+0000 7f78d64c2700 1 --2- 172.21.15.84:0/3451353627 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78d0075650 0x7f78d0075a30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:51.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.611+0000 7f78d64c2700 1 --2- 172.21.15.84:0/3451353627 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78d01145f0 0x7f78d01169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:51.616 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.611+0000 7f78d64c2700 1 --2- 172.21.15.84:0/3451353627 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78d0075f70 0x7f78d01140b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:51.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.611+0000 7f78d64c2700 1 -- 172.21.15.84:0/3451353627 >> 172.21.15.84:0/3451353627 conn(0x7f78d007a580 msgr2=0x7f78d007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:51.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.611+0000 7f78d64c2700 1 -- 172.21.15.84:0/3451353627 shutdown_connections 2024-04-26T18:26:51.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.612+0000 7f78d64c2700 1 -- 172.21.15.84:0/3451353627 wait complete. 2024-04-26T18:26:51.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.612+0000 7f78d64c2700 1 Processor -- start 2024-04-26T18:26:51.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.612+0000 7f78d64c2700 1 -- start start 2024-04-26T18:26:51.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78d64c2700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78d0075650 0x7f78d01a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78d64c2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78d0075f70 0x7f78d01a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78d64c2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78d01145f0 0x7f78d01a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78d64c2700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0119c50 con 0x7f78d0075f70 2024-04-26T18:26:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78d64c2700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0119ad0 con 0x7f78d0075650 2024-04-26T18:26:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78d64c2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0119dd0 con 0x7f78d01145f0 2024-04-26T18:26:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78cf7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78d0075f70 0x7f78d01a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78cf7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78d0075f70 0x7f78d01a0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46638/0 (socket says 172.21.15.84:46638) 2024-04-26T18:26:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78cf7fe700 1 -- 172.21.15.84:0/2156614112 learned_addr learned my addr 172.21.15.84:0/2156614112 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78d4a5f700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78d01145f0 0x7f78d01a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78cffff700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78d0075650 0x7f78d01a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:51.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78cf7fe700 1 -- 172.21.15.84:0/2156614112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78d01145f0 msgr2=0x7f78d01a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:51.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78cf7fe700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78d01145f0 0x7f78d01a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:51.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78cf7fe700 1 -- 172.21.15.84:0/2156614112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78d0075650 msgr2=0x7f78d01a0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:51.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78cf7fe700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78d0075650 0x7f78d01a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:51.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78cf7fe700 1 -- 172.21.15.84:0/2156614112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d01a5110 con 0x7f78d0075f70 2024-04-26T18:26:51.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78cf7fe700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78d0075f70 0x7f78d01a0640 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f78c400c9b0 tx=0x7f78c400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:51.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78cd7fa700 1 -- 172.21.15.84:0/2156614112 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4007b00 con 0x7f78d0075f70 2024-04-26T18:26:51.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78cd7fa700 1 -- 172.21.15.84:0/2156614112 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c4007ca0 con 0x7f78d0075f70 2024-04-26T18:26:51.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78d64c2700 1 -- 172.21.15.84:0/2156614112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d01a5400 con 0x7f78d0075f70 2024-04-26T18:26:51.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.613+0000 7f78cd7fa700 1 -- 172.21.15.84:0/2156614112 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4018730 con 0x7f78d0075f70 2024-04-26T18:26:51.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.614+0000 7f78d64c2700 1 -- 172.21.15.84:0/2156614112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d01acc80 con 0x7f78d0075f70 2024-04-26T18:26:51.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.614+0000 7f78d64c2700 1 -- 172.21.15.84:0/2156614112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d006a110 con 0x7f78d0075f70 2024-04-26T18:26:51.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.616+0000 7f78cd7fa700 1 -- 172.21.15.84:0/2156614112 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f78c400f450 con 0x7f78d0075f70 2024-04-26T18:26:51.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.616+0000 7f78cd7fa700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f78b80a3c40 0x7f78b80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:51.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.616+0000 7f78cffff700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f78b80a3c40 0x7f78b80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:51.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.616+0000 7f78cd7fa700 1 -- 172.21.15.84:0/2156614112 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f78c40d07e0 con 0x7f78d0075f70 2024-04-26T18:26:51.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.617+0000 7f78cffff700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f78b80a3c40 0x7f78b80a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f78c0005cd0 tx=0x7f78c0005c40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:51.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.620+0000 7f78cd7fa700 1 -- 172.21.15.84:0/2156614112 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f78c4014030 con 0x7f78d0075f70 2024-04-26T18:26:51.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.868+0000 7f78d64c2700 1 -- 172.21.15.84:0/2156614112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f78d01a1460 con 0x7f78d0075f70 2024-04-26T18:26:51.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.868+0000 7f78cd7fa700 1 -- 172.21.15.84:0/2156614112 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11936 (secure 0 0 0) 0x7f78d01a1460 con 0x7f78d0075f70 2024-04-26T18:26:51.869 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:26:51.869 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:24:41.425794+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24407,24418,24422,24430,24434]},"id":1}]} 2024-04-26T18:26:51.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.870+0000 7f78d64c2700 1 -- 172.21.15.84:0/2156614112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f78b80a3c40 msgr2=0x7f78b80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.870+0000 7f78d64c2700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f78b80a3c40 0x7f78b80a6100 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f78c0005cd0 tx=0x7f78c0005c40 comp rx=0 tx=0).stop 2024-04-26T18:26:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.871+0000 7f78d64c2700 1 -- 172.21.15.84:0/2156614112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78d0075f70 msgr2=0x7f78d01a0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.871+0000 7f78d64c2700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78d0075f70 0x7f78d01a0640 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f78c400c9b0 tx=0x7f78c400cd70 comp rx=0 tx=0).stop 2024-04-26T18:26:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.871+0000 7f78d64c2700 1 -- 172.21.15.84:0/2156614112 shutdown_connections 2024-04-26T18:26:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.871+0000 7f78d64c2700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f78b80a3c40 0x7f78b80a6100 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.871+0000 7f78d64c2700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78d01145f0 0x7f78d01a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.871+0000 7f78d64c2700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78d0075650 0x7f78d01a0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.871+0000 7f78d64c2700 1 --2- 172.21.15.84:0/2156614112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78d0075f70 0x7f78d01a0640 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.871+0000 7f78d64c2700 1 -- 172.21.15.84:0/2156614112 >> 172.21.15.84:0/2156614112 conn(0x7f78d007a580 msgr2=0x7f78d01133c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.871+0000 7f78d64c2700 1 -- 172.21.15.84:0/2156614112 shutdown_connections 2024-04-26T18:26:51.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:51.871+0000 7f78d64c2700 1 -- 172.21.15.84:0/2156614112 wait complete. 2024-04-26T18:26:51.872 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 28 2024-04-26T18:26:52.507 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:52 smithi084 ceph-mon[29141]: pgmap v376: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:52.507 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:52 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2156614112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:26:52.534 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T18:24:22.810992+0000', 'modified': '2024-04-26T18:24:41.425794+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24434, 'mds_1': 24430, 'mds_2': 24422, 'mds_3': 24418, 'mds_4': 24407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14598': {'gid': 14598, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6839/96876655', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6838', 'nonce': 96876655}, {'type': 'v1', 'addr': '172.21.15.84:6839', 'nonce': 96876655}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14610': {'gid': 14610, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6841/2504248516', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6840', 'nonce': 2504248516}, {'type': 'v1', 'addr': '172.21.15.84:6841', 'nonce': 2504248516}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24385': {'gid': 24385, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/280377862', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 280377862}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 280377862}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6833/3711970972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6832', 'nonce': 3711970972}, {'type': 'v1', 'addr': '172.21.15.103:6833', 'nonce': 3711970972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24406': {'gid': 24406, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/1310802824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 1310802824}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 1310802824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.103:6835/2529642104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6834', 'nonce': 2529642104}, {'type': 'v1', 'addr': '172.21.15.103:6835', 'nonce': 2529642104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.174:6837/1532309720', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 1532309720}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 1532309720}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.103:6837/2271231481', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6836', 'nonce': 2271231481}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 2271231481}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.174:6839/518809558', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 518809558}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 518809558}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.103:6839/2465442393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 2465442393}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 2465442393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24434, 'qdb_cluster': [24407, 24418, 24422, 24430, 24434]} 2024-04-26T18:26:52.534 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-26T18:26:52.534 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.e status 2024-04-26T18:26:52.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:52 smithi103 ceph-mon[32727]: pgmap v376: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:52.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:52 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2156614112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:26:52.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:52 smithi174 ceph-mon[32705]: pgmap v376: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:52.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:52 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2156614112' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:26:52.773 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:54.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.471+0000 7f32eb813700 1 -- 172.21.15.84:0/758188820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e4102320 msgr2=0x7f32e41027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:54.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.471+0000 7f32eb813700 1 --2- 172.21.15.84:0/758188820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e4102320 0x7f32e41027a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009990 tx=0x7f32e0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.471+0000 7f32eb813700 1 -- 172.21.15.84:0/758188820 shutdown_connections 2024-04-26T18:26:54.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.471+0000 7f32eb813700 1 --2- 172.21.15.84:0/758188820 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32e4102ce0 0x7f32e410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.471+0000 7f32eb813700 1 --2- 172.21.15.84:0/758188820 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32e4108390 0x7f32e4108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.471+0000 7f32eb813700 1 --2- 172.21.15.84:0/758188820 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e4102320 0x7f32e41027a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.471+0000 7f32eb813700 1 -- 172.21.15.84:0/758188820 >> 172.21.15.84:0/758188820 conn(0x7f32e40792d0 msgr2=0x7f32e40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:54.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.471+0000 7f32eb813700 1 -- 172.21.15.84:0/758188820 shutdown_connections 2024-04-26T18:26:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.471+0000 7f32eb813700 1 -- 172.21.15.84:0/758188820 wait complete. 2024-04-26T18:26:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.471+0000 7f32eb813700 1 Processor -- start 2024-04-26T18:26:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.471+0000 7f32eb813700 1 -- start start 2024-04-26T18:26:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.471+0000 7f32eb813700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32e4102320 0x7f32e419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32eb813700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32e4102ce0 0x7f32e419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32eb813700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e4108390 0x7f32e4199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32eb813700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f32e4112530 con 0x7f32e4108390 2024-04-26T18:26:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32eb813700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41123b0 con 0x7f32e4102ce0 2024-04-26T18:26:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32eb813700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f32e41126b0 con 0x7f32e4102320 2024-04-26T18:26:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32e9db0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e4108390 0x7f32e4199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32e9db0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e4108390 0x7f32e4199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46658/0 (socket says 172.21.15.84:46658) 2024-04-26T18:26:54.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32e9db0700 1 -- 172.21.15.84:0/90833288 learned_addr learned my addr 172.21.15.84:0/90833288 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:54.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32e8dae700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32e4102ce0 0x7f32e419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32e95af700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32e4102320 0x7f32e419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32e9db0700 1 -- 172.21.15.84:0/90833288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32e4102320 msgr2=0x7f32e419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:54.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32e9db0700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32e4102320 0x7f32e419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32e9db0700 1 -- 172.21.15.84:0/90833288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32e4102ce0 msgr2=0x7f32e419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:54.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32e9db0700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32e4102ce0 0x7f32e419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32e9db0700 1 -- 172.21.15.84:0/90833288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32e419a0a0 con 0x7f32e4108390 2024-04-26T18:26:54.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32e95af700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32e4102320 0x7f32e419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:54.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32e9db0700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e4108390 0x7f32e4199940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f32dc007ee0 tx=0x7f32dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:54.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32da7fc700 1 -- 172.21.15.84:0/90833288 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc00dbd0 con 0x7f32e4108390 2024-04-26T18:26:54.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32da7fc700 1 -- 172.21.15.84:0/90833288 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32dc00dd70 con 0x7f32e4108390 2024-04-26T18:26:54.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32eb813700 1 -- 172.21.15.84:0/90833288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32e419a390 con 0x7f32e4108390 2024-04-26T18:26:54.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32da7fc700 1 -- 172.21.15.84:0/90833288 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc005990 con 0x7f32e4108390 2024-04-26T18:26:54.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.472+0000 7f32eb813700 1 -- 172.21.15.84:0/90833288 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32e41a6650 con 0x7f32e4108390 2024-04-26T18:26:54.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.474+0000 7f32da7fc700 1 -- 172.21.15.84:0/90833288 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f32dc01a020 con 0x7f32e4108390 2024-04-26T18:26:54.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.475+0000 7f32da7fc700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f32d00a3c40 0x7f32d00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.475+0000 7f32e95af700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f32d00a3c40 0x7f32d00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.475+0000 7f32da7fc700 1 -- 172.21.15.84:0/90833288 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f32dc0cfc00 con 0x7f32e4108390 2024-04-26T18:26:54.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.475+0000 7f32e95af700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f32d00a3c40 0x7f32d00a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f32e419aa80 tx=0x7f32d4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:54.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.476+0000 7f32f60a5b80 1 Processor -- start 2024-04-26T18:26:54.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.476+0000 7f32f60a5b80 1 -- start start 2024-04-26T18:26:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.476+0000 7f32f60a5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd749ad0 0x555cbd749eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.476+0000 7f32f60a5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd74a3f0 0x555cbd743cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.476+0000 7f32f60a5b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 0x555cbd744710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.476+0000 7f32f60a5b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555cbd6966f0 con 0x555cbd7442d0 2024-04-26T18:26:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.476+0000 7f32f60a5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555cbd635590 con 0x555cbd749ad0 2024-04-26T18:26:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.476+0000 7f32f60a5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555cbd6926b0 con 0x555cbd74a3f0 2024-04-26T18:26:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.476+0000 7f32e9db0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 0x555cbd744710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.476+0000 7f32e9db0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 0x555cbd744710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46670/0 (socket says 172.21.15.84:46670) 2024-04-26T18:26:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.476+0000 7f32e9db0700 1 -- 172.21.15.84:0/2091779412 learned_addr learned my addr 172.21.15.84:0/2091779412 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.476+0000 7f32e95af700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd749ad0 0x555cbd749eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.477+0000 7f32e8dae700 1 --2- 172.21.15.84:0/2091779412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd74a3f0 0x555cbd743cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.477+0000 7f32e9db0700 1 -- 172.21.15.84:0/2091779412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd74a3f0 msgr2=0x555cbd743cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.477+0000 7f32e9db0700 1 --2- 172.21.15.84:0/2091779412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd74a3f0 0x555cbd743cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.477+0000 7f32e9db0700 1 -- 172.21.15.84:0/2091779412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd749ad0 msgr2=0x555cbd749eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.477+0000 7f32e9db0700 1 --2- 172.21.15.84:0/2091779412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd749ad0 0x555cbd749eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.477+0000 7f32e9db0700 1 -- 172.21.15.84:0/2091779412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cbd5e3880 con 0x555cbd7442d0 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.477+0000 7f32e95af700 1 --2- 172.21.15.84:0/2091779412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd749ad0 0x555cbd749eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.477+0000 7f32e9db0700 1 --2- 172.21.15.84:0/2091779412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 0x555cbd744710 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f32dc098310 tx=0x7f32dc09bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789a2e9bc26d110c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.477+0000 7f32cf7fe700 1 -- 172.21.15.84:0/2091779412 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0a0260 con 0x555cbd7442d0 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.477+0000 7f32cf7fe700 1 -- 172.21.15.84:0/2091779412 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32dc0a4b20 con 0x555cbd7442d0 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.477+0000 7f32cf7fe700 1 -- 172.21.15.84:0/2091779412 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32dc0a4690 con 0x555cbd7442d0 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.478+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2091779412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 msgr2=0x555cbd744710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.478+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2091779412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 0x555cbd744710 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f32dc098310 tx=0x7f32dc09bd90 comp rx=0 tx=0).stop 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.478+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2091779412 shutdown_connections 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.478+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2091779412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 0x555cbd744710 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.478+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2091779412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd749ad0 0x555cbd749eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.478+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2091779412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd74a3f0 0x555cbd743cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.478+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2091779412 >> 172.21.15.84:0/2091779412 conn(0x555cbd670cf0 msgr2=0x555cbd73f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.478+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2091779412 shutdown_connections 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.478+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2091779412 wait complete. 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.478+0000 7f32f60a5b80 1 -- start start 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.478+0000 7f32f60a5b80 1 Processor -- start 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32f60a5b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 0x555cbd7500f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32f60a5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd749ad0 0x555cbd74d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32f60a5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd74a3f0 0x555cbd74d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32f60a5b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555cbd750f10 con 0x555cbd7442d0 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32f60a5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555cbd5fc900 con 0x555cbd749ad0 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32f60a5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555cbd751090 con 0x555cbd74a3f0 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32f60a5b80 10 client.0 _ll_register_callbacks cb 0x555cbd62a530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:54.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32e95af700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 0x555cbd7500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32e95af700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 0x555cbd7500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46672/0 (socket says 172.21.15.84:46672) 2024-04-26T18:26:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32e95af700 1 -- 172.21.15.84:0/2788252378 learned_addr learned my addr 172.21.15.84:0/2788252378 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32e8dae700 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd749ad0 0x555cbd74d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32e9db0700 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd74a3f0 0x555cbd74d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32e95af700 1 -- 172.21.15.84:0/2788252378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd74a3f0 msgr2=0x555cbd74d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32e95af700 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd74a3f0 0x555cbd74d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32e95af700 1 -- 172.21.15.84:0/2788252378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd749ad0 msgr2=0x555cbd74d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32e95af700 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd749ad0 0x555cbd74d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32e95af700 1 -- 172.21.15.84:0/2788252378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cbd74df70 con 0x555cbd7442d0 2024-04-26T18:26:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32e9db0700 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd74a3f0 0x555cbd74d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32e95af700 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 0x555cbd7500f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f32d400ce60 tx=0x7f32d4004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d62d55d91c126f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:26:54.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32ccff9700 1 -- 172.21.15.84:0/2788252378 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32d4023020 con 0x555cbd7442d0 2024-04-26T18:26:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32ccff9700 1 -- 172.21.15.84:0/2788252378 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f32d401f3f0 con 0x555cbd7442d0 2024-04-26T18:26:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.479+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2788252378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32dc0a4b20 con 0x555cbd7442d0 2024-04-26T18:26:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.480+0000 7f32ccff9700 1 -- 172.21.15.84:0/2788252378 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32d402e050 con 0x555cbd7442d0 2024-04-26T18:26:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.480+0000 7f32f60a5b80 10 client.14988 fetch_fsmap learned FSMap version 28 2024-04-26T18:26:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.480+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2788252378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555cbd75b990 con 0x555cbd7442d0 2024-04-26T18:26:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.480+0000 7f32ccff9700 1 -- 172.21.15.84:0/2788252378 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32dc0a4b20 con 0x555cbd7442d0 2024-04-26T18:26:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.480+0000 7f32ccff9700 1 -- 172.21.15.84:0/2788252378 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f32d4011460 con 0x555cbd7442d0 2024-04-26T18:26:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.480+0000 7f32f60a5b80 10 client.14988 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:26:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.480+0000 7f32f60a5b80 10 client.14988 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T18:26:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.480+0000 7f32f60a5b80 20 client.14988 populate_metadata read hostname 'smithi084' 2024-04-26T18:26:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.480+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x555cbd75e030 0x555cbd75f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.480+0000 7f32f60a5b80 4 client.14988 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:26:54.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.480+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2788252378 --> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cbd6a40d0 con 0x555cbd75e030 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.480+0000 7f32e8dae700 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x555cbd75e030 0x555cbd75f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.481+0000 7f32e8dae700 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x555cbd75e030 0x555cbd75f4a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=419bb9be271134d1 server_cookie=9688624c4dbbf591 in_seq=0 out_seq=0 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.481+0000 7f32ccff9700 10 client.14988 ms_handle_connect on v2:172.21.15.103:6834/2529642104 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.481+0000 7f32ccff9700 1 -- 172.21.15.84:0/2788252378 <== mds.4 v2:172.21.15.103:6834/2529642104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cbd6a40d0 con 0x555cbd75e030 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.481+0000 7f32ccff9700 10 client.14988 handle_command_reply: tid=0 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.482+0000 7f32f60a5b80 1 client.14988 shutdown 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.482+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2788252378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 msgr2=0x555cbd7500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.482+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 0x555cbd7500f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f32d400ce60 tx=0x7f32d4004860 comp rx=0 tx=0).stop 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.482+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2788252378 shutdown_connections 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.482+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x555cbd75e030 0x555cbd75f4a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.482+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd7442d0 0x555cbd7500f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.482+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd74a3f0 0x555cbd74d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.482+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2788252378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd749ad0 0x555cbd74d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.482+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2788252378 >> 172.21.15.84:0/2788252378 conn(0x555cbd670cf0 msgr2=0x555cbd73e330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:54.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.482+0000 7f32ccff9700 0 client.14988 ms_handle_reset on v2:172.21.15.103:6834/2529642104 2024-04-26T18:26:54.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.482+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2788252378 shutdown_connections 2024-04-26T18:26:54.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.482+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2788252378 wait complete. 2024-04-26T18:26:54.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.482+0000 7f32f60a5b80 20 client.14988 trim_cache size 0 max 16384 2024-04-26T18:26:54.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.502+0000 7f32f60a5b80 1 Processor -- start 2024-04-26T18:26:54.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.502+0000 7f32f60a5b80 1 -- start start 2024-04-26T18:26:54.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.502+0000 7f32f60a5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd7442d0 0x555cbd743cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.502+0000 7f32f60a5b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 0x555cbd75f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.502+0000 7f32f60a5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd74a3f0 0x555cbd74a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.502+0000 7f32f60a5b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555cbd75d7e0 con 0x555cbd749ad0 2024-04-26T18:26:54.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.502+0000 7f32f60a5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555cbd75deb0 con 0x555cbd74a3f0 2024-04-26T18:26:54.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.502+0000 7f32f60a5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555cbd75e580 con 0x555cbd7442d0 2024-04-26T18:26:54.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.502+0000 7f32e8dae700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 0x555cbd75f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.502+0000 7f32e8dae700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 0x555cbd75f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46676/0 (socket says 172.21.15.84:46676) 2024-04-26T18:26:54.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.502+0000 7f32e8dae700 1 -- 172.21.15.84:0/2947407799 learned_addr learned my addr 172.21.15.84:0/2947407799 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:54.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.503+0000 7f32e9db0700 1 --2- 172.21.15.84:0/2947407799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd74a3f0 0x555cbd74a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.503+0000 7f32e95af700 1 --2- 172.21.15.84:0/2947407799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd7442d0 0x555cbd743cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.503+0000 7f32e8dae700 1 -- 172.21.15.84:0/2947407799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd7442d0 msgr2=0x555cbd743cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:54.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.503+0000 7f32e8dae700 1 --2- 172.21.15.84:0/2947407799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd7442d0 0x555cbd743cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.503+0000 7f32e8dae700 1 -- 172.21.15.84:0/2947407799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd74a3f0 msgr2=0x555cbd74a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:54.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.503+0000 7f32e8dae700 1 --2- 172.21.15.84:0/2947407799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd74a3f0 0x555cbd74a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.503+0000 7f32e8dae700 1 -- 172.21.15.84:0/2947407799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cbd74b960 con 0x555cbd749ad0 2024-04-26T18:26:54.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.503+0000 7f32e9db0700 1 --2- 172.21.15.84:0/2947407799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd74a3f0 0x555cbd74a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:54.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.503+0000 7f32e8dae700 1 --2- 172.21.15.84:0/2947407799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 0x555cbd75f4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009990 tx=0x7f32e0005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b4321025b5f7bf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:54.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.503+0000 7f32ceffd700 1 -- 172.21.15.84:0/2947407799 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0003df0 con 0x555cbd749ad0 2024-04-26T18:26:54.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.503+0000 7f32ceffd700 1 -- 172.21.15.84:0/2947407799 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e0005570 con 0x555cbd749ad0 2024-04-26T18:26:54.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.503+0000 7f32ceffd700 1 -- 172.21.15.84:0/2947407799 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e000f460 con 0x555cbd749ad0 2024-04-26T18:26:54.507 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:54 smithi084 ceph-mon[29141]: pgmap v377: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:54.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.504+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2947407799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 msgr2=0x555cbd75f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:54.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.504+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2947407799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 0x555cbd75f4a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009990 tx=0x7f32e0005960 comp rx=0 tx=0).stop 2024-04-26T18:26:54.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.504+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2947407799 shutdown_connections 2024-04-26T18:26:54.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.504+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2947407799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd74a3f0 0x555cbd74a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.504+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2947407799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd7442d0 0x555cbd743cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.504+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2947407799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 0x555cbd75f4a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.504+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2947407799 >> 172.21.15.84:0/2947407799 conn(0x555cbd670cf0 msgr2=0x555cbd747620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:54.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.504+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2947407799 shutdown_connections 2024-04-26T18:26:54.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.504+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2947407799 wait complete. 2024-04-26T18:26:54.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.504+0000 7f32f60a5b80 1 -- start start 2024-04-26T18:26:54.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.504+0000 7f32f60a5b80 1 Processor -- start 2024-04-26T18:26:54.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32f60a5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd7442d0 0x555cbd74bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32f60a5b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 0x555cbd74c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32f60a5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd74a3f0 0x555cbd74c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32f60a5b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555cbd75f320 con 0x555cbd749ad0 2024-04-26T18:26:54.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32f60a5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555cbd75ec50 con 0x555cbd74a3f0 2024-04-26T18:26:54.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32f60a5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555cbd75f9f0 con 0x555cbd7442d0 2024-04-26T18:26:54.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32f60a5b80 10 client.0 _ll_register_callbacks cb 0x555cbd62a530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:54.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32e8dae700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 0x555cbd74c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32e8dae700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 0x555cbd74c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46688/0 (socket says 172.21.15.84:46688) 2024-04-26T18:26:54.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32e8dae700 1 -- 172.21.15.84:0/2870611771 learned_addr learned my addr 172.21.15.84:0/2870611771 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:54.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32e9db0700 1 --2- 172.21.15.84:0/2870611771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd74a3f0 0x555cbd74c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32e95af700 1 --2- 172.21.15.84:0/2870611771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd7442d0 0x555cbd74bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32e8dae700 1 -- 172.21.15.84:0/2870611771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd7442d0 msgr2=0x555cbd74bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:54.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32e8dae700 1 --2- 172.21.15.84:0/2870611771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd7442d0 0x555cbd74bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32e8dae700 1 -- 172.21.15.84:0/2870611771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd74a3f0 msgr2=0x555cbd74c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32e8dae700 1 --2- 172.21.15.84:0/2870611771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd74a3f0 0x555cbd74c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32e8dae700 1 -- 172.21.15.84:0/2870611771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cbd7c8260 con 0x555cbd749ad0 2024-04-26T18:26:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32e8dae700 1 --2- 172.21.15.84:0/2870611771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 0x555cbd74c220 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f32e002c000 tx=0x7f32e000b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5e566a858077f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.505+0000 7f32ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:26:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32ccff9700 1 -- 172.21.15.84:0/2870611771 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32e0036060 con 0x555cbd749ad0 2024-04-26T18:26:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32ccff9700 1 -- 172.21.15.84:0/2870611771 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f32e000bdc0 con 0x555cbd749ad0 2024-04-26T18:26:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2870611771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32e0005570 con 0x555cbd749ad0 2024-04-26T18:26:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32ccff9700 1 -- 172.21.15.84:0/2870611771 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f32e00059d0 con 0x555cbd749ad0 2024-04-26T18:26:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32f60a5b80 10 client.14994 fetch_fsmap learned FSMap version 28 2024-04-26T18:26:54.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2870611771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555cbd7c9540 con 0x555cbd749ad0 2024-04-26T18:26:54.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32ccff9700 1 -- 172.21.15.84:0/2870611771 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32e0005570 con 0x555cbd749ad0 2024-04-26T18:26:54.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32ccff9700 1 -- 172.21.15.84:0/2870611771 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f32e000fb70 con 0x555cbd749ad0 2024-04-26T18:26:54.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32f60a5b80 10 client.14994 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:26:54.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32f60a5b80 10 client.14994 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T18:26:54.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32f60a5b80 20 client.14994 populate_metadata read hostname 'smithi084' 2024-04-26T18:26:54.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2870611771 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x555cbd7cdba0 0x555cbd7cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:54.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32f60a5b80 4 client.14994 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:26:54.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.506+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2870611771 --> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] -- command(tid 0: {"prefix": "status"}) -- 0x555cbd69e170 con 0x555cbd7cdba0 2024-04-26T18:26:54.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32e95af700 1 --2- 172.21.15.84:0/2870611771 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x555cbd7cdba0 0x555cbd7cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:54.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32e95af700 1 --2- 172.21.15.84:0/2870611771 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x555cbd7cdba0 0x555cbd7cc8b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=71b3ed0e7052ae91 server_cookie=b49761caaacfaa77 in_seq=0 out_seq=0 2024-04-26T18:26:54.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32ccff9700 10 client.14994 ms_handle_connect on v2:172.21.15.103:6834/2529642104 2024-04-26T18:26:54.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32ccff9700 1 -- 172.21.15.84:0/2870611771 <== mds.4 v2:172.21.15.103:6834/2529642104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555cbd69e170 con 0x555cbd7cdba0 2024-04-26T18:26:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32ccff9700 10 client.14994 handle_command_reply: tid=0 2024-04-26T18:26:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32f60a5b80 1 client.14994 shutdown 2024-04-26T18:26:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2870611771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 msgr2=0x555cbd74c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2870611771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 0x555cbd74c220 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f32e002c000 tx=0x7f32e000b6f0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2870611771 shutdown_connections 2024-04-26T18:26:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2870611771 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x555cbd7cdba0 0x555cbd7cc8b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2870611771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555cbd74a3f0 0x555cbd74c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2870611771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555cbd7442d0 0x555cbd74bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32ccff9700 0 client.14994 ms_handle_reset on v2:172.21.15.103:6834/2529642104 2024-04-26T18:26:54.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32f60a5b80 1 --2- 172.21.15.84:0/2870611771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555cbd749ad0 0x555cbd74c220 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.507+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2870611771 >> 172.21.15.84:0/2870611771 conn(0x555cbd670cf0 msgr2=0x555cbd74a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2870611771 shutdown_connections 2024-04-26T18:26:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32f60a5b80 1 -- 172.21.15.84:0/2870611771 wait complete. 2024-04-26T18:26:54.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32f60a5b80 20 client.14994 trim_cache size 0 max 16384 2024-04-26T18:26:54.518 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:26:54.518 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:26:54.518 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 4, 2024-04-26T18:26:54.519 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24407, 2024-04-26T18:26:54.519 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:26:54.519 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:26:54.519 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:26:54.519 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 136.21508704600001, 2024-04-26T18:26:54.519 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 28, 2024-04-26T18:26:54.519 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 93, 2024-04-26T18:26:54.519 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 89, 2024-04-26T18:26:54.519 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 186.887007342 2024-04-26T18:26:54.519 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:26:54.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32eb813700 1 -- 172.21.15.84:0/90833288 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f32d00a3c40 msgr2=0x7f32d00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:54.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32eb813700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f32d00a3c40 0x7f32d00a6100 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f32e419aa80 tx=0x7f32d4008040 comp rx=0 tx=0).stop 2024-04-26T18:26:54.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32eb813700 1 -- 172.21.15.84:0/90833288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e4108390 msgr2=0x7f32e4199940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:54.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32eb813700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e4108390 0x7f32e4199940 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f32dc007ee0 tx=0x7f32dc00d3b0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32eb813700 1 -- 172.21.15.84:0/90833288 shutdown_connections 2024-04-26T18:26:54.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32eb813700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f32d00a3c40 0x7f32d00a6100 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32eb813700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f32e4108390 0x7f32e4199940 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32eb813700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f32e4102320 0x7f32e419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32eb813700 1 --2- 172.21.15.84:0/90833288 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f32e4102ce0 0x7f32e419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:54.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32eb813700 1 -- 172.21.15.84:0/90833288 >> 172.21.15.84:0/90833288 conn(0x7f32e40792d0 msgr2=0x7f32e4101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:54.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.508+0000 7f32eb813700 1 -- 172.21.15.84:0/90833288 shutdown_connections 2024-04-26T18:26:54.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:54.509+0000 7f32eb813700 1 -- 172.21.15.84:0/90833288 wait complete. 2024-04-26T18:26:54.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:54 smithi103 ceph-mon[32727]: pgmap v377: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:54.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:54 smithi174 ceph-mon[32705]: pgmap v377: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:26:55.196 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-26T18:26:55.439 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:56.534 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:56 smithi084 ceph-mon[29141]: pgmap v378: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:56.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:56 smithi103 ceph-mon[32727]: pgmap v378: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:56.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:56 smithi174 ceph-mon[32705]: pgmap v378: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.081+0000 7f546fd32700 1 -- 172.21.15.84:0/450094115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54681034c0 msgr2=0x7f5468103940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.081+0000 7f546fd32700 1 --2- 172.21.15.84:0/450094115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54681034c0 0x7f5468103940 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f54580099c0 tx=0x7f5458009cd0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.081+0000 7f546fd32700 1 -- 172.21.15.84:0/450094115 shutdown_connections 2024-04-26T18:26:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.081+0000 7f546fd32700 1 --2- 172.21.15.84:0/450094115 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5468103e80 0x7f5468110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.081+0000 7f546fd32700 1 --2- 172.21.15.84:0/450094115 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468109560 0x7f5468109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.081+0000 7f546fd32700 1 --2- 172.21.15.84:0/450094115 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f54681034c0 0x7f5468103940 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.081+0000 7f546fd32700 1 -- 172.21.15.84:0/450094115 >> 172.21.15.84:0/450094115 conn(0x7f546807a5d0 msgr2=0x7f546807a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.081+0000 7f546fd32700 1 -- 172.21.15.84:0/450094115 shutdown_connections 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.081+0000 7f546fd32700 1 -- 172.21.15.84:0/450094115 wait complete. 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.082+0000 7f546fd32700 1 Processor -- start 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.082+0000 7f546fd32700 1 -- start start 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.082+0000 7f546fd32700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54681034c0 0x7f5468075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.082+0000 7f546fd32700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468103e80 0x7f5468075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.082+0000 7f546fd32700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5468109560 0x7f54680761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.082+0000 7f546fd32700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f54681136e0 con 0x7f5468109560 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.082+0000 7f546fd32700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5468113560 con 0x7f54681034c0 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.082+0000 7f546fd32700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5468113860 con 0x7f5468103e80 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.082+0000 7f546e2cf700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5468109560 0x7f54680761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.082+0000 7f546e2cf700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5468109560 0x7f54680761a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44544/0 (socket says 172.21.15.84:44544) 2024-04-26T18:26:57.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.082+0000 7f546e2cf700 1 -- 172.21.15.84:0/3790918251 learned_addr learned my addr 172.21.15.84:0/3790918251 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.082+0000 7f546dace700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54681034c0 0x7f5468075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.082+0000 7f546d2cd700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468103e80 0x7f5468075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.083+0000 7f546e2cf700 1 -- 172.21.15.84:0/3790918251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468103e80 msgr2=0x7f5468075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.083+0000 7f546e2cf700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468103e80 0x7f5468075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.083+0000 7f546e2cf700 1 -- 172.21.15.84:0/3790918251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54681034c0 msgr2=0x7f5468075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.083+0000 7f546e2cf700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54681034c0 0x7f5468075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.083+0000 7f546e2cf700 1 -- 172.21.15.84:0/3790918251 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54681a8060 con 0x7f5468109560 2024-04-26T18:26:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.083+0000 7f546e2cf700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5468109560 0x7f54680761a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f546000b970 tx=0x7f546000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.083+0000 7f545effd700 1 -- 172.21.15.84:0/3790918251 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546000d860 con 0x7f5468109560 2024-04-26T18:26:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.083+0000 7f546fd32700 1 -- 172.21.15.84:0/3790918251 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54681a8350 con 0x7f5468109560 2024-04-26T18:26:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.083+0000 7f545effd700 1 -- 172.21.15.84:0/3790918251 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f546000e430 con 0x7f5468109560 2024-04-26T18:26:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.083+0000 7f545effd700 1 -- 172.21.15.84:0/3790918251 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5460017660 con 0x7f5468109560 2024-04-26T18:26:57.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.083+0000 7f546fd32700 1 -- 172.21.15.84:0/3790918251 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54681a8860 con 0x7f5468109560 2024-04-26T18:26:57.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.085+0000 7f545effd700 1 -- 172.21.15.84:0/3790918251 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f546000da00 con 0x7f5468109560 2024-04-26T18:26:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.086+0000 7f545effd700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f54540a3c40 0x7f54540a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.086+0000 7f546dace700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f54540a3c40 0x7f54540a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.086+0000 7f545effd700 1 -- 172.21.15.84:0/3790918251 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f54600cff90 con 0x7f5468109560 2024-04-26T18:26:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.086+0000 7f546dace700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f54540a3c40 0x7f54540a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5464004630 tx=0x7f546400b3c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.086+0000 7f547a5c4b80 1 Processor -- start 2024-04-26T18:26:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f547a5c4b80 1 -- start start 2024-04-26T18:26:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f547a5c4b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 0x563b8138fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f547a5c4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b81389d50 0x563b8138a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f547a5c4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b8138a6f0 0x563b81396770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f547a5c4b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563b812dc7e0 con 0x563b8138fa20 2024-04-26T18:26:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f547a5c4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563b812428e0 con 0x563b8138a6f0 2024-04-26T18:26:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f547a5c4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b812d87a0 con 0x563b81389d50 2024-04-26T18:26:57.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f546dace700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 0x563b8138fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f546dace700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 0x563b8138fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44548/0 (socket says 172.21.15.84:44548) 2024-04-26T18:26:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f546dace700 1 -- 172.21.15.84:0/1283863395 learned_addr learned my addr 172.21.15.84:0/1283863395 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f546e2cf700 1 --2- 172.21.15.84:0/1283863395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b8138a6f0 0x563b81396770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f546d2cd700 1 --2- 172.21.15.84:0/1283863395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b81389d50 0x563b8138a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f546dace700 1 -- 172.21.15.84:0/1283863395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b81389d50 msgr2=0x563b8138a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f546dace700 1 --2- 172.21.15.84:0/1283863395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b81389d50 0x563b8138a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f546dace700 1 -- 172.21.15.84:0/1283863395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b8138a6f0 msgr2=0x563b81396770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f546dace700 1 --2- 172.21.15.84:0/1283863395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b8138a6f0 0x563b81396770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f546dace700 1 -- 172.21.15.84:0/1283863395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b81229880 con 0x563b8138fa20 2024-04-26T18:26:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f546dace700 1 --2- 172.21.15.84:0/1283863395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 0x563b8138fe00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5464003b10 tx=0x7f5464013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29678449f09ec929 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.087+0000 7f5453fff700 1 -- 172.21.15.84:0/1283863395 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5464011ad0 con 0x563b8138fa20 2024-04-26T18:26:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.088+0000 7f5453fff700 1 -- 172.21.15.84:0/1283863395 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5464011c70 con 0x563b8138fa20 2024-04-26T18:26:57.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.088+0000 7f5453fff700 1 -- 172.21.15.84:0/1283863395 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f546401c3f0 con 0x563b8138fa20 2024-04-26T18:26:57.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.088+0000 7f547a5c4b80 1 -- 172.21.15.84:0/1283863395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 msgr2=0x563b8138fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:57.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.088+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/1283863395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 0x563b8138fe00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5464003b10 tx=0x7f5464013000 comp rx=0 tx=0).stop 2024-04-26T18:26:57.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.088+0000 7f547a5c4b80 1 -- 172.21.15.84:0/1283863395 shutdown_connections 2024-04-26T18:26:57.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.088+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/1283863395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 0x563b8138fe00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.088+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/1283863395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b8138a6f0 0x563b81396770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.088+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/1283863395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b81389d50 0x563b8138a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.088+0000 7f547a5c4b80 1 -- 172.21.15.84:0/1283863395 >> 172.21.15.84:0/1283863395 conn(0x563b812b8570 msgr2=0x563b81384900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.088+0000 7f547a5c4b80 1 -- 172.21.15.84:0/1283863395 shutdown_connections 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.088+0000 7f547a5c4b80 1 -- 172.21.15.84:0/1283863395 wait complete. 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.089+0000 7f547a5c4b80 1 -- start start 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.089+0000 7f547a5c4b80 1 Processor -- start 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.089+0000 7f547a5c4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b81389d50 0x563b813962b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.089+0000 7f547a5c4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b8138a6f0 0x563b81393650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.089+0000 7f547a5c4b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 0x563b81393b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.089+0000 7f547a5c4b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563b81396cb0 con 0x563b8138fa20 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.089+0000 7f547a5c4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563b81296a30 con 0x563b81389d50 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.089+0000 7f547a5c4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b81396e30 con 0x563b8138a6f0 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.089+0000 7f547a5c4b80 10 client.0 _ll_register_callbacks cb 0x563b81274900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.089+0000 7f546e2cf700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 0x563b81393b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.089+0000 7f546e2cf700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 0x563b81393b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44560/0 (socket says 172.21.15.84:44560) 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.089+0000 7f546e2cf700 1 -- 172.21.15.84:0/3650190237 learned_addr learned my addr 172.21.15.84:0/3650190237 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.089+0000 7f546dace700 1 --2- 172.21.15.84:0/3650190237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b81389d50 0x563b813962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f546d2cd700 1 --2- 172.21.15.84:0/3650190237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b8138a6f0 0x563b81393650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f546e2cf700 1 -- 172.21.15.84:0/3650190237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b8138a6f0 msgr2=0x563b81393650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f546e2cf700 1 --2- 172.21.15.84:0/3650190237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b8138a6f0 0x563b81393650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f546e2cf700 1 -- 172.21.15.84:0/3650190237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b81389d50 msgr2=0x563b813962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f546e2cf700 1 --2- 172.21.15.84:0/3650190237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b81389d50 0x563b813962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f546e2cf700 1 -- 172.21.15.84:0/3650190237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b8129a430 con 0x563b8138fa20 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f546e2cf700 1 --2- 172.21.15.84:0/3650190237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 0x563b81393b90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f546000bfd0 tx=0x7f546009c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1116d1641dbf0616 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f54517fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f54517fa700 1 -- 172.21.15.84:0/3650190237 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54600a4eb0 con 0x563b8138fa20 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f54517fa700 1 -- 172.21.15.84:0/3650190237 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f546000e8e0 con 0x563b8138fa20 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3650190237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5464011c70 con 0x563b8138fa20 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f54517fa700 1 -- 172.21.15.84:0/3650190237 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f54600bb170 con 0x563b8138fa20 2024-04-26T18:26:57.095 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f547a5c4b80 10 client.15006 fetch_fsmap learned FSMap version 28 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3650190237 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b813a1450 con 0x563b8138fa20 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.090+0000 7f54517fa700 1 -- 172.21.15.84:0/3650190237 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5464011c70 con 0x563b8138fa20 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.091+0000 7f54517fa700 1 -- 172.21.15.84:0/3650190237 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f54600b0ce0 con 0x563b8138fa20 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.091+0000 7f547a5c4b80 10 client.15006 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.091+0000 7f547a5c4b80 10 client.15006 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.091+0000 7f547a5c4b80 20 client.15006 populate_metadata read hostname 'smithi084' 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.091+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/3650190237 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x563b813a39e0 0x563b813a4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.091+0000 7f547a5c4b80 4 client.15006 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.091+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3650190237 --> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b812ea1c0 con 0x563b813a39e0 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.091+0000 7f546dace700 1 --2- 172.21.15.84:0/3650190237 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x563b813a39e0 0x563b813a4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.091+0000 7f54517fa700 10 client.15006 ms_handle_connect on v2:172.21.15.174:6836/1532309720 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.091+0000 7f546dace700 1 --2- 172.21.15.84:0/3650190237 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x563b813a39e0 0x563b813a4e50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=423610e7ce48a3ee server_cookie=459fc3f6e17716c4 in_seq=0 out_seq=0 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f54517fa700 1 -- 172.21.15.84:0/3650190237 <== mds.3 v2:172.21.15.174:6836/1532309720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b812ea1c0 con 0x563b813a39e0 2024-04-26T18:26:57.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f54517fa700 10 client.15006 handle_command_reply: tid=0 2024-04-26T18:26:57.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f547a5c4b80 1 client.15006 shutdown 2024-04-26T18:26:57.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3650190237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 msgr2=0x563b81393b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:57.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/3650190237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 0x563b81393b90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f546000bfd0 tx=0x7f546009c9c0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3650190237 shutdown_connections 2024-04-26T18:26:57.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/3650190237 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x563b813a39e0 0x563b813a4e50 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/3650190237 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138fa20 0x563b81393b90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/3650190237 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b81389d50 0x563b813962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f54517fa700 0 client.15006 ms_handle_reset on v2:172.21.15.174:6836/1532309720 2024-04-26T18:26:57.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/3650190237 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b8138a6f0 0x563b81393650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3650190237 >> 172.21.15.84:0/3650190237 conn(0x563b812b8570 msgr2=0x563b81388d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:57.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3650190237 shutdown_connections 2024-04-26T18:26:57.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.092+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3650190237 wait complete. 2024-04-26T18:26:57.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.093+0000 7f547a5c4b80 20 client.15006 trim_cache size 0 max 16384 2024-04-26T18:26:57.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.113+0000 7f547a5c4b80 1 Processor -- start 2024-04-26T18:26:57.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.113+0000 7f547a5c4b80 1 -- start start 2024-04-26T18:26:57.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.113+0000 7f547a5c4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b81389d50 0x563b81393610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.113+0000 7f547a5c4b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 0x563b81393b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.113+0000 7f547a5c4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b8138fa20 0x563b813844d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.113+0000 7f547a5c4b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563b813a3190 con 0x563b8138a6f0 2024-04-26T18:26:57.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.113+0000 7f547a5c4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563b813a3860 con 0x563b8138fa20 2024-04-26T18:26:57.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.113+0000 7f547a5c4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b813a3f30 con 0x563b81389d50 2024-04-26T18:26:57.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.113+0000 7f546d2cd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 0x563b81393b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.113+0000 7f546d2cd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 0x563b81393b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44566/0 (socket says 172.21.15.84:44566) 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.113+0000 7f546d2cd700 1 -- 172.21.15.84:0/2172962561 learned_addr learned my addr 172.21.15.84:0/2172962561 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.114+0000 7f546dace700 1 --2- 172.21.15.84:0/2172962561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b81389d50 0x563b81393610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.114+0000 7f546d2cd700 1 -- 172.21.15.84:0/2172962561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b81389d50 msgr2=0x563b81393610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.114+0000 7f546e2cf700 1 --2- 172.21.15.84:0/2172962561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b8138fa20 0x563b813844d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.114+0000 7f546d2cd700 1 --2- 172.21.15.84:0/2172962561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b81389d50 0x563b81393610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.114+0000 7f546d2cd700 1 -- 172.21.15.84:0/2172962561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b8138fa20 msgr2=0x563b813844d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.114+0000 7f546d2cd700 1 --2- 172.21.15.84:0/2172962561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b8138fa20 0x563b813844d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.114+0000 7f546d2cd700 1 -- 172.21.15.84:0/2172962561 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b8138a4f0 con 0x563b8138a6f0 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.114+0000 7f546e2cf700 1 --2- 172.21.15.84:0/2172962561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b8138fa20 0x563b813844d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.114+0000 7f546d2cd700 1 --2- 172.21.15.84:0/2172962561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 0x563b81393b50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f54580098e0 tx=0x7f54580096a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9dc916778501d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.114+0000 7f54537fe700 1 -- 172.21.15.84:0/2172962561 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458027070 con 0x563b8138a6f0 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.114+0000 7f54537fe700 1 -- 172.21.15.84:0/2172962561 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5458003d10 con 0x563b8138a6f0 2024-04-26T18:26:57.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.114+0000 7f54537fe700 1 -- 172.21.15.84:0/2172962561 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458005660 con 0x563b8138a6f0 2024-04-26T18:26:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 -- 172.21.15.84:0/2172962561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 msgr2=0x563b81393b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/2172962561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 0x563b81393b50 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f54580098e0 tx=0x7f54580096a0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 -- 172.21.15.84:0/2172962561 shutdown_connections 2024-04-26T18:26:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/2172962561 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b8138fa20 0x563b813844d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/2172962561 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b81389d50 0x563b81393610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/2172962561 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 0x563b81393b50 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 -- 172.21.15.84:0/2172962561 >> 172.21.15.84:0/2172962561 conn(0x563b812b8570 msgr2=0x563b81392bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 -- 172.21.15.84:0/2172962561 shutdown_connections 2024-04-26T18:26:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 -- 172.21.15.84:0/2172962561 wait complete. 2024-04-26T18:26:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 -- start start 2024-04-26T18:26:57.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 Processor -- start 2024-04-26T18:26:57.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b81389d50 0x563b813ae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 0x563b813a9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b8138fa20 0x563b813a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563b813a4cd0 con 0x563b8138a6f0 2024-04-26T18:26:57.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563b813a4600 con 0x563b81389d50 2024-04-26T18:26:57.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.115+0000 7f547a5c4b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b813a53a0 con 0x563b8138fa20 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f547a5c4b80 10 client.0 _ll_register_callbacks cb 0x563b81274900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f546d2cd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 0x563b813a9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f546d2cd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 0x563b813a9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44578/0 (socket says 172.21.15.84:44578) 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f546d2cd700 1 -- 172.21.15.84:0/3721400814 learned_addr learned my addr 172.21.15.84:0/3721400814 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f546dace700 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b81389d50 0x563b813ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f546e2cf700 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b8138fa20 0x563b813a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f546d2cd700 1 -- 172.21.15.84:0/3721400814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b8138fa20 msgr2=0x563b813a97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f546d2cd700 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b8138fa20 0x563b813a97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f546d2cd700 1 -- 172.21.15.84:0/3721400814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b81389d50 msgr2=0x563b813ae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f546d2cd700 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b81389d50 0x563b813ae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f546d2cd700 1 -- 172.21.15.84:0/3721400814 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b813aa000 con 0x563b8138a6f0 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f546dace700 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b81389d50 0x563b813ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f546d2cd700 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 0x563b813a9280 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5458000c00 tx=0x7f5458005100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65b676f7c5dc391 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:57.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f54517fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:26:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f54517fa700 1 -- 172.21.15.84:0/3721400814 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5458027040 con 0x563b8138a6f0 2024-04-26T18:26:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3721400814 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5458003d10 con 0x563b8138a6f0 2024-04-26T18:26:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f54517fa700 1 -- 172.21.15.84:0/3721400814 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f545800ba00 con 0x563b8138a6f0 2024-04-26T18:26:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f54517fa700 1 -- 172.21.15.84:0/3721400814 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f545803d030 con 0x563b8138a6f0 2024-04-26T18:26:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f547a5c4b80 10 client.15012 fetch_fsmap learned FSMap version 28 2024-04-26T18:26:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3721400814 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563b8140baf0 con 0x563b8138a6f0 2024-04-26T18:26:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.116+0000 7f54517fa700 1 -- 172.21.15.84:0/3721400814 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5458003d10 con 0x563b8138a6f0 2024-04-26T18:26:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.117+0000 7f54517fa700 1 -- 172.21.15.84:0/3721400814 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f545800f460 con 0x563b8138a6f0 2024-04-26T18:26:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.117+0000 7f547a5c4b80 10 client.15012 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:26:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.117+0000 7f547a5c4b80 10 client.15012 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T18:26:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.117+0000 7f547a5c4b80 20 client.15012 populate_metadata read hostname 'smithi084' 2024-04-26T18:26:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.117+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x563b81410140 0x563b8140ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:57.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.117+0000 7f547a5c4b80 4 client.15012 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.117+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3721400814 --> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] -- command(tid 0: {"prefix": "status"}) -- 0x563b812e4260 con 0x563b81410140 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.117+0000 7f546dace700 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x563b81410140 0x563b8140ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.117+0000 7f546dace700 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x563b81410140 0x563b8140ee50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=4c23474f8fb1a02d server_cookie=97574f096cb11c07 in_seq=0 out_seq=0 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.117+0000 7f54517fa700 10 client.15012 ms_handle_connect on v2:172.21.15.174:6836/1532309720 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f54517fa700 1 -- 172.21.15.84:0/3721400814 <== mds.3 v2:172.21.15.174:6836/1532309720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x563b812e4260 con 0x563b81410140 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f54517fa700 10 client.15012 handle_command_reply: tid=0 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f547a5c4b80 1 client.15012 shutdown 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3721400814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 msgr2=0x563b813a9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 0x563b813a9280 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5458000c00 tx=0x7f5458005100 comp rx=0 tx=0).stop 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3721400814 shutdown_connections 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b8138fa20 0x563b813a97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b81389d50 0x563b813ae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x563b81410140 0x563b8140ee50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f547a5c4b80 1 --2- 172.21.15.84:0/3721400814 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b8138a6f0 0x563b813a9280 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3721400814 >> 172.21.15.84:0/3721400814 conn(0x563b812b8570 msgr2=0x563b81393610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:57.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f54517fa700 0 client.15012 ms_handle_reset on v2:172.21.15.174:6836/1532309720 2024-04-26T18:26:57.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3721400814 shutdown_connections 2024-04-26T18:26:57.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f547a5c4b80 1 -- 172.21.15.84:0/3721400814 wait complete. 2024-04-26T18:26:57.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f547a5c4b80 20 client.15012 trim_cache size 0 max 16384 2024-04-26T18:26:57.124 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:26:57.124 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:26:57.124 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 3, 2024-04-26T18:26:57.124 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24418, 2024-04-26T18:26:57.124 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:26:57.124 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:26:57.125 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:26:57.125 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 139.83285229500001, 2024-04-26T18:26:57.125 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 28, 2024-04-26T18:26:57.125 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 93, 2024-04-26T18:26:57.125 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 89, 2024-04-26T18:26:57.125 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 183.98406563 2024-04-26T18:26:57.125 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:26:57.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f546fd32700 1 -- 172.21.15.84:0/3790918251 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f54540a3c40 msgr2=0x7f54540a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:57.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f546fd32700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f54540a3c40 0x7f54540a6100 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5464004630 tx=0x7f546400b3c0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f546fd32700 1 -- 172.21.15.84:0/3790918251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5468109560 msgr2=0x7f54680761a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:57.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f546fd32700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5468109560 0x7f54680761a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f546000b970 tx=0x7f546000bc80 comp rx=0 tx=0).stop 2024-04-26T18:26:57.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f546fd32700 1 -- 172.21.15.84:0/3790918251 shutdown_connections 2024-04-26T18:26:57.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f546fd32700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f54540a3c40 0x7f54540a6100 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f546fd32700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5468109560 0x7f54680761a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f546fd32700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f54681034c0 0x7f5468075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f546fd32700 1 --2- 172.21.15.84:0/3790918251 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5468103e80 0x7f5468075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:57.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.118+0000 7f546fd32700 1 -- 172.21.15.84:0/3790918251 >> 172.21.15.84:0/3790918251 conn(0x7f546807a5d0 msgr2=0x7f54681021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:57.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.119+0000 7f546fd32700 1 -- 172.21.15.84:0/3790918251 shutdown_connections 2024-04-26T18:26:57.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:57.119+0000 7f546fd32700 1 -- 172.21.15.84:0/3790918251 wait complete. 2024-04-26T18:26:57.803 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-26T18:26:58.042 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:26:58.413 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:58 smithi084 ceph-mon[29141]: pgmap v379: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:58 smithi103 ceph-mon[32727]: pgmap v379: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:58.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:58 smithi174 ceph-mon[32705]: pgmap v379: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:26:59.441 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:26:59 smithi084 ceph-mon[29141]: pgmap v380: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:26:59 smithi103 ceph-mon[32727]: pgmap v380: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:26:59 smithi174 ceph-mon[32705]: pgmap v380: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:26:59.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.775+0000 7fe2d7b7d700 1 -- 172.21.15.84:0/1272259181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d0101a10 msgr2=0x7fe2d0101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.775+0000 7fe2d7b7d700 1 --2- 172.21.15.84:0/1272259181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d0101a10 0x7fe2d0101e70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0009990 tx=0x7fe2c0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.775+0000 7fe2d7b7d700 1 -- 172.21.15.84:0/1272259181 shutdown_connections 2024-04-26T18:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.775+0000 7fe2d7b7d700 1 --2- 172.21.15.84:0/1272259181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d01023b0 0x7fe2d01102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.775+0000 7fe2d7b7d700 1 --2- 172.21.15.84:0/1272259181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d01094b0 0x7fe2d0109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.775+0000 7fe2d7b7d700 1 --2- 172.21.15.84:0/1272259181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d0101a10 0x7fe2d0101e70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.775+0000 7fe2d7b7d700 1 -- 172.21.15.84:0/1272259181 >> 172.21.15.84:0/1272259181 conn(0x7fe2d0078990 msgr2=0x7fe2d00ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.775+0000 7fe2d7b7d700 1 -- 172.21.15.84:0/1272259181 shutdown_connections 2024-04-26T18:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.775+0000 7fe2d7b7d700 1 -- 172.21.15.84:0/1272259181 wait complete. 2024-04-26T18:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.776+0000 7fe2d7b7d700 1 Processor -- start 2024-04-26T18:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.776+0000 7fe2d7b7d700 1 -- start start 2024-04-26T18:26:59.778 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.776+0000 7fe2d7b7d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d0101a10 0x7fe2d019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.776+0000 7fe2d7b7d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d01023b0 0x7fe2d019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.776+0000 7fe2d7b7d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d01094b0 0x7fe2d0199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.776+0000 7fe2d7b7d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d0113660 con 0x7fe2d01023b0 2024-04-26T18:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.776+0000 7fe2d7b7d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d01134e0 con 0x7fe2d0101a10 2024-04-26T18:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.776+0000 7fe2d7b7d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe2d01137e0 con 0x7fe2d01094b0 2024-04-26T18:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.776+0000 7fe2d5118700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d01023b0 0x7fe2d019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.776+0000 7fe2d5118700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d01023b0 0x7fe2d019fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44594/0 (socket says 172.21.15.84:44594) 2024-04-26T18:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.776+0000 7fe2d5118700 1 -- 172.21.15.84:0/4002659248 learned_addr learned my addr 172.21.15.84:0/4002659248 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.776+0000 7fe2d5919700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d0101a10 0x7fe2d019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.776+0000 7fe2d611a700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d01094b0 0x7fe2d0199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.777+0000 7fe2d5118700 1 -- 172.21.15.84:0/4002659248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d01094b0 msgr2=0x7fe2d0199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.777+0000 7fe2d5118700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d01094b0 0x7fe2d0199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.779 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.777+0000 7fe2d5118700 1 -- 172.21.15.84:0/4002659248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d0101a10 msgr2=0x7fe2d019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:59.780 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.777+0000 7fe2d5118700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d0101a10 0x7fe2d019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.780 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.777+0000 7fe2d5118700 1 -- 172.21.15.84:0/4002659248 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2d019a0e0 con 0x7fe2d01023b0 2024-04-26T18:26:59.780 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.777+0000 7fe2d5919700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d0101a10 0x7fe2d019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:26:59.780 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.777+0000 7fe2d5118700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d01023b0 0x7fe2d019fd00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00057c0 tx=0x7fe2c00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:59.780 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.777+0000 7fe2c6ffd700 1 -- 172.21.15.84:0/4002659248 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0027070 con 0x7fe2d01023b0 2024-04-26T18:26:59.780 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.777+0000 7fe2d7b7d700 1 -- 172.21.15.84:0/4002659248 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2d019a370 con 0x7fe2d01023b0 2024-04-26T18:26:59.780 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.777+0000 7fe2c6ffd700 1 -- 172.21.15.84:0/4002659248 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c0003f20 con 0x7fe2d01023b0 2024-04-26T18:26:59.780 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.777+0000 7fe2c6ffd700 1 -- 172.21.15.84:0/4002659248 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0022b50 con 0x7fe2d01023b0 2024-04-26T18:26:59.780 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.777+0000 7fe2d7b7d700 1 -- 172.21.15.84:0/4002659248 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2d01a6690 con 0x7fe2d01023b0 2024-04-26T18:26:59.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.779+0000 7fe2c6ffd700 1 -- 172.21.15.84:0/4002659248 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe2c002a070 con 0x7fe2d01023b0 2024-04-26T18:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.780+0000 7fe2c6ffd700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe2bc0a3c40 0x7fe2bc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.780+0000 7fe2d5919700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe2bc0a3c40 0x7fe2bc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.780+0000 7fe2c6ffd700 1 -- 172.21.15.84:0/4002659248 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fe2c00d1370 con 0x7fe2d01023b0 2024-04-26T18:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.780+0000 7fe2d5919700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe2bc0a3c40 0x7fe2bc0a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe2d019aac0 tx=0x7fe2cc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2e240fb80 1 Processor -- start 2024-04-26T18:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2e240fb80 1 -- start start 2024-04-26T18:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2e240fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d55a20 0x557185d55e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2e240fb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 0x557185d501b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2e240fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d506f0 0x557185d5c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2e240fb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557185ca27e0 con 0x557185d4fd50 2024-04-26T18:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2e240fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557185c088e0 con 0x557185d506f0 2024-04-26T18:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2e240fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557185c9e7a0 con 0x557185d55a20 2024-04-26T18:26:59.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2d5118700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 0x557185d501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2d5118700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 0x557185d501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44604/0 (socket says 172.21.15.84:44604) 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2d5118700 1 -- 172.21.15.84:0/1184302807 learned_addr learned my addr 172.21.15.84:0/1184302807 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2d5919700 1 --2- 172.21.15.84:0/1184302807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d55a20 0x557185d55e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2d611a700 1 --2- 172.21.15.84:0/1184302807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d506f0 0x557185d5c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2d5118700 1 -- 172.21.15.84:0/1184302807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d55a20 msgr2=0x557185d55e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2d5118700 1 --2- 172.21.15.84:0/1184302807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d55a20 0x557185d55e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2d5118700 1 -- 172.21.15.84:0/1184302807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d506f0 msgr2=0x557185d5c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2d5118700 1 --2- 172.21.15.84:0/1184302807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d506f0 0x557185d5c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2d5118700 1 -- 172.21.15.84:0/1184302807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557185bef880 con 0x557185d4fd50 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2d5919700 1 --2- 172.21.15.84:0/1184302807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d55a20 0x557185d55e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.781+0000 7fe2d611a700 1 --2- 172.21.15.84:0/1184302807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d506f0 0x557185d5c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.782+0000 7fe2d5118700 1 --2- 172.21.15.84:0/1184302807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 0x557185d501b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe2c009d640 tx=0x7fe2c0005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e74b944a08be7ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.782+0000 7fe2bbfff700 1 -- 172.21.15.84:0/1184302807 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0022470 con 0x557185d4fd50 2024-04-26T18:26:59.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.782+0000 7fe2bbfff700 1 -- 172.21.15.84:0/1184302807 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c0099990 con 0x557185d4fd50 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.782+0000 7fe2bbfff700 1 -- 172.21.15.84:0/1184302807 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c00040d0 con 0x557185d4fd50 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.782+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1184302807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 msgr2=0x557185d501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.783+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1184302807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 0x557185d501b0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe2c009d640 tx=0x7fe2c0005a00 comp rx=0 tx=0).stop 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.783+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1184302807 shutdown_connections 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.783+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1184302807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d506f0 0x557185d5c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.783+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1184302807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d55a20 0x557185d55e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.783+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1184302807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 0x557185d501b0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.783+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1184302807 >> 172.21.15.84:0/1184302807 conn(0x557185c7e570 msgr2=0x557185d4a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.783+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1184302807 shutdown_connections 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.783+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1184302807 wait complete. 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.783+0000 7fe2e240fb80 1 -- start start 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.783+0000 7fe2e240fb80 1 Processor -- start 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2e240fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d4fd50 0x557185d5c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2e240fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d506f0 0x557185d59750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2e240fb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 0x557185d59c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2e240fb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557185d56420 con 0x557185d55a20 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2e240fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557185c5ca30 con 0x557185d4fd50 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2e240fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557185d610c0 con 0x557185d506f0 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2d611a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 0x557185d59c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2e240fb80 10 client.0 _ll_register_callbacks cb 0x557185c3a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2d611a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 0x557185d59c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44620/0 (socket says 172.21.15.84:44620) 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2d611a700 1 -- 172.21.15.84:0/1033460919 learned_addr learned my addr 172.21.15.84:0/1033460919 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2d5919700 1 --2- 172.21.15.84:0/1033460919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d4fd50 0x557185d5c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2d5118700 1 --2- 172.21.15.84:0/1033460919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d506f0 0x557185d59750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2d611a700 1 -- 172.21.15.84:0/1033460919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d506f0 msgr2=0x557185d59750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2d611a700 1 --2- 172.21.15.84:0/1033460919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d506f0 0x557185d59750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2d611a700 1 -- 172.21.15.84:0/1033460919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d4fd50 msgr2=0x557185d5c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:59.789 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2d611a700 1 --2- 172.21.15.84:0/1033460919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d4fd50 0x557185d5c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2d611a700 1 -- 172.21.15.84:0/1033460919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557185c60430 con 0x557185d55a20 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2d611a700 1 --2- 172.21.15.84:0/1033460919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 0x557185d59c90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800aeb0 tx=0x7fe2c800aee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4103a8e3aada20 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2b97fa700 1 -- 172.21.15.84:0/1033460919 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8007920 con 0x557185d55a20 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2b97fa700 1 -- 172.21.15.84:0/1033460919 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fe2c8007ac0 con 0x557185d55a20 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.784+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1033460919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c0099990 con 0x557185d55a20 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.785+0000 7fe2b97fa700 1 -- 172.21.15.84:0/1033460919 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe2c8007cc0 con 0x557185d55a20 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.785+0000 7fe2e240fb80 10 client.15024 fetch_fsmap learned FSMap version 28 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.785+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1033460919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557185d6b650 con 0x557185d55a20 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.785+0000 7fe2b97fa700 1 -- 172.21.15.84:0/1033460919 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c0099990 con 0x557185d55a20 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.785+0000 7fe2b97fa700 1 -- 172.21.15.84:0/1033460919 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe2c80153f0 con 0x557185d55a20 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.785+0000 7fe2e240fb80 10 client.15024 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.785+0000 7fe2e240fb80 10 client.15024 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T18:26:59.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.785+0000 7fe2e240fb80 20 client.15024 populate_metadata read hostname 'smithi084' 2024-04-26T18:26:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.785+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1033460919 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x557185d6dcb0 0x557185d6f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.785+0000 7fe2e240fb80 4 client.15024 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:26:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.785+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1033460919 --> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557185cb01c0 con 0x557185d6dcb0 2024-04-26T18:26:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.785+0000 7fe2d5919700 1 --2- 172.21.15.84:0/1033460919 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x557185d6dcb0 0x557185d6f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.786+0000 7fe2d5919700 1 --2- 172.21.15.84:0/1033460919 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x557185d6dcb0 0x557185d6f120 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=af8bc26442e25fcf server_cookie=307473e218f6da54 in_seq=0 out_seq=0 2024-04-26T18:26:59.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.786+0000 7fe2b97fa700 10 client.15024 ms_handle_connect on v2:172.21.15.103:6836/2271231481 2024-04-26T18:26:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2b97fa700 1 -- 172.21.15.84:0/1033460919 <== mds.2 v2:172.21.15.103:6836/2271231481 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557185cb01c0 con 0x557185d6dcb0 2024-04-26T18:26:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2b97fa700 10 client.15024 handle_command_reply: tid=0 2024-04-26T18:26:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2e240fb80 1 client.15024 shutdown 2024-04-26T18:26:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1033460919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 msgr2=0x557185d59c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1033460919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 0x557185d59c90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fe2c800aeb0 tx=0x7fe2c800aee0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.792 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1033460919 shutdown_connections 2024-04-26T18:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1033460919 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x557185d6dcb0 0x557185d6f120 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1033460919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 0x557185d59c90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1033460919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d4fd50 0x557185d5c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2b97fa700 0 client.15024 ms_handle_reset on v2:172.21.15.103:6836/2271231481 2024-04-26T18:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1033460919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d506f0 0x557185d59750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1033460919 >> 172.21.15.84:0/1033460919 conn(0x557185c7e570 msgr2=0x557185d4ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1033460919 shutdown_connections 2024-04-26T18:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1033460919 wait complete. 2024-04-26T18:26:59.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.787+0000 7fe2e240fb80 20 client.15024 trim_cache size 0 max 16384 2024-04-26T18:26:59.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.808+0000 7fe2e240fb80 1 Processor -- start 2024-04-26T18:26:59.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.808+0000 7fe2e240fb80 1 -- start start 2024-04-26T18:26:59.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.808+0000 7fe2e240fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d4fd50 0x557185d59cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.808+0000 7fe2e240fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d506f0 0x557185d4ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.808+0000 7fe2e240fb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 0x557185d4a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.808+0000 7fe2e240fb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557185d6d460 con 0x557185d55a20 2024-04-26T18:26:59.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.808+0000 7fe2e240fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557185d6db30 con 0x557185d4fd50 2024-04-26T18:26:59.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.808+0000 7fe2e240fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557185d6e200 con 0x557185d506f0 2024-04-26T18:26:59.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.808+0000 7fe2d611a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 0x557185d4a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.808+0000 7fe2d611a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 0x557185d4a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44636/0 (socket says 172.21.15.84:44636) 2024-04-26T18:26:59.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.808+0000 7fe2d611a700 1 -- 172.21.15.84:0/550409396 learned_addr learned my addr 172.21.15.84:0/550409396 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:59.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.809+0000 7fe2d5118700 1 --2- 172.21.15.84:0/550409396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d506f0 0x557185d4ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.809+0000 7fe2d5919700 1 --2- 172.21.15.84:0/550409396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d4fd50 0x557185d59cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.809+0000 7fe2d611a700 1 -- 172.21.15.84:0/550409396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d506f0 msgr2=0x557185d4ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.809+0000 7fe2d611a700 1 --2- 172.21.15.84:0/550409396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d506f0 0x557185d4ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.809+0000 7fe2d611a700 1 -- 172.21.15.84:0/550409396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d4fd50 msgr2=0x557185d59cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.809+0000 7fe2d611a700 1 --2- 172.21.15.84:0/550409396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d4fd50 0x557185d59cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.809+0000 7fe2d611a700 1 -- 172.21.15.84:0/550409396 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557185d563d0 con 0x557185d55a20 2024-04-26T18:26:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.809+0000 7fe2d611a700 1 --2- 172.21.15.84:0/550409396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 0x557185d4a4d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8007ff0 tx=0x7fe2c8007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aeef478b02c98fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.809+0000 7fe2bb7fe700 1 -- 172.21.15.84:0/550409396 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8009b40 con 0x557185d55a20 2024-04-26T18:26:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.809+0000 7fe2bb7fe700 1 -- 172.21.15.84:0/550409396 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c8015e50 con 0x557185d55a20 2024-04-26T18:26:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.809+0000 7fe2bb7fe700 1 -- 172.21.15.84:0/550409396 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2c80206d0 con 0x557185d55a20 2024-04-26T18:26:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.810+0000 7fe2e240fb80 1 -- 172.21.15.84:0/550409396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 msgr2=0x557185d4a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.810+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/550409396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 0x557185d4a4d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe2c8007ff0 tx=0x7fe2c8007590 comp rx=0 tx=0).stop 2024-04-26T18:26:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.810+0000 7fe2e240fb80 1 -- 172.21.15.84:0/550409396 shutdown_connections 2024-04-26T18:26:59.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.810+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/550409396 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d55a20 0x557185d4a4d0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.810+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/550409396 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d4fd50 0x557185d59cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.810+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/550409396 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d506f0 0x557185d4ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.810+0000 7fe2e240fb80 1 -- 172.21.15.84:0/550409396 >> 172.21.15.84:0/550409396 conn(0x557185c7e570 msgr2=0x557185d4f5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.810+0000 7fe2e240fb80 1 -- 172.21.15.84:0/550409396 shutdown_connections 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.810+0000 7fe2e240fb80 1 -- 172.21.15.84:0/550409396 wait complete. 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.810+0000 7fe2e240fb80 1 -- start start 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2e240fb80 1 Processor -- start 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2e240fb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 0x557185d785a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2e240fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d506f0 0x557185d78ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2e240fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d55a20 0x557185d71540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2e240fb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557185d6efa0 con 0x557185d4fd50 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2e240fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557185d6e8d0 con 0x557185d506f0 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2e240fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557185d6f670 con 0x557185d55a20 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2e240fb80 10 client.0 _ll_register_callbacks cb 0x557185c3a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:26:59.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2d5919700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 0x557185d785a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2d5919700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 0x557185d785a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44648/0 (socket says 172.21.15.84:44648) 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2d5919700 1 -- 172.21.15.84:0/1057728028 learned_addr learned my addr 172.21.15.84:0/1057728028 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2d5118700 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d506f0 0x557185d78ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2d611a700 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d55a20 0x557185d71540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2d5919700 1 -- 172.21.15.84:0/1057728028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d55a20 msgr2=0x557185d71540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2d5919700 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d55a20 0x557185d71540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2d5919700 1 -- 172.21.15.84:0/1057728028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d506f0 msgr2=0x557185d78ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.811+0000 7fe2d5919700 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d506f0 0x557185d78ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.812+0000 7fe2d5919700 1 -- 172.21.15.84:0/1057728028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557185d471f0 con 0x557185d4fd50 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.812+0000 7fe2d5118700 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d506f0 0x557185d78ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.812+0000 7fe2d5919700 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 0x557185d785a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x557185d6fc30 tx=0x7fe2cc013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b5fb99a65c07db server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.812+0000 7fe2b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.812+0000 7fe2b97fa700 1 -- 172.21.15.84:0/1057728028 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2cc004220 con 0x557185d4fd50 2024-04-26T18:26:59.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.812+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1057728028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2c8015e50 con 0x557185d4fd50 2024-04-26T18:26:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.812+0000 7fe2b97fa700 1 -- 172.21.15.84:0/1057728028 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fe2cc0043c0 con 0x557185d4fd50 2024-04-26T18:26:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.812+0000 7fe2b97fa700 1 -- 172.21.15.84:0/1057728028 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe2cc027240 con 0x557185d4fd50 2024-04-26T18:26:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.812+0000 7fe2e240fb80 10 client.15030 fetch_fsmap learned FSMap version 28 2024-04-26T18:26:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.812+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1057728028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557185dd5720 con 0x557185d4fd50 2024-04-26T18:26:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.812+0000 7fe2b97fa700 1 -- 172.21.15.84:0/1057728028 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2c8015e50 con 0x557185d4fd50 2024-04-26T18:26:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.812+0000 7fe2b97fa700 1 -- 172.21.15.84:0/1057728028 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fe2cc02e070 con 0x557185d4fd50 2024-04-26T18:26:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.812+0000 7fe2e240fb80 10 client.15030 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:26:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.813+0000 7fe2e240fb80 10 client.15030 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T18:26:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.813+0000 7fe2e240fb80 20 client.15030 populate_metadata read hostname 'smithi084' 2024-04-26T18:26:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.813+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x557185dda3a0 0x557185dd90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:26:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.813+0000 7fe2e240fb80 4 client.15030 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:26:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.813+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1057728028 --> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] -- command(tid 0: {"prefix": "status"}) -- 0x557185caa260 con 0x557185dda3a0 2024-04-26T18:26:59.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.813+0000 7fe2d5118700 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x557185dda3a0 0x557185dd90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.813+0000 7fe2b97fa700 10 client.15030 ms_handle_connect on v2:172.21.15.103:6836/2271231481 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.813+0000 7fe2d5118700 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x557185dda3a0 0x557185dd90b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b0220264d111220e server_cookie=6c749e3e86b1710f in_seq=0 out_seq=0 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.813+0000 7fe2b97fa700 1 -- 172.21.15.84:0/1057728028 <== mds.2 v2:172.21.15.103:6836/2271231481 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557185caa260 con 0x557185dda3a0 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.813+0000 7fe2b97fa700 10 client.15030 handle_command_reply: tid=0 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.813+0000 7fe2e240fb80 1 client.15030 shutdown 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1057728028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 msgr2=0x557185d785a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 0x557185d785a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x557185d6fc30 tx=0x7fe2cc013070 comp rx=0 tx=0).stop 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1057728028 shutdown_connections 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x557185dda3a0 0x557185dd90b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557185d4fd50 0x557185d785a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557185d55a20 0x557185d71540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2e240fb80 1 --2- 172.21.15.84:0/1057728028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557185d506f0 0x557185d78ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2b97fa700 0 client.15030 ms_handle_reset on v2:172.21.15.103:6836/2271231481 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1057728028 >> 172.21.15.84:0/1057728028 conn(0x557185c7e570 msgr2=0x557185d57f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:59.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1057728028 shutdown_connections 2024-04-26T18:26:59.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2e240fb80 1 -- 172.21.15.84:0/1057728028 wait complete. 2024-04-26T18:26:59.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2e240fb80 20 client.15030 trim_cache size 0 max 16384 2024-04-26T18:26:59.820 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:26:59.820 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:26:59.820 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 2, 2024-04-26T18:26:59.820 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24422, 2024-04-26T18:26:59.820 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:26:59.820 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:26:59.820 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:26:59.820 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 143.540338166, 2024-04-26T18:26:59.821 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 28, 2024-04-26T18:26:59.821 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 93, 2024-04-26T18:26:59.821 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 89, 2024-04-26T18:26:59.821 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 183.72003043399999 2024-04-26T18:26:59.821 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:26:59.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2d7b7d700 1 -- 172.21.15.84:0/4002659248 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe2bc0a3c40 msgr2=0x7fe2bc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:59.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2d7b7d700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe2bc0a3c40 0x7fe2bc0a6100 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe2d019aac0 tx=0x7fe2cc00b410 comp rx=0 tx=0).stop 2024-04-26T18:26:59.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2d7b7d700 1 -- 172.21.15.84:0/4002659248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d01023b0 msgr2=0x7fe2d019fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:26:59.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.814+0000 7fe2d7b7d700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d01023b0 0x7fe2d019fd00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00057c0 tx=0x7fe2c00057f0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.815+0000 7fe2d7b7d700 1 -- 172.21.15.84:0/4002659248 shutdown_connections 2024-04-26T18:26:59.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.815+0000 7fe2d7b7d700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fe2bc0a3c40 0x7fe2bc0a6100 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.815+0000 7fe2d7b7d700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe2d01094b0 0x7fe2d0199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.815+0000 7fe2d7b7d700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe2d0101a10 0x7fe2d019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.815+0000 7fe2d7b7d700 1 --2- 172.21.15.84:0/4002659248 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe2d01023b0 0x7fe2d019fd00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:26:59.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.815+0000 7fe2d7b7d700 1 -- 172.21.15.84:0/4002659248 >> 172.21.15.84:0/4002659248 conn(0x7fe2d0078990 msgr2=0x7fe2d01064f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:26:59.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.815+0000 7fe2d7b7d700 1 -- 172.21.15.84:0/4002659248 shutdown_connections 2024-04-26T18:26:59.822 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:26:59.815+0000 7fe2d7b7d700 1 -- 172.21.15.84:0/4002659248 wait complete. 2024-04-26T18:27:00.731 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T18:27:00.971 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:02.547 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:02 smithi084 ceph-mon[29141]: pgmap v381: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:02.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:02 smithi103 ceph-mon[32727]: pgmap v381: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:02 smithi174 ceph-mon[32705]: pgmap v381: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.746+0000 7f4004647700 1 -- 172.21.15.84:0/881650501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ffc102ce0 msgr2=0x7f3ffc10f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.746+0000 7f4004647700 1 --2- 172.21.15.84:0/881650501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ffc102ce0 0x7f3ffc10f1d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f3ff400b0a0 tx=0x7f3ff400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.746+0000 7f4004647700 1 -- 172.21.15.84:0/881650501 shutdown_connections 2024-04-26T18:27:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.746+0000 7f4004647700 1 --2- 172.21.15.84:0/881650501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ffc102ce0 0x7f3ffc10f1d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.746+0000 7f4004647700 1 --2- 172.21.15.84:0/881650501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ffc108390 0x7f3ffc108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.746+0000 7f4004647700 1 --2- 172.21.15.84:0/881650501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ffc102320 0x7f3ffc1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.746+0000 7f4004647700 1 -- 172.21.15.84:0/881650501 >> 172.21.15.84:0/881650501 conn(0x7f3ffc0792d0 msgr2=0x7f3ffc0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.746+0000 7f4004647700 1 -- 172.21.15.84:0/881650501 shutdown_connections 2024-04-26T18:27:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.746+0000 7f4004647700 1 -- 172.21.15.84:0/881650501 wait complete. 2024-04-26T18:27:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.747+0000 7f4004647700 1 Processor -- start 2024-04-26T18:27:02.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.747+0000 7f4004647700 1 -- start start 2024-04-26T18:27:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.747+0000 7f4004647700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ffc102320 0x7f3ffc104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.747+0000 7f4004647700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ffc102ce0 0x7f3ffc104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.747+0000 7f4004647700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ffc108390 0x7f3ffc104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.747+0000 7f4004647700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc1124d0 con 0x7f3ffc102320 2024-04-26T18:27:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.747+0000 7f4004647700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc112350 con 0x7f3ffc108390 2024-04-26T18:27:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.747+0000 7f4004647700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ffc112650 con 0x7f3ffc102ce0 2024-04-26T18:27:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.747+0000 7f40023e3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ffc102320 0x7f3ffc104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.749 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.747+0000 7f40023e3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ffc102320 0x7f3ffc104000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44670/0 (socket says 172.21.15.84:44670) 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.747+0000 7f40023e3700 1 -- 172.21.15.84:0/3857374102 learned_addr learned my addr 172.21.15.84:0/3857374102 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.748+0000 7f4002be4700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ffc108390 0x7f3ffc104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.748+0000 7f4001be2700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ffc102ce0 0x7f3ffc104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.748+0000 7f40023e3700 1 -- 172.21.15.84:0/3857374102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ffc102ce0 msgr2=0x7f3ffc104540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.748+0000 7f40023e3700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ffc102ce0 0x7f3ffc104540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.748+0000 7f40023e3700 1 -- 172.21.15.84:0/3857374102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ffc108390 msgr2=0x7f3ffc104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.748+0000 7f40023e3700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ffc108390 0x7f3ffc104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.748+0000 7f40023e3700 1 -- 172.21.15.84:0/3857374102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ffc1a7fb0 con 0x7f3ffc102320 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.748+0000 7f40023e3700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ffc102320 0x7f3ffc104000 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c970 tx=0x7f3ff800cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.748+0000 7f3ff37fe700 1 -- 172.21.15.84:0/3857374102 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff8007aa0 con 0x7f3ffc102320 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.748+0000 7f4004647700 1 -- 172.21.15.84:0/3857374102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ffc1a82a0 con 0x7f3ffc102320 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.748+0000 7f3ff37fe700 1 -- 172.21.15.84:0/3857374102 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff8004ce0 con 0x7f3ffc102320 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.748+0000 7f3ff37fe700 1 -- 172.21.15.84:0/3857374102 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80055c0 con 0x7f3ffc102320 2024-04-26T18:27:02.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.748+0000 7f4004647700 1 -- 172.21.15.84:0/3857374102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ffc1a8790 con 0x7f3ffc102320 2024-04-26T18:27:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.750+0000 7f3ff37fe700 1 -- 172.21.15.84:0/3857374102 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3ff8007c40 con 0x7f3ffc102320 2024-04-26T18:27:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.751+0000 7f3ff37fe700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3fe80a3c40 0x7f3fe80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.751+0000 7f4001be2700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3fe80a3c40 0x7f3fe80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.751+0000 7f3ff37fe700 1 -- 172.21.15.84:0/3857374102 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f3ff80d0260 con 0x7f3ffc102320 2024-04-26T18:27:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.751+0000 7f4001be2700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3fe80a3c40 0x7f3fe80a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3fec005fd0 tx=0x7f3fec00a560 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f400eed9b80 1 Processor -- start 2024-04-26T18:27:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f400eed9b80 1 -- start start 2024-04-26T18:27:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f400eed9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f7a20 0x556cec8f7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f400eed9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f1d50 0x556cec8f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f400eed9b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f26f0 0x556cec8fe770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f400eed9b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556cec8447e0 con 0x556cec8f26f0 2024-04-26T18:27:02.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f400eed9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556cec7aa8e0 con 0x556cec8f7a20 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f400eed9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556cec8407a0 con 0x556cec8f1d50 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f4002be4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f26f0 0x556cec8fe770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f4002be4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f26f0 0x556cec8fe770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44682/0 (socket says 172.21.15.84:44682) 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f4002be4700 1 -- 172.21.15.84:0/451667961 learned_addr learned my addr 172.21.15.84:0/451667961 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f4001be2700 1 --2- 172.21.15.84:0/451667961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f1d50 0x556cec8f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f40023e3700 1 --2- 172.21.15.84:0/451667961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f7a20 0x556cec8f7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f4002be4700 1 -- 172.21.15.84:0/451667961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f1d50 msgr2=0x556cec8f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f4002be4700 1 --2- 172.21.15.84:0/451667961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f1d50 0x556cec8f21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f4002be4700 1 -- 172.21.15.84:0/451667961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f7a20 msgr2=0x556cec8f7e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f4002be4700 1 --2- 172.21.15.84:0/451667961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f7a20 0x556cec8f7e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f4002be4700 1 -- 172.21.15.84:0/451667961 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cec791880 con 0x556cec8f26f0 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f4001be2700 1 --2- 172.21.15.84:0/451667961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f1d50 0x556cec8f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.752+0000 7f40023e3700 1 --2- 172.21.15.84:0/451667961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f7a20 0x556cec8f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:02.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.753+0000 7f4002be4700 1 --2- 172.21.15.84:0/451667961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f26f0 0x556cec8fe770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40092e0 tx=0x7f3ff4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891ba5fac7ef8217 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:02.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.753+0000 7f3ff0ff9700 1 -- 172.21.15.84:0/451667961 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff4004a50 con 0x556cec8f26f0 2024-04-26T18:27:02.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.753+0000 7f3ff0ff9700 1 -- 172.21.15.84:0/451667961 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff4004bf0 con 0x556cec8f26f0 2024-04-26T18:27:02.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.753+0000 7f3ff0ff9700 1 -- 172.21.15.84:0/451667961 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff401c6e0 con 0x556cec8f26f0 2024-04-26T18:27:02.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.753+0000 7f400eed9b80 1 -- 172.21.15.84:0/451667961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f26f0 msgr2=0x556cec8fe770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:02.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.753+0000 7f400eed9b80 1 --2- 172.21.15.84:0/451667961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f26f0 0x556cec8fe770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f3ff40092e0 tx=0x7f3ff4009310 comp rx=0 tx=0).stop 2024-04-26T18:27:02.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 -- 172.21.15.84:0/451667961 shutdown_connections 2024-04-26T18:27:02.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 --2- 172.21.15.84:0/451667961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f26f0 0x556cec8fe770 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 --2- 172.21.15.84:0/451667961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f7a20 0x556cec8f7e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 --2- 172.21.15.84:0/451667961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f1d50 0x556cec8f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 -- 172.21.15.84:0/451667961 >> 172.21.15.84:0/451667961 conn(0x556cec820570 msgr2=0x556cec8ec900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:02.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 -- 172.21.15.84:0/451667961 shutdown_connections 2024-04-26T18:27:02.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 -- 172.21.15.84:0/451667961 wait complete. 2024-04-26T18:27:02.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 -- start start 2024-04-26T18:27:02.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 Processor -- start 2024-04-26T18:27:02.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 0x556cec8fe320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f26f0 0x556cec8fb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f7a20 0x556cec8fbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556cec8fecb0 con 0x556cec8f1d50 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556cec7fea30 con 0x556cec8f7a20 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556cec8fee30 con 0x556cec8f26f0 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f400eed9b80 10 client.0 _ll_register_callbacks cb 0x556cec7dc900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f40023e3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 0x556cec8fe320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f40023e3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 0x556cec8fe320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44692/0 (socket says 172.21.15.84:44692) 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f40023e3700 1 -- 172.21.15.84:0/3619300346 learned_addr learned my addr 172.21.15.84:0/3619300346 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.754+0000 7f4001be2700 1 --2- 172.21.15.84:0/3619300346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f26f0 0x556cec8fb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f4002be4700 1 --2- 172.21.15.84:0/3619300346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f7a20 0x556cec8fbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f40023e3700 1 -- 172.21.15.84:0/3619300346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f26f0 msgr2=0x556cec8fb6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f40023e3700 1 --2- 172.21.15.84:0/3619300346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f26f0 0x556cec8fb6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f40023e3700 1 -- 172.21.15.84:0/3619300346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f7a20 msgr2=0x556cec8fbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f40023e3700 1 --2- 172.21.15.84:0/3619300346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f7a20 0x556cec8fbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f40023e3700 1 -- 172.21.15.84:0/3619300346 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cec802430 con 0x556cec8f1d50 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f40023e3700 1 --2- 172.21.15.84:0/3619300346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 0x556cec8fe320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80cdd30 tx=0x7f3ff809cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436399227227f58c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f3fe5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f3fe5ffb700 1 -- 172.21.15.84:0/3619300346 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80b4020 con 0x556cec8f1d50 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f3fe5ffb700 1 -- 172.21.15.84:0/3619300346 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f3ff8085d30 con 0x556cec8f1d50 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f400eed9b80 1 -- 172.21.15.84:0/3619300346 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff4004bf0 con 0x556cec8f1d50 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f3fe5ffb700 1 -- 172.21.15.84:0/3619300346 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3ff809cca0 con 0x556cec8f1d50 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f400eed9b80 10 client.15042 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f400eed9b80 1 -- 172.21.15.84:0/3619300346 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556cec9093e0 con 0x556cec8f1d50 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f3fe5ffb700 1 -- 172.21.15.84:0/3619300346 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff4004bf0 con 0x556cec8f1d50 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f3fe5ffb700 1 -- 172.21.15.84:0/3619300346 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3ff80bb230 con 0x556cec8f1d50 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.755+0000 7f400eed9b80 10 client.15042 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.756+0000 7f400eed9b80 10 client.15042 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.756+0000 7f400eed9b80 20 client.15042 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:02.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.756+0000 7f400eed9b80 1 --2- 172.21.15.84:0/3619300346 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x556cec90b9b0 0x556cec90ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.756+0000 7f400eed9b80 4 client.15042 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.756+0000 7f400eed9b80 1 -- 172.21.15.84:0/3619300346 --> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cec8521c0 con 0x556cec90b9b0 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.756+0000 7f4001be2700 1 --2- 172.21.15.84:0/3619300346 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x556cec90b9b0 0x556cec90ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.756+0000 7f4001be2700 1 --2- 172.21.15.84:0/3619300346 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x556cec90b9b0 0x556cec90ce20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dd5d4d003b62f733 server_cookie=7d30b1919af2d2dd in_seq=0 out_seq=0 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.756+0000 7f3fe5ffb700 10 client.15042 ms_handle_connect on v2:172.21.15.174:6838/518809558 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f3fe5ffb700 1 -- 172.21.15.84:0/3619300346 <== mds.1 v2:172.21.15.174:6838/518809558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556cec8521c0 con 0x556cec90b9b0 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f3fe5ffb700 10 client.15042 handle_command_reply: tid=0 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f400eed9b80 1 client.15042 shutdown 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f400eed9b80 1 -- 172.21.15.84:0/3619300346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 msgr2=0x556cec8fe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f400eed9b80 1 --2- 172.21.15.84:0/3619300346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 0x556cec8fe320 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3ff80cdd30 tx=0x7f3ff809cae0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f400eed9b80 1 -- 172.21.15.84:0/3619300346 shutdown_connections 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f400eed9b80 1 --2- 172.21.15.84:0/3619300346 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x556cec90b9b0 0x556cec90ce20 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f400eed9b80 1 --2- 172.21.15.84:0/3619300346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 0x556cec8fe320 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f400eed9b80 1 --2- 172.21.15.84:0/3619300346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f7a20 0x556cec8fbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f400eed9b80 1 --2- 172.21.15.84:0/3619300346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f26f0 0x556cec8fb6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f400eed9b80 1 -- 172.21.15.84:0/3619300346 >> 172.21.15.84:0/3619300346 conn(0x556cec820570 msgr2=0x556cec8f0d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:02.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f3fe5ffb700 0 client.15042 ms_handle_reset on v2:172.21.15.174:6838/518809558 2024-04-26T18:27:02.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f400eed9b80 1 -- 172.21.15.84:0/3619300346 shutdown_connections 2024-04-26T18:27:02.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f400eed9b80 1 -- 172.21.15.84:0/3619300346 wait complete. 2024-04-26T18:27:02.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.757+0000 7f400eed9b80 20 client.15042 trim_cache size 0 max 16384 2024-04-26T18:27:02.781 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.777+0000 7f400eed9b80 1 Processor -- start 2024-04-26T18:27:02.781 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.777+0000 7f400eed9b80 1 -- start start 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f400eed9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f1d50 0x556cec8f5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f400eed9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f26f0 0x556cec8f5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f400eed9b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f7a20 0x556cec8f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f400eed9b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556cec90b160 con 0x556cec8f7a20 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f400eed9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556cec90b830 con 0x556cec8f26f0 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f400eed9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556cec90bf00 con 0x556cec8f1d50 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f4002be4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f7a20 0x556cec8f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f4002be4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f7a20 0x556cec8f0d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44702/0 (socket says 172.21.15.84:44702) 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f4002be4700 1 -- 172.21.15.84:0/1703189203 learned_addr learned my addr 172.21.15.84:0/1703189203 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f4001be2700 1 --2- 172.21.15.84:0/1703189203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f26f0 0x556cec8f5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f4002be4700 1 -- 172.21.15.84:0/1703189203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f1d50 msgr2=0x556cec8f5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f4002be4700 1 --2- 172.21.15.84:0/1703189203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f1d50 0x556cec8f5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f4002be4700 1 -- 172.21.15.84:0/1703189203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f26f0 msgr2=0x556cec8f5c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:02.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f4002be4700 1 --2- 172.21.15.84:0/1703189203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f26f0 0x556cec8f5c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f4002be4700 1 -- 172.21.15.84:0/1703189203 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cec90b650 con 0x556cec8f7a20 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f4002be4700 1 --2- 172.21.15.84:0/1703189203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f7a20 0x556cec8f0d40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009820 tx=0x7f3ff400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ed5e5dafeaead8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f3fe7fff700 1 -- 172.21.15.84:0/1703189203 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff402a030 con 0x556cec8f7a20 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f3fe7fff700 1 -- 172.21.15.84:0/1703189203 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff4007950 con 0x556cec8f7a20 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.778+0000 7f3fe7fff700 1 -- 172.21.15.84:0/1703189203 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff401c9d0 con 0x556cec8f7a20 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.779+0000 7f400eed9b80 1 -- 172.21.15.84:0/1703189203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f7a20 msgr2=0x556cec8f0d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.779+0000 7f400eed9b80 1 --2- 172.21.15.84:0/1703189203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f7a20 0x556cec8f0d40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4009820 tx=0x7f3ff400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.779+0000 7f400eed9b80 1 -- 172.21.15.84:0/1703189203 shutdown_connections 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.779+0000 7f400eed9b80 1 --2- 172.21.15.84:0/1703189203 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f7a20 0x556cec8f0d40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.779+0000 7f400eed9b80 1 --2- 172.21.15.84:0/1703189203 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f1d50 0x556cec8f5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.779+0000 7f400eed9b80 1 --2- 172.21.15.84:0/1703189203 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f26f0 0x556cec8f5c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.779+0000 7f400eed9b80 1 -- 172.21.15.84:0/1703189203 >> 172.21.15.84:0/1703189203 conn(0x556cec820570 msgr2=0x556cec8ec4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.779+0000 7f400eed9b80 1 -- 172.21.15.84:0/1703189203 shutdown_connections 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.779+0000 7f400eed9b80 1 -- 172.21.15.84:0/1703189203 wait complete. 2024-04-26T18:27:02.783 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.779+0000 7f400eed9b80 1 -- start start 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.779+0000 7f400eed9b80 1 Processor -- start 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f400eed9b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 0x556cec8f4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f400eed9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f26f0 0x556cec916290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f400eed9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f7a20 0x556cec8fcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f400eed9b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556cec90cca0 con 0x556cec8f1d50 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f400eed9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556cec90c5d0 con 0x556cec8f26f0 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f400eed9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556cec90d370 con 0x556cec8f7a20 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f400eed9b80 10 client.0 _ll_register_callbacks cb 0x556cec7dc900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f40023e3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 0x556cec8f4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f40023e3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 0x556cec8f4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44714/0 (socket says 172.21.15.84:44714) 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f40023e3700 1 -- 172.21.15.84:0/1886136447 learned_addr learned my addr 172.21.15.84:0/1886136447 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f4001be2700 1 --2- 172.21.15.84:0/1886136447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f26f0 0x556cec916290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f4002be4700 1 --2- 172.21.15.84:0/1886136447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f7a20 0x556cec8fcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f40023e3700 1 -- 172.21.15.84:0/1886136447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f7a20 msgr2=0x556cec8fcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:02.784 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f40023e3700 1 --2- 172.21.15.84:0/1886136447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f7a20 0x556cec8fcea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f40023e3700 1 -- 172.21.15.84:0/1886136447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f26f0 msgr2=0x556cec916290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f40023e3700 1 --2- 172.21.15.84:0/1886136447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f26f0 0x556cec916290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f40023e3700 1 -- 172.21.15.84:0/1886136447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cec916b50 con 0x556cec8f1d50 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f40023e3700 1 --2- 172.21.15.84:0/1886136447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 0x556cec8f4c70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8098f70 tx=0x7f3ff80985f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd1b173b86ec771 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f3fe5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f3fe5ffb700 1 -- 172.21.15.84:0/1886136447 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ff80b4070 con 0x556cec8f1d50 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f3fe5ffb700 1 -- 172.21.15.84:0/1886136447 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f3ff809b9f0 con 0x556cec8f1d50 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.780+0000 7f400eed9b80 1 -- 172.21.15.84:0/1886136447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ff4007950 con 0x556cec8f1d50 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.781+0000 7f3fe5ffb700 1 -- 172.21.15.84:0/1886136447 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f3ff809c0e0 con 0x556cec8f1d50 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.781+0000 7f400eed9b80 10 client.15048 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.781+0000 7f400eed9b80 1 -- 172.21.15.84:0/1886136447 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x556cec973140 con 0x556cec8f1d50 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.781+0000 7f3fe5ffb700 1 -- 172.21.15.84:0/1886136447 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ff4007950 con 0x556cec8f1d50 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.781+0000 7f3fe5ffb700 1 -- 172.21.15.84:0/1886136447 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f3ff8004750 con 0x556cec8f1d50 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.781+0000 7f400eed9b80 10 client.15048 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:02.785 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.781+0000 7f400eed9b80 10 client.15048 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:27:02.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.781+0000 7f400eed9b80 20 client.15048 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:02.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.781+0000 7f400eed9b80 1 --2- 172.21.15.84:0/1886136447 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x556cec977d00 0x556cec976a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:02.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.781+0000 7f400eed9b80 4 client.15048 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:27:02.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.781+0000 7f400eed9b80 1 -- 172.21.15.84:0/1886136447 --> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] -- command(tid 0: {"prefix": "status"}) -- 0x556cec84c260 con 0x556cec977d00 2024-04-26T18:27:02.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.781+0000 7f4001be2700 1 --2- 172.21.15.84:0/1886136447 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x556cec977d00 0x556cec976a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:02.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f3fe5ffb700 10 client.15048 ms_handle_connect on v2:172.21.15.174:6838/518809558 2024-04-26T18:27:02.786 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f4001be2700 1 --2- 172.21.15.84:0/1886136447 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x556cec977d00 0x556cec976a10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3d141ee0739e9058 server_cookie=d555f6b5a81b9c48 in_seq=0 out_seq=0 2024-04-26T18:27:02.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f3fe5ffb700 1 -- 172.21.15.84:0/1886136447 <== mds.1 v2:172.21.15.174:6838/518809558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x556cec84c260 con 0x556cec977d00 2024-04-26T18:27:02.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f3fe5ffb700 10 client.15048 handle_command_reply: tid=0 2024-04-26T18:27:02.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f400eed9b80 1 client.15048 shutdown 2024-04-26T18:27:02.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f400eed9b80 1 -- 172.21.15.84:0/1886136447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 msgr2=0x556cec8f4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:02.787 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f400eed9b80 1 --2- 172.21.15.84:0/1886136447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 0x556cec8f4c70 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3ff8098f70 tx=0x7f3ff80985f0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f400eed9b80 1 -- 172.21.15.84:0/1886136447 shutdown_connections 2024-04-26T18:27:02.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f400eed9b80 1 --2- 172.21.15.84:0/1886136447 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x556cec977d00 0x556cec976a10 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f400eed9b80 1 --2- 172.21.15.84:0/1886136447 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556cec8f1d50 0x556cec8f4c70 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f400eed9b80 1 --2- 172.21.15.84:0/1886136447 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556cec8f7a20 0x556cec8fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f400eed9b80 1 --2- 172.21.15.84:0/1886136447 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556cec8f26f0 0x556cec916290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f400eed9b80 1 -- 172.21.15.84:0/1886136447 >> 172.21.15.84:0/1886136447 conn(0x556cec820570 msgr2=0x556cec8f5710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:02.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f3fe5ffb700 0 client.15048 ms_handle_reset on v2:172.21.15.174:6838/518809558 2024-04-26T18:27:02.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f400eed9b80 1 -- 172.21.15.84:0/1886136447 shutdown_connections 2024-04-26T18:27:02.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f400eed9b80 1 -- 172.21.15.84:0/1886136447 wait complete. 2024-04-26T18:27:02.788 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.782+0000 7f400eed9b80 20 client.15048 trim_cache size 0 max 16384 2024-04-26T18:27:02.788 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:27:02.789 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:27:02.789 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 1, 2024-04-26T18:27:02.789 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24430, 2024-04-26T18:27:02.789 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:27:02.789 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:27:02.789 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:27:02.789 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 147.520878772, 2024-04-26T18:27:02.789 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 28, 2024-04-26T18:27:02.789 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 93, 2024-04-26T18:27:02.789 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 89, 2024-04-26T18:27:02.789 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 183.91844990000001 2024-04-26T18:27:02.789 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:27:02.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.783+0000 7f4004647700 1 -- 172.21.15.84:0/3857374102 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3fe80a3c40 msgr2=0x7f3fe80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:02.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.783+0000 7f4004647700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3fe80a3c40 0x7f3fe80a6100 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3fec005fd0 tx=0x7f3fec00a560 comp rx=0 tx=0).stop 2024-04-26T18:27:02.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.783+0000 7f4004647700 1 -- 172.21.15.84:0/3857374102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ffc102320 msgr2=0x7f3ffc104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:02.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.783+0000 7f4004647700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ffc102320 0x7f3ffc104000 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3ff800c970 tx=0x7f3ff800cd30 comp rx=0 tx=0).stop 2024-04-26T18:27:02.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.783+0000 7f4004647700 1 -- 172.21.15.84:0/3857374102 shutdown_connections 2024-04-26T18:27:02.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.783+0000 7f4004647700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f3fe80a3c40 0x7f3fe80a6100 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.783+0000 7f4004647700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ffc102320 0x7f3ffc104000 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.783+0000 7f4004647700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ffc108390 0x7f3ffc104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.790 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.783+0000 7f4004647700 1 --2- 172.21.15.84:0/3857374102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ffc102ce0 0x7f3ffc104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:02.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.783+0000 7f4004647700 1 -- 172.21.15.84:0/3857374102 >> 172.21.15.84:0/3857374102 conn(0x7f3ffc0792d0 msgr2=0x7f3ffc1015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:02.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.783+0000 7f4004647700 1 -- 172.21.15.84:0/3857374102 shutdown_connections 2024-04-26T18:27:02.791 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:02.783+0000 7f4004647700 1 -- 172.21.15.84:0/3857374102 wait complete. 2024-04-26T18:27:03.658 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T18:27:03.900 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:04.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:04 smithi084 ceph-mon[29141]: pgmap v382: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:04.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:04 smithi103 ceph-mon[32727]: pgmap v382: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:04.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:04 smithi174 ceph-mon[32705]: pgmap v382: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:05.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.680+0000 7f6910477700 1 -- 172.21.15.84:0/3749862907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6908103e30 msgr2=0x7f6908110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:05.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.680+0000 7f6910477700 1 --2- 172.21.15.84:0/3749862907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6908103e30 0x7f6908110290 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f690000b0a0 tx=0x7f690000b3b0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.680+0000 7f6910477700 1 -- 172.21.15.84:0/3749862907 shutdown_connections 2024-04-26T18:27:05.684 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.680+0000 7f6910477700 1 --2- 172.21.15.84:0/3749862907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6908103e30 0x7f6908110290 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.680+0000 7f6910477700 1 --2- 172.21.15.84:0/3749862907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69081094c0 0x7f69081098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.680+0000 7f6910477700 1 --2- 172.21.15.84:0/3749862907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6908103470 0x7f69081038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.680+0000 7f6910477700 1 -- 172.21.15.84:0/3749862907 >> 172.21.15.84:0/3749862907 conn(0x7f690807a580 msgr2=0x7f690807a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:05.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.680+0000 7f6910477700 1 -- 172.21.15.84:0/3749862907 shutdown_connections 2024-04-26T18:27:05.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.680+0000 7f6910477700 1 -- 172.21.15.84:0/3749862907 wait complete. 2024-04-26T18:27:05.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.680+0000 7f6910477700 1 Processor -- start 2024-04-26T18:27:05.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.680+0000 7f6910477700 1 -- start start 2024-04-26T18:27:05.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f6910477700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6908103470 0x7f690819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f6910477700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6908103e30 0x7f690819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f6910477700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69081094c0 0x7f6908199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f6910477700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69081135c0 con 0x7f6908103470 2024-04-26T18:27:05.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f6910477700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6908113440 con 0x7f69081094c0 2024-04-26T18:27:05.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f6910477700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6908113740 con 0x7f6908103e30 2024-04-26T18:27:05.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f690e213700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6908103470 0x7f690819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f690e213700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6908103470 0x7f690819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44720/0 (socket says 172.21.15.84:44720) 2024-04-26T18:27:05.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f690e213700 1 -- 172.21.15.84:0/2117625777 learned_addr learned my addr 172.21.15.84:0/2117625777 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:05.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f690da12700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6908103e30 0x7f690819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f690ea14700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69081094c0 0x7f6908199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f690e213700 1 -- 172.21.15.84:0/2117625777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6908103e30 msgr2=0x7f690819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:05.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f690e213700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6908103e30 0x7f690819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f690e213700 1 -- 172.21.15.84:0/2117625777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69081094c0 msgr2=0x7f6908199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:05.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f690e213700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69081094c0 0x7f6908199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f690e213700 1 -- 172.21.15.84:0/2117625777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690819a000 con 0x7f6908103470 2024-04-26T18:27:05.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f690e213700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6908103470 0x7f690819f6a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f68f800c9b0 tx=0x7f68f800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:05.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f68ff7fe700 1 -- 172.21.15.84:0/2117625777 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8007b00 con 0x7f6908103470 2024-04-26T18:27:05.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f68ff7fe700 1 -- 172.21.15.84:0/2117625777 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68f8007ca0 con 0x7f6908103470 2024-04-26T18:27:05.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f6910477700 1 -- 172.21.15.84:0/2117625777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f690819a2f0 con 0x7f6908103470 2024-04-26T18:27:05.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.681+0000 7f68ff7fe700 1 -- 172.21.15.84:0/2117625777 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8018660 con 0x7f6908103470 2024-04-26T18:27:05.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.682+0000 7f6910477700 1 -- 172.21.15.84:0/2117625777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69081a6500 con 0x7f6908103470 2024-04-26T18:27:05.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.684+0000 7f68ff7fe700 1 -- 172.21.15.84:0/2117625777 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f68f800f450 con 0x7f6908103470 2024-04-26T18:27:05.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.684+0000 7f68ff7fe700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f68f40a3c40 0x7f68f40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.684+0000 7f690da12700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f68f40a3c40 0x7f68f40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.684+0000 7f68ff7fe700 1 -- 172.21.15.84:0/2117625777 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f68f80d0390 con 0x7f6908103470 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.685+0000 7f690da12700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f68f40a3c40 0x7f68f40a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f690807ab80 tx=0x7f690400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.685+0000 7f691ad09b80 1 Processor -- start 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.685+0000 7f691ad09b80 1 -- start start 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.685+0000 7f691ad09b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0bad0 0x55d588d0beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.685+0000 7f691ad09b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d0c3f0 0x55d588d05cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.685+0000 7f691ad09b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d062d0 0x55d588d06710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.685+0000 7f691ad09b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d588c586f0 con 0x55d588d062d0 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.685+0000 7f691ad09b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d588bf7590 con 0x55d588d0c3f0 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.685+0000 7f691ad09b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d588c546b0 con 0x55d588d0bad0 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f690ea14700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d062d0 0x55d588d06710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f690ea14700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d062d0 0x55d588d06710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44726/0 (socket says 172.21.15.84:44726) 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f690ea14700 1 -- 172.21.15.84:0/1944231737 learned_addr learned my addr 172.21.15.84:0/1944231737 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f690da12700 1 --2- 172.21.15.84:0/1944231737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d0c3f0 0x55d588d05cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f690e213700 1 --2- 172.21.15.84:0/1944231737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0bad0 0x55d588d0beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f690ea14700 1 -- 172.21.15.84:0/1944231737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0bad0 msgr2=0x55d588d0beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f690ea14700 1 --2- 172.21.15.84:0/1944231737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0bad0 0x55d588d0beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f690ea14700 1 -- 172.21.15.84:0/1944231737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d0c3f0 msgr2=0x55d588d05cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f690ea14700 1 --2- 172.21.15.84:0/1944231737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d0c3f0 0x55d588d05cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f690ea14700 1 -- 172.21.15.84:0/1944231737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d588ba5880 con 0x55d588d062d0 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f690da12700 1 --2- 172.21.15.84:0/1944231737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d0c3f0 0x55d588d05cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f690ea14700 1 --2- 172.21.15.84:0/1944231737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d062d0 0x55d588d06710 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f69000090d0 tx=0x7f6900009100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fcff115ba7a650e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f68fcd70700 1 -- 172.21.15.84:0/1944231737 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6900004a50 con 0x55d588d062d0 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f68fcd70700 1 -- 172.21.15.84:0/1944231737 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6900004bf0 con 0x55d588d062d0 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.686+0000 7f68fcd70700 1 -- 172.21.15.84:0/1944231737 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690001c6d0 con 0x55d588d062d0 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.687+0000 7f691ad09b80 1 -- 172.21.15.84:0/1944231737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d062d0 msgr2=0x55d588d06710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.687+0000 7f691ad09b80 1 --2- 172.21.15.84:0/1944231737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d062d0 0x55d588d06710 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f69000090d0 tx=0x7f6900009100 comp rx=0 tx=0).stop 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.687+0000 7f691ad09b80 1 -- 172.21.15.84:0/1944231737 shutdown_connections 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.687+0000 7f691ad09b80 1 --2- 172.21.15.84:0/1944231737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d062d0 0x55d588d06710 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.687+0000 7f691ad09b80 1 --2- 172.21.15.84:0/1944231737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0bad0 0x55d588d0beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.687+0000 7f691ad09b80 1 --2- 172.21.15.84:0/1944231737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d0c3f0 0x55d588d05cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.687+0000 7f691ad09b80 1 -- 172.21.15.84:0/1944231737 >> 172.21.15.84:0/1944231737 conn(0x55d588c32cf0 msgr2=0x55d588d011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.687+0000 7f691ad09b80 1 -- 172.21.15.84:0/1944231737 shutdown_connections 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.687+0000 7f691ad09b80 1 -- 172.21.15.84:0/1944231737 wait complete. 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.687+0000 7f691ad09b80 1 -- start start 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.687+0000 7f691ad09b80 1 Processor -- start 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f691ad09b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d062d0 0x55d588d120f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f691ad09b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d0f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f691ad09b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d0c3f0 0x55d588d0f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f691ad09b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d588d12eb0 con 0x55d588d0bad0 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f691ad09b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d588bbe900 con 0x55d588d0c3f0 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f691ad09b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d588d13030 con 0x55d588d062d0 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f691ad09b80 10 client.0 _ll_register_callbacks cb 0x55d588bec530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:05.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f690da12700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d0f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f690da12700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d0f3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44740/0 (socket says 172.21.15.84:44740) 2024-04-26T18:27:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f690da12700 1 -- 172.21.15.84:0/2715727944 learned_addr learned my addr 172.21.15.84:0/2715727944 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f690e213700 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d062d0 0x55d588d120f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f690da12700 1 -- 172.21.15.84:0/2715727944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d062d0 msgr2=0x55d588d120f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f690ea14700 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d0c3f0 0x55d588d0f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f690da12700 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d062d0 0x55d588d120f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f690da12700 1 -- 172.21.15.84:0/2715727944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d0c3f0 msgr2=0x55d588d0f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f690da12700 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d0c3f0 0x55d588d0f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f690da12700 1 -- 172.21.15.84:0/2715727944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d588d0ff70 con 0x55d588d0bad0 2024-04-26T18:27:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f690ea14700 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d0c3f0 0x55d588d0f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f690da12700 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d0f3f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6904011770 tx=0x7f69040047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fc0ce57d779455 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f68f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:05.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f68f1ffb700 1 -- 172.21.15.84:0/2715727944 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904004a30 con 0x55d588d0bad0 2024-04-26T18:27:05.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f68f1ffb700 1 -- 172.21.15.84:0/2715727944 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f6904004bd0 con 0x55d588d0bad0 2024-04-26T18:27:05.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.688+0000 7f691ad09b80 1 -- 172.21.15.84:0/2715727944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6900004bf0 con 0x55d588d0bad0 2024-04-26T18:27:05.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.689+0000 7f68f1ffb700 1 -- 172.21.15.84:0/2715727944 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f6904004e10 con 0x55d588d0bad0 2024-04-26T18:27:05.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.689+0000 7f691ad09b80 10 client.15060 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:05.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.689+0000 7f691ad09b80 1 -- 172.21.15.84:0/2715727944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d588d1d910 con 0x55d588d0bad0 2024-04-26T18:27:05.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.689+0000 7f68f1ffb700 1 -- 172.21.15.84:0/2715727944 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6900004bf0 con 0x55d588d0bad0 2024-04-26T18:27:05.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.689+0000 7f68f1ffb700 1 -- 172.21.15.84:0/2715727944 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6904030070 con 0x55d588d0bad0 2024-04-26T18:27:05.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.689+0000 7f691ad09b80 10 client.15060 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:05.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.689+0000 7f691ad09b80 10 client.15060 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:27:05.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.689+0000 7f691ad09b80 20 client.15060 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:05.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.689+0000 7f691ad09b80 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d588d1ff80 0x55d588d213f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.689+0000 7f691ad09b80 4 client.15060 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:05.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.689+0000 7f691ad09b80 1 -- 172.21.15.84:0/2715727944 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d588c660d0 con 0x55d588d1ff80 2024-04-26T18:27:05.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.689+0000 7f690e213700 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d588d1ff80 0x55d588d213f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.690+0000 7f690e213700 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d588d1ff80 0x55d588d213f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a91b5ff2e405cfe5 server_cookie=79125aff63e0d0ca in_seq=0 out_seq=0 2024-04-26T18:27:05.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.690+0000 7f68f1ffb700 10 client.15060 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:27:05.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f68f1ffb700 1 -- 172.21.15.84:0/2715727944 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d588c660d0 con 0x55d588d1ff80 2024-04-26T18:27:05.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f68f1ffb700 10 client.15060 handle_command_reply: tid=0 2024-04-26T18:27:05.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f691ad09b80 1 client.15060 shutdown 2024-04-26T18:27:05.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f691ad09b80 1 -- 172.21.15.84:0/2715727944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 msgr2=0x55d588d0f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:05.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f691ad09b80 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d0f3f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6904011770 tx=0x7f69040047c0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f691ad09b80 1 -- 172.21.15.84:0/2715727944 shutdown_connections 2024-04-26T18:27:05.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f691ad09b80 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d588d1ff80 0x55d588d213f0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f691ad09b80 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d0c3f0 0x55d588d0f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f691ad09b80 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d062d0 0x55d588d120f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f68f1ffb700 0 client.15060 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:27:05.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f691ad09b80 1 --2- 172.21.15.84:0/2715727944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d0f3f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f691ad09b80 1 -- 172.21.15.84:0/2715727944 >> 172.21.15.84:0/2715727944 conn(0x55d588c32cf0 msgr2=0x55d588d00330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:05.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f691ad09b80 1 -- 172.21.15.84:0/2715727944 shutdown_connections 2024-04-26T18:27:05.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f691ad09b80 1 -- 172.21.15.84:0/2715727944 wait complete. 2024-04-26T18:27:05.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.691+0000 7f691ad09b80 20 client.15060 trim_cache size 0 max 16384 2024-04-26T18:27:05.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.711+0000 7f691ad09b80 1 Processor -- start 2024-04-26T18:27:05.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.711+0000 7f691ad09b80 1 -- start start 2024-04-26T18:27:05.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.711+0000 7f691ad09b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d062d0 0x55d588d05cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.711+0000 7f691ad09b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d0dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.711+0000 7f691ad09b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0c3f0 0x55d588d0e220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.711+0000 7f691ad09b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d588d1f730 con 0x55d588d0bad0 2024-04-26T18:27:05.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.711+0000 7f691ad09b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d588d1fe00 con 0x55d588d062d0 2024-04-26T18:27:05.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.711+0000 7f691ad09b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d588d204d0 con 0x55d588d0c3f0 2024-04-26T18:27:05.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f690da12700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d0dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f690da12700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d0dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44748/0 (socket says 172.21.15.84:44748) 2024-04-26T18:27:05.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f690da12700 1 -- 172.21.15.84:0/209924834 learned_addr learned my addr 172.21.15.84:0/209924834 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:05.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f690e213700 1 --2- 172.21.15.84:0/209924834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d062d0 0x55d588d05cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f690ea14700 1 --2- 172.21.15.84:0/209924834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0c3f0 0x55d588d0e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f690da12700 1 -- 172.21.15.84:0/209924834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0c3f0 msgr2=0x55d588d0e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:05.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f690da12700 1 --2- 172.21.15.84:0/209924834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0c3f0 0x55d588d0e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f690da12700 1 -- 172.21.15.84:0/209924834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d062d0 msgr2=0x55d588d05cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:05.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f690da12700 1 --2- 172.21.15.84:0/209924834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d062d0 0x55d588d05cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f690da12700 1 -- 172.21.15.84:0/209924834 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d588d0d960 con 0x55d588d0bad0 2024-04-26T18:27:05.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f690ea14700 1 --2- 172.21.15.84:0/209924834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0c3f0 0x55d588d0e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:05.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f690e213700 1 --2- 172.21.15.84:0/209924834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d062d0 0x55d588d05cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:05.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f690da12700 1 --2- 172.21.15.84:0/209924834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d0dce0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6904004790 tx=0x7f6904010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b15e1db06e7be55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:05.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f68f3fff700 1 -- 172.21.15.84:0/209924834 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f690401f640 con 0x55d588d0bad0 2024-04-26T18:27:05.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f68f3fff700 1 -- 172.21.15.84:0/209924834 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f690402d070 con 0x55d588d0bad0 2024-04-26T18:27:05.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.712+0000 7f68f3fff700 1 -- 172.21.15.84:0/209924834 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904005530 con 0x55d588d0bad0 2024-04-26T18:27:05.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.713+0000 7f691ad09b80 1 -- 172.21.15.84:0/209924834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 msgr2=0x55d588d0dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:05.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.713+0000 7f691ad09b80 1 --2- 172.21.15.84:0/209924834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d0dce0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6904004790 tx=0x7f6904010a40 comp rx=0 tx=0).stop 2024-04-26T18:27:05.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.713+0000 7f691ad09b80 1 -- 172.21.15.84:0/209924834 shutdown_connections 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.713+0000 7f691ad09b80 1 --2- 172.21.15.84:0/209924834 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0c3f0 0x55d588d0e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.713+0000 7f691ad09b80 1 --2- 172.21.15.84:0/209924834 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d062d0 0x55d588d05cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.713+0000 7f691ad09b80 1 --2- 172.21.15.84:0/209924834 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d0dce0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.713+0000 7f691ad09b80 1 -- 172.21.15.84:0/209924834 >> 172.21.15.84:0/209924834 conn(0x55d588c32cf0 msgr2=0x55d588d09620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.713+0000 7f691ad09b80 1 -- 172.21.15.84:0/209924834 shutdown_connections 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.713+0000 7f691ad09b80 1 -- 172.21.15.84:0/209924834 wait complete. 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f691ad09b80 1 -- start start 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f691ad09b80 1 Processor -- start 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f691ad09b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d062d0 0x55d588d1b630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f691ad09b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d1bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f691ad09b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0c3f0 0x55d588d8a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f691ad09b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d588d21270 con 0x55d588d0bad0 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f691ad09b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d588d20ba0 con 0x55d588d062d0 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f691ad09b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d588d21940 con 0x55d588d0c3f0 2024-04-26T18:27:05.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f690da12700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d1bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f690da12700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d1bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44762/0 (socket says 172.21.15.84:44762) 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f690da12700 1 -- 172.21.15.84:0/66273426 learned_addr learned my addr 172.21.15.84:0/66273426 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f691ad09b80 10 client.0 _ll_register_callbacks cb 0x55d588bec530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f690e213700 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d062d0 0x55d588d1b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.714+0000 7f690ea14700 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0c3f0 0x55d588d8a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f690da12700 1 -- 172.21.15.84:0/66273426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0c3f0 msgr2=0x55d588d8a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f690da12700 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0c3f0 0x55d588d8a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f690da12700 1 -- 172.21.15.84:0/66273426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d062d0 msgr2=0x55d588d1b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f690da12700 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d062d0 0x55d588d1b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f690da12700 1 -- 172.21.15.84:0/66273426 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d588d8a6a0 con 0x55d588d0bad0 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f690e213700 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d062d0 0x55d588d1b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f690da12700 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d1bb70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f690401fca0 tx=0x7f6904004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=168e50c5d8790406 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f68f1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:05.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f68f1ffb700 1 -- 172.21.15.84:0/66273426 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6904010e50 con 0x55d588d0bad0 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f68f1ffb700 1 -- 172.21.15.84:0/66273426 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f6904005340 con 0x55d588d0bad0 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f691ad09b80 1 -- 172.21.15.84:0/66273426 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f690402d070 con 0x55d588d0bad0 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f68f1ffb700 1 -- 172.21.15.84:0/66273426 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f690404b030 con 0x55d588d0bad0 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f691ad09b80 10 client.15066 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f691ad09b80 1 -- 172.21.15.84:0/66273426 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55d588d8b980 con 0x55d588d0bad0 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.715+0000 7f68f1ffb700 1 -- 172.21.15.84:0/66273426 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f690402d070 con 0x55d588d0bad0 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.716+0000 7f68f1ffb700 1 -- 172.21.15.84:0/66273426 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f6904030040 con 0x55d588d0bad0 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.716+0000 7f691ad09b80 10 client.15066 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.716+0000 7f691ad09b80 10 client.15066 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.716+0000 7f691ad09b80 20 client.15066 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.716+0000 7f691ad09b80 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d588d90130 0x55d588d8ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.716+0000 7f691ad09b80 4 client.15066 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.716+0000 7f691ad09b80 1 -- 172.21.15.84:0/66273426 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "status"}) -- 0x55d588c60170 con 0x55d588d90130 2024-04-26T18:27:05.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.716+0000 7f690e213700 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d588d90130 0x55d588d8ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.716+0000 7f690e213700 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d588d90130 0x55d588d8ee40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6067dd27045de40 server_cookie=df9dfca97bf93b5c in_seq=0 out_seq=0 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.716+0000 7f68f1ffb700 10 client.15066 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f68f1ffb700 1 -- 172.21.15.84:0/66273426 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d588c60170 con 0x55d588d90130 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f68f1ffb700 10 client.15066 handle_command_reply: tid=0 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f691ad09b80 1 client.15066 shutdown 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f691ad09b80 1 -- 172.21.15.84:0/66273426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 msgr2=0x55d588d1bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f691ad09b80 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d1bb70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f690401fca0 tx=0x7f6904004aa0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f691ad09b80 1 -- 172.21.15.84:0/66273426 shutdown_connections 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f691ad09b80 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d588d90130 0x55d588d8ee40 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f691ad09b80 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d588d0c3f0 0x55d588d8a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f691ad09b80 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d588d062d0 0x55d588d1b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f68f1ffb700 0 client.15066 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f691ad09b80 1 --2- 172.21.15.84:0/66273426 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d588d0bad0 0x55d588d1bb70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f691ad09b80 1 -- 172.21.15.84:0/66273426 >> 172.21.15.84:0/66273426 conn(0x55d588c32cf0 msgr2=0x55d588d0db70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:05.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f691ad09b80 1 -- 172.21.15.84:0/66273426 shutdown_connections 2024-04-26T18:27:05.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f691ad09b80 1 -- 172.21.15.84:0/66273426 wait complete. 2024-04-26T18:27:05.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f691ad09b80 20 client.15066 trim_cache size 0 max 16384 2024-04-26T18:27:05.723 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:27:05.723 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:27:05.723 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2024-04-26T18:27:05.723 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24434, 2024-04-26T18:27:05.723 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:27:05.723 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:27:05.723 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:27:05.723 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 162.76639639800001, 2024-04-26T18:27:05.724 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 28, 2024-04-26T18:27:05.724 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 93, 2024-04-26T18:27:05.724 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 89, 2024-04-26T18:27:05.724 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 177.023650395 2024-04-26T18:27:05.724 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:27:05.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f6910477700 1 -- 172.21.15.84:0/2117625777 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f68f40a3c40 msgr2=0x7f68f40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:05.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f6910477700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f68f40a3c40 0x7f68f40a6100 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f690807ab80 tx=0x7f690400b410 comp rx=0 tx=0).stop 2024-04-26T18:27:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f6910477700 1 -- 172.21.15.84:0/2117625777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6908103470 msgr2=0x7f690819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.717+0000 7f6910477700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6908103470 0x7f690819f6a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f68f800c9b0 tx=0x7f68f800ccc0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.718+0000 7f6910477700 1 -- 172.21.15.84:0/2117625777 shutdown_connections 2024-04-26T18:27:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.718+0000 7f6910477700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f68f40a3c40 0x7f68f40a6100 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.718+0000 7f6910477700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6908103470 0x7f690819f6a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.718+0000 7f6910477700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f69081094c0 0x7f6908199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.718+0000 7f6910477700 1 --2- 172.21.15.84:0/2117625777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6908103e30 0x7f690819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.718+0000 7f6910477700 1 -- 172.21.15.84:0/2117625777 >> 172.21.15.84:0/2117625777 conn(0x7f690807a580 msgr2=0x7f6908102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.718+0000 7f6910477700 1 -- 172.21.15.84:0/2117625777 shutdown_connections 2024-04-26T18:27:05.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:05.718+0000 7f6910477700 1 -- 172.21.15.84:0/2117625777 wait complete. 2024-04-26T18:27:06.530 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T18:27:06.530 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T18:27:06.542 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi084.front.sepia.ceph.com 2024-04-26T18:27:06.542 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T18:27:06.543 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:06 smithi084 ceph-mon[29141]: pgmap v383: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:06.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:06 smithi103 ceph-mon[32727]: pgmap v383: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:06.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:06 smithi174 ceph-mon[32705]: pgmap v383: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:06.782 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:08.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.499+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/22393740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08103580 msgr2=0x7fbc081039e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:08.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.499+0000 7fbc0f3fe700 1 --2- 172.21.15.84:0/22393740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08103580 0x7fbc081039e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fbc040099c0 tx=0x7fbc04009cd0 comp rx=0 tx=0).stop 2024-04-26T18:27:08.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.499+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/22393740 shutdown_connections 2024-04-26T18:27:08.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.499+0000 7fbc0f3fe700 1 --2- 172.21.15.84:0/22393740 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc08103f20 0x7fbc0810a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:08.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.499+0000 7fbc0f3fe700 1 --2- 172.21.15.84:0/22393740 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc08102380 0x7fbc081027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:08.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.499+0000 7fbc0f3fe700 1 --2- 172.21.15.84:0/22393740 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08103580 0x7fbc081039e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:08.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.499+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/22393740 >> 172.21.15.84:0/22393740 conn(0x7fbc080fd920 msgr2=0x7fbc080ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:08.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.499+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/22393740 shutdown_connections 2024-04-26T18:27:08.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.499+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/22393740 wait complete. 2024-04-26T18:27:08.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0f3fe700 1 Processor -- start 2024-04-26T18:27:08.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0f3fe700 1 -- start start 2024-04-26T18:27:08.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0f3fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc08102380 0x7fbc081062c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0f3fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08103580 0x7fbc08104960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0f3fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc08103f20 0x7fbc08104ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0f3fe700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0810de90 con 0x7fbc08103580 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0f3fe700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0810dd10 con 0x7fbc08102380 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0f3fe700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc0810e010 con 0x7fbc08103f20 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0c999700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08103580 0x7fbc08104960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0c999700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08103580 0x7fbc08104960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52618/0 (socket says 172.21.15.84:52618) 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0c999700 1 -- 172.21.15.84:0/3423561112 learned_addr learned my addr 172.21.15.84:0/3423561112 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0d19a700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc08102380 0x7fbc081062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0d99b700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc08103f20 0x7fbc08104ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0c999700 1 -- 172.21.15.84:0/3423561112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc08103f20 msgr2=0x7fbc08104ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0c999700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc08103f20 0x7fbc08104ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0c999700 1 -- 172.21.15.84:0/3423561112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc08102380 msgr2=0x7fbc081062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:08.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0c999700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc08102380 0x7fbc081062c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0c999700 1 -- 172.21.15.84:0/3423561112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc081053e0 con 0x7fbc08103580 2024-04-26T18:27:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0d99b700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc08103f20 0x7fbc08104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.500+0000 7fbc0c999700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08103580 0x7fbc08104960 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbc04003790 tx=0x7fbc04003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.501+0000 7fbbfe7fc700 1 -- 172.21.15.84:0/3423561112 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc04027070 con 0x7fbc08103580 2024-04-26T18:27:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.501+0000 7fbbfe7fc700 1 -- 172.21.15.84:0/3423561112 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc04003f80 con 0x7fbc08103580 2024-04-26T18:27:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.501+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/3423561112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc08105610 con 0x7fbc08103580 2024-04-26T18:27:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.501+0000 7fbbfe7fc700 1 -- 172.21.15.84:0/3423561112 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0400f460 con 0x7fbc08103580 2024-04-26T18:27:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.501+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/3423561112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc081a80b0 con 0x7fbc08103580 2024-04-26T18:27:08.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.502+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/3423561112 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc0806a110 con 0x7fbc08103580 2024-04-26T18:27:08.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.503+0000 7fbbfe7fc700 1 -- 172.21.15.84:0/3423561112 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbc0402a070 con 0x7fbc08103580 2024-04-26T18:27:08.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.503+0000 7fbbfe7fc700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fbbf40a3cc0 0x7fbbf40a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:08.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.503+0000 7fbc0d19a700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fbbf40a3cc0 0x7fbbf40a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:08.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.504+0000 7fbbfe7fc700 1 -- 172.21.15.84:0/3423561112 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fbc040d1800 con 0x7fbc08103580 2024-04-26T18:27:08.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.504+0000 7fbc0d19a700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fbbf40a3cc0 0x7fbbf40a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8006fd0 tx=0x7fbbf8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:08.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.507+0000 7fbbfe7fc700 1 -- 172.21.15.84:0/3423561112 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc04013070 con 0x7fbc08103580 2024-04-26T18:27:08.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:08 smithi084 ceph-mon[29141]: pgmap v384: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:08.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:08 smithi103 ceph-mon[32727]: pgmap v384: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:08.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:08 smithi174 ceph-mon[32705]: pgmap v384: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:08.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.726+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/3423561112 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fbc080ffb60 con 0x7fbbf40a3cc0 2024-04-26T18:27:08.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.727+0000 7fbbfe7fc700 1 -- 172.21.15.84:0/3423561112 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fbc080ffb60 con 0x7fbbf40a3cc0 2024-04-26T18:27:08.727 INFO:teuthology.orchestra.run.smithi084.stdout:Backend: cephadm 2024-04-26T18:27:08.727 INFO:teuthology.orchestra.run.smithi084.stdout:Available: Yes 2024-04-26T18:27:08.727 INFO:teuthology.orchestra.run.smithi084.stdout:Paused: No 2024-04-26T18:27:08.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.729+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/3423561112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fbbf40a3cc0 msgr2=0x7fbbf40a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:08.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.729+0000 7fbc0f3fe700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fbbf40a3cc0 0x7fbbf40a6180 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fbbf8006fd0 tx=0x7fbbf8008040 comp rx=0 tx=0).stop 2024-04-26T18:27:08.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.729+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/3423561112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08103580 msgr2=0x7fbc08104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:08.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.729+0000 7fbc0f3fe700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08103580 0x7fbc08104960 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fbc04003790 tx=0x7fbc04003a10 comp rx=0 tx=0).stop 2024-04-26T18:27:08.729 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.729+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/3423561112 shutdown_connections 2024-04-26T18:27:08.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.729+0000 7fbc0f3fe700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fbbf40a3cc0 0x7fbbf40a6180 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:08.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.729+0000 7fbc0f3fe700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc08103f20 0x7fbc08104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:08.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.729+0000 7fbc0f3fe700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc08102380 0x7fbc081062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:08.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.729+0000 7fbc0f3fe700 1 --2- 172.21.15.84:0/3423561112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc08103580 0x7fbc08104960 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:08.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.729+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/3423561112 >> 172.21.15.84:0/3423561112 conn(0x7fbc080fd920 msgr2=0x7fbc080ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:08.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.729+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/3423561112 shutdown_connections 2024-04-26T18:27:08.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:08.729+0000 7fbc0f3fe700 1 -- 172.21.15.84:0/3423561112 wait complete. 2024-04-26T18:27:09.441 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T18:27:09.682 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:10.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:10 smithi084 ceph-mon[29141]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:10.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:10 smithi084 ceph-mon[29141]: pgmap v385: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:10.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:10 smithi103 ceph-mon[32727]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:10.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:10 smithi103 ceph-mon[32727]: pgmap v385: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:10 smithi174 ceph-mon[32705]: from='client.15072 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:10 smithi174 ceph-mon[32705]: pgmap v385: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:11.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.385+0000 7f0288d8f700 1 -- 172.21.15.84:0/2522832537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0284105110 msgr2=0x7f028410b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:11.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.385+0000 7f0288d8f700 1 --2- 172.21.15.84:0/2522832537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0284105110 0x7f028410b9c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f027c00b0a0 tx=0x7f027c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:27:11.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.386+0000 7f0288d8f700 1 -- 172.21.15.84:0/2522832537 shutdown_connections 2024-04-26T18:27:11.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.386+0000 7f0288d8f700 1 --2- 172.21.15.84:0/2522832537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0284105110 0x7f028410b9c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:11.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.386+0000 7f0288d8f700 1 --2- 172.21.15.84:0/2522832537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0284101790 0x7f0284101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:11.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.386+0000 7f0288d8f700 1 --2- 172.21.15.84:0/2522832537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0284104770 0x7f0284104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:11.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.386+0000 7f0288d8f700 1 -- 172.21.15.84:0/2522832537 >> 172.21.15.84:0/2522832537 conn(0x7f02840fd0f0 msgr2=0x7f02840ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:11.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.386+0000 7f0288d8f700 1 -- 172.21.15.84:0/2522832537 shutdown_connections 2024-04-26T18:27:11.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.386+0000 7f0288d8f700 1 -- 172.21.15.84:0/2522832537 wait complete. 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.386+0000 7f0288d8f700 1 Processor -- start 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.386+0000 7f0288d8f700 1 -- start start 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.386+0000 7f0288d8f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0284101790 0x7f028419b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.386+0000 7f0288d8f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0284104770 0x7f028419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0288d8f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0284105110 0x7f02841a2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0288d8f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f028410ef40 con 0x7f0284104770 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0288d8f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f028410edc0 con 0x7f0284101790 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0288d8f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f028410f0c0 con 0x7f0284105110 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0281d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0284104770 0x7f028419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0281d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0284104770 0x7f028419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52636/0 (socket says 172.21.15.84:52636) 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0281d9b700 1 -- 172.21.15.84:0/3500222805 learned_addr learned my addr 172.21.15.84:0/3500222805 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0282d9d700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0284105110 0x7f02841a2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f028259c700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0284101790 0x7f028419b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:11.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0281d9b700 1 -- 172.21.15.84:0/3500222805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0284105110 msgr2=0x7f02841a2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0281d9b700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0284105110 0x7f02841a2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0281d9b700 1 -- 172.21.15.84:0/3500222805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0284101790 msgr2=0x7f028419b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0281d9b700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0284101790 0x7f028419b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0281d9b700 1 -- 172.21.15.84:0/3500222805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02841a3390 con 0x7f0284104770 2024-04-26T18:27:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0281d9b700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0284104770 0x7f028419bc70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f027800d980 tx=0x7f027800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f02737fe700 1 -- 172.21.15.84:0/3500222805 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0278016070 con 0x7f0284104770 2024-04-26T18:27:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f02737fe700 1 -- 172.21.15.84:0/3500222805 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0278009d80 con 0x7f0284104770 2024-04-26T18:27:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0288d8f700 1 -- 172.21.15.84:0/3500222805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02841a3680 con 0x7f0284104770 2024-04-26T18:27:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f02737fe700 1 -- 172.21.15.84:0/3500222805 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027800f3c0 con 0x7f0284104770 2024-04-26T18:27:11.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.387+0000 7f0288d8f700 1 -- 172.21.15.84:0/3500222805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02841a3c10 con 0x7f0284104770 2024-04-26T18:27:11.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.388+0000 7f0288d8f700 1 -- 172.21.15.84:0/3500222805 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f028406a110 con 0x7f0284104770 2024-04-26T18:27:11.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.389+0000 7f02737fe700 1 -- 172.21.15.84:0/3500222805 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0278019070 con 0x7f0284104770 2024-04-26T18:27:11.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.390+0000 7f02737fe700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f026c0a3cc0 0x7f026c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:11.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.390+0000 7f028259c700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f026c0a3cc0 0x7f026c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:11.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.390+0000 7f02737fe700 1 -- 172.21.15.84:0/3500222805 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f02780cfe80 con 0x7f0284104770 2024-04-26T18:27:11.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.391+0000 7f028259c700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f026c0a3cc0 0x7f026c0a6180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f028406b950 tx=0x7f027400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:11.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.394+0000 7f02737fe700 1 -- 172.21.15.84:0/3500222805 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f027809c7d0 con 0x7f0284104770 2024-04-26T18:27:11.604 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.602+0000 7f0288d8f700 1 -- 172.21.15.84:0/3500222805 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f0284103c80 con 0x7f026c0a3cc0 2024-04-26T18:27:11.617 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.616+0000 7f02737fe700 1 -- 172.21.15.84:0/3500222805 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7f0284103c80 con 0x7f026c0a3cc0 2024-04-26T18:27:11.617 INFO:teuthology.orchestra.run.smithi084.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T18:27:11.617 INFO:teuthology.orchestra.run.smithi084.stdout:mds.a smithi084 running (3m) 38s ago 3m 18.3M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b c63b0eed8f43 2024-04-26T18:27:11.617 INFO:teuthology.orchestra.run.smithi084.stdout:mds.b smithi103 running (3m) 49s ago 3m 29.9M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 9008ada52e45 2024-04-26T18:27:11.617 INFO:teuthology.orchestra.run.smithi084.stdout:mds.c smithi174 running (3m) 49s ago 3m 19.4M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 8cc2cd012148 2024-04-26T18:27:11.617 INFO:teuthology.orchestra.run.smithi084.stdout:mds.d smithi084 running (3m) 38s ago 3m 16.1M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 8acd3260a497 2024-04-26T18:27:11.617 INFO:teuthology.orchestra.run.smithi084.stdout:mds.e smithi103 running (3m) 49s ago 3m 19.8M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 3c7b2ab98ba0 2024-04-26T18:27:11.618 INFO:teuthology.orchestra.run.smithi084.stdout:mds.f smithi174 running (3m) 49s ago 3m 20.3M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 7321510a55a7 2024-04-26T18:27:11.618 INFO:teuthology.orchestra.run.smithi084.stdout:mds.g smithi084 running (3m) 38s ago 3m 19.4M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b da32549d9cfa 2024-04-26T18:27:11.618 INFO:teuthology.orchestra.run.smithi084.stdout:mds.h smithi103 running (3m) 49s ago 3m 18.9M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 90fc70c84d82 2024-04-26T18:27:11.618 INFO:teuthology.orchestra.run.smithi084.stdout:mds.i smithi174 running (3m) 49s ago 3m 20.2M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b d8edfcd179c3 2024-04-26T18:27:11.618 INFO:teuthology.orchestra.run.smithi084.stdout:mds.j smithi084 running (3m) 38s ago 3m 14.5M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b b83ce86a9044 2024-04-26T18:27:11.618 INFO:teuthology.orchestra.run.smithi084.stdout:mds.k smithi103 running (3m) 49s ago 3m 20.8M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 4956ca44d39c 2024-04-26T18:27:11.618 INFO:teuthology.orchestra.run.smithi084.stdout:mds.l smithi174 running (3m) 49s ago 3m 19.3M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b e004f7cad9de 2024-04-26T18:27:11.618 INFO:teuthology.orchestra.run.smithi084.stdout:mgr.x smithi084 *:9283,8765 running (10m) 38s ago 10m 516M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b aab0ee9dc54e 2024-04-26T18:27:11.618 INFO:teuthology.orchestra.run.smithi084.stdout:mgr.y smithi103 *:8443,8765 running (8m) 49s ago 8m 454M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b e973bf7c756d 2024-04-26T18:27:11.618 INFO:teuthology.orchestra.run.smithi084.stdout:mgr.z smithi174 *:8443,8765 running (8m) 49s ago 8m 453M - 19.0.0-3267-gbc94a2a9 3a6bde214c5b 04b25399f61f 2024-04-26T18:27:11.618 INFO:teuthology.orchestra.run.smithi084.stdout:mon.a smithi084 running (10m) 38s ago 11m 53.6M 2048M 19.0.0-3267-gbc94a2a9 3a6bde214c5b b25e6427f5ba 2024-04-26T18:27:11.618 INFO:teuthology.orchestra.run.smithi084.stdout:mon.b smithi103 running (9m) 49s ago 9m 43.5M 2048M 19.0.0-3267-gbc94a2a9 3a6bde214c5b d58834cea0b8 2024-04-26T18:27:11.618 INFO:teuthology.orchestra.run.smithi084.stdout:mon.c smithi174 running (9m) 49s ago 9m 41.7M 2048M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 8f4977f04f29 2024-04-26T18:27:11.619 INFO:teuthology.orchestra.run.smithi084.stdout:osd.0 smithi084 running (8m) 38s ago 8m 57.2M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b e7a8a1969e64 2024-04-26T18:27:11.619 INFO:teuthology.orchestra.run.smithi084.stdout:osd.1 smithi103 running (8m) 49s ago 8m 62.1M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b d09f6fd496d1 2024-04-26T18:27:11.619 INFO:teuthology.orchestra.run.smithi084.stdout:osd.2 smithi174 running (7m) 49s ago 7m 47.8M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b a5b9bff37f57 2024-04-26T18:27:11.619 INFO:teuthology.orchestra.run.smithi084.stdout:osd.3 smithi084 running (7m) 38s ago 7m 45.3M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b f3c1755e9995 2024-04-26T18:27:11.619 INFO:teuthology.orchestra.run.smithi084.stdout:osd.4 smithi103 running (6m) 49s ago 6m 43.8M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b bee3b91c288b 2024-04-26T18:27:11.619 INFO:teuthology.orchestra.run.smithi084.stdout:osd.5 smithi174 running (6m) 49s ago 6m 58.4M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 261f011c9b75 2024-04-26T18:27:11.619 INFO:teuthology.orchestra.run.smithi084.stdout:osd.6 smithi084 running (6m) 38s ago 6m 42.5M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 1e7566819412 2024-04-26T18:27:11.619 INFO:teuthology.orchestra.run.smithi084.stdout:osd.7 smithi103 running (5m) 49s ago 5m 57.0M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b d6bde4067942 2024-04-26T18:27:11.619 INFO:teuthology.orchestra.run.smithi084.stdout:osd.8 smithi174 running (5m) 49s ago 5m 55.2M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b ae9212cbe92e 2024-04-26T18:27:11.619 INFO:teuthology.orchestra.run.smithi084.stdout:osd.9 smithi084 running (4m) 38s ago 4m 53.4M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 1a7fd3385d64 2024-04-26T18:27:11.619 INFO:teuthology.orchestra.run.smithi084.stdout:osd.10 smithi103 running (4m) 49s ago 4m 56.2M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b e17472705489 2024-04-26T18:27:11.619 INFO:teuthology.orchestra.run.smithi084.stdout:osd.11 smithi174 running (3m) 49s ago 3m 52.3M 4272M 19.0.0-3267-gbc94a2a9 3a6bde214c5b 24f8d78d28c6 2024-04-26T18:27:11.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.618+0000 7f0288d8f700 1 -- 172.21.15.84:0/3500222805 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f026c0a3cc0 msgr2=0x7f026c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:11.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.618+0000 7f0288d8f700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f026c0a3cc0 0x7f026c0a6180 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f028406b950 tx=0x7f027400b410 comp rx=0 tx=0).stop 2024-04-26T18:27:11.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.618+0000 7f0288d8f700 1 -- 172.21.15.84:0/3500222805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0284104770 msgr2=0x7f028419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:11.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.618+0000 7f0288d8f700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0284104770 0x7f028419bc70 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f027800d980 tx=0x7f027800dc90 comp rx=0 tx=0).stop 2024-04-26T18:27:11.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.618+0000 7f0288d8f700 1 -- 172.21.15.84:0/3500222805 shutdown_connections 2024-04-26T18:27:11.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.618+0000 7f0288d8f700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f026c0a3cc0 0x7f026c0a6180 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:11.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.618+0000 7f0288d8f700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0284105110 0x7f02841a2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:11.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.618+0000 7f0288d8f700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0284101790 0x7f028419b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:11.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.618+0000 7f0288d8f700 1 --2- 172.21.15.84:0/3500222805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0284104770 0x7f028419bc70 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:11.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.618+0000 7f0288d8f700 1 -- 172.21.15.84:0/3500222805 >> 172.21.15.84:0/3500222805 conn(0x7f02840fd0f0 msgr2=0x7f0284102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:11.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.618+0000 7f0288d8f700 1 -- 172.21.15.84:0/3500222805 shutdown_connections 2024-04-26T18:27:11.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:11.618+0000 7f0288d8f700 1 -- 172.21.15.84:0/3500222805 wait complete. 2024-04-26T18:27:12.457 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T18:27:12.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:12 smithi084 ceph-mon[29141]: pgmap v386: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:12.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:12 smithi103 ceph-mon[32727]: pgmap v386: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:12.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:12 smithi174 ceph-mon[32705]: pgmap v386: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:12.697 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:13.511 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:13 smithi084 ceph-mon[29141]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:13.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:13 smithi103 ceph-mon[32727]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:13 smithi174 ceph-mon[32705]: from='client.15078 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.494+0000 7f1295da1700 1 -- 172.21.15.84:0/839309775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290103fa0 msgr2=0x7f1290106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.494+0000 7f1295da1700 1 --2- 172.21.15.84:0/839309775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290103fa0 0x7f1290106390 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1280009990 tx=0x7f1280009ca0 comp rx=0 tx=0).stop 2024-04-26T18:27:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 -- 172.21.15.84:0/839309775 shutdown_connections 2024-04-26T18:27:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 --2- 172.21.15.84:0/839309775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290103fa0 0x7f1290106390 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 --2- 172.21.15.84:0/839309775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1290109200 0x7f129010b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 --2- 172.21.15.84:0/839309775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12901068d0 0x7f1290108cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 -- 172.21.15.84:0/839309775 >> 172.21.15.84:0/839309775 conn(0x7f12900fd910 msgr2=0x7f12900ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 -- 172.21.15.84:0/839309775 shutdown_connections 2024-04-26T18:27:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 -- 172.21.15.84:0/839309775 wait complete. 2024-04-26T18:27:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 Processor -- start 2024-04-26T18:27:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 -- start start 2024-04-26T18:27:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1290103fa0 0x7f1290199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12901068d0 0x7f1290199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290109200 0x7f12901a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f129010eef0 con 0x7f1290109200 2024-04-26T18:27:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f129010ed70 con 0x7f1290103fa0 2024-04-26T18:27:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.495+0000 7f1295da1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f129010f070 con 0x7f12901068d0 2024-04-26T18:27:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290109200 0x7f12901a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290109200 0x7f12901a0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52662/0 (socket says 172.21.15.84:52662) 2024-04-26T18:27:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128ffff700 1 -- 172.21.15.84:0/3636763300 learned_addr learned my addr 172.21.15.84:0/3636763300 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128effd700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12901068d0 0x7f1290199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128f7fe700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1290103fa0 0x7f1290199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128ffff700 1 -- 172.21.15.84:0/3636763300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12901068d0 msgr2=0x7f1290199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128ffff700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12901068d0 0x7f1290199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128ffff700 1 -- 172.21.15.84:0/3636763300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1290103fa0 msgr2=0x7f1290199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128ffff700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1290103fa0 0x7f1290199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128ffff700 1 -- 172.21.15.84:0/3636763300 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12901a1190 con 0x7f1290109200 2024-04-26T18:27:14.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128f7fe700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1290103fa0 0x7f1290199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128ffff700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290109200 0x7f12901a0af0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1288007d90 tx=0x7f128800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128cff9700 1 -- 172.21.15.84:0/3636763300 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f128800ad00 con 0x7f1290109200 2024-04-26T18:27:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128cff9700 1 -- 172.21.15.84:0/3636763300 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1288004030 con 0x7f1290109200 2024-04-26T18:27:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f128cff9700 1 -- 172.21.15.84:0/3636763300 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1288012680 con 0x7f1290109200 2024-04-26T18:27:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f1295da1700 1 -- 172.21.15.84:0/3636763300 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12901a1480 con 0x7f1290109200 2024-04-26T18:27:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.496+0000 7f1295da1700 1 -- 172.21.15.84:0/3636763300 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12901a1970 con 0x7f1290109200 2024-04-26T18:27:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.497+0000 7f1295da1700 1 -- 172.21.15.84:0/3636763300 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f129006a110 con 0x7f1290109200 2024-04-26T18:27:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.498+0000 7f128cff9700 1 -- 172.21.15.84:0/3636763300 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f1288003a00 con 0x7f1290109200 2024-04-26T18:27:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.499+0000 7f128cff9700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12780a3c40 0x7f12780a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.499+0000 7f128f7fe700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12780a3c40 0x7f12780a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.499+0000 7f128cff9700 1 -- 172.21.15.84:0/3636763300 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f12880d06b0 con 0x7f1290109200 2024-04-26T18:27:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.499+0000 7f128f7fe700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12780a3c40 0x7f12780a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1280006010 tx=0x7f1280009510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.503+0000 7f128cff9700 1 -- 172.21.15.84:0/3636763300 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f128809d060 con 0x7f1290109200 2024-04-26T18:27:14.513 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:14 smithi084 ceph-mon[29141]: pgmap v387: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:14.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:14 smithi103 ceph-mon[32727]: pgmap v387: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:14.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:14 smithi174 ceph-mon[32705]: pgmap v387: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:14.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.710+0000 7f1295da1700 1 -- 172.21.15.84:0/3636763300 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f1290063860 con 0x7f12780a3c40 2024-04-26T18:27:14.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.716+0000 7f128cff9700 1 -- 172.21.15.84:0/3636763300 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f1290063860 con 0x7f12780a3c40 2024-04-26T18:27:14.717 INFO:teuthology.orchestra.run.smithi084.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T18:27:14.717 INFO:teuthology.orchestra.run.smithi084.stdout:mds.all 12/12 52s ago 3m smithi084=a;smithi084=d;smithi084=g;smithi084=j;smithi103=b;smithi103=e;smithi103=h;smithi103=k;smithi174=c;smithi174=f;smithi174=i;smithi174=l;count:12 2024-04-26T18:27:14.717 INFO:teuthology.orchestra.run.smithi084.stdout:mgr 3/3 52s ago 9m smithi084=x;smithi103=y;smithi174=z;count:3 2024-04-26T18:27:14.717 INFO:teuthology.orchestra.run.smithi084.stdout:mon 3/3 52s ago 9m smithi084:172.21.15.84=a;smithi103:172.21.15.103=b;smithi174:172.21.15.174=c;count:3 2024-04-26T18:27:14.717 INFO:teuthology.orchestra.run.smithi084.stdout:osd 12 52s ago - 2024-04-26T18:27:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.718+0000 7f1295da1700 1 -- 172.21.15.84:0/3636763300 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12780a3c40 msgr2=0x7f12780a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.718+0000 7f1295da1700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12780a3c40 0x7f12780a6100 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1280006010 tx=0x7f1280009510 comp rx=0 tx=0).stop 2024-04-26T18:27:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.718+0000 7f1295da1700 1 -- 172.21.15.84:0/3636763300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290109200 msgr2=0x7f12901a0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.718+0000 7f1295da1700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290109200 0x7f12901a0af0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1288007d90 tx=0x7f128800a340 comp rx=0 tx=0).stop 2024-04-26T18:27:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.718+0000 7f1295da1700 1 -- 172.21.15.84:0/3636763300 shutdown_connections 2024-04-26T18:27:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.718+0000 7f1295da1700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f12780a3c40 0x7f12780a6100 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.718+0000 7f1295da1700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1290109200 0x7f12901a0af0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.718+0000 7f1295da1700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1290103fa0 0x7f1290199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.718+0000 7f1295da1700 1 --2- 172.21.15.84:0/3636763300 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f12901068d0 0x7f1290199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.718+0000 7f1295da1700 1 -- 172.21.15.84:0/3636763300 >> 172.21.15.84:0/3636763300 conn(0x7f12900fd910 msgr2=0x7f1290104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:14.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.718+0000 7f1295da1700 1 -- 172.21.15.84:0/3636763300 shutdown_connections 2024-04-26T18:27:14.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:14.718+0000 7f1295da1700 1 -- 172.21.15.84:0/3636763300 wait complete. 2024-04-26T18:27:15.494 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T18:27:15.735 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:16.521 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:16 smithi084 ceph-mon[29141]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:16.521 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:16 smithi084 ceph-mon[29141]: pgmap v388: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:16.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:16 smithi103 ceph-mon[32727]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:16.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:16 smithi103 ceph-mon[32727]: pgmap v388: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:16.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:16 smithi174 ceph-mon[32705]: from='client.15084 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:16.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:16 smithi174 ceph-mon[32705]: pgmap v388: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:17.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.470+0000 7fb659038700 1 -- 172.21.15.84:0/1879707724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6541051a0 msgr2=0x7fb65410ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:17.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.470+0000 7fb659038700 1 --2- 172.21.15.84:0/1879707724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6541051a0 0x7fb65410ba50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00b0a0 tx=0x7fb64c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.470+0000 7fb659038700 1 -- 172.21.15.84:0/1879707724 shutdown_connections 2024-04-26T18:27:17.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.470+0000 7fb659038700 1 --2- 172.21.15.84:0/1879707724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6541051a0 0x7fb65410ba50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.470+0000 7fb659038700 1 --2- 172.21.15.84:0/1879707724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654101820 0x7fb654101c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.470+0000 7fb659038700 1 --2- 172.21.15.84:0/1879707724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb654104800 0x7fb654104c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.470+0000 7fb659038700 1 -- 172.21.15.84:0/1879707724 >> 172.21.15.84:0/1879707724 conn(0x7fb6540fd180 msgr2=0x7fb6540ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:17.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.470+0000 7fb659038700 1 -- 172.21.15.84:0/1879707724 shutdown_connections 2024-04-26T18:27:17.471 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.470+0000 7fb659038700 1 -- 172.21.15.84:0/1879707724 wait complete. 2024-04-26T18:27:17.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.472+0000 7fb659038700 1 Processor -- start 2024-04-26T18:27:17.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.472+0000 7fb659038700 1 -- start start 2024-04-26T18:27:17.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.472+0000 7fb659038700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654101820 0x7fb654199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.472+0000 7fb659038700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb654104800 0x7fb654199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.472+0000 7fb659038700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6541051a0 0x7fb6541a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.472+0000 7fb659038700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb65410eff0 con 0x7fb6541051a0 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.472+0000 7fb659038700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb65410ee70 con 0x7fb654104800 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.472+0000 7fb659038700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb65410f170 con 0x7fb654101820 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.472+0000 7fb65359e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6541051a0 0x7fb6541a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.472+0000 7fb65359e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6541051a0 0x7fb6541a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54014/0 (socket says 172.21.15.84:54014) 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.472+0000 7fb65359e700 1 -- 172.21.15.84:0/1915299135 learned_addr learned my addr 172.21.15.84:0/1915299135 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.472+0000 7fb65259c700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb654104800 0x7fb654199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.472+0000 7fb652d9d700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654101820 0x7fb654199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.473+0000 7fb65359e700 1 -- 172.21.15.84:0/1915299135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654101820 msgr2=0x7fb654199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.473+0000 7fb65359e700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654101820 0x7fb654199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.473+0000 7fb65359e700 1 -- 172.21.15.84:0/1915299135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb654104800 msgr2=0x7fb654199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:17.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.473+0000 7fb65359e700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb654104800 0x7fb654199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.473+0000 7fb65359e700 1 -- 172.21.15.84:0/1915299135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6541a1270 con 0x7fb6541051a0 2024-04-26T18:27:17.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.473+0000 7fb65359e700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6541051a0 0x7fb6541a0bd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00b070 tx=0x7fb64c004e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:17.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.473+0000 7fb63bfff700 1 -- 172.21.15.84:0/1915299135 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c021070 con 0x7fb6541051a0 2024-04-26T18:27:17.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.473+0000 7fb63bfff700 1 -- 172.21.15.84:0/1915299135 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c007c00 con 0x7fb6541051a0 2024-04-26T18:27:17.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.473+0000 7fb63bfff700 1 -- 172.21.15.84:0/1915299135 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c0047c0 con 0x7fb6541051a0 2024-04-26T18:27:17.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.473+0000 7fb659038700 1 -- 172.21.15.84:0/1915299135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6541a1500 con 0x7fb6541051a0 2024-04-26T18:27:17.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.473+0000 7fb659038700 1 -- 172.21.15.84:0/1915299135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6541a19f0 con 0x7fb6541051a0 2024-04-26T18:27:17.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.474+0000 7fb659038700 1 -- 172.21.15.84:0/1915299135 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb65406a110 con 0x7fb6541051a0 2024-04-26T18:27:17.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.476+0000 7fb63bfff700 1 -- 172.21.15.84:0/1915299135 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb64c003a00 con 0x7fb6541051a0 2024-04-26T18:27:17.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.476+0000 7fb63bfff700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb63c0a3cc0 0x7fb63c0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:17.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.476+0000 7fb652d9d700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb63c0a3cc0 0x7fb63c0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:17.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.476+0000 7fb63bfff700 1 -- 172.21.15.84:0/1915299135 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fb64c00f020 con 0x7fb6541051a0 2024-04-26T18:27:17.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.477+0000 7fb652d9d700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb63c0a3cc0 0x7fb63c0a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb65406b930 tx=0x7fb64400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:17.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.480+0000 7fb63bfff700 1 -- 172.21.15.84:0/1915299135 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb64c02a020 con 0x7fb6541051a0 2024-04-26T18:27:17.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.683+0000 7fb659038700 1 -- 172.21.15.84:0/1915299135 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7fb654063860 con 0x7fb63c0a3cc0 2024-04-26T18:27:17.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.685+0000 7fb63bfff700 1 -- 172.21.15.84:0/1915299135 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7fb654063860 con 0x7fb63c0a3cc0 2024-04-26T18:27:17.685 INFO:teuthology.orchestra.run.smithi084.stdout:HOST ADDR LABELS STATUS 2024-04-26T18:27:17.685 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084 172.21.15.84 2024-04-26T18:27:17.685 INFO:teuthology.orchestra.run.smithi084.stdout:smithi103 172.21.15.103 2024-04-26T18:27:17.685 INFO:teuthology.orchestra.run.smithi084.stdout:smithi174 172.21.15.174 2024-04-26T18:27:17.685 INFO:teuthology.orchestra.run.smithi084.stdout:3 hosts in cluster 2024-04-26T18:27:17.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.687+0000 7fb659038700 1 -- 172.21.15.84:0/1915299135 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb63c0a3cc0 msgr2=0x7fb63c0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:17.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.687+0000 7fb659038700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb63c0a3cc0 0x7fb63c0a6180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb65406b930 tx=0x7fb64400b410 comp rx=0 tx=0).stop 2024-04-26T18:27:17.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.687+0000 7fb659038700 1 -- 172.21.15.84:0/1915299135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6541051a0 msgr2=0x7fb6541a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:17.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.687+0000 7fb659038700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6541051a0 0x7fb6541a0bd0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fb64c00b070 tx=0x7fb64c004e40 comp rx=0 tx=0).stop 2024-04-26T18:27:17.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.687+0000 7fb659038700 1 -- 172.21.15.84:0/1915299135 shutdown_connections 2024-04-26T18:27:17.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.687+0000 7fb659038700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb63c0a3cc0 0x7fb63c0a6180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.687+0000 7fb659038700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb6541051a0 0x7fb6541a0bd0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.687+0000 7fb659038700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb654101820 0x7fb654199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.687+0000 7fb659038700 1 --2- 172.21.15.84:0/1915299135 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb654104800 0x7fb654199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:17.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.687+0000 7fb659038700 1 -- 172.21.15.84:0/1915299135 >> 172.21.15.84:0/1915299135 conn(0x7fb6540fd180 msgr2=0x7fb654107b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:17.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.687+0000 7fb659038700 1 -- 172.21.15.84:0/1915299135 shutdown_connections 2024-04-26T18:27:17.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:17.687+0000 7fb659038700 1 -- 172.21.15.84:0/1915299135 wait complete. 2024-04-26T18:27:18.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:18 smithi084 ceph-mon[29141]: pgmap v389: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:18.391 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T18:27:18.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:18 smithi103 ceph-mon[32727]: pgmap v389: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:18.629 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:18.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:18 smithi174 ceph-mon[32705]: pgmap v389: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:19.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:19 smithi084 ceph-mon[29141]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:19.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:19 smithi103 ceph-mon[32727]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:19.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:19 smithi174 ceph-mon[32705]: from='client.15090 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.380+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/3940272924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4d0103490 msgr2=0x7ff4d01038b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.380+0000 7ff4d6ac0700 1 --2- 172.21.15.84:0/3940272924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4d0103490 0x7ff4d01038b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff4c40099c0 tx=0x7ff4c4009cd0 comp rx=0 tx=0).stop 2024-04-26T18:27:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.380+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/3940272924 shutdown_connections 2024-04-26T18:27:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.380+0000 7ff4d6ac0700 1 --2- 172.21.15.84:0/3940272924 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4d0103490 0x7ff4d01038b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.380+0000 7ff4d6ac0700 1 --2- 172.21.15.84:0/3940272924 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4d0105030 0x7ff4d010b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:20.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.380+0000 7ff4d6ac0700 1 --2- 172.21.15.84:0/3940272924 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4d0104690 0x7ff4d0104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:20.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.380+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/3940272924 >> 172.21.15.84:0/3940272924 conn(0x7ff4d00fea30 msgr2=0x7ff4d0100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:20.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.380+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/3940272924 shutdown_connections 2024-04-26T18:27:20.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.381+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/3940272924 wait complete. 2024-04-26T18:27:20.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.381+0000 7ff4d6ac0700 1 Processor -- start 2024-04-26T18:27:20.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.381+0000 7ff4d6ac0700 1 -- start start 2024-04-26T18:27:20.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.381+0000 7ff4d6ac0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4d0103490 0x7ff4d0199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:20.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.381+0000 7ff4d6ac0700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4d0104690 0x7ff4d0199b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:20.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.381+0000 7ff4d6ac0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4d0105030 0x7ff4d01a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:20.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.381+0000 7ff4d6ac0700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d010ef80 con 0x7ff4d0103490 2024-04-26T18:27:20.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.381+0000 7ff4d6ac0700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d010ee00 con 0x7ff4d0104690 2024-04-26T18:27:20.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.381+0000 7ff4d6ac0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff4d010f100 con 0x7ff4d0105030 2024-04-26T18:27:20.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.381+0000 7ff4d485c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4d0103490 0x7ff4d0199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:20.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.381+0000 7ff4d485c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4d0103490 0x7ff4d0199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54040/0 (socket says 172.21.15.84:54040) 2024-04-26T18:27:20.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.381+0000 7ff4d485c700 1 -- 172.21.15.84:0/398788825 learned_addr learned my addr 172.21.15.84:0/398788825 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:20.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4d485c700 1 -- 172.21.15.84:0/398788825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4d0105030 msgr2=0x7ff4d01a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:20.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4d505d700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4d0105030 0x7ff4d01a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:20.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4cffff700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4d0104690 0x7ff4d0199b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4d485c700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4d0105030 0x7ff4d01a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4d485c700 1 -- 172.21.15.84:0/398788825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4d0104690 msgr2=0x7ff4d0199b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4d485c700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4d0104690 0x7ff4d0199b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4d485c700 1 -- 172.21.15.84:0/398788825 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4d01a12b0 con 0x7ff4d0103490 2024-04-26T18:27:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4d505d700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4d0105030 0x7ff4d01a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4cffff700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4d0104690 0x7ff4d0199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4d485c700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4d0103490 0x7ff4d0199650 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4003790 tx=0x7ff4c4003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4cdffb700 1 -- 172.21.15.84:0/398788825 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c4027070 con 0x7ff4d0103490 2024-04-26T18:27:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4cdffb700 1 -- 172.21.15.84:0/398788825 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4c4022470 con 0x7ff4d0103490 2024-04-26T18:27:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/398788825 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4d01a1540 con 0x7ff4d0103490 2024-04-26T18:27:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4cdffb700 1 -- 172.21.15.84:0/398788825 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4c400f6c0 con 0x7ff4d0103490 2024-04-26T18:27:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.382+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/398788825 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4d01a1a20 con 0x7ff4d0103490 2024-04-26T18:27:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.384+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/398788825 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4d006a110 con 0x7ff4d0103490 2024-04-26T18:27:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.384+0000 7ff4cdffb700 1 -- 172.21.15.84:0/398788825 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff4c402a070 con 0x7ff4d0103490 2024-04-26T18:27:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.385+0000 7ff4cdffb700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff4c00a3c40 0x7ff4c00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.385+0000 7ff4cffff700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff4c00a3c40 0x7ff4c00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.385+0000 7ff4cdffb700 1 -- 172.21.15.84:0/398788825 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7ff4c40d1360 con 0x7ff4d0103490 2024-04-26T18:27:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.385+0000 7ff4cffff700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff4c00a3c40 0x7ff4c00a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc007900 tx=0x7ff4bc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:20.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.390+0000 7ff4cdffb700 1 -- 172.21.15.84:0/398788825 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff4c4013070 con 0x7ff4d0103490 2024-04-26T18:27:20.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:20 smithi084 ceph-mon[29141]: pgmap v390: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:20.596 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.597+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/398788825 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7ff4d0063860 con 0x7ff4c00a3c40 2024-04-26T18:27:20.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.598+0000 7ff4cdffb700 1 -- 172.21.15.84:0/398788825 <== mgr.14150 v2:172.21.15.84:6800/2059208901 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7ff4d0063860 con 0x7ff4c00a3c40 2024-04-26T18:27:20.599 INFO:teuthology.orchestra.run.smithi084.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T18:27:20.599 INFO:teuthology.orchestra.run.smithi084.stdout:smithi084 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WF400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T18:27:20.599 INFO:teuthology.orchestra.run.smithi084.stdout:smithi103 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204012Z400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-04-26T18:27:20.599 INFO:teuthology.orchestra.run.smithi084.stdout:smithi174 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050074400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-26T18:27:20.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.600+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/398788825 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff4c00a3c40 msgr2=0x7ff4c00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:20.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.600+0000 7ff4d6ac0700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff4c00a3c40 0x7ff4c00a6100 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff4bc007900 tx=0x7ff4bc008040 comp rx=0 tx=0).stop 2024-04-26T18:27:20.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.600+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/398788825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4d0103490 msgr2=0x7ff4d0199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:20.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.600+0000 7ff4d6ac0700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4d0103490 0x7ff4d0199650 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff4c4003790 tx=0x7ff4c4003a10 comp rx=0 tx=0).stop 2024-04-26T18:27:20.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.600+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/398788825 shutdown_connections 2024-04-26T18:27:20.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.600+0000 7ff4d6ac0700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff4c00a3c40 0x7ff4c00a6100 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:20.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.600+0000 7ff4d6ac0700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4d0103490 0x7ff4d0199650 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:20.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.600+0000 7ff4d6ac0700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff4d0105030 0x7ff4d01a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:20.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.600+0000 7ff4d6ac0700 1 --2- 172.21.15.84:0/398788825 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4d0104690 0x7ff4d0199b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:20.601 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.600+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/398788825 >> 172.21.15.84:0/398788825 conn(0x7ff4d00fea30 msgr2=0x7ff4d0107a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:20.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.600+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/398788825 shutdown_connections 2024-04-26T18:27:20.602 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:20.601+0000 7ff4d6ac0700 1 -- 172.21.15.84:0/398788825 wait complete. 2024-04-26T18:27:20.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:20 smithi103 ceph-mon[32727]: pgmap v390: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:20.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:20 smithi174 ceph-mon[32705]: pgmap v390: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:21.411 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T18:27:21.421 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi084.front.sepia.ceph.com 2024-04-26T18:27:21.421 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T18:27:21.661 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:22.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:22 smithi084 ceph-mon[29141]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:22.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:22 smithi084 ceph-mon[29141]: pgmap v391: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:22 smithi103 ceph-mon[32727]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:22.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:22 smithi103 ceph-mon[32727]: pgmap v391: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:22.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:22 smithi174 ceph-mon[32705]: from='client.15096 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:27:22.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:22 smithi174 ceph-mon[32705]: pgmap v391: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:23.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.266+0000 7fbc6378c700 1 -- 172.21.15.84:0/3324474964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc5c101eb0 msgr2=0x7fbc5c10ba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:23.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.266+0000 7fbc6378c700 1 --2- 172.21.15.84:0/3324474964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc5c101eb0 0x7fbc5c10ba80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbc5400b0a0 tx=0x7fbc5400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:27:23.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.266+0000 7fbc6378c700 1 -- 172.21.15.84:0/3324474964 shutdown_connections 2024-04-26T18:27:23.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.266+0000 7fbc6378c700 1 --2- 172.21.15.84:0/3324474964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc5c101eb0 0x7fbc5c10ba80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:23.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.266+0000 7fbc6378c700 1 --2- 172.21.15.84:0/3324474964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc5c104a90 0x7fbc5c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:23.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.266+0000 7fbc6378c700 1 --2- 172.21.15.84:0/3324474964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5c101510 0x7fbc5c101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:23.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.266+0000 7fbc6378c700 1 -- 172.21.15.84:0/3324474964 >> 172.21.15.84:0/3324474964 conn(0x7fbc5c0fd0d0 msgr2=0x7fbc5c0ff510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:23.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.266+0000 7fbc6378c700 1 -- 172.21.15.84:0/3324474964 shutdown_connections 2024-04-26T18:27:23.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.266+0000 7fbc6378c700 1 -- 172.21.15.84:0/3324474964 wait complete. 2024-04-26T18:27:23.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc6378c700 1 Processor -- start 2024-04-26T18:27:23.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc6378c700 1 -- start start 2024-04-26T18:27:23.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc6378c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc5c101510 0x7fbc5c199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:23.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc6378c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc5c101eb0 0x7fbc5c199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:23.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc6378c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5c104a90 0x7fbc5c1a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:23.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc6378c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c10f040 con 0x7fbc5c101510 2024-04-26T18:27:23.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc6378c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c10eec0 con 0x7fbc5c104a90 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc6378c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbc5c10f1c0 con 0x7fbc5c101eb0 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc61528700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc5c101510 0x7fbc5c199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc61528700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc5c101510 0x7fbc5c199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54044/0 (socket says 172.21.15.84:54044) 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc61528700 1 -- 172.21.15.84:0/1193487901 learned_addr learned my addr 172.21.15.84:0/1193487901 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc61d29700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5c104a90 0x7fbc5c1a0c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc60d27700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc5c101eb0 0x7fbc5c199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc61528700 1 -- 172.21.15.84:0/1193487901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc5c101eb0 msgr2=0x7fbc5c199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc61528700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc5c101eb0 0x7fbc5c199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc61528700 1 -- 172.21.15.84:0/1193487901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5c104a90 msgr2=0x7fbc5c1a0c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc61528700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5c104a90 0x7fbc5c1a0c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc61528700 1 -- 172.21.15.84:0/1193487901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc5c1a12e0 con 0x7fbc5c101510 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc60d27700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc5c101eb0 0x7fbc5c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.267+0000 7fbc61d29700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5c104a90 0x7fbc5c1a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:23.271 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.268+0000 7fbc61528700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc5c101510 0x7fbc5c199680 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800b530 tx=0x7fbc5800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:23.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.268+0000 7fbc527fc700 1 -- 172.21.15.84:0/1193487901 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58015020 con 0x7fbc5c101510 2024-04-26T18:27:23.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.268+0000 7fbc6378c700 1 -- 172.21.15.84:0/1193487901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc5c1a15d0 con 0x7fbc5c101510 2024-04-26T18:27:23.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.268+0000 7fbc527fc700 1 -- 172.21.15.84:0/1193487901 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc58011ce0 con 0x7fbc5c101510 2024-04-26T18:27:23.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.268+0000 7fbc527fc700 1 -- 172.21.15.84:0/1193487901 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc58010d60 con 0x7fbc5c101510 2024-04-26T18:27:23.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.268+0000 7fbc6378c700 1 -- 172.21.15.84:0/1193487901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc5c1a1be0 con 0x7fbc5c101510 2024-04-26T18:27:23.272 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.269+0000 7fbc6378c700 1 -- 172.21.15.84:0/1193487901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc5c06a110 con 0x7fbc5c101510 2024-04-26T18:27:23.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.270+0000 7fbc527fc700 1 -- 172.21.15.84:0/1193487901 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fbc58018020 con 0x7fbc5c101510 2024-04-26T18:27:23.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.271+0000 7fbc527fc700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fbc480a3c40 0x7fbc480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:23.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.271+0000 7fbc60d27700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fbc480a3c40 0x7fbc480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:23.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.271+0000 7fbc527fc700 1 -- 172.21.15.84:0/1193487901 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fbc580d01c0 con 0x7fbc5c101510 2024-04-26T18:27:23.273 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.271+0000 7fbc60d27700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fbc480a3c40 0x7fbc480a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c007900 tx=0x7fbc4c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:23.274 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.275+0000 7fbc527fc700 1 -- 172.21.15.84:0/1193487901 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbc5801f020 con 0x7fbc5c101510 2024-04-26T18:27:23.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.520+0000 7fbc6378c700 1 -- 172.21.15.84:0/1193487901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fbc5c02e4c0 con 0x7fbc5c101510 2024-04-26T18:27:23.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.520+0000 7fbc527fc700 1 -- 172.21.15.84:0/1193487901 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3094 (secure 0 0 0) 0x7fbc5809cb70 con 0x7fbc5c101510 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout:e28 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout:legacy client fscid: 1 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout:Filesystem 'cephfs' (1) 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout:fs_name cephfs 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout:epoch 28 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout:created 2024-04-26T18:24:22.810992+0000 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout:modified 2024-04-26T18:24:41.425794+0000 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout:tableserver 0 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout:root 0 2024-04-26T18:27:23.521 INFO:teuthology.orchestra.run.smithi084.stdout:session_timeout 300 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:session_autoclose 300 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:max_file_size 1099511627776 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:max_xattr_size 65536 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:required_client_features {} 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:last_failure 0 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:last_failure_osd_epoch 0 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:max_mds 5 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:in 0,1,2,3,4 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:up {0=24434,1=24430,2=24422,3=24418,4=24407} 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:failed 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:damaged 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:stopped 2024-04-26T18:27:23.522 INFO:teuthology.orchestra.run.smithi084.stdout:data_pools [3] 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:metadata_pool 2 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:inline_data disabled 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:balancer 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:bal_rank_mask -1 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:standby_count_wanted 0 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:qdb_cluster leader: 24434 members: 24407,24418,24422,24430,24434 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.b{0:24434} state up:active seq 5 addr [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.f{0:24406} state up:standby-replay seq 1 addr [v2:172.21.15.174:6834/1310802824,v1:172.21.15.174:6835/1310802824] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.i{1:24430} state up:active seq 11 addr [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.k{1:24389} state up:standby-replay seq 1 addr [v2:172.21.15.103:6832/3711970972,v1:172.21.15.103:6833/3711970972] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.h{2:24422} state up:active seq 12 addr [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.l{2:24385} state up:standby-replay seq 1 addr [v2:172.21.15.174:6832/280377862,v1:172.21.15.174:6833/280377862] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.c{3:24418} state up:active seq 13 addr [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.a{3:14610} state up:standby-replay seq 1 addr [v2:172.21.15.84:6840/2504248516,v1:172.21.15.84:6841/2504248516] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.e{4:24407} state up:active seq 14 addr [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:27:23.523 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.g{4:14598} state up:standby-replay seq 1 addr [v2:172.21.15.84:6838/96876655,v1:172.21.15.84:6839/96876655] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:27:23.524 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:27:23.524 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:27:23.524 INFO:teuthology.orchestra.run.smithi084.stdout:Standby daemons: 2024-04-26T18:27:23.524 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:27:23.524 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.84:6834/1853556606,v1:172.21.15.84:6835/1853556606] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:27:23.524 INFO:teuthology.orchestra.run.smithi084.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.84:6836/3254529801,v1:172.21.15.84:6837/3254529801] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T18:27:23.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.522+0000 7fbc6378c700 1 -- 172.21.15.84:0/1193487901 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fbc480a3c40 msgr2=0x7fbc480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:23.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.522+0000 7fbc6378c700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fbc480a3c40 0x7fbc480a6100 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbc4c007900 tx=0x7fbc4c008040 comp rx=0 tx=0).stop 2024-04-26T18:27:23.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.522+0000 7fbc6378c700 1 -- 172.21.15.84:0/1193487901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc5c101510 msgr2=0x7fbc5c199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:23.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.522+0000 7fbc6378c700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc5c101510 0x7fbc5c199680 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800b530 tx=0x7fbc5800b8f0 comp rx=0 tx=0).stop 2024-04-26T18:27:23.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.522+0000 7fbc6378c700 1 -- 172.21.15.84:0/1193487901 shutdown_connections 2024-04-26T18:27:23.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.522+0000 7fbc6378c700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fbc480a3c40 0x7fbc480a6100 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:23.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.522+0000 7fbc6378c700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbc5c101510 0x7fbc5c199680 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:23.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.522+0000 7fbc6378c700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbc5c104a90 0x7fbc5c1a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:23.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.522+0000 7fbc6378c700 1 --2- 172.21.15.84:0/1193487901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbc5c101eb0 0x7fbc5c199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:23.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.522+0000 7fbc6378c700 1 -- 172.21.15.84:0/1193487901 >> 172.21.15.84:0/1193487901 conn(0x7fbc5c0fd0d0 msgr2=0x7fbc5c107bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:23.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.523+0000 7fbc6378c700 1 -- 172.21.15.84:0/1193487901 shutdown_connections 2024-04-26T18:27:23.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:23.523+0000 7fbc6378c700 1 -- 172.21.15.84:0/1193487901 wait complete. 2024-04-26T18:27:23.526 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 28 2024-04-26T18:27:24.305 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-26T18:27:24.547 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:24.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:24 smithi084 ceph-mon[29141]: pgmap v392: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:24.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:24 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1193487901' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T18:27:24.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:24 smithi103 ceph-mon[32727]: pgmap v392: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:24.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:24 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1193487901' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T18:27:24.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:24 smithi174 ceph-mon[32705]: pgmap v392: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:24.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:24 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1193487901' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T18:27:25.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:25 smithi084 ceph-mon[29141]: pgmap v393: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:25.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:25 smithi103 ceph-mon[32727]: pgmap v393: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:25.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:25 smithi174 ceph-mon[32705]: pgmap v393: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:26.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.364+0000 7ff7206d9700 1 -- 172.21.15.84:0/1316553857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff718105020 msgr2=0x7ff71810b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.364+0000 7ff7206d9700 1 --2- 172.21.15.84:0/1316553857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff718105020 0x7ff71810b8d0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff71000b0d0 tx=0x7ff71000b3e0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.364+0000 7ff7206d9700 1 -- 172.21.15.84:0/1316553857 shutdown_connections 2024-04-26T18:27:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.364+0000 7ff7206d9700 1 --2- 172.21.15.84:0/1316553857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff718105020 0x7ff71810b8d0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.364+0000 7ff7206d9700 1 --2- 172.21.15.84:0/1316553857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff718103480 0x7ff7181038a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.364+0000 7ff7206d9700 1 --2- 172.21.15.84:0/1316553857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff718104680 0x7ff718104ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.364+0000 7ff7206d9700 1 -- 172.21.15.84:0/1316553857 >> 172.21.15.84:0/1316553857 conn(0x7ff7180fea20 msgr2=0x7ff718100e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.364+0000 7ff7206d9700 1 -- 172.21.15.84:0/1316553857 shutdown_connections 2024-04-26T18:27:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.364+0000 7ff7206d9700 1 -- 172.21.15.84:0/1316553857 wait complete. 2024-04-26T18:27:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.365+0000 7ff7206d9700 1 Processor -- start 2024-04-26T18:27:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.365+0000 7ff7206d9700 1 -- start start 2024-04-26T18:27:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.365+0000 7ff7206d9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff718103480 0x7ff718199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.365+0000 7ff7206d9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff718104680 0x7ff718199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.365+0000 7ff7206d9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff718105020 0x7ff7181a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.365+0000 7ff7206d9700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff71810ef20 con 0x7ff718104680 2024-04-26T18:27:26.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.365+0000 7ff7206d9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff71810eda0 con 0x7ff718103480 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.365+0000 7ff7206d9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff71810f0a0 con 0x7ff718105020 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.365+0000 7ff71dc74700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff718104680 0x7ff718199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.365+0000 7ff71dc74700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff718104680 0x7ff718199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54054/0 (socket says 172.21.15.84:54054) 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.365+0000 7ff71dc74700 1 -- 172.21.15.84:0/3708970972 learned_addr learned my addr 172.21.15.84:0/3708970972 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.365+0000 7ff71e475700 1 --2- 172.21.15.84:0/3708970972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff718103480 0x7ff718199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.366+0000 7ff71dc74700 1 -- 172.21.15.84:0/3708970972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff718105020 msgr2=0x7ff7181a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.366+0000 7ff71dc74700 1 --2- 172.21.15.84:0/3708970972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff718105020 0x7ff7181a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.366+0000 7ff71dc74700 1 -- 172.21.15.84:0/3708970972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff718103480 msgr2=0x7ff718199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.366+0000 7ff71dc74700 1 --2- 172.21.15.84:0/3708970972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff718103480 0x7ff718199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.366+0000 7ff71dc74700 1 -- 172.21.15.84:0/3708970972 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7181a1190 con 0x7ff718104680 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.366+0000 7ff71dc74700 1 --2- 172.21.15.84:0/3708970972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff718104680 0x7ff718199a70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c9a0 tx=0x7ff71400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.366+0000 7ff70f7fe700 1 -- 172.21.15.84:0/3708970972 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714007a60 con 0x7ff718104680 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.366+0000 7ff7206d9700 1 -- 172.21.15.84:0/3708970972 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7181a1480 con 0x7ff718104680 2024-04-26T18:27:26.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.366+0000 7ff70f7fe700 1 -- 172.21.15.84:0/3708970972 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff714004ce0 con 0x7ff718104680 2024-04-26T18:27:26.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.366+0000 7ff70f7fe700 1 -- 172.21.15.84:0/3708970972 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff714005560 con 0x7ff718104680 2024-04-26T18:27:26.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.366+0000 7ff7206d9700 1 -- 172.21.15.84:0/3708970972 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7181a1970 con 0x7ff718104680 2024-04-26T18:27:26.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.367+0000 7ff7206d9700 1 -- 172.21.15.84:0/3708970972 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff71806a110 con 0x7ff718104680 2024-04-26T18:27:26.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.368+0000 7ff70f7fe700 1 -- 172.21.15.84:0/3708970972 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ff714007c00 con 0x7ff718104680 2024-04-26T18:27:26.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.369+0000 7ff70f7fe700 1 --2- 172.21.15.84:0/3708970972 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff7040a3c40 0x7ff7040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:26.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.369+0000 7ff71e475700 1 --2- 172.21.15.84:0/3708970972 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff7040a3c40 0x7ff7040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:26.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.369+0000 7ff70f7fe700 1 -- 172.21.15.84:0/3708970972 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7ff7140d0570 con 0x7ff718104680 2024-04-26T18:27:26.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.369+0000 7ff71e475700 1 --2- 172.21.15.84:0/3708970972 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff7040a3c40 0x7ff7040a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7181044e0 tx=0x7ff70800a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:26.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.372+0000 7ff70f7fe700 1 -- 172.21.15.84:0/3708970972 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff71409cfb0 con 0x7ff718104680 2024-04-26T18:27:26.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.576+0000 7ff7206d9700 1 -- 172.21.15.84:0/3708970972 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7ff71802e560 con 0x7ff718104680 2024-04-26T18:27:26.576 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.577+0000 7ff70f7fe700 1 -- 172.21.15.84:0/3708970972 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5326 (secure 0 0 0) 0x7ff71409c8b0 con 0x7ff718104680 2024-04-26T18:27:26.577 INFO:teuthology.orchestra.run.smithi084.stdout:epoch 93 2024-04-26T18:27:26.577 INFO:teuthology.orchestra.run.smithi084.stdout:fsid df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:27:26.577 INFO:teuthology.orchestra.run.smithi084.stdout:created 2024-04-26T18:16:06.267694+0000 2024-04-26T18:27:26.577 INFO:teuthology.orchestra.run.smithi084.stdout:modified 2024-04-26T18:26:07.290168+0000 2024-04-26T18:27:26.577 INFO:teuthology.orchestra.run.smithi084.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-26T18:27:26.577 INFO:teuthology.orchestra.run.smithi084.stdout:crush_version 26 2024-04-26T18:27:26.577 INFO:teuthology.orchestra.run.smithi084.stdout:full_ratio 0.95 2024-04-26T18:27:26.577 INFO:teuthology.orchestra.run.smithi084.stdout:backfillfull_ratio 0.9 2024-04-26T18:27:26.577 INFO:teuthology.orchestra.run.smithi084.stdout:nearfull_ratio 0.85 2024-04-26T18:27:26.577 INFO:teuthology.orchestra.run.smithi084.stdout:require_min_compat_client luminous 2024-04-26T18:27:26.577 INFO:teuthology.orchestra.run.smithi084.stdout:min_compat_client luminous 2024-04-26T18:27:26.578 INFO:teuthology.orchestra.run.smithi084.stdout:require_osd_release squid 2024-04-26T18:27:26.578 INFO:teuthology.orchestra.run.smithi084.stdout:stretch_mode_enabled false 2024-04-26T18:27:26.578 INFO:teuthology.orchestra.run.smithi084.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-26T18:27:26.578 INFO:teuthology.orchestra.run.smithi084.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-26T18:27:26.578 INFO:teuthology.orchestra.run.smithi084.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 89 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-04-26T18:27:26.578 INFO:teuthology.orchestra.run.smithi084.stdout:max_osd 12 2024-04-26T18:27:26.578 INFO:teuthology.orchestra.run.smithi084.stdout:osd.0 up in weight 1 up_from 8 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6802/140050011,v1:172.21.15.84:6803/140050011] [v2:172.21.15.84:6804/140050011,v1:172.21.15.84:6805/140050011] exists,up b1076912-986e-4825-b1d1-5fa9d69bc07d 2024-04-26T18:27:26.578 INFO:teuthology.orchestra.run.smithi084.stdout:osd.1 up in weight 1 up_from 13 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6800/2512857657,v1:172.21.15.103:6801/2512857657] [v2:172.21.15.103:6802/2512857657,v1:172.21.15.103:6803/2512857657] exists,up fa93293d-9058-4804-ba4b-7e1e674b0fb8 2024-04-26T18:27:26.578 INFO:teuthology.orchestra.run.smithi084.stdout:osd.2 up in weight 1 up_from 18 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6800/76340924,v1:172.21.15.174:6801/76340924] [v2:172.21.15.174:6802/76340924,v1:172.21.15.174:6803/76340924] exists,up bf78a44f-80a2-4cd7-974f-5abdf771e650 2024-04-26T18:27:26.578 INFO:teuthology.orchestra.run.smithi084.stdout:osd.3 up in weight 1 up_from 26 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6810/2352789256,v1:172.21.15.84:6811/2352789256] [v2:172.21.15.84:6812/2352789256,v1:172.21.15.84:6813/2352789256] exists,up 55fd0035-bc76-4235-b8b5-5cf5ef770139 2024-04-26T18:27:26.578 INFO:teuthology.orchestra.run.smithi084.stdout:osd.4 up in weight 1 up_from 34 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6808/1619258252,v1:172.21.15.103:6809/1619258252] [v2:172.21.15.103:6810/1619258252,v1:172.21.15.103:6811/1619258252] exists,up c278a0f3-e436-4d1c-a009-f9165ebf3c60 2024-04-26T18:27:26.578 INFO:teuthology.orchestra.run.smithi084.stdout:osd.5 up in weight 1 up_from 39 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6808/3168291768,v1:172.21.15.174:6809/3168291768] [v2:172.21.15.174:6810/3168291768,v1:172.21.15.174:6811/3168291768] exists,up 6505be9b-c1b0-4d8b-944e-9920f90cb88c 2024-04-26T18:27:26.578 INFO:teuthology.orchestra.run.smithi084.stdout:osd.6 up in weight 1 up_from 47 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6818/886635297,v1:172.21.15.84:6819/886635297] [v2:172.21.15.84:6820/886635297,v1:172.21.15.84:6821/886635297] exists,up a037b66a-7488-469d-b73a-14cc0d35cb5e 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:osd.7 up in weight 1 up_from 52 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6816/1738278079,v1:172.21.15.103:6817/1738278079] [v2:172.21.15.103:6818/1738278079,v1:172.21.15.103:6819/1738278079] exists,up c425a13f-8019-4484-86a0-a34fa4d8de81 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:osd.8 up in weight 1 up_from 60 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6816/38410548,v1:172.21.15.174:6817/38410548] [v2:172.21.15.174:6818/38410548,v1:172.21.15.174:6819/38410548] exists,up 687e6c08-bfa4-440c-b3bc-90a5748989d3 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:osd.9 up in weight 1 up_from 68 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.84:6826/2926393927,v1:172.21.15.84:6827/2926393927] [v2:172.21.15.84:6828/2926393927,v1:172.21.15.84:6829/2926393927] exists,up 059eb22d-6dfb-4dd0-b192-84ed42ee1fbc 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:osd.10 up in weight 1 up_from 73 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.103:6824/2705726915,v1:172.21.15.103:6825/2705726915] [v2:172.21.15.103:6826/2705726915,v1:172.21.15.103:6827/2705726915] exists,up da31e305-5220-4a87-a131-005b7c7f76b6 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:osd.11 up in weight 1 up_from 81 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.174:6824/1668672128,v1:172.21.15.174:6825/1668672128] [v2:172.21.15.174:6826/1668672128,v1:172.21.15.174:6827/1668672128] exists,up 3fde6c8f-ee02-41ee-a34f-73072b4761e0 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 2.b [8,11] 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 2.c [2,1] 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 2.f [8,11] 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 2.1d [2,4] 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 3.0 [4,8,5,9] 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 3.13 [4,9] 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 3.28 [3,6] 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 3.31 [4,7] 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 3.36 [4,9] 2024-04-26T18:27:26.579 INFO:teuthology.orchestra.run.smithi084.stdout:pg_upmap_items 3.39 [5,6] 2024-04-26T18:27:26.580 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:0/1604680842 expires 2024-04-27T18:17:05.083192+0000 2024-04-26T18:27:26.580 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:6801/3919933286 expires 2024-04-27T18:17:05.083192+0000 2024-04-26T18:27:26.580 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:6800/3919933286 expires 2024-04-27T18:17:05.083192+0000 2024-04-26T18:27:26.580 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:6801/1448727861 expires 2024-04-27T18:16:37.859669+0000 2024-04-26T18:27:26.580 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:0/538363975 expires 2024-04-27T18:16:37.859669+0000 2024-04-26T18:27:26.580 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:0/952052990 expires 2024-04-27T18:17:05.083192+0000 2024-04-26T18:27:26.580 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:0/1266942705 expires 2024-04-27T18:16:37.859669+0000 2024-04-26T18:27:26.580 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:6800/1448727861 expires 2024-04-27T18:16:37.859669+0000 2024-04-26T18:27:26.580 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:0/3970542149 expires 2024-04-27T18:16:37.859669+0000 2024-04-26T18:27:26.580 INFO:teuthology.orchestra.run.smithi084.stdout:blocklist 172.21.15.84:0/3607125049 expires 2024-04-27T18:17:05.083192+0000 2024-04-26T18:27:26.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.579+0000 7ff7206d9700 1 -- 172.21.15.84:0/3708970972 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff7040a3c40 msgr2=0x7ff7040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.579+0000 7ff7206d9700 1 --2- 172.21.15.84:0/3708970972 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff7040a3c40 0x7ff7040a6100 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff7181044e0 tx=0x7ff70800a380 comp rx=0 tx=0).stop 2024-04-26T18:27:26.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.579+0000 7ff7206d9700 1 -- 172.21.15.84:0/3708970972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff718104680 msgr2=0x7ff718199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:26.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.579+0000 7ff7206d9700 1 --2- 172.21.15.84:0/3708970972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff718104680 0x7ff718199a70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7ff71400c9a0 tx=0x7ff71400ccb0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.579+0000 7ff7206d9700 1 -- 172.21.15.84:0/3708970972 shutdown_connections 2024-04-26T18:27:26.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.579+0000 7ff7206d9700 1 --2- 172.21.15.84:0/3708970972 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7ff7040a3c40 0x7ff7040a6100 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.581 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.579+0000 7ff7206d9700 1 --2- 172.21.15.84:0/3708970972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff718105020 0x7ff7181a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.579+0000 7ff7206d9700 1 --2- 172.21.15.84:0/3708970972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff718103480 0x7ff718199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.579+0000 7ff7206d9700 1 --2- 172.21.15.84:0/3708970972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff718104680 0x7ff718199a70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:26.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.579+0000 7ff7206d9700 1 -- 172.21.15.84:0/3708970972 >> 172.21.15.84:0/3708970972 conn(0x7ff7180fea20 msgr2=0x7ff718107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:26.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.579+0000 7ff7206d9700 1 -- 172.21.15.84:0/3708970972 shutdown_connections 2024-04-26T18:27:26.582 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:26.579+0000 7ff7206d9700 1 -- 172.21.15.84:0/3708970972 wait complete. 2024-04-26T18:27:26.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:26 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3708970972' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T18:27:26.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:26 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3708970972' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T18:27:27.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:26 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3708970972' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-26T18:27:27.397 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-26T18:27:27.410 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:27:27.652 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:27.905 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:27 smithi084 ceph-mon[29141]: pgmap v394: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:27.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:27 smithi174 ceph-mon[32705]: pgmap v394: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:28.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:27 smithi103 ceph-mon[32727]: pgmap v394: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:29.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.280+0000 7f4380973700 1 -- 172.21.15.84:0/47646095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f437c102330 msgr2=0x7f437c1102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:29.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.280+0000 7f4380973700 1 --2- 172.21.15.84:0/47646095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f437c102330 0x7f437c1102d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f437400b0d0 tx=0x7f437400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:27:29.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.280+0000 7f4380973700 1 -- 172.21.15.84:0/47646095 shutdown_connections 2024-04-26T18:27:29.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.280+0000 7f4380973700 1 --2- 172.21.15.84:0/47646095 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f437c102330 0x7f437c1102d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:29.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.280+0000 7f4380973700 1 --2- 172.21.15.84:0/47646095 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f437c1094d0 0x7f437c1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:29.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.280+0000 7f4380973700 1 --2- 172.21.15.84:0/47646095 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c101970 0x7f437c101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:29.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.280+0000 7f4380973700 1 -- 172.21.15.84:0/47646095 >> 172.21.15.84:0/47646095 conn(0x7f437c078920 msgr2=0x7f437c0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:29.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.280+0000 7f4380973700 1 -- 172.21.15.84:0/47646095 shutdown_connections 2024-04-26T18:27:29.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.280+0000 7f4380973700 1 -- 172.21.15.84:0/47646095 wait complete. 2024-04-26T18:27:29.282 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.281+0000 7f4380973700 1 Processor -- start 2024-04-26T18:27:29.282 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.281+0000 7f4380973700 1 -- start start 2024-04-26T18:27:29.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.281+0000 7f4380973700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f437c101970 0x7f437c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:29.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.281+0000 7f4380973700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f437c102330 0x7f437c19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:29.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.281+0000 7f4380973700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c1094d0 0x7f437c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:29.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.281+0000 7f4380973700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f437c1135e0 con 0x7f437c101970 2024-04-26T18:27:29.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.281+0000 7f4380973700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f437c113460 con 0x7f437c102330 2024-04-26T18:27:29.283 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.281+0000 7f4380973700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f437c113760 con 0x7f437c1094d0 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.282+0000 7f437a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f437c101970 0x7f437c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.282+0000 7f437a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f437c101970 0x7f437c19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47142/0 (socket says 172.21.15.84:47142) 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.282+0000 7f437a59c700 1 -- 172.21.15.84:0/3071284366 learned_addr learned my addr 172.21.15.84:0/3071284366 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.282+0000 7f4379d9b700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f437c102330 0x7f437c19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.282+0000 7f437ad9d700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c1094d0 0x7f437c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.282+0000 7f437a59c700 1 -- 172.21.15.84:0/3071284366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c1094d0 msgr2=0x7f437c199820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.282+0000 7f437a59c700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c1094d0 0x7f437c199820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.282+0000 7f437a59c700 1 -- 172.21.15.84:0/3071284366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f437c102330 msgr2=0x7f437c19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.282+0000 7f437a59c700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f437c102330 0x7f437c19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.282+0000 7f437a59c700 1 -- 172.21.15.84:0/3071284366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f437c19a000 con 0x7f437c101970 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.282+0000 7f437a59c700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f437c101970 0x7f437c19f6a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f436c00d950 tx=0x7f436c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.283+0000 7f436b7fe700 1 -- 172.21.15.84:0/3071284366 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c016070 con 0x7f437c101970 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.283+0000 7f4380973700 1 -- 172.21.15.84:0/3071284366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f437c19a2f0 con 0x7f437c101970 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.283+0000 7f436b7fe700 1 -- 172.21.15.84:0/3071284366 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f436c00fd40 con 0x7f437c101970 2024-04-26T18:27:29.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.283+0000 7f436b7fe700 1 -- 172.21.15.84:0/3071284366 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f436c00e3b0 con 0x7f437c101970 2024-04-26T18:27:29.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.283+0000 7f4380973700 1 -- 172.21.15.84:0/3071284366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f437c102a80 con 0x7f437c101970 2024-04-26T18:27:29.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.284+0000 7f4380973700 1 -- 172.21.15.84:0/3071284366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f437c06a110 con 0x7f437c101970 2024-04-26T18:27:29.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.285+0000 7f436b7fe700 1 -- 172.21.15.84:0/3071284366 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f436c019070 con 0x7f437c101970 2024-04-26T18:27:29.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.285+0000 7f436b7fe700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f43640a3c40 0x7f43640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:29.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.285+0000 7f4379d9b700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f43640a3c40 0x7f43640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:29.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.285+0000 7f436b7fe700 1 -- 172.21.15.84:0/3071284366 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f436c0cfe60 con 0x7f437c101970 2024-04-26T18:27:29.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.286+0000 7f4379d9b700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f43640a3c40 0x7f43640a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f437c108a60 tx=0x7f437000a380 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:29.290 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.290+0000 7f436b7fe700 1 -- 172.21.15.84:0/3071284366 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f436c09c810 con 0x7f437c101970 2024-04-26T18:27:29.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.536+0000 7f4380973700 1 -- 172.21.15.84:0/3071284366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f437c19ad50 con 0x7f437c101970 2024-04-26T18:27:29.536 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.536+0000 7f436b7fe700 1 -- 172.21.15.84:0/3071284366 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11936 (secure 0 0 0) 0x7f437c19ad50 con 0x7f437c101970 2024-04-26T18:27:29.536 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:27:29.537 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:24:41.425794+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24407,24418,24422,24430,24434]},"id":1}]} 2024-04-26T18:27:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.538+0000 7f4380973700 1 -- 172.21.15.84:0/3071284366 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f43640a3c40 msgr2=0x7f43640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.538+0000 7f4380973700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f43640a3c40 0x7f43640a6100 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f437c108a60 tx=0x7f437000a380 comp rx=0 tx=0).stop 2024-04-26T18:27:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.538+0000 7f4380973700 1 -- 172.21.15.84:0/3071284366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f437c101970 msgr2=0x7f437c19f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.538+0000 7f4380973700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f437c101970 0x7f437c19f6a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f436c00d950 tx=0x7f436c00dc60 comp rx=0 tx=0).stop 2024-04-26T18:27:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.538+0000 7f4380973700 1 -- 172.21.15.84:0/3071284366 shutdown_connections 2024-04-26T18:27:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.538+0000 7f4380973700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f43640a3c40 0x7f43640a6100 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.538+0000 7f4380973700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f437c101970 0x7f437c19f6a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.538+0000 7f4380973700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f437c1094d0 0x7f437c199820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:29.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.538+0000 7f4380973700 1 --2- 172.21.15.84:0/3071284366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f437c102330 0x7f437c19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.538+0000 7f4380973700 1 -- 172.21.15.84:0/3071284366 >> 172.21.15.84:0/3071284366 conn(0x7f437c078920 msgr2=0x7f437c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.538+0000 7f4380973700 1 -- 172.21.15.84:0/3071284366 shutdown_connections 2024-04-26T18:27:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:29.538+0000 7f4380973700 1 -- 172.21.15.84:0/3071284366 wait complete. 2024-04-26T18:27:29.540 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 28 2024-04-26T18:27:30.372 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:27:30.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:30 smithi084 ceph-mon[29141]: pgmap v395: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:30.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:30 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3071284366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:27:30.616 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:30 smithi103 ceph-mon[32727]: pgmap v395: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:30.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:30 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3071284366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:27:30.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:30 smithi174 ceph-mon[32705]: pgmap v395: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:30.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:30 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3071284366' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:27:32.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.381+0000 7fdd57f55700 1 -- 172.21.15.84:0/3683055044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd50102ce0 msgr2=0x7fdd5010f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.381+0000 7fdd57f55700 1 --2- 172.21.15.84:0/3683055044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd50102ce0 0x7fdd5010f1d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fdd40009990 tx=0x7fdd40009ca0 comp rx=0 tx=0).stop 2024-04-26T18:27:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.381+0000 7fdd57f55700 1 -- 172.21.15.84:0/3683055044 shutdown_connections 2024-04-26T18:27:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.381+0000 7fdd57f55700 1 --2- 172.21.15.84:0/3683055044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd50102ce0 0x7fdd5010f1d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.381+0000 7fdd57f55700 1 --2- 172.21.15.84:0/3683055044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd50108390 0x7fdd50108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.381+0000 7fdd57f55700 1 --2- 172.21.15.84:0/3683055044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd50102320 0x7fdd501027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.381+0000 7fdd57f55700 1 -- 172.21.15.84:0/3683055044 >> 172.21.15.84:0/3683055044 conn(0x7fdd500792d0 msgr2=0x7fdd500796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.381+0000 7fdd57f55700 1 -- 172.21.15.84:0/3683055044 shutdown_connections 2024-04-26T18:27:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.381+0000 7fdd57f55700 1 -- 172.21.15.84:0/3683055044 wait complete. 2024-04-26T18:27:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd57f55700 1 Processor -- start 2024-04-26T18:27:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd57f55700 1 -- start start 2024-04-26T18:27:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd57f55700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd50102320 0x7fdd50104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:32.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd57f55700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd50102ce0 0x7fdd50104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd57f55700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd50108390 0x7fdd50104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd57f55700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdd501124e0 con 0x7fdd50108390 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd57f55700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd50112360 con 0x7fdd50102ce0 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd57f55700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdd50112660 con 0x7fdd50102320 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd564f2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd50108390 0x7fdd50104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd564f2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd50108390 0x7fdd50104a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47160/0 (socket says 172.21.15.84:47160) 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd564f2700 1 -- 172.21.15.84:0/643095533 learned_addr learned my addr 172.21.15.84:0/643095533 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd55cf1700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd50102320 0x7fdd50104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd554f0700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd50102ce0 0x7fdd50104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd564f2700 1 -- 172.21.15.84:0/643095533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd50102320 msgr2=0x7fdd50104000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd564f2700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd50102320 0x7fdd50104000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd564f2700 1 -- 172.21.15.84:0/643095533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd50102ce0 msgr2=0x7fdd50104540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd564f2700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd50102ce0 0x7fdd50104540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:32.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.382+0000 7fdd564f2700 1 -- 172.21.15.84:0/643095533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd501a8030 con 0x7fdd50108390 2024-04-26T18:27:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.383+0000 7fdd564f2700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd50108390 0x7fdd50104a80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdd400047b0 tx=0x7fdd40005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.383+0000 7fdd4effd700 1 -- 172.21.15.84:0/643095533 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40027070 con 0x7fdd50108390 2024-04-26T18:27:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.383+0000 7fdd4effd700 1 -- 172.21.15.84:0/643095533 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd40003f20 con 0x7fdd50108390 2024-04-26T18:27:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.383+0000 7fdd57f55700 1 -- 172.21.15.84:0/643095533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd501a82c0 con 0x7fdd50108390 2024-04-26T18:27:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.383+0000 7fdd4effd700 1 -- 172.21.15.84:0/643095533 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd40022bb0 con 0x7fdd50108390 2024-04-26T18:27:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.383+0000 7fdd57f55700 1 -- 172.21.15.84:0/643095533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd501a86c0 con 0x7fdd50108390 2024-04-26T18:27:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.384+0000 7fdd57f55700 1 -- 172.21.15.84:0/643095533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd5006a110 con 0x7fdd50108390 2024-04-26T18:27:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.385+0000 7fdd4effd700 1 -- 172.21.15.84:0/643095533 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fdd4002a070 con 0x7fdd50108390 2024-04-26T18:27:32.387 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:32 smithi084 ceph-mon[29141]: pgmap v396: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:32.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.386+0000 7fdd4effd700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdd380a3c40 0x7fdd380a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:32.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.386+0000 7fdd55cf1700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdd380a3c40 0x7fdd380a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:32.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.386+0000 7fdd4effd700 1 -- 172.21.15.84:0/643095533 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fdd400d1a70 con 0x7fdd50108390 2024-04-26T18:27:32.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.386+0000 7fdd55cf1700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdd380a3c40 0x7fdd380a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdd48007fd0 tx=0x7fdd4800c040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:32.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.391+0000 7fdd4effd700 1 -- 172.21.15.84:0/643095533 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd40031070 con 0x7fdd50108390 2024-04-26T18:27:32.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:32 smithi103 ceph-mon[32727]: pgmap v396: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:32.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.635+0000 7fdd57f55700 1 -- 172.21.15.84:0/643095533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdd501a8920 con 0x7fdd50108390 2024-04-26T18:27:32.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.636+0000 7fdd4effd700 1 -- 172.21.15.84:0/643095533 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11936 (secure 0 0 0) 0x7fdd40013070 con 0x7fdd50108390 2024-04-26T18:27:32.636 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:27:32.636 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:24:41.425794+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24407,24418,24422,24430,24434]},"id":1}]} 2024-04-26T18:27:32.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.638+0000 7fdd57f55700 1 -- 172.21.15.84:0/643095533 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdd380a3c40 msgr2=0x7fdd380a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:32.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.638+0000 7fdd57f55700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdd380a3c40 0x7fdd380a6100 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fdd48007fd0 tx=0x7fdd4800c040 comp rx=0 tx=0).stop 2024-04-26T18:27:32.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.638+0000 7fdd57f55700 1 -- 172.21.15.84:0/643095533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd50108390 msgr2=0x7fdd50104a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:32.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.638+0000 7fdd57f55700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd50108390 0x7fdd50104a80 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fdd400047b0 tx=0x7fdd40005e80 comp rx=0 tx=0).stop 2024-04-26T18:27:32.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.638+0000 7fdd57f55700 1 -- 172.21.15.84:0/643095533 shutdown_connections 2024-04-26T18:27:32.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.638+0000 7fdd57f55700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fdd380a3c40 0x7fdd380a6100 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:32.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.638+0000 7fdd57f55700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd50108390 0x7fdd50104a80 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:32.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.638+0000 7fdd57f55700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd50102320 0x7fdd50104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:32.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.638+0000 7fdd57f55700 1 --2- 172.21.15.84:0/643095533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd50102ce0 0x7fdd50104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:32.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.638+0000 7fdd57f55700 1 -- 172.21.15.84:0/643095533 >> 172.21.15.84:0/643095533 conn(0x7fdd500792d0 msgr2=0x7fdd501015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:32.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.638+0000 7fdd57f55700 1 -- 172.21.15.84:0/643095533 shutdown_connections 2024-04-26T18:27:32.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:32.638+0000 7fdd57f55700 1 -- 172.21.15.84:0/643095533 wait complete. 2024-04-26T18:27:32.639 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 28 2024-04-26T18:27:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:32 smithi174 ceph-mon[32705]: pgmap v396: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:33.321 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:27:33.560 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:33.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:33 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/643095533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:27:33.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:33 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/643095533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:27:33.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:33 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/643095533' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:27:34.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:34 smithi084 ceph-mon[29141]: pgmap v397: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:34.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:34 smithi103 ceph-mon[32727]: pgmap v397: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:34.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:34 smithi174 ceph-mon[32705]: pgmap v397: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:35.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.226+0000 7f609119e700 1 -- 172.21.15.84:0/2105032170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c1075e0 msgr2=0x7f608c10f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.226+0000 7f609119e700 1 --2- 172.21.15.84:0/2105032170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c1075e0 0x7f608c10f1b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f608400b0d0 tx=0x7f608400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:27:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.226+0000 7f609119e700 1 -- 172.21.15.84:0/2105032170 shutdown_connections 2024-04-26T18:27:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.226+0000 7f609119e700 1 --2- 172.21.15.84:0/2105032170 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c1075e0 0x7f608c10f1b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.226+0000 7f609119e700 1 --2- 172.21.15.84:0/2105032170 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f608c102500 0x7f608c1028e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.226+0000 7f609119e700 1 --2- 172.21.15.84:0/2105032170 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f608c102eb0 0x7f608c106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.226+0000 7f609119e700 1 -- 172.21.15.84:0/2105032170 >> 172.21.15.84:0/2105032170 conn(0x7f608c0fddb0 msgr2=0x7f608c1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.226+0000 7f609119e700 1 -- 172.21.15.84:0/2105032170 shutdown_connections 2024-04-26T18:27:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.226+0000 7f609119e700 1 -- 172.21.15.84:0/2105032170 wait complete. 2024-04-26T18:27:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f609119e700 1 Processor -- start 2024-04-26T18:27:35.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f609119e700 1 -- start start 2024-04-26T18:27:35.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f609119e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f608c102500 0x7f608c19a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:35.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f609119e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f608c102eb0 0x7f608c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:35.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f609119e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c1075e0 0x7f608c1a5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:35.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f609119e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f608c1124a0 con 0x7f608c1075e0 2024-04-26T18:27:35.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f609119e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f608c112320 con 0x7f608c102eb0 2024-04-26T18:27:35.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f609119e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f608c112620 con 0x7f608c102500 2024-04-26T18:27:35.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f609099d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c1075e0 0x7f608c1a5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:35.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f608b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f608c102eb0 0x7f608c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f609099d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c1075e0 0x7f608c1a5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47174/0 (socket says 172.21.15.84:47174) 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f608b7fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f608c102eb0 0x7f608c19ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:54264/0 (socket says 172.21.15.84:54264) 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f609099d700 1 -- 172.21.15.84:0/2177986831 learned_addr learned my addr 172.21.15.84:0/2177986831 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f608bfff700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f608c102500 0x7f608c19a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f608b7fe700 1 -- 172.21.15.84:0/2177986831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f608c102500 msgr2=0x7f608c19a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f608b7fe700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f608c102500 0x7f608c19a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f608b7fe700 1 -- 172.21.15.84:0/2177986831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c1075e0 msgr2=0x7f608c1a5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f608b7fe700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c1075e0 0x7f608c1a5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.227+0000 7f608b7fe700 1 -- 172.21.15.84:0/2177986831 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f608c1a5e80 con 0x7f608c102eb0 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.228+0000 7f609099d700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c1075e0 0x7f608c1a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.228+0000 7f608b7fe700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f608c102eb0 0x7f608c19ae00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f608000e9c0 tx=0x7f608000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.228+0000 7f60897fa700 1 -- 172.21.15.84:0/2177986831 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f608000cbd0 con 0x7f608c102eb0 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.228+0000 7f609119e700 1 -- 172.21.15.84:0/2177986831 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f608c1a60b0 con 0x7f608c102eb0 2024-04-26T18:27:35.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.228+0000 7f60897fa700 1 -- 172.21.15.84:0/2177986831 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f608000cd70 con 0x7f608c102eb0 2024-04-26T18:27:35.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.228+0000 7f609119e700 1 -- 172.21.15.84:0/2177986831 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f608c1a6610 con 0x7f608c102eb0 2024-04-26T18:27:35.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.228+0000 7f60897fa700 1 -- 172.21.15.84:0/2177986831 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6080018680 con 0x7f608c102eb0 2024-04-26T18:27:35.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.229+0000 7f609119e700 1 -- 172.21.15.84:0/2177986831 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f608c06a110 con 0x7f608c102eb0 2024-04-26T18:27:35.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.230+0000 7f60897fa700 1 -- 172.21.15.84:0/2177986831 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6080010430 con 0x7f608c102eb0 2024-04-26T18:27:35.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.230+0000 7f60897fa700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f60740a3c40 0x7f60740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:35.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.231+0000 7f608bfff700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f60740a3c40 0x7f60740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:35.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.231+0000 7f60897fa700 1 -- 172.21.15.84:0/2177986831 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f60800d0570 con 0x7f608c102eb0 2024-04-26T18:27:35.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.231+0000 7f608bfff700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f60740a3c40 0x7f60740a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f607c005950 tx=0x7f607c005eb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:35.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.234+0000 7f60897fa700 1 -- 172.21.15.84:0/2177986831 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f608009cec0 con 0x7f608c102eb0 2024-04-26T18:27:35.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.440+0000 7f609119e700 1 -- 172.21.15.84:0/2177986831 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f608c06bf80 con 0x7f608c102eb0 2024-04-26T18:27:35.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.442+0000 7f60897fa700 1 -- 172.21.15.84:0/2177986831 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+22727 (secure 0 0 0) 0x7f608009c7c0 con 0x7f608c102eb0 2024-04-26T18:27:35.443 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:27:35.443 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":93,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:16:06.267694+0000","modified":"2024-04-26T18:26:07.290168+0000","last_up_change":"2024-04-26T18:23:19.906511+0000","last_in_change":"2024-04-26T18:22:59.541869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:19:31.174024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:24:16.533128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:24:19.742656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1076912-986e-4825-b1d1-5fa9d69bc07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6803","nonce":140050011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6805","nonce":140050011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6809","nonce":140050011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6807","nonce":140050011}]},"public_addr":"172.21.15.84:6803/140050011","cluster_addr":"172.21.15.84:6805/140050011","heartbeat_back_addr":"172.21.15.84:6809/140050011","heartbeat_front_addr":"172.21.15.84:6807/140050011","state":["exists","up"]},{"osd":1,"uuid":"fa93293d-9058-4804-ba4b-7e1e674b0fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6801","nonce":2512857657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6803","nonce":2512857657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6807","nonce":2512857657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6805","nonce":2512857657}]},"public_addr":"172.21.15.103:6801/2512857657","cluster_addr":"172.21.15.103:6803/2512857657","heartbeat_back_addr":"172.21.15.103:6807/2512857657","heartbeat_front_addr":"172.21.15.103:6805/2512857657","state":["exists","up"]},{"osd":2,"uuid":"bf78a44f-80a2-4cd7-974f-5abdf771e650","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6801","nonce":76340924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6803","nonce":76340924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6807","nonce":76340924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6805","nonce":76340924}]},"public_addr":"172.21.15.174:6801/76340924","cluster_addr":"172.21.15.174:6803/76340924","heartbeat_back_addr":"172.21.15.174:6807/76340924","heartbeat_front_addr":"172.21.15.174:6805/76340924","state":["exists","up"]},{"osd":3,"uuid":"55fd0035-bc76-4235-b8b5-5cf5ef770139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6811","nonce":2352789256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6813","nonce":2352789256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6817","nonce":2352789256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6815","nonce":2352789256}]},"public_addr":"172.21.15.84:6811/2352789256","cluster_addr":"172.21.15.84:6813/2352789256","heartbeat_back_addr":"172.21.15.84:6817/2352789256","heartbeat_front_addr":"172.21.15.84:6815/2352789256","state":["exists","up"]},{"osd":4,"uuid":"c278a0f3-e436-4d1c-a009-f9165ebf3c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6809","nonce":1619258252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6811","nonce":1619258252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6815","nonce":1619258252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6813","nonce":1619258252}]},"public_addr":"172.21.15.103:6809/1619258252","cluster_addr":"172.21.15.103:6811/1619258252","heartbeat_back_addr":"172.21.15.103:6815/1619258252","heartbeat_front_addr":"172.21.15.103:6813/1619258252","state":["exists","up"]},{"osd":5,"uuid":"6505be9b-c1b0-4d8b-944e-9920f90cb88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6809","nonce":3168291768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6811","nonce":3168291768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6815","nonce":3168291768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6813","nonce":3168291768}]},"public_addr":"172.21.15.174:6809/3168291768","cluster_addr":"172.21.15.174:6811/3168291768","heartbeat_back_addr":"172.21.15.174:6815/3168291768","heartbeat_front_addr":"172.21.15.174:6813/3168291768","state":["exists","up"]},{"osd":6,"uuid":"a037b66a-7488-469d-b73a-14cc0d35cb5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6819","nonce":886635297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6821","nonce":886635297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6825","nonce":886635297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6823","nonce":886635297}]},"public_addr":"172.21.15.84:6819/886635297","cluster_addr":"172.21.15.84:6821/886635297","heartbeat_back_addr":"172.21.15.84:6825/886635297","heartbeat_front_addr":"172.21.15.84:6823/886635297","state":["exists","up"]},{"osd":7,"uuid":"c425a13f-8019-4484-86a0-a34fa4d8de81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6817","nonce":1738278079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6819","nonce":1738278079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6823","nonce":1738278079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6821","nonce":1738278079}]},"public_addr":"172.21.15.103:6817/1738278079","cluster_addr":"172.21.15.103:6819/1738278079","heartbeat_back_addr":"172.21.15.103:6823/1738278079","heartbeat_front_addr":"172.21.15.103:6821/1738278079","state":["exists","up"]},{"osd":8,"uuid":"687e6c08-bfa4-440c-b3bc-90a5748989d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6817","nonce":38410548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6819","nonce":38410548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6823","nonce":38410548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6821","nonce":38410548}]},"public_addr":"172.21.15.174:6817/38410548","cluster_addr":"172.21.15.174:6819/38410548","heartbeat_back_addr":"172.21.15.174:6823/38410548","heartbeat_front_addr":"172.21.15.174:6821/38410548","state":["exists","up"]},{"osd":9,"uuid":"059eb22d-6dfb-4dd0-b192-84ed42ee1fbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6827","nonce":2926393927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6829","nonce":2926393927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6833","nonce":2926393927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6831","nonce":2926393927}]},"public_addr":"172.21.15.84:6827/2926393927","cluster_addr":"172.21.15.84:6829/2926393927","heartbeat_back_addr":"172.21.15.84:6833/2926393927","heartbeat_front_addr":"172.21.15.84:6831/2926393927","state":["exists","up"]},{"osd":10,"uuid":"da31e305-5220-4a87-a131-005b7c7f76b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6825","nonce":2705726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6827","nonce":2705726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6831","nonce":2705726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6829","nonce":2705726915}]},"public_addr":"172.21.15.103:6825/2705726915","cluster_addr":"172.21.15.103:6827/2705726915","heartbeat_back_addr":"172.21.15.103:6831/2705726915","heartbeat_front_addr":"172.21.15.103:6829/2705726915","state":["exists","up"]},{"osd":11,"uuid":"3fde6c8f-ee02-41ee-a34f-73072b4761e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6825","nonce":1668672128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6827","nonce":1668672128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6831","nonce":1668672128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6829","nonce":1668672128}]},"public_addr":"172.21.15.174:6825/1668672128","cluster_addr":"172.21.15.174:6827/1668672128","heartbeat_back_addr":"172.21.15.174:6831/1668672128","heartbeat_front_addr":"172.21.15.174:6829/1668672128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:39.099188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:03.327608+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:27.714108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:52.506015+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:17.631670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:42.702489+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:07.606697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:34.123214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:59.534658+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:25.357720+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:51.543106+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:23:18.208230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/1604680842":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6800/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/538363975":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/952052990":"2024-04-27T18:17:05.083192+0000","172.21.15.84:0/1266942705":"2024-04-27T18:16:37.859669+0000","172.21.15.84:6800/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3970542149":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3607125049":"2024-04-27T18:17:05.083192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:27:35.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.445+0000 7f609119e700 1 -- 172.21.15.84:0/2177986831 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f60740a3c40 msgr2=0x7f60740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:35.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.445+0000 7f609119e700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f60740a3c40 0x7f60740a6100 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f607c005950 tx=0x7f607c005eb0 comp rx=0 tx=0).stop 2024-04-26T18:27:35.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.445+0000 7f609119e700 1 -- 172.21.15.84:0/2177986831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f608c102eb0 msgr2=0x7f608c19ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:35.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.445+0000 7f609119e700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f608c102eb0 0x7f608c19ae00 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f608000e9c0 tx=0x7f608000ecd0 comp rx=0 tx=0).stop 2024-04-26T18:27:35.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.446+0000 7f609119e700 1 -- 172.21.15.84:0/2177986831 shutdown_connections 2024-04-26T18:27:35.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.446+0000 7f609119e700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f60740a3c40 0x7f60740a6100 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:35.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.446+0000 7f609119e700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f608c1075e0 0x7f608c1a5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:35.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.446+0000 7f609119e700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f608c102500 0x7f608c19a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:35.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.446+0000 7f609119e700 1 --2- 172.21.15.84:0/2177986831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f608c102eb0 0x7f608c19ae00 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:35.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.446+0000 7f609119e700 1 -- 172.21.15.84:0/2177986831 >> 172.21.15.84:0/2177986831 conn(0x7f608c0fddb0 msgr2=0x7f608c100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:35.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.446+0000 7f609119e700 1 -- 172.21.15.84:0/2177986831 shutdown_connections 2024-04-26T18:27:35.447 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:35.446+0000 7f609119e700 1 -- 172.21.15.84:0/2177986831 wait complete. 2024-04-26T18:27:35.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:35 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:27:35.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:35 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:35.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:35 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:35.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:35 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:35.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:35 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:35.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:35 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:27:35.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:35 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:35.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:35 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:35.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:35 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:35.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:35 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:35 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:27:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:35 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:35 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:35 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:35 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:36.197 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T18:24:22.810992+0000', 'modified': '2024-04-26T18:24:41.425794+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24434, 'mds_1': 24430, 'mds_2': 24422, 'mds_3': 24418, 'mds_4': 24407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14598': {'gid': 14598, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6839/96876655', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6838', 'nonce': 96876655}, {'type': 'v1', 'addr': '172.21.15.84:6839', 'nonce': 96876655}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14610': {'gid': 14610, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6841/2504248516', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6840', 'nonce': 2504248516}, {'type': 'v1', 'addr': '172.21.15.84:6841', 'nonce': 2504248516}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24385': {'gid': 24385, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/280377862', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 280377862}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 280377862}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6833/3711970972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6832', 'nonce': 3711970972}, {'type': 'v1', 'addr': '172.21.15.103:6833', 'nonce': 3711970972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24406': {'gid': 24406, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/1310802824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 1310802824}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 1310802824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.103:6835/2529642104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6834', 'nonce': 2529642104}, {'type': 'v1', 'addr': '172.21.15.103:6835', 'nonce': 2529642104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.174:6837/1532309720', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 1532309720}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 1532309720}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.103:6837/2271231481', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6836', 'nonce': 2271231481}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 2271231481}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.174:6839/518809558', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 518809558}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 518809558}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.103:6839/2465442393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 2465442393}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 2465442393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24434, 'qdb_cluster': [24407, 24418, 24422, 24430, 24434]} 2024-04-26T18:27:36.197 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-26T18:27:36.198 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.e status 2024-04-26T18:27:36.443 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:36.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:36 smithi084 ceph-mon[29141]: pgmap v398: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:36.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:36 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2177986831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:27:36.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:36 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:27:36.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:36 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:27:36.565 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:36 smithi084 ceph-mon[29141]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:36 smithi103 ceph-mon[32727]: pgmap v398: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:36 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2177986831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:27:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:36 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:27:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:36 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:27:36.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:36 smithi103 ceph-mon[32727]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:36.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:36 smithi174 ceph-mon[32705]: pgmap v398: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:36.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:36 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2177986831' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:27:36.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:36 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:27:36.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:36 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:27:36.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:36 smithi174 ceph-mon[32705]: from='mgr.14150 172.21.15.84:0/2796454633' entity='mgr.x' 2024-04-26T18:27:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.167+0000 7f10aa31e700 1 -- 172.21.15.84:0/389649750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a41094f0 msgr2=0x7f10a41098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.167+0000 7f10aa31e700 1 --2- 172.21.15.84:0/389649750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a41094f0 0x7f10a41098d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1094009990 tx=0x7f1094009ca0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.167+0000 7f10aa31e700 1 -- 172.21.15.84:0/389649750 shutdown_connections 2024-04-26T18:27:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.167+0000 7f10aa31e700 1 --2- 172.21.15.84:0/389649750 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a41094f0 0x7f10a41098d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.167+0000 7f10aa31e700 1 --2- 172.21.15.84:0/389649750 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a4103e90 0x7f10a41102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.167+0000 7f10aa31e700 1 --2- 172.21.15.84:0/389649750 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10a41034d0 0x7f10a4103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.167+0000 7f10aa31e700 1 -- 172.21.15.84:0/389649750 >> 172.21.15.84:0/389649750 conn(0x7f10a407a4b0 msgr2=0x7f10a407a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:38.170 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.167+0000 7f10aa31e700 1 -- 172.21.15.84:0/389649750 shutdown_connections 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.167+0000 7f10aa31e700 1 -- 172.21.15.84:0/389649750 wait complete. 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.168+0000 7f10aa31e700 1 Processor -- start 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.168+0000 7f10aa31e700 1 -- start start 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.168+0000 7f10aa31e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a41034d0 0x7f10a419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.168+0000 7f10aa31e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a4103e90 0x7f10a419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.168+0000 7f10aa31e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10a41094f0 0x7f10a4199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.168+0000 7f10aa31e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f10a41136f0 con 0x7f10a41034d0 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.168+0000 7f10aa31e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10a4113570 con 0x7f10a41094f0 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.168+0000 7f10aa31e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10a4113870 con 0x7f10a4103e90 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.168+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a41034d0 0x7f10a419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.168+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a41034d0 0x7f10a419f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49564/0 (socket says 172.21.15.84:49564) 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10a3fff700 1 -- 172.21.15.84:0/2440312003 learned_addr learned my addr 172.21.15.84:0/2440312003 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10a88bb700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10a41094f0 0x7f10a4199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.171 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10a37fe700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a4103e90 0x7f10a419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10a3fff700 1 -- 172.21.15.84:0/2440312003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a4103e90 msgr2=0x7f10a419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10a3fff700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a4103e90 0x7f10a419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10a3fff700 1 -- 172.21.15.84:0/2440312003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10a41094f0 msgr2=0x7f10a4199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10a3fff700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10a41094f0 0x7f10a4199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10a3fff700 1 -- 172.21.15.84:0/2440312003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a419a210 con 0x7f10a41034d0 2024-04-26T18:27:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10a37fe700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a4103e90 0x7f10a419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10a3fff700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a41034d0 0x7f10a419f8f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1094000c00 tx=0x7f10940047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10a17fa700 1 -- 172.21.15.84:0/2440312003 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094027070 con 0x7f10a41034d0 2024-04-26T18:27:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10aa31e700 1 -- 172.21.15.84:0/2440312003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a419a4a0 con 0x7f10a41034d0 2024-04-26T18:27:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10a17fa700 1 -- 172.21.15.84:0/2440312003 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10940041d0 con 0x7f10a41034d0 2024-04-26T18:27:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10a17fa700 1 -- 172.21.15.84:0/2440312003 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109400f460 con 0x7f10a41034d0 2024-04-26T18:27:38.172 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.169+0000 7f10aa31e700 1 -- 172.21.15.84:0/2440312003 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a41a67c0 con 0x7f10a41034d0 2024-04-26T18:27:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.171+0000 7f10a17fa700 1 -- 172.21.15.84:0/2440312003 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f109402a070 con 0x7f10a41034d0 2024-04-26T18:27:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.172+0000 7f10a17fa700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f108c0a3c40 0x7f108c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.172+0000 7f10a37fe700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f108c0a3c40 0x7f108c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.172+0000 7f10a17fa700 1 -- 172.21.15.84:0/2440312003 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f10940d1470 con 0x7f10a41034d0 2024-04-26T18:27:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10a37fe700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f108c0a3c40 0x7f108c0a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f10a407ab70 tx=0x7f1098008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10b4bb0b80 1 Processor -- start 2024-04-26T18:27:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10b4bb0b80 1 -- start start 2024-04-26T18:27:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10b4bb0b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec45cc0 0x5580bec460a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10b4bb0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec465e0 0x5580bec40160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10b4bb0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec40780 0x5580bec40be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10b4bb0b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5580beb8e7e0 con 0x5580bec45cc0 2024-04-26T18:27:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10b4bb0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5580beaf48e0 con 0x5580bec40780 2024-04-26T18:27:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10b4bb0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5580beb8a7a0 con 0x5580bec465e0 2024-04-26T18:27:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec45cc0 0x5580bec460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec45cc0 0x5580bec460a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49570/0 (socket says 172.21.15.84:49570) 2024-04-26T18:27:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10a3fff700 1 -- 172.21.15.84:0/4022655125 learned_addr learned my addr 172.21.15.84:0/4022655125 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:38.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10a88bb700 1 --2- 172.21.15.84:0/4022655125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec40780 0x5580bec40be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10a3fff700 1 -- 172.21.15.84:0/4022655125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec465e0 msgr2=0x5580bec40160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10a3fff700 1 --2- 172.21.15.84:0/4022655125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec465e0 0x5580bec40160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10a3fff700 1 -- 172.21.15.84:0/4022655125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec40780 msgr2=0x5580bec40be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10a3fff700 1 --2- 172.21.15.84:0/4022655125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec40780 0x5580bec40be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.173+0000 7f10a3fff700 1 -- 172.21.15.84:0/4022655125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580beadb880 con 0x5580bec45cc0 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.174+0000 7f10a3fff700 1 --2- 172.21.15.84:0/4022655125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec45cc0 0x5580bec460a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1094009350 tx=0x7f109409d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9236d09050bda49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.174+0000 7f108a7fc700 1 -- 172.21.15.84:0/4022655125 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1094022470 con 0x5580bec45cc0 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.174+0000 7f108a7fc700 1 -- 172.21.15.84:0/4022655125 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f109409d750 con 0x5580bec45cc0 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.174+0000 7f108a7fc700 1 -- 172.21.15.84:0/4022655125 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109409dc10 con 0x5580bec45cc0 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.174+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/4022655125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec45cc0 msgr2=0x5580bec460a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/4022655125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec45cc0 0x5580bec460a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f1094009350 tx=0x7f109409d360 comp rx=0 tx=0).stop 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/4022655125 shutdown_connections 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/4022655125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec45cc0 0x5580bec460a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/4022655125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec40780 0x5580bec40be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/4022655125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec465e0 0x5580bec40160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/4022655125 >> 172.21.15.84:0/4022655125 conn(0x5580beb6a570 msgr2=0x5580bec36900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/4022655125 shutdown_connections 2024-04-26T18:27:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/4022655125 wait complete. 2024-04-26T18:27:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 -- start start 2024-04-26T18:27:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 Processor -- start 2024-04-26T18:27:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec40780 0x5580bec4c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec45cc0 0x5580bec497c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 0x5580bec49d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5580beb8e7e0 con 0x5580bec465e0 2024-04-26T18:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5580beb48a30 con 0x5580bec40780 2024-04-26T18:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.175+0000 7f10b4bb0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5580bec4ce80 con 0x5580bec45cc0 2024-04-26T18:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10b4bb0b80 10 client.0 _ll_register_callbacks cb 0x5580beb26900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10a88bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 0x5580bec49d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10a88bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 0x5580bec49d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49578/0 (socket says 172.21.15.84:49578) 2024-04-26T18:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10a88bb700 1 -- 172.21.15.84:0/3027635283 learned_addr learned my addr 172.21.15.84:0/3027635283 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10a37fe700 1 --2- 172.21.15.84:0/3027635283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec45cc0 0x5580bec497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10a3fff700 1 --2- 172.21.15.84:0/3027635283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec40780 0x5580bec4c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10a88bb700 1 -- 172.21.15.84:0/3027635283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec45cc0 msgr2=0x5580bec497c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10a88bb700 1 --2- 172.21.15.84:0/3027635283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec45cc0 0x5580bec497c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10a88bb700 1 -- 172.21.15.84:0/3027635283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec40780 msgr2=0x5580bec4c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10a88bb700 1 --2- 172.21.15.84:0/3027635283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec40780 0x5580bec4c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10a88bb700 1 -- 172.21.15.84:0/3027635283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580beb4c430 con 0x5580bec465e0 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10a88bb700 1 --2- 172.21.15.84:0/3027635283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 0x5580bec49d00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f109c009a20 tx=0x7f109c009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d82a98f4c899040b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10837fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10837fe700 1 -- 172.21.15.84:0/3027635283 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c01a060 con 0x5580bec465e0 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10837fe700 1 -- 172.21.15.84:0/3027635283 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f109c013710 con 0x5580bec465e0 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/3027635283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f109409d750 con 0x5580bec465e0 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.176+0000 7f10837fe700 1 -- 172.21.15.84:0/3027635283 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f109c013950 con 0x5580bec465e0 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.177+0000 7f10b4bb0b80 10 client.15138 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.177+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/3027635283 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5580bec57710 con 0x5580bec465e0 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.177+0000 7f10837fe700 1 -- 172.21.15.84:0/3027635283 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f109c020410 con 0x5580bec465e0 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.177+0000 7f10837fe700 1 -- 172.21.15.84:0/3027635283 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f109c015400 con 0x5580bec465e0 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.177+0000 7f10b4bb0b80 10 client.15138 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.177+0000 7f10b4bb0b80 10 client.15138 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.177+0000 7f10b4bb0b80 20 client.15138 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.177+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/3027635283 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5580bec59cc0 0x5580bec5b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.177+0000 7f10b4bb0b80 4 client.15138 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:38.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.177+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/3027635283 --> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f109409d750 con 0x5580bec59cc0 2024-04-26T18:27:38.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.177+0000 7f10a3fff700 1 --2- 172.21.15.84:0/3027635283 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5580bec59cc0 0x5580bec5b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.178+0000 7f10837fe700 10 client.15138 ms_handle_connect on v2:172.21.15.103:6834/2529642104 2024-04-26T18:27:38.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.178+0000 7f10a3fff700 1 --2- 172.21.15.84:0/3027635283 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5580bec59cc0 0x5580bec5b130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ade112bf6faa2967 server_cookie=6d1ad3aec4153419 in_seq=0 out_seq=0 2024-04-26T18:27:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10837fe700 1 -- 172.21.15.84:0/3027635283 <== mds.4 v2:172.21.15.103:6834/2529642104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f109409d750 con 0x5580bec59cc0 2024-04-26T18:27:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10837fe700 10 client.15138 handle_command_reply: tid=0 2024-04-26T18:27:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10b4bb0b80 1 client.15138 shutdown 2024-04-26T18:27:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/3027635283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 msgr2=0x5580bec49d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/3027635283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 0x5580bec49d00 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f109c009a20 tx=0x7f109c009a50 comp rx=0 tx=0).stop 2024-04-26T18:27:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/3027635283 shutdown_connections 2024-04-26T18:27:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/3027635283 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5580bec59cc0 0x5580bec5b130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/3027635283 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 0x5580bec49d00 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10837fe700 0 client.15138 ms_handle_reset on v2:172.21.15.103:6834/2529642104 2024-04-26T18:27:38.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/3027635283 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec40780 0x5580bec4c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/3027635283 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec45cc0 0x5580bec497c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/3027635283 >> 172.21.15.84:0/3027635283 conn(0x5580beb6a570 msgr2=0x5580bec3ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:38.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/3027635283 shutdown_connections 2024-04-26T18:27:38.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/3027635283 wait complete. 2024-04-26T18:27:38.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.179+0000 7f10b4bb0b80 20 client.15138 trim_cache size 0 max 16384 2024-04-26T18:27:38.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10b4bb0b80 1 Processor -- start 2024-04-26T18:27:38.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10b4bb0b80 1 -- start start 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10b4bb0b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec40780 0x5580bec49d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10b4bb0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec45cc0 0x5580bec64590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10b4bb0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec465e0 0x5580bec64ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10b4bb0b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5580bec59470 con 0x5580bec40780 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10b4bb0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5580bec59b40 con 0x5580bec45cc0 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10b4bb0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5580bec5a210 con 0x5580bec465e0 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec40780 0x5580bec49d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10a3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec40780 0x5580bec49d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49588/0 (socket says 172.21.15.84:49588) 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10a3fff700 1 -- 172.21.15.84:0/978013515 learned_addr learned my addr 172.21.15.84:0/978013515 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10a88bb700 1 --2- 172.21.15.84:0/978013515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec465e0 0x5580bec64ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10a3fff700 1 -- 172.21.15.84:0/978013515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec465e0 msgr2=0x5580bec64ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10a3fff700 1 --2- 172.21.15.84:0/978013515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec465e0 0x5580bec64ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10a3fff700 1 -- 172.21.15.84:0/978013515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec45cc0 msgr2=0x5580bec64590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:38.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10a3fff700 1 --2- 172.21.15.84:0/978013515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec45cc0 0x5580bec64590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10a3fff700 1 -- 172.21.15.84:0/978013515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580bec375e0 con 0x5580bec40780 2024-04-26T18:27:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.203+0000 7f10a3fff700 1 --2- 172.21.15.84:0/978013515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec40780 0x5580bec49d30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f10940098e0 tx=0x7f1094022a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a78af9328bd0102 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.204+0000 7f1089ffb700 1 -- 172.21.15.84:0/978013515 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10940ba070 con 0x5580bec40780 2024-04-26T18:27:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.204+0000 7f1089ffb700 1 -- 172.21.15.84:0/978013515 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f109409ba50 con 0x5580bec40780 2024-04-26T18:27:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.204+0000 7f1089ffb700 1 -- 172.21.15.84:0/978013515 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10940a5ea0 con 0x5580bec40780 2024-04-26T18:27:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.204+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/978013515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec40780 msgr2=0x5580bec49d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.204+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/978013515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec40780 0x5580bec49d30 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f10940098e0 tx=0x7f1094022a10 comp rx=0 tx=0).stop 2024-04-26T18:27:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.204+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/978013515 shutdown_connections 2024-04-26T18:27:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.204+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/978013515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec40780 0x5580bec49d30 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.204+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/978013515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec465e0 0x5580bec64ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.204+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/978013515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec45cc0 0x5580bec64590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.204+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/978013515 >> 172.21.15.84:0/978013515 conn(0x5580beb6a570 msgr2=0x5580bec480b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:38.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.204+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/978013515 shutdown_connections 2024-04-26T18:27:38.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.204+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/978013515 wait complete. 2024-04-26T18:27:38.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10b4bb0b80 1 -- start start 2024-04-26T18:27:38.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10b4bb0b80 1 Processor -- start 2024-04-26T18:27:38.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10b4bb0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec40780 0x5580bec64790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10b4bb0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec45cc0 0x5580bec5f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10b4bb0b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 0x5580bec5faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10b4bb0b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5580bec5afb0 con 0x5580bec465e0 2024-04-26T18:27:38.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10b4bb0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5580bec5a8e0 con 0x5580bec45cc0 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10b4bb0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5580bec5b680 con 0x5580bec40780 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10b4bb0b80 10 client.0 _ll_register_callbacks cb 0x5580beb26900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10a88bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 0x5580bec5faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10a88bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 0x5580bec5faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49604/0 (socket says 172.21.15.84:49604) 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10a88bb700 1 -- 172.21.15.84:0/2817979718 learned_addr learned my addr 172.21.15.84:0/2817979718 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10a37fe700 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec45cc0 0x5580bec5f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.205+0000 7f10a3fff700 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec40780 0x5580bec64790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10a88bb700 1 -- 172.21.15.84:0/2817979718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec40780 msgr2=0x5580bec64790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10a88bb700 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec40780 0x5580bec64790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10a88bb700 1 -- 172.21.15.84:0/2817979718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec45cc0 msgr2=0x5580bec5f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10a88bb700 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec45cc0 0x5580bec5f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10a88bb700 1 -- 172.21.15.84:0/2817979718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5580bec5ffe0 con 0x5580bec465e0 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10a3fff700 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec40780 0x5580bec64790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:38.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10a88bb700 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 0x5580bec5faa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f109c004c20 tx=0x7f109c015760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bd4c3c3cbd110b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10837fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10837fe700 1 -- 172.21.15.84:0/2817979718 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109c015e60 con 0x5580bec465e0 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10837fe700 1 -- 172.21.15.84:0/2817979718 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f109c00a400 con 0x5580bec465e0 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/2817979718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f109409ba50 con 0x5580bec465e0 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10837fe700 1 -- 172.21.15.84:0/2817979718 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f109c01a030 con 0x5580bec465e0 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10b4bb0b80 10 client.15144 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/2817979718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5580becc1ab0 con 0x5580bec465e0 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10837fe700 1 -- 172.21.15.84:0/2817979718 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f109409ba50 con 0x5580bec465e0 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10837fe700 1 -- 172.21.15.84:0/2817979718 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f109c0289b0 con 0x5580bec465e0 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10b4bb0b80 10 client.15144 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10b4bb0b80 10 client.15144 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10b4bb0b80 20 client.15144 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5580becc65d0 0x5580becc52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:38.212 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10b4bb0b80 4 client.15144 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.206+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/2817979718 --> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] -- command(tid 0: {"prefix": "status"}) -- 0x5580beb9c1c0 con 0x5580becc65d0 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.207+0000 7f10a3fff700 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5580becc65d0 0x5580becc52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.207+0000 7f10a3fff700 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5580becc65d0 0x5580becc52e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8709dfbf965c01ea server_cookie=87f00b79a144d49a in_seq=0 out_seq=0 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.207+0000 7f10837fe700 10 client.15144 ms_handle_connect on v2:172.21.15.103:6834/2529642104 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.207+0000 7f10837fe700 1 -- 172.21.15.84:0/2817979718 <== mds.4 v2:172.21.15.103:6834/2529642104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5580beb9c1c0 con 0x5580becc65d0 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.207+0000 7f10837fe700 10 client.15144 handle_command_reply: tid=0 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.207+0000 7f10b4bb0b80 1 client.15144 shutdown 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.207+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/2817979718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 msgr2=0x5580bec5faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.207+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 0x5580bec5faa0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f109c004c20 tx=0x7f109c015760 comp rx=0 tx=0).stop 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.207+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/2817979718 shutdown_connections 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.207+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5580becc65d0 0x5580becc52e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5580bec465e0 0x5580bec5faa0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5580bec40780 0x5580bec64790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10837fe700 0 client.15144 ms_handle_reset on v2:172.21.15.103:6834/2529642104 2024-04-26T18:27:38.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10b4bb0b80 1 --2- 172.21.15.84:0/2817979718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5580bec45cc0 0x5580bec5f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/2817979718 >> 172.21.15.84:0/2817979718 conn(0x5580beb6a570 msgr2=0x5580bec437b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:38.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/2817979718 shutdown_connections 2024-04-26T18:27:38.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10b4bb0b80 1 -- 172.21.15.84:0/2817979718 wait complete. 2024-04-26T18:27:38.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10b4bb0b80 20 client.15144 trim_cache size 0 max 16384 2024-04-26T18:27:38.214 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:27:38.214 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:27:38.214 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 4, 2024-04-26T18:27:38.214 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24407, 2024-04-26T18:27:38.214 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:27:38.214 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:27:38.215 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:27:38.215 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 179.91521785699999, 2024-04-26T18:27:38.215 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 28, 2024-04-26T18:27:38.215 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 93, 2024-04-26T18:27:38.215 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 89, 2024-04-26T18:27:38.215 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 230.587123695 2024-04-26T18:27:38.215 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:27:38.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10aa31e700 1 -- 172.21.15.84:0/2440312003 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f108c0a3c40 msgr2=0x7f108c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:38.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10aa31e700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f108c0a3c40 0x7f108c0a6100 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f10a407ab70 tx=0x7f1098008040 comp rx=0 tx=0).stop 2024-04-26T18:27:38.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10aa31e700 1 -- 172.21.15.84:0/2440312003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a41034d0 msgr2=0x7f10a419f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:38.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10aa31e700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a41034d0 0x7f10a419f8f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1094000c00 tx=0x7f10940047b0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10aa31e700 1 -- 172.21.15.84:0/2440312003 shutdown_connections 2024-04-26T18:27:38.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10aa31e700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f108c0a3c40 0x7f108c0a6100 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10aa31e700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10a41034d0 0x7f10a419f8f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10aa31e700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10a41094f0 0x7f10a4199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10aa31e700 1 --2- 172.21.15.84:0/2440312003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10a4103e90 0x7f10a419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:38.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.208+0000 7f10aa31e700 1 -- 172.21.15.84:0/2440312003 >> 172.21.15.84:0/2440312003 conn(0x7f10a407a4b0 msgr2=0x7f10a4102220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:38.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.209+0000 7f10aa31e700 1 -- 172.21.15.84:0/2440312003 shutdown_connections 2024-04-26T18:27:38.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:38.209+0000 7f10aa31e700 1 -- 172.21.15.84:0/2440312003 wait complete. 2024-04-26T18:27:38.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:38 smithi084 ceph-mon[29141]: pgmap v399: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:38.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:38 smithi103 ceph-mon[32727]: pgmap v399: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:38.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:38 smithi174 ceph-mon[32705]: pgmap v399: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:38.905 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-26T18:27:39.145 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:40.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:40 smithi084 ceph-mon[29141]: pgmap v400: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:40.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:40 smithi103 ceph-mon[32727]: pgmap v400: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:40.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:40 smithi174 ceph-mon[32705]: pgmap v400: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:40.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.868+0000 7f04219cb700 1 -- 172.21.15.84:0/2740829327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c075f70 msgr2=0x7f041c1140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:40.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.868+0000 7f04219cb700 1 --2- 172.21.15.84:0/2740829327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c075f70 0x7f041c1140b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f0410009990 tx=0x7f0410009ca0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.868+0000 7f04219cb700 1 -- 172.21.15.84:0/2740829327 shutdown_connections 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.868+0000 7f04219cb700 1 --2- 172.21.15.84:0/2740829327 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f041c1145f0 0x7f041c1169e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.868+0000 7f04219cb700 1 --2- 172.21.15.84:0/2740829327 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f041c075650 0x7f041c075a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.868+0000 7f04219cb700 1 --2- 172.21.15.84:0/2740829327 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c075f70 0x7f041c1140b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.869+0000 7f04219cb700 1 -- 172.21.15.84:0/2740829327 >> 172.21.15.84:0/2740829327 conn(0x7f041c07a580 msgr2=0x7f041c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.869+0000 7f04219cb700 1 -- 172.21.15.84:0/2740829327 shutdown_connections 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.869+0000 7f04219cb700 1 -- 172.21.15.84:0/2740829327 wait complete. 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.869+0000 7f04219cb700 1 Processor -- start 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.869+0000 7f04219cb700 1 -- start start 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f04219cb700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f041c075650 0x7f041c1a02c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f04219cb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c075f70 0x7f041c1a0800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f04219cb700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f041c1145f0 0x7f041c1a4b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f04219cb700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f041c119cd0 con 0x7f041c075f70 2024-04-26T18:27:40.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f04219cb700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f041c119b50 con 0x7f041c1145f0 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f04219cb700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f041c119e50 con 0x7f041c075650 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f041a7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c075f70 0x7f041c1a0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f041a7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c075f70 0x7f041c1a0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49618/0 (socket says 172.21.15.84:49618) 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f041a7fc700 1 -- 172.21.15.84:0/2573158799 learned_addr learned my addr 172.21.15.84:0/2573158799 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f041affd700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f041c075650 0x7f041c1a02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f041a7fc700 1 -- 172.21.15.84:0/2573158799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f041c075650 msgr2=0x7f041c1a02c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f041a7fc700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f041c075650 0x7f041c1a02c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f041a7fc700 1 -- 172.21.15.84:0/2573158799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f041c1145f0 msgr2=0x7f041c1a4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f041a7fc700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f041c1145f0 0x7f041c1a4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f041a7fc700 1 -- 172.21.15.84:0/2573158799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f041c1a52d0 con 0x7f041c075f70 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f041affd700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f041c075650 0x7f041c1a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f041a7fc700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c075f70 0x7f041c1a0800 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0410000c00 tx=0x7f04100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f04209c9700 1 -- 172.21.15.84:0/2573158799 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410027070 con 0x7f041c075f70 2024-04-26T18:27:40.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f04219cb700 1 -- 172.21.15.84:0/2573158799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f041c1a5560 con 0x7f041c075f70 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f04209c9700 1 -- 172.21.15.84:0/2573158799 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04100041d0 con 0x7f041c075f70 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f04209c9700 1 -- 172.21.15.84:0/2573158799 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041000f460 con 0x7f041c075f70 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.870+0000 7f04219cb700 1 -- 172.21.15.84:0/2573158799 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f041c1ace40 con 0x7f041c075f70 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.873+0000 7f04209c9700 1 -- 172.21.15.84:0/2573158799 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f041002a070 con 0x7f041c075f70 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.873+0000 7f04209c9700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f04040a3c40 0x7f04040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.873+0000 7f041affd700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f04040a3c40 0x7f04040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.874+0000 7f04209c9700 1 -- 172.21.15.84:0/2573158799 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f04100d1470 con 0x7f041c075f70 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.874+0000 7f041affd700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f04040a3c40 0x7f04040a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f040c0099d0 tx=0x7f040c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.874+0000 7f042c25db80 1 Processor -- start 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.874+0000 7f042c25db80 1 -- start start 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.874+0000 7f042c25db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b8ad0 0x5615817b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.874+0000 7f042c25db80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 0x5615817b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.874+0000 7f042c25db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b32d0 0x5615817b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.874+0000 7f042c25db80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5615817056f0 con 0x5615817b93f0 2024-04-26T18:27:40.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.874+0000 7f042c25db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615816a4590 con 0x5615817b8ad0 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.874+0000 7f042c25db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615817016b0 con 0x5615817b32d0 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.875+0000 7f041a7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 0x5615817b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.875+0000 7f041a7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 0x5615817b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49624/0 (socket says 172.21.15.84:49624) 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.875+0000 7f041a7fc700 1 -- 172.21.15.84:0/1850611764 learned_addr learned my addr 172.21.15.84:0/1850611764 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.875+0000 7f041a7fc700 1 -- 172.21.15.84:0/1850611764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b32d0 msgr2=0x5615817b3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.875+0000 7f041affd700 1 --2- 172.21.15.84:0/1850611764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b8ad0 0x5615817b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.875+0000 7f041a7fc700 1 --2- 172.21.15.84:0/1850611764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b32d0 0x5615817b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.875+0000 7f041a7fc700 1 -- 172.21.15.84:0/1850611764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b8ad0 msgr2=0x5615817b8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.875+0000 7f041a7fc700 1 --2- 172.21.15.84:0/1850611764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b8ad0 0x5615817b8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.875+0000 7f041a7fc700 1 -- 172.21.15.84:0/1850611764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561581652880 con 0x5615817b93f0 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.875+0000 7f041a7fc700 1 --2- 172.21.15.84:0/1850611764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 0x5615817b2cb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0410009350 tx=0x7f041009d360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746ec1755504cc24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.875+0000 7f0401ffb700 1 -- 172.21.15.84:0/1850611764 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0410022470 con 0x5615817b93f0 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.875+0000 7f0401ffb700 1 -- 172.21.15.84:0/1850611764 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f041009d750 con 0x5615817b93f0 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.875+0000 7f0401ffb700 1 -- 172.21.15.84:0/1850611764 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041009dc10 con 0x5615817b93f0 2024-04-26T18:27:40.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.876+0000 7f042c25db80 1 -- 172.21.15.84:0/1850611764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 msgr2=0x5615817b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.876+0000 7f042c25db80 1 --2- 172.21.15.84:0/1850611764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 0x5615817b2cb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0410009350 tx=0x7f041009d360 comp rx=0 tx=0).stop 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.876+0000 7f042c25db80 1 -- 172.21.15.84:0/1850611764 shutdown_connections 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.876+0000 7f042c25db80 1 --2- 172.21.15.84:0/1850611764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b32d0 0x5615817b3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.876+0000 7f042c25db80 1 --2- 172.21.15.84:0/1850611764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b8ad0 0x5615817b8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.876+0000 7f042c25db80 1 --2- 172.21.15.84:0/1850611764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 0x5615817b2cb0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.876+0000 7f042c25db80 1 -- 172.21.15.84:0/1850611764 >> 172.21.15.84:0/1850611764 conn(0x5615816dfcf0 msgr2=0x5615817ae1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.876+0000 7f042c25db80 1 -- 172.21.15.84:0/1850611764 shutdown_connections 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.876+0000 7f042c25db80 1 -- 172.21.15.84:0/1850611764 wait complete. 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.876+0000 7f042c25db80 1 -- start start 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.876+0000 7f042c25db80 1 Processor -- start 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f042c25db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b32d0 0x5615817bf180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f042c25db80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 0x5615817bc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f042c25db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b93f0 0x5615817bc9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f042c25db80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5615817056f0 con 0x5615817b8ad0 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f042c25db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56158166b900 con 0x5615817b93f0 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f042c25db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615817bfcd0 con 0x5615817b32d0 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f042c25db80 10 client.0 _ll_register_callbacks cb 0x561581699530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f041a7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 0x5615817bc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f041a7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 0x5615817bc480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49638/0 (socket says 172.21.15.84:49638) 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f041a7fc700 1 -- 172.21.15.84:0/1468724520 learned_addr learned my addr 172.21.15.84:0/1468724520 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f041affd700 1 --2- 172.21.15.84:0/1468724520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b32d0 0x5615817bf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f041b7fe700 1 --2- 172.21.15.84:0/1468724520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b93f0 0x5615817bc9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f041a7fc700 1 -- 172.21.15.84:0/1468724520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b32d0 msgr2=0x5615817bf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f041a7fc700 1 --2- 172.21.15.84:0/1468724520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b32d0 0x5615817bf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f041a7fc700 1 -- 172.21.15.84:0/1468724520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b93f0 msgr2=0x5615817bc9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f041a7fc700 1 --2- 172.21.15.84:0/1468724520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b93f0 0x5615817bc9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f041a7fc700 1 -- 172.21.15.84:0/1468724520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615817bd070 con 0x5615817b8ad0 2024-04-26T18:27:40.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f041a7fc700 1 --2- 172.21.15.84:0/1468724520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 0x5615817bc480 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0410099bb0 tx=0x7f04100a4130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=660a3ac4bb093b36 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f03faffd700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f03faffd700 1 -- 172.21.15.84:0/1468724520 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04100b5070 con 0x5615817b8ad0 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f03faffd700 1 -- 172.21.15.84:0/1468724520 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f041009dbb0 con 0x5615817b8ad0 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.877+0000 7f042c25db80 1 -- 172.21.15.84:0/1468724520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f041009d750 con 0x5615817b8ad0 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.878+0000 7f03faffd700 1 -- 172.21.15.84:0/1468724520 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0410022470 con 0x5615817b8ad0 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.878+0000 7f042c25db80 10 client.15156 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.878+0000 7f042c25db80 1 -- 172.21.15.84:0/1468724520 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615817ca610 con 0x5615817b8ad0 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.878+0000 7f03faffd700 1 -- 172.21.15.84:0/1468724520 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f041009d750 con 0x5615817b8ad0 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.878+0000 7f03faffd700 1 -- 172.21.15.84:0/1468724520 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0410084020 con 0x5615817b8ad0 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.878+0000 7f042c25db80 10 client.15156 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.878+0000 7f042c25db80 10 client.15156 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.878+0000 7f042c25db80 20 client.15156 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.878+0000 7f042c25db80 1 --2- 172.21.15.84:0/1468724520 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x5615817cc9d0 0x5615817cde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.878+0000 7f042c25db80 4 client.15156 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:40.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.878+0000 7f042c25db80 1 -- 172.21.15.84:0/1468724520 --> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615817130d0 con 0x5615817cc9d0 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.878+0000 7f041affd700 1 --2- 172.21.15.84:0/1468724520 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x5615817cc9d0 0x5615817cde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.879+0000 7f03faffd700 10 client.15156 ms_handle_connect on v2:172.21.15.174:6836/1532309720 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.879+0000 7f041affd700 1 --2- 172.21.15.84:0/1468724520 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x5615817cc9d0 0x5615817cde40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6332a7769857b9c0 server_cookie=29f699fd5215a879 in_seq=0 out_seq=0 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.879+0000 7f03faffd700 1 -- 172.21.15.84:0/1468724520 <== mds.3 v2:172.21.15.174:6836/1532309720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615817130d0 con 0x5615817cc9d0 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.879+0000 7f03faffd700 10 client.15156 handle_command_reply: tid=0 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.879+0000 7f042c25db80 1 client.15156 shutdown 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.880+0000 7f042c25db80 1 -- 172.21.15.84:0/1468724520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 msgr2=0x5615817bc480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.880+0000 7f042c25db80 1 --2- 172.21.15.84:0/1468724520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 0x5615817bc480 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0410099bb0 tx=0x7f04100a4130 comp rx=0 tx=0).stop 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.880+0000 7f042c25db80 1 -- 172.21.15.84:0/1468724520 shutdown_connections 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.880+0000 7f042c25db80 1 --2- 172.21.15.84:0/1468724520 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b93f0 0x5615817bc9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.880+0000 7f042c25db80 1 --2- 172.21.15.84:0/1468724520 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b32d0 0x5615817bf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.880+0000 7f042c25db80 1 --2- 172.21.15.84:0/1468724520 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x5615817cc9d0 0x5615817cde40 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.880+0000 7f042c25db80 1 --2- 172.21.15.84:0/1468724520 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 0x5615817bc480 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.880+0000 7f042c25db80 1 -- 172.21.15.84:0/1468724520 >> 172.21.15.84:0/1468724520 conn(0x5615816dfcf0 msgr2=0x5615817adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:40.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.880+0000 7f03faffd700 0 client.15156 ms_handle_reset on v2:172.21.15.174:6836/1532309720 2024-04-26T18:27:40.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.880+0000 7f042c25db80 1 -- 172.21.15.84:0/1468724520 shutdown_connections 2024-04-26T18:27:40.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.880+0000 7f042c25db80 1 -- 172.21.15.84:0/1468724520 wait complete. 2024-04-26T18:27:40.891 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.880+0000 7f042c25db80 20 client.15156 trim_cache size 0 max 16384 2024-04-26T18:27:40.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.900+0000 7f042c25db80 1 Processor -- start 2024-04-26T18:27:40.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.900+0000 7f042c25db80 1 -- start start 2024-04-26T18:27:40.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.900+0000 7f042c25db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b32d0 0x5615817cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.900+0000 7f042c25db80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 0x5615817b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.900+0000 7f042c25db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b93f0 0x5615817bad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.900+0000 7f042c25db80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5615817cc180 con 0x5615817b8ad0 2024-04-26T18:27:40.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.900+0000 7f042c25db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615817cc850 con 0x5615817b32d0 2024-04-26T18:27:40.902 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.900+0000 7f042c25db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615817ccf20 con 0x5615817b93f0 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.900+0000 7f041a7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 0x5615817b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.900+0000 7f041a7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 0x5615817b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49648/0 (socket says 172.21.15.84:49648) 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.900+0000 7f041a7fc700 1 -- 172.21.15.84:0/1324382355 learned_addr learned my addr 172.21.15.84:0/1324382355 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.901+0000 7f041b7fe700 1 --2- 172.21.15.84:0/1324382355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b93f0 0x5615817bad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.901+0000 7f041affd700 1 --2- 172.21.15.84:0/1324382355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b32d0 0x5615817cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.901+0000 7f041a7fc700 1 -- 172.21.15.84:0/1324382355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b93f0 msgr2=0x5615817bad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.901+0000 7f041a7fc700 1 --2- 172.21.15.84:0/1324382355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b93f0 0x5615817bad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.901+0000 7f041a7fc700 1 -- 172.21.15.84:0/1324382355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b32d0 msgr2=0x5615817cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.901+0000 7f041a7fc700 1 --2- 172.21.15.84:0/1324382355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b32d0 0x5615817cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.901+0000 7f041a7fc700 1 -- 172.21.15.84:0/1324382355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615817af720 con 0x5615817b8ad0 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.901+0000 7f041b7fe700 1 --2- 172.21.15.84:0/1324382355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b93f0 0x5615817bad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.901+0000 7f041affd700 1 --2- 172.21.15.84:0/1324382355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b32d0 0x5615817cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.901+0000 7f041a7fc700 1 --2- 172.21.15.84:0/1324382355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 0x5615817b2cb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0410029000 tx=0x7f041009bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7fc630495c368f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.901+0000 7f04017fa700 1 -- 172.21.15.84:0/1324382355 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041009d750 con 0x5615817b8ad0 2024-04-26T18:27:40.903 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.901+0000 7f04017fa700 1 -- 172.21.15.84:0/1324382355 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04100bb070 con 0x5615817b8ad0 2024-04-26T18:27:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.901+0000 7f04017fa700 1 -- 172.21.15.84:0/1324382355 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041009d9b0 con 0x5615817b8ad0 2024-04-26T18:27:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.902+0000 7f042c25db80 1 -- 172.21.15.84:0/1324382355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 msgr2=0x5615817b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.902+0000 7f042c25db80 1 --2- 172.21.15.84:0/1324382355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 0x5615817b2cb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f0410029000 tx=0x7f041009bc60 comp rx=0 tx=0).stop 2024-04-26T18:27:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.902+0000 7f042c25db80 1 -- 172.21.15.84:0/1324382355 shutdown_connections 2024-04-26T18:27:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.902+0000 7f042c25db80 1 --2- 172.21.15.84:0/1324382355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b93f0 0x5615817bad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.902+0000 7f042c25db80 1 --2- 172.21.15.84:0/1324382355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b32d0 0x5615817cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.902+0000 7f042c25db80 1 --2- 172.21.15.84:0/1324382355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b8ad0 0x5615817b2cb0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.902+0000 7f042c25db80 1 -- 172.21.15.84:0/1324382355 >> 172.21.15.84:0/1324382355 conn(0x5615816dfcf0 msgr2=0x5615817b6620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.902+0000 7f042c25db80 1 -- 172.21.15.84:0/1324382355 shutdown_connections 2024-04-26T18:27:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.902+0000 7f042c25db80 1 -- 172.21.15.84:0/1324382355 wait complete. 2024-04-26T18:27:40.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.902+0000 7f042c25db80 1 -- start start 2024-04-26T18:27:40.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f042c25db80 1 Processor -- start 2024-04-26T18:27:40.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f042c25db80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b32d0 0x5615817c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f042c25db80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b8ad0 0x5615817c8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f042c25db80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 0x5615817c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f042c25db80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5615817cdcc0 con 0x5615817b93f0 2024-04-26T18:27:40.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f042c25db80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615817cd5f0 con 0x5615817b32d0 2024-04-26T18:27:40.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f042c25db80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615817ce390 con 0x5615817b8ad0 2024-04-26T18:27:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f042c25db80 10 client.0 _ll_register_callbacks cb 0x561581699530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f041b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 0x5615817c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f041b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 0x5615817c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49656/0 (socket says 172.21.15.84:49656) 2024-04-26T18:27:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f041b7fe700 1 -- 172.21.15.84:0/3276115029 learned_addr learned my addr 172.21.15.84:0/3276115029 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f041affd700 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b32d0 0x5615817c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f041a7fc700 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b8ad0 0x5615817c8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f041b7fe700 1 -- 172.21.15.84:0/3276115029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b8ad0 msgr2=0x5615817c8500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f041b7fe700 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b8ad0 0x5615817c8500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f041b7fe700 1 -- 172.21.15.84:0/3276115029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b32d0 msgr2=0x5615817c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f041b7fe700 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b32d0 0x5615817c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f041b7fe700 1 -- 172.21.15.84:0/3276115029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615817d63b0 con 0x5615817b93f0 2024-04-26T18:27:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f041a7fc700 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b8ad0 0x5615817c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:40.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.903+0000 7f041affd700 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b32d0 0x5615817c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f041b7fe700 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 0x5615817c8a40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f041400ca80 tx=0x7f041400ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1d0e3632af83fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f03faffd700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f03faffd700 1 -- 172.21.15.84:0/3276115029 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041401c060 con 0x5615817b93f0 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f03faffd700 1 -- 172.21.15.84:0/3276115029 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f04140098c0 con 0x5615817b93f0 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f042c25db80 1 -- 172.21.15.84:0/3276115029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04100bb070 con 0x5615817b93f0 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f03faffd700 1 -- 172.21.15.84:0/3276115029 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f0414009ac0 con 0x5615817b93f0 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f042c25db80 10 client.15162 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f042c25db80 1 -- 172.21.15.84:0/3276115029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5615817d7660 con 0x5615817b93f0 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f03faffd700 1 -- 172.21.15.84:0/3276115029 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04100bb070 con 0x5615817b93f0 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f03faffd700 1 -- 172.21.15.84:0/3276115029 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f0414022410 con 0x5615817b93f0 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f042c25db80 10 client.15162 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f042c25db80 10 client.15162 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f042c25db80 20 client.15162 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:40.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f042c25db80 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x561581838050 0x561581836d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f042c25db80 4 client.15162 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.904+0000 7f042c25db80 1 -- 172.21.15.84:0/3276115029 --> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] -- command(tid 0: {"prefix": "status"}) -- 0x56158170d170 con 0x561581838050 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.905+0000 7f041affd700 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x561581838050 0x561581836d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.905+0000 7f03faffd700 10 client.15162 ms_handle_connect on v2:172.21.15.174:6836/1532309720 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.905+0000 7f041affd700 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x561581838050 0x561581836d60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=bc72fb004fb09e23 server_cookie=df4813546cbd4972 in_seq=0 out_seq=0 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.905+0000 7f03faffd700 1 -- 172.21.15.84:0/3276115029 <== mds.3 v2:172.21.15.174:6836/1532309720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56158170d170 con 0x561581838050 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f03faffd700 10 client.15162 handle_command_reply: tid=0 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f042c25db80 1 client.15162 shutdown 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f042c25db80 1 -- 172.21.15.84:0/3276115029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 msgr2=0x5615817c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f042c25db80 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 0x5615817c8a40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f041400ca80 tx=0x7f041400ce40 comp rx=0 tx=0).stop 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f042c25db80 1 -- 172.21.15.84:0/3276115029 shutdown_connections 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f042c25db80 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x561581838050 0x561581836d60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f042c25db80 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615817b93f0 0x5615817c8a40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f042c25db80 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615817b32d0 0x5615817c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f042c25db80 1 --2- 172.21.15.84:0/3276115029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615817b8ad0 0x5615817c8500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f03faffd700 0 client.15162 ms_handle_reset on v2:172.21.15.174:6836/1532309720 2024-04-26T18:27:40.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f042c25db80 1 -- 172.21.15.84:0/3276115029 >> 172.21.15.84:0/3276115029 conn(0x5615816dfcf0 msgr2=0x5615817bacd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:40.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f042c25db80 1 -- 172.21.15.84:0/3276115029 shutdown_connections 2024-04-26T18:27:40.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f042c25db80 1 -- 172.21.15.84:0/3276115029 wait complete. 2024-04-26T18:27:40.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f042c25db80 20 client.15162 trim_cache size 0 max 16384 2024-04-26T18:27:40.912 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:27:40.912 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:27:40.912 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 3, 2024-04-26T18:27:40.912 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24418, 2024-04-26T18:27:40.912 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:27:40.912 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:27:40.913 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:27:40.913 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 183.620798541, 2024-04-26T18:27:40.913 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 28, 2024-04-26T18:27:40.913 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 93, 2024-04-26T18:27:40.913 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 89, 2024-04-26T18:27:40.913 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 227.77201178300001 2024-04-26T18:27:40.913 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:27:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.906+0000 7f04219cb700 1 -- 172.21.15.84:0/2573158799 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f04040a3c40 msgr2=0x7f04040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.907+0000 7f04219cb700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f04040a3c40 0x7f04040a6100 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f040c0099d0 tx=0x7f040c008040 comp rx=0 tx=0).stop 2024-04-26T18:27:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.907+0000 7f04219cb700 1 -- 172.21.15.84:0/2573158799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c075f70 msgr2=0x7f041c1a0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.907+0000 7f04219cb700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c075f70 0x7f041c1a0800 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f0410000c00 tx=0x7f04100047b0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.907+0000 7f04219cb700 1 -- 172.21.15.84:0/2573158799 shutdown_connections 2024-04-26T18:27:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.907+0000 7f04219cb700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f04040a3c40 0x7f04040a6100 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.907+0000 7f04219cb700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f041c1145f0 0x7f041c1a4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.907+0000 7f04219cb700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f041c075650 0x7f041c1a02c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.907+0000 7f04219cb700 1 --2- 172.21.15.84:0/2573158799 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f041c075f70 0x7f041c1a0800 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.907+0000 7f04219cb700 1 -- 172.21.15.84:0/2573158799 >> 172.21.15.84:0/2573158799 conn(0x7f041c07a580 msgr2=0x7f041c113580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.907+0000 7f04219cb700 1 -- 172.21.15.84:0/2573158799 shutdown_connections 2024-04-26T18:27:40.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:40.907+0000 7f04219cb700 1 -- 172.21.15.84:0/2573158799 wait complete. 2024-04-26T18:27:41.606 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-26T18:27:41.842 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:42.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:42 smithi084 ceph-mon[29141]: pgmap v401: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:42.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:42 smithi103 ceph-mon[32727]: pgmap v401: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:42.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:42 smithi174 ceph-mon[32705]: pgmap v401: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:43.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.553+0000 7fea206e6700 1 -- 172.21.15.84:0/949360638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea18102320 msgr2=0x7fea181027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:43.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.553+0000 7fea206e6700 1 --2- 172.21.15.84:0/949360638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea18102320 0x7fea181027a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fea14009990 tx=0x7fea14009ca0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.553+0000 7fea206e6700 1 -- 172.21.15.84:0/949360638 shutdown_connections 2024-04-26T18:27:43.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.553+0000 7fea206e6700 1 --2- 172.21.15.84:0/949360638 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea18102ce0 0x7fea1810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.553+0000 7fea206e6700 1 --2- 172.21.15.84:0/949360638 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea18108390 0x7fea18108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.553+0000 7fea206e6700 1 --2- 172.21.15.84:0/949360638 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea18102320 0x7fea181027a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.553+0000 7fea206e6700 1 -- 172.21.15.84:0/949360638 >> 172.21.15.84:0/949360638 conn(0x7fea180792d0 msgr2=0x7fea180796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:43.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.553+0000 7fea206e6700 1 -- 172.21.15.84:0/949360638 shutdown_connections 2024-04-26T18:27:43.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.553+0000 7fea206e6700 1 -- 172.21.15.84:0/949360638 wait complete. 2024-04-26T18:27:43.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea206e6700 1 Processor -- start 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea206e6700 1 -- start start 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea206e6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea18102320 0x7fea1819f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea206e6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea18102ce0 0x7fea1819fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea206e6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea18108390 0x7fea18199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea206e6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fea181125b0 con 0x7fea18102ce0 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea206e6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fea18112430 con 0x7fea18102320 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea206e6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fea18112730 con 0x7fea18108390 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea1e482700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea18102320 0x7fea1819f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea1dc81700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea18102ce0 0x7fea1819fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea1dc81700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea18102ce0 0x7fea1819fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49680/0 (socket says 172.21.15.84:49680) 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea1e482700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea18102320 0x7fea1819f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:47148/0 (socket says 172.21.15.84:47148) 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea1dc81700 1 -- 172.21.15.84:0/475969087 learned_addr learned my addr 172.21.15.84:0/475969087 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea1ec83700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea18108390 0x7fea18199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea1dc81700 1 -- 172.21.15.84:0/475969087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea18108390 msgr2=0x7fea18199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:43.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea1dc81700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea18108390 0x7fea18199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea1dc81700 1 -- 172.21.15.84:0/475969087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea18102320 msgr2=0x7fea1819f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:43.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea1dc81700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea18102320 0x7fea1819f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.554+0000 7fea1dc81700 1 -- 172.21.15.84:0/475969087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea1819a1e0 con 0x7fea18102ce0 2024-04-26T18:27:43.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.555+0000 7fea1e482700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea18102320 0x7fea1819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:43.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.555+0000 7fea1dc81700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea18102ce0 0x7fea1819fe00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fea140057f0 tx=0x7fea14003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:43.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.555+0000 7fea0f7fe700 1 -- 172.21.15.84:0/475969087 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14027070 con 0x7fea18102ce0 2024-04-26T18:27:43.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.555+0000 7fea0f7fe700 1 -- 172.21.15.84:0/475969087 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea14004020 con 0x7fea18102ce0 2024-04-26T18:27:43.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.555+0000 7fea206e6700 1 -- 172.21.15.84:0/475969087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea1819a470 con 0x7fea18102ce0 2024-04-26T18:27:43.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.555+0000 7fea0f7fe700 1 -- 172.21.15.84:0/475969087 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea14022cf0 con 0x7fea18102ce0 2024-04-26T18:27:43.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.555+0000 7fea206e6700 1 -- 172.21.15.84:0/475969087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea181a6790 con 0x7fea18102ce0 2024-04-26T18:27:43.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.558+0000 7fea0f7fe700 1 -- 172.21.15.84:0/475969087 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fea1402a070 con 0x7fea18102ce0 2024-04-26T18:27:43.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.558+0000 7fea0f7fe700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fea040a3c40 0x7fea040a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.558+0000 7fea1e482700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fea040a3c40 0x7fea040a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.558+0000 7fea0f7fe700 1 -- 172.21.15.84:0/475969087 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fea140d1280 con 0x7fea18102ce0 2024-04-26T18:27:43.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.559+0000 7fea1e482700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fea040a3c40 0x7fea040a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fea1819abc0 tx=0x7fea08009190 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.559+0000 7fea2af78b80 1 Processor -- start 2024-04-26T18:27:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.559+0000 7fea2af78b80 1 -- start start 2024-04-26T18:27:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.559+0000 7fea2af78b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 0x55f4d15a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.559+0000 7fea2af78b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 0x55f4d15a01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.559+0000 7fea2af78b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a06f0 0x55f4d15ac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.559+0000 7fea2af78b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f4d14f27e0 con 0x55f4d15a5a20 2024-04-26T18:27:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.559+0000 7fea2af78b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f4d14588e0 con 0x55f4d15a06f0 2024-04-26T18:27:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.559+0000 7fea2af78b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4d14ee7a0 con 0x55f4d159fd50 2024-04-26T18:27:43.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.559+0000 7fea1e482700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 0x55f4d15a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.560+0000 7fea1e482700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 0x55f4d15a5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49692/0 (socket says 172.21.15.84:49692) 2024-04-26T18:27:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.560+0000 7fea1e482700 1 -- 172.21.15.84:0/1539353505 learned_addr learned my addr 172.21.15.84:0/1539353505 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.560+0000 7fea1dc81700 1 --2- 172.21.15.84:0/1539353505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 0x55f4d15a01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.560+0000 7fea1ec83700 1 --2- 172.21.15.84:0/1539353505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a06f0 0x55f4d15ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.560+0000 7fea1e482700 1 -- 172.21.15.84:0/1539353505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 msgr2=0x55f4d15a01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.560+0000 7fea1e482700 1 --2- 172.21.15.84:0/1539353505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 0x55f4d15a01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.560+0000 7fea1e482700 1 -- 172.21.15.84:0/1539353505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a06f0 msgr2=0x55f4d15ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.560+0000 7fea1e482700 1 --2- 172.21.15.84:0/1539353505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a06f0 0x55f4d15ac770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.560+0000 7fea1e482700 1 -- 172.21.15.84:0/1539353505 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4d143f880 con 0x55f4d15a5a20 2024-04-26T18:27:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.560+0000 7fea1e482700 1 --2- 172.21.15.84:0/1539353505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 0x55f4d15a5e00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fea0800ee30 tx=0x7fea08011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd2f65367080f480 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.560+0000 7fea0cff9700 1 -- 172.21.15.84:0/1539353505 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0801b040 con 0x55f4d15a5a20 2024-04-26T18:27:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.560+0000 7fea0cff9700 1 -- 172.21.15.84:0/1539353505 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea08024070 con 0x55f4d15a5a20 2024-04-26T18:27:43.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.560+0000 7fea0cff9700 1 -- 172.21.15.84:0/1539353505 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08013b20 con 0x55f4d15a5a20 2024-04-26T18:27:43.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.561+0000 7fea2af78b80 1 -- 172.21.15.84:0/1539353505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 msgr2=0x55f4d15a5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:43.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.561+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1539353505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 0x55f4d15a5e00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fea0800ee30 tx=0x7fea08011000 comp rx=0 tx=0).stop 2024-04-26T18:27:43.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.561+0000 7fea2af78b80 1 -- 172.21.15.84:0/1539353505 shutdown_connections 2024-04-26T18:27:43.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.561+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1539353505 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 0x55f4d15a5e00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.565 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.561+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1539353505 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a06f0 0x55f4d15ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.561+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1539353505 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 0x55f4d15a01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.561+0000 7fea2af78b80 1 -- 172.21.15.84:0/1539353505 >> 172.21.15.84:0/1539353505 conn(0x55f4d14ce570 msgr2=0x55f4d159a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.561+0000 7fea2af78b80 1 -- 172.21.15.84:0/1539353505 shutdown_connections 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.561+0000 7fea2af78b80 1 -- 172.21.15.84:0/1539353505 wait complete. 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.561+0000 7fea2af78b80 1 -- start start 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.561+0000 7fea2af78b80 1 Processor -- start 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea2af78b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 0x55f4d15ac310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea2af78b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a06f0 0x55f4d15a96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea2af78b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a5a20 0x55f4d15a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea2af78b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f4d15accb0 con 0x55f4d15a06f0 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea2af78b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f4d14aca30 con 0x55f4d15a5a20 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea2af78b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4d15ace30 con 0x55f4d159fd50 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea2af78b80 10 client.0 _ll_register_callbacks cb 0x55f4d148a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:43.566 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea1dc81700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a06f0 0x55f4d15a96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea1dc81700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a06f0 0x55f4d15a96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49694/0 (socket says 172.21.15.84:49694) 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea1dc81700 1 -- 172.21.15.84:0/1254016125 learned_addr learned my addr 172.21.15.84:0/1254016125 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea1e482700 1 --2- 172.21.15.84:0/1254016125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 0x55f4d15ac310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea1dc81700 1 -- 172.21.15.84:0/1254016125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 msgr2=0x55f4d15ac310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea1dc81700 1 --2- 172.21.15.84:0/1254016125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 0x55f4d15ac310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea1dc81700 1 -- 172.21.15.84:0/1254016125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a5a20 msgr2=0x55f4d15a9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea1ec83700 1 --2- 172.21.15.84:0/1254016125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a5a20 0x55f4d15a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea1dc81700 1 --2- 172.21.15.84:0/1254016125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a5a20 0x55f4d15a9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea1dc81700 1 -- 172.21.15.84:0/1254016125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4d14b0430 con 0x55f4d15a06f0 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea1dc81700 1 --2- 172.21.15.84:0/1254016125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a06f0 0x55f4d15a96b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fea140057c0 tx=0x7fea14005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8790dc0650790255 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea01ffb700 1 -- 172.21.15.84:0/1254016125 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1409cf20 con 0x55f4d15a06f0 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea01ffb700 1 -- 172.21.15.84:0/1254016125 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fea140a5d10 con 0x55f4d15a06f0 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.562+0000 7fea2af78b80 1 -- 172.21.15.84:0/1254016125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea08024070 con 0x55f4d15a06f0 2024-04-26T18:27:43.567 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.563+0000 7fea01ffb700 1 -- 172.21.15.84:0/1254016125 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea140041d0 con 0x55f4d15a06f0 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.563+0000 7fea2af78b80 10 client.15174 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.563+0000 7fea2af78b80 1 -- 172.21.15.84:0/1254016125 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f4d15b7450 con 0x55f4d15a06f0 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.563+0000 7fea01ffb700 1 -- 172.21.15.84:0/1254016125 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea08024070 con 0x55f4d15a06f0 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.563+0000 7fea01ffb700 1 -- 172.21.15.84:0/1254016125 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fea14003b70 con 0x55f4d15a06f0 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.563+0000 7fea2af78b80 10 client.15174 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.563+0000 7fea2af78b80 10 client.15174 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.563+0000 7fea2af78b80 20 client.15174 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.563+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1254016125 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x55f4d15b9b40 0x55f4d15bafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.563+0000 7fea2af78b80 4 client.15174 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.563+0000 7fea2af78b80 1 -- 172.21.15.84:0/1254016125 --> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4d15001c0 con 0x55f4d15b9b40 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.563+0000 7fea1e482700 1 --2- 172.21.15.84:0/1254016125 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x55f4d15b9b40 0x55f4d15bafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.564+0000 7fea1e482700 1 --2- 172.21.15.84:0/1254016125 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x55f4d15b9b40 0x55f4d15bafb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=14585c1e98ae92d6 server_cookie=db97e8f23620df56 in_seq=0 out_seq=0 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.564+0000 7fea01ffb700 10 client.15174 ms_handle_connect on v2:172.21.15.103:6836/2271231481 2024-04-26T18:27:43.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.564+0000 7fea01ffb700 1 -- 172.21.15.84:0/1254016125 <== mds.2 v2:172.21.15.103:6836/2271231481 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f4d15001c0 con 0x55f4d15b9b40 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.564+0000 7fea01ffb700 10 client.15174 handle_command_reply: tid=0 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.565+0000 7fea2af78b80 1 client.15174 shutdown 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.565+0000 7fea2af78b80 1 -- 172.21.15.84:0/1254016125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a06f0 msgr2=0x55f4d15a96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.565+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1254016125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a06f0 0x55f4d15a96b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fea140057c0 tx=0x7fea14005a00 comp rx=0 tx=0).stop 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.565+0000 7fea2af78b80 1 -- 172.21.15.84:0/1254016125 shutdown_connections 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.565+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1254016125 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a5a20 0x55f4d15a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.565+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1254016125 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 0x55f4d15ac310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.565+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1254016125 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x55f4d15b9b40 0x55f4d15bafb0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.565+0000 7fea01ffb700 0 client.15174 ms_handle_reset on v2:172.21.15.103:6836/2271231481 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.565+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1254016125 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a06f0 0x55f4d15a96b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.565+0000 7fea2af78b80 1 -- 172.21.15.84:0/1254016125 >> 172.21.15.84:0/1254016125 conn(0x55f4d14ce570 msgr2=0x55f4d159ed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.565+0000 7fea2af78b80 1 -- 172.21.15.84:0/1254016125 shutdown_connections 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.565+0000 7fea2af78b80 1 -- 172.21.15.84:0/1254016125 wait complete. 2024-04-26T18:27:43.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.565+0000 7fea2af78b80 20 client.15174 trim_cache size 0 max 16384 2024-04-26T18:27:43.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.585+0000 7fea2af78b80 1 Processor -- start 2024-04-26T18:27:43.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.585+0000 7fea2af78b80 1 -- start start 2024-04-26T18:27:43.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.585+0000 7fea2af78b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 0x55f4d15a7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.585+0000 7fea2af78b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a06f0 0x55f4d15a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.585+0000 7fea2af78b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 0x55f4d15c4410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.585+0000 7fea2af78b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f4d15b92f0 con 0x55f4d15a5a20 2024-04-26T18:27:43.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.585+0000 7fea2af78b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f4d15b99c0 con 0x55f4d15a06f0 2024-04-26T18:27:43.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.585+0000 7fea2af78b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4d15ba090 con 0x55f4d159fd50 2024-04-26T18:27:43.586 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.585+0000 7fea1ec83700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 0x55f4d15c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.585+0000 7fea1ec83700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 0x55f4d15c4410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49706/0 (socket says 172.21.15.84:49706) 2024-04-26T18:27:43.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.585+0000 7fea1ec83700 1 -- 172.21.15.84:0/1314987652 learned_addr learned my addr 172.21.15.84:0/1314987652 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:43.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.585+0000 7fea1ec83700 1 -- 172.21.15.84:0/1314987652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 msgr2=0x55f4d15a7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:43.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.586+0000 7fea1ec83700 1 --2- 172.21.15.84:0/1314987652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 0x55f4d15a7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.586+0000 7fea1ec83700 1 -- 172.21.15.84:0/1314987652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a06f0 msgr2=0x55f4d15a84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:43.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.586+0000 7fea1ec83700 1 --2- 172.21.15.84:0/1314987652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a06f0 0x55f4d15a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.586+0000 7fea1ec83700 1 -- 172.21.15.84:0/1314987652 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4d15b6740 con 0x55f4d15a5a20 2024-04-26T18:27:43.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.586+0000 7fea1ec83700 1 --2- 172.21.15.84:0/1314987652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 0x55f4d15c4410 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fea10009f80 tx=0x7fea10009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1be6acbb1d405 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:43.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.586+0000 7fea03fff700 1 -- 172.21.15.84:0/1314987652 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1000ae50 con 0x55f4d15a5a20 2024-04-26T18:27:43.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.586+0000 7fea03fff700 1 -- 172.21.15.84:0/1314987652 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea1001b070 con 0x55f4d15a5a20 2024-04-26T18:27:43.587 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.586+0000 7fea03fff700 1 -- 172.21.15.84:0/1314987652 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea10015860 con 0x55f4d15a5a20 2024-04-26T18:27:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 -- 172.21.15.84:0/1314987652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 msgr2=0x55f4d15c4410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1314987652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 0x55f4d15c4410 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fea10009f80 tx=0x7fea10009650 comp rx=0 tx=0).stop 2024-04-26T18:27:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 -- 172.21.15.84:0/1314987652 shutdown_connections 2024-04-26T18:27:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1314987652 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d15a5a20 0x55f4d15c4410 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1314987652 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d159fd50 0x55f4d15a7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 --2- 172.21.15.84:0/1314987652 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a06f0 0x55f4d15a84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 -- 172.21.15.84:0/1314987652 >> 172.21.15.84:0/1314987652 conn(0x55f4d14ce570 msgr2=0x55f4d15b87a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 -- 172.21.15.84:0/1314987652 shutdown_connections 2024-04-26T18:27:43.591 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 -- 172.21.15.84:0/1314987652 wait complete. 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 -- start start 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 Processor -- start 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d159fd50 0x55f4d15c4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d15a06f0 0x55f4d15a2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a5a20 0x55f4d15a2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f4d15bae30 con 0x55f4d159fd50 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f4d15ba760 con 0x55f4d15a5a20 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4d15bb500 con 0x55f4d15a06f0 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.587+0000 7fea2af78b80 10 client.0 _ll_register_callbacks cb 0x55f4d148a900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea1e482700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d159fd50 0x55f4d15c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea1e482700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d159fd50 0x55f4d15c4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49720/0 (socket says 172.21.15.84:49720) 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea1e482700 1 -- 172.21.15.84:0/392373138 learned_addr learned my addr 172.21.15.84:0/392373138 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea1ec83700 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a5a20 0x55f4d15a2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.592 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea1dc81700 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d15a06f0 0x55f4d15a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea1e482700 1 -- 172.21.15.84:0/392373138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d15a06f0 msgr2=0x55f4d15a2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea1e482700 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d15a06f0 0x55f4d15a2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea1e482700 1 -- 172.21.15.84:0/392373138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a5a20 msgr2=0x55f4d15a2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea1e482700 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a5a20 0x55f4d15a2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea1e482700 1 -- 172.21.15.84:0/392373138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4d15a3040 con 0x55f4d159fd50 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea1dc81700 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d15a06f0 0x55f4d15a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea1e482700 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d159fd50 0x55f4d15c4680 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fea0800ee00 tx=0x7fea080157f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a8c94217cac02c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea01ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea01ffb700 1 -- 172.21.15.84:0/392373138 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea08035070 con 0x55f4d159fd50 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea01ffb700 1 -- 172.21.15.84:0/392373138 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fea08015b90 con 0x55f4d159fd50 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.588+0000 7fea2af78b80 1 -- 172.21.15.84:0/392373138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea1001b070 con 0x55f4d159fd50 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.589+0000 7fea01ffb700 1 -- 172.21.15.84:0/392373138 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fea0800ee60 con 0x55f4d159fd50 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.589+0000 7fea2af78b80 10 client.15180 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.589+0000 7fea2af78b80 1 -- 172.21.15.84:0/392373138 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55f4d15c3480 con 0x55f4d159fd50 2024-04-26T18:27:43.593 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.589+0000 7fea01ffb700 1 -- 172.21.15.84:0/392373138 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea1001b070 con 0x55f4d159fd50 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.589+0000 7fea01ffb700 1 -- 172.21.15.84:0/392373138 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7fea0801e030 con 0x55f4d159fd50 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.589+0000 7fea2af78b80 10 client.15180 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.589+0000 7fea2af78b80 10 client.15180 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.589+0000 7fea2af78b80 20 client.15180 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.589+0000 7fea2af78b80 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x55f4d15b9b40 0x55f4d1625250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.589+0000 7fea2af78b80 4 client.15180 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.589+0000 7fea2af78b80 1 -- 172.21.15.84:0/392373138 --> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] -- command(tid 0: {"prefix": "status"}) -- 0x55f4d14fa260 con 0x55f4d15b9b40 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.589+0000 7fea1dc81700 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x55f4d15b9b40 0x55f4d1625250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea1dc81700 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x55f4d15b9b40 0x55f4d1625250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3eda933f048890fb server_cookie=650ab8e9b1ba9fe2 in_seq=0 out_seq=0 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea01ffb700 10 client.15180 ms_handle_connect on v2:172.21.15.103:6836/2271231481 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea01ffb700 1 -- 172.21.15.84:0/392373138 <== mds.2 v2:172.21.15.103:6836/2271231481 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55f4d14fa260 con 0x55f4d15b9b40 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea01ffb700 10 client.15180 handle_command_reply: tid=0 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea2af78b80 1 client.15180 shutdown 2024-04-26T18:27:43.594 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea2af78b80 1 -- 172.21.15.84:0/392373138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d159fd50 msgr2=0x55f4d15c4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea2af78b80 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d159fd50 0x55f4d15c4680 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fea0800ee00 tx=0x7fea080157f0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea2af78b80 1 -- 172.21.15.84:0/392373138 shutdown_connections 2024-04-26T18:27:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea2af78b80 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x55f4d15b9b40 0x55f4d1625250 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea2af78b80 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4d159fd50 0x55f4d15c4680 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea2af78b80 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4d15a5a20 0x55f4d15a2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea2af78b80 1 --2- 172.21.15.84:0/392373138 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4d15a06f0 0x55f4d15a2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea2af78b80 1 -- 172.21.15.84:0/392373138 >> 172.21.15.84:0/392373138 conn(0x55f4d14ce570 msgr2=0x55f4d15aacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea01ffb700 0 client.15180 ms_handle_reset on v2:172.21.15.103:6836/2271231481 2024-04-26T18:27:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea2af78b80 1 -- 172.21.15.84:0/392373138 shutdown_connections 2024-04-26T18:27:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea2af78b80 1 -- 172.21.15.84:0/392373138 wait complete. 2024-04-26T18:27:43.595 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.590+0000 7fea2af78b80 20 client.15180 trim_cache size 0 max 16384 2024-04-26T18:27:43.596 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:27:43.596 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:27:43.596 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 2, 2024-04-26T18:27:43.596 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24422, 2024-04-26T18:27:43.596 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:27:43.596 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:27:43.596 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:27:43.596 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 187.316772759, 2024-04-26T18:27:43.596 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 28, 2024-04-26T18:27:43.596 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 93, 2024-04-26T18:27:43.596 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 89, 2024-04-26T18:27:43.596 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 227.49646432500001 2024-04-26T18:27:43.596 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:27:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.591+0000 7fea206e6700 1 -- 172.21.15.84:0/475969087 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fea040a3c40 msgr2=0x7fea040a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.591+0000 7fea206e6700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fea040a3c40 0x7fea040a6100 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fea1819abc0 tx=0x7fea08009190 comp rx=0 tx=0).stop 2024-04-26T18:27:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.591+0000 7fea206e6700 1 -- 172.21.15.84:0/475969087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea18102ce0 msgr2=0x7fea1819fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.591+0000 7fea206e6700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea18102ce0 0x7fea1819fe00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fea140057f0 tx=0x7fea14003680 comp rx=0 tx=0).stop 2024-04-26T18:27:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.591+0000 7fea206e6700 1 -- 172.21.15.84:0/475969087 shutdown_connections 2024-04-26T18:27:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.591+0000 7fea206e6700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fea040a3c40 0x7fea040a6100 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.597 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.591+0000 7fea206e6700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fea18108390 0x7fea18199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.591+0000 7fea206e6700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fea18102320 0x7fea1819f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.591+0000 7fea206e6700 1 --2- 172.21.15.84:0/475969087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fea18102ce0 0x7fea1819fe00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:43.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.591+0000 7fea206e6700 1 -- 172.21.15.84:0/475969087 >> 172.21.15.84:0/475969087 conn(0x7fea180792d0 msgr2=0x7fea1806b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:43.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.591+0000 7fea206e6700 1 -- 172.21.15.84:0/475969087 shutdown_connections 2024-04-26T18:27:43.598 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:43.591+0000 7fea206e6700 1 -- 172.21.15.84:0/475969087 wait complete. 2024-04-26T18:27:44.336 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T18:27:44.572 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:44.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:44 smithi084 ceph-mon[29141]: pgmap v402: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:44.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:44 smithi103 ceph-mon[32727]: pgmap v402: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:44.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:44 smithi174 ceph-mon[32705]: pgmap v402: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:27:46.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.203+0000 7f275a746700 1 -- 172.21.15.84:0/36493982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2754102320 msgr2=0x7f27541027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:46.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.203+0000 7f275a746700 1 --2- 172.21.15.84:0/36493982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2754102320 0x7f27541027a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2748009990 tx=0x7f2748009ca0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.203+0000 7f275a746700 1 -- 172.21.15.84:0/36493982 shutdown_connections 2024-04-26T18:27:46.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.203+0000 7f275a746700 1 --2- 172.21.15.84:0/36493982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2754102ce0 0x7f275410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.203+0000 7f275a746700 1 --2- 172.21.15.84:0/36493982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2754108390 0x7f2754108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.203+0000 7f275a746700 1 --2- 172.21.15.84:0/36493982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2754102320 0x7f27541027a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.203+0000 7f275a746700 1 -- 172.21.15.84:0/36493982 >> 172.21.15.84:0/36493982 conn(0x7f27540792d0 msgr2=0x7f27540796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:46.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.203+0000 7f275a746700 1 -- 172.21.15.84:0/36493982 shutdown_connections 2024-04-26T18:27:46.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.203+0000 7f275a746700 1 -- 172.21.15.84:0/36493982 wait complete. 2024-04-26T18:27:46.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.204+0000 7f275a746700 1 Processor -- start 2024-04-26T18:27:46.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.204+0000 7f275a746700 1 -- start start 2024-04-26T18:27:46.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.204+0000 7f275a746700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2754102320 0x7f275419ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.204+0000 7f275a746700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2754102ce0 0x7f275419b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.204+0000 7f275a746700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2754108390 0x7f27541a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.204+0000 7f275a746700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2754112600 con 0x7f2754102ce0 2024-04-26T18:27:46.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.204+0000 7f275a746700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2754112480 con 0x7f2754102320 2024-04-26T18:27:46.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.204+0000 7f275a746700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2754112780 con 0x7f2754108390 2024-04-26T18:27:46.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.204+0000 7f27537fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2754102ce0 0x7f275419b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.204+0000 7f27537fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2754102ce0 0x7f275419b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49742/0 (socket says 172.21.15.84:49742) 2024-04-26T18:27:46.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.204+0000 7f27537fe700 1 -- 172.21.15.84:0/1060122085 learned_addr learned my addr 172.21.15.84:0/1060122085 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:46.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.204+0000 7f2753fff700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2754102320 0x7f275419ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.205+0000 7f27537fe700 1 -- 172.21.15.84:0/1060122085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2754108390 msgr2=0x7f27541a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:46.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.205+0000 7f27537fe700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2754108390 0x7f27541a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.205+0000 7f27537fe700 1 -- 172.21.15.84:0/1060122085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2754102320 msgr2=0x7f275419ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:46.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.205+0000 7f27537fe700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2754102320 0x7f275419ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.205+0000 7f27537fe700 1 -- 172.21.15.84:0/1060122085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27541a60c0 con 0x7f2754102ce0 2024-04-26T18:27:46.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.205+0000 7f2753fff700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2754102320 0x7f275419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:46.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.205+0000 7f27537fe700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2754102ce0 0x7f275419b040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2748005910 tx=0x7f2748003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:46.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.205+0000 7f27517fa700 1 -- 172.21.15.84:0/1060122085 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748027070 con 0x7f2754102ce0 2024-04-26T18:27:46.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.205+0000 7f275a746700 1 -- 172.21.15.84:0/1060122085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27541a6290 con 0x7f2754102ce0 2024-04-26T18:27:46.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.205+0000 7f27517fa700 1 -- 172.21.15.84:0/1060122085 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2748004140 con 0x7f2754102ce0 2024-04-26T18:27:46.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.205+0000 7f27517fa700 1 -- 172.21.15.84:0/1060122085 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274800f460 con 0x7f2754102ce0 2024-04-26T18:27:46.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.205+0000 7f275a746700 1 -- 172.21.15.84:0/1060122085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27541a66f0 con 0x7f2754102ce0 2024-04-26T18:27:46.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.207+0000 7f27517fa700 1 -- 172.21.15.84:0/1060122085 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f274802a070 con 0x7f2754102ce0 2024-04-26T18:27:46.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.208+0000 7f27517fa700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f273c0a3c40 0x7f273c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.208+0000 7f2753fff700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f273c0a3c40 0x7f273c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.208+0000 7f27517fa700 1 -- 172.21.15.84:0/1060122085 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f27480d1440 con 0x7f2754102ce0 2024-04-26T18:27:46.215 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.208+0000 7f2753fff700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f273c0a3c40 0x7f273c0a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f27440099d0 tx=0x7f2744008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:46.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2764fd8b80 1 Processor -- start 2024-04-26T18:27:46.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2764fd8b80 1 -- start start 2024-04-26T18:27:46.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2764fd8b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 0x55bcd44e6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2764fd8b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44e0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2764fd8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e12d0 0x55bcd44e1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2764fd8b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bcd44336f0 con 0x55bcd44e6ad0 2024-04-26T18:27:46.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2764fd8b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bcd43d2590 con 0x55bcd44e73f0 2024-04-26T18:27:46.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2764fd8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bcd442f6b0 con 0x55bcd44e12d0 2024-04-26T18:27:46.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2753fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 0x55bcd44e6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2753fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 0x55bcd44e6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49752/0 (socket says 172.21.15.84:49752) 2024-04-26T18:27:46.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2753fff700 1 -- 172.21.15.84:0/331794010 learned_addr learned my addr 172.21.15.84:0/331794010 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:46.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f27537fe700 1 --2- 172.21.15.84:0/331794010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44e0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2758ce3700 1 --2- 172.21.15.84:0/331794010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e12d0 0x55bcd44e1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2753fff700 1 -- 172.21.15.84:0/331794010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e12d0 msgr2=0x55bcd44e1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2753fff700 1 --2- 172.21.15.84:0/331794010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e12d0 0x55bcd44e1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2753fff700 1 -- 172.21.15.84:0/331794010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 msgr2=0x55bcd44e0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2753fff700 1 --2- 172.21.15.84:0/331794010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44e0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2753fff700 1 -- 172.21.15.84:0/331794010 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd4380880 con 0x55bcd44e6ad0 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f27537fe700 1 --2- 172.21.15.84:0/331794010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44e0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2758ce3700 1 --2- 172.21.15.84:0/331794010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e12d0 0x55bcd44e1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.209+0000 7f2753fff700 1 --2- 172.21.15.84:0/331794010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 0x55bcd44e6eb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f274400d7c0 tx=0x7f2744010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b712ac5779b240b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.210+0000 7f273a7fc700 1 -- 172.21.15.84:0/331794010 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744024070 con 0x55bcd44e6ad0 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.210+0000 7f273a7fc700 1 -- 172.21.15.84:0/331794010 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2744014af0 con 0x55bcd44e6ad0 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.210+0000 7f273a7fc700 1 -- 172.21.15.84:0/331794010 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f274401fc30 con 0x55bcd44e6ad0 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.210+0000 7f2764fd8b80 1 -- 172.21.15.84:0/331794010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 msgr2=0x55bcd44e6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.210+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/331794010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 0x55bcd44e6eb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f274400d7c0 tx=0x7f2744010000 comp rx=0 tx=0).stop 2024-04-26T18:27:46.217 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.210+0000 7f2764fd8b80 1 -- 172.21.15.84:0/331794010 shutdown_connections 2024-04-26T18:27:46.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.210+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/331794010 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 0x55bcd44e6eb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.210+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/331794010 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e12d0 0x55bcd44e1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.210+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/331794010 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44e0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.210+0000 7f2764fd8b80 1 -- 172.21.15.84:0/331794010 >> 172.21.15.84:0/331794010 conn(0x55bcd440dcf0 msgr2=0x55bcd44dc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:46.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.210+0000 7f2764fd8b80 1 -- 172.21.15.84:0/331794010 shutdown_connections 2024-04-26T18:27:46.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.210+0000 7f2764fd8b80 1 -- 172.21.15.84:0/331794010 wait complete. 2024-04-26T18:27:46.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f2764fd8b80 1 -- start start 2024-04-26T18:27:46.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f2764fd8b80 1 Processor -- start 2024-04-26T18:27:46.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f2764fd8b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 0x55bcd44ed110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f2764fd8b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e6ad0 0x55bcd44ea410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f2764fd8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e73f0 0x55bcd44ea950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f2764fd8b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bcd44edf10 con 0x55bcd44e12d0 2024-04-26T18:27:46.218 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f2764fd8b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bcd4399900 con 0x55bcd44e6ad0 2024-04-26T18:27:46.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f2764fd8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bcd44ee090 con 0x55bcd44e73f0 2024-04-26T18:27:46.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f2764fd8b80 10 client.0 _ll_register_callbacks cb 0x55bcd43c7530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:46.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f2753fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 0x55bcd44ed110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f2753fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 0x55bcd44ed110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49762/0 (socket says 172.21.15.84:49762) 2024-04-26T18:27:46.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f2753fff700 1 -- 172.21.15.84:0/2159429868 learned_addr learned my addr 172.21.15.84:0/2159429868 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:46.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f27537fe700 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e6ad0 0x55bcd44ea410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.211+0000 7f2758ce3700 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e73f0 0x55bcd44ea950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f2753fff700 1 -- 172.21.15.84:0/2159429868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e73f0 msgr2=0x55bcd44ea950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:46.219 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f2753fff700 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e73f0 0x55bcd44ea950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f2753fff700 1 -- 172.21.15.84:0/2159429868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e6ad0 msgr2=0x55bcd44ea410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:46.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f2753fff700 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e6ad0 0x55bcd44ea410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f2753fff700 1 -- 172.21.15.84:0/2159429868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd44eaf90 con 0x55bcd44e12d0 2024-04-26T18:27:46.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f27537fe700 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e6ad0 0x55bcd44ea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:46.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f2753fff700 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 0x55bcd44ed110 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2744014ac0 tx=0x7f274401f4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b348b6b80887be47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:46.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f27337fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:46.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f27337fe700 1 -- 172.21.15.84:0/2159429868 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744024040 con 0x55bcd44e12d0 2024-04-26T18:27:46.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f27337fe700 1 -- 172.21.15.84:0/2159429868 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f274401f840 con 0x55bcd44e12d0 2024-04-26T18:27:46.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f2764fd8b80 1 -- 172.21.15.84:0/2159429868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2744014af0 con 0x55bcd44e12d0 2024-04-26T18:27:46.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f27337fe700 1 -- 172.21.15.84:0/2159429868 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f274401fa40 con 0x55bcd44e12d0 2024-04-26T18:27:46.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f2764fd8b80 10 client.15192 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:46.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f2764fd8b80 1 -- 172.21.15.84:0/2159429868 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bcd44f88a0 con 0x55bcd44e12d0 2024-04-26T18:27:46.220 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f27337fe700 1 -- 172.21.15.84:0/2159429868 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2744014af0 con 0x55bcd44e12d0 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f27337fe700 1 -- 172.21.15.84:0/2159429868 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2744013070 con 0x55bcd44e12d0 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.212+0000 7f2764fd8b80 10 client.15192 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.213+0000 7f2764fd8b80 10 client.15192 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.213+0000 7f2764fd8b80 20 client.15192 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.213+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x55bcd44fabf0 0x55bcd44fc060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.213+0000 7f2764fd8b80 4 client.15192 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.213+0000 7f2764fd8b80 1 -- 172.21.15.84:0/2159429868 --> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bcd44410d0 con 0x55bcd44fabf0 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.213+0000 7f27537fe700 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x55bcd44fabf0 0x55bcd44fc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.213+0000 7f27537fe700 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x55bcd44fabf0 0x55bcd44fc060 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b0a882594b6c3cc5 server_cookie=48278c1d1822a7fe in_seq=0 out_seq=0 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.213+0000 7f27337fe700 10 client.15192 ms_handle_connect on v2:172.21.15.174:6838/518809558 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.214+0000 7f27337fe700 1 -- 172.21.15.84:0/2159429868 <== mds.1 v2:172.21.15.174:6838/518809558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bcd44410d0 con 0x55bcd44fabf0 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.214+0000 7f27337fe700 10 client.15192 handle_command_reply: tid=0 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.214+0000 7f2764fd8b80 1 client.15192 shutdown 2024-04-26T18:27:46.221 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.214+0000 7f2764fd8b80 1 -- 172.21.15.84:0/2159429868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 msgr2=0x55bcd44ed110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:46.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.214+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 0x55bcd44ed110 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f2744014ac0 tx=0x7f274401f4b0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.214+0000 7f2764fd8b80 1 -- 172.21.15.84:0/2159429868 shutdown_connections 2024-04-26T18:27:46.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.214+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x55bcd44fabf0 0x55bcd44fc060 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.214+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 0x55bcd44ed110 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.214+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e73f0 0x55bcd44ea950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.214+0000 7f27337fe700 0 client.15192 ms_handle_reset on v2:172.21.15.174:6838/518809558 2024-04-26T18:27:46.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.214+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/2159429868 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e6ad0 0x55bcd44ea410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.214+0000 7f2764fd8b80 1 -- 172.21.15.84:0/2159429868 >> 172.21.15.84:0/2159429868 conn(0x55bcd440dcf0 msgr2=0x55bcd44dbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:46.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.215+0000 7f2764fd8b80 1 -- 172.21.15.84:0/2159429868 shutdown_connections 2024-04-26T18:27:46.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.215+0000 7f2764fd8b80 1 -- 172.21.15.84:0/2159429868 wait complete. 2024-04-26T18:27:46.222 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.215+0000 7f2764fd8b80 20 client.15192 trim_cache size 0 max 16384 2024-04-26T18:27:46.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.234+0000 7f2764fd8b80 1 Processor -- start 2024-04-26T18:27:46.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f2764fd8b80 1 -- start start 2024-04-26T18:27:46.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f2764fd8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e12d0 0x55bcd44e0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f2764fd8b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 0x55bcd44fa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f2764fd8b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44dbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f2764fd8b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bcd44fa3a0 con 0x55bcd44e6ad0 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f2764fd8b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bcd44faa70 con 0x55bcd44e73f0 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f2764fd8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bcd44fb140 con 0x55bcd44e12d0 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f27537fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 0x55bcd44fa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f27537fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 0x55bcd44fa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49776/0 (socket says 172.21.15.84:49776) 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f27537fe700 1 -- 172.21.15.84:0/1313540277 learned_addr learned my addr 172.21.15.84:0/1313540277 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f2758ce3700 1 --2- 172.21.15.84:0/1313540277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44dbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f27537fe700 1 -- 172.21.15.84:0/1313540277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e12d0 msgr2=0x55bcd44e0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f2753fff700 1 --2- 172.21.15.84:0/1313540277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e12d0 0x55bcd44e0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f27537fe700 1 --2- 172.21.15.84:0/1313540277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e12d0 0x55bcd44e0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f27537fe700 1 -- 172.21.15.84:0/1313540277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 msgr2=0x55bcd44dbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f27537fe700 1 --2- 172.21.15.84:0/1313540277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44dbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.237 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f27537fe700 1 -- 172.21.15.84:0/1313540277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcd44ed850 con 0x55bcd44e6ad0 2024-04-26T18:27:46.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f2758ce3700 1 --2- 172.21.15.84:0/1313540277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44dbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:46.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f27537fe700 1 --2- 172.21.15.84:0/1313540277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 0x55bcd44fa520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f27480cef10 tx=0x7f27480cede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7188ca66511eb1d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:46.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f2739ffb700 1 -- 172.21.15.84:0/1313540277 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27480bb070 con 0x55bcd44e6ad0 2024-04-26T18:27:46.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f2739ffb700 1 -- 172.21.15.84:0/1313540277 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27480a6360 con 0x55bcd44e6ad0 2024-04-26T18:27:46.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.235+0000 7f2739ffb700 1 -- 172.21.15.84:0/1313540277 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2748088310 con 0x55bcd44e6ad0 2024-04-26T18:27:46.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.236+0000 7f2764fd8b80 1 -- 172.21.15.84:0/1313540277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 msgr2=0x55bcd44fa520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:46.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.236+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/1313540277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 0x55bcd44fa520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f27480cef10 tx=0x7f27480cede0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.236+0000 7f2764fd8b80 1 -- 172.21.15.84:0/1313540277 shutdown_connections 2024-04-26T18:27:46.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.236+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/1313540277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44dbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.236+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/1313540277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e12d0 0x55bcd44e0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.236+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/1313540277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e6ad0 0x55bcd44fa520 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.236+0000 7f2764fd8b80 1 -- 172.21.15.84:0/1313540277 >> 172.21.15.84:0/1313540277 conn(0x55bcd440dcf0 msgr2=0x55bcd44ecd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:46.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.236+0000 7f2764fd8b80 1 -- 172.21.15.84:0/1313540277 shutdown_connections 2024-04-26T18:27:46.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.236+0000 7f2764fd8b80 1 -- 172.21.15.84:0/1313540277 wait complete. 2024-04-26T18:27:46.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.237+0000 7f2764fd8b80 1 -- start start 2024-04-26T18:27:46.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.237+0000 7f2764fd8b80 1 Processor -- start 2024-04-26T18:27:46.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.237+0000 7f2764fd8b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 0x55bcd44f6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.237+0000 7f2764fd8b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e6ad0 0x55bcd44f6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.237+0000 7f2764fd8b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.237+0000 7f2764fd8b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bcd44fbee0 con 0x55bcd44e12d0 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.237+0000 7f2764fd8b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bcd44fb810 con 0x55bcd44e73f0 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.237+0000 7f2764fd8b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bcd44fc5b0 con 0x55bcd44e6ad0 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.237+0000 7f2764fd8b80 10 client.0 _ll_register_callbacks cb 0x55bcd43c7530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.237+0000 7f2753fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 0x55bcd44f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.237+0000 7f2753fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 0x55bcd44f6350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49792/0 (socket says 172.21.15.84:49792) 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.237+0000 7f2753fff700 1 -- 172.21.15.84:0/1374955797 learned_addr learned my addr 172.21.15.84:0/1374955797 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.237+0000 7f2758ce3700 1 --2- 172.21.15.84:0/1374955797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f27537fe700 1 --2- 172.21.15.84:0/1374955797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e6ad0 0x55bcd44f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f2753fff700 1 -- 172.21.15.84:0/1374955797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e6ad0 msgr2=0x55bcd44f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f2753fff700 1 --2- 172.21.15.84:0/1374955797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e6ad0 0x55bcd44f6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f2753fff700 1 -- 172.21.15.84:0/1374955797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 msgr2=0x55bcd44f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f2753fff700 1 --2- 172.21.15.84:0/1374955797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44f6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f2753fff700 1 -- 172.21.15.84:0/1374955797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcd4504640 con 0x55bcd44e12d0 2024-04-26T18:27:46.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f2753fff700 1 --2- 172.21.15.84:0/1374955797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 0x55bcd44f6350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f274400de60 tx=0x7f2744014d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c145cae4951dd76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f27337fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f27337fe700 1 -- 172.21.15.84:0/1374955797 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2744024070 con 0x55bcd44e12d0 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f27337fe700 1 -- 172.21.15.84:0/1374955797 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f274400f050 con 0x55bcd44e12d0 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f2764fd8b80 1 -- 172.21.15.84:0/1374955797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27480a6360 con 0x55bcd44e12d0 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f27337fe700 1 -- 172.21.15.84:0/1374955797 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f27440090c0 con 0x55bcd44e12d0 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f2764fd8b80 10 client.15198 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f2764fd8b80 1 -- 172.21.15.84:0/1374955797 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bcd45059a0 con 0x55bcd44e12d0 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f27337fe700 1 -- 172.21.15.84:0/1374955797 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27480a6360 con 0x55bcd44e12d0 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f27337fe700 1 -- 172.21.15.84:0/1374955797 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f2744034440 con 0x55bcd44e12d0 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f2764fd8b80 10 client.15198 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f2764fd8b80 10 client.15198 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f2764fd8b80 20 client.15198 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.238+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/1374955797 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x55bcd45663a0 0x55bcd45650b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:46.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.239+0000 7f2764fd8b80 4 client.15198 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.239+0000 7f2764fd8b80 1 -- 172.21.15.84:0/1374955797 --> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] -- command(tid 0: {"prefix": "status"}) -- 0x55bcd443b170 con 0x55bcd45663a0 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.239+0000 7f27537fe700 1 --2- 172.21.15.84:0/1374955797 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x55bcd45663a0 0x55bcd45650b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.239+0000 7f27337fe700 10 client.15198 ms_handle_connect on v2:172.21.15.174:6838/518809558 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.239+0000 7f27537fe700 1 --2- 172.21.15.84:0/1374955797 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x55bcd45663a0 0x55bcd45650b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cbcaa1ab9950df9a server_cookie=dd9f59bd1aff5d19 in_seq=0 out_seq=0 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.239+0000 7f27337fe700 1 -- 172.21.15.84:0/1374955797 <== mds.1 v2:172.21.15.174:6838/518809558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bcd443b170 con 0x55bcd45663a0 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.239+0000 7f27337fe700 10 client.15198 handle_command_reply: tid=0 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.239+0000 7f2764fd8b80 1 client.15198 shutdown 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f2764fd8b80 1 -- 172.21.15.84:0/1374955797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 msgr2=0x55bcd44f6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/1374955797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 0x55bcd44f6350 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f274400de60 tx=0x7f2744014d80 comp rx=0 tx=0).stop 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f2764fd8b80 1 -- 172.21.15.84:0/1374955797 shutdown_connections 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/1374955797 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x55bcd45663a0 0x55bcd45650b0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/1374955797 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bcd44e12d0 0x55bcd44f6350 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/1374955797 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bcd44e73f0 0x55bcd44f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f2764fd8b80 1 --2- 172.21.15.84:0/1374955797 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bcd44e6ad0 0x55bcd44f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f2764fd8b80 1 -- 172.21.15.84:0/1374955797 >> 172.21.15.84:0/1374955797 conn(0x55bcd440dcf0 msgr2=0x55bcd44eccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:46.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f27337fe700 0 client.15198 ms_handle_reset on v2:172.21.15.174:6838/518809558 2024-04-26T18:27:46.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f2764fd8b80 1 -- 172.21.15.84:0/1374955797 shutdown_connections 2024-04-26T18:27:46.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f2764fd8b80 1 -- 172.21.15.84:0/1374955797 wait complete. 2024-04-26T18:27:46.246 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f2764fd8b80 20 client.15198 trim_cache size 0 max 16384 2024-04-26T18:27:46.246 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:27:46.246 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:27:46.246 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 1, 2024-04-26T18:27:46.246 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24430, 2024-04-26T18:27:46.246 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:27:46.246 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:27:46.247 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:27:46.247 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 190.97833761800001, 2024-04-26T18:27:46.247 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 28, 2024-04-26T18:27:46.247 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 93, 2024-04-26T18:27:46.247 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 89, 2024-04-26T18:27:46.247 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 227.37590938100001 2024-04-26T18:27:46.247 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:27:46.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f275a746700 1 -- 172.21.15.84:0/1060122085 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f273c0a3c40 msgr2=0x7f273c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:46.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f275a746700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f273c0a3c40 0x7f273c0a6100 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f27440099d0 tx=0x7f2744008040 comp rx=0 tx=0).stop 2024-04-26T18:27:46.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f275a746700 1 -- 172.21.15.84:0/1060122085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2754102ce0 msgr2=0x7f275419b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:46.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f275a746700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2754102ce0 0x7f275419b040 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f2748005910 tx=0x7f2748003730 comp rx=0 tx=0).stop 2024-04-26T18:27:46.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f275a746700 1 -- 172.21.15.84:0/1060122085 shutdown_connections 2024-04-26T18:27:46.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f275a746700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f273c0a3c40 0x7f273c0a6100 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f275a746700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2754108390 0x7f27541a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f275a746700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2754102320 0x7f275419ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f275a746700 1 --2- 172.21.15.84:0/1060122085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2754102ce0 0x7f275419b040 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:46.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f275a746700 1 -- 172.21.15.84:0/1060122085 >> 172.21.15.84:0/1060122085 conn(0x7f27540792d0 msgr2=0x7f2754101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:46.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f275a746700 1 -- 172.21.15.84:0/1060122085 shutdown_connections 2024-04-26T18:27:46.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:46.240+0000 7f275a746700 1 -- 172.21.15.84:0/1060122085 wait complete. 2024-04-26T18:27:46.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:46 smithi084 ceph-mon[29141]: pgmap v403: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:46.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:46 smithi103 ceph-mon[32727]: pgmap v403: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:46.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:46 smithi174 ceph-mon[32705]: pgmap v403: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:47.045 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T18:27:47.282 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:48.576 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:48 smithi084 ceph-mon[29141]: pgmap v404: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:48.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:48 smithi103 ceph-mon[32727]: pgmap v404: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:48.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:48 smithi174 ceph-mon[32705]: pgmap v404: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-26T18:27:48.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.938+0000 7f9e8e3a1700 1 -- 172.21.15.84:0/410612408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e88101de0 msgr2=0x7f9e88102260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:48.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.938+0000 7f9e8e3a1700 1 --2- 172.21.15.84:0/410612408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e88101de0 0x7f9e88102260 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009990 tx=0x7f9e7c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.938+0000 7f9e8e3a1700 1 -- 172.21.15.84:0/410612408 shutdown_connections 2024-04-26T18:27:48.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.938+0000 7f9e8e3a1700 1 --2- 172.21.15.84:0/410612408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e881027a0 0x7f9e88110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.938+0000 7f9e8e3a1700 1 --2- 172.21.15.84:0/410612408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e88108500 0x7f9e881088e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.938+0000 7f9e8e3a1700 1 --2- 172.21.15.84:0/410612408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e88101de0 0x7f9e88102260 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.938+0000 7f9e8e3a1700 1 -- 172.21.15.84:0/410612408 >> 172.21.15.84:0/410612408 conn(0x7f9e88077640 msgr2=0x7f9e88077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:48.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.938+0000 7f9e8e3a1700 1 -- 172.21.15.84:0/410612408 shutdown_connections 2024-04-26T18:27:48.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.938+0000 7f9e8e3a1700 1 -- 172.21.15.84:0/410612408 wait complete. 2024-04-26T18:27:48.939 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.939+0000 7f9e8e3a1700 1 Processor -- start 2024-04-26T18:27:48.940 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.939+0000 7f9e8e3a1700 1 -- start start 2024-04-26T18:27:48.941 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.939+0000 7f9e8e3a1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e88101de0 0x7f9e8819f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.939+0000 7f9e8e3a1700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e881027a0 0x7f9e8819fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.939+0000 7f9e8e3a1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e88108500 0x7f9e88199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.939+0000 7f9e8e3a1700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88113710 con 0x7f9e88101de0 2024-04-26T18:27:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.939+0000 7f9e8e3a1700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88113590 con 0x7f9e881027a0 2024-04-26T18:27:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.939+0000 7f9e8e3a1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9e88113890 con 0x7f9e88108500 2024-04-26T18:27:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.939+0000 7f9e87fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e88101de0 0x7f9e8819f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.939+0000 7f9e87fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e88101de0 0x7f9e8819f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54160/0 (socket says 172.21.15.84:54160) 2024-04-26T18:27:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.939+0000 7f9e87fff700 1 -- 172.21.15.84:0/4143883490 learned_addr learned my addr 172.21.15.84:0/4143883490 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.939+0000 7f9e877fe700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e881027a0 0x7f9e8819fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.939+0000 7f9e8c93e700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e88108500 0x7f9e88199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.940+0000 7f9e87fff700 1 -- 172.21.15.84:0/4143883490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e88108500 msgr2=0x7f9e88199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.940+0000 7f9e87fff700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e88108500 0x7f9e88199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.942 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.940+0000 7f9e87fff700 1 -- 172.21.15.84:0/4143883490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e881027a0 msgr2=0x7f9e8819fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:48.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.940+0000 7f9e87fff700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e881027a0 0x7f9e8819fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.940+0000 7f9e87fff700 1 -- 172.21.15.84:0/4143883490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e8819a1f0 con 0x7f9e88101de0 2024-04-26T18:27:48.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.940+0000 7f9e8c93e700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e88108500 0x7f9e88199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:48.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.940+0000 7f9e87fff700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e88101de0 0x7f9e8819f880 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800b500 tx=0x7f9e7800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:48.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.940+0000 7f9e857fa700 1 -- 172.21.15.84:0/4143883490 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78015020 con 0x7f9e88101de0 2024-04-26T18:27:48.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.940+0000 7f9e8e3a1700 1 -- 172.21.15.84:0/4143883490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e8819a4e0 con 0x7f9e88101de0 2024-04-26T18:27:48.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.940+0000 7f9e857fa700 1 -- 172.21.15.84:0/4143883490 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e78011ce0 con 0x7f9e88101de0 2024-04-26T18:27:48.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.940+0000 7f9e857fa700 1 -- 172.21.15.84:0/4143883490 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78010d60 con 0x7f9e88101de0 2024-04-26T18:27:48.943 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.940+0000 7f9e8e3a1700 1 -- 172.21.15.84:0/4143883490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e881a6750 con 0x7f9e88101de0 2024-04-26T18:27:48.949 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.942+0000 7f9e857fa700 1 -- 172.21.15.84:0/4143883490 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9e78018020 con 0x7f9e88101de0 2024-04-26T18:27:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.943+0000 7f9e857fa700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f9e700a3c40 0x7f9e700a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.943+0000 7f9e877fe700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f9e700a3c40 0x7f9e700a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.943+0000 7f9e857fa700 1 -- 172.21.15.84:0/4143883490 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f9e780cfc30 con 0x7f9e88101de0 2024-04-26T18:27:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.943+0000 7f9e877fe700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f9e700a3c40 0x7f9e700a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c006010 tx=0x7f9e7c01a040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.943+0000 7f9e98c33b80 1 Processor -- start 2024-04-26T18:27:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.943+0000 7f9e98c33b80 1 -- start start 2024-04-26T18:27:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e98c33b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f662eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e98c33b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f6633f0 0x56443f65ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e98c33b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f65d2d0 0x56443f65d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e98c33b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56443f5af6f0 con 0x56443f6633f0 2024-04-26T18:27:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e98c33b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56443f54e590 con 0x56443f65d2d0 2024-04-26T18:27:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e98c33b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56443f5ab6b0 con 0x56443f662ad0 2024-04-26T18:27:48.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e877fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f6633f0 0x56443f65ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e877fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f6633f0 0x56443f65ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54172/0 (socket says 172.21.15.84:54172) 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e877fe700 1 -- 172.21.15.84:0/3066404698 learned_addr learned my addr 172.21.15.84:0/3066404698 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e87fff700 1 --2- 172.21.15.84:0/3066404698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f662eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e8c93e700 1 --2- 172.21.15.84:0/3066404698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f65d2d0 0x56443f65d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e877fe700 1 -- 172.21.15.84:0/3066404698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 msgr2=0x56443f662eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e877fe700 1 --2- 172.21.15.84:0/3066404698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f662eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e877fe700 1 -- 172.21.15.84:0/3066404698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f65d2d0 msgr2=0x56443f65d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e877fe700 1 --2- 172.21.15.84:0/3066404698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f65d2d0 0x56443f65d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e877fe700 1 -- 172.21.15.84:0/3066404698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56443f4fc880 con 0x56443f6633f0 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e8c93e700 1 --2- 172.21.15.84:0/3066404698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f65d2d0 0x56443f65d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e877fe700 1 --2- 172.21.15.84:0/3066404698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f6633f0 0x56443f65ccb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009990 tx=0x7f9e7c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=545fb3a35c97f4e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e6e7fc700 1 -- 172.21.15.84:0/3066404698 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c00fa40 con 0x56443f6633f0 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e6e7fc700 1 -- 172.21.15.84:0/3066404698 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e7c032070 con 0x56443f6633f0 2024-04-26T18:27:48.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.944+0000 7f9e6e7fc700 1 -- 172.21.15.84:0/3066404698 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7c023dd0 con 0x56443f6633f0 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.945+0000 7f9e98c33b80 1 -- 172.21.15.84:0/3066404698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f6633f0 msgr2=0x56443f65ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.945+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/3066404698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f6633f0 0x56443f65ccb0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c009990 tx=0x7f9e7c01e000 comp rx=0 tx=0).stop 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.945+0000 7f9e98c33b80 1 -- 172.21.15.84:0/3066404698 shutdown_connections 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.945+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/3066404698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f65d2d0 0x56443f65d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.945+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/3066404698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f662eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.945+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/3066404698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f6633f0 0x56443f65ccb0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.945+0000 7f9e98c33b80 1 -- 172.21.15.84:0/3066404698 >> 172.21.15.84:0/3066404698 conn(0x56443f589cf0 msgr2=0x56443f6581b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.945+0000 7f9e98c33b80 1 -- 172.21.15.84:0/3066404698 shutdown_connections 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.945+0000 7f9e98c33b80 1 -- 172.21.15.84:0/3066404698 wait complete. 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.945+0000 7f9e98c33b80 1 -- start start 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e98c33b80 1 Processor -- start 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e98c33b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f65d2d0 0x56443f6691c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e98c33b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f6664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e98c33b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f6633f0 0x56443f666a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e98c33b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56443f669eb0 con 0x56443f65d2d0 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e98c33b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56443f515900 con 0x56443f6633f0 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e98c33b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56443f66a030 con 0x56443f662ad0 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e98c33b80 10 client.0 _ll_register_callbacks cb 0x56443f543530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e87fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f65d2d0 0x56443f6691c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e87fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f65d2d0 0x56443f6691c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54178/0 (socket says 172.21.15.84:54178) 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e87fff700 1 -- 172.21.15.84:0/3796345180 learned_addr learned my addr 172.21.15.84:0/3796345180 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e8c93e700 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f6633f0 0x56443f666a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e877fe700 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f6664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e87fff700 1 -- 172.21.15.84:0/3796345180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 msgr2=0x56443f6664c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e87fff700 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f6664c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e87fff700 1 -- 172.21.15.84:0/3796345180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f6633f0 msgr2=0x56443f666a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e87fff700 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f6633f0 0x56443f666a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e87fff700 1 -- 172.21.15.84:0/3796345180 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56443f6670d0 con 0x56443f65d2d0 2024-04-26T18:27:48.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e877fe700 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f6664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e87fff700 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f65d2d0 0x56443f6691c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9e780cd700 tx=0x7f9e7809c5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee364518a11b983 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.946+0000 7f9e677fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e677fe700 1 -- 172.21.15.84:0/3796345180 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e780ba060 con 0x56443f65d2d0 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e677fe700 1 -- 172.21.15.84:0/3796345180 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f9e7809c140 con 0x56443f65d2d0 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e98c33b80 1 -- 172.21.15.84:0/3796345180 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e7c032070 con 0x56443f65d2d0 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e677fe700 1 -- 172.21.15.84:0/3796345180 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e780b5070 con 0x56443f65d2d0 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e98c33b80 10 client.15210 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e98c33b80 1 -- 172.21.15.84:0/3796345180 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56443f6749c0 con 0x56443f65d2d0 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e677fe700 1 -- 172.21.15.84:0/3796345180 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e7c032070 con 0x56443f65d2d0 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e677fe700 1 -- 172.21.15.84:0/3796345180 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9e780a4750 con 0x56443f65d2d0 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e98c33b80 10 client.15210 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e98c33b80 10 client.15210 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e98c33b80 20 client.15210 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:48.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56443f676f80 0x56443f6783f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e98c33b80 4 client.15210 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:27:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.947+0000 7f9e98c33b80 1 -- 172.21.15.84:0/3796345180 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56443f5bd0d0 con 0x56443f676f80 2024-04-26T18:27:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.948+0000 7f9e877fe700 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56443f676f80 0x56443f6783f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.948+0000 7f9e677fe700 10 client.15210 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:27:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.948+0000 7f9e877fe700 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56443f676f80 0x56443f6783f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cce1d2c2482394b9 server_cookie=6c654d422bd2d282 in_seq=0 out_seq=0 2024-04-26T18:27:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e677fe700 1 -- 172.21.15.84:0/3796345180 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56443f5bd0d0 con 0x56443f676f80 2024-04-26T18:27:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e677fe700 10 client.15210 handle_command_reply: tid=0 2024-04-26T18:27:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e98c33b80 1 client.15210 shutdown 2024-04-26T18:27:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e98c33b80 1 -- 172.21.15.84:0/3796345180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f65d2d0 msgr2=0x56443f6691c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:48.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f65d2d0 0x56443f6691c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9e780cd700 tx=0x7f9e7809c5e0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e98c33b80 1 -- 172.21.15.84:0/3796345180 shutdown_connections 2024-04-26T18:27:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56443f676f80 0x56443f6783f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e677fe700 0 client.15210 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:27:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f65d2d0 0x56443f6691c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f6633f0 0x56443f666a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/3796345180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f6664c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e98c33b80 1 -- 172.21.15.84:0/3796345180 >> 172.21.15.84:0/3796345180 conn(0x56443f589cf0 msgr2=0x56443f657da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e98c33b80 1 -- 172.21.15.84:0/3796345180 shutdown_connections 2024-04-26T18:27:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e98c33b80 1 -- 172.21.15.84:0/3796345180 wait complete. 2024-04-26T18:27:48.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.949+0000 7f9e98c33b80 20 client.15210 trim_cache size 0 max 16384 2024-04-26T18:27:48.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e98c33b80 1 Processor -- start 2024-04-26T18:27:48.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e98c33b80 1 -- start start 2024-04-26T18:27:48.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e98c33b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f589cf0 0x56443f65ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e98c33b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f65d2d0 0x56443f681c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.978 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e98c33b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f663ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e98c33b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56443f676730 con 0x56443f589cf0 2024-04-26T18:27:48.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e98c33b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56443f676e00 con 0x56443f65d2d0 2024-04-26T18:27:48.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e98c33b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56443f6774d0 con 0x56443f662ad0 2024-04-26T18:27:48.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e87fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f589cf0 0x56443f65ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e87fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f589cf0 0x56443f65ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54192/0 (socket says 172.21.15.84:54192) 2024-04-26T18:27:48.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e87fff700 1 -- 172.21.15.84:0/1879134361 learned_addr learned my addr 172.21.15.84:0/1879134361 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:48.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e8c93e700 1 --2- 172.21.15.84:0/1879134361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f663ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.979 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e877fe700 1 --2- 172.21.15.84:0/1879134361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f65d2d0 0x56443f681c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e87fff700 1 -- 172.21.15.84:0/1879134361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 msgr2=0x56443f663ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:48.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e87fff700 1 --2- 172.21.15.84:0/1879134361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f663ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e87fff700 1 -- 172.21.15.84:0/1879134361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f65d2d0 msgr2=0x56443f681c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:48.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e87fff700 1 --2- 172.21.15.84:0/1879134361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f65d2d0 0x56443f681c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e87fff700 1 -- 172.21.15.84:0/1879134361 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56443f660c90 con 0x56443f589cf0 2024-04-26T18:27:48.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e8c93e700 1 --2- 172.21.15.84:0/1879134361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f663ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:48.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.972+0000 7f9e87fff700 1 --2- 172.21.15.84:0/1879134361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f589cf0 0x56443f65ccb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e7809a190 tx=0x7f9e78004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80cc2aec34fd7be0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:48.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.973+0000 7f9e6dffb700 1 -- 172.21.15.84:0/1879134361 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e78098180 con 0x56443f589cf0 2024-04-26T18:27:48.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.973+0000 7f9e6dffb700 1 -- 172.21.15.84:0/1879134361 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e780a48c0 con 0x56443f589cf0 2024-04-26T18:27:48.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.973+0000 7f9e6dffb700 1 -- 172.21.15.84:0/1879134361 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e7809c140 con 0x56443f589cf0 2024-04-26T18:27:48.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.973+0000 7f9e98c33b80 1 -- 172.21.15.84:0/1879134361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f589cf0 msgr2=0x56443f65ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:48.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.973+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/1879134361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f589cf0 0x56443f65ccb0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e7809a190 tx=0x7f9e78004860 comp rx=0 tx=0).stop 2024-04-26T18:27:48.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.973+0000 7f9e98c33b80 1 -- 172.21.15.84:0/1879134361 shutdown_connections 2024-04-26T18:27:48.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.973+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/1879134361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f589cf0 0x56443f65ccb0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.973+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/1879134361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f662ad0 0x56443f663ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.973+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/1879134361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f65d2d0 0x56443f681c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.981 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.973+0000 7f9e98c33b80 1 -- 172.21.15.84:0/1879134361 >> 172.21.15.84:0/1879134361 conn(0x56443f653ee0 msgr2=0x56443f6637d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:48.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.973+0000 7f9e98c33b80 1 -- 172.21.15.84:0/1879134361 shutdown_connections 2024-04-26T18:27:48.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e98c33b80 1 -- 172.21.15.84:0/1879134361 wait complete. 2024-04-26T18:27:48.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e98c33b80 1 -- start start 2024-04-26T18:27:48.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e98c33b80 1 Processor -- start 2024-04-26T18:27:48.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e98c33b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f589cf0 0x56443f669050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e98c33b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f65d2d0 0x56443f667cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e98c33b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f662ad0 0x56443f668270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e98c33b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56443f678270 con 0x56443f662ad0 2024-04-26T18:27:48.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e98c33b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56443f677ba0 con 0x56443f589cf0 2024-04-26T18:27:48.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e98c33b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56443f678940 con 0x56443f65d2d0 2024-04-26T18:27:48.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e98c33b80 10 client.0 _ll_register_callbacks cb 0x56443f543530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:27:48.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e8c93e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f662ad0 0x56443f668270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e8c93e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f662ad0 0x56443f668270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54206/0 (socket says 172.21.15.84:54206) 2024-04-26T18:27:48.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e8c93e700 1 -- 172.21.15.84:0/85144470 learned_addr learned my addr 172.21.15.84:0/85144470 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:48.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.974+0000 7f9e87fff700 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f589cf0 0x56443f669050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e877fe700 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f65d2d0 0x56443f667cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e8c93e700 1 -- 172.21.15.84:0/85144470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f65d2d0 msgr2=0x56443f667cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:48.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e8c93e700 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f65d2d0 0x56443f667cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e8c93e700 1 -- 172.21.15.84:0/85144470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f589cf0 msgr2=0x56443f669050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:27:48.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e8c93e700 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f589cf0 0x56443f669050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e8c93e700 1 -- 172.21.15.84:0/85144470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56443f668b00 con 0x56443f662ad0 2024-04-26T18:27:48.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e877fe700 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f65d2d0 0x56443f667cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:48.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e87fff700 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f589cf0 0x56443f669050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:27:48.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e8c93e700 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f662ad0 0x56443f668270 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9e80011f10 tx=0x7f9e8000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc3b808afbfdab29 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:48.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e677fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:27:48.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e677fe700 1 -- 172.21.15.84:0/85144470 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e8001f060 con 0x56443f662ad0 2024-04-26T18:27:48.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e677fe700 1 -- 172.21.15.84:0/85144470 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f9e8001a410 con 0x56443f662ad0 2024-04-26T18:27:48.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e98c33b80 1 -- 172.21.15.84:0/85144470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e780a48c0 con 0x56443f662ad0 2024-04-26T18:27:48.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e677fe700 1 -- 172.21.15.84:0/85144470 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f9e8000fe30 con 0x56443f662ad0 2024-04-26T18:27:48.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e98c33b80 10 client.15216 fetch_fsmap learned FSMap version 28 2024-04-26T18:27:48.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e98c33b80 1 -- 172.21.15.84:0/85144470 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x56443f6e31f0 con 0x56443f662ad0 2024-04-26T18:27:48.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.975+0000 7f9e677fe700 1 -- 172.21.15.84:0/85144470 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e780a48c0 con 0x56443f662ad0 2024-04-26T18:27:48.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.976+0000 7f9e677fe700 1 -- 172.21.15.84:0/85144470 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 28) ==== 8354+0+0 (secure 0 0 0) 0x7f9e80024470 con 0x56443f662ad0 2024-04-26T18:27:48.985 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.976+0000 7f9e98c33b80 10 client.15216 fetch_fsmap finished waiting for FSMap version 28 2024-04-26T18:27:48.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.976+0000 7f9e98c33b80 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:27:48.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.976+0000 7f9e98c33b80 20 client.15216 populate_metadata read hostname 'smithi084' 2024-04-26T18:27:48.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.976+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56443f6633f0 0x56443f6e6b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:48.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.976+0000 7f9e98c33b80 4 client.15216 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:27:48.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.976+0000 7f9e98c33b80 1 -- 172.21.15.84:0/85144470 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "status"}) -- 0x56443f5b7170 con 0x56443f6633f0 2024-04-26T18:27:48.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.976+0000 7f9e87fff700 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56443f6633f0 0x56443f6e6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:48.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.976+0000 7f9e677fe700 10 client.15216 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:27:48.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.976+0000 7f9e87fff700 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56443f6633f0 0x56443f6e6b40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc6fc1022815e74c server_cookie=efa78a3182290ec5 in_seq=0 out_seq=0 2024-04-26T18:27:48.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e677fe700 1 -- 172.21.15.84:0/85144470 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56443f5b7170 con 0x56443f6633f0 2024-04-26T18:27:48.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e677fe700 10 client.15216 handle_command_reply: tid=0 2024-04-26T18:27:48.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e98c33b80 1 client.15216 shutdown 2024-04-26T18:27:48.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e98c33b80 1 -- 172.21.15.84:0/85144470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f662ad0 msgr2=0x56443f668270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:48.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f662ad0 0x56443f668270 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9e80011f10 tx=0x7f9e8000f690 comp rx=0 tx=0).stop 2024-04-26T18:27:48.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e98c33b80 1 -- 172.21.15.84:0/85144470 shutdown_connections 2024-04-26T18:27:48.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56443f6633f0 0x56443f6e6b40 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56443f662ad0 0x56443f668270 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e677fe700 0 client.15216 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:27:48.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56443f589cf0 0x56443f669050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e98c33b80 1 --2- 172.21.15.84:0/85144470 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56443f65d2d0 0x56443f667cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e98c33b80 1 -- 172.21.15.84:0/85144470 >> 172.21.15.84:0/85144470 conn(0x56443f653ee0 msgr2=0x56443f664060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:48.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e98c33b80 1 -- 172.21.15.84:0/85144470 shutdown_connections 2024-04-26T18:27:48.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e98c33b80 1 -- 172.21.15.84:0/85144470 wait complete. 2024-04-26T18:27:48.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.977+0000 7f9e98c33b80 20 client.15216 trim_cache size 0 max 16384 2024-04-26T18:27:48.989 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:27:48.989 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:27:48.989 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2024-04-26T18:27:48.989 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24434, 2024-04-26T18:27:48.989 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:27:48.989 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:27:48.990 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:27:48.990 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 206.02647680499999, 2024-04-26T18:27:48.990 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 28, 2024-04-26T18:27:48.990 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 93, 2024-04-26T18:27:48.990 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 89, 2024-04-26T18:27:48.990 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 220.28373180700001 2024-04-26T18:27:48.990 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:27:48.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.978+0000 7f9e8e3a1700 1 -- 172.21.15.84:0/4143883490 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f9e700a3c40 msgr2=0x7f9e700a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:48.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.978+0000 7f9e8e3a1700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f9e700a3c40 0x7f9e700a6100 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c006010 tx=0x7f9e7c01a040 comp rx=0 tx=0).stop 2024-04-26T18:27:48.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.978+0000 7f9e8e3a1700 1 -- 172.21.15.84:0/4143883490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e88101de0 msgr2=0x7f9e8819f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:48.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.978+0000 7f9e8e3a1700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e88101de0 0x7f9e8819f880 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9e7800b500 tx=0x7f9e7800b8c0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.978+0000 7f9e8e3a1700 1 -- 172.21.15.84:0/4143883490 shutdown_connections 2024-04-26T18:27:48.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.978+0000 7f9e8e3a1700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f9e700a3c40 0x7f9e700a6100 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.991 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.978+0000 7f9e8e3a1700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9e88101de0 0x7f9e8819f880 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.978+0000 7f9e8e3a1700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9e88108500 0x7f9e88199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.978+0000 7f9e8e3a1700 1 --2- 172.21.15.84:0/4143883490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9e881027a0 0x7f9e8819fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:48.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.978+0000 7f9e8e3a1700 1 -- 172.21.15.84:0/4143883490 >> 172.21.15.84:0/4143883490 conn(0x7f9e88077640 msgr2=0x7f9e88100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:48.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.978+0000 7f9e8e3a1700 1 -- 172.21.15.84:0/4143883490 shutdown_connections 2024-04-26T18:27:48.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:48.978+0000 7f9e8e3a1700 1 -- 172.21.15.84:0/4143883490 wait complete. 2024-04-26T18:27:49.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:49 smithi084 ceph-mon[29141]: pgmap v405: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:49.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:49 smithi103 ceph-mon[32727]: pgmap v405: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:49.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:49 smithi174 ceph-mon[32705]: pgmap v405: 97 pgs: 97 active+clean; 648 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-26T18:27:49.799 INFO:teuthology.run_tasks:Running task sequential... 2024-04-26T18:27:49.813 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-26T18:27:49.814 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-26T18:27:49.815 INFO:teuthology.task.print:Enabling mgr modules 2024-04-26T18:27:49.815 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-26T18:27:49.816 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T18:27:49.816 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi084.front.sepia.ceph.com 2024-04-26T18:27:49.816 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-26T18:27:49.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.917+0000 7f0bd5685640 1 -- 172.21.15.84:0/433805223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd0104540 msgr2=0x7f0bd010add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:49.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.917+0000 7f0bd5685640 1 --2- 172.21.15.84:0/433805223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd0104540 0x7f0bd010add0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f0bc8009ca0 tx=0x7f0bc802a8c0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.917+0000 7f0bd5685640 1 -- 172.21.15.84:0/433805223 shutdown_connections 2024-04-26T18:27:49.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.917+0000 7f0bd5685640 1 --2- 172.21.15.84:0/433805223 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd0104540 0x7f0bd010add0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.917+0000 7f0bd5685640 1 --2- 172.21.15.84:0/433805223 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bd0103b80 0x7f0bd0104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.917+0000 7f0bd5685640 1 --2- 172.21.15.84:0/433805223 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0102990 0x7f0bd0102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.917+0000 7f0bd5685640 1 -- 172.21.15.84:0/433805223 >> 172.21.15.84:0/433805223 conn(0x7f0bd00fdf20 msgr2=0x7f0bd0100340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:49.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.918+0000 7f0bd5685640 1 -- 172.21.15.84:0/433805223 shutdown_connections 2024-04-26T18:27:49.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.918+0000 7f0bd5685640 1 -- 172.21.15.84:0/433805223 wait complete. 2024-04-26T18:27:49.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.918+0000 7f0bd5685640 1 Processor -- start 2024-04-26T18:27:49.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.918+0000 7f0bd5685640 1 -- start start 2024-04-26T18:27:49.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.918+0000 7f0bd5685640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd0102990 0x7f0bd0198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:49.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.918+0000 7f0bd5685640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0103b80 0x7f0bd0199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:49.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.918+0000 7f0bd5685640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bd0104540 0x7f0bd01a00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:49.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.918+0000 7f0bd5685640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd010e530 con 0x7f0bd0102990 2024-04-26T18:27:49.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.918+0000 7f0bd5685640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd010e3b0 con 0x7f0bd0104540 2024-04-26T18:27:49.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.918+0000 7f0bd5685640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd010e6b0 con 0x7f0bd0103b80 2024-04-26T18:27:49.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.918+0000 7f0bceffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd0102990 0x7f0bd0198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:49.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bceffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd0102990 0x7f0bd0198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54212/0 (socket says 172.21.15.84:54212) 2024-04-26T18:27:49.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bceffd640 1 -- 172.21.15.84:0/1538304240 learned_addr learned my addr 172.21.15.84:0/1538304240 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:49.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bceffd640 1 -- 172.21.15.84:0/1538304240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0103b80 msgr2=0x7f0bd0199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:49.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bc67fc640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0103b80 0x7f0bd0199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:49.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bceffd640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0103b80 0x7f0bd0199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bceffd640 1 -- 172.21.15.84:0/1538304240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bd0104540 msgr2=0x7f0bd01a00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:49.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bcf7fe640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bd0104540 0x7f0bd01a00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:49.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bceffd640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bd0104540 0x7f0bd01a00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:49.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bceffd640 1 -- 172.21.15.84:0/1538304240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd01a0730 con 0x7f0bd0102990 2024-04-26T18:27:49.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bc67fc640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0103b80 0x7f0bd0199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:49.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bcf7fe640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bd0104540 0x7f0bd01a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:49.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bceffd640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd0102990 0x7f0bd0198b10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000b450 tx=0x7f0bc000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:49.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bccff9640 1 -- 172.21.15.84:0/1538304240 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0015020 con 0x7f0bd0102990 2024-04-26T18:27:49.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bccff9640 1 -- 172.21.15.84:0/1538304240 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0bc0011d40 con 0x7f0bd0102990 2024-04-26T18:27:49.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bd5685640 1 -- 172.21.15.84:0/1538304240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd01a0a20 con 0x7f0bd0102990 2024-04-26T18:27:49.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bccff9640 1 -- 172.21.15.84:0/1538304240 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc000f340 con 0x7f0bd0102990 2024-04-26T18:27:49.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.919+0000 7f0bd5685640 1 -- 172.21.15.84:0/1538304240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd01a0fa0 con 0x7f0bd0102990 2024-04-26T18:27:49.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.920+0000 7f0bd5685640 1 -- 172.21.15.84:0/1538304240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd0109c50 con 0x7f0bd0102990 2024-04-26T18:27:49.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.921+0000 7f0bccff9640 1 -- 172.21.15.84:0/1538304240 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0bc0018020 con 0x7f0bd0102990 2024-04-26T18:27:49.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.922+0000 7f0bccff9640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0bbc0a3c50 0x7f0bbc0a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:49.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.922+0000 7f0bc67fc640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0bbc0a3c50 0x7f0bbc0a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:49.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.922+0000 7f0bccff9640 1 -- 172.21.15.84:0/1538304240 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f0bc00cff90 con 0x7f0bd0102990 2024-04-26T18:27:49.924 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.923+0000 7f0bc67fc640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0bbc0a3c50 0x7f0bbc0a6110 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0bd019a020 tx=0x7f0bb8005d30 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:49.927 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:49.927+0000 7f0bccff9640 1 -- 172.21.15.84:0/1538304240 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0bc001f020 con 0x7f0bd0102990 2024-04-26T18:27:50.159 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.159+0000 7f0bd5685640 1 -- 172.21.15.84:0/1538304240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f0bd00506c0 con 0x7f0bd0102990 2024-04-26T18:27:50.327 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.327+0000 7f0bccff9640 1 -- 172.21.15.84:0/1538304240 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f0bc000d040 con 0x7f0bd0102990 2024-04-26T18:27:50.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.330+0000 7f0bd5685640 1 -- 172.21.15.84:0/1538304240 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0bbc0a3c50 msgr2=0x7f0bbc0a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:50.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.330+0000 7f0bd5685640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0bbc0a3c50 0x7f0bbc0a6110 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0bd019a020 tx=0x7f0bb8005d30 comp rx=0 tx=0).stop 2024-04-26T18:27:50.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.330+0000 7f0bd5685640 1 -- 172.21.15.84:0/1538304240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd0102990 msgr2=0x7f0bd0198b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:50.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.330+0000 7f0bd5685640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd0102990 0x7f0bd0198b10 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000b450 tx=0x7f0bc000b910 comp rx=0 tx=0).stop 2024-04-26T18:27:50.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.330+0000 7f0bd5685640 1 -- 172.21.15.84:0/1538304240 shutdown_connections 2024-04-26T18:27:50.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.330+0000 7f0bd5685640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f0bbc0a3c50 0x7f0bbc0a6110 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.330+0000 7f0bd5685640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0bd0104540 0x7f0bd01a00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.330+0000 7f0bd5685640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0bd0103b80 0x7f0bd0199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.330 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.330+0000 7f0bd5685640 1 --2- 172.21.15.84:0/1538304240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0bd0102990 0x7f0bd0198b10 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.330+0000 7f0bd5685640 1 -- 172.21.15.84:0/1538304240 >> 172.21.15.84:0/1538304240 conn(0x7f0bd00fdf20 msgr2=0x7f0bd0106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:50.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.330+0000 7f0bd5685640 1 -- 172.21.15.84:0/1538304240 shutdown_connections 2024-04-26T18:27:50.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.330+0000 7f0bd5685640 1 -- 172.21.15.84:0/1538304240 wait complete. 2024-04-26T18:27:50.349 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-26T18:27:50.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.452+0000 7fc081524640 1 -- 172.21.15.84:0/2779759548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c102a80 msgr2=0x7fc07c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:50.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.452+0000 7fc081524640 1 --2- 172.21.15.84:0/2779759548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c102a80 0x7fc07c102f00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc070009ca0 tx=0x7fc07002a8c0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.452+0000 7fc081524640 1 -- 172.21.15.84:0/2779759548 shutdown_connections 2024-04-26T18:27:50.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.452+0000 7fc081524640 1 --2- 172.21.15.84:0/2779759548 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc07c103440 0x7fc07c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.452+0000 7fc081524640 1 --2- 172.21.15.84:0/2779759548 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c102a80 0x7fc07c102f00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.452+0000 7fc081524640 1 --2- 172.21.15.84:0/2779759548 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc07c101890 0x7fc07c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.454 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.452+0000 7fc081524640 1 -- 172.21.15.84:0/2779759548 >> 172.21.15.84:0/2779759548 conn(0x7fc07c0fce00 msgr2=0x7fc07c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.452+0000 7fc081524640 1 -- 172.21.15.84:0/2779759548 shutdown_connections 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.452+0000 7fc081524640 1 -- 172.21.15.84:0/2779759548 wait complete. 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc081524640 1 Processor -- start 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc081524640 1 -- start start 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc081524640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c101890 0x7fc07c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc081524640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc07c102a80 0x7fc07c1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc081524640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc07c103440 0x7fc07c1a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc081524640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c10e4e0 con 0x7fc07c101890 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc081524640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c10e360 con 0x7fc07c102a80 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc081524640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc07c10e660 con 0x7fc07c103440 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc07affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c101890 0x7fc07c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc07affd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c101890 0x7fc07c198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54238/0 (socket says 172.21.15.84:54238) 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc07affd640 1 -- 172.21.15.84:0/1196535160 learned_addr learned my addr 172.21.15.84:0/1196535160 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc07a7fc640 1 --2- 172.21.15.84:0/1196535160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc07c102a80 0x7fc07c1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:50.455 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc07b7fe640 1 --2- 172.21.15.84:0/1196535160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc07c103440 0x7fc07c1a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc07affd640 1 -- 172.21.15.84:0/1196535160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc07c103440 msgr2=0x7fc07c1a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc07affd640 1 --2- 172.21.15.84:0/1196535160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc07c103440 0x7fc07c1a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc07affd640 1 -- 172.21.15.84:0/1196535160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc07c102a80 msgr2=0x7fc07c1990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc07affd640 1 --2- 172.21.15.84:0/1196535160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc07c102a80 0x7fc07c1990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc07affd640 1 -- 172.21.15.84:0/1196535160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc07c1a0780 con 0x7fc07c101890 2024-04-26T18:27:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.453+0000 7fc07affd640 1 --2- 172.21.15.84:0/1196535160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c101890 0x7fc07c198b60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc07400d950 tx=0x7fc07400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.454+0000 7fc063fff640 1 -- 172.21.15.84:0/1196535160 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc074017070 con 0x7fc07c101890 2024-04-26T18:27:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.454+0000 7fc063fff640 1 -- 172.21.15.84:0/1196535160 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc074009c30 con 0x7fc07c101890 2024-04-26T18:27:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.454+0000 7fc081524640 1 -- 172.21.15.84:0/1196535160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc07c1a0a70 con 0x7fc07c101890 2024-04-26T18:27:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.454+0000 7fc063fff640 1 -- 172.21.15.84:0/1196535160 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc07400f620 con 0x7fc07c101890 2024-04-26T18:27:50.456 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.454+0000 7fc081524640 1 -- 172.21.15.84:0/1196535160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc07c1a0f70 con 0x7fc07c101890 2024-04-26T18:27:50.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.455+0000 7fc081524640 1 -- 172.21.15.84:0/1196535160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc07c108c30 con 0x7fc07c101890 2024-04-26T18:27:50.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.456+0000 7fc063fff640 1 -- 172.21.15.84:0/1196535160 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fc07401a070 con 0x7fc07c101890 2024-04-26T18:27:50.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.457+0000 7fc063fff640 1 --2- 172.21.15.84:0/1196535160 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc0640a3ca0 0x7fc0640a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:50.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.457+0000 7fc07a7fc640 1 -- 172.21.15.84:0/1196535160 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc0640a3ca0 msgr2=0x7fc0640a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/2059208901 2024-04-26T18:27:50.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.457+0000 7fc063fff640 1 -- 172.21.15.84:0/1196535160 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fc0740cfe20 con 0x7fc07c101890 2024-04-26T18:27:50.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.457+0000 7fc07a7fc640 1 --2- 172.21.15.84:0/1196535160 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc0640a3ca0 0x7fc0640a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:27:50.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.462+0000 7fc063fff640 1 -- 172.21.15.84:0/1196535160 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc074009de0 con 0x7fc07c101890 2024-04-26T18:27:50.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:50 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1538304240' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T18:27:50.595 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:50 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: ignoring --setuser ceph since I am not root 2024-04-26T18:27:50.595 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:50 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: ignoring --setgroup ceph since I am not root 2024-04-26T18:27:50.595 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:50 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:50.514+0000 7f73cf1b9200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:27:50.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:50 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1538304240' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T18:27:50.627 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:50 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: ignoring --setuser ceph since I am not root 2024-04-26T18:27:50.627 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:50 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: ignoring --setgroup ceph since I am not root 2024-04-26T18:27:50.627 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:50 smithi103 ceph-mgr[34122]: -- 172.21.15.103:0/3700354782 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556723a3e5a0 con 0x556723a12800 2024-04-26T18:27:50.627 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:50 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:50.511+0000 7f5659ce3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:27:50.646 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.647+0000 7fc081524640 1 -- 172.21.15.84:0/1196535160 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fc07c02dad0 con 0x7fc07c101890 2024-04-26T18:27:50.650 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.650+0000 7fc063fff640 1 -- 172.21.15.84:0/1196535160 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7fc07409c180 con 0x7fc07c101890 2024-04-26T18:27:50.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.652+0000 7fc081524640 1 -- 172.21.15.84:0/1196535160 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc0640a3ca0 msgr2=0x7fc0640a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:27:50.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.652+0000 7fc081524640 1 --2- 172.21.15.84:0/1196535160 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc0640a3ca0 0x7fc0640a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.652+0000 7fc081524640 1 -- 172.21.15.84:0/1196535160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c101890 msgr2=0x7fc07c198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:50.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.652+0000 7fc081524640 1 --2- 172.21.15.84:0/1196535160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c101890 0x7fc07c198b60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc07400d950 tx=0x7fc07400de10 comp rx=0 tx=0).stop 2024-04-26T18:27:50.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.652+0000 7fc081524640 1 -- 172.21.15.84:0/1196535160 shutdown_connections 2024-04-26T18:27:50.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.652+0000 7fc081524640 1 --2- 172.21.15.84:0/1196535160 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fc0640a3ca0 0x7fc0640a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.652+0000 7fc081524640 1 --2- 172.21.15.84:0/1196535160 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc07c103440 0x7fc07c1a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.652+0000 7fc081524640 1 --2- 172.21.15.84:0/1196535160 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc07c102a80 0x7fc07c1990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.652+0000 7fc081524640 1 --2- 172.21.15.84:0/1196535160 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc07c101890 0x7fc07c198b60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.652+0000 7fc081524640 1 -- 172.21.15.84:0/1196535160 >> 172.21.15.84:0/1196535160 conn(0x7fc07c0fce00 msgr2=0x7fc07c105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:50.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.653+0000 7fc081524640 1 -- 172.21.15.84:0/1196535160 shutdown_connections 2024-04-26T18:27:50.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.653+0000 7fc081524640 1 -- 172.21.15.84:0/1196535160 wait complete. 2024-04-26T18:27:50.664 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-26T18:27:50.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:50 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1538304240' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-26T18:27:50.689 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:50 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: ignoring --setuser ceph since I am not root 2024-04-26T18:27:50.689 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:50 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: ignoring --setgroup ceph since I am not root 2024-04-26T18:27:50.689 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:50 smithi174 ceph-mgr[34566]: -- 172.21.15.174:0/1977982084 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c28dcb45a0 con 0x55c28dc89000 2024-04-26T18:27:50.689 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:50 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:50.509+0000 7fed8199f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-26T18:27:50.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.765+0000 7f6d8b437640 1 -- 172.21.15.84:0/112917624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84102a80 msgr2=0x7f6d84102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:50.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.765+0000 7f6d8b437640 1 --2- 172.21.15.84:0/112917624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84102a80 0x7f6d84102f00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f6d74009900 tx=0x7f6d7402a410 comp rx=0 tx=0).stop 2024-04-26T18:27:50.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.765+0000 7f6d8b437640 1 -- 172.21.15.84:0/112917624 shutdown_connections 2024-04-26T18:27:50.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.765+0000 7f6d8b437640 1 --2- 172.21.15.84:0/112917624 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d84103440 0x7f6d84109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.765+0000 7f6d8b437640 1 --2- 172.21.15.84:0/112917624 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84102a80 0x7f6d84102f00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.765+0000 7f6d8b437640 1 --2- 172.21.15.84:0/112917624 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d84101890 0x7f6d84101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.765+0000 7f6d8b437640 1 -- 172.21.15.84:0/112917624 >> 172.21.15.84:0/112917624 conn(0x7f6d840fce00 msgr2=0x7f6d840ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:50.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.765+0000 7f6d8b437640 1 -- 172.21.15.84:0/112917624 shutdown_connections 2024-04-26T18:27:50.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.765+0000 7f6d8b437640 1 -- 172.21.15.84:0/112917624 wait complete. 2024-04-26T18:27:50.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.765+0000 7f6d8b437640 1 Processor -- start 2024-04-26T18:27:50.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.765+0000 7f6d8b437640 1 -- start start 2024-04-26T18:27:50.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d8b437640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d84101890 0x7f6d8406ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:50.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d8b437640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84102a80 0x7f6d8419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:50.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d8b437640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d84103440 0x7f6d8419fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:50.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d8b437640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8410d3c0 con 0x7f6d84102a80 2024-04-26T18:27:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d8b437640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8410d240 con 0x7f6d84103440 2024-04-26T18:27:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d8b437640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6d8410d540 con 0x7f6d84101890 2024-04-26T18:27:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d889d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84102a80 0x7f6d8419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d889d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84102a80 0x7f6d8419d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54262/0 (socket says 172.21.15.84:54262) 2024-04-26T18:27:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d889d2640 1 -- 172.21.15.84:0/1950800550 learned_addr learned my addr 172.21.15.84:0/1950800550 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d891d3640 1 --2- 172.21.15.84:0/1950800550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d84101890 0x7f6d8406ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d899d4640 1 --2- 172.21.15.84:0/1950800550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d84103440 0x7f6d8419fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d889d2640 1 -- 172.21.15.84:0/1950800550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d84101890 msgr2=0x7f6d8406ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d889d2640 1 --2- 172.21.15.84:0/1950800550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d84101890 0x7f6d8406ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d889d2640 1 -- 172.21.15.84:0/1950800550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d84103440 msgr2=0x7f6d8419fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d889d2640 1 --2- 172.21.15.84:0/1950800550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d84103440 0x7f6d8419fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d889d2640 1 -- 172.21.15.84:0/1950800550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d841a07b0 con 0x7f6d84102a80 2024-04-26T18:27:50.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d889d2640 1 --2- 172.21.15.84:0/1950800550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84102a80 0x7f6d8419d880 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6d74005940 tx=0x7f6d74031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:50.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d7a7fc640 1 -- 172.21.15.84:0/1950800550 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d7403f030 con 0x7f6d84102a80 2024-04-26T18:27:50.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d8b437640 1 -- 172.21.15.84:0/1950800550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d841a09e0 con 0x7f6d84102a80 2024-04-26T18:27:50.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d7a7fc640 1 -- 172.21.15.84:0/1950800550 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d74003aa0 con 0x7f6d84102a80 2024-04-26T18:27:50.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d7a7fc640 1 -- 172.21.15.84:0/1950800550 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d74035070 con 0x7f6d84102a80 2024-04-26T18:27:50.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.766+0000 7f6d8b437640 1 -- 172.21.15.84:0/1950800550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d841a0f20 con 0x7f6d84102a80 2024-04-26T18:27:50.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.767+0000 7f6d8b437640 1 -- 172.21.15.84:0/1950800550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d84108bd0 con 0x7f6d84102a80 2024-04-26T18:27:50.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.768+0000 7f6d7a7fc640 1 -- 172.21.15.84:0/1950800550 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f6d74042030 con 0x7f6d84102a80 2024-04-26T18:27:50.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.769+0000 7f6d7a7fc640 1 --2- 172.21.15.84:0/1950800550 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6d700a3ca0 0x7f6d700a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:50.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.769+0000 7f6d891d3640 1 -- 172.21.15.84:0/1950800550 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6d700a3ca0 msgr2=0x7f6d700a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/2059208901 2024-04-26T18:27:50.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.769+0000 7f6d7a7fc640 1 -- 172.21.15.84:0/1950800550 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f6d740f0a20 con 0x7f6d84102a80 2024-04-26T18:27:50.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.769+0000 7f6d891d3640 1 --2- 172.21.15.84:0/1950800550 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6d700a3ca0 0x7f6d700a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:27:50.777 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.778+0000 7f6d7a7fc640 1 -- 172.21.15.84:0/1950800550 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d740bcd80 con 0x7f6d84102a80 2024-04-26T18:27:50.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.960+0000 7f6d8b437640 1 -- 172.21.15.84:0/1950800550 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f6d840635a0 con 0x7f6d84102a80 2024-04-26T18:27:50.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.964+0000 7f6d7a7fc640 1 -- 172.21.15.84:0/1950800550 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v32)=0 v32) ==== 133+0+0 (secure 0 0 0) 0x7f6d740c1080 con 0x7f6d84102a80 2024-04-26T18:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.967+0000 7f6d8b437640 1 -- 172.21.15.84:0/1950800550 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6d700a3ca0 msgr2=0x7f6d700a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.967+0000 7f6d8b437640 1 --2- 172.21.15.84:0/1950800550 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6d700a3ca0 0x7f6d700a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.967+0000 7f6d8b437640 1 -- 172.21.15.84:0/1950800550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84102a80 msgr2=0x7f6d8419d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.967+0000 7f6d8b437640 1 --2- 172.21.15.84:0/1950800550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84102a80 0x7f6d8419d880 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f6d74005940 tx=0x7f6d74031a50 comp rx=0 tx=0).stop 2024-04-26T18:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.967+0000 7f6d8b437640 1 -- 172.21.15.84:0/1950800550 shutdown_connections 2024-04-26T18:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.967+0000 7f6d8b437640 1 --2- 172.21.15.84:0/1950800550 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f6d700a3ca0 0x7f6d700a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.967+0000 7f6d8b437640 1 --2- 172.21.15.84:0/1950800550 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6d84103440 0x7f6d8419fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.967+0000 7f6d8b437640 1 --2- 172.21.15.84:0/1950800550 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6d84102a80 0x7f6d8419d880 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.967+0000 7f6d8b437640 1 --2- 172.21.15.84:0/1950800550 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6d84101890 0x7f6d8406ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:50.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.967+0000 7f6d8b437640 1 -- 172.21.15.84:0/1950800550 >> 172.21.15.84:0/1950800550 conn(0x7f6d840fce00 msgr2=0x7f6d84105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:50.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.967+0000 7f6d8b437640 1 -- 172.21.15.84:0/1950800550 shutdown_connections 2024-04-26T18:27:50.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:50.967+0000 7f6d8b437640 1 -- 172.21.15.84:0/1950800550 wait complete. 2024-04-26T18:27:50.977 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:50 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:50.706+0000 7f73cf1b9200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:27:50.977 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:50 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:50.818+0000 7f73cf1b9200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:27:50.978 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T18:27:51.005 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T18:27:51.005 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-26T18:27:51.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:27:51.006 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2024-04-26T18:27:51.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:27:51.022 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2024-04-26T18:27:51.087 INFO:teuthology.orchestra.run.smithi084.stderr:Cannot find device "ceph-brx" 2024-04-26T18:27:51.088 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:27:51.089 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-26T18:27:51.089 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T18:27:51.089 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T18:27:51.089 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T18:27:51.097 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:50 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:50.693+0000 7fed8199f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:27:51.097 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:50 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:50.795+0000 7fed8199f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:27:51.112 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:50 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:50.698+0000 7f5659ce3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:27:51.112 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:50 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:50.801+0000 7f5659ce3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-26T18:27:51.227 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:51 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:51.139+0000 7f73cf1b9200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:27:51.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:51 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1538304240' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T18:27:51.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:51 smithi103 ceph-mon[32727]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-26T18:27:51.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:51 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1196535160' entity='client.admin' 2024-04-26T18:27:51.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:51 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1950800550' entity='client.admin' 2024-04-26T18:27:51.377 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:51 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:51.111+0000 7f5659ce3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:27:51.377 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:51 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:51.198+0000 7f5659ce3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:27:51.378 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:27:51.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:51 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1538304240' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T18:27:51.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:51 smithi174 ceph-mon[32705]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-26T18:27:51.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:51 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1196535160' entity='client.admin' 2024-04-26T18:27:51.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:51 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1950800550' entity='client.admin' 2024-04-26T18:27:51.439 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:51 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:51.097+0000 7fed8199f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-26T18:27:51.439 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:51 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:51.180+0000 7fed8199f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:27:51.500 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:51 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:51.227+0000 7f73cf1b9200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-26T18:27:51.500 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:51 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1538304240' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-26T18:27:51.500 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:51 smithi084 ceph-mon[29141]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-04-26T18:27:51.500 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:51 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1196535160' entity='client.admin' 2024-04-26T18:27:51.500 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:51 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1950800550' entity='client.admin' 2024-04-26T18:27:52.688 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:52 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:52.354+0000 7fed8199f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:27:52.689 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:52 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:52.523+0000 7fed8199f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:27:52.712 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:52 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:52.445+0000 7f73cf1b9200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:27:52.712 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:52 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:52.624+0000 7f73cf1b9200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:27:52.728 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:52 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:52.388+0000 7f5659ce3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-26T18:27:52.729 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:52 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:52.558+0000 7f5659ce3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-26T18:27:52.729 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:52 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:52.729+0000 7f5659ce3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:27:52.973 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:52 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:52.797+0000 7f73cf1b9200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:27:52.982 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:52 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:52.695+0000 7fed8199f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-26T18:27:53.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.087+0000 7f149e7c6700 1 -- 172.21.15.84:0/1275721942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14981083c0 msgr2=0x7f14981087a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:53.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.087+0000 7f149e7c6700 1 --2- 172.21.15.84:0/1275721942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14981083c0 0x7f14981087a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1488009990 tx=0x7f1488009ca0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.087+0000 7f149e7c6700 1 -- 172.21.15.84:0/1275721942 shutdown_connections 2024-04-26T18:27:53.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.087+0000 7f149e7c6700 1 --2- 172.21.15.84:0/1275721942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14981083c0 0x7f14981087a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.087+0000 7f149e7c6700 1 --2- 172.21.15.84:0/1275721942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1498102d10 0x7f149810f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.087+0000 7f149e7c6700 1 --2- 172.21.15.84:0/1275721942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1498102350 0x7f14981027d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.087+0000 7f149e7c6700 1 -- 172.21.15.84:0/1275721942 >> 172.21.15.84:0/1275721942 conn(0x7f1498079360 msgr2=0x7f1498079770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:53.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.087+0000 7f149e7c6700 1 -- 172.21.15.84:0/1275721942 shutdown_connections 2024-04-26T18:27:53.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.087+0000 7f149e7c6700 1 -- 172.21.15.84:0/1275721942 wait complete. 2024-04-26T18:27:53.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f149e7c6700 1 Processor -- start 2024-04-26T18:27:53.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f149e7c6700 1 -- start start 2024-04-26T18:27:53.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f149e7c6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1498102350 0x7f149819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:53.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f149e7c6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1498102d10 0x7f149819fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f149e7c6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14981083c0 0x7f1498199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f149e7c6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1498112510 con 0x7f14981083c0 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f149e7c6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1498112390 con 0x7f1498102d10 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f149e7c6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1498112690 con 0x7f1498102350 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f1497fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1498102350 0x7f149819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f149cd63700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14981083c0 0x7f1498199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f14977fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1498102d10 0x7f149819fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f149cd63700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14981083c0 0x7f1498199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54284/0 (socket says 172.21.15.84:54284) 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f1497fff700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1498102350 0x7f149819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.84:40908/0 (socket says 172.21.15.84:40908) 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.088+0000 7f149cd63700 1 -- 172.21.15.84:0/3044606112 learned_addr learned my addr 172.21.15.84:0/3044606112 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.089+0000 7f14977fe700 1 -- 172.21.15.84:0/3044606112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1498102350 msgr2=0x7f149819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.089+0000 7f14977fe700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1498102350 0x7f149819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.089+0000 7f14977fe700 1 -- 172.21.15.84:0/3044606112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14981083c0 msgr2=0x7f1498199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:53.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.089+0000 7f14977fe700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14981083c0 0x7f1498199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.089+0000 7f14977fe700 1 -- 172.21.15.84:0/3044606112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f149819a0e0 con 0x7f1498102d10 2024-04-26T18:27:53.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.089+0000 7f1497fff700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1498102350 0x7f149819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:53.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.089+0000 7f149cd63700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14981083c0 0x7f1498199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:53.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.089+0000 7f14977fe700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1498102d10 0x7f149819fcb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f148c00e9f0 tx=0x7f148c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:53.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.089+0000 7f14957fa700 1 -- 172.21.15.84:0/3044606112 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c004220 con 0x7f1498102d10 2024-04-26T18:27:53.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.089+0000 7f149e7c6700 1 -- 172.21.15.84:0/3044606112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f149819a3d0 con 0x7f1498102d10 2024-04-26T18:27:53.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.089+0000 7f14957fa700 1 -- 172.21.15.84:0/3044606112 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f148c0043c0 con 0x7f1498102d10 2024-04-26T18:27:53.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.089+0000 7f14957fa700 1 -- 172.21.15.84:0/3044606112 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f148c005020 con 0x7f1498102d10 2024-04-26T18:27:53.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.089+0000 7f149e7c6700 1 -- 172.21.15.84:0/3044606112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14981a6640 con 0x7f1498102d10 2024-04-26T18:27:53.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.090+0000 7f149e7c6700 1 -- 172.21.15.84:0/3044606112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1498103a70 con 0x7f1498102d10 2024-04-26T18:27:53.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.091+0000 7f14957fa700 1 -- 172.21.15.84:0/3044606112 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f148c004750 con 0x7f1498102d10 2024-04-26T18:27:53.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.092+0000 7f14957fa700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14800a3c90 0x7f14800a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:53.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.092+0000 7f1497fff700 1 -- 172.21.15.84:0/3044606112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14800a3c90 msgr2=0x7f14800a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/2059208901 2024-04-26T18:27:53.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.092+0000 7f14957fa700 1 -- 172.21.15.84:0/3044606112 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f148c0d0570 con 0x7f1498102d10 2024-04-26T18:27:53.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.092+0000 7f1497fff700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14800a3c90 0x7f14800a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:27:53.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.096+0000 7f14957fa700 1 -- 172.21.15.84:0/3044606112 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f148c042e20 con 0x7f1498102d10 2024-04-26T18:27:53.280 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:52 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:52.982+0000 7fed8199f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:27:53.280 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:53 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:53.075+0000 7fed8199f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:27:53.280 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:53 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:53.280+0000 7fed8199f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:27:53.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.293+0000 7f1497fff700 1 -- 172.21.15.84:0/3044606112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14800a3c90 msgr2=0x7f14800a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/2059208901 2024-04-26T18:27:53.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.293+0000 7f1497fff700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14800a3c90 0x7f14800a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:27:53.293 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:53 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:53.113+0000 7f73cf1b9200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:27:53.293 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:53 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:53.209+0000 7f73cf1b9200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:27:53.314 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:53 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:53.018+0000 7f5659ce3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-26T18:27:53.315 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:53 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:53.111+0000 7f5659ce3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-26T18:27:53.315 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:53 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:53.315+0000 7f5659ce3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:27:53.349 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.349+0000 7f149e7c6700 1 -- 172.21.15.84:0/3044606112 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f149806a110 con 0x7f1498102d10 2024-04-26T18:27:53.350 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.350+0000 7f14957fa700 1 -- 172.21.15.84:0/3044606112 <== mon.2 v2:172.21.15.103:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+73 (secure 0 0 0) 0x7f149806a110 con 0x7f1498102d10 2024-04-26T18:27:53.350 INFO:teuthology.orchestra.run.smithi084.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-26T18:27:53.352 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.352+0000 7f149e7c6700 1 -- 172.21.15.84:0/3044606112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14800a3c90 msgr2=0x7f14800a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:27:53.352 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.352+0000 7f149e7c6700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14800a3c90 0x7f14800a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.352 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.352+0000 7f149e7c6700 1 -- 172.21.15.84:0/3044606112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1498102d10 msgr2=0x7f149819fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:53.352 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.352+0000 7f149e7c6700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1498102d10 0x7f149819fcb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f148c00e9f0 tx=0x7f148c00ed00 comp rx=0 tx=0).stop 2024-04-26T18:27:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.352+0000 7f149e7c6700 1 -- 172.21.15.84:0/3044606112 shutdown_connections 2024-04-26T18:27:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.352+0000 7f149e7c6700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f14800a3c90 0x7f14800a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.352+0000 7f149e7c6700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14981083c0 0x7f1498199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.352+0000 7f149e7c6700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1498102350 0x7f149819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.352+0000 7f149e7c6700 1 --2- 172.21.15.84:0/3044606112 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1498102d10 0x7f149819fcb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.352+0000 7f149e7c6700 1 -- 172.21.15.84:0/3044606112 >> 172.21.15.84:0/3044606112 conn(0x7f1498079360 msgr2=0x7f1498101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.352+0000 7f149e7c6700 1 -- 172.21.15.84:0/3044606112 shutdown_connections 2024-04-26T18:27:53.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:53.352+0000 7f149e7c6700 1 -- 172.21.15.84:0/3044606112 wait complete. 2024-04-26T18:27:53.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:53 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3044606112' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T18:27:53.595 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:53 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:53.422+0000 7f73cf1b9200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-26T18:27:53.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:53 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3044606112' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T18:27:53.627 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:53 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:53.509+0000 7f5659ce3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:27:53.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:53 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3044606112' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T18:27:53.689 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:53 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:53.474+0000 7fed8199f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:27:54.045 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:53 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:53.617+0000 7f73cf1b9200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-26T18:27:54.075 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T18:27:54.077 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T18:27:54.077 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi084.front.sepia.ceph.com 2024-04-26T18:27:54.077 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T18:27:54.077 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T18:27:54.077 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T18:27:54.077 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T18:27:54.077 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T18:27:54.077 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T18:27:54.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:27:54.078 DEBUG:teuthology.orchestra.run.smithi084:> ip addr 2024-04-26T18:27:54.094 INFO:teuthology.orchestra.run.smithi084.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T18:27:54.094 INFO:teuthology.orchestra.run.smithi084.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T18:27:54.094 INFO:teuthology.orchestra.run.smithi084.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T18:27:54.094 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2024-04-26T18:27:54.094 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 ::1/128 scope host 2024-04-26T18:27:54.094 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2024-04-26T18:27:54.094 INFO:teuthology.orchestra.run.smithi084.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T18:27:54.094 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:68 brd ff:ff:ff:ff:ff:ff 2024-04-26T18:27:54.094 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f0 2024-04-26T18:27:54.095 INFO:teuthology.orchestra.run.smithi084.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T18:27:54.095 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:d8:da:69 brd ff:ff:ff:ff:ff:ff 2024-04-26T18:27:54.095 INFO:teuthology.orchestra.run.smithi084.stdout: altname enp7s0f1 2024-04-26T18:27:54.095 INFO:teuthology.orchestra.run.smithi084.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T18:27:54.095 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:50 brd ff:ff:ff:ff:ff:ff 2024-04-26T18:27:54.095 INFO:teuthology.orchestra.run.smithi084.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T18:27:54.095 INFO:teuthology.orchestra.run.smithi084.stdout: link/ether 0c:c4:7a:8f:d7:51 brd ff:ff:ff:ff:ff:ff 2024-04-26T18:27:54.095 INFO:teuthology.orchestra.run.smithi084.stdout: inet 172.21.15.84/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T18:27:54.095 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft 41984sec preferred_lft 41984sec 2024-04-26T18:27:54.095 INFO:teuthology.orchestra.run.smithi084.stdout: inet6 fe80::9516:45:208e:5bbf/64 scope link noprefixroute 2024-04-26T18:27:54.095 INFO:teuthology.orchestra.run.smithi084.stdout: valid_lft forever preferred_lft forever 2024-04-26T18:27:54.096 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T18:27:54.096 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:27:54.096 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-04-26T18:27:54.096 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add name ceph-brx type bridge 2024-04-26T18:27:54.096 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr flush dev ceph-brx 2024-04-26T18:27:54.096 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set ceph-brx up 2024-04-26T18:27:54.096 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T18:27:54.096 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-04-26T18:27:54.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:27:54.275 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:27:54.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:27:54.281 DEBUG:teuthology.orchestra.run.smithi084:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T18:27:54.297 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:54 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:54.234+0000 7f73cf1b9200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:27:54.305 INFO:teuthology.orchestra.run.smithi084.stdout:1 2024-04-26T18:27:54.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:27:54.307 DEBUG:teuthology.orchestra.run.smithi084:> route 2024-04-26T18:27:54.377 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:54 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:54.115+0000 7f5659ce3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:27:54.391 INFO:teuthology.orchestra.run.smithi084.stdout:Kernel IP routing table 2024-04-26T18:27:54.391 INFO:teuthology.orchestra.run.smithi084.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T18:27:54.391 INFO:teuthology.orchestra.run.smithi084.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T18:27:54.391 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T18:27:54.391 INFO:teuthology.orchestra.run.smithi084.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T18:27:54.391 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:27:54.392 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-04-26T18:27:54.392 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T18:27:54.392 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T18:27:54.392 DEBUG:teuthology.orchestra.run.smithi084:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T18:27:54.392 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-04-26T18:27:54.438 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:54 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:54.078+0000 7fed8199f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-26T18:27:54.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:27:54.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:27:54.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:27:54.692 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2024-04-26T18:27:54.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:27:54.708 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list-id 2024-04-26T18:27:54.763 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:27:54.763 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-04-26T18:27:54.763 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T18:27:54.763 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T18:27:54.763 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-04-26T18:27:54.798 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:54 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:54.499+0000 7fed8199f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:27:54.798 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:54 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:54.595+0000 7fed8199f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:27:54.798 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:54 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:54.798+0000 7fed8199f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:27:54.842 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:54 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:54.541+0000 7f5659ce3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:27:54.842 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:54 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:54.640+0000 7f5659ce3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:27:54.842 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:54 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:54.842+0000 7f5659ce3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:27:54.844 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:54 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:54.691+0000 7f73cf1b9200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-26T18:27:54.844 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:54 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:54.794+0000 7f73cf1b9200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-26T18:27:54.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:27:54.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:27:54.880 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T18:27:54.880 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:27:54.880 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-04-26T18:27:54.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T18:27:54.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T18:27:54.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T18:27:54.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T18:27:54.880 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T18:27:54.880 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-04-26T18:27:54.968 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:27:55.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:27:55.044 DEBUG:teuthology.orchestra.run.smithi084:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T18:27:55.044 DEBUG:teuthology.orchestra.run.smithi084:> set -e 2024-04-26T18:27:55.044 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set brx.0 up 2024-04-26T18:27:55.044 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T18:27:55.044 DEBUG:teuthology.orchestra.run.smithi084:> ') 2024-04-26T18:27:55.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:27:55.110 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:55 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:55.013+0000 7f73cf1b9200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-26T18:27:55.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:27:55.121 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T18:27:55.121 DEBUG:teuthology.orchestra.run.smithi084:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T18:27:55.136 INFO:teuthology.orchestra.run.smithi084.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T18:27:55.136 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T18:27:55.136 DEBUG:teuthology.orchestra.run.smithi084:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T18:27:55.191 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-26T18:27:55.191 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-26T18:27:55.191 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/44f1e2f4-deda-464d-9481-4db402deadff 2024-04-26T18:27:55.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:27:55.191 DEBUG:teuthology.orchestra.run.smithi084:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/44f1e2f4-deda-464d-9481-4db402deadff /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-26T18:27:55.286 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-26T18:27:55.286 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-26T18:27:55.286 INFO:teuthology.orchestra.run.smithi084.stdout:invalid new device string format 2024-04-26T18:27:55.397 INFO:teuthology.orchestra.run.smithi084.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync 2024-04-26T18:27:55.397 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync". 2024-04-26T18:27:55.397 INFO:teuthology.orchestra.run.smithi084.stdout:invalid new device string format 2024-04-26T18:27:55.398 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: resolved to: "172.21.15.84:3300,172.21.15.103:3300,172.21.15.174:3300" 2024-04-26T18:27:55.398 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.84:3300,172.21.15.103:3300,172.21.15.174:3300:/volumes/qa/sv_1/44f1e2f4-deda-464d-9481-4db402deadff 2024-04-26T18:27:55.398 INFO:teuthology.orchestra.run.smithi084.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,nowsync,key=0,fsid=df3f3816-03f8-11ef-bc93-c7b262605968" will pass to kernel 2024-04-26T18:27:55.399 INFO:teuthology.orchestra.run.smithi084.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T18:27:55.399 INFO:teuthology.orchestra.run.smithi084.stdout: You just mounted a file system that supports labels which does not 2024-04-26T18:27:55.399 INFO:teuthology.orchestra.run.smithi084.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T18:27:55.399 INFO:teuthology.orchestra.run.smithi084.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T18:27:55.399 INFO:teuthology.orchestra.run.smithi084.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T18:27:55.401 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T18:27:55.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:27:55.401 DEBUG:teuthology.orchestra.run.smithi084:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T18:27:55.425 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T18:27:55.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T18:27:55.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:27:55.523 INFO:teuthology.orchestra.run.smithi084.stdout:client24818 2024-04-26T18:27:55.523 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:27:55.523 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T18:27:55.539 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T18:27:55.627 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:55 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:55.365+0000 7f5659ce3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:27:55.627 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:55 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:55.461+0000 7f5659ce3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:27:55.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T18:27:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:27:55.637 INFO:teuthology.orchestra.run.smithi084.stdout:client24818 2024-04-26T18:27:55.637 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T18:27:55.656 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:55 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:55.320+0000 7fed8199f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:27:55.656 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:55 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:55.412+0000 7fed8199f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:27:55.669 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:55 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:55.567+0000 7f73cf1b9200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-26T18:27:55.669 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:55 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:55.669+0000 7f73cf1b9200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-26T18:27:55.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T18:27:55.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:27:55.693 INFO:teuthology.orchestra.run.smithi084.stdout:client24818 2024-04-26T18:27:55.693 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/df3f3816-03f8-11ef-bc93-c7b262605968.client24818/status') 2024-04-26T18:27:55.782 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T18:27:55.792 INFO:teuthology.orchestra.run.smithi084.stdout:instance: client.24818 (3)172.21.15.84:0/452953064 2024-04-26T18:27:55.792 INFO:teuthology.orchestra.run.smithi084.stdout:blocklisted: false 2024-04-26T18:27:55.792 INFO:teuthology.orchestra.run.smithi084.stderr:0+1 records in 2024-04-26T18:27:55.792 INFO:teuthology.orchestra.run.smithi084.stderr:0+1 records out 2024-04-26T18:27:55.792 INFO:teuthology.orchestra.run.smithi084.stderr:70 bytes copied, 3.8576e-05 s, 1.8 MB/s 2024-04-26T18:27:55.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T18:27:55.800 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T18:27:55.809 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T18:27:55.809 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi084.front.sepia.ceph.com 2024-04-26T18:27:55.809 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-04-26T18:27:55.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.911+0000 7fb80f389640 1 -- 172.21.15.84:0/698491865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb808102a80 msgr2=0x7fb808102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:55.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.911+0000 7fb80f389640 1 --2- 172.21.15.84:0/698491865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb808102a80 0x7fb808102f00 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb7f8009900 tx=0x7fb7f802a410 comp rx=0 tx=0).stop 2024-04-26T18:27:55.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.911+0000 7fb80f389640 1 -- 172.21.15.84:0/698491865 shutdown_connections 2024-04-26T18:27:55.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.911+0000 7fb80f389640 1 --2- 172.21.15.84:0/698491865 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb808103440 0x7fb808109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:55.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.911+0000 7fb80f389640 1 --2- 172.21.15.84:0/698491865 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb808102a80 0x7fb808102f00 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:55.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.911+0000 7fb80f389640 1 --2- 172.21.15.84:0/698491865 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb808101890 0x7fb808101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:55.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.911+0000 7fb80f389640 1 -- 172.21.15.84:0/698491865 >> 172.21.15.84:0/698491865 conn(0x7fb8080fce00 msgr2=0x7fb8080ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:55.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.911+0000 7fb80f389640 1 -- 172.21.15.84:0/698491865 shutdown_connections 2024-04-26T18:27:55.912 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.911+0000 7fb80f389640 1 -- 172.21.15.84:0/698491865 wait complete. 2024-04-26T18:27:55.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.912+0000 7fb80f389640 1 Processor -- start 2024-04-26T18:27:55.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.912+0000 7fb80f389640 1 -- start start 2024-04-26T18:27:55.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.912+0000 7fb80f389640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb808101890 0x7fb808198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:55.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.912+0000 7fb80f389640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb808102a80 0x7fb8081990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:55.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.912+0000 7fb80f389640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb808103440 0x7fb8081a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:55.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.912+0000 7fb80f389640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb80810e4e0 con 0x7fb808101890 2024-04-26T18:27:55.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.912+0000 7fb80f389640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb80810e360 con 0x7fb808103440 2024-04-26T18:27:55.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.912+0000 7fb80f389640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb80810e660 con 0x7fb808102a80 2024-04-26T18:27:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80d125640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb808101890 0x7fb808198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80d125640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb808101890 0x7fb808198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54308/0 (socket says 172.21.15.84:54308) 2024-04-26T18:27:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80d125640 1 -- 172.21.15.84:0/1585179270 learned_addr learned my addr 172.21.15.84:0/1585179270 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80d926640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb808103440 0x7fb8081a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80c924640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb808102a80 0x7fb8081990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80d125640 1 -- 172.21.15.84:0/1585179270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb808102a80 msgr2=0x7fb8081990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80d125640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb808102a80 0x7fb8081990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80d125640 1 -- 172.21.15.84:0/1585179270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb808103440 msgr2=0x7fb8081a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80d125640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb808103440 0x7fb8081a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80d125640 1 -- 172.21.15.84:0/1585179270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8081a0780 con 0x7fb808101890 2024-04-26T18:27:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80d926640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb808103440 0x7fb8081a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:55.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80d125640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb808101890 0x7fb808198bb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb80400e9c0 tx=0x7fb80400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:55.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb7fe7fc640 1 -- 172.21.15.84:0/1585179270 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80400cd80 con 0x7fb808101890 2024-04-26T18:27:55.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb7fe7fc640 1 -- 172.21.15.84:0/1585179270 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8040035f0 con 0x7fb808101890 2024-04-26T18:27:55.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80f389640 1 -- 172.21.15.84:0/1585179270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8081a0a70 con 0x7fb808101890 2024-04-26T18:27:55.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb7fe7fc640 1 -- 172.21.15.84:0/1585179270 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804010690 con 0x7fb808101890 2024-04-26T18:27:55.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.913+0000 7fb80f389640 1 -- 172.21.15.84:0/1585179270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8081a0f40 con 0x7fb808101890 2024-04-26T18:27:55.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.914+0000 7fb80f389640 1 -- 172.21.15.84:0/1585179270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb808108c80 con 0x7fb808101890 2024-04-26T18:27:55.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.915+0000 7fb7fe7fc640 1 -- 172.21.15.84:0/1585179270 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fb804004000 con 0x7fb808101890 2024-04-26T18:27:55.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.916+0000 7fb7fe7fc640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7f40a3ca0 0x7fb7f40a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:55.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.916+0000 7fb80c924640 1 -- 172.21.15.84:0/1585179270 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7f40a3ca0 msgr2=0x7fb7f40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/2059208901 2024-04-26T18:27:55.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.916+0000 7fb7fe7fc640 1 -- 172.21.15.84:0/1585179270 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7fb8040d0970 con 0x7fb808101890 2024-04-26T18:27:55.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.916+0000 7fb80c924640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7f40a3ca0 0x7fb7f40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:27:55.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:55.920+0000 7fb7fe7fc640 1 -- 172.21.15.84:0/1585179270 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb80409d320 con 0x7fb808101890 2024-04-26T18:27:55.939 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:55 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:55.656+0000 7fed8199f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:27:55.939 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:55 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:55.788+0000 7fed8199f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:27:55.939 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:55 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:55.884+0000 7fed8199f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:27:56.050 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:55 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:55.706+0000 7f5659ce3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:27:56.051 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:55 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:55.838+0000 7f5659ce3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:27:56.051 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:55 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:55.935+0000 7f5659ce3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:27:56.051 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:56 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:56.051+0000 7f5659ce3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:27:56.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.117+0000 7fb80c924640 1 -- 172.21.15.84:0/1585179270 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7f40a3ca0 msgr2=0x7fb7f40a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/2059208901 2024-04-26T18:27:56.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.117+0000 7fb80c924640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7f40a3ca0 0x7fb7f40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:27:56.146 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.146+0000 7fb80f389640 1 -- 172.21.15.84:0/1585179270 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fb808063560 con 0x7fb808101890 2024-04-26T18:27:56.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.198+0000 7fb7fe7fc640 1 -- 172.21.15.84:0/1585179270 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v29) ==== 118+0+0 (secure 0 0 0) 0x7fb80409cc20 con 0x7fb808101890 2024-04-26T18:27:56.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.201+0000 7fb80f389640 1 -- 172.21.15.84:0/1585179270 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7f40a3ca0 msgr2=0x7fb7f40a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:27:56.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.201+0000 7fb80f389640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7f40a3ca0 0x7fb7f40a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:56.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.201+0000 7fb80f389640 1 -- 172.21.15.84:0/1585179270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb808101890 msgr2=0x7fb808198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:56.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.201+0000 7fb80f389640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb808101890 0x7fb808198bb0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb80400e9c0 tx=0x7fb80400ee80 comp rx=0 tx=0).stop 2024-04-26T18:27:56.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.201+0000 7fb80f389640 1 -- 172.21.15.84:0/1585179270 shutdown_connections 2024-04-26T18:27:56.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.201+0000 7fb80f389640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7fb7f40a3ca0 0x7fb7f40a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:56.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.201+0000 7fb80f389640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb808103440 0x7fb8081a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:56.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.201+0000 7fb80f389640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb808102a80 0x7fb8081990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:56.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.201+0000 7fb80f389640 1 --2- 172.21.15.84:0/1585179270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb808101890 0x7fb808198bb0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:56.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.201+0000 7fb80f389640 1 -- 172.21.15.84:0/1585179270 >> 172.21.15.84:0/1585179270 conn(0x7fb8080fce00 msgr2=0x7fb808105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:56.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.201+0000 7fb80f389640 1 -- 172.21.15.84:0/1585179270 shutdown_connections 2024-04-26T18:27:56.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.201+0000 7fb80f389640 1 -- 172.21.15.84:0/1585179270 wait complete. 2024-04-26T18:27:56.202 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:55 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:55.931+0000 7f73cf1b9200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-26T18:27:56.203 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:56 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:56.071+0000 7f73cf1b9200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-26T18:27:56.203 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:56 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:56.170+0000 7f73cf1b9200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-26T18:27:56.212 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-04-26T18:27:56.305 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:56 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1585179270' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T18:27:56.306 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:56 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:56.000+0000 7fed8199f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:27:56.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.315+0000 7f578d0b0640 1 -- 172.21.15.84:0/156175645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5788101890 msgr2=0x7f5788101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:56.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.315+0000 7f578d0b0640 1 --2- 172.21.15.84:0/156175645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5788101890 0x7f5788101cb0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f5778009900 tx=0x7f577802a410 comp rx=0 tx=0).stop 2024-04-26T18:27:56.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.315+0000 7f578d0b0640 1 -- 172.21.15.84:0/156175645 shutdown_connections 2024-04-26T18:27:56.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.315+0000 7f578d0b0640 1 --2- 172.21.15.84:0/156175645 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5788103440 0x7f5788109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:56.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.315+0000 7f578d0b0640 1 --2- 172.21.15.84:0/156175645 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788102a80 0x7f5788102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:56.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.315+0000 7f578d0b0640 1 --2- 172.21.15.84:0/156175645 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5788101890 0x7f5788101cb0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:56.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.315+0000 7f578d0b0640 1 -- 172.21.15.84:0/156175645 >> 172.21.15.84:0/156175645 conn(0x7f57880fce00 msgr2=0x7f57880ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:56.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.315+0000 7f578d0b0640 1 -- 172.21.15.84:0/156175645 shutdown_connections 2024-04-26T18:27:56.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.316+0000 7f578d0b0640 1 -- 172.21.15.84:0/156175645 wait complete. 2024-04-26T18:27:56.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.316+0000 7f578d0b0640 1 Processor -- start 2024-04-26T18:27:56.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.316+0000 7f578d0b0640 1 -- start start 2024-04-26T18:27:56.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578d0b0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788101890 0x7f5788105820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:56.318 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578d0b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5788102a80 0x7f5788103e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578d0b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5788103440 0x7f57881043a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578d0b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f578810d3c0 con 0x7f5788102a80 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578d0b0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f578810d240 con 0x7f5788101890 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578d0b0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f578810d540 con 0x7f5788103440 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5788102a80 0x7f5788103e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5788102a80 0x7f5788103e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54328/0 (socket says 172.21.15.84:54328) 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578659c640 1 -- 172.21.15.84:0/4253666079 learned_addr learned my addr 172.21.15.84:0/4253666079 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578759e640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5788103440 0x7f57881043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f5786d9d640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788101890 0x7f5788105820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578659c640 1 -- 172.21.15.84:0/4253666079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5788103440 msgr2=0x7f57881043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578659c640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5788103440 0x7f57881043a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578659c640 1 -- 172.21.15.84:0/4253666079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788101890 msgr2=0x7f5788105820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578659c640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5788101890 0x7f5788105820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:56.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578659c640 1 -- 172.21.15.84:0/4253666079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5788104c30 con 0x7f5788102a80 2024-04-26T18:27:56.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578759e640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5788103440 0x7f57881043a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:56.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578659c640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5788102a80 0x7f5788103e60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f577c00d950 tx=0x7f577c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:56.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f576ffff640 1 -- 172.21.15.84:0/4253666079 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c017070 con 0x7f5788102a80 2024-04-26T18:27:56.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f576ffff640 1 -- 172.21.15.84:0/4253666079 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577c009c30 con 0x7f5788102a80 2024-04-26T18:27:56.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578d0b0640 1 -- 172.21.15.84:0/4253666079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57881a2c80 con 0x7f5788102a80 2024-04-26T18:27:56.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f576ffff640 1 -- 172.21.15.84:0/4253666079 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577c00f620 con 0x7f5788102a80 2024-04-26T18:27:56.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.317+0000 7f578d0b0640 1 -- 172.21.15.84:0/4253666079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57881a31d0 con 0x7f5788102a80 2024-04-26T18:27:56.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.318+0000 7f578d0b0640 1 -- 172.21.15.84:0/4253666079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57881a33a0 con 0x7f5788102a80 2024-04-26T18:27:56.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.320+0000 7f576ffff640 1 -- 172.21.15.84:0/4253666079 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f577c005030 con 0x7f5788102a80 2024-04-26T18:27:56.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.320+0000 7f576ffff640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f57700a3ca0 0x7f57700a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:56.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.320+0000 7f5786d9d640 1 -- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f57700a3ca0 msgr2=0x7f57700a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/2059208901 2024-04-26T18:27:56.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.321+0000 7f576ffff640 1 -- 172.21.15.84:0/4253666079 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8277+0+0 (secure 0 0 0) 0x7f577c0cfff0 con 0x7f5788102a80 2024-04-26T18:27:56.321 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.321+0000 7f5786d9d640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f57700a3ca0 0x7f57700a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T18:27:56.324 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.324+0000 7f576ffff640 1 -- 172.21.15.84:0/4253666079 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f577c009de0 con 0x7f5788102a80 2024-04-26T18:27:56.353 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:56 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1585179270' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T18:27:56.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.522+0000 7f5786d9d640 1 -- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f57700a3ca0 msgr2=0x7f57700a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/2059208901 2024-04-26T18:27:56.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.522+0000 7f5786d9d640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f57700a3ca0 0x7f57700a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T18:27:56.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.532+0000 7f578d0b0640 1 -- 172.21.15.84:0/4253666079 --> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f57880ff000 con 0x7f57700a3ca0 2024-04-26T18:27:56.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:56 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1585179270' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-04-26T18:27:56.595 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:56 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:56.296+0000 7f73cf1b9200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-26T18:27:56.627 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:56 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:56.353+0000 7f5659ce3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:27:56.689 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:56 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:56.305+0000 7fed8199f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:27:56.922 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.923+0000 7f5786d9d640 1 -- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f57700a3ca0 msgr2=0x7f57700a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.84:6800/2059208901 2024-04-26T18:27:56.923 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:56.923+0000 7f5786d9d640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f57700a3ca0 0x7f57700a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T18:27:57.094 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:56 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:56.606+0000 7f73cf1b9200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-26T18:27:57.189 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:56 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:56.904+0000 7fed8199f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:27:57.189 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z[34562]: 2024-04-26T18:27:57.121+0000 7fed8199f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:27:57.216 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:57.217+0000 7f576ffff640 1 -- 172.21.15.84:0/4253666079 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f577c0989c0 con 0x7f5788102a80 2024-04-26T18:27:57.377 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:56 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:56.960+0000 7f5659ce3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:27:57.377 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y[34118]: 2024-04-26T18:27:57.179+0000 7f5659ce3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:27:57.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1585179270' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T18:27:57.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T18:27:57.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: Standby manager daemon z restarted 2024-04-26T18:27:57.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: Standby manager daemon z started 2024-04-26T18:27:57.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.174:0/3457464374' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:27:57.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.174:0/3457464374' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:27:57.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.174:0/3457464374' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:27:57.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.174:0/3457464374' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:27:57.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: Standby manager daemon y restarted 2024-04-26T18:27:57.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: Standby manager daemon y started 2024-04-26T18:27:57.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.103:0/4246398511' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:27:57.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.103:0/4246398511' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:27:57.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.103:0/4246398511' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:27:57.378 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:57 smithi103 ceph-mon[32727]: from='mgr.? 172.21.15.103:0/4246398511' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:27:57.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:57.457+0000 7f576ffff640 1 -- 172.21.15.84:0/4253666079 <== mon.0 v2:172.21.15.84:3300/0 8 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f577c0980a0 con 0x7f5788102a80 2024-04-26T18:27:57.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:57.457+0000 7f576ffff640 1 -- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f57700a3ca0 msgr2=0x7f57700a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T18:27:57.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:57.457+0000 7f576ffff640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/2059208901,v1:172.21.15.84:6801/2059208901] conn(0x7f57700a3ca0 0x7f57700a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:57.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1585179270' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T18:27:57.586 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T18:27:57.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: Standby manager daemon z restarted 2024-04-26T18:27:57.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: Standby manager daemon z started 2024-04-26T18:27:57.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.174:0/3457464374' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:27:57.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.174:0/3457464374' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:27:57.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.174:0/3457464374' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:27:57.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.174:0/3457464374' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:27:57.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: Standby manager daemon y restarted 2024-04-26T18:27:57.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: Standby manager daemon y started 2024-04-26T18:27:57.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.103:0/4246398511' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:27:57.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.103:0/4246398511' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:27:57.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.103:0/4246398511' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:27:57.587 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:57 smithi174 ceph-mon[32705]: from='mgr.? 172.21.15.103:0/4246398511' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:27:57.588 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:57.220+0000 7f73cf1b9200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-26T18:27:57.588 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:57.442+0000 7f73cf1b9200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-26T18:27:57.588 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1585179270' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-04-26T18:27:57.588 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-26T18:27:57.588 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: Standby manager daemon z restarted 2024-04-26T18:27:57.588 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: Standby manager daemon z started 2024-04-26T18:27:57.588 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.174:0/3457464374' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-26T18:27:57.588 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.174:0/3457464374' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:27:57.588 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.174:0/3457464374' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-26T18:27:57.589 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.174:0/3457464374' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:27:57.589 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: Standby manager daemon y restarted 2024-04-26T18:27:57.589 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: Standby manager daemon y started 2024-04-26T18:27:57.589 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.103:0/4246398511' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-26T18:27:57.589 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.103:0/4246398511' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T18:27:57.589 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.103:0/4246398511' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-26T18:27:57.589 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-mon[29141]: from='mgr.? 172.21.15.103:0/4246398511' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T18:27:57.845 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:57.686+0000 7f732db13700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:57.845 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:57.686+0000 7f732db13700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:57.845 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:57.686+0000 7f732db13700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:57.845 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:57.686+0000 7f732db13700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:57.845 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:57.686+0000 7f732db13700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:57.845 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:57.688+0000 7f733131a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:57.845 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:57.688+0000 7f733131a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:57.845 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:57.688+0000 7f733131a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:57.845 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:57.688+0000 7f733131a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:57.845 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:57.689+0000 7f733131a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:58.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.460+0000 7f576ffff640 1 -- 172.21.15.84:0/4253666079 <== mon.0 v2:172.21.15.84:3300/0 9 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f577c0cdb10 con 0x7f5788102a80 2024-04-26T18:27:58.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.461+0000 7f576ffff640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f5770035480 0x7f5770035860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:58.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.461+0000 7f576ffff640 1 -- 172.21.15.84:0/4253666079 --> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f577c0a4860 con 0x7f5770035480 2024-04-26T18:27:58.466 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.466+0000 7f5786d9d640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f5770035480 0x7f5770035860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:58.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.469+0000 7f5786d9d640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f5770035480 0x7f5770035860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5778009230 tx=0x7f57780314a0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:58.476 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: Active manager daemon x restarted 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: Activating manager daemon x 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: mgrmap e21: x(active, starting, since 0.0158814s), standbys: y, z 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:27:58.477 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:27:58.478 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:27:58.479 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:27:58.479 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:27:58.479 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:27:58.479 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:27:58.479 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:27:58.479 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:27:58.479 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:27:58.479 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:27:58.479 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:27:58.479 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: Manager daemon x is now available 2024-04-26T18:27:58.479 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:27:58.479 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:27:58.479 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:27:58.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.488+0000 7f576ffff640 1 -- 172.21.15.84:0/4253666079 <== mgr.15240 v2:172.21.15.84:6800/3453983695 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f577c0a4860 con 0x7f5770035480 2024-04-26T18:27:58.488 INFO:teuthology.orchestra.run.smithi084.stdout:{} 2024-04-26T18:27:58.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.490+0000 7f578d0b0640 1 -- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f5770035480 msgr2=0x7f5770035860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:58.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.490+0000 7f578d0b0640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f5770035480 0x7f5770035860 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f5778009230 tx=0x7f57780314a0 comp rx=0 tx=0).stop 2024-04-26T18:27:58.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.490+0000 7f578d0b0640 1 -- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5788102a80 msgr2=0x7f5788103e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:58.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.490+0000 7f578d0b0640 1 --2- 172.21.15.84:0/4253666079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5788102a80 0x7f5788103e60 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f577c00d950 tx=0x7f577c00de10 comp rx=0 tx=0).stop 2024-04-26T18:27:58.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.490+0000 7f578759e640 1 -- 172.21.15.84:0/4253666079 reap_dead start 2024-04-26T18:27:58.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.490+0000 7f578d0b0640 1 -- 172.21.15.84:0/4253666079 shutdown_connections 2024-04-26T18:27:58.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.490+0000 7f578d0b0640 1 -- 172.21.15.84:0/4253666079 >> 172.21.15.84:0/4253666079 conn(0x7f57880fce00 msgr2=0x7f57880fe8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:58.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.490+0000 7f578d0b0640 1 -- 172.21.15.84:0/4253666079 shutdown_connections 2024-04-26T18:27:58.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.490+0000 7f578d0b0640 1 -- 172.21.15.84:0/4253666079 wait complete. 2024-04-26T18:27:58.501 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-26T18:27:58.513 INFO:teuthology.run_tasks:Running task exec... 2024-04-26T18:27:58.555 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T18:27:58.555 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi084.front.sepia.ceph.com 2024-04-26T18:27:58.555 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-26T18:27:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-26T18:27:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: Active manager daemon x restarted 2024-04-26T18:27:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: Activating manager daemon x 2024-04-26T18:27:58.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T18:27:58.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: mgrmap e21: x(active, starting, since 0.0158814s), standbys: y, z 2024-04-26T18:27:58.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:27:58.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:27:58.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:27:58.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:27:58.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:27:58.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:27:58.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:27:58.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:27:58.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:27:58.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:27:58.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:27:58.628 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:27:58.629 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:27:58.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:27:58.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:27:58.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:27:58.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:27:58.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:27:58.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: Manager daemon x is now available 2024-04-26T18:27:58.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:27:58.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:27:58.630 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:58 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:27:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.660+0000 7fd6237af640 1 -- 172.21.15.84:0/459616261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd61c1086f0 msgr2=0x7fd61c10aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.660+0000 7fd6237af640 1 --2- 172.21.15.84:0/459616261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd61c1086f0 0x7fd61c10aae0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd614009ca0 tx=0x7fd61402a8c0 comp rx=0 tx=0).stop 2024-04-26T18:27:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.660+0000 7fd6237af640 1 -- 172.21.15.84:0/459616261 shutdown_connections 2024-04-26T18:27:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.660+0000 7fd6237af640 1 --2- 172.21.15.84:0/459616261 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd61c1086f0 0x7fd61c10aae0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:58.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.660+0000 7fd6237af640 1 --2- 172.21.15.84:0/459616261 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd61c105dc0 0x7fd61c1081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.660+0000 7fd6237af640 1 --2- 172.21.15.84:0/459616261 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c103490 0x7fd61c105880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.660+0000 7fd6237af640 1 -- 172.21.15.84:0/459616261 >> 172.21.15.84:0/459616261 conn(0x7fd61c0fce00 msgr2=0x7fd61c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.660+0000 7fd6237af640 1 -- 172.21.15.84:0/459616261 shutdown_connections 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.660+0000 7fd6237af640 1 -- 172.21.15.84:0/459616261 wait complete. 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd6237af640 1 Processor -- start 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd6237af640 1 -- start start 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd6237af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c103490 0x7fd61c19ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd6237af640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd61c105dc0 0x7fd61c19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd6237af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd61c1086f0 0x7fd61c1a22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd6237af640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c10e4e0 con 0x7fd61c105dc0 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd6237af640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c10e360 con 0x7fd61c1086f0 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd6237af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd61c10e660 con 0x7fd61c103490 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd620d4a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd61c105dc0 0x7fd61c19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:58.663 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd620d4a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd61c105dc0 0x7fd61c19b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43238/0 (socket says 172.21.15.84:43238) 2024-04-26T18:27:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd620d4a640 1 -- 172.21.15.84:0/3243973455 learned_addr learned my addr 172.21.15.84:0/3243973455 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd621d4c640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd61c1086f0 0x7fd61c1a22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd62154b640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c103490 0x7fd61c19ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd620d4a640 1 -- 172.21.15.84:0/3243973455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c103490 msgr2=0x7fd61c19ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd620d4a640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c103490 0x7fd61c19ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd620d4a640 1 -- 172.21.15.84:0/3243973455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd61c1086f0 msgr2=0x7fd61c1a22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd620d4a640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd61c1086f0 0x7fd61c1a22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd620d4a640 1 -- 172.21.15.84:0/3243973455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd61c1a2960 con 0x7fd61c105dc0 2024-04-26T18:27:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.661+0000 7fd621d4c640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd61c1086f0 0x7fd61c1a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.662+0000 7fd620d4a640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd61c105dc0 0x7fd61c19b290 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd61800e9a0 tx=0x7fd61800ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.662+0000 7fd6127fc640 1 -- 172.21.15.84:0/3243973455 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd61800cd80 con 0x7fd61c105dc0 2024-04-26T18:27:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.662+0000 7fd6237af640 1 -- 172.21.15.84:0/3243973455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd61c1a2c50 con 0x7fd61c105dc0 2024-04-26T18:27:58.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.662+0000 7fd6127fc640 1 -- 172.21.15.84:0/3243973455 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6180035f0 con 0x7fd61c105dc0 2024-04-26T18:27:58.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.662+0000 7fd6127fc640 1 -- 172.21.15.84:0/3243973455 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618010690 con 0x7fd61c105dc0 2024-04-26T18:27:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.662+0000 7fd6237af640 1 -- 172.21.15.84:0/3243973455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd61c1a31a0 con 0x7fd61c105dc0 2024-04-26T18:27:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.663+0000 7fd6237af640 1 -- 172.21.15.84:0/3243973455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd61c10a1e0 con 0x7fd61c105dc0 2024-04-26T18:27:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.664+0000 7fd6127fc640 1 -- 172.21.15.84:0/3243973455 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fd618004000 con 0x7fd61c105dc0 2024-04-26T18:27:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.665+0000 7fd6127fc640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd6080a3a60 0x7fd6080a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.665+0000 7fd62154b640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd6080a3a60 0x7fd6080a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.665+0000 7fd6127fc640 1 -- 172.21.15.84:0/3243973455 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fd6180d0400 con 0x7fd61c105dc0 2024-04-26T18:27:58.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.665+0000 7fd62154b640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd6080a3a60 0x7fd6080a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd61c19c220 tx=0x7fd60c005f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:58.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.674+0000 7fd6127fc640 1 -- 172.21.15.84:0/3243973455 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd6180042f0 con 0x7fd61c105dc0 2024-04-26T18:27:58.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: mgrmap e20: x(active, since 10m), standbys: y, z 2024-04-26T18:27:58.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: Active manager daemon x restarted 2024-04-26T18:27:58.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: Activating manager daemon x 2024-04-26T18:27:58.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: osdmap e94: 12 total, 12 up, 12 in 2024-04-26T18:27:58.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: mgrmap e21: x(active, starting, since 0.0158814s), standbys: y, z 2024-04-26T18:27:58.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:27:58.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-26T18:27:58.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T18:27:58.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T18:27:58.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T18:27:58.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: Manager daemon x is now available 2024-04-26T18:27:58.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:27:58.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-26T18:27:58.692 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:58 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-26T18:27:58.783 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:58.475+0000 7f732b30e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:58.783 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:58.475+0000 7f732b30e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:58.783 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:58.475+0000 7f732b30e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:58.783 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:58.475+0000 7f732b30e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:58.783 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:27:58 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:27:58.475+0000 7f732b30e700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:27:58.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.887+0000 7fd6237af640 1 -- 172.21.15.84:0/3243973455 --> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fd61c062d70 con 0x7fd6080a3a60 2024-04-26T18:27:58.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.953+0000 7fd6127fc640 1 -- 172.21.15.84:0/3243973455 <== mgr.15240 v2:172.21.15.84:6800/3453983695 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd61c062d70 con 0x7fd6080a3a60 2024-04-26T18:27:58.953 INFO:teuthology.orchestra.run.smithi084.stdout:Schedule set for path / 2024-04-26T18:27:58.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.955+0000 7fd6237af640 1 -- 172.21.15.84:0/3243973455 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd6080a3a60 msgr2=0x7fd6080a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:58.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.955+0000 7fd6237af640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd6080a3a60 0x7fd6080a5f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd61c19c220 tx=0x7fd60c005f90 comp rx=0 tx=0).stop 2024-04-26T18:27:58.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.955+0000 7fd6237af640 1 -- 172.21.15.84:0/3243973455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd61c105dc0 msgr2=0x7fd61c19b290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:58.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.955+0000 7fd6237af640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd61c105dc0 0x7fd61c19b290 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fd61800e9a0 tx=0x7fd61800ee60 comp rx=0 tx=0).stop 2024-04-26T18:27:58.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.955+0000 7fd6237af640 1 -- 172.21.15.84:0/3243973455 shutdown_connections 2024-04-26T18:27:58.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.955+0000 7fd6237af640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd6080a3a60 0x7fd6080a5f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:58.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.955+0000 7fd6237af640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd61c1086f0 0x7fd61c1a22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:58.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.955+0000 7fd6237af640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd61c105dc0 0x7fd61c19b290 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:58.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.955+0000 7fd6237af640 1 --2- 172.21.15.84:0/3243973455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd61c103490 0x7fd61c19ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:58.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.955+0000 7fd6237af640 1 -- 172.21.15.84:0/3243973455 >> 172.21.15.84:0/3243973455 conn(0x7fd61c0fce00 msgr2=0x7fd61c102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:58.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.955+0000 7fd6237af640 1 -- 172.21.15.84:0/3243973455 shutdown_connections 2024-04-26T18:27:58.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:58.955+0000 7fd6237af640 1 -- 172.21.15.84:0/3243973455 wait complete. 2024-04-26T18:27:58.966 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-26T18:27:59.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 -- 172.21.15.84:0/2346228102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e0101890 msgr2=0x7f67e0101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:59.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 --2- 172.21.15.84:0/2346228102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e0101890 0x7f67e0101cb0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009900 tx=0x7f67dc02a410 comp rx=0 tx=0).stop 2024-04-26T18:27:59.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 -- 172.21.15.84:0/2346228102 shutdown_connections 2024-04-26T18:27:59.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 --2- 172.21.15.84:0/2346228102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e0103440 0x7f67e0109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 --2- 172.21.15.84:0/2346228102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67e0102a80 0x7f67e0102f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 --2- 172.21.15.84:0/2346228102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e0101890 0x7f67e0101cb0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 -- 172.21.15.84:0/2346228102 >> 172.21.15.84:0/2346228102 conn(0x7f67e00fce00 msgr2=0x7f67e00ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:59.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 -- 172.21.15.84:0/2346228102 shutdown_connections 2024-04-26T18:27:59.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 -- 172.21.15.84:0/2346228102 wait complete. 2024-04-26T18:27:59.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 Processor -- start 2024-04-26T18:27:59.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 -- start start 2024-04-26T18:27:59.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67e0101890 0x7f67e0198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e0102a80 0x7f67e01990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e0103440 0x7f67e01a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f67e010d390 con 0x7f67e0102a80 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f67e010d210 con 0x7f67e0103440 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.072+0000 7f67e811c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f67e010d510 con 0x7f67e0101890 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e56b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e0102a80 0x7f67e01990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e56b7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e0102a80 0x7f67e01990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43254/0 (socket says 172.21.15.84:43254) 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e56b7640 1 -- 172.21.15.84:0/2683986528 learned_addr learned my addr 172.21.15.84:0/2683986528 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e5eb8640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67e0101890 0x7f67e0198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e66b9640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e0103440 0x7f67e01a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e56b7640 1 -- 172.21.15.84:0/2683986528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67e0101890 msgr2=0x7f67e0198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e56b7640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67e0101890 0x7f67e0198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e56b7640 1 -- 172.21.15.84:0/2683986528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e0103440 msgr2=0x7f67e01a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:59.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e56b7640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e0103440 0x7f67e01a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e56b7640 1 -- 172.21.15.84:0/2683986528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e01a0760 con 0x7f67e0102a80 2024-04-26T18:27:59.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e66b9640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e0103440 0x7f67e01a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:59.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e5eb8640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67e0101890 0x7f67e0198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:27:59.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e56b7640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e0102a80 0x7f67e01990d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f67d000e9f0 tx=0x7f67d000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:59.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67d6ffd640 1 -- 172.21.15.84:0/2683986528 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d000cd80 con 0x7f67e0102a80 2024-04-26T18:27:59.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e811c640 1 -- 172.21.15.84:0/2683986528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e01a0a50 con 0x7f67e0102a80 2024-04-26T18:27:59.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67d6ffd640 1 -- 172.21.15.84:0/2683986528 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67d00108e0 con 0x7f67e0102a80 2024-04-26T18:27:59.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67d6ffd640 1 -- 172.21.15.84:0/2683986528 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d0018980 con 0x7f67e0102a80 2024-04-26T18:27:59.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.073+0000 7f67e811c640 1 -- 172.21.15.84:0/2683986528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e01a0fa0 con 0x7f67e0102a80 2024-04-26T18:27:59.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.074+0000 7f67e811c640 1 -- 172.21.15.84:0/2683986528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e01a1170 con 0x7f67e0102a80 2024-04-26T18:27:59.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.075+0000 7f67d6ffd640 1 -- 172.21.15.84:0/2683986528 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f67d0010430 con 0x7f67e0102a80 2024-04-26T18:27:59.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.076+0000 7f67d6ffd640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f67cc0a3a60 0x7f67cc0a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:59.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.076+0000 7f67e5eb8640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f67cc0a3a60 0x7f67cc0a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:59.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.076+0000 7f67d6ffd640 1 -- 172.21.15.84:0/2683986528 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f67d00d0a60 con 0x7f67e0102a80 2024-04-26T18:27:59.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.076+0000 7f67e5eb8640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f67cc0a3a60 0x7f67cc0a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f67dc006010 tx=0x7f67dc000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:59.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.080+0000 7f67d6ffd640 1 -- 172.21.15.84:0/2683986528 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67d009d600 con 0x7f67e0102a80 2024-04-26T18:27:59.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.325+0000 7f67e811c640 1 -- 172.21.15.84:0/2683986528 --> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f67e006a6f0 con 0x7f67cc0a3a60 2024-04-26T18:27:59.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.390+0000 7f67d6ffd640 1 -- 172.21.15.84:0/2683986528 <== mgr.15240 v2:172.21.15.84:6800/3453983695 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f67e006a6f0 con 0x7f67cc0a3a60 2024-04-26T18:27:59.390 INFO:teuthology.orchestra.run.smithi084.stdout:Retention added to path / 2024-04-26T18:27:59.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.392+0000 7f67e811c640 1 -- 172.21.15.84:0/2683986528 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f67cc0a3a60 msgr2=0x7f67cc0a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:59.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.392+0000 7f67e811c640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f67cc0a3a60 0x7f67cc0a5f20 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f67dc006010 tx=0x7f67dc000bc0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.392+0000 7f67e811c640 1 -- 172.21.15.84:0/2683986528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e0102a80 msgr2=0x7f67e01990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:59.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.392+0000 7f67e811c640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e0102a80 0x7f67e01990d0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f67d000e9f0 tx=0x7f67d000eeb0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.392+0000 7f67e811c640 1 -- 172.21.15.84:0/2683986528 shutdown_connections 2024-04-26T18:27:59.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.392+0000 7f67e811c640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f67cc0a3a60 0x7f67cc0a5f20 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.392+0000 7f67e811c640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f67e0103440 0x7f67e01a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.392+0000 7f67e811c640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f67e0102a80 0x7f67e01990d0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.392+0000 7f67e811c640 1 --2- 172.21.15.84:0/2683986528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f67e0101890 0x7f67e0198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.392+0000 7f67e811c640 1 -- 172.21.15.84:0/2683986528 >> 172.21.15.84:0/2683986528 conn(0x7f67e00fce00 msgr2=0x7f67e0069fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:59.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.392+0000 7f67e811c640 1 -- 172.21.15.84:0/2683986528 shutdown_connections 2024-04-26T18:27:59.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.392+0000 7f67e811c640 1 -- 172.21.15.84:0/2683986528 wait complete. 2024-04-26T18:27:59.407 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T18:27:59.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.523+0000 7ff92f1f7640 1 -- 172.21.15.84:0/2517509991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff928105de0 msgr2=0x7ff9281081d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:59.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.523+0000 7ff92f1f7640 1 --2- 172.21.15.84:0/2517509991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff928105de0 0x7ff9281081d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7ff914009900 tx=0x7ff91402a410 comp rx=0 tx=0).stop 2024-04-26T18:27:59.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.523+0000 7ff92f1f7640 1 -- 172.21.15.84:0/2517509991 shutdown_connections 2024-04-26T18:27:59.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.523+0000 7ff92f1f7640 1 --2- 172.21.15.84:0/2517509991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff928108710 0x7ff92810ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.523+0000 7ff92f1f7640 1 --2- 172.21.15.84:0/2517509991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff928105de0 0x7ff9281081d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.523+0000 7ff92f1f7640 1 --2- 172.21.15.84:0/2517509991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9281034b0 0x7ff9281058a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.523+0000 7ff92f1f7640 1 -- 172.21.15.84:0/2517509991 >> 172.21.15.84:0/2517509991 conn(0x7ff9280fce40 msgr2=0x7ff9280ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:59.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.523+0000 7ff92f1f7640 1 -- 172.21.15.84:0/2517509991 shutdown_connections 2024-04-26T18:27:59.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.523+0000 7ff92f1f7640 1 -- 172.21.15.84:0/2517509991 wait complete. 2024-04-26T18:27:59.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.524+0000 7ff92f1f7640 1 Processor -- start 2024-04-26T18:27:59.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.524+0000 7ff92f1f7640 1 -- start start 2024-04-26T18:27:59.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.524+0000 7ff92f1f7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9281034b0 0x7ff928102e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:59.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.524+0000 7ff92f1f7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff928105de0 0x7ff928101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:59.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.524+0000 7ff92f1f7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff928108710 0x7ff9281019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:59.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.524+0000 7ff92f1f7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff92810e4f0 con 0x7ff928105de0 2024-04-26T18:27:59.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.524+0000 7ff92f1f7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff92810e370 con 0x7ff9281034b0 2024-04-26T18:27:59.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.524+0000 7ff92f1f7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff92810e670 con 0x7ff928108710 2024-04-26T18:27:59.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff92cf93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9281034b0 0x7ff928102e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:59.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff927fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff928105de0 0x7ff928101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:59.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff92cf93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9281034b0 0x7ff928102e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:49224/0 (socket says 172.21.15.84:49224) 2024-04-26T18:27:59.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff92cf93640 1 -- 172.21.15.84:0/741178790 learned_addr learned my addr 172.21.15.84:0/741178790 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:59.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff92d794640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff928108710 0x7ff9281019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:59.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff92cf93640 1 -- 172.21.15.84:0/741178790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff928108710 msgr2=0x7ff9281019a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:59.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff92cf93640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff928108710 0x7ff9281019a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff92cf93640 1 -- 172.21.15.84:0/741178790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff928105de0 msgr2=0x7ff928101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:59.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff92cf93640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff928105de0 0x7ff928101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff92cf93640 1 -- 172.21.15.84:0/741178790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff928102230 con 0x7ff9281034b0 2024-04-26T18:27:59.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff927fff640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff928105de0 0x7ff928101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:27:59.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff92cf93640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9281034b0 0x7ff928102e20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff91c00e9f0 tx=0x7ff91c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:59.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff925ffb640 1 -- 172.21.15.84:0/741178790 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c00cd80 con 0x7ff9281034b0 2024-04-26T18:27:59.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff92f1f7640 1 -- 172.21.15.84:0/741178790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9281a2c80 con 0x7ff9281034b0 2024-04-26T18:27:59.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff925ffb640 1 -- 172.21.15.84:0/741178790 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff91c0054d0 con 0x7ff9281034b0 2024-04-26T18:27:59.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.525+0000 7ff925ffb640 1 -- 172.21.15.84:0/741178790 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff91c018630 con 0x7ff9281034b0 2024-04-26T18:27:59.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.526+0000 7ff92f1f7640 1 -- 172.21.15.84:0/741178790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9281a31a0 con 0x7ff9281034b0 2024-04-26T18:27:59.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.527+0000 7ff92f1f7640 1 -- 172.21.15.84:0/741178790 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9280506c0 con 0x7ff9281034b0 2024-04-26T18:27:59.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.528+0000 7ff925ffb640 1 -- 172.21.15.84:0/741178790 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7ff91c004e70 con 0x7ff9281034b0 2024-04-26T18:27:59.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.528+0000 7ff925ffb640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff9180a3a60 0x7ff9180a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:59.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.529+0000 7ff927fff640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff9180a3a60 0x7ff9180a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:59.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.529+0000 7ff925ffb640 1 -- 172.21.15.84:0/741178790 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7ff91c0d06f0 con 0x7ff9281034b0 2024-04-26T18:27:59.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.529+0000 7ff927fff640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff9180a3a60 0x7ff9180a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff914009230 tx=0x7ff9140314a0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:59.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.535+0000 7ff925ffb640 1 -- 172.21.15.84:0/741178790 <== mon.2 v2:172.21.15.103:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff91c09d250 con 0x7ff9281034b0 2024-04-26T18:27:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:59 smithi103 ceph-mon[32727]: [26/Apr/2024:18:27:57] ENGINE Bus STARTING 2024-04-26T18:27:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:59 smithi103 ceph-mon[32727]: [26/Apr/2024:18:27:58] ENGINE Serving on https://172.21.15.84:7150 2024-04-26T18:27:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:59 smithi103 ceph-mon[32727]: [26/Apr/2024:18:27:58] ENGINE Serving on http://172.21.15.84:8765 2024-04-26T18:27:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:59 smithi103 ceph-mon[32727]: [26/Apr/2024:18:27:58] ENGINE Bus STARTED 2024-04-26T18:27:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:59 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:27:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:59 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:27:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:27:59 smithi103 ceph-mon[32727]: mgrmap e22: x(active, since 1.02197s), standbys: y, z 2024-04-26T18:27:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:59 smithi174 ceph-mon[32705]: [26/Apr/2024:18:27:57] ENGINE Bus STARTING 2024-04-26T18:27:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:59 smithi174 ceph-mon[32705]: [26/Apr/2024:18:27:58] ENGINE Serving on https://172.21.15.84:7150 2024-04-26T18:27:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:59 smithi174 ceph-mon[32705]: [26/Apr/2024:18:27:58] ENGINE Serving on http://172.21.15.84:8765 2024-04-26T18:27:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:59 smithi174 ceph-mon[32705]: [26/Apr/2024:18:27:58] ENGINE Bus STARTED 2024-04-26T18:27:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:59 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:27:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:59 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:27:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:27:59 smithi174 ceph-mon[32705]: mgrmap e22: x(active, since 1.02197s), standbys: y, z 2024-04-26T18:27:59.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.824+0000 7ff92f1f7640 1 -- 172.21.15.84:0/741178790 --> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ff928062e30 con 0x7ff9180a3a60 2024-04-26T18:27:59.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.832+0000 7ff925ffb640 1 -- 172.21.15.84:0/741178790 <== mgr.15240 v2:172.21.15.84:6800/3453983695 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7ff928062e30 con 0x7ff9180a3a60 2024-04-26T18:27:59.832 INFO:teuthology.orchestra.run.smithi084.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T18:27:58", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-26T18:27:59.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.834+0000 7ff92f1f7640 1 -- 172.21.15.84:0/741178790 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff9180a3a60 msgr2=0x7ff9180a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:59.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.834+0000 7ff92f1f7640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff9180a3a60 0x7ff9180a5f20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7ff914009230 tx=0x7ff9140314a0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.834+0000 7ff92f1f7640 1 -- 172.21.15.84:0/741178790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9281034b0 msgr2=0x7ff928102e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:59.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.834+0000 7ff92f1f7640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9281034b0 0x7ff928102e20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff91c00e9f0 tx=0x7ff91c00edf0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.834+0000 7ff92f1f7640 1 -- 172.21.15.84:0/741178790 shutdown_connections 2024-04-26T18:27:59.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.834+0000 7ff92f1f7640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff9180a3a60 0x7ff9180a5f20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.834+0000 7ff92f1f7640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff928108710 0x7ff9281019a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.834+0000 7ff92f1f7640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff928105de0 0x7ff928101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.834+0000 7ff92f1f7640 1 --2- 172.21.15.84:0/741178790 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9281034b0 0x7ff928102e20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.834+0000 7ff92f1f7640 1 -- 172.21.15.84:0/741178790 >> 172.21.15.84:0/741178790 conn(0x7ff9280fce40 msgr2=0x7ff928103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:59.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.835+0000 7ff92f1f7640 1 -- 172.21.15.84:0/741178790 shutdown_connections 2024-04-26T18:27:59.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.835+0000 7ff92f1f7640 1 -- 172.21.15.84:0/741178790 wait complete. 2024-04-26T18:27:59.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:59 smithi084 ceph-mon[29141]: [26/Apr/2024:18:27:57] ENGINE Bus STARTING 2024-04-26T18:27:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:59 smithi084 ceph-mon[29141]: [26/Apr/2024:18:27:58] ENGINE Serving on https://172.21.15.84:7150 2024-04-26T18:27:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:59 smithi084 ceph-mon[29141]: [26/Apr/2024:18:27:58] ENGINE Serving on http://172.21.15.84:8765 2024-04-26T18:27:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:59 smithi084 ceph-mon[29141]: [26/Apr/2024:18:27:58] ENGINE Bus STARTED 2024-04-26T18:27:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:59 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:27:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:59 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:27:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:27:59 smithi084 ceph-mon[29141]: mgrmap e22: x(active, since 1.02197s), standbys: y, z 2024-04-26T18:27:59.846 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T18:27:59.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.948+0000 7f4672304640 1 -- 172.21.15.84:0/3832034901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c102a80 msgr2=0x7f466c102f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:59.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.948+0000 7f4672304640 1 --2- 172.21.15.84:0/3832034901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c102a80 0x7f466c102f00 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f4660009900 tx=0x7f466002a410 comp rx=0 tx=0).stop 2024-04-26T18:27:59.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.948+0000 7f4672304640 1 -- 172.21.15.84:0/3832034901 shutdown_connections 2024-04-26T18:27:59.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.948+0000 7f4672304640 1 --2- 172.21.15.84:0/3832034901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f466c103440 0x7f466c109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.948+0000 7f4672304640 1 --2- 172.21.15.84:0/3832034901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c102a80 0x7f466c102f00 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.948+0000 7f4672304640 1 --2- 172.21.15.84:0/3832034901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f466c101890 0x7f466c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.948+0000 7f4672304640 1 -- 172.21.15.84:0/3832034901 >> 172.21.15.84:0/3832034901 conn(0x7f466c0fce00 msgr2=0x7f466c0ff240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:27:59.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.948+0000 7f4672304640 1 -- 172.21.15.84:0/3832034901 shutdown_connections 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.948+0000 7f4672304640 1 -- 172.21.15.84:0/3832034901 wait complete. 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.948+0000 7f4672304640 1 Processor -- start 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.948+0000 7f4672304640 1 -- start start 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f4672304640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f466c101890 0x7f466c06ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f4672304640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c102a80 0x7f466c19d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f4672304640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f466c103440 0x7f466c19feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f4672304640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f466c10d3a0 con 0x7f466c102a80 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f4672304640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f466c10d220 con 0x7f466c103440 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f4672304640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f466c10d520 con 0x7f466c101890 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f466b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c102a80 0x7f466c19d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f466b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c102a80 0x7f466c19d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43294/0 (socket says 172.21.15.84:43294) 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f466b7fe640 1 -- 172.21.15.84:0/2025146016 learned_addr learned my addr 172.21.15.84:0/2025146016 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f466bfff640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f466c101890 0x7f466c06ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f46708a1640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f466c103440 0x7f466c19feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:59.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f466b7fe640 1 -- 172.21.15.84:0/2025146016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f466c101890 msgr2=0x7f466c06ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f466b7fe640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f466c101890 0x7f466c06ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f466b7fe640 1 -- 172.21.15.84:0/2025146016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f466c103440 msgr2=0x7f466c19feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:27:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f466b7fe640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f466c103440 0x7f466c19feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:27:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f466b7fe640 1 -- 172.21.15.84:0/2025146016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f466c1a07c0 con 0x7f466c102a80 2024-04-26T18:27:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f466b7fe640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c102a80 0x7f466c19d890 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4660005940 tx=0x7f4660004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f46697fa640 1 -- 172.21.15.84:0/2025146016 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f466003f030 con 0x7f466c102a80 2024-04-26T18:27:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f46697fa640 1 -- 172.21.15.84:0/2025146016 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4660003cb0 con 0x7f466c102a80 2024-04-26T18:27:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f4672304640 1 -- 172.21.15.84:0/2025146016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f466c1a09f0 con 0x7f466c102a80 2024-04-26T18:27:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.949+0000 7f46697fa640 1 -- 172.21.15.84:0/2025146016 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4660035070 con 0x7f466c102a80 2024-04-26T18:27:59.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.950+0000 7f4672304640 1 -- 172.21.15.84:0/2025146016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f466c1a0ee0 con 0x7f466c102a80 2024-04-26T18:27:59.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.950+0000 7f4672304640 1 -- 172.21.15.84:0/2025146016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f466c108bd0 con 0x7f466c102a80 2024-04-26T18:27:59.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.952+0000 7f46697fa640 1 -- 172.21.15.84:0/2025146016 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f4660042030 con 0x7f466c102a80 2024-04-26T18:27:59.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.952+0000 7f46697fa640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46540a3a60 0x7f46540a5f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:27:59.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.952+0000 7f466bfff640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46540a3a60 0x7f46540a5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:27:59.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.952+0000 7f46697fa640 1 -- 172.21.15.84:0/2025146016 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f46600f0990 con 0x7f466c102a80 2024-04-26T18:27:59.955 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.953+0000 7f466bfff640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46540a3a60 0x7f46540a5f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f466c1028e0 tx=0x7f465c00b390 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:27:59.956 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:27:59.957+0000 7f46697fa640 1 -- 172.21.15.84:0/2025146016 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4660003e60 con 0x7f466c102a80 2024-04-26T18:28:00.167 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.167+0000 7f4672304640 1 -- 172.21.15.84:0/2025146016 --> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f466c062d70 con 0x7f46540a3a60 2024-04-26T18:28:00.175 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.175+0000 7f46697fa640 1 -- 172.21.15.84:0/2025146016 <== mgr.15240 v2:172.21.15.84:6800/3453983695 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f466c062d70 con 0x7f46540a3a60 2024-04-26T18:28:00.175 INFO:teuthology.orchestra.run.smithi084.stdout:/ 1m 6m3h 2024-04-26T18:28:00.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.178+0000 7f4672304640 1 -- 172.21.15.84:0/2025146016 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46540a3a60 msgr2=0x7f46540a5f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:00.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.178+0000 7f4672304640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46540a3a60 0x7f46540a5f20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f466c1028e0 tx=0x7f465c00b390 comp rx=0 tx=0).stop 2024-04-26T18:28:00.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.178+0000 7f4672304640 1 -- 172.21.15.84:0/2025146016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c102a80 msgr2=0x7f466c19d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:00.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.178+0000 7f4672304640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c102a80 0x7f466c19d890 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f4660005940 tx=0x7f4660004430 comp rx=0 tx=0).stop 2024-04-26T18:28:00.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.178+0000 7f4672304640 1 -- 172.21.15.84:0/2025146016 shutdown_connections 2024-04-26T18:28:00.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.178+0000 7f4672304640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46540a3a60 0x7f46540a5f20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:00.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.178+0000 7f4672304640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f466c103440 0x7f466c19feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:00.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.178+0000 7f4672304640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f466c102a80 0x7f466c19d890 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:00.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.178+0000 7f4672304640 1 --2- 172.21.15.84:0/2025146016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f466c101890 0x7f466c06ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:00.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.178+0000 7f4672304640 1 -- 172.21.15.84:0/2025146016 >> 172.21.15.84:0/2025146016 conn(0x7f466c0fce00 msgr2=0x7f466c105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:00.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.178+0000 7f4672304640 1 -- 172.21.15.84:0/2025146016 shutdown_connections 2024-04-26T18:28:00.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:00.178+0000 7f4672304640 1 -- 172.21.15.84:0/2025146016 wait complete. 2024-04-26T18:28:00.190 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-26T18:28:00.215 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-26T18:28:00.228 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-26T18:28:00.238 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi084.front.sepia.ceph.com 2024-04-26T18:28:00.238 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-26T18:28:00.257 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-26T18:28:00.310 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T18:28:00.310 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:28:00.587 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:00.594 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:28:00.399+0000 7f730a68d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:28:00.594 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:28:00.399+0000 7f730a68d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:28:00.594 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:28:00.399+0000 7f730a68d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:28:00.594 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:28:00.399+0000 7f730a68d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:28:00.595 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:28:00.399+0000 7f730a68d700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-26T18:28:00.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-mon[29141]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:00.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-mon[29141]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:28:00.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-mon[29141]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:00.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-mon[29141]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:00.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:00 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:00 smithi174 ceph-mon[32705]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:00.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:00 smithi174 ceph-mon[32705]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:28:00.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:00 smithi174 ceph-mon[32705]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:00.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:00 smithi174 ceph-mon[32705]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:00.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:00 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:00 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:00 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:00 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:00 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:00 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:00 smithi103 ceph-mon[32727]: from='client.15270 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:00 smithi103 ceph-mon[32727]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:28:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:00 smithi103 ceph-mon[32727]: from='client.15318 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:00 smithi103 ceph-mon[32727]: from='client.15324 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:00 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:00 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:00 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:00 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:00 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:00.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:00 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:28:01.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:01.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:01.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-26T18:28:01.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:28:01.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:01 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:28:01.817 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.818 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:28:01.819 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:01 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:28:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:28:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='client.24875 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:28:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-04-26T18:28:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:28:01.879 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:28:01.879 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:01 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:28:02.449 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi103 to 176.2M 2024-04-26T18:28:02.449 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: Unable to set osd_memory_target on smithi103 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:28:02.449 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-04-26T18:28:02.449 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: Unable to set osd_memory_target on smithi084 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-26T18:28:02.449 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi174 to 176.2M 2024-04-26T18:28:02.449 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: Unable to set osd_memory_target on smithi174 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:28:02.449 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: Updating smithi084:/etc/ceph/ceph.conf 2024-04-26T18:28:02.450 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-26T18:28:02.450 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: Updating smithi174:/etc/ceph/ceph.conf 2024-04-26T18:28:02.450 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: Updating smithi084:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:28:02.450 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: Updating smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:28:02.450 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: Updating smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:28:02.450 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.450 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.450 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.450 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.450 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.450 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.451 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.451 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:28:02.451 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:28:02.451 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:28:02.451 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:02 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.548+0000 7f2a02e8f700 1 -- 172.21.15.84:0/3988741855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29fc102320 msgr2=0x7f29fc1027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.548+0000 7f2a02e8f700 1 --2- 172.21.15.84:0/3988741855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29fc102320 0x7f29fc1027a0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f29ec009990 tx=0x7f29ec009ca0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.548+0000 7f2a02e8f700 1 -- 172.21.15.84:0/3988741855 shutdown_connections 2024-04-26T18:28:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.548+0000 7f2a02e8f700 1 --2- 172.21.15.84:0/3988741855 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29fc102ce0 0x7f29fc10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.548+0000 7f2a02e8f700 1 --2- 172.21.15.84:0/3988741855 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc108390 0x7f29fc108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.548+0000 7f2a02e8f700 1 --2- 172.21.15.84:0/3988741855 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29fc102320 0x7f29fc1027a0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.548+0000 7f2a02e8f700 1 -- 172.21.15.84:0/3988741855 >> 172.21.15.84:0/3988741855 conn(0x7f29fc0792d0 msgr2=0x7f29fc0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.548+0000 7f2a02e8f700 1 -- 172.21.15.84:0/3988741855 shutdown_connections 2024-04-26T18:28:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.549+0000 7f2a02e8f700 1 -- 172.21.15.84:0/3988741855 wait complete. 2024-04-26T18:28:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.549+0000 7f2a02e8f700 1 Processor -- start 2024-04-26T18:28:02.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.549+0000 7f2a02e8f700 1 -- start start 2024-04-26T18:28:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.549+0000 7f2a02e8f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29fc102320 0x7f29fc104370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.549+0000 7f2a02e8f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc102ce0 0x7f29fc1048b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.549+0000 7f2a02e8f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29fc108390 0x7f29fc104df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.549+0000 7f2a02e8f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc1124e0 con 0x7f29fc102320 2024-04-26T18:28:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.549+0000 7f2a02e8f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc112360 con 0x7f29fc102ce0 2024-04-26T18:28:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.549+0000 7f2a02e8f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc112660 con 0x7f29fc108390 2024-04-26T18:28:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f2a00c2b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29fc102320 0x7f29fc104370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f2a00c2b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29fc102320 0x7f29fc104370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43326/0 (socket says 172.21.15.84:43326) 2024-04-26T18:28:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f2a00c2b700 1 -- 172.21.15.84:0/1519499964 learned_addr learned my addr 172.21.15.84:0/1519499964 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f2a0142c700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29fc108390 0x7f29fc104df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f29fbfff700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc102ce0 0x7f29fc1048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f2a00c2b700 1 -- 172.21.15.84:0/1519499964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29fc108390 msgr2=0x7f29fc104df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f2a00c2b700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29fc108390 0x7f29fc104df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f2a00c2b700 1 -- 172.21.15.84:0/1519499964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc102ce0 msgr2=0x7f29fc1048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f2a00c2b700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc102ce0 0x7f29fc1048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f2a00c2b700 1 -- 172.21.15.84:0/1519499964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29fc1a8050 con 0x7f29fc102320 2024-04-26T18:28:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f29fbfff700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc102ce0 0x7f29fc1048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f2a0142c700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29fc108390 0x7f29fc104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f2a00c2b700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29fc102320 0x7f29fc104370 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f29f4007f10 tx=0x7f29f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f29f9ffb700 1 -- 172.21.15.84:0/1519499964 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f4004db0 con 0x7f29fc102320 2024-04-26T18:28:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f29f9ffb700 1 -- 172.21.15.84:0/1519499964 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29f4003a00 con 0x7f29fc102320 2024-04-26T18:28:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f2a02e8f700 1 -- 172.21.15.84:0/1519499964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29fc1a8340 con 0x7f29fc102320 2024-04-26T18:28:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.550+0000 7f29f9ffb700 1 -- 172.21.15.84:0/1519499964 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f40045e0 con 0x7f29fc102320 2024-04-26T18:28:02.553 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.551+0000 7f2a02e8f700 1 -- 172.21.15.84:0/1519499964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29fc1a88a0 con 0x7f29fc102320 2024-04-26T18:28:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.552+0000 7f2a02e8f700 1 -- 172.21.15.84:0/1519499964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29fc06a110 con 0x7f29fc102320 2024-04-26T18:28:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.552+0000 7f29f9ffb700 1 -- 172.21.15.84:0/1519499964 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f29f401c020 con 0x7f29fc102320 2024-04-26T18:28:02.554 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.553+0000 7f29f9ffb700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f29e80a3fe0 0x7f29e80a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:02.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.553+0000 7f29fbfff700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f29e80a3fe0 0x7f29e80a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:02.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.553+0000 7f29f9ffb700 1 -- 172.21.15.84:0/1519499964 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f29f40d0200 con 0x7f29fc102320 2024-04-26T18:28:02.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.553+0000 7f29fbfff700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f29e80a3fe0 0x7f29e80a64a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00b2e0 tx=0x7f29ec01a040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:02.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.558+0000 7f29f9ffb700 1 -- 172.21.15.84:0/1519499964 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f29f4010070 con 0x7f29fc102320 2024-04-26T18:28:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi103 to 176.2M 2024-04-26T18:28:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: Unable to set osd_memory_target on smithi103 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:28:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-04-26T18:28:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: Unable to set osd_memory_target on smithi084 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-26T18:28:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi174 to 176.2M 2024-04-26T18:28:02.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: Unable to set osd_memory_target on smithi174 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:28:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: Updating smithi084:/etc/ceph/ceph.conf 2024-04-26T18:28:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-26T18:28:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: Updating smithi174:/etc/ceph/ceph.conf 2024-04-26T18:28:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: Updating smithi084:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:28:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: Updating smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:28:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: Updating smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:28:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:28:02.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:28:02.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:28:02.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:02 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.756+0000 7f2a02e8f700 1 -- 172.21.15.84:0/1519499964 --> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f29fc06b100 con 0x7f29e80a3fe0 2024-04-26T18:28:02.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.763+0000 7f29f9ffb700 1 -- 172.21.15.84:0/1519499964 <== mgr.15240 v2:172.21.15.84:6800/3453983695 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258113 (secure 0 0 0) 0x7f29fc06b100 con 0x7f29e80a3fe0 2024-04-26T18:28:02.767 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:28:02.769 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.766+0000 7f2a02e8f700 1 -- 172.21.15.84:0/1519499964 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f29e80a3fe0 msgr2=0x7f29e80a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.766+0000 7f2a02e8f700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f29e80a3fe0 0x7f29e80a64a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f29ec00b2e0 tx=0x7f29ec01a040 comp rx=0 tx=0).stop 2024-04-26T18:28:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.766+0000 7f2a02e8f700 1 -- 172.21.15.84:0/1519499964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29fc102320 msgr2=0x7f29fc104370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.766+0000 7f2a02e8f700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29fc102320 0x7f29fc104370 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f29f4007f10 tx=0x7f29f400d3b0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.766+0000 7f2a02e8f700 1 -- 172.21.15.84:0/1519499964 shutdown_connections 2024-04-26T18:28:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.766+0000 7f2a02e8f700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f29e80a3fe0 0x7f29e80a64a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.766+0000 7f2a02e8f700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f29fc102320 0x7f29fc104370 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.766+0000 7f2a02e8f700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f29fc108390 0x7f29fc104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.766+0000 7f2a02e8f700 1 --2- 172.21.15.84:0/1519499964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f29fc102ce0 0x7f29fc1048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.767+0000 7f2a02e8f700 1 -- 172.21.15.84:0/1519499964 >> 172.21.15.84:0/1519499964 conn(0x7f29fc0792d0 msgr2=0x7f29fc1065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.767+0000 7f2a02e8f700 1 -- 172.21.15.84:0/1519499964 shutdown_connections 2024-04-26T18:28:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:02.767+0000 7f2a02e8f700 1 -- 172.21.15.84:0/1519499964 wait complete. 2024-04-26T18:28:02.770 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-04-26T18:28:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi103 to 176.2M 2024-04-26T18:28:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: Unable to set osd_memory_target on smithi103 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:28:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-04-26T18:28:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: Unable to set osd_memory_target on smithi084 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-26T18:28:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi174 to 176.2M 2024-04-26T18:28:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: Unable to set osd_memory_target on smithi174 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:28:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: Updating smithi084:/etc/ceph/ceph.conf 2024-04-26T18:28:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: Updating smithi103:/etc/ceph/ceph.conf 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: Updating smithi174:/etc/ceph/ceph.conf 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: Updating smithi084:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: Updating smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: Updating smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:28:02.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:28:02.879 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:02 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:28:03.431 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":5,"stamp":"2024-04-26T18:28:01.465649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1195513,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2234,"num_read_kb":1291,"num_write":558,"num_write_kb":4408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182319,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":941,"ondisk_log_size":941,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370304,"kb_used_data":12780,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124571520,"statfs":{"total":1151940427776,"available":1151561236480,"internally_reserved":0,"allocated":13086720,"data_stored":9276087,"data_compressed":337142,"data_compressed_allocated":458752,"data_compressed_original":3424256,"omap_allocated":19143,"internal_metadata":365933881},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.990008"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984569+0000","last_change":"2024-04-26T18:24:20.951581+0000","last_active":"2024-04-26T18:27:57.984569+0000","last_peered":"2024-04-26T18:27:57.984569+0000","last_clean":"2024-04-26T18:27:57.984569+0000","last_became_active":"2024-04-26T18:24:20.951260+0000","last_became_peered":"2024-04-26T18:24:20.951260+0000","last_unstale":"2024-04-26T18:27:57.984569+0000","last_undegraded":"2024-04-26T18:27:57.984569+0000","last_fullsized":"2024-04-26T18:27:57.984569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:54:28.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697639+0000","last_change":"2024-04-26T18:24:20.952263+0000","last_active":"2024-04-26T18:27:57.697639+0000","last_peered":"2024-04-26T18:27:57.697639+0000","last_clean":"2024-04-26T18:27:57.697639+0000","last_became_active":"2024-04-26T18:24:20.951930+0000","last_became_peered":"2024-04-26T18:24:20.951930+0000","last_unstale":"2024-04-26T18:27:57.697639+0000","last_undegraded":"2024-04-26T18:27:57.697639+0000","last_fullsized":"2024-04-26T18:27:57.697639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:54.816572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697346+0000","last_change":"2024-04-26T18:24:20.952280+0000","last_active":"2024-04-26T18:27:57.697346+0000","last_peered":"2024-04-26T18:27:57.697346+0000","last_clean":"2024-04-26T18:27:57.697346+0000","last_became_active":"2024-04-26T18:24:20.951623+0000","last_became_peered":"2024-04-26T18:24:20.951623+0000","last_unstale":"2024-04-26T18:27:57.697346+0000","last_undegraded":"2024-04-26T18:27:57.697346+0000","last_fullsized":"2024-04-26T18:27:57.697346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:02.502261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.695472+0000","last_change":"2024-04-26T18:24:20.962274+0000","last_active":"2024-04-26T18:27:57.695472+0000","last_peered":"2024-04-26T18:27:57.695472+0000","last_clean":"2024-04-26T18:27:57.695472+0000","last_became_active":"2024-04-26T18:24:20.961920+0000","last_became_peered":"2024-04-26T18:24:20.961920+0000","last_unstale":"2024-04-26T18:27:57.695472+0000","last_undegraded":"2024-04-26T18:27:57.695472+0000","last_fullsized":"2024-04-26T18:27:57.695472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:49:46.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545742+0000","last_change":"2024-04-26T18:24:20.949096+0000","last_active":"2024-04-26T18:27:57.545742+0000","last_peered":"2024-04-26T18:27:57.545742+0000","last_clean":"2024-04-26T18:27:57.545742+0000","last_became_active":"2024-04-26T18:24:20.948737+0000","last_became_peered":"2024-04-26T18:24:20.948737+0000","last_unstale":"2024-04-26T18:27:57.545742+0000","last_undegraded":"2024-04-26T18:27:57.545742+0000","last_fullsized":"2024-04-26T18:27:57.545742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:12:40.371589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451728+0000","last_change":"2024-04-26T18:24:20.962670+0000","last_active":"2024-04-26T18:27:57.451728+0000","last_peered":"2024-04-26T18:27:57.451728+0000","last_clean":"2024-04-26T18:27:57.451728+0000","last_became_active":"2024-04-26T18:24:20.962309+0000","last_became_peered":"2024-04-26T18:24:20.962309+0000","last_unstale":"2024-04-26T18:27:57.451728+0000","last_undegraded":"2024-04-26T18:27:57.451728+0000","last_fullsized":"2024-04-26T18:27:57.451728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:40.315864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591196+0000","last_change":"2024-04-26T18:25:06.350579+0000","last_active":"2024-04-26T18:27:57.591196+0000","last_peered":"2024-04-26T18:27:57.591196+0000","last_clean":"2024-04-26T18:27:57.591196+0000","last_became_active":"2024-04-26T18:25:06.350104+0000","last_became_peered":"2024-04-26T18:25:06.350104+0000","last_unstale":"2024-04-26T18:27:57.591196+0000","last_undegraded":"2024-04-26T18:27:57.591196+0000","last_fullsized":"2024-04-26T18:27:57.591196+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:00.950883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452339+0000","last_change":"2024-04-26T18:24:20.961096+0000","last_active":"2024-04-26T18:27:57.452339+0000","last_peered":"2024-04-26T18:27:57.452339+0000","last_clean":"2024-04-26T18:27:57.452339+0000","last_became_active":"2024-04-26T18:24:20.960796+0000","last_became_peered":"2024-04-26T18:24:20.960796+0000","last_unstale":"2024-04-26T18:27:57.452339+0000","last_undegraded":"2024-04-26T18:27:57.452339+0000","last_fullsized":"2024-04-26T18:27:57.452339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:04.538836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589942+0000","last_change":"2024-04-26T18:24:20.959871+0000","last_active":"2024-04-26T18:27:57.589942+0000","last_peered":"2024-04-26T18:27:57.589942+0000","last_clean":"2024-04-26T18:27:57.589942+0000","last_became_active":"2024-04-26T18:24:20.959404+0000","last_became_peered":"2024-04-26T18:24:20.959404+0000","last_unstale":"2024-04-26T18:27:57.589942+0000","last_undegraded":"2024-04-26T18:27:57.589942+0000","last_fullsized":"2024-04-26T18:27:57.589942+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:43.053618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451941+0000","last_change":"2024-04-26T18:26:07.431543+0000","last_active":"2024-04-26T18:27:57.451941+0000","last_peered":"2024-04-26T18:27:57.451941+0000","last_clean":"2024-04-26T18:27:57.451941+0000","last_became_active":"2024-04-26T18:26:07.430976+0000","last_became_peered":"2024-04-26T18:26:07.430976+0000","last_unstale":"2024-04-26T18:27:57.451941+0000","last_undegraded":"2024-04-26T18:27:57.451941+0000","last_fullsized":"2024-04-26T18:27:57.451941+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:31.430010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591606+0000","last_change":"2024-04-26T18:24:20.952109+0000","last_active":"2024-04-26T18:27:57.591606+0000","last_peered":"2024-04-26T18:27:57.591606+0000","last_clean":"2024-04-26T18:27:57.591606+0000","last_became_active":"2024-04-26T18:24:20.951776+0000","last_became_peered":"2024-04-26T18:24:20.951776+0000","last_unstale":"2024-04-26T18:27:57.591606+0000","last_undegraded":"2024-04-26T18:27:57.591606+0000","last_fullsized":"2024-04-26T18:27:57.591606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:53.220246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.695451+0000","last_change":"2024-04-26T18:24:20.963965+0000","last_active":"2024-04-26T18:27:57.695451+0000","last_peered":"2024-04-26T18:27:57.695451+0000","last_clean":"2024-04-26T18:27:57.695451+0000","last_became_active":"2024-04-26T18:24:20.963648+0000","last_became_peered":"2024-04-26T18:24:20.963648+0000","last_unstale":"2024-04-26T18:27:57.695451+0000","last_undegraded":"2024-04-26T18:27:57.695451+0000","last_fullsized":"2024-04-26T18:27:57.695451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:04.080813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694857+0000","last_change":"2024-04-26T18:24:20.954186+0000","last_active":"2024-04-26T18:27:57.694857+0000","last_peered":"2024-04-26T18:27:57.694857+0000","last_clean":"2024-04-26T18:27:57.694857+0000","last_became_active":"2024-04-26T18:24:20.953431+0000","last_became_peered":"2024-04-26T18:24:20.953431+0000","last_unstale":"2024-04-26T18:27:57.694857+0000","last_undegraded":"2024-04-26T18:27:57.694857+0000","last_fullsized":"2024-04-26T18:27:57.694857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:21.498476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.696970+0000","last_change":"2024-04-26T18:24:20.945392+0000","last_active":"2024-04-26T18:27:57.696970+0000","last_peered":"2024-04-26T18:27:57.696970+0000","last_clean":"2024-04-26T18:27:57.696970+0000","last_became_active":"2024-04-26T18:24:20.944971+0000","last_became_peered":"2024-04-26T18:24:20.944971+0000","last_unstale":"2024-04-26T18:27:57.696970+0000","last_undegraded":"2024-04-26T18:27:57.696970+0000","last_fullsized":"2024-04-26T18:27:57.696970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:06.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697005+0000","last_change":"2024-04-26T18:25:06.299351+0000","last_active":"2024-04-26T18:27:57.697005+0000","last_peered":"2024-04-26T18:27:57.697005+0000","last_clean":"2024-04-26T18:27:57.697005+0000","last_became_active":"2024-04-26T18:25:06.298895+0000","last_became_peered":"2024-04-26T18:25:06.298895+0000","last_unstale":"2024-04-26T18:27:57.697005+0000","last_undegraded":"2024-04-26T18:27:57.697005+0000","last_fullsized":"2024-04-26T18:27:57.697005+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:37.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589273+0000","last_change":"2024-04-26T18:24:20.954610+0000","last_active":"2024-04-26T18:27:57.589273+0000","last_peered":"2024-04-26T18:27:57.589273+0000","last_clean":"2024-04-26T18:27:57.589273+0000","last_became_active":"2024-04-26T18:24:20.954239+0000","last_became_peered":"2024-04-26T18:24:20.954239+0000","last_unstale":"2024-04-26T18:27:57.589273+0000","last_undegraded":"2024-04-26T18:27:57.589273+0000","last_fullsized":"2024-04-26T18:27:57.589273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:41.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984496+0000","last_change":"2024-04-26T18:24:20.956932+0000","last_active":"2024-04-26T18:27:57.984496+0000","last_peered":"2024-04-26T18:27:57.984496+0000","last_clean":"2024-04-26T18:27:57.984496+0000","last_became_active":"2024-04-26T18:24:20.956464+0000","last_became_peered":"2024-04-26T18:24:20.956464+0000","last_unstale":"2024-04-26T18:27:57.984496+0000","last_undegraded":"2024-04-26T18:27:57.984496+0000","last_fullsized":"2024-04-26T18:27:57.984496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:01.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.543737+0000","last_change":"2024-04-26T18:24:20.954870+0000","last_active":"2024-04-26T18:27:57.543737+0000","last_peered":"2024-04-26T18:27:57.543737+0000","last_clean":"2024-04-26T18:27:57.543737+0000","last_became_active":"2024-04-26T18:24:20.954542+0000","last_became_peered":"2024-04-26T18:24:20.954542+0000","last_unstale":"2024-04-26T18:27:57.543737+0000","last_undegraded":"2024-04-26T18:27:57.543737+0000","last_fullsized":"2024-04-26T18:27:57.543737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:13:51.932625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452391+0000","last_change":"2024-04-26T18:24:20.950994+0000","last_active":"2024-04-26T18:27:57.452391+0000","last_peered":"2024-04-26T18:27:57.452391+0000","last_clean":"2024-04-26T18:27:57.452391+0000","last_became_active":"2024-04-26T18:24:20.950343+0000","last_became_peered":"2024-04-26T18:24:20.950343+0000","last_unstale":"2024-04-26T18:27:57.452391+0000","last_undegraded":"2024-04-26T18:27:57.452391+0000","last_fullsized":"2024-04-26T18:27:57.452391+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:25.274049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"94'14","reported_seq":247,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589437+0000","last_change":"2024-04-26T18:24:17.929731+0000","last_active":"2024-04-26T18:27:57.589437+0000","last_peered":"2024-04-26T18:27:57.589437+0000","last_clean":"2024-04-26T18:27:57.589437+0000","last_became_active":"2024-04-26T18:24:17.929379+0000","last_became_peered":"2024-04-26T18:24:17.929379+0000","last_unstale":"2024-04-26T18:27:57.589437+0000","last_undegraded":"2024-04-26T18:27:57.589437+0000","last_fullsized":"2024-04-26T18:27:57.589437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:44.821660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452602+0000","last_change":"2024-04-26T18:24:20.962885+0000","last_active":"2024-04-26T18:27:57.452602+0000","last_peered":"2024-04-26T18:27:57.452602+0000","last_clean":"2024-04-26T18:27:57.452602+0000","last_became_active":"2024-04-26T18:24:20.962674+0000","last_became_peered":"2024-04-26T18:24:20.962674+0000","last_unstale":"2024-04-26T18:27:57.452602+0000","last_undegraded":"2024-04-26T18:27:57.452602+0000","last_fullsized":"2024-04-26T18:27:57.452602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:32.444889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452837+0000","last_change":"2024-04-26T18:24:18.141500+0000","last_active":"2024-04-26T18:27:57.452837+0000","last_peered":"2024-04-26T18:27:57.452837+0000","last_clean":"2024-04-26T18:27:57.452837+0000","last_became_active":"2024-04-26T18:24:18.140870+0000","last_became_peered":"2024-04-26T18:24:18.140870+0000","last_unstale":"2024-04-26T18:27:57.452837+0000","last_undegraded":"2024-04-26T18:27:57.452837+0000","last_fullsized":"2024-04-26T18:27:57.452837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:13.821872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697816+0000","last_change":"2024-04-26T18:24:20.949183+0000","last_active":"2024-04-26T18:27:57.697816+0000","last_peered":"2024-04-26T18:27:57.697816+0000","last_clean":"2024-04-26T18:27:57.697816+0000","last_became_active":"2024-04-26T18:24:20.948677+0000","last_became_peered":"2024-04-26T18:24:20.948677+0000","last_unstale":"2024-04-26T18:27:57.697816+0000","last_undegraded":"2024-04-26T18:27:57.697816+0000","last_fullsized":"2024-04-26T18:27:57.697816+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:55.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":68,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.546095+0000","last_change":"2024-04-26T18:25:06.312239+0000","last_active":"2024-04-26T18:27:57.546095+0000","last_peered":"2024-04-26T18:27:57.546095+0000","last_clean":"2024-04-26T18:27:57.546095+0000","last_became_active":"2024-04-26T18:25:06.301304+0000","last_became_peered":"2024-04-26T18:25:06.301304+0000","last_unstale":"2024-04-26T18:27:57.546095+0000","last_undegraded":"2024-04-26T18:27:57.546095+0000","last_fullsized":"2024-04-26T18:27:57.546095+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:36.159348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589460+0000","last_change":"2024-04-26T18:24:20.955555+0000","last_active":"2024-04-26T18:27:57.589460+0000","last_peered":"2024-04-26T18:27:57.589460+0000","last_clean":"2024-04-26T18:27:57.589460+0000","last_became_active":"2024-04-26T18:24:20.955176+0000","last_became_peered":"2024-04-26T18:24:20.955176+0000","last_unstale":"2024-04-26T18:27:57.589460+0000","last_undegraded":"2024-04-26T18:27:57.589460+0000","last_fullsized":"2024-04-26T18:27:57.589460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:02:30.368971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545963+0000","last_change":"2024-04-26T18:24:17.930919+0000","last_active":"2024-04-26T18:27:57.545963+0000","last_peered":"2024-04-26T18:27:57.545963+0000","last_clean":"2024-04-26T18:27:57.545963+0000","last_became_active":"2024-04-26T18:24:17.930596+0000","last_became_peered":"2024-04-26T18:24:17.930596+0000","last_unstale":"2024-04-26T18:27:57.545963+0000","last_undegraded":"2024-04-26T18:27:57.545963+0000","last_fullsized":"2024-04-26T18:27:57.545963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:57.476027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550891+0000","last_change":"2024-04-26T18:24:20.955796+0000","last_active":"2024-04-26T18:27:57.550891+0000","last_peered":"2024-04-26T18:27:57.550891+0000","last_clean":"2024-04-26T18:27:57.550891+0000","last_became_active":"2024-04-26T18:24:20.955432+0000","last_became_peered":"2024-04-26T18:24:20.955432+0000","last_unstale":"2024-04-26T18:27:57.550891+0000","last_undegraded":"2024-04-26T18:27:57.550891+0000","last_fullsized":"2024-04-26T18:27:57.550891+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:25.161797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":243,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550866+0000","last_change":"2024-04-26T18:24:17.929505+0000","last_active":"2024-04-26T18:27:57.550866+0000","last_peered":"2024-04-26T18:27:57.550866+0000","last_clean":"2024-04-26T18:27:57.550866+0000","last_became_active":"2024-04-26T18:24:17.929205+0000","last_became_peered":"2024-04-26T18:24:17.929205+0000","last_unstale":"2024-04-26T18:27:57.550866+0000","last_undegraded":"2024-04-26T18:27:57.550866+0000","last_fullsized":"2024-04-26T18:27:57.550866+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:42.460063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":203,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545699+0000","last_change":"2024-04-26T18:24:20.952134+0000","last_active":"2024-04-26T18:27:57.545699+0000","last_peered":"2024-04-26T18:27:57.545699+0000","last_clean":"2024-04-26T18:27:57.545699+0000","last_became_active":"2024-04-26T18:24:20.951758+0000","last_became_peered":"2024-04-26T18:24:20.951758+0000","last_unstale":"2024-04-26T18:27:57.545699+0000","last_undegraded":"2024-04-26T18:27:57.545699+0000","last_fullsized":"2024-04-26T18:27:57.545699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:25.389221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451604+0000","last_change":"2024-04-26T18:24:17.928704+0000","last_active":"2024-04-26T18:27:57.451604+0000","last_peered":"2024-04-26T18:27:57.451604+0000","last_clean":"2024-04-26T18:27:57.451604+0000","last_became_active":"2024-04-26T18:24:17.928343+0000","last_became_peered":"2024-04-26T18:24:17.928343+0000","last_unstale":"2024-04-26T18:27:57.451604+0000","last_undegraded":"2024-04-26T18:27:57.451604+0000","last_fullsized":"2024-04-26T18:27:57.451604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:49.400481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550744+0000","last_change":"2024-04-26T18:24:20.956808+0000","last_active":"2024-04-26T18:27:57.550744+0000","last_peered":"2024-04-26T18:27:57.550744+0000","last_clean":"2024-04-26T18:27:57.550744+0000","last_became_active":"2024-04-26T18:24:20.956466+0000","last_became_peered":"2024-04-26T18:24:20.956466+0000","last_unstale":"2024-04-26T18:27:57.550744+0000","last_undegraded":"2024-04-26T18:27:57.550744+0000","last_fullsized":"2024-04-26T18:27:57.550744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:31.150670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.693067+0000","last_change":"2024-04-26T18:24:17.929134+0000","last_active":"2024-04-26T18:27:57.693067+0000","last_peered":"2024-04-26T18:27:57.693067+0000","last_clean":"2024-04-26T18:27:57.693067+0000","last_became_active":"2024-04-26T18:24:17.928798+0000","last_became_peered":"2024-04-26T18:24:17.928798+0000","last_unstale":"2024-04-26T18:27:57.693067+0000","last_undegraded":"2024-04-26T18:27:57.693067+0000","last_fullsized":"2024-04-26T18:27:57.693067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:10.450825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452339+0000","last_change":"2024-04-26T18:24:20.951594+0000","last_active":"2024-04-26T18:27:57.452339+0000","last_peered":"2024-04-26T18:27:57.452339+0000","last_clean":"2024-04-26T18:27:57.452339+0000","last_became_active":"2024-04-26T18:24:20.951148+0000","last_became_peered":"2024-04-26T18:24:20.951148+0000","last_unstale":"2024-04-26T18:27:57.452339+0000","last_undegraded":"2024-04-26T18:27:57.452339+0000","last_fullsized":"2024-04-26T18:27:57.452339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:18.703292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550654+0000","last_change":"2024-04-26T18:24:17.931101+0000","last_active":"2024-04-26T18:27:57.550654+0000","last_peered":"2024-04-26T18:27:57.550654+0000","last_clean":"2024-04-26T18:27:57.550654+0000","last_became_active":"2024-04-26T18:24:17.930819+0000","last_became_peered":"2024-04-26T18:24:17.930819+0000","last_unstale":"2024-04-26T18:27:57.550654+0000","last_undegraded":"2024-04-26T18:27:57.550654+0000","last_fullsized":"2024-04-26T18:27:57.550654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.543387+0000","last_change":"2024-04-26T18:24:20.948866+0000","last_active":"2024-04-26T18:27:57.543387+0000","last_peered":"2024-04-26T18:27:57.543387+0000","last_clean":"2024-04-26T18:27:57.543387+0000","last_became_active":"2024-04-26T18:24:20.948562+0000","last_became_peered":"2024-04-26T18:24:20.948562+0000","last_unstale":"2024-04-26T18:27:57.543387+0000","last_undegraded":"2024-04-26T18:27:57.543387+0000","last_fullsized":"2024-04-26T18:27:57.543387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:33:19.616660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":436,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.587833+0000","last_change":"2024-04-26T18:24:17.926384+0000","last_active":"2024-04-26T18:27:57.587833+0000","last_peered":"2024-04-26T18:27:57.587833+0000","last_clean":"2024-04-26T18:27:57.587833+0000","last_became_active":"2024-04-26T18:24:17.926037+0000","last_became_peered":"2024-04-26T18:24:17.926037+0000","last_unstale":"2024-04-26T18:27:57.587833+0000","last_undegraded":"2024-04-26T18:27:57.587833+0000","last_fullsized":"2024-04-26T18:27:57.587833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:05.283755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":397,"num_read_kb":397,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589843+0000","last_change":"2024-04-26T18:24:20.954257+0000","last_active":"2024-04-26T18:27:57.589843+0000","last_peered":"2024-04-26T18:27:57.589843+0000","last_clean":"2024-04-26T18:27:57.589843+0000","last_became_active":"2024-04-26T18:24:20.953913+0000","last_became_peered":"2024-04-26T18:24:20.953913+0000","last_unstale":"2024-04-26T18:27:57.589843+0000","last_undegraded":"2024-04-26T18:27:57.589843+0000","last_fullsized":"2024-04-26T18:27:57.589843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:06.814156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591540+0000","last_change":"2024-04-26T18:24:17.932843+0000","last_active":"2024-04-26T18:27:57.591540+0000","last_peered":"2024-04-26T18:27:57.591540+0000","last_clean":"2024-04-26T18:27:57.591540+0000","last_became_active":"2024-04-26T18:24:17.932502+0000","last_became_peered":"2024-04-26T18:24:17.932502+0000","last_unstale":"2024-04-26T18:27:57.591540+0000","last_undegraded":"2024-04-26T18:27:57.591540+0000","last_fullsized":"2024-04-26T18:27:57.591540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:49.727094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697781+0000","last_change":"2024-04-26T18:24:20.959881+0000","last_active":"2024-04-26T18:27:57.697781+0000","last_peered":"2024-04-26T18:27:57.697781+0000","last_clean":"2024-04-26T18:27:57.697781+0000","last_became_active":"2024-04-26T18:24:20.959541+0000","last_became_peered":"2024-04-26T18:24:20.959541+0000","last_unstale":"2024-04-26T18:27:57.697781+0000","last_undegraded":"2024-04-26T18:27:57.697781+0000","last_fullsized":"2024-04-26T18:27:57.697781+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:23.393521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.551017+0000","last_change":"2024-04-26T18:24:18.142146+0000","last_active":"2024-04-26T18:27:57.551017+0000","last_peered":"2024-04-26T18:27:57.551017+0000","last_clean":"2024-04-26T18:27:57.551017+0000","last_became_active":"2024-04-26T18:24:18.141829+0000","last_became_peered":"2024-04-26T18:24:18.141829+0000","last_unstale":"2024-04-26T18:27:57.551017+0000","last_undegraded":"2024-04-26T18:27:57.551017+0000","last_fullsized":"2024-04-26T18:27:57.551017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:49.341638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589611+0000","last_change":"2024-04-26T18:24:20.957993+0000","last_active":"2024-04-26T18:27:57.589611+0000","last_peered":"2024-04-26T18:27:57.589611+0000","last_clean":"2024-04-26T18:27:57.589611+0000","last_became_active":"2024-04-26T18:24:20.957443+0000","last_became_peered":"2024-04-26T18:24:20.957443+0000","last_unstale":"2024-04-26T18:27:57.589611+0000","last_undegraded":"2024-04-26T18:27:57.589611+0000","last_fullsized":"2024-04-26T18:27:57.589611+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:06:32.960546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'40","reported_seq":482,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.691920+0000","last_change":"2024-04-26T18:24:17.931283+0000","last_active":"2024-04-26T18:27:57.691920+0000","last_peered":"2024-04-26T18:27:57.691920+0000","last_clean":"2024-04-26T18:27:57.691920+0000","last_became_active":"2024-04-26T18:24:17.930943+0000","last_became_peered":"2024-04-26T18:24:17.930943+0000","last_unstale":"2024-04-26T18:27:57.691920+0000","last_undegraded":"2024-04-26T18:27:57.691920+0000","last_fullsized":"2024-04-26T18:27:57.691920+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:50.511677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":71436,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":409,"num_read_kb":268,"num_write":42,"num_write_kb":86,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545864+0000","last_change":"2024-04-26T18:24:20.947509+0000","last_active":"2024-04-26T18:27:57.545864+0000","last_peered":"2024-04-26T18:27:57.545864+0000","last_clean":"2024-04-26T18:27:57.545864+0000","last_became_active":"2024-04-26T18:24:20.947181+0000","last_became_peered":"2024-04-26T18:24:20.947181+0000","last_unstale":"2024-04-26T18:27:57.545864+0000","last_undegraded":"2024-04-26T18:27:57.545864+0000","last_fullsized":"2024-04-26T18:27:57.545864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.262312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452359+0000","last_change":"2024-04-26T18:24:17.934042+0000","last_active":"2024-04-26T18:27:57.452359+0000","last_peered":"2024-04-26T18:27:57.452359+0000","last_clean":"2024-04-26T18:27:57.452359+0000","last_became_active":"2024-04-26T18:24:17.933537+0000","last_became_peered":"2024-04-26T18:24:17.933537+0000","last_unstale":"2024-04-26T18:27:57.452359+0000","last_undegraded":"2024-04-26T18:27:57.452359+0000","last_fullsized":"2024-04-26T18:27:57.452359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:35:06.904160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.692822+0000","last_change":"2024-04-26T18:24:20.957672+0000","last_active":"2024-04-26T18:27:57.692822+0000","last_peered":"2024-04-26T18:27:57.692822+0000","last_clean":"2024-04-26T18:27:57.692822+0000","last_became_active":"2024-04-26T18:24:20.957214+0000","last_became_peered":"2024-04-26T18:24:20.957214+0000","last_unstale":"2024-04-26T18:27:57.692822+0000","last_undegraded":"2024-04-26T18:27:57.692822+0000","last_fullsized":"2024-04-26T18:27:57.692822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:42:35.341858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550631+0000","last_change":"2024-04-26T18:24:17.930324+0000","last_active":"2024-04-26T18:27:57.550631+0000","last_peered":"2024-04-26T18:27:57.550631+0000","last_clean":"2024-04-26T18:27:57.550631+0000","last_became_active":"2024-04-26T18:24:17.930022+0000","last_became_peered":"2024-04-26T18:24:17.930022+0000","last_unstale":"2024-04-26T18:27:57.550631+0000","last_undegraded":"2024-04-26T18:27:57.550631+0000","last_fullsized":"2024-04-26T18:27:57.550631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:36.864816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591382+0000","last_change":"2024-04-26T18:25:06.350388+0000","last_active":"2024-04-26T18:27:57.591382+0000","last_peered":"2024-04-26T18:27:57.591382+0000","last_clean":"2024-04-26T18:27:57.591382+0000","last_became_active":"2024-04-26T18:25:06.349836+0000","last_became_peered":"2024-04-26T18:25:06.349836+0000","last_unstale":"2024-04-26T18:27:57.591382+0000","last_undegraded":"2024-04-26T18:27:57.591382+0000","last_fullsized":"2024-04-26T18:27:57.591382+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:57.762115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'7","reported_seq":241,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697538+0000","last_change":"2024-04-26T18:24:17.934676+0000","last_active":"2024-04-26T18:27:57.697538+0000","last_peered":"2024-04-26T18:27:57.697538+0000","last_clean":"2024-04-26T18:27:57.697538+0000","last_became_active":"2024-04-26T18:24:17.934307+0000","last_became_peered":"2024-04-26T18:24:17.934307+0000","last_unstale":"2024-04-26T18:27:57.697538+0000","last_undegraded":"2024-04-26T18:27:57.697538+0000","last_fullsized":"2024-04-26T18:27:57.697538+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:27.312248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1370,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":202,"num_read_kb":3,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591082+0000","last_change":"2024-04-26T18:24:20.947371+0000","last_active":"2024-04-26T18:27:57.591082+0000","last_peered":"2024-04-26T18:27:57.591082+0000","last_clean":"2024-04-26T18:27:57.591082+0000","last_became_active":"2024-04-26T18:24:20.947048+0000","last_became_peered":"2024-04-26T18:24:20.947048+0000","last_unstale":"2024-04-26T18:27:57.591082+0000","last_undegraded":"2024-04-26T18:27:57.591082+0000","last_fullsized":"2024-04-26T18:27:57.591082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:55.038074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545656+0000","last_change":"2024-04-26T18:24:18.142680+0000","last_active":"2024-04-26T18:27:57.545656+0000","last_peered":"2024-04-26T18:27:57.545656+0000","last_clean":"2024-04-26T18:27:57.545656+0000","last_became_active":"2024-04-26T18:24:18.142392+0000","last_became_peered":"2024-04-26T18:24:18.142392+0000","last_unstale":"2024-04-26T18:27:57.545656+0000","last_undegraded":"2024-04-26T18:27:57.545656+0000","last_fullsized":"2024-04-26T18:27:57.545656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:52.654813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550596+0000","last_change":"2024-04-26T18:24:20.958131+0000","last_active":"2024-04-26T18:27:57.550596+0000","last_peered":"2024-04-26T18:27:57.550596+0000","last_clean":"2024-04-26T18:27:57.550596+0000","last_became_active":"2024-04-26T18:24:20.957701+0000","last_became_peered":"2024-04-26T18:24:20.957701+0000","last_unstale":"2024-04-26T18:27:57.550596+0000","last_undegraded":"2024-04-26T18:27:57.550596+0000","last_fullsized":"2024-04-26T18:27:57.550596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:02.018158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984426+0000","last_change":"2024-04-26T18:25:06.319173+0000","last_active":"2024-04-26T18:27:57.984426+0000","last_peered":"2024-04-26T18:27:57.984426+0000","last_clean":"2024-04-26T18:27:57.984426+0000","last_became_active":"2024-04-26T18:25:06.302935+0000","last_became_peered":"2024-04-26T18:25:06.302935+0000","last_unstale":"2024-04-26T18:27:57.984426+0000","last_undegraded":"2024-04-26T18:27:57.984426+0000","last_fullsized":"2024-04-26T18:27:57.984426+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:40.198105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984341+0000","last_change":"2024-04-26T18:24:20.961230+0000","last_active":"2024-04-26T18:27:57.984341+0000","last_peered":"2024-04-26T18:27:57.984341+0000","last_clean":"2024-04-26T18:27:57.984341+0000","last_became_active":"2024-04-26T18:24:20.960807+0000","last_became_peered":"2024-04-26T18:24:20.960807+0000","last_unstale":"2024-04-26T18:27:57.984341+0000","last_undegraded":"2024-04-26T18:27:57.984341+0000","last_fullsized":"2024-04-26T18:27:57.984341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:39.535100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.543569+0000","last_change":"2024-04-26T18:24:18.142082+0000","last_active":"2024-04-26T18:27:57.543569+0000","last_peered":"2024-04-26T18:27:57.543569+0000","last_clean":"2024-04-26T18:27:57.543569+0000","last_became_active":"2024-04-26T18:24:18.141765+0000","last_became_peered":"2024-04-26T18:24:18.141765+0000","last_unstale":"2024-04-26T18:27:57.543569+0000","last_undegraded":"2024-04-26T18:27:57.543569+0000","last_fullsized":"2024-04-26T18:27:57.543569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:02.140027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451762+0000","last_change":"2024-04-26T18:24:20.952867+0000","last_active":"2024-04-26T18:27:57.451762+0000","last_peered":"2024-04-26T18:27:57.451762+0000","last_clean":"2024-04-26T18:27:57.451762+0000","last_became_active":"2024-04-26T18:24:20.952525+0000","last_became_peered":"2024-04-26T18:24:20.952525+0000","last_unstale":"2024-04-26T18:27:57.451762+0000","last_undegraded":"2024-04-26T18:27:57.451762+0000","last_fullsized":"2024-04-26T18:27:57.451762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:52.804312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'6","reported_seq":239,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.592036+0000","last_change":"2024-04-26T18:24:17.932270+0000","last_active":"2024-04-26T18:27:57.592036+0000","last_peered":"2024-04-26T18:27:57.592036+0000","last_clean":"2024-04-26T18:27:57.592036+0000","last_became_active":"2024-04-26T18:24:17.931901+0000","last_became_peered":"2024-04-26T18:24:17.931901+0000","last_unstale":"2024-04-26T18:27:57.592036+0000","last_undegraded":"2024-04-26T18:27:57.592036+0000","last_fullsized":"2024-04-26T18:27:57.592036+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:13.524174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":200,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451763+0000","last_change":"2024-04-26T18:24:20.960612+0000","last_active":"2024-04-26T18:27:57.451763+0000","last_peered":"2024-04-26T18:27:57.451763+0000","last_clean":"2024-04-26T18:27:57.451763+0000","last_became_active":"2024-04-26T18:24:20.960280+0000","last_became_peered":"2024-04-26T18:24:20.960280+0000","last_unstale":"2024-04-26T18:27:57.451763+0000","last_undegraded":"2024-04-26T18:27:57.451763+0000","last_fullsized":"2024-04-26T18:27:57.451763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:53.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.543368+0000","last_change":"2024-04-26T18:25:06.299736+0000","last_active":"2024-04-26T18:27:57.543368+0000","last_peered":"2024-04-26T18:27:57.543368+0000","last_clean":"2024-04-26T18:27:57.543368+0000","last_became_active":"2024-04-26T18:25:06.299265+0000","last_became_peered":"2024-04-26T18:25:06.299265+0000","last_unstale":"2024-04-26T18:27:57.543368+0000","last_undegraded":"2024-04-26T18:27:57.543368+0000","last_fullsized":"2024-04-26T18:27:57.543368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:32.427551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":239,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.693247+0000","last_change":"2024-04-26T18:24:18.141181+0000","last_active":"2024-04-26T18:27:57.693247+0000","last_peered":"2024-04-26T18:27:57.693247+0000","last_clean":"2024-04-26T18:27:57.693247+0000","last_became_active":"2024-04-26T18:24:18.140854+0000","last_became_peered":"2024-04-26T18:24:18.140854+0000","last_unstale":"2024-04-26T18:27:57.693247+0000","last_undegraded":"2024-04-26T18:27:57.693247+0000","last_fullsized":"2024-04-26T18:27:57.693247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:27.900701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452738+0000","last_change":"2024-04-26T18:24:20.961653+0000","last_active":"2024-04-26T18:27:57.452738+0000","last_peered":"2024-04-26T18:27:57.452738+0000","last_clean":"2024-04-26T18:27:57.452738+0000","last_became_active":"2024-04-26T18:24:20.961330+0000","last_became_peered":"2024-04-26T18:24:20.961330+0000","last_unstale":"2024-04-26T18:27:57.452738+0000","last_undegraded":"2024-04-26T18:27:57.452738+0000","last_fullsized":"2024-04-26T18:27:57.452738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:19.033932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.695409+0000","last_change":"2024-04-26T18:24:17.933613+0000","last_active":"2024-04-26T18:27:57.695409+0000","last_peered":"2024-04-26T18:27:57.695409+0000","last_clean":"2024-04-26T18:27:57.695409+0000","last_became_active":"2024-04-26T18:24:17.933219+0000","last_became_peered":"2024-04-26T18:24:17.933219+0000","last_unstale":"2024-04-26T18:27:57.695409+0000","last_undegraded":"2024-04-26T18:27:57.695409+0000","last_fullsized":"2024-04-26T18:27:57.695409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:04.215821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":366,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591110+0000","last_change":"2024-04-26T18:22:57.585634+0000","last_active":"2024-04-26T18:27:57.591110+0000","last_peered":"2024-04-26T18:27:57.591110+0000","last_clean":"2024-04-26T18:27:57.591110+0000","last_became_active":"2024-04-26T18:22:57.585198+0000","last_became_peered":"2024-04-26T18:22:57.585198+0000","last_unstale":"2024-04-26T18:27:57.591110+0000","last_undegraded":"2024-04-26T18:27:57.591110+0000","last_fullsized":"2024-04-26T18:27:57.591110+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_clean_scrub_stamp":"2024-04-26T18:19:31.594684+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:56:11.861930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.692768+0000","last_change":"2024-04-26T18:24:20.950303+0000","last_active":"2024-04-26T18:27:57.692768+0000","last_peered":"2024-04-26T18:27:57.692768+0000","last_clean":"2024-04-26T18:27:57.692768+0000","last_became_active":"2024-04-26T18:24:20.950002+0000","last_became_peered":"2024-04-26T18:24:20.950002+0000","last_unstale":"2024-04-26T18:27:57.692768+0000","last_undegraded":"2024-04-26T18:27:57.692768+0000","last_fullsized":"2024-04-26T18:27:57.692768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:00.912752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591327+0000","last_change":"2024-04-26T18:24:17.934924+0000","last_active":"2024-04-26T18:27:57.591327+0000","last_peered":"2024-04-26T18:27:57.591327+0000","last_clean":"2024-04-26T18:27:57.591327+0000","last_became_active":"2024-04-26T18:24:17.934598+0000","last_became_peered":"2024-04-26T18:24:17.934598+0000","last_unstale":"2024-04-26T18:27:57.591327+0000","last_undegraded":"2024-04-26T18:27:57.591327+0000","last_fullsized":"2024-04-26T18:27:57.591327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:28.690043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545940+0000","last_change":"2024-04-26T18:24:18.142560+0000","last_active":"2024-04-26T18:27:57.545940+0000","last_peered":"2024-04-26T18:27:57.545940+0000","last_clean":"2024-04-26T18:27:57.545940+0000","last_became_active":"2024-04-26T18:24:18.142184+0000","last_became_peered":"2024-04-26T18:24:18.142184+0000","last_unstale":"2024-04-26T18:27:57.545940+0000","last_undegraded":"2024-04-26T18:27:57.545940+0000","last_fullsized":"2024-04-26T18:27:57.545940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:26.656113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694369+0000","last_change":"2024-04-26T18:24:20.952073+0000","last_active":"2024-04-26T18:27:57.694369+0000","last_peered":"2024-04-26T18:27:57.694369+0000","last_clean":"2024-04-26T18:27:57.694369+0000","last_became_active":"2024-04-26T18:24:20.951761+0000","last_became_peered":"2024-04-26T18:24:20.951761+0000","last_unstale":"2024-04-26T18:27:57.694369+0000","last_undegraded":"2024-04-26T18:27:57.694369+0000","last_fullsized":"2024-04-26T18:27:57.694369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:30.244060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'50","reported_seq":106,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.655257+0000","last_change":"2024-04-26T18:24:17.927171+0000","last_active":"2024-04-26T18:27:57.655257+0000","last_peered":"2024-04-26T18:27:57.655257+0000","last_clean":"2024-04-26T18:27:57.655257+0000","last_became_active":"2024-04-26T18:24:17.926816+0000","last_became_peered":"2024-04-26T18:24:17.926816+0000","last_unstale":"2024-04-26T18:27:57.655257+0000","last_undegraded":"2024-04-26T18:27:57.655257+0000","last_fullsized":"2024-04-26T18:27:57.655257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:18.249363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":89,"num_write_kb":837,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452834+0000","last_change":"2024-04-26T18:24:20.951684+0000","last_active":"2024-04-26T18:27:57.452834+0000","last_peered":"2024-04-26T18:27:57.452834+0000","last_clean":"2024-04-26T18:27:57.452834+0000","last_became_active":"2024-04-26T18:24:20.951364+0000","last_became_peered":"2024-04-26T18:24:20.951364+0000","last_unstale":"2024-04-26T18:27:57.452834+0000","last_undegraded":"2024-04-26T18:27:57.452834+0000","last_fullsized":"2024-04-26T18:27:57.452834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:28.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.695676+0000","last_change":"2024-04-26T18:24:18.229770+0000","last_active":"2024-04-26T18:27:57.695676+0000","last_peered":"2024-04-26T18:27:57.695676+0000","last_clean":"2024-04-26T18:27:57.695676+0000","last_became_active":"2024-04-26T18:24:18.229446+0000","last_became_peered":"2024-04-26T18:24:18.229446+0000","last_unstale":"2024-04-26T18:27:57.695676+0000","last_undegraded":"2024-04-26T18:27:57.695676+0000","last_fullsized":"2024-04-26T18:27:57.695676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:14.794797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697561+0000","last_change":"2024-04-26T18:24:20.960513+0000","last_active":"2024-04-26T18:27:57.697561+0000","last_peered":"2024-04-26T18:27:57.697561+0000","last_clean":"2024-04-26T18:27:57.697561+0000","last_became_active":"2024-04-26T18:24:20.959549+0000","last_became_peered":"2024-04-26T18:24:20.959549+0000","last_unstale":"2024-04-26T18:27:57.697561+0000","last_undegraded":"2024-04-26T18:27:57.697561+0000","last_fullsized":"2024-04-26T18:27:57.697561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:18:37.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589273+0000","last_change":"2024-04-26T18:24:18.141549+0000","last_active":"2024-04-26T18:27:57.589273+0000","last_peered":"2024-04-26T18:27:57.589273+0000","last_clean":"2024-04-26T18:27:57.589273+0000","last_became_active":"2024-04-26T18:24:18.141172+0000","last_became_peered":"2024-04-26T18:24:18.141172+0000","last_unstale":"2024-04-26T18:27:57.589273+0000","last_undegraded":"2024-04-26T18:27:57.589273+0000","last_fullsized":"2024-04-26T18:27:57.589273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:15.878130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545672+0000","last_change":"2024-04-26T18:24:20.948121+0000","last_active":"2024-04-26T18:27:57.545672+0000","last_peered":"2024-04-26T18:27:57.545672+0000","last_clean":"2024-04-26T18:27:57.545672+0000","last_became_active":"2024-04-26T18:24:20.947738+0000","last_became_peered":"2024-04-26T18:24:20.947738+0000","last_unstale":"2024-04-26T18:27:57.545672+0000","last_undegraded":"2024-04-26T18:27:57.545672+0000","last_fullsized":"2024-04-26T18:27:57.545672+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:04.237302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452547+0000","last_change":"2024-04-26T18:24:17.926631+0000","last_active":"2024-04-26T18:27:57.452547+0000","last_peered":"2024-04-26T18:27:57.452547+0000","last_clean":"2024-04-26T18:27:57.452547+0000","last_became_active":"2024-04-26T18:24:17.926259+0000","last_became_peered":"2024-04-26T18:24:17.926259+0000","last_unstale":"2024-04-26T18:27:57.452547+0000","last_undegraded":"2024-04-26T18:27:57.452547+0000","last_fullsized":"2024-04-26T18:27:57.452547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:47.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984177+0000","last_change":"2024-04-26T18:24:20.947283+0000","last_active":"2024-04-26T18:27:57.984177+0000","last_peered":"2024-04-26T18:27:57.984177+0000","last_clean":"2024-04-26T18:27:57.984177+0000","last_became_active":"2024-04-26T18:24:20.946812+0000","last_became_peered":"2024-04-26T18:24:20.946812+0000","last_unstale":"2024-04-26T18:27:57.984177+0000","last_undegraded":"2024-04-26T18:27:57.984177+0000","last_fullsized":"2024-04-26T18:27:57.984177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:33.730698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452290+0000","last_change":"2024-04-26T18:24:17.931843+0000","last_active":"2024-04-26T18:27:57.452290+0000","last_peered":"2024-04-26T18:27:57.452290+0000","last_clean":"2024-04-26T18:27:57.452290+0000","last_became_active":"2024-04-26T18:24:17.931442+0000","last_became_peered":"2024-04-26T18:24:17.931442+0000","last_unstale":"2024-04-26T18:27:57.452290+0000","last_undegraded":"2024-04-26T18:27:57.452290+0000","last_fullsized":"2024-04-26T18:27:57.452290+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:09.649435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452567+0000","last_change":"2024-04-26T18:24:20.955661+0000","last_active":"2024-04-26T18:27:57.452567+0000","last_peered":"2024-04-26T18:27:57.452567+0000","last_clean":"2024-04-26T18:27:57.452567+0000","last_became_active":"2024-04-26T18:24:20.955344+0000","last_became_peered":"2024-04-26T18:24:20.955344+0000","last_unstale":"2024-04-26T18:27:57.452567+0000","last_undegraded":"2024-04-26T18:27:57.452567+0000","last_fullsized":"2024-04-26T18:27:57.452567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:44.274381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984200+0000","last_change":"2024-04-26T18:25:06.315466+0000","last_active":"2024-04-26T18:27:57.984200+0000","last_peered":"2024-04-26T18:27:57.984200+0000","last_clean":"2024-04-26T18:27:57.984200+0000","last_became_active":"2024-04-26T18:25:06.301788+0000","last_became_peered":"2024-04-26T18:25:06.301788+0000","last_unstale":"2024-04-26T18:27:57.984200+0000","last_undegraded":"2024-04-26T18:27:57.984200+0000","last_fullsized":"2024-04-26T18:27:57.984200+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:12.998583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591736+0000","last_change":"2024-04-26T18:24:20.955746+0000","last_active":"2024-04-26T18:27:57.591736+0000","last_peered":"2024-04-26T18:27:57.591736+0000","last_clean":"2024-04-26T18:27:57.591736+0000","last_became_active":"2024-04-26T18:24:20.953652+0000","last_became_peered":"2024-04-26T18:24:20.953652+0000","last_unstale":"2024-04-26T18:27:57.591736+0000","last_undegraded":"2024-04-26T18:27:57.591736+0000","last_fullsized":"2024-04-26T18:27:57.591736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:31.414185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":200,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589452+0000","last_change":"2024-04-26T18:25:06.309393+0000","last_active":"2024-04-26T18:27:57.589452+0000","last_peered":"2024-04-26T18:27:57.589452+0000","last_clean":"2024-04-26T18:27:57.589452+0000","last_became_active":"2024-04-26T18:25:06.302434+0000","last_became_peered":"2024-04-26T18:25:06.302434+0000","last_unstale":"2024-04-26T18:27:57.589452+0000","last_undegraded":"2024-04-26T18:27:57.589452+0000","last_fullsized":"2024-04-26T18:27:57.589452+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:54.730295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":178,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591717+0000","last_change":"2024-04-26T18:24:20.954238+0000","last_active":"2024-04-26T18:27:57.591717+0000","last_peered":"2024-04-26T18:27:57.591717+0000","last_clean":"2024-04-26T18:27:57.591717+0000","last_became_active":"2024-04-26T18:24:20.953883+0000","last_became_peered":"2024-04-26T18:24:20.953883+0000","last_unstale":"2024-04-26T18:27:57.591717+0000","last_undegraded":"2024-04-26T18:27:57.591717+0000","last_fullsized":"2024-04-26T18:27:57.591717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:10.580835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.653236+0000","last_change":"2024-04-26T18:24:17.926901+0000","last_active":"2024-04-26T18:27:57.653236+0000","last_peered":"2024-04-26T18:27:57.653236+0000","last_clean":"2024-04-26T18:27:57.653236+0000","last_became_active":"2024-04-26T18:24:17.926291+0000","last_became_peered":"2024-04-26T18:24:17.926291+0000","last_unstale":"2024-04-26T18:27:57.653236+0000","last_undegraded":"2024-04-26T18:27:57.653236+0000","last_fullsized":"2024-04-26T18:27:57.653236+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:17.695198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591253+0000","last_change":"2024-04-26T18:24:20.950578+0000","last_active":"2024-04-26T18:27:57.591253+0000","last_peered":"2024-04-26T18:27:57.591253+0000","last_clean":"2024-04-26T18:27:57.591253+0000","last_became_active":"2024-04-26T18:24:20.950174+0000","last_became_peered":"2024-04-26T18:24:20.950174+0000","last_unstale":"2024-04-26T18:27:57.591253+0000","last_undegraded":"2024-04-26T18:27:57.591253+0000","last_fullsized":"2024-04-26T18:27:57.591253+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694486+0000","last_change":"2024-04-26T18:24:17.928054+0000","last_active":"2024-04-26T18:27:57.694486+0000","last_peered":"2024-04-26T18:27:57.694486+0000","last_clean":"2024-04-26T18:27:57.694486+0000","last_became_active":"2024-04-26T18:24:17.927715+0000","last_became_peered":"2024-04-26T18:24:17.927715+0000","last_unstale":"2024-04-26T18:27:57.694486+0000","last_undegraded":"2024-04-26T18:27:57.694486+0000","last_fullsized":"2024-04-26T18:27:57.694486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.155436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591822+0000","last_change":"2024-04-26T18:24:20.944512+0000","last_active":"2024-04-26T18:27:57.591822+0000","last_peered":"2024-04-26T18:27:57.591822+0000","last_clean":"2024-04-26T18:27:57.591822+0000","last_became_active":"2024-04-26T18:24:20.944123+0000","last_became_peered":"2024-04-26T18:24:20.944123+0000","last_unstale":"2024-04-26T18:27:57.591822+0000","last_undegraded":"2024-04-26T18:27:57.591822+0000","last_fullsized":"2024-04-26T18:27:57.591822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.240206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984272+0000","last_change":"2024-04-26T18:24:20.959138+0000","last_active":"2024-04-26T18:27:57.984272+0000","last_peered":"2024-04-26T18:27:57.984272+0000","last_clean":"2024-04-26T18:27:57.984272+0000","last_became_active":"2024-04-26T18:24:20.958675+0000","last_became_peered":"2024-04-26T18:24:20.958675+0000","last_unstale":"2024-04-26T18:27:57.984272+0000","last_undegraded":"2024-04-26T18:27:57.984272+0000","last_fullsized":"2024-04-26T18:27:57.984272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:34:32.103380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452173+0000","last_change":"2024-04-26T18:24:20.961689+0000","last_active":"2024-04-26T18:27:57.452173+0000","last_peered":"2024-04-26T18:27:57.452173+0000","last_clean":"2024-04-26T18:27:57.452173+0000","last_became_active":"2024-04-26T18:24:20.961374+0000","last_became_peered":"2024-04-26T18:24:20.961374+0000","last_unstale":"2024-04-26T18:27:57.452173+0000","last_undegraded":"2024-04-26T18:27:57.452173+0000","last_fullsized":"2024-04-26T18:27:57.452173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:55.243887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452150+0000","last_change":"2024-04-26T18:24:20.950711+0000","last_active":"2024-04-26T18:27:57.452150+0000","last_peered":"2024-04-26T18:27:57.452150+0000","last_clean":"2024-04-26T18:27:57.452150+0000","last_became_active":"2024-04-26T18:24:20.950371+0000","last_became_peered":"2024-04-26T18:24:20.950371+0000","last_unstale":"2024-04-26T18:27:57.452150+0000","last_undegraded":"2024-04-26T18:27:57.452150+0000","last_fullsized":"2024-04-26T18:27:57.452150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:35.307921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697143+0000","last_change":"2024-04-26T18:24:20.959990+0000","last_active":"2024-04-26T18:27:57.697143+0000","last_peered":"2024-04-26T18:27:57.697143+0000","last_clean":"2024-04-26T18:27:57.697143+0000","last_became_active":"2024-04-26T18:24:20.959514+0000","last_became_peered":"2024-04-26T18:24:20.959514+0000","last_unstale":"2024-04-26T18:27:57.697143+0000","last_undegraded":"2024-04-26T18:27:57.697143+0000","last_fullsized":"2024-04-26T18:27:57.697143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:00.184448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694549+0000","last_change":"2024-04-26T18:24:20.957338+0000","last_active":"2024-04-26T18:27:57.694549+0000","last_peered":"2024-04-26T18:27:57.694549+0000","last_clean":"2024-04-26T18:27:57.694549+0000","last_became_active":"2024-04-26T18:24:20.956999+0000","last_became_peered":"2024-04-26T18:24:20.956999+0000","last_unstale":"2024-04-26T18:27:57.694549+0000","last_undegraded":"2024-04-26T18:27:57.694549+0000","last_fullsized":"2024-04-26T18:27:57.694549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.066902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984324+0000","last_change":"2024-04-26T18:24:20.953634+0000","last_active":"2024-04-26T18:27:57.984324+0000","last_peered":"2024-04-26T18:27:57.984324+0000","last_clean":"2024-04-26T18:27:57.984324+0000","last_became_active":"2024-04-26T18:24:20.953259+0000","last_became_peered":"2024-04-26T18:24:20.953259+0000","last_unstale":"2024-04-26T18:27:57.984324+0000","last_undegraded":"2024-04-26T18:27:57.984324+0000","last_fullsized":"2024-04-26T18:27:57.984324+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:00.095344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452650+0000","last_change":"2024-04-26T18:24:20.954997+0000","last_active":"2024-04-26T18:27:57.452650+0000","last_peered":"2024-04-26T18:27:57.452650+0000","last_clean":"2024-04-26T18:27:57.452650+0000","last_became_active":"2024-04-26T18:24:20.954223+0000","last_became_peered":"2024-04-26T18:24:20.954223+0000","last_unstale":"2024-04-26T18:27:57.452650+0000","last_undegraded":"2024-04-26T18:27:57.452650+0000","last_fullsized":"2024-04-26T18:27:57.452650+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:14.092455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452832+0000","last_change":"2024-04-26T18:24:20.957056+0000","last_active":"2024-04-26T18:27:57.452832+0000","last_peered":"2024-04-26T18:27:57.452832+0000","last_clean":"2024-04-26T18:27:57.452832+0000","last_became_active":"2024-04-26T18:24:20.956737+0000","last_became_peered":"2024-04-26T18:24:20.956737+0000","last_unstale":"2024-04-26T18:27:57.452832+0000","last_undegraded":"2024-04-26T18:27:57.452832+0000","last_fullsized":"2024-04-26T18:27:57.452832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:48.358862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550741+0000","last_change":"2024-04-26T18:25:06.350206+0000","last_active":"2024-04-26T18:27:57.550741+0000","last_peered":"2024-04-26T18:27:57.550741+0000","last_clean":"2024-04-26T18:27:57.550741+0000","last_became_active":"2024-04-26T18:25:06.349866+0000","last_became_peered":"2024-04-26T18:25:06.349866+0000","last_unstale":"2024-04-26T18:27:57.550741+0000","last_undegraded":"2024-04-26T18:27:57.550741+0000","last_fullsized":"2024-04-26T18:27:57.550741+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:41.909056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452351+0000","last_change":"2024-04-26T18:24:20.956000+0000","last_active":"2024-04-26T18:27:57.452351+0000","last_peered":"2024-04-26T18:27:57.452351+0000","last_clean":"2024-04-26T18:27:57.452351+0000","last_became_active":"2024-04-26T18:24:20.955677+0000","last_became_peered":"2024-04-26T18:24:20.955677+0000","last_unstale":"2024-04-26T18:27:57.452351+0000","last_undegraded":"2024-04-26T18:27:57.452351+0000","last_fullsized":"2024-04-26T18:27:57.452351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:20.895674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589421+0000","last_change":"2024-04-26T18:24:20.959317+0000","last_active":"2024-04-26T18:27:57.589421+0000","last_peered":"2024-04-26T18:27:57.589421+0000","last_clean":"2024-04-26T18:27:57.589421+0000","last_became_active":"2024-04-26T18:24:20.958963+0000","last_became_peered":"2024-04-26T18:24:20.958963+0000","last_unstale":"2024-04-26T18:27:57.589421+0000","last_undegraded":"2024-04-26T18:27:57.589421+0000","last_fullsized":"2024-04-26T18:27:57.589421+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:08.231089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694689+0000","last_change":"2024-04-26T18:24:20.954445+0000","last_active":"2024-04-26T18:27:57.694689+0000","last_peered":"2024-04-26T18:27:57.694689+0000","last_clean":"2024-04-26T18:27:57.694689+0000","last_became_active":"2024-04-26T18:24:20.953852+0000","last_became_peered":"2024-04-26T18:24:20.953852+0000","last_unstale":"2024-04-26T18:27:57.694689+0000","last_undegraded":"2024-04-26T18:27:57.694689+0000","last_fullsized":"2024-04-26T18:27:57.694689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:05.636897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984195+0000","last_change":"2024-04-26T18:24:20.958023+0000","last_active":"2024-04-26T18:27:57.984195+0000","last_peered":"2024-04-26T18:27:57.984195+0000","last_clean":"2024-04-26T18:27:57.984195+0000","last_became_active":"2024-04-26T18:24:20.957642+0000","last_became_peered":"2024-04-26T18:24:20.957642+0000","last_unstale":"2024-04-26T18:27:57.984195+0000","last_undegraded":"2024-04-26T18:27:57.984195+0000","last_fullsized":"2024-04-26T18:27:57.984195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:46.332710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106773,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":604911,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2098,"num_read_kb":1172,"num_write":375,"num_write_kb":1468,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1583,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":724992,"data_stored":1814733,"data_compressed":81435,"data_compressed_allocated":110592,"data_compressed_original":1572864,"omap_allocated":0,"internal_metadata":0},"log_size":302,"ondisk_log_size":302,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351034,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32180,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712972,"statfs":{"total":95995035648,"available":95962083328,"internally_reserved":0,"allocated":1024000,"data_stored":501214,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612671,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32364,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712788,"statfs":{"total":95995035648,"available":95961894912,"internally_reserved":0,"allocated":1216512,"data_stored":1304500,"data_compressed":58084,"data_compressed_allocated":77824,"data_compressed_original":761856,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776197,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32212,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712940,"statfs":{"total":95995035648,"available":95962050560,"internally_reserved":0,"allocated":1056768,"data_stored":509583,"data_compressed":14494,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037834,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1073152,"data_stored":903155,"data_compressed":33640,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299470,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32260,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712892,"statfs":{"total":95995035648,"available":95962001408,"internally_reserved":0,"allocated":1105920,"data_stored":1103793,"data_compressed":50085,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462997,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":509691,"data_compressed":14494,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574957,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1092,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1118208,"data_stored":583742,"data_compressed":16337,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738482,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":511171,"data_compressed":14494,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411433,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1122304,"data_stored":581231,"data_compressed":16337,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149795,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28092,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717060,"statfs":{"total":95995035648,"available":95966269440,"internally_reserved":0,"allocated":1101824,"data_stored":1033941,"data_compressed":39796,"data_compressed_allocated":53248,"data_compressed_original":557056,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888159,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":500168,"data_compressed":11570,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724632,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712852,"statfs":{"total":95995035648,"available":95961960448,"internally_reserved":0,"allocated":1138688,"data_stored":1233898,"data_compressed":56241,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":75281,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":72770,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":525246,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":134952,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":205554,"data_compressed":7999,"data_compressed_allocated":12288,"data_compressed_original":131072,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1746,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:28:03.436 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:28:03.676 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:03.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:03 smithi174 ceph-mon[32705]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:28:03.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:03 smithi174 ceph-mon[32705]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:28:03.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:03 smithi174 ceph-mon[32705]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:28:03.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:03 smithi174 ceph-mon[32705]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:28:03.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:03 smithi174 ceph-mon[32705]: Updating smithi084:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:28:03.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:03 smithi174 ceph-mon[32705]: Updating smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:28:03.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:03 smithi174 ceph-mon[32705]: Updating smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:28:03.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:03 smithi174 ceph-mon[32705]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-26T18:28:03.798 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:03 smithi084 ceph-mon[29141]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:28:03.798 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:03 smithi084 ceph-mon[29141]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:28:03.798 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:03 smithi084 ceph-mon[29141]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:28:03.798 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:03 smithi084 ceph-mon[29141]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:28:03.799 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:03 smithi084 ceph-mon[29141]: Updating smithi084:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:28:03.799 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:03 smithi084 ceph-mon[29141]: Updating smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:28:03.799 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:03 smithi084 ceph-mon[29141]: Updating smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:28:03.799 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:03 smithi084 ceph-mon[29141]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-26T18:28:03.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:03 smithi103 ceph-mon[32727]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:28:03.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:03 smithi103 ceph-mon[32727]: Updating smithi084:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:28:03.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:03 smithi103 ceph-mon[32727]: Updating smithi103:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:28:03.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:03 smithi103 ceph-mon[32727]: Updating smithi174:/etc/ceph/ceph.client.admin.keyring 2024-04-26T18:28:03.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:03 smithi103 ceph-mon[32727]: Updating smithi084:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:28:03.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:03 smithi103 ceph-mon[32727]: Updating smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:28:03.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:03 smithi103 ceph-mon[32727]: Updating smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T18:28:03.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:03 smithi103 ceph-mon[32727]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-04-26T18:28:04.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:04 smithi084 ceph-mon[29141]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:28:04.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:04 smithi084 ceph-mon[29141]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 121 KiB/s wr, 16 op/s 2024-04-26T18:28:04.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:04 smithi174 ceph-mon[32705]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:28:04.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:04 smithi174 ceph-mon[32705]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 121 KiB/s wr, 16 op/s 2024-04-26T18:28:04.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:04 smithi103 ceph-mon[32727]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:28:04.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:04 smithi103 ceph-mon[32727]: pgmap v6: 97 pgs: 97 active+clean; 1.3 MiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 121 KiB/s wr, 16 op/s 2024-04-26T18:28:05.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.391+0000 7f10d11a3700 1 -- 172.21.15.84:0/1304268320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10cc102ce0 msgr2=0x7f10cc10f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:05.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.391+0000 7f10d11a3700 1 --2- 172.21.15.84:0/1304268320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10cc102ce0 0x7f10cc10f1d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f10bc009990 tx=0x7f10bc009ca0 comp rx=0 tx=0).stop 2024-04-26T18:28:05.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.391+0000 7f10d11a3700 1 -- 172.21.15.84:0/1304268320 shutdown_connections 2024-04-26T18:28:05.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.391+0000 7f10d11a3700 1 --2- 172.21.15.84:0/1304268320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10cc102ce0 0x7f10cc10f1d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:05.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.391+0000 7f10d11a3700 1 --2- 172.21.15.84:0/1304268320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc108390 0x7f10cc108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:05.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.391+0000 7f10d11a3700 1 --2- 172.21.15.84:0/1304268320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10cc102320 0x7f10cc1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:05.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.391+0000 7f10d11a3700 1 -- 172.21.15.84:0/1304268320 >> 172.21.15.84:0/1304268320 conn(0x7f10cc0792d0 msgr2=0x7f10cc0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:05.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.391+0000 7f10d11a3700 1 -- 172.21.15.84:0/1304268320 shutdown_connections 2024-04-26T18:28:05.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.391+0000 7f10d11a3700 1 -- 172.21.15.84:0/1304268320 wait complete. 2024-04-26T18:28:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.392+0000 7f10d11a3700 1 Processor -- start 2024-04-26T18:28:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.392+0000 7f10d11a3700 1 -- start start 2024-04-26T18:28:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10d11a3700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10cc102320 0x7f10cc19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10d11a3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc102ce0 0x7f10cc19fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10d11a3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10cc108390 0x7f10cc1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10d11a3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc1124b0 con 0x7f10cc108390 2024-04-26T18:28:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10d11a3700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc112330 con 0x7f10cc102320 2024-04-26T18:28:05.394 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10d11a3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f10cc112630 con 0x7f10cc102ce0 2024-04-26T18:28:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10cb59e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10cc108390 0x7f10cc1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10cb59e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10cc108390 0x7f10cc1998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43342/0 (socket says 172.21.15.84:43342) 2024-04-26T18:28:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10cb59e700 1 -- 172.21.15.84:0/3401399769 learned_addr learned my addr 172.21.15.84:0/3401399769 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10ca59c700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc102ce0 0x7f10cc19fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10cad9d700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10cc102320 0x7f10cc19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10cb59e700 1 -- 172.21.15.84:0/3401399769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc102ce0 msgr2=0x7f10cc19fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10cb59e700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc102ce0 0x7f10cc19fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10cb59e700 1 -- 172.21.15.84:0/3401399769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10cc102320 msgr2=0x7f10cc19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10cb59e700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10cc102320 0x7f10cc19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.393+0000 7f10cb59e700 1 -- 172.21.15.84:0/3401399769 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10cc19a010 con 0x7f10cc108390 2024-04-26T18:28:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.394+0000 7f10cb59e700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10cc108390 0x7f10cc1998b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0098e0 tx=0x7f10bc009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.394+0000 7f10b3fff700 1 -- 172.21.15.84:0/3401399769 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc026070 con 0x7f10cc108390 2024-04-26T18:28:05.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.394+0000 7f10d11a3700 1 -- 172.21.15.84:0/3401399769 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10cc19a2a0 con 0x7f10cc108390 2024-04-26T18:28:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.394+0000 7f10b3fff700 1 -- 172.21.15.84:0/3401399769 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10bc0054e0 con 0x7f10cc108390 2024-04-26T18:28:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.394+0000 7f10d11a3700 1 -- 172.21.15.84:0/3401399769 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10cc1a6570 con 0x7f10cc108390 2024-04-26T18:28:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.394+0000 7f10b3fff700 1 -- 172.21.15.84:0/3401399769 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10bc0046c0 con 0x7f10cc108390 2024-04-26T18:28:05.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.395+0000 7f10d11a3700 1 -- 172.21.15.84:0/3401399769 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10b8005330 con 0x7f10cc108390 2024-04-26T18:28:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.397+0000 7f10b3fff700 1 -- 172.21.15.84:0/3401399769 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f10bc029070 con 0x7f10cc108390 2024-04-26T18:28:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.397+0000 7f10b3fff700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f10b40a3f60 0x7f10b40a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.397+0000 7f10b3fff700 1 -- 172.21.15.84:0/3401399769 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f10bc0d13a0 con 0x7f10cc108390 2024-04-26T18:28:05.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.397+0000 7f10cad9d700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f10b40a3f60 0x7f10b40a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:05.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.398+0000 7f10cad9d700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f10b40a3f60 0x7f10b40a6420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10cc100b70 tx=0x7f10c400b410 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:05.401 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.401+0000 7f10b3fff700 1 -- 172.21.15.84:0/3401399769 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f10bc013070 con 0x7f10cc108390 2024-04-26T18:28:05.600 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.600+0000 7f10d11a3700 1 -- 172.21.15.84:0/3401399769 --> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f10b8000bc0 con 0x7f10b40a3f60 2024-04-26T18:28:05.607 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.607+0000 7f10b3fff700 1 -- 172.21.15.84:0/3401399769 <== mgr.15240 v2:172.21.15.84:6800/3453983695 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258145 (secure 0 0 0) 0x7f10b8000bc0 con 0x7f10b40a3f60 2024-04-26T18:28:05.610 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:28:05.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.610+0000 7f10d11a3700 1 -- 172.21.15.84:0/3401399769 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f10b40a3f60 msgr2=0x7f10b40a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:05.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.610+0000 7f10d11a3700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f10b40a3f60 0x7f10b40a6420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f10cc100b70 tx=0x7f10c400b410 comp rx=0 tx=0).stop 2024-04-26T18:28:05.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.610+0000 7f10d11a3700 1 -- 172.21.15.84:0/3401399769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10cc108390 msgr2=0x7f10cc1998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:05.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.610+0000 7f10d11a3700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10cc108390 0x7f10cc1998b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f10bc0098e0 tx=0x7f10bc009f50 comp rx=0 tx=0).stop 2024-04-26T18:28:05.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.610+0000 7f10d11a3700 1 -- 172.21.15.84:0/3401399769 shutdown_connections 2024-04-26T18:28:05.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.610+0000 7f10d11a3700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f10b40a3f60 0x7f10b40a6420 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:05.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.610+0000 7f10d11a3700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f10cc108390 0x7f10cc1998b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:05.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.610+0000 7f10d11a3700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f10cc102320 0x7f10cc19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:05.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.610+0000 7f10d11a3700 1 --2- 172.21.15.84:0/3401399769 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f10cc102ce0 0x7f10cc19fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:05.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.610+0000 7f10d11a3700 1 -- 172.21.15.84:0/3401399769 >> 172.21.15.84:0/3401399769 conn(0x7f10cc0792d0 msgr2=0x7f10cc101600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:05.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.611+0000 7f10d11a3700 1 -- 172.21.15.84:0/3401399769 shutdown_connections 2024-04-26T18:28:05.613 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:05.611+0000 7f10d11a3700 1 -- 172.21.15.84:0/3401399769 wait complete. 2024-04-26T18:28:05.614 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-04-26T18:28:06.314 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-04-26T18:28:05.466396+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1398058,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2768,"num_read_kb":1885,"num_write":773,"num_write_kb":5599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182319,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1076,"ondisk_log_size":1076,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374580,"kb_used_data":12912,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567244,"statfs":{"total":1151940427776,"available":1151556857856,"internally_reserved":0,"allocated":13221888,"data_stored":10295706,"data_compressed":389724,"data_compressed_allocated":528384,"data_compressed_original":4386816,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":202545,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":534,"num_read_kb":594,"num_write":215,"num_write_kb":1191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.990756"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984569+0000","last_change":"2024-04-26T18:24:20.951581+0000","last_active":"2024-04-26T18:27:57.984569+0000","last_peered":"2024-04-26T18:27:57.984569+0000","last_clean":"2024-04-26T18:27:57.984569+0000","last_became_active":"2024-04-26T18:24:20.951260+0000","last_became_peered":"2024-04-26T18:24:20.951260+0000","last_unstale":"2024-04-26T18:27:57.984569+0000","last_undegraded":"2024-04-26T18:27:57.984569+0000","last_fullsized":"2024-04-26T18:27:57.984569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:54:28.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697639+0000","last_change":"2024-04-26T18:24:20.952263+0000","last_active":"2024-04-26T18:27:57.697639+0000","last_peered":"2024-04-26T18:27:57.697639+0000","last_clean":"2024-04-26T18:27:57.697639+0000","last_became_active":"2024-04-26T18:24:20.951930+0000","last_became_peered":"2024-04-26T18:24:20.951930+0000","last_unstale":"2024-04-26T18:27:57.697639+0000","last_undegraded":"2024-04-26T18:27:57.697639+0000","last_fullsized":"2024-04-26T18:27:57.697639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:54.816572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697346+0000","last_change":"2024-04-26T18:24:20.952280+0000","last_active":"2024-04-26T18:27:57.697346+0000","last_peered":"2024-04-26T18:27:57.697346+0000","last_clean":"2024-04-26T18:27:57.697346+0000","last_became_active":"2024-04-26T18:24:20.951623+0000","last_became_peered":"2024-04-26T18:24:20.951623+0000","last_unstale":"2024-04-26T18:27:57.697346+0000","last_undegraded":"2024-04-26T18:27:57.697346+0000","last_fullsized":"2024-04-26T18:27:57.697346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:02.502261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.695472+0000","last_change":"2024-04-26T18:24:20.962274+0000","last_active":"2024-04-26T18:27:57.695472+0000","last_peered":"2024-04-26T18:27:57.695472+0000","last_clean":"2024-04-26T18:27:57.695472+0000","last_became_active":"2024-04-26T18:24:20.961920+0000","last_became_peered":"2024-04-26T18:24:20.961920+0000","last_unstale":"2024-04-26T18:27:57.695472+0000","last_undegraded":"2024-04-26T18:27:57.695472+0000","last_fullsized":"2024-04-26T18:27:57.695472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:49:46.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545742+0000","last_change":"2024-04-26T18:24:20.949096+0000","last_active":"2024-04-26T18:27:57.545742+0000","last_peered":"2024-04-26T18:27:57.545742+0000","last_clean":"2024-04-26T18:27:57.545742+0000","last_became_active":"2024-04-26T18:24:20.948737+0000","last_became_peered":"2024-04-26T18:24:20.948737+0000","last_unstale":"2024-04-26T18:27:57.545742+0000","last_undegraded":"2024-04-26T18:27:57.545742+0000","last_fullsized":"2024-04-26T18:27:57.545742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:12:40.371589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451728+0000","last_change":"2024-04-26T18:24:20.962670+0000","last_active":"2024-04-26T18:27:57.451728+0000","last_peered":"2024-04-26T18:27:57.451728+0000","last_clean":"2024-04-26T18:27:57.451728+0000","last_became_active":"2024-04-26T18:24:20.962309+0000","last_became_peered":"2024-04-26T18:24:20.962309+0000","last_unstale":"2024-04-26T18:27:57.451728+0000","last_undegraded":"2024-04-26T18:27:57.451728+0000","last_fullsized":"2024-04-26T18:27:57.451728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:40.315864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591196+0000","last_change":"2024-04-26T18:25:06.350579+0000","last_active":"2024-04-26T18:27:57.591196+0000","last_peered":"2024-04-26T18:27:57.591196+0000","last_clean":"2024-04-26T18:27:57.591196+0000","last_became_active":"2024-04-26T18:25:06.350104+0000","last_became_peered":"2024-04-26T18:25:06.350104+0000","last_unstale":"2024-04-26T18:27:57.591196+0000","last_undegraded":"2024-04-26T18:27:57.591196+0000","last_fullsized":"2024-04-26T18:27:57.591196+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:00.950883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452339+0000","last_change":"2024-04-26T18:24:20.961096+0000","last_active":"2024-04-26T18:27:57.452339+0000","last_peered":"2024-04-26T18:27:57.452339+0000","last_clean":"2024-04-26T18:27:57.452339+0000","last_became_active":"2024-04-26T18:24:20.960796+0000","last_became_peered":"2024-04-26T18:24:20.960796+0000","last_unstale":"2024-04-26T18:27:57.452339+0000","last_undegraded":"2024-04-26T18:27:57.452339+0000","last_fullsized":"2024-04-26T18:27:57.452339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:04.538836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589942+0000","last_change":"2024-04-26T18:24:20.959871+0000","last_active":"2024-04-26T18:27:57.589942+0000","last_peered":"2024-04-26T18:27:57.589942+0000","last_clean":"2024-04-26T18:27:57.589942+0000","last_became_active":"2024-04-26T18:24:20.959404+0000","last_became_peered":"2024-04-26T18:24:20.959404+0000","last_unstale":"2024-04-26T18:27:57.589942+0000","last_undegraded":"2024-04-26T18:27:57.589942+0000","last_fullsized":"2024-04-26T18:27:57.589942+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:43.053618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451941+0000","last_change":"2024-04-26T18:26:07.431543+0000","last_active":"2024-04-26T18:27:57.451941+0000","last_peered":"2024-04-26T18:27:57.451941+0000","last_clean":"2024-04-26T18:27:57.451941+0000","last_became_active":"2024-04-26T18:26:07.430976+0000","last_became_peered":"2024-04-26T18:26:07.430976+0000","last_unstale":"2024-04-26T18:27:57.451941+0000","last_undegraded":"2024-04-26T18:27:57.451941+0000","last_fullsized":"2024-04-26T18:27:57.451941+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:31.430010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591606+0000","last_change":"2024-04-26T18:24:20.952109+0000","last_active":"2024-04-26T18:27:57.591606+0000","last_peered":"2024-04-26T18:27:57.591606+0000","last_clean":"2024-04-26T18:27:57.591606+0000","last_became_active":"2024-04-26T18:24:20.951776+0000","last_became_peered":"2024-04-26T18:24:20.951776+0000","last_unstale":"2024-04-26T18:27:57.591606+0000","last_undegraded":"2024-04-26T18:27:57.591606+0000","last_fullsized":"2024-04-26T18:27:57.591606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:53.220246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.695451+0000","last_change":"2024-04-26T18:24:20.963965+0000","last_active":"2024-04-26T18:27:57.695451+0000","last_peered":"2024-04-26T18:27:57.695451+0000","last_clean":"2024-04-26T18:27:57.695451+0000","last_became_active":"2024-04-26T18:24:20.963648+0000","last_became_peered":"2024-04-26T18:24:20.963648+0000","last_unstale":"2024-04-26T18:27:57.695451+0000","last_undegraded":"2024-04-26T18:27:57.695451+0000","last_fullsized":"2024-04-26T18:27:57.695451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:04.080813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694857+0000","last_change":"2024-04-26T18:24:20.954186+0000","last_active":"2024-04-26T18:27:57.694857+0000","last_peered":"2024-04-26T18:27:57.694857+0000","last_clean":"2024-04-26T18:27:57.694857+0000","last_became_active":"2024-04-26T18:24:20.953431+0000","last_became_peered":"2024-04-26T18:24:20.953431+0000","last_unstale":"2024-04-26T18:27:57.694857+0000","last_undegraded":"2024-04-26T18:27:57.694857+0000","last_fullsized":"2024-04-26T18:27:57.694857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:21.498476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.696970+0000","last_change":"2024-04-26T18:24:20.945392+0000","last_active":"2024-04-26T18:27:57.696970+0000","last_peered":"2024-04-26T18:27:57.696970+0000","last_clean":"2024-04-26T18:27:57.696970+0000","last_became_active":"2024-04-26T18:24:20.944971+0000","last_became_peered":"2024-04-26T18:24:20.944971+0000","last_unstale":"2024-04-26T18:27:57.696970+0000","last_undegraded":"2024-04-26T18:27:57.696970+0000","last_fullsized":"2024-04-26T18:27:57.696970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:06.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697005+0000","last_change":"2024-04-26T18:25:06.299351+0000","last_active":"2024-04-26T18:27:57.697005+0000","last_peered":"2024-04-26T18:27:57.697005+0000","last_clean":"2024-04-26T18:27:57.697005+0000","last_became_active":"2024-04-26T18:25:06.298895+0000","last_became_peered":"2024-04-26T18:25:06.298895+0000","last_unstale":"2024-04-26T18:27:57.697005+0000","last_undegraded":"2024-04-26T18:27:57.697005+0000","last_fullsized":"2024-04-26T18:27:57.697005+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:37.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589273+0000","last_change":"2024-04-26T18:24:20.954610+0000","last_active":"2024-04-26T18:27:57.589273+0000","last_peered":"2024-04-26T18:27:57.589273+0000","last_clean":"2024-04-26T18:27:57.589273+0000","last_became_active":"2024-04-26T18:24:20.954239+0000","last_became_peered":"2024-04-26T18:24:20.954239+0000","last_unstale":"2024-04-26T18:27:57.589273+0000","last_undegraded":"2024-04-26T18:27:57.589273+0000","last_fullsized":"2024-04-26T18:27:57.589273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:41.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984496+0000","last_change":"2024-04-26T18:24:20.956932+0000","last_active":"2024-04-26T18:27:57.984496+0000","last_peered":"2024-04-26T18:27:57.984496+0000","last_clean":"2024-04-26T18:27:57.984496+0000","last_became_active":"2024-04-26T18:24:20.956464+0000","last_became_peered":"2024-04-26T18:24:20.956464+0000","last_unstale":"2024-04-26T18:27:57.984496+0000","last_undegraded":"2024-04-26T18:27:57.984496+0000","last_fullsized":"2024-04-26T18:27:57.984496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:01.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.543737+0000","last_change":"2024-04-26T18:24:20.954870+0000","last_active":"2024-04-26T18:27:57.543737+0000","last_peered":"2024-04-26T18:27:57.543737+0000","last_clean":"2024-04-26T18:27:57.543737+0000","last_became_active":"2024-04-26T18:24:20.954542+0000","last_became_peered":"2024-04-26T18:24:20.954542+0000","last_unstale":"2024-04-26T18:27:57.543737+0000","last_undegraded":"2024-04-26T18:27:57.543737+0000","last_fullsized":"2024-04-26T18:27:57.543737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:13:51.932625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452391+0000","last_change":"2024-04-26T18:24:20.950994+0000","last_active":"2024-04-26T18:27:57.452391+0000","last_peered":"2024-04-26T18:27:57.452391+0000","last_clean":"2024-04-26T18:27:57.452391+0000","last_became_active":"2024-04-26T18:24:20.950343+0000","last_became_peered":"2024-04-26T18:24:20.950343+0000","last_unstale":"2024-04-26T18:27:57.452391+0000","last_undegraded":"2024-04-26T18:27:57.452391+0000","last_fullsized":"2024-04-26T18:27:57.452391+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:25.274049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"94'14","reported_seq":252,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:01.596025+0000","last_change":"2024-04-26T18:24:17.929731+0000","last_active":"2024-04-26T18:28:01.596025+0000","last_peered":"2024-04-26T18:28:01.596025+0000","last_clean":"2024-04-26T18:28:01.596025+0000","last_became_active":"2024-04-26T18:24:17.929379+0000","last_became_peered":"2024-04-26T18:24:17.929379+0000","last_unstale":"2024-04-26T18:28:01.596025+0000","last_undegraded":"2024-04-26T18:28:01.596025+0000","last_fullsized":"2024-04-26T18:28:01.596025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:44.821660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":205,"num_read_kb":205,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452602+0000","last_change":"2024-04-26T18:24:20.962885+0000","last_active":"2024-04-26T18:27:57.452602+0000","last_peered":"2024-04-26T18:27:57.452602+0000","last_clean":"2024-04-26T18:27:57.452602+0000","last_became_active":"2024-04-26T18:24:20.962674+0000","last_became_peered":"2024-04-26T18:24:20.962674+0000","last_unstale":"2024-04-26T18:27:57.452602+0000","last_undegraded":"2024-04-26T18:27:57.452602+0000","last_fullsized":"2024-04-26T18:27:57.452602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:32.444889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452837+0000","last_change":"2024-04-26T18:24:18.141500+0000","last_active":"2024-04-26T18:27:57.452837+0000","last_peered":"2024-04-26T18:27:57.452837+0000","last_clean":"2024-04-26T18:27:57.452837+0000","last_became_active":"2024-04-26T18:24:18.140870+0000","last_became_peered":"2024-04-26T18:24:18.140870+0000","last_unstale":"2024-04-26T18:27:57.452837+0000","last_undegraded":"2024-04-26T18:27:57.452837+0000","last_fullsized":"2024-04-26T18:27:57.452837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:13.821872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697816+0000","last_change":"2024-04-26T18:24:20.949183+0000","last_active":"2024-04-26T18:27:57.697816+0000","last_peered":"2024-04-26T18:27:57.697816+0000","last_clean":"2024-04-26T18:27:57.697816+0000","last_became_active":"2024-04-26T18:24:20.948677+0000","last_became_peered":"2024-04-26T18:24:20.948677+0000","last_unstale":"2024-04-26T18:27:57.697816+0000","last_undegraded":"2024-04-26T18:27:57.697816+0000","last_fullsized":"2024-04-26T18:27:57.697816+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:55.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":68,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.546095+0000","last_change":"2024-04-26T18:25:06.312239+0000","last_active":"2024-04-26T18:27:57.546095+0000","last_peered":"2024-04-26T18:27:57.546095+0000","last_clean":"2024-04-26T18:27:57.546095+0000","last_became_active":"2024-04-26T18:25:06.301304+0000","last_became_peered":"2024-04-26T18:25:06.301304+0000","last_unstale":"2024-04-26T18:27:57.546095+0000","last_undegraded":"2024-04-26T18:27:57.546095+0000","last_fullsized":"2024-04-26T18:27:57.546095+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:36.159348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589460+0000","last_change":"2024-04-26T18:24:20.955555+0000","last_active":"2024-04-26T18:27:57.589460+0000","last_peered":"2024-04-26T18:27:57.589460+0000","last_clean":"2024-04-26T18:27:57.589460+0000","last_became_active":"2024-04-26T18:24:20.955176+0000","last_became_peered":"2024-04-26T18:24:20.955176+0000","last_unstale":"2024-04-26T18:27:57.589460+0000","last_undegraded":"2024-04-26T18:27:57.589460+0000","last_fullsized":"2024-04-26T18:27:57.589460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:02:30.368971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545963+0000","last_change":"2024-04-26T18:24:17.930919+0000","last_active":"2024-04-26T18:27:57.545963+0000","last_peered":"2024-04-26T18:27:57.545963+0000","last_clean":"2024-04-26T18:27:57.545963+0000","last_became_active":"2024-04-26T18:24:17.930596+0000","last_became_peered":"2024-04-26T18:24:17.930596+0000","last_unstale":"2024-04-26T18:27:57.545963+0000","last_undegraded":"2024-04-26T18:27:57.545963+0000","last_fullsized":"2024-04-26T18:27:57.545963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:57.476027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550891+0000","last_change":"2024-04-26T18:24:20.955796+0000","last_active":"2024-04-26T18:27:57.550891+0000","last_peered":"2024-04-26T18:27:57.550891+0000","last_clean":"2024-04-26T18:27:57.550891+0000","last_became_active":"2024-04-26T18:24:20.955432+0000","last_became_peered":"2024-04-26T18:24:20.955432+0000","last_unstale":"2024-04-26T18:27:57.550891+0000","last_undegraded":"2024-04-26T18:27:57.550891+0000","last_fullsized":"2024-04-26T18:27:57.550891+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:25.161797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"94'9","reported_seq":252,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:01.596448+0000","last_change":"2024-04-26T18:24:17.929505+0000","last_active":"2024-04-26T18:28:01.596448+0000","last_peered":"2024-04-26T18:28:01.596448+0000","last_clean":"2024-04-26T18:28:01.596448+0000","last_became_active":"2024-04-26T18:24:17.929205+0000","last_became_peered":"2024-04-26T18:24:17.929205+0000","last_unstale":"2024-04-26T18:28:01.596448+0000","last_undegraded":"2024-04-26T18:28:01.596448+0000","last_fullsized":"2024-04-26T18:28:01.596448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:42.460063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":210,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545699+0000","last_change":"2024-04-26T18:24:20.952134+0000","last_active":"2024-04-26T18:27:57.545699+0000","last_peered":"2024-04-26T18:27:57.545699+0000","last_clean":"2024-04-26T18:27:57.545699+0000","last_became_active":"2024-04-26T18:24:20.951758+0000","last_became_peered":"2024-04-26T18:24:20.951758+0000","last_unstale":"2024-04-26T18:27:57.545699+0000","last_undegraded":"2024-04-26T18:27:57.545699+0000","last_fullsized":"2024-04-26T18:27:57.545699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:25.389221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451604+0000","last_change":"2024-04-26T18:24:17.928704+0000","last_active":"2024-04-26T18:27:57.451604+0000","last_peered":"2024-04-26T18:27:57.451604+0000","last_clean":"2024-04-26T18:27:57.451604+0000","last_became_active":"2024-04-26T18:24:17.928343+0000","last_became_peered":"2024-04-26T18:24:17.928343+0000","last_unstale":"2024-04-26T18:27:57.451604+0000","last_undegraded":"2024-04-26T18:27:57.451604+0000","last_fullsized":"2024-04-26T18:27:57.451604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:49.400481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550744+0000","last_change":"2024-04-26T18:24:20.956808+0000","last_active":"2024-04-26T18:27:57.550744+0000","last_peered":"2024-04-26T18:27:57.550744+0000","last_clean":"2024-04-26T18:27:57.550744+0000","last_became_active":"2024-04-26T18:24:20.956466+0000","last_became_peered":"2024-04-26T18:24:20.956466+0000","last_unstale":"2024-04-26T18:27:57.550744+0000","last_undegraded":"2024-04-26T18:27:57.550744+0000","last_fullsized":"2024-04-26T18:27:57.550744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:31.150670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.693067+0000","last_change":"2024-04-26T18:24:17.929134+0000","last_active":"2024-04-26T18:27:57.693067+0000","last_peered":"2024-04-26T18:27:57.693067+0000","last_clean":"2024-04-26T18:27:57.693067+0000","last_became_active":"2024-04-26T18:24:17.928798+0000","last_became_peered":"2024-04-26T18:24:17.928798+0000","last_unstale":"2024-04-26T18:27:57.693067+0000","last_undegraded":"2024-04-26T18:27:57.693067+0000","last_fullsized":"2024-04-26T18:27:57.693067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:10.450825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452339+0000","last_change":"2024-04-26T18:24:20.951594+0000","last_active":"2024-04-26T18:27:57.452339+0000","last_peered":"2024-04-26T18:27:57.452339+0000","last_clean":"2024-04-26T18:27:57.452339+0000","last_became_active":"2024-04-26T18:24:20.951148+0000","last_became_peered":"2024-04-26T18:24:20.951148+0000","last_unstale":"2024-04-26T18:27:57.452339+0000","last_undegraded":"2024-04-26T18:27:57.452339+0000","last_fullsized":"2024-04-26T18:27:57.452339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:18.703292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550654+0000","last_change":"2024-04-26T18:24:17.931101+0000","last_active":"2024-04-26T18:27:57.550654+0000","last_peered":"2024-04-26T18:27:57.550654+0000","last_clean":"2024-04-26T18:27:57.550654+0000","last_became_active":"2024-04-26T18:24:17.930819+0000","last_became_peered":"2024-04-26T18:24:17.930819+0000","last_unstale":"2024-04-26T18:27:57.550654+0000","last_undegraded":"2024-04-26T18:27:57.550654+0000","last_fullsized":"2024-04-26T18:27:57.550654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.543387+0000","last_change":"2024-04-26T18:24:20.948866+0000","last_active":"2024-04-26T18:27:57.543387+0000","last_peered":"2024-04-26T18:27:57.543387+0000","last_clean":"2024-04-26T18:27:57.543387+0000","last_became_active":"2024-04-26T18:24:20.948562+0000","last_became_peered":"2024-04-26T18:24:20.948562+0000","last_unstale":"2024-04-26T18:27:57.543387+0000","last_undegraded":"2024-04-26T18:27:57.543387+0000","last_fullsized":"2024-04-26T18:27:57.543387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:33:19.616660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"94'8","reported_seq":448,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:02.596461+0000","last_change":"2024-04-26T18:24:17.926384+0000","last_active":"2024-04-26T18:28:02.596461+0000","last_peered":"2024-04-26T18:28:02.596461+0000","last_clean":"2024-04-26T18:28:02.596461+0000","last_became_active":"2024-04-26T18:24:17.926037+0000","last_became_peered":"2024-04-26T18:24:17.926037+0000","last_unstale":"2024-04-26T18:28:02.596461+0000","last_undegraded":"2024-04-26T18:28:02.596461+0000","last_fullsized":"2024-04-26T18:28:02.596461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:05.283755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":407,"num_read_kb":407,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589843+0000","last_change":"2024-04-26T18:24:20.954257+0000","last_active":"2024-04-26T18:27:57.589843+0000","last_peered":"2024-04-26T18:27:57.589843+0000","last_clean":"2024-04-26T18:27:57.589843+0000","last_became_active":"2024-04-26T18:24:20.953913+0000","last_became_peered":"2024-04-26T18:24:20.953913+0000","last_unstale":"2024-04-26T18:27:57.589843+0000","last_undegraded":"2024-04-26T18:27:57.589843+0000","last_fullsized":"2024-04-26T18:27:57.589843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:06.814156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591540+0000","last_change":"2024-04-26T18:24:17.932843+0000","last_active":"2024-04-26T18:27:57.591540+0000","last_peered":"2024-04-26T18:27:57.591540+0000","last_clean":"2024-04-26T18:27:57.591540+0000","last_became_active":"2024-04-26T18:24:17.932502+0000","last_became_peered":"2024-04-26T18:24:17.932502+0000","last_unstale":"2024-04-26T18:27:57.591540+0000","last_undegraded":"2024-04-26T18:27:57.591540+0000","last_fullsized":"2024-04-26T18:27:57.591540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:49.727094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697781+0000","last_change":"2024-04-26T18:24:20.959881+0000","last_active":"2024-04-26T18:27:57.697781+0000","last_peered":"2024-04-26T18:27:57.697781+0000","last_clean":"2024-04-26T18:27:57.697781+0000","last_became_active":"2024-04-26T18:24:20.959541+0000","last_became_peered":"2024-04-26T18:24:20.959541+0000","last_unstale":"2024-04-26T18:27:57.697781+0000","last_undegraded":"2024-04-26T18:27:57.697781+0000","last_fullsized":"2024-04-26T18:27:57.697781+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:23.393521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.551017+0000","last_change":"2024-04-26T18:24:18.142146+0000","last_active":"2024-04-26T18:27:57.551017+0000","last_peered":"2024-04-26T18:27:57.551017+0000","last_clean":"2024-04-26T18:27:57.551017+0000","last_became_active":"2024-04-26T18:24:18.141829+0000","last_became_peered":"2024-04-26T18:24:18.141829+0000","last_unstale":"2024-04-26T18:27:57.551017+0000","last_undegraded":"2024-04-26T18:27:57.551017+0000","last_fullsized":"2024-04-26T18:27:57.551017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:49.341638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589611+0000","last_change":"2024-04-26T18:24:20.957993+0000","last_active":"2024-04-26T18:27:57.589611+0000","last_peered":"2024-04-26T18:27:57.589611+0000","last_clean":"2024-04-26T18:27:57.589611+0000","last_became_active":"2024-04-26T18:24:20.957443+0000","last_became_peered":"2024-04-26T18:24:20.957443+0000","last_unstale":"2024-04-26T18:27:57.589611+0000","last_undegraded":"2024-04-26T18:27:57.589611+0000","last_fullsized":"2024-04-26T18:27:57.589611+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:06:32.960546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'45","reported_seq":500,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:02.597472+0000","last_change":"2024-04-26T18:24:17.931283+0000","last_active":"2024-04-26T18:28:02.597472+0000","last_peered":"2024-04-26T18:28:02.597472+0000","last_clean":"2024-04-26T18:28:02.597472+0000","last_became_active":"2024-04-26T18:24:17.930943+0000","last_became_peered":"2024-04-26T18:24:17.930943+0000","last_unstale":"2024-04-26T18:28:02.597472+0000","last_undegraded":"2024-04-26T18:28:02.597472+0000","last_fullsized":"2024-04-26T18:28:02.597472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:50.511677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75785,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":422,"num_read_kb":285,"num_write":47,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545864+0000","last_change":"2024-04-26T18:24:20.947509+0000","last_active":"2024-04-26T18:27:57.545864+0000","last_peered":"2024-04-26T18:27:57.545864+0000","last_clean":"2024-04-26T18:27:57.545864+0000","last_became_active":"2024-04-26T18:24:20.947181+0000","last_became_peered":"2024-04-26T18:24:20.947181+0000","last_unstale":"2024-04-26T18:27:57.545864+0000","last_undegraded":"2024-04-26T18:27:57.545864+0000","last_fullsized":"2024-04-26T18:27:57.545864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.262312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452359+0000","last_change":"2024-04-26T18:24:17.934042+0000","last_active":"2024-04-26T18:27:57.452359+0000","last_peered":"2024-04-26T18:27:57.452359+0000","last_clean":"2024-04-26T18:27:57.452359+0000","last_became_active":"2024-04-26T18:24:17.933537+0000","last_became_peered":"2024-04-26T18:24:17.933537+0000","last_unstale":"2024-04-26T18:27:57.452359+0000","last_undegraded":"2024-04-26T18:27:57.452359+0000","last_fullsized":"2024-04-26T18:27:57.452359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:35:06.904160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.692822+0000","last_change":"2024-04-26T18:24:20.957672+0000","last_active":"2024-04-26T18:27:57.692822+0000","last_peered":"2024-04-26T18:27:57.692822+0000","last_clean":"2024-04-26T18:27:57.692822+0000","last_became_active":"2024-04-26T18:24:20.957214+0000","last_became_peered":"2024-04-26T18:24:20.957214+0000","last_unstale":"2024-04-26T18:27:57.692822+0000","last_undegraded":"2024-04-26T18:27:57.692822+0000","last_fullsized":"2024-04-26T18:27:57.692822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:42:35.341858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550631+0000","last_change":"2024-04-26T18:24:17.930324+0000","last_active":"2024-04-26T18:27:57.550631+0000","last_peered":"2024-04-26T18:27:57.550631+0000","last_clean":"2024-04-26T18:27:57.550631+0000","last_became_active":"2024-04-26T18:24:17.930022+0000","last_became_peered":"2024-04-26T18:24:17.930022+0000","last_unstale":"2024-04-26T18:27:57.550631+0000","last_undegraded":"2024-04-26T18:27:57.550631+0000","last_fullsized":"2024-04-26T18:27:57.550631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:36.864816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591382+0000","last_change":"2024-04-26T18:25:06.350388+0000","last_active":"2024-04-26T18:27:57.591382+0000","last_peered":"2024-04-26T18:27:57.591382+0000","last_clean":"2024-04-26T18:27:57.591382+0000","last_became_active":"2024-04-26T18:25:06.349836+0000","last_became_peered":"2024-04-26T18:25:06.349836+0000","last_unstale":"2024-04-26T18:27:57.591382+0000","last_undegraded":"2024-04-26T18:27:57.591382+0000","last_fullsized":"2024-04-26T18:27:57.591382+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:57.762115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'8","reported_seq":249,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:01.701789+0000","last_change":"2024-04-26T18:24:17.934676+0000","last_active":"2024-04-26T18:28:01.701789+0000","last_peered":"2024-04-26T18:28:01.701789+0000","last_clean":"2024-04-26T18:28:01.701789+0000","last_became_active":"2024-04-26T18:24:17.934307+0000","last_became_peered":"2024-04-26T18:24:17.934307+0000","last_unstale":"2024-04-26T18:28:01.701789+0000","last_undegraded":"2024-04-26T18:28:01.701789+0000","last_fullsized":"2024-04-26T18:28:01.701789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:27.312248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1761,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":209,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591082+0000","last_change":"2024-04-26T18:24:20.947371+0000","last_active":"2024-04-26T18:27:57.591082+0000","last_peered":"2024-04-26T18:27:57.591082+0000","last_clean":"2024-04-26T18:27:57.591082+0000","last_became_active":"2024-04-26T18:24:20.947048+0000","last_became_peered":"2024-04-26T18:24:20.947048+0000","last_unstale":"2024-04-26T18:27:57.591082+0000","last_undegraded":"2024-04-26T18:27:57.591082+0000","last_fullsized":"2024-04-26T18:27:57.591082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:55.038074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545656+0000","last_change":"2024-04-26T18:24:18.142680+0000","last_active":"2024-04-26T18:27:57.545656+0000","last_peered":"2024-04-26T18:27:57.545656+0000","last_clean":"2024-04-26T18:27:57.545656+0000","last_became_active":"2024-04-26T18:24:18.142392+0000","last_became_peered":"2024-04-26T18:24:18.142392+0000","last_unstale":"2024-04-26T18:27:57.545656+0000","last_undegraded":"2024-04-26T18:27:57.545656+0000","last_fullsized":"2024-04-26T18:27:57.545656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:52.654813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550596+0000","last_change":"2024-04-26T18:24:20.958131+0000","last_active":"2024-04-26T18:27:57.550596+0000","last_peered":"2024-04-26T18:27:57.550596+0000","last_clean":"2024-04-26T18:27:57.550596+0000","last_became_active":"2024-04-26T18:24:20.957701+0000","last_became_peered":"2024-04-26T18:24:20.957701+0000","last_unstale":"2024-04-26T18:27:57.550596+0000","last_undegraded":"2024-04-26T18:27:57.550596+0000","last_fullsized":"2024-04-26T18:27:57.550596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:02.018158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984426+0000","last_change":"2024-04-26T18:25:06.319173+0000","last_active":"2024-04-26T18:27:57.984426+0000","last_peered":"2024-04-26T18:27:57.984426+0000","last_clean":"2024-04-26T18:27:57.984426+0000","last_became_active":"2024-04-26T18:25:06.302935+0000","last_became_peered":"2024-04-26T18:25:06.302935+0000","last_unstale":"2024-04-26T18:27:57.984426+0000","last_undegraded":"2024-04-26T18:27:57.984426+0000","last_fullsized":"2024-04-26T18:27:57.984426+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:40.198105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984341+0000","last_change":"2024-04-26T18:24:20.961230+0000","last_active":"2024-04-26T18:27:57.984341+0000","last_peered":"2024-04-26T18:27:57.984341+0000","last_clean":"2024-04-26T18:27:57.984341+0000","last_became_active":"2024-04-26T18:24:20.960807+0000","last_became_peered":"2024-04-26T18:24:20.960807+0000","last_unstale":"2024-04-26T18:27:57.984341+0000","last_undegraded":"2024-04-26T18:27:57.984341+0000","last_fullsized":"2024-04-26T18:27:57.984341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:39.535100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.543569+0000","last_change":"2024-04-26T18:24:18.142082+0000","last_active":"2024-04-26T18:27:57.543569+0000","last_peered":"2024-04-26T18:27:57.543569+0000","last_clean":"2024-04-26T18:27:57.543569+0000","last_became_active":"2024-04-26T18:24:18.141765+0000","last_became_peered":"2024-04-26T18:24:18.141765+0000","last_unstale":"2024-04-26T18:27:57.543569+0000","last_undegraded":"2024-04-26T18:27:57.543569+0000","last_fullsized":"2024-04-26T18:27:57.543569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:02.140027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451762+0000","last_change":"2024-04-26T18:24:20.952867+0000","last_active":"2024-04-26T18:27:57.451762+0000","last_peered":"2024-04-26T18:27:57.451762+0000","last_clean":"2024-04-26T18:27:57.451762+0000","last_became_active":"2024-04-26T18:24:20.952525+0000","last_became_peered":"2024-04-26T18:24:20.952525+0000","last_unstale":"2024-04-26T18:27:57.451762+0000","last_undegraded":"2024-04-26T18:27:57.451762+0000","last_fullsized":"2024-04-26T18:27:57.451762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:52.804312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'6","reported_seq":245,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:01.596387+0000","last_change":"2024-04-26T18:24:17.932270+0000","last_active":"2024-04-26T18:28:01.596387+0000","last_peered":"2024-04-26T18:28:01.596387+0000","last_clean":"2024-04-26T18:28:01.596387+0000","last_became_active":"2024-04-26T18:24:17.931901+0000","last_became_peered":"2024-04-26T18:24:17.931901+0000","last_unstale":"2024-04-26T18:28:01.596387+0000","last_undegraded":"2024-04-26T18:28:01.596387+0000","last_fullsized":"2024-04-26T18:28:01.596387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:13.524174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":206,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451763+0000","last_change":"2024-04-26T18:24:20.960612+0000","last_active":"2024-04-26T18:27:57.451763+0000","last_peered":"2024-04-26T18:27:57.451763+0000","last_clean":"2024-04-26T18:27:57.451763+0000","last_became_active":"2024-04-26T18:24:20.960280+0000","last_became_peered":"2024-04-26T18:24:20.960280+0000","last_unstale":"2024-04-26T18:27:57.451763+0000","last_undegraded":"2024-04-26T18:27:57.451763+0000","last_fullsized":"2024-04-26T18:27:57.451763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:53.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.543368+0000","last_change":"2024-04-26T18:25:06.299736+0000","last_active":"2024-04-26T18:27:57.543368+0000","last_peered":"2024-04-26T18:27:57.543368+0000","last_clean":"2024-04-26T18:27:57.543368+0000","last_became_active":"2024-04-26T18:25:06.299265+0000","last_became_peered":"2024-04-26T18:25:06.299265+0000","last_unstale":"2024-04-26T18:27:57.543368+0000","last_undegraded":"2024-04-26T18:27:57.543368+0000","last_fullsized":"2024-04-26T18:27:57.543368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:32.427551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":244,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:01.701339+0000","last_change":"2024-04-26T18:24:18.141181+0000","last_active":"2024-04-26T18:28:01.701339+0000","last_peered":"2024-04-26T18:28:01.701339+0000","last_clean":"2024-04-26T18:28:01.701339+0000","last_became_active":"2024-04-26T18:24:18.140854+0000","last_became_peered":"2024-04-26T18:24:18.140854+0000","last_unstale":"2024-04-26T18:28:01.701339+0000","last_undegraded":"2024-04-26T18:28:01.701339+0000","last_fullsized":"2024-04-26T18:28:01.701339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:27.900701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":205,"num_read_kb":205,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452738+0000","last_change":"2024-04-26T18:24:20.961653+0000","last_active":"2024-04-26T18:27:57.452738+0000","last_peered":"2024-04-26T18:27:57.452738+0000","last_clean":"2024-04-26T18:27:57.452738+0000","last_became_active":"2024-04-26T18:24:20.961330+0000","last_became_peered":"2024-04-26T18:24:20.961330+0000","last_unstale":"2024-04-26T18:27:57.452738+0000","last_undegraded":"2024-04-26T18:27:57.452738+0000","last_fullsized":"2024-04-26T18:27:57.452738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:19.033932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.695409+0000","last_change":"2024-04-26T18:24:17.933613+0000","last_active":"2024-04-26T18:27:57.695409+0000","last_peered":"2024-04-26T18:27:57.695409+0000","last_clean":"2024-04-26T18:27:57.695409+0000","last_became_active":"2024-04-26T18:24:17.933219+0000","last_became_peered":"2024-04-26T18:24:17.933219+0000","last_unstale":"2024-04-26T18:27:57.695409+0000","last_undegraded":"2024-04-26T18:27:57.695409+0000","last_fullsized":"2024-04-26T18:27:57.695409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:04.215821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"94'148","reported_seq":410,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:59.571396+0000","last_change":"2024-04-26T18:22:57.585634+0000","last_active":"2024-04-26T18:27:59.571396+0000","last_peered":"2024-04-26T18:27:59.571396+0000","last_clean":"2024-04-26T18:27:59.571396+0000","last_became_active":"2024-04-26T18:22:57.585198+0000","last_became_peered":"2024-04-26T18:22:57.585198+0000","last_unstale":"2024-04-26T18:27:59.571396+0000","last_undegraded":"2024-04-26T18:27:59.571396+0000","last_fullsized":"2024-04-26T18:27:59.571396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_clean_scrub_stamp":"2024-04-26T18:19:31.594684+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:56:11.861930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.692768+0000","last_change":"2024-04-26T18:24:20.950303+0000","last_active":"2024-04-26T18:27:57.692768+0000","last_peered":"2024-04-26T18:27:57.692768+0000","last_clean":"2024-04-26T18:27:57.692768+0000","last_became_active":"2024-04-26T18:24:20.950002+0000","last_became_peered":"2024-04-26T18:24:20.950002+0000","last_unstale":"2024-04-26T18:27:57.692768+0000","last_undegraded":"2024-04-26T18:27:57.692768+0000","last_fullsized":"2024-04-26T18:27:57.692768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:00.912752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591327+0000","last_change":"2024-04-26T18:24:17.934924+0000","last_active":"2024-04-26T18:27:57.591327+0000","last_peered":"2024-04-26T18:27:57.591327+0000","last_clean":"2024-04-26T18:27:57.591327+0000","last_became_active":"2024-04-26T18:24:17.934598+0000","last_became_peered":"2024-04-26T18:24:17.934598+0000","last_unstale":"2024-04-26T18:27:57.591327+0000","last_undegraded":"2024-04-26T18:27:57.591327+0000","last_fullsized":"2024-04-26T18:27:57.591327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:28.690043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545940+0000","last_change":"2024-04-26T18:24:18.142560+0000","last_active":"2024-04-26T18:27:57.545940+0000","last_peered":"2024-04-26T18:27:57.545940+0000","last_clean":"2024-04-26T18:27:57.545940+0000","last_became_active":"2024-04-26T18:24:18.142184+0000","last_became_peered":"2024-04-26T18:24:18.142184+0000","last_unstale":"2024-04-26T18:27:57.545940+0000","last_undegraded":"2024-04-26T18:27:57.545940+0000","last_fullsized":"2024-04-26T18:27:57.545940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:26.656113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694369+0000","last_change":"2024-04-26T18:24:20.952073+0000","last_active":"2024-04-26T18:27:57.694369+0000","last_peered":"2024-04-26T18:27:57.694369+0000","last_clean":"2024-04-26T18:27:57.694369+0000","last_became_active":"2024-04-26T18:24:20.951761+0000","last_became_peered":"2024-04-26T18:24:20.951761+0000","last_unstale":"2024-04-26T18:27:57.694369+0000","last_undegraded":"2024-04-26T18:27:57.694369+0000","last_fullsized":"2024-04-26T18:27:57.694369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:30.244060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'128","reported_seq":286,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:00.463784+0000","last_change":"2024-04-26T18:24:17.927171+0000","last_active":"2024-04-26T18:28:00.463784+0000","last_peered":"2024-04-26T18:28:00.463784+0000","last_clean":"2024-04-26T18:28:00.463784+0000","last_became_active":"2024-04-26T18:24:17.926816+0000","last_became_peered":"2024-04-26T18:24:17.926816+0000","last_unstale":"2024-04-26T18:28:00.463784+0000","last_undegraded":"2024-04-26T18:28:00.463784+0000","last_fullsized":"2024-04-26T18:28:00.463784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:18.249363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":422,"num_read_kb":376,"num_write":236,"num_write_kb":1414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452834+0000","last_change":"2024-04-26T18:24:20.951684+0000","last_active":"2024-04-26T18:27:57.452834+0000","last_peered":"2024-04-26T18:27:57.452834+0000","last_clean":"2024-04-26T18:27:57.452834+0000","last_became_active":"2024-04-26T18:24:20.951364+0000","last_became_peered":"2024-04-26T18:24:20.951364+0000","last_unstale":"2024-04-26T18:27:57.452834+0000","last_undegraded":"2024-04-26T18:27:57.452834+0000","last_fullsized":"2024-04-26T18:27:57.452834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:28.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.695676+0000","last_change":"2024-04-26T18:24:18.229770+0000","last_active":"2024-04-26T18:27:57.695676+0000","last_peered":"2024-04-26T18:27:57.695676+0000","last_clean":"2024-04-26T18:27:57.695676+0000","last_became_active":"2024-04-26T18:24:18.229446+0000","last_became_peered":"2024-04-26T18:24:18.229446+0000","last_unstale":"2024-04-26T18:27:57.695676+0000","last_undegraded":"2024-04-26T18:27:57.695676+0000","last_fullsized":"2024-04-26T18:27:57.695676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:14.794797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697561+0000","last_change":"2024-04-26T18:24:20.960513+0000","last_active":"2024-04-26T18:27:57.697561+0000","last_peered":"2024-04-26T18:27:57.697561+0000","last_clean":"2024-04-26T18:27:57.697561+0000","last_became_active":"2024-04-26T18:24:20.959549+0000","last_became_peered":"2024-04-26T18:24:20.959549+0000","last_unstale":"2024-04-26T18:27:57.697561+0000","last_undegraded":"2024-04-26T18:27:57.697561+0000","last_fullsized":"2024-04-26T18:27:57.697561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:18:37.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589273+0000","last_change":"2024-04-26T18:24:18.141549+0000","last_active":"2024-04-26T18:27:57.589273+0000","last_peered":"2024-04-26T18:27:57.589273+0000","last_clean":"2024-04-26T18:27:57.589273+0000","last_became_active":"2024-04-26T18:24:18.141172+0000","last_became_peered":"2024-04-26T18:24:18.141172+0000","last_unstale":"2024-04-26T18:27:57.589273+0000","last_undegraded":"2024-04-26T18:27:57.589273+0000","last_fullsized":"2024-04-26T18:27:57.589273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:15.878130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545672+0000","last_change":"2024-04-26T18:24:20.948121+0000","last_active":"2024-04-26T18:27:57.545672+0000","last_peered":"2024-04-26T18:27:57.545672+0000","last_clean":"2024-04-26T18:27:57.545672+0000","last_became_active":"2024-04-26T18:24:20.947738+0000","last_became_peered":"2024-04-26T18:24:20.947738+0000","last_unstale":"2024-04-26T18:27:57.545672+0000","last_undegraded":"2024-04-26T18:27:57.545672+0000","last_fullsized":"2024-04-26T18:27:57.545672+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:04.237302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452547+0000","last_change":"2024-04-26T18:24:17.926631+0000","last_active":"2024-04-26T18:27:57.452547+0000","last_peered":"2024-04-26T18:27:57.452547+0000","last_clean":"2024-04-26T18:27:57.452547+0000","last_became_active":"2024-04-26T18:24:17.926259+0000","last_became_peered":"2024-04-26T18:24:17.926259+0000","last_unstale":"2024-04-26T18:27:57.452547+0000","last_undegraded":"2024-04-26T18:27:57.452547+0000","last_fullsized":"2024-04-26T18:27:57.452547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:47.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984177+0000","last_change":"2024-04-26T18:24:20.947283+0000","last_active":"2024-04-26T18:27:57.984177+0000","last_peered":"2024-04-26T18:27:57.984177+0000","last_clean":"2024-04-26T18:27:57.984177+0000","last_became_active":"2024-04-26T18:24:20.946812+0000","last_became_peered":"2024-04-26T18:24:20.946812+0000","last_unstale":"2024-04-26T18:27:57.984177+0000","last_undegraded":"2024-04-26T18:27:57.984177+0000","last_fullsized":"2024-04-26T18:27:57.984177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:33.730698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"94'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452339+0000","last_change":"2024-04-26T18:24:17.931843+0000","last_active":"2024-04-26T18:27:57.452339+0000","last_peered":"2024-04-26T18:27:57.452339+0000","last_clean":"2024-04-26T18:27:57.452339+0000","last_became_active":"2024-04-26T18:24:17.931442+0000","last_became_peered":"2024-04-26T18:24:17.931442+0000","last_unstale":"2024-04-26T18:27:57.452339+0000","last_undegraded":"2024-04-26T18:27:57.452339+0000","last_fullsized":"2024-04-26T18:27:57.452339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:09.649435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452567+0000","last_change":"2024-04-26T18:24:20.955661+0000","last_active":"2024-04-26T18:27:57.452567+0000","last_peered":"2024-04-26T18:27:57.452567+0000","last_clean":"2024-04-26T18:27:57.452567+0000","last_became_active":"2024-04-26T18:24:20.955344+0000","last_became_peered":"2024-04-26T18:24:20.955344+0000","last_unstale":"2024-04-26T18:27:57.452567+0000","last_undegraded":"2024-04-26T18:27:57.452567+0000","last_fullsized":"2024-04-26T18:27:57.452567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:44.274381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984200+0000","last_change":"2024-04-26T18:25:06.315466+0000","last_active":"2024-04-26T18:27:57.984200+0000","last_peered":"2024-04-26T18:27:57.984200+0000","last_clean":"2024-04-26T18:27:57.984200+0000","last_became_active":"2024-04-26T18:25:06.301788+0000","last_became_peered":"2024-04-26T18:25:06.301788+0000","last_unstale":"2024-04-26T18:27:57.984200+0000","last_undegraded":"2024-04-26T18:27:57.984200+0000","last_fullsized":"2024-04-26T18:27:57.984200+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:12.998583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591736+0000","last_change":"2024-04-26T18:24:20.955746+0000","last_active":"2024-04-26T18:27:57.591736+0000","last_peered":"2024-04-26T18:27:57.591736+0000","last_clean":"2024-04-26T18:27:57.591736+0000","last_became_active":"2024-04-26T18:24:20.953652+0000","last_became_peered":"2024-04-26T18:24:20.953652+0000","last_unstale":"2024-04-26T18:27:57.591736+0000","last_undegraded":"2024-04-26T18:27:57.591736+0000","last_fullsized":"2024-04-26T18:27:57.591736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:31.414185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":207,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:01.595662+0000","last_change":"2024-04-26T18:25:06.309393+0000","last_active":"2024-04-26T18:28:01.595662+0000","last_peered":"2024-04-26T18:28:01.595662+0000","last_clean":"2024-04-26T18:28:01.595662+0000","last_became_active":"2024-04-26T18:25:06.302434+0000","last_became_peered":"2024-04-26T18:25:06.302434+0000","last_unstale":"2024-04-26T18:28:01.595662+0000","last_undegraded":"2024-04-26T18:28:01.595662+0000","last_fullsized":"2024-04-26T18:28:01.595662+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:54.730295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591717+0000","last_change":"2024-04-26T18:24:20.954238+0000","last_active":"2024-04-26T18:27:57.591717+0000","last_peered":"2024-04-26T18:27:57.591717+0000","last_clean":"2024-04-26T18:27:57.591717+0000","last_became_active":"2024-04-26T18:24:20.953883+0000","last_became_peered":"2024-04-26T18:24:20.953883+0000","last_unstale":"2024-04-26T18:27:57.591717+0000","last_undegraded":"2024-04-26T18:27:57.591717+0000","last_fullsized":"2024-04-26T18:27:57.591717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:10.580835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'75","reported_seq":150,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:00.459215+0000","last_change":"2024-04-26T18:24:17.926901+0000","last_active":"2024-04-26T18:28:00.459215+0000","last_peered":"2024-04-26T18:28:00.459215+0000","last_clean":"2024-04-26T18:28:00.459215+0000","last_became_active":"2024-04-26T18:24:17.926291+0000","last_became_peered":"2024-04-26T18:24:17.926291+0000","last_unstale":"2024-04-26T18:28:00.459215+0000","last_undegraded":"2024-04-26T18:28:00.459215+0000","last_fullsized":"2024-04-26T18:28:00.459215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:17.695198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":44,"num_write":84,"num_write_kb":1075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591253+0000","last_change":"2024-04-26T18:24:20.950578+0000","last_active":"2024-04-26T18:27:57.591253+0000","last_peered":"2024-04-26T18:27:57.591253+0000","last_clean":"2024-04-26T18:27:57.591253+0000","last_became_active":"2024-04-26T18:24:20.950174+0000","last_became_peered":"2024-04-26T18:24:20.950174+0000","last_unstale":"2024-04-26T18:27:57.591253+0000","last_undegraded":"2024-04-26T18:27:57.591253+0000","last_fullsized":"2024-04-26T18:27:57.591253+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694486+0000","last_change":"2024-04-26T18:24:17.928054+0000","last_active":"2024-04-26T18:27:57.694486+0000","last_peered":"2024-04-26T18:27:57.694486+0000","last_clean":"2024-04-26T18:27:57.694486+0000","last_became_active":"2024-04-26T18:24:17.927715+0000","last_became_peered":"2024-04-26T18:24:17.927715+0000","last_unstale":"2024-04-26T18:27:57.694486+0000","last_undegraded":"2024-04-26T18:27:57.694486+0000","last_fullsized":"2024-04-26T18:27:57.694486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.155436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591822+0000","last_change":"2024-04-26T18:24:20.944512+0000","last_active":"2024-04-26T18:27:57.591822+0000","last_peered":"2024-04-26T18:27:57.591822+0000","last_clean":"2024-04-26T18:27:57.591822+0000","last_became_active":"2024-04-26T18:24:20.944123+0000","last_became_peered":"2024-04-26T18:24:20.944123+0000","last_unstale":"2024-04-26T18:27:57.591822+0000","last_undegraded":"2024-04-26T18:27:57.591822+0000","last_fullsized":"2024-04-26T18:27:57.591822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.240206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984272+0000","last_change":"2024-04-26T18:24:20.959138+0000","last_active":"2024-04-26T18:27:57.984272+0000","last_peered":"2024-04-26T18:27:57.984272+0000","last_clean":"2024-04-26T18:27:57.984272+0000","last_became_active":"2024-04-26T18:24:20.958675+0000","last_became_peered":"2024-04-26T18:24:20.958675+0000","last_unstale":"2024-04-26T18:27:57.984272+0000","last_undegraded":"2024-04-26T18:27:57.984272+0000","last_fullsized":"2024-04-26T18:27:57.984272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:34:32.103380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452173+0000","last_change":"2024-04-26T18:24:20.961689+0000","last_active":"2024-04-26T18:27:57.452173+0000","last_peered":"2024-04-26T18:27:57.452173+0000","last_clean":"2024-04-26T18:27:57.452173+0000","last_became_active":"2024-04-26T18:24:20.961374+0000","last_became_peered":"2024-04-26T18:24:20.961374+0000","last_unstale":"2024-04-26T18:27:57.452173+0000","last_undegraded":"2024-04-26T18:27:57.452173+0000","last_fullsized":"2024-04-26T18:27:57.452173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:55.243887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452150+0000","last_change":"2024-04-26T18:24:20.950711+0000","last_active":"2024-04-26T18:27:57.452150+0000","last_peered":"2024-04-26T18:27:57.452150+0000","last_clean":"2024-04-26T18:27:57.452150+0000","last_became_active":"2024-04-26T18:24:20.950371+0000","last_became_peered":"2024-04-26T18:24:20.950371+0000","last_unstale":"2024-04-26T18:27:57.452150+0000","last_undegraded":"2024-04-26T18:27:57.452150+0000","last_fullsized":"2024-04-26T18:27:57.452150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:35.307921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697143+0000","last_change":"2024-04-26T18:24:20.959990+0000","last_active":"2024-04-26T18:27:57.697143+0000","last_peered":"2024-04-26T18:27:57.697143+0000","last_clean":"2024-04-26T18:27:57.697143+0000","last_became_active":"2024-04-26T18:24:20.959514+0000","last_became_peered":"2024-04-26T18:24:20.959514+0000","last_unstale":"2024-04-26T18:27:57.697143+0000","last_undegraded":"2024-04-26T18:27:57.697143+0000","last_fullsized":"2024-04-26T18:27:57.697143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:00.184448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694549+0000","last_change":"2024-04-26T18:24:20.957338+0000","last_active":"2024-04-26T18:27:57.694549+0000","last_peered":"2024-04-26T18:27:57.694549+0000","last_clean":"2024-04-26T18:27:57.694549+0000","last_became_active":"2024-04-26T18:24:20.956999+0000","last_became_peered":"2024-04-26T18:24:20.956999+0000","last_unstale":"2024-04-26T18:27:57.694549+0000","last_undegraded":"2024-04-26T18:27:57.694549+0000","last_fullsized":"2024-04-26T18:27:57.694549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.066902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984324+0000","last_change":"2024-04-26T18:24:20.953634+0000","last_active":"2024-04-26T18:27:57.984324+0000","last_peered":"2024-04-26T18:27:57.984324+0000","last_clean":"2024-04-26T18:27:57.984324+0000","last_became_active":"2024-04-26T18:24:20.953259+0000","last_became_peered":"2024-04-26T18:24:20.953259+0000","last_unstale":"2024-04-26T18:27:57.984324+0000","last_undegraded":"2024-04-26T18:27:57.984324+0000","last_fullsized":"2024-04-26T18:27:57.984324+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:00.095344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452650+0000","last_change":"2024-04-26T18:24:20.954997+0000","last_active":"2024-04-26T18:27:57.452650+0000","last_peered":"2024-04-26T18:27:57.452650+0000","last_clean":"2024-04-26T18:27:57.452650+0000","last_became_active":"2024-04-26T18:24:20.954223+0000","last_became_peered":"2024-04-26T18:24:20.954223+0000","last_unstale":"2024-04-26T18:27:57.452650+0000","last_undegraded":"2024-04-26T18:27:57.452650+0000","last_fullsized":"2024-04-26T18:27:57.452650+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:14.092455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452832+0000","last_change":"2024-04-26T18:24:20.957056+0000","last_active":"2024-04-26T18:27:57.452832+0000","last_peered":"2024-04-26T18:27:57.452832+0000","last_clean":"2024-04-26T18:27:57.452832+0000","last_became_active":"2024-04-26T18:24:20.956737+0000","last_became_peered":"2024-04-26T18:24:20.956737+0000","last_unstale":"2024-04-26T18:27:57.452832+0000","last_undegraded":"2024-04-26T18:27:57.452832+0000","last_fullsized":"2024-04-26T18:27:57.452832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:48.358862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550741+0000","last_change":"2024-04-26T18:25:06.350206+0000","last_active":"2024-04-26T18:27:57.550741+0000","last_peered":"2024-04-26T18:27:57.550741+0000","last_clean":"2024-04-26T18:27:57.550741+0000","last_became_active":"2024-04-26T18:25:06.349866+0000","last_became_peered":"2024-04-26T18:25:06.349866+0000","last_unstale":"2024-04-26T18:27:57.550741+0000","last_undegraded":"2024-04-26T18:27:57.550741+0000","last_fullsized":"2024-04-26T18:27:57.550741+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:41.909056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452351+0000","last_change":"2024-04-26T18:24:20.956000+0000","last_active":"2024-04-26T18:27:57.452351+0000","last_peered":"2024-04-26T18:27:57.452351+0000","last_clean":"2024-04-26T18:27:57.452351+0000","last_became_active":"2024-04-26T18:24:20.955677+0000","last_became_peered":"2024-04-26T18:24:20.955677+0000","last_unstale":"2024-04-26T18:27:57.452351+0000","last_undegraded":"2024-04-26T18:27:57.452351+0000","last_fullsized":"2024-04-26T18:27:57.452351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:20.895674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589421+0000","last_change":"2024-04-26T18:24:20.959317+0000","last_active":"2024-04-26T18:27:57.589421+0000","last_peered":"2024-04-26T18:27:57.589421+0000","last_clean":"2024-04-26T18:27:57.589421+0000","last_became_active":"2024-04-26T18:24:20.958963+0000","last_became_peered":"2024-04-26T18:24:20.958963+0000","last_unstale":"2024-04-26T18:27:57.589421+0000","last_undegraded":"2024-04-26T18:27:57.589421+0000","last_fullsized":"2024-04-26T18:27:57.589421+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:08.231089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694689+0000","last_change":"2024-04-26T18:24:20.954445+0000","last_active":"2024-04-26T18:27:57.694689+0000","last_peered":"2024-04-26T18:27:57.694689+0000","last_clean":"2024-04-26T18:27:57.694689+0000","last_became_active":"2024-04-26T18:24:20.953852+0000","last_became_peered":"2024-04-26T18:24:20.953852+0000","last_unstale":"2024-04-26T18:27:57.694689+0000","last_undegraded":"2024-04-26T18:27:57.694689+0000","last_fullsized":"2024-04-26T18:27:57.694689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:05.636897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984195+0000","last_change":"2024-04-26T18:24:20.958023+0000","last_active":"2024-04-26T18:27:57.984195+0000","last_peered":"2024-04-26T18:27:57.984195+0000","last_clean":"2024-04-26T18:27:57.984195+0000","last_became_active":"2024-04-26T18:24:20.957642+0000","last_became_peered":"2024-04-26T18:24:20.957642+0000","last_unstale":"2024-04-26T18:27:57.984195+0000","last_undegraded":"2024-04-26T18:27:57.984195+0000","last_fullsized":"2024-04-26T18:27:57.984195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:46.332710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106773,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":807456,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2531,"num_read_kb":1555,"num_write":570,"num_write_kb":2659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1583,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":811008,"data_stored":2422368,"data_compressed":109023,"data_compressed_allocated":147456,"data_compressed_original":2125824,"omap_allocated":0,"internal_metadata":0},"log_size":426,"ondisk_log_size":426,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351035,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":510598,"data_compressed":14494,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612672,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36560,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708592,"statfs":{"total":95995035648,"available":95957598208,"internally_reserved":0,"allocated":1245184,"data_stored":1505872,"data_compressed":67130,"data_compressed_allocated":90112,"data_compressed_original":946176,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":902799,"data_compressed":33790,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037835,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":903155,"data_compressed":33790,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299471,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":1104966,"data_compressed":50085,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462998,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":509691,"data_compressed":14494,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":588482,"data_compressed":16337,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":511562,"data_compressed":14494,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411434,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":586362,"data_compressed":16337,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149796,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":1230573,"data_compressed":48992,"data_compressed_allocated":65536,"data_compressed_original":741376,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888160,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":510334,"data_compressed":14494,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724633,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712832,"statfs":{"total":95995035648,"available":95961939968,"internally_reserved":0,"allocated":1163264,"data_stored":1431312,"data_compressed":65287,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":80021,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":77901,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34498,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":332366,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19296,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19296,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":406926,"data_compressed":17045,"data_compressed_allocated":24576,"data_compressed_original":315392,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:28:06.318 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T18:28:06.318 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T18:28:06.558 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:06.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:06 smithi084 ceph-mon[29141]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 170 KiB/s wr, 107 op/s 2024-04-26T18:28:06.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:06 smithi084 ceph-mon[29141]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:28:06.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:06 smithi103 ceph-mon[32727]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 170 KiB/s wr, 107 op/s 2024-04-26T18:28:06.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:06 smithi103 ceph-mon[32727]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:28:06.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:06 smithi174 ceph-mon[32705]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 170 KiB/s wr, 107 op/s 2024-04-26T18:28:06.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:06 smithi174 ceph-mon[32705]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:28:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.298+0000 7f93ff98b700 1 -- 172.21.15.84:0/2892573383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f8102360 msgr2=0x7f93f81027c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.298+0000 7f93ff98b700 1 --2- 172.21.15.84:0/2892573383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f8102360 0x7f93f81027c0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f93e8009990 tx=0x7f93e8009ca0 comp rx=0 tx=0).stop 2024-04-26T18:28:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 -- 172.21.15.84:0/2892573383 shutdown_connections 2024-04-26T18:28:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 --2- 172.21.15.84:0/2892573383 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8102d00 0x7f93f810f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 --2- 172.21.15.84:0/2892573383 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93f81083a0 0x7f93f8108780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 --2- 172.21.15.84:0/2892573383 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f8102360 0x7f93f81027c0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 -- 172.21.15.84:0/2892573383 >> 172.21.15.84:0/2892573383 conn(0x7f93f8079250 msgr2=0x7f93f8079660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 -- 172.21.15.84:0/2892573383 shutdown_connections 2024-04-26T18:28:08.302 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 -- 172.21.15.84:0/2892573383 wait complete. 2024-04-26T18:28:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 Processor -- start 2024-04-26T18:28:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 -- start start 2024-04-26T18:28:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8102360 0x7f93f819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93f8102d00 0x7f93f819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f81083a0 0x7f93f81988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8112510 con 0x7f93f81083a0 2024-04-26T18:28:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8112390 con 0x7f93f8102360 2024-04-26T18:28:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.299+0000 7f93ff98b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f93f8112690 con 0x7f93f8102d00 2024-04-26T18:28:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93fdf28700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f81083a0 0x7f93f81988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93fdf28700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f81083a0 0x7f93f81988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45672/0 (socket says 172.21.15.84:45672) 2024-04-26T18:28:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93fdf28700 1 -- 172.21.15.84:0/1504124218 learned_addr learned my addr 172.21.15.84:0/1504124218 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93fd727700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8102360 0x7f93f819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:08.303 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93fcf26700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93f8102d00 0x7f93f819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:08.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93fdf28700 1 -- 172.21.15.84:0/1504124218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93f8102d00 msgr2=0x7f93f819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:08.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93fdf28700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93f8102d00 0x7f93f819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:08.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93fdf28700 1 -- 172.21.15.84:0/1504124218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8102360 msgr2=0x7f93f819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:08.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93fdf28700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8102360 0x7f93f819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:08.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93fdf28700 1 -- 172.21.15.84:0/1504124218 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93f8199010 con 0x7f93f81083a0 2024-04-26T18:28:08.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93fdf28700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f81083a0 0x7f93f81988b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f93f000ada0 tx=0x7f93f000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:08.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93ee7fc700 1 -- 172.21.15.84:0/1504124218 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0017020 con 0x7f93f81083a0 2024-04-26T18:28:08.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93ff98b700 1 -- 172.21.15.84:0/1504124218 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93f8199300 con 0x7f93f81083a0 2024-04-26T18:28:08.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93ee7fc700 1 -- 172.21.15.84:0/1504124218 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93f000cd80 con 0x7f93f81083a0 2024-04-26T18:28:08.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93ee7fc700 1 -- 172.21.15.84:0/1504124218 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93f0005260 con 0x7f93f81083a0 2024-04-26T18:28:08.304 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.300+0000 7f93ff98b700 1 -- 172.21.15.84:0/1504124218 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93f806bbb0 con 0x7f93f81083a0 2024-04-26T18:28:08.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.301+0000 7f93ff98b700 1 -- 172.21.15.84:0/1504124218 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93f806a110 con 0x7f93f81083a0 2024-04-26T18:28:08.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.303+0000 7f93ee7fc700 1 -- 172.21.15.84:0/1504124218 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f93f000a090 con 0x7f93f81083a0 2024-04-26T18:28:08.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.303+0000 7f93ee7fc700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f93e40a3f60 0x7f93e40a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:08.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.303+0000 7f93fd727700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f93e40a3f60 0x7f93e40a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:08.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.303+0000 7f93ee7fc700 1 -- 172.21.15.84:0/1504124218 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f93f00d07d0 con 0x7f93f81083a0 2024-04-26T18:28:08.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.304+0000 7f93fd727700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f93e40a3f60 0x7f93e40a6420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f93f81999f0 tx=0x7f93f400f040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:08.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.307+0000 7f93ee7fc700 1 -- 172.21.15.84:0/1504124218 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f93f0021020 con 0x7f93f81083a0 2024-04-26T18:28:08.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.506+0000 7f93ff98b700 1 -- 172.21.15.84:0/1504124218 --> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f93f806be10 con 0x7f93e40a3f60 2024-04-26T18:28:08.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.513+0000 7f93ee7fc700 1 -- 172.21.15.84:0/1504124218 <== mgr.15240 v2:172.21.15.84:6800/3453983695 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258145 (secure 0 0 0) 0x7f93f806be10 con 0x7f93e40a3f60 2024-04-26T18:28:08.517 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:28:08.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.516+0000 7f93ff98b700 1 -- 172.21.15.84:0/1504124218 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f93e40a3f60 msgr2=0x7f93e40a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:08.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.516+0000 7f93ff98b700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f93e40a3f60 0x7f93e40a6420 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f93f81999f0 tx=0x7f93f400f040 comp rx=0 tx=0).stop 2024-04-26T18:28:08.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.516+0000 7f93ff98b700 1 -- 172.21.15.84:0/1504124218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f81083a0 msgr2=0x7f93f81988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:08.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.516+0000 7f93ff98b700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f81083a0 0x7f93f81988b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f93f000ada0 tx=0x7f93f000c330 comp rx=0 tx=0).stop 2024-04-26T18:28:08.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.516+0000 7f93ff98b700 1 -- 172.21.15.84:0/1504124218 shutdown_connections 2024-04-26T18:28:08.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.516+0000 7f93ff98b700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f93e40a3f60 0x7f93e40a6420 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:08.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.516+0000 7f93ff98b700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f93f81083a0 0x7f93f81988b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:08.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.516+0000 7f93ff98b700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f93f8102360 0x7f93f819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:08.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.516+0000 7f93ff98b700 1 --2- 172.21.15.84:0/1504124218 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f93f8102d00 0x7f93f819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:08.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.516+0000 7f93ff98b700 1 -- 172.21.15.84:0/1504124218 >> 172.21.15.84:0/1504124218 conn(0x7f93f8079250 msgr2=0x7f93f8101250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:08.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.516+0000 7f93ff98b700 1 -- 172.21.15.84:0/1504124218 shutdown_connections 2024-04-26T18:28:08.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:08.516+0000 7f93ff98b700 1 -- 172.21.15.84:0/1504124218 wait complete. 2024-04-26T18:28:08.520 INFO:teuthology.orchestra.run.smithi084.stderr:dumped all 2024-04-26T18:28:09.242 INFO:teuthology.orchestra.run.smithi084.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-26T18:28:07.466980+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1398058,"num_objects":104,"num_object_clones":0,"num_object_copies":312,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2768,"num_read_kb":1885,"num_write":773,"num_write_kb":5599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":1182319,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1076,"ondisk_log_size":1076,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":374580,"kb_used_data":12912,"kb_used_omap":18,"kb_used_meta":361517,"kb_avail":1124567244,"statfs":{"total":1151940427776,"available":1151556857856,"internally_reserved":0,"allocated":13221888,"data_stored":10295706,"data_compressed":389724,"data_compressed_allocated":528384,"data_compressed_original":4386816,"omap_allocated":19143,"internal_metadata":370193721},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":202545,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":534,"num_read_kb":594,"num_write":215,"num_write_kb":1191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.991340"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984569+0000","last_change":"2024-04-26T18:24:20.951581+0000","last_active":"2024-04-26T18:27:57.984569+0000","last_peered":"2024-04-26T18:27:57.984569+0000","last_clean":"2024-04-26T18:27:57.984569+0000","last_became_active":"2024-04-26T18:24:20.951260+0000","last_became_peered":"2024-04-26T18:24:20.951260+0000","last_unstale":"2024-04-26T18:27:57.984569+0000","last_undegraded":"2024-04-26T18:27:57.984569+0000","last_fullsized":"2024-04-26T18:27:57.984569+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:54:28.815699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697639+0000","last_change":"2024-04-26T18:24:20.952263+0000","last_active":"2024-04-26T18:27:57.697639+0000","last_peered":"2024-04-26T18:27:57.697639+0000","last_clean":"2024-04-26T18:27:57.697639+0000","last_became_active":"2024-04-26T18:24:20.951930+0000","last_became_peered":"2024-04-26T18:24:20.951930+0000","last_unstale":"2024-04-26T18:27:57.697639+0000","last_undegraded":"2024-04-26T18:27:57.697639+0000","last_fullsized":"2024-04-26T18:27:57.697639+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:54:54.816572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697346+0000","last_change":"2024-04-26T18:24:20.952280+0000","last_active":"2024-04-26T18:27:57.697346+0000","last_peered":"2024-04-26T18:27:57.697346+0000","last_clean":"2024-04-26T18:27:57.697346+0000","last_became_active":"2024-04-26T18:24:20.951623+0000","last_became_peered":"2024-04-26T18:24:20.951623+0000","last_unstale":"2024-04-26T18:27:57.697346+0000","last_undegraded":"2024-04-26T18:27:57.697346+0000","last_fullsized":"2024-04-26T18:27:57.697346+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:19:02.502261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.695472+0000","last_change":"2024-04-26T18:24:20.962274+0000","last_active":"2024-04-26T18:27:57.695472+0000","last_peered":"2024-04-26T18:27:57.695472+0000","last_clean":"2024-04-26T18:27:57.695472+0000","last_became_active":"2024-04-26T18:24:20.961920+0000","last_became_peered":"2024-04-26T18:24:20.961920+0000","last_unstale":"2024-04-26T18:27:57.695472+0000","last_undegraded":"2024-04-26T18:27:57.695472+0000","last_fullsized":"2024-04-26T18:27:57.695472+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:49:46.034633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545742+0000","last_change":"2024-04-26T18:24:20.949096+0000","last_active":"2024-04-26T18:27:57.545742+0000","last_peered":"2024-04-26T18:27:57.545742+0000","last_clean":"2024-04-26T18:27:57.545742+0000","last_became_active":"2024-04-26T18:24:20.948737+0000","last_became_peered":"2024-04-26T18:24:20.948737+0000","last_unstale":"2024-04-26T18:27:57.545742+0000","last_undegraded":"2024-04-26T18:27:57.545742+0000","last_fullsized":"2024-04-26T18:27:57.545742+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:12:40.371589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451728+0000","last_change":"2024-04-26T18:24:20.962670+0000","last_active":"2024-04-26T18:27:57.451728+0000","last_peered":"2024-04-26T18:27:57.451728+0000","last_clean":"2024-04-26T18:27:57.451728+0000","last_became_active":"2024-04-26T18:24:20.962309+0000","last_became_peered":"2024-04-26T18:24:20.962309+0000","last_unstale":"2024-04-26T18:27:57.451728+0000","last_undegraded":"2024-04-26T18:27:57.451728+0000","last_fullsized":"2024-04-26T18:27:57.451728+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:38:40.315864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591196+0000","last_change":"2024-04-26T18:25:06.350579+0000","last_active":"2024-04-26T18:27:57.591196+0000","last_peered":"2024-04-26T18:27:57.591196+0000","last_clean":"2024-04-26T18:27:57.591196+0000","last_became_active":"2024-04-26T18:25:06.350104+0000","last_became_peered":"2024-04-26T18:25:06.350104+0000","last_unstale":"2024-04-26T18:27:57.591196+0000","last_undegraded":"2024-04-26T18:27:57.591196+0000","last_fullsized":"2024-04-26T18:27:57.591196+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:37:00.950883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452339+0000","last_change":"2024-04-26T18:24:20.961096+0000","last_active":"2024-04-26T18:27:57.452339+0000","last_peered":"2024-04-26T18:27:57.452339+0000","last_clean":"2024-04-26T18:27:57.452339+0000","last_became_active":"2024-04-26T18:24:20.960796+0000","last_became_peered":"2024-04-26T18:24:20.960796+0000","last_unstale":"2024-04-26T18:27:57.452339+0000","last_undegraded":"2024-04-26T18:27:57.452339+0000","last_fullsized":"2024-04-26T18:27:57.452339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:06:04.538836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589942+0000","last_change":"2024-04-26T18:24:20.959871+0000","last_active":"2024-04-26T18:27:57.589942+0000","last_peered":"2024-04-26T18:27:57.589942+0000","last_clean":"2024-04-26T18:27:57.589942+0000","last_became_active":"2024-04-26T18:24:20.959404+0000","last_became_peered":"2024-04-26T18:24:20.959404+0000","last_unstale":"2024-04-26T18:27:57.589942+0000","last_undegraded":"2024-04-26T18:27:57.589942+0000","last_fullsized":"2024-04-26T18:27:57.589942+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:41:43.053618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":49,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451941+0000","last_change":"2024-04-26T18:26:07.431543+0000","last_active":"2024-04-26T18:27:57.451941+0000","last_peered":"2024-04-26T18:27:57.451941+0000","last_clean":"2024-04-26T18:27:57.451941+0000","last_became_active":"2024-04-26T18:26:07.430976+0000","last_became_peered":"2024-04-26T18:26:07.430976+0000","last_unstale":"2024-04-26T18:27:57.451941+0000","last_undegraded":"2024-04-26T18:27:57.451941+0000","last_fullsized":"2024-04-26T18:27:57.451941+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:01:31.430010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591606+0000","last_change":"2024-04-26T18:24:20.952109+0000","last_active":"2024-04-26T18:27:57.591606+0000","last_peered":"2024-04-26T18:27:57.591606+0000","last_clean":"2024-04-26T18:27:57.591606+0000","last_became_active":"2024-04-26T18:24:20.951776+0000","last_became_peered":"2024-04-26T18:24:20.951776+0000","last_unstale":"2024-04-26T18:27:57.591606+0000","last_undegraded":"2024-04-26T18:27:57.591606+0000","last_fullsized":"2024-04-26T18:27:57.591606+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:58:53.220246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.695451+0000","last_change":"2024-04-26T18:24:20.963965+0000","last_active":"2024-04-26T18:27:57.695451+0000","last_peered":"2024-04-26T18:27:57.695451+0000","last_clean":"2024-04-26T18:27:57.695451+0000","last_became_active":"2024-04-26T18:24:20.963648+0000","last_became_peered":"2024-04-26T18:24:20.963648+0000","last_unstale":"2024-04-26T18:27:57.695451+0000","last_undegraded":"2024-04-26T18:27:57.695451+0000","last_fullsized":"2024-04-26T18:27:57.695451+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:04.080813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694857+0000","last_change":"2024-04-26T18:24:20.954186+0000","last_active":"2024-04-26T18:27:57.694857+0000","last_peered":"2024-04-26T18:27:57.694857+0000","last_clean":"2024-04-26T18:27:57.694857+0000","last_became_active":"2024-04-26T18:24:20.953431+0000","last_became_peered":"2024-04-26T18:24:20.953431+0000","last_unstale":"2024-04-26T18:27:57.694857+0000","last_undegraded":"2024-04-26T18:27:57.694857+0000","last_fullsized":"2024-04-26T18:27:57.694857+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:48:21.498476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.696970+0000","last_change":"2024-04-26T18:24:20.945392+0000","last_active":"2024-04-26T18:27:57.696970+0000","last_peered":"2024-04-26T18:27:57.696970+0000","last_clean":"2024-04-26T18:27:57.696970+0000","last_became_active":"2024-04-26T18:24:20.944971+0000","last_became_peered":"2024-04-26T18:24:20.944971+0000","last_unstale":"2024-04-26T18:27:57.696970+0000","last_undegraded":"2024-04-26T18:27:57.696970+0000","last_fullsized":"2024-04-26T18:27:57.696970+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:03:06.968689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697005+0000","last_change":"2024-04-26T18:25:06.299351+0000","last_active":"2024-04-26T18:27:57.697005+0000","last_peered":"2024-04-26T18:27:57.697005+0000","last_clean":"2024-04-26T18:27:57.697005+0000","last_became_active":"2024-04-26T18:25:06.298895+0000","last_became_peered":"2024-04-26T18:25:06.298895+0000","last_unstale":"2024-04-26T18:27:57.697005+0000","last_undegraded":"2024-04-26T18:27:57.697005+0000","last_fullsized":"2024-04-26T18:27:57.697005+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:37.517653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589273+0000","last_change":"2024-04-26T18:24:20.954610+0000","last_active":"2024-04-26T18:27:57.589273+0000","last_peered":"2024-04-26T18:27:57.589273+0000","last_clean":"2024-04-26T18:27:57.589273+0000","last_became_active":"2024-04-26T18:24:20.954239+0000","last_became_peered":"2024-04-26T18:24:20.954239+0000","last_unstale":"2024-04-26T18:27:57.589273+0000","last_undegraded":"2024-04-26T18:27:57.589273+0000","last_fullsized":"2024-04-26T18:27:57.589273+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:41.276659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984496+0000","last_change":"2024-04-26T18:24:20.956932+0000","last_active":"2024-04-26T18:27:57.984496+0000","last_peered":"2024-04-26T18:27:57.984496+0000","last_clean":"2024-04-26T18:27:57.984496+0000","last_became_active":"2024-04-26T18:24:20.956464+0000","last_became_peered":"2024-04-26T18:24:20.956464+0000","last_unstale":"2024-04-26T18:27:57.984496+0000","last_undegraded":"2024-04-26T18:27:57.984496+0000","last_fullsized":"2024-04-26T18:27:57.984496+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:33:01.902525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.543737+0000","last_change":"2024-04-26T18:24:20.954870+0000","last_active":"2024-04-26T18:27:57.543737+0000","last_peered":"2024-04-26T18:27:57.543737+0000","last_clean":"2024-04-26T18:27:57.543737+0000","last_became_active":"2024-04-26T18:24:20.954542+0000","last_became_peered":"2024-04-26T18:24:20.954542+0000","last_unstale":"2024-04-26T18:27:57.543737+0000","last_undegraded":"2024-04-26T18:27:57.543737+0000","last_fullsized":"2024-04-26T18:27:57.543737+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:13:51.932625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452391+0000","last_change":"2024-04-26T18:24:20.950994+0000","last_active":"2024-04-26T18:27:57.452391+0000","last_peered":"2024-04-26T18:27:57.452391+0000","last_clean":"2024-04-26T18:27:57.452391+0000","last_became_active":"2024-04-26T18:24:20.950343+0000","last_became_peered":"2024-04-26T18:24:20.950343+0000","last_unstale":"2024-04-26T18:27:57.452391+0000","last_undegraded":"2024-04-26T18:27:57.452391+0000","last_fullsized":"2024-04-26T18:27:57.452391+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:29:25.274049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"94'14","reported_seq":252,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:01.596025+0000","last_change":"2024-04-26T18:24:17.929731+0000","last_active":"2024-04-26T18:28:01.596025+0000","last_peered":"2024-04-26T18:28:01.596025+0000","last_clean":"2024-04-26T18:28:01.596025+0000","last_became_active":"2024-04-26T18:24:17.929379+0000","last_became_peered":"2024-04-26T18:24:17.929379+0000","last_unstale":"2024-04-26T18:28:01.596025+0000","last_undegraded":"2024-04-26T18:28:01.596025+0000","last_fullsized":"2024-04-26T18:28:01.596025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:03:44.821660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":205,"num_read_kb":205,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452602+0000","last_change":"2024-04-26T18:24:20.962885+0000","last_active":"2024-04-26T18:27:57.452602+0000","last_peered":"2024-04-26T18:27:57.452602+0000","last_clean":"2024-04-26T18:27:57.452602+0000","last_became_active":"2024-04-26T18:24:20.962674+0000","last_became_peered":"2024-04-26T18:24:20.962674+0000","last_unstale":"2024-04-26T18:27:57.452602+0000","last_undegraded":"2024-04-26T18:27:57.452602+0000","last_fullsized":"2024-04-26T18:27:57.452602+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:23:32.444889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452837+0000","last_change":"2024-04-26T18:24:18.141500+0000","last_active":"2024-04-26T18:27:57.452837+0000","last_peered":"2024-04-26T18:27:57.452837+0000","last_clean":"2024-04-26T18:27:57.452837+0000","last_became_active":"2024-04-26T18:24:18.140870+0000","last_became_peered":"2024-04-26T18:24:18.140870+0000","last_unstale":"2024-04-26T18:27:57.452837+0000","last_undegraded":"2024-04-26T18:27:57.452837+0000","last_fullsized":"2024-04-26T18:27:57.452837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:17:13.821872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697816+0000","last_change":"2024-04-26T18:24:20.949183+0000","last_active":"2024-04-26T18:27:57.697816+0000","last_peered":"2024-04-26T18:27:57.697816+0000","last_clean":"2024-04-26T18:27:57.697816+0000","last_became_active":"2024-04-26T18:24:20.948677+0000","last_became_peered":"2024-04-26T18:24:20.948677+0000","last_unstale":"2024-04-26T18:27:57.697816+0000","last_undegraded":"2024-04-26T18:27:57.697816+0000","last_fullsized":"2024-04-26T18:27:57.697816+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:55.064055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":68,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.546095+0000","last_change":"2024-04-26T18:25:06.312239+0000","last_active":"2024-04-26T18:27:57.546095+0000","last_peered":"2024-04-26T18:27:57.546095+0000","last_clean":"2024-04-26T18:27:57.546095+0000","last_became_active":"2024-04-26T18:25:06.301304+0000","last_became_peered":"2024-04-26T18:25:06.301304+0000","last_unstale":"2024-04-26T18:27:57.546095+0000","last_undegraded":"2024-04-26T18:27:57.546095+0000","last_fullsized":"2024-04-26T18:27:57.546095+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:28:36.159348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,4,7],"acting":[8,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589460+0000","last_change":"2024-04-26T18:24:20.955555+0000","last_active":"2024-04-26T18:27:57.589460+0000","last_peered":"2024-04-26T18:27:57.589460+0000","last_clean":"2024-04-26T18:27:57.589460+0000","last_became_active":"2024-04-26T18:24:20.955176+0000","last_became_peered":"2024-04-26T18:24:20.955176+0000","last_unstale":"2024-04-26T18:27:57.589460+0000","last_undegraded":"2024-04-26T18:27:57.589460+0000","last_fullsized":"2024-04-26T18:27:57.589460+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:02:30.368971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545963+0000","last_change":"2024-04-26T18:24:17.930919+0000","last_active":"2024-04-26T18:27:57.545963+0000","last_peered":"2024-04-26T18:27:57.545963+0000","last_clean":"2024-04-26T18:27:57.545963+0000","last_became_active":"2024-04-26T18:24:17.930596+0000","last_became_peered":"2024-04-26T18:24:17.930596+0000","last_unstale":"2024-04-26T18:27:57.545963+0000","last_undegraded":"2024-04-26T18:27:57.545963+0000","last_fullsized":"2024-04-26T18:27:57.545963+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:12:57.476027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550891+0000","last_change":"2024-04-26T18:24:20.955796+0000","last_active":"2024-04-26T18:27:57.550891+0000","last_peered":"2024-04-26T18:27:57.550891+0000","last_clean":"2024-04-26T18:27:57.550891+0000","last_became_active":"2024-04-26T18:24:20.955432+0000","last_became_peered":"2024-04-26T18:24:20.955432+0000","last_unstale":"2024-04-26T18:27:57.550891+0000","last_undegraded":"2024-04-26T18:27:57.550891+0000","last_fullsized":"2024-04-26T18:27:57.550891+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:14:25.161797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"94'9","reported_seq":252,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:01.596448+0000","last_change":"2024-04-26T18:24:17.929505+0000","last_active":"2024-04-26T18:28:01.596448+0000","last_peered":"2024-04-26T18:28:01.596448+0000","last_clean":"2024-04-26T18:28:01.596448+0000","last_became_active":"2024-04-26T18:24:17.929205+0000","last_became_peered":"2024-04-26T18:24:17.929205+0000","last_unstale":"2024-04-26T18:28:01.596448+0000","last_undegraded":"2024-04-26T18:28:01.596448+0000","last_fullsized":"2024-04-26T18:28:01.596448+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:20:42.460063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":210,"num_read_kb":6,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545699+0000","last_change":"2024-04-26T18:24:20.952134+0000","last_active":"2024-04-26T18:27:57.545699+0000","last_peered":"2024-04-26T18:27:57.545699+0000","last_clean":"2024-04-26T18:27:57.545699+0000","last_became_active":"2024-04-26T18:24:20.951758+0000","last_became_peered":"2024-04-26T18:24:20.951758+0000","last_unstale":"2024-04-26T18:27:57.545699+0000","last_undegraded":"2024-04-26T18:27:57.545699+0000","last_fullsized":"2024-04-26T18:27:57.545699+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:44:25.389221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451604+0000","last_change":"2024-04-26T18:24:17.928704+0000","last_active":"2024-04-26T18:27:57.451604+0000","last_peered":"2024-04-26T18:27:57.451604+0000","last_clean":"2024-04-26T18:27:57.451604+0000","last_became_active":"2024-04-26T18:24:17.928343+0000","last_became_peered":"2024-04-26T18:24:17.928343+0000","last_unstale":"2024-04-26T18:27:57.451604+0000","last_undegraded":"2024-04-26T18:27:57.451604+0000","last_fullsized":"2024-04-26T18:27:57.451604+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:15:49.400481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550744+0000","last_change":"2024-04-26T18:24:20.956808+0000","last_active":"2024-04-26T18:27:57.550744+0000","last_peered":"2024-04-26T18:27:57.550744+0000","last_clean":"2024-04-26T18:27:57.550744+0000","last_became_active":"2024-04-26T18:24:20.956466+0000","last_became_peered":"2024-04-26T18:24:20.956466+0000","last_unstale":"2024-04-26T18:27:57.550744+0000","last_undegraded":"2024-04-26T18:27:57.550744+0000","last_fullsized":"2024-04-26T18:27:57.550744+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:31.150670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.693067+0000","last_change":"2024-04-26T18:24:17.929134+0000","last_active":"2024-04-26T18:27:57.693067+0000","last_peered":"2024-04-26T18:27:57.693067+0000","last_clean":"2024-04-26T18:27:57.693067+0000","last_became_active":"2024-04-26T18:24:17.928798+0000","last_became_peered":"2024-04-26T18:24:17.928798+0000","last_unstale":"2024-04-26T18:27:57.693067+0000","last_undegraded":"2024-04-26T18:27:57.693067+0000","last_fullsized":"2024-04-26T18:27:57.693067+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:10.450825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452339+0000","last_change":"2024-04-26T18:24:20.951594+0000","last_active":"2024-04-26T18:27:57.452339+0000","last_peered":"2024-04-26T18:27:57.452339+0000","last_clean":"2024-04-26T18:27:57.452339+0000","last_became_active":"2024-04-26T18:24:20.951148+0000","last_became_peered":"2024-04-26T18:24:20.951148+0000","last_unstale":"2024-04-26T18:27:57.452339+0000","last_undegraded":"2024-04-26T18:27:57.452339+0000","last_fullsized":"2024-04-26T18:27:57.452339+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:18.703292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550654+0000","last_change":"2024-04-26T18:24:17.931101+0000","last_active":"2024-04-26T18:27:57.550654+0000","last_peered":"2024-04-26T18:27:57.550654+0000","last_clean":"2024-04-26T18:27:57.550654+0000","last_became_active":"2024-04-26T18:24:17.930819+0000","last_became_peered":"2024-04-26T18:24:17.930819+0000","last_unstale":"2024-04-26T18:27:57.550654+0000","last_undegraded":"2024-04-26T18:27:57.550654+0000","last_fullsized":"2024-04-26T18:27:57.550654+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:20:34.260175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.543387+0000","last_change":"2024-04-26T18:24:20.948866+0000","last_active":"2024-04-26T18:27:57.543387+0000","last_peered":"2024-04-26T18:27:57.543387+0000","last_clean":"2024-04-26T18:27:57.543387+0000","last_became_active":"2024-04-26T18:24:20.948562+0000","last_became_peered":"2024-04-26T18:24:20.948562+0000","last_unstale":"2024-04-26T18:27:57.543387+0000","last_undegraded":"2024-04-26T18:27:57.543387+0000","last_fullsized":"2024-04-26T18:27:57.543387+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:33:19.616660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"94'8","reported_seq":448,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:02.596461+0000","last_change":"2024-04-26T18:24:17.926384+0000","last_active":"2024-04-26T18:28:02.596461+0000","last_peered":"2024-04-26T18:28:02.596461+0000","last_clean":"2024-04-26T18:28:02.596461+0000","last_became_active":"2024-04-26T18:24:17.926037+0000","last_became_peered":"2024-04-26T18:24:17.926037+0000","last_unstale":"2024-04-26T18:28:02.596461+0000","last_undegraded":"2024-04-26T18:28:02.596461+0000","last_fullsized":"2024-04-26T18:28:02.596461+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:19:05.283755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":407,"num_read_kb":407,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589843+0000","last_change":"2024-04-26T18:24:20.954257+0000","last_active":"2024-04-26T18:27:57.589843+0000","last_peered":"2024-04-26T18:27:57.589843+0000","last_clean":"2024-04-26T18:27:57.589843+0000","last_became_active":"2024-04-26T18:24:20.953913+0000","last_became_peered":"2024-04-26T18:24:20.953913+0000","last_unstale":"2024-04-26T18:27:57.589843+0000","last_undegraded":"2024-04-26T18:27:57.589843+0000","last_fullsized":"2024-04-26T18:27:57.589843+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:06.814156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591540+0000","last_change":"2024-04-26T18:24:17.932843+0000","last_active":"2024-04-26T18:27:57.591540+0000","last_peered":"2024-04-26T18:27:57.591540+0000","last_clean":"2024-04-26T18:27:57.591540+0000","last_became_active":"2024-04-26T18:24:17.932502+0000","last_became_peered":"2024-04-26T18:24:17.932502+0000","last_unstale":"2024-04-26T18:27:57.591540+0000","last_undegraded":"2024-04-26T18:27:57.591540+0000","last_fullsized":"2024-04-26T18:27:57.591540+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:55:49.727094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697781+0000","last_change":"2024-04-26T18:24:20.959881+0000","last_active":"2024-04-26T18:27:57.697781+0000","last_peered":"2024-04-26T18:27:57.697781+0000","last_clean":"2024-04-26T18:27:57.697781+0000","last_became_active":"2024-04-26T18:24:20.959541+0000","last_became_peered":"2024-04-26T18:24:20.959541+0000","last_unstale":"2024-04-26T18:27:57.697781+0000","last_undegraded":"2024-04-26T18:27:57.697781+0000","last_fullsized":"2024-04-26T18:27:57.697781+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:47:23.393521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.551017+0000","last_change":"2024-04-26T18:24:18.142146+0000","last_active":"2024-04-26T18:27:57.551017+0000","last_peered":"2024-04-26T18:27:57.551017+0000","last_clean":"2024-04-26T18:27:57.551017+0000","last_became_active":"2024-04-26T18:24:18.141829+0000","last_became_peered":"2024-04-26T18:24:18.141829+0000","last_unstale":"2024-04-26T18:27:57.551017+0000","last_undegraded":"2024-04-26T18:27:57.551017+0000","last_fullsized":"2024-04-26T18:27:57.551017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:11:49.341638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589611+0000","last_change":"2024-04-26T18:24:20.957993+0000","last_active":"2024-04-26T18:27:57.589611+0000","last_peered":"2024-04-26T18:27:57.589611+0000","last_clean":"2024-04-26T18:27:57.589611+0000","last_became_active":"2024-04-26T18:24:20.957443+0000","last_became_peered":"2024-04-26T18:24:20.957443+0000","last_unstale":"2024-04-26T18:27:57.589611+0000","last_undegraded":"2024-04-26T18:27:57.589611+0000","last_fullsized":"2024-04-26T18:27:57.589611+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:06:32.960546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"94'45","reported_seq":500,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:02.597472+0000","last_change":"2024-04-26T18:24:17.931283+0000","last_active":"2024-04-26T18:28:02.597472+0000","last_peered":"2024-04-26T18:28:02.597472+0000","last_clean":"2024-04-26T18:28:02.597472+0000","last_became_active":"2024-04-26T18:24:17.930943+0000","last_became_peered":"2024-04-26T18:24:17.930943+0000","last_unstale":"2024-04-26T18:28:02.597472+0000","last_undegraded":"2024-04-26T18:28:02.597472+0000","last_fullsized":"2024-04-26T18:28:02.597472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:59:50.511677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":75785,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":422,"num_read_kb":285,"num_write":47,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545864+0000","last_change":"2024-04-26T18:24:20.947509+0000","last_active":"2024-04-26T18:27:57.545864+0000","last_peered":"2024-04-26T18:27:57.545864+0000","last_clean":"2024-04-26T18:27:57.545864+0000","last_became_active":"2024-04-26T18:24:20.947181+0000","last_became_peered":"2024-04-26T18:24:20.947181+0000","last_unstale":"2024-04-26T18:27:57.545864+0000","last_undegraded":"2024-04-26T18:27:57.545864+0000","last_fullsized":"2024-04-26T18:27:57.545864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T00:39:53.262312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452359+0000","last_change":"2024-04-26T18:24:17.934042+0000","last_active":"2024-04-26T18:27:57.452359+0000","last_peered":"2024-04-26T18:27:57.452359+0000","last_clean":"2024-04-26T18:27:57.452359+0000","last_became_active":"2024-04-26T18:24:17.933537+0000","last_became_peered":"2024-04-26T18:24:17.933537+0000","last_unstale":"2024-04-26T18:27:57.452359+0000","last_undegraded":"2024-04-26T18:27:57.452359+0000","last_fullsized":"2024-04-26T18:27:57.452359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:35:06.904160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.692822+0000","last_change":"2024-04-26T18:24:20.957672+0000","last_active":"2024-04-26T18:27:57.692822+0000","last_peered":"2024-04-26T18:27:57.692822+0000","last_clean":"2024-04-26T18:27:57.692822+0000","last_became_active":"2024-04-26T18:24:20.957214+0000","last_became_peered":"2024-04-26T18:24:20.957214+0000","last_unstale":"2024-04-26T18:27:57.692822+0000","last_undegraded":"2024-04-26T18:27:57.692822+0000","last_fullsized":"2024-04-26T18:27:57.692822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:42:35.341858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550631+0000","last_change":"2024-04-26T18:24:17.930324+0000","last_active":"2024-04-26T18:27:57.550631+0000","last_peered":"2024-04-26T18:27:57.550631+0000","last_clean":"2024-04-26T18:27:57.550631+0000","last_became_active":"2024-04-26T18:24:17.930022+0000","last_became_peered":"2024-04-26T18:24:17.930022+0000","last_unstale":"2024-04-26T18:27:57.550631+0000","last_undegraded":"2024-04-26T18:27:57.550631+0000","last_fullsized":"2024-04-26T18:27:57.550631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:27:36.864816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":48,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591382+0000","last_change":"2024-04-26T18:25:06.350388+0000","last_active":"2024-04-26T18:27:57.591382+0000","last_peered":"2024-04-26T18:27:57.591382+0000","last_clean":"2024-04-26T18:27:57.591382+0000","last_became_active":"2024-04-26T18:25:06.349836+0000","last_became_peered":"2024-04-26T18:25:06.349836+0000","last_unstale":"2024-04-26T18:27:57.591382+0000","last_undegraded":"2024-04-26T18:27:57.591382+0000","last_fullsized":"2024-04-26T18:27:57.591382+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:57.762115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"94'8","reported_seq":249,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:01.701789+0000","last_change":"2024-04-26T18:24:17.934676+0000","last_active":"2024-04-26T18:28:01.701789+0000","last_peered":"2024-04-26T18:28:01.701789+0000","last_clean":"2024-04-26T18:28:01.701789+0000","last_became_active":"2024-04-26T18:24:17.934307+0000","last_became_peered":"2024-04-26T18:24:17.934307+0000","last_unstale":"2024-04-26T18:28:01.701789+0000","last_undegraded":"2024-04-26T18:28:01.701789+0000","last_fullsized":"2024-04-26T18:28:01.701789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:08:27.312248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1761,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":209,"num_read_kb":5,"num_write":8,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591082+0000","last_change":"2024-04-26T18:24:20.947371+0000","last_active":"2024-04-26T18:27:57.591082+0000","last_peered":"2024-04-26T18:27:57.591082+0000","last_clean":"2024-04-26T18:27:57.591082+0000","last_became_active":"2024-04-26T18:24:20.947048+0000","last_became_peered":"2024-04-26T18:24:20.947048+0000","last_unstale":"2024-04-26T18:27:57.591082+0000","last_undegraded":"2024-04-26T18:27:57.591082+0000","last_fullsized":"2024-04-26T18:27:57.591082+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:55.038074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545656+0000","last_change":"2024-04-26T18:24:18.142680+0000","last_active":"2024-04-26T18:27:57.545656+0000","last_peered":"2024-04-26T18:27:57.545656+0000","last_clean":"2024-04-26T18:27:57.545656+0000","last_became_active":"2024-04-26T18:24:18.142392+0000","last_became_peered":"2024-04-26T18:24:18.142392+0000","last_unstale":"2024-04-26T18:27:57.545656+0000","last_undegraded":"2024-04-26T18:27:57.545656+0000","last_fullsized":"2024-04-26T18:27:57.545656+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:26:52.654813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550596+0000","last_change":"2024-04-26T18:24:20.958131+0000","last_active":"2024-04-26T18:27:57.550596+0000","last_peered":"2024-04-26T18:27:57.550596+0000","last_clean":"2024-04-26T18:27:57.550596+0000","last_became_active":"2024-04-26T18:24:20.957701+0000","last_became_peered":"2024-04-26T18:24:20.957701+0000","last_unstale":"2024-04-26T18:27:57.550596+0000","last_undegraded":"2024-04-26T18:27:57.550596+0000","last_fullsized":"2024-04-26T18:27:57.550596+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:02.018158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984426+0000","last_change":"2024-04-26T18:25:06.319173+0000","last_active":"2024-04-26T18:27:57.984426+0000","last_peered":"2024-04-26T18:27:57.984426+0000","last_clean":"2024-04-26T18:27:57.984426+0000","last_became_active":"2024-04-26T18:25:06.302935+0000","last_became_peered":"2024-04-26T18:25:06.302935+0000","last_unstale":"2024-04-26T18:27:57.984426+0000","last_undegraded":"2024-04-26T18:27:57.984426+0000","last_fullsized":"2024-04-26T18:27:57.984426+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:54:40.198105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984341+0000","last_change":"2024-04-26T18:24:20.961230+0000","last_active":"2024-04-26T18:27:57.984341+0000","last_peered":"2024-04-26T18:27:57.984341+0000","last_clean":"2024-04-26T18:27:57.984341+0000","last_became_active":"2024-04-26T18:24:20.960807+0000","last_became_peered":"2024-04-26T18:24:20.960807+0000","last_unstale":"2024-04-26T18:27:57.984341+0000","last_undegraded":"2024-04-26T18:27:57.984341+0000","last_fullsized":"2024-04-26T18:27:57.984341+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:44:39.535100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.543569+0000","last_change":"2024-04-26T18:24:18.142082+0000","last_active":"2024-04-26T18:27:57.543569+0000","last_peered":"2024-04-26T18:27:57.543569+0000","last_clean":"2024-04-26T18:27:57.543569+0000","last_became_active":"2024-04-26T18:24:18.141765+0000","last_became_peered":"2024-04-26T18:24:18.141765+0000","last_unstale":"2024-04-26T18:27:57.543569+0000","last_undegraded":"2024-04-26T18:27:57.543569+0000","last_fullsized":"2024-04-26T18:27:57.543569+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:59:02.140027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451762+0000","last_change":"2024-04-26T18:24:20.952867+0000","last_active":"2024-04-26T18:27:57.451762+0000","last_peered":"2024-04-26T18:27:57.451762+0000","last_clean":"2024-04-26T18:27:57.451762+0000","last_became_active":"2024-04-26T18:24:20.952525+0000","last_became_peered":"2024-04-26T18:24:20.952525+0000","last_unstale":"2024-04-26T18:27:57.451762+0000","last_undegraded":"2024-04-26T18:27:57.451762+0000","last_fullsized":"2024-04-26T18:27:57.451762+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:20:52.804312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"94'6","reported_seq":245,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:01.596387+0000","last_change":"2024-04-26T18:24:17.932270+0000","last_active":"2024-04-26T18:28:01.596387+0000","last_peered":"2024-04-26T18:28:01.596387+0000","last_clean":"2024-04-26T18:28:01.596387+0000","last_became_active":"2024-04-26T18:24:17.931901+0000","last_became_peered":"2024-04-26T18:24:17.931901+0000","last_unstale":"2024-04-26T18:28:01.596387+0000","last_undegraded":"2024-04-26T18:28:01.596387+0000","last_fullsized":"2024-04-26T18:28:01.596387+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:08:13.524174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":206,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.451763+0000","last_change":"2024-04-26T18:24:20.960612+0000","last_active":"2024-04-26T18:27:57.451763+0000","last_peered":"2024-04-26T18:27:57.451763+0000","last_clean":"2024-04-26T18:27:57.451763+0000","last_became_active":"2024-04-26T18:24:20.960280+0000","last_became_peered":"2024-04-26T18:24:20.960280+0000","last_unstale":"2024-04-26T18:27:57.451763+0000","last_undegraded":"2024-04-26T18:27:57.451763+0000","last_fullsized":"2024-04-26T18:27:57.451763+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:34:53.310532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.543368+0000","last_change":"2024-04-26T18:25:06.299736+0000","last_active":"2024-04-26T18:27:57.543368+0000","last_peered":"2024-04-26T18:27:57.543368+0000","last_clean":"2024-04-26T18:27:57.543368+0000","last_became_active":"2024-04-26T18:25:06.299265+0000","last_became_peered":"2024-04-26T18:25:06.299265+0000","last_unstale":"2024-04-26T18:27:57.543368+0000","last_undegraded":"2024-04-26T18:27:57.543368+0000","last_fullsized":"2024-04-26T18:27:57.543368+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:32:32.427551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,8],"acting":[9,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"94'6","reported_seq":244,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:01.701339+0000","last_change":"2024-04-26T18:24:18.141181+0000","last_active":"2024-04-26T18:28:01.701339+0000","last_peered":"2024-04-26T18:28:01.701339+0000","last_clean":"2024-04-26T18:28:01.701339+0000","last_became_active":"2024-04-26T18:24:18.140854+0000","last_became_peered":"2024-04-26T18:24:18.140854+0000","last_unstale":"2024-04-26T18:28:01.701339+0000","last_undegraded":"2024-04-26T18:28:01.701339+0000","last_fullsized":"2024-04-26T18:28:01.701339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:19:27.900701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":205,"num_read_kb":205,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452738+0000","last_change":"2024-04-26T18:24:20.961653+0000","last_active":"2024-04-26T18:27:57.452738+0000","last_peered":"2024-04-26T18:27:57.452738+0000","last_clean":"2024-04-26T18:27:57.452738+0000","last_became_active":"2024-04-26T18:24:20.961330+0000","last_became_peered":"2024-04-26T18:24:20.961330+0000","last_unstale":"2024-04-26T18:27:57.452738+0000","last_undegraded":"2024-04-26T18:27:57.452738+0000","last_fullsized":"2024-04-26T18:27:57.452738+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:04:19.033932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.695409+0000","last_change":"2024-04-26T18:24:17.933613+0000","last_active":"2024-04-26T18:27:57.695409+0000","last_peered":"2024-04-26T18:27:57.695409+0000","last_clean":"2024-04-26T18:27:57.695409+0000","last_became_active":"2024-04-26T18:24:17.933219+0000","last_became_peered":"2024-04-26T18:24:17.933219+0000","last_unstale":"2024-04-26T18:27:57.695409+0000","last_undegraded":"2024-04-26T18:27:57.695409+0000","last_fullsized":"2024-04-26T18:27:57.695409+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:22:04.215821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"94'148","reported_seq":410,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:59.571396+0000","last_change":"2024-04-26T18:22:57.585634+0000","last_active":"2024-04-26T18:27:59.571396+0000","last_peered":"2024-04-26T18:27:59.571396+0000","last_clean":"2024-04-26T18:27:59.571396+0000","last_became_active":"2024-04-26T18:22:57.585198+0000","last_became_peered":"2024-04-26T18:22:57.585198+0000","last_unstale":"2024-04-26T18:27:59.571396+0000","last_undegraded":"2024-04-26T18:27:59.571396+0000","last_fullsized":"2024-04-26T18:27:59.571396+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:19:31.594684+0000","last_clean_scrub_stamp":"2024-04-26T18:19:31.594684+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:56:11.861930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.692768+0000","last_change":"2024-04-26T18:24:20.950303+0000","last_active":"2024-04-26T18:27:57.692768+0000","last_peered":"2024-04-26T18:27:57.692768+0000","last_clean":"2024-04-26T18:27:57.692768+0000","last_became_active":"2024-04-26T18:24:20.950002+0000","last_became_peered":"2024-04-26T18:24:20.950002+0000","last_unstale":"2024-04-26T18:27:57.692768+0000","last_undegraded":"2024-04-26T18:27:57.692768+0000","last_fullsized":"2024-04-26T18:27:57.692768+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:44:00.912752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591327+0000","last_change":"2024-04-26T18:24:17.934924+0000","last_active":"2024-04-26T18:27:57.591327+0000","last_peered":"2024-04-26T18:27:57.591327+0000","last_clean":"2024-04-26T18:27:57.591327+0000","last_became_active":"2024-04-26T18:24:17.934598+0000","last_became_peered":"2024-04-26T18:24:17.934598+0000","last_unstale":"2024-04-26T18:27:57.591327+0000","last_undegraded":"2024-04-26T18:27:57.591327+0000","last_fullsized":"2024-04-26T18:27:57.591327+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:22:28.690043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545940+0000","last_change":"2024-04-26T18:24:18.142560+0000","last_active":"2024-04-26T18:27:57.545940+0000","last_peered":"2024-04-26T18:27:57.545940+0000","last_clean":"2024-04-26T18:27:57.545940+0000","last_became_active":"2024-04-26T18:24:18.142184+0000","last_became_peered":"2024-04-26T18:24:18.142184+0000","last_unstale":"2024-04-26T18:27:57.545940+0000","last_undegraded":"2024-04-26T18:27:57.545940+0000","last_fullsized":"2024-04-26T18:27:57.545940+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:00:26.656113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694369+0000","last_change":"2024-04-26T18:24:20.952073+0000","last_active":"2024-04-26T18:27:57.694369+0000","last_peered":"2024-04-26T18:27:57.694369+0000","last_clean":"2024-04-26T18:27:57.694369+0000","last_became_active":"2024-04-26T18:24:20.951761+0000","last_became_peered":"2024-04-26T18:24:20.951761+0000","last_unstale":"2024-04-26T18:27:57.694369+0000","last_undegraded":"2024-04-26T18:27:57.694369+0000","last_fullsized":"2024-04-26T18:27:57.694369+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:38:30.244060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"94'128","reported_seq":286,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:00.463784+0000","last_change":"2024-04-26T18:24:17.927171+0000","last_active":"2024-04-26T18:28:00.463784+0000","last_peered":"2024-04-26T18:28:00.463784+0000","last_clean":"2024-04-26T18:28:00.463784+0000","last_became_active":"2024-04-26T18:24:17.926816+0000","last_became_peered":"2024-04-26T18:24:17.926816+0000","last_unstale":"2024-04-26T18:28:00.463784+0000","last_undegraded":"2024-04-26T18:28:00.463784+0000","last_fullsized":"2024-04-26T18:28:00.463784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:56:18.249363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":422,"num_read_kb":376,"num_write":236,"num_write_kb":1414,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452834+0000","last_change":"2024-04-26T18:24:20.951684+0000","last_active":"2024-04-26T18:27:57.452834+0000","last_peered":"2024-04-26T18:27:57.452834+0000","last_clean":"2024-04-26T18:27:57.452834+0000","last_became_active":"2024-04-26T18:24:20.951364+0000","last_became_peered":"2024-04-26T18:24:20.951364+0000","last_unstale":"2024-04-26T18:27:57.452834+0000","last_undegraded":"2024-04-26T18:27:57.452834+0000","last_fullsized":"2024-04-26T18:27:57.452834+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:30:28.074579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.695676+0000","last_change":"2024-04-26T18:24:18.229770+0000","last_active":"2024-04-26T18:27:57.695676+0000","last_peered":"2024-04-26T18:27:57.695676+0000","last_clean":"2024-04-26T18:27:57.695676+0000","last_became_active":"2024-04-26T18:24:18.229446+0000","last_became_peered":"2024-04-26T18:24:18.229446+0000","last_unstale":"2024-04-26T18:27:57.695676+0000","last_undegraded":"2024-04-26T18:27:57.695676+0000","last_fullsized":"2024-04-26T18:27:57.695676+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:14.794797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697561+0000","last_change":"2024-04-26T18:24:20.960513+0000","last_active":"2024-04-26T18:27:57.697561+0000","last_peered":"2024-04-26T18:27:57.697561+0000","last_clean":"2024-04-26T18:27:57.697561+0000","last_became_active":"2024-04-26T18:24:20.959549+0000","last_became_peered":"2024-04-26T18:24:20.959549+0000","last_unstale":"2024-04-26T18:27:57.697561+0000","last_undegraded":"2024-04-26T18:27:57.697561+0000","last_fullsized":"2024-04-26T18:27:57.697561+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T20:18:37.766521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589273+0000","last_change":"2024-04-26T18:24:18.141549+0000","last_active":"2024-04-26T18:27:57.589273+0000","last_peered":"2024-04-26T18:27:57.589273+0000","last_clean":"2024-04-26T18:27:57.589273+0000","last_became_active":"2024-04-26T18:24:18.141172+0000","last_became_peered":"2024-04-26T18:24:18.141172+0000","last_unstale":"2024-04-26T18:27:57.589273+0000","last_undegraded":"2024-04-26T18:27:57.589273+0000","last_fullsized":"2024-04-26T18:27:57.589273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T18:38:15.878130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.545672+0000","last_change":"2024-04-26T18:24:20.948121+0000","last_active":"2024-04-26T18:27:57.545672+0000","last_peered":"2024-04-26T18:27:57.545672+0000","last_clean":"2024-04-26T18:27:57.545672+0000","last_became_active":"2024-04-26T18:24:20.947738+0000","last_became_peered":"2024-04-26T18:24:20.947738+0000","last_unstale":"2024-04-26T18:27:57.545672+0000","last_undegraded":"2024-04-26T18:27:57.545672+0000","last_fullsized":"2024-04-26T18:27:57.545672+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:45:04.237302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452547+0000","last_change":"2024-04-26T18:24:17.926631+0000","last_active":"2024-04-26T18:27:57.452547+0000","last_peered":"2024-04-26T18:27:57.452547+0000","last_clean":"2024-04-26T18:27:57.452547+0000","last_became_active":"2024-04-26T18:24:17.926259+0000","last_became_peered":"2024-04-26T18:24:17.926259+0000","last_unstale":"2024-04-26T18:27:57.452547+0000","last_undegraded":"2024-04-26T18:27:57.452547+0000","last_fullsized":"2024-04-26T18:27:57.452547+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T02:18:47.265122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984177+0000","last_change":"2024-04-26T18:24:20.947283+0000","last_active":"2024-04-26T18:27:57.984177+0000","last_peered":"2024-04-26T18:27:57.984177+0000","last_clean":"2024-04-26T18:27:57.984177+0000","last_became_active":"2024-04-26T18:24:20.946812+0000","last_became_peered":"2024-04-26T18:24:20.946812+0000","last_unstale":"2024-04-26T18:27:57.984177+0000","last_undegraded":"2024-04-26T18:27:57.984177+0000","last_fullsized":"2024-04-26T18:27:57.984177+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:11:33.730698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"94'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452339+0000","last_change":"2024-04-26T18:24:17.931843+0000","last_active":"2024-04-26T18:27:57.452339+0000","last_peered":"2024-04-26T18:27:57.452339+0000","last_clean":"2024-04-26T18:27:57.452339+0000","last_became_active":"2024-04-26T18:24:17.931442+0000","last_became_peered":"2024-04-26T18:24:17.931442+0000","last_unstale":"2024-04-26T18:27:57.452339+0000","last_undegraded":"2024-04-26T18:27:57.452339+0000","last_fullsized":"2024-04-26T18:27:57.452339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:40:09.649435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452567+0000","last_change":"2024-04-26T18:24:20.955661+0000","last_active":"2024-04-26T18:27:57.452567+0000","last_peered":"2024-04-26T18:27:57.452567+0000","last_clean":"2024-04-26T18:27:57.452567+0000","last_became_active":"2024-04-26T18:24:20.955344+0000","last_became_peered":"2024-04-26T18:24:20.955344+0000","last_unstale":"2024-04-26T18:27:57.452567+0000","last_undegraded":"2024-04-26T18:27:57.452567+0000","last_fullsized":"2024-04-26T18:27:57.452567+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:29:44.274381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984200+0000","last_change":"2024-04-26T18:25:06.315466+0000","last_active":"2024-04-26T18:27:57.984200+0000","last_peered":"2024-04-26T18:27:57.984200+0000","last_clean":"2024-04-26T18:27:57.984200+0000","last_became_active":"2024-04-26T18:25:06.301788+0000","last_became_peered":"2024-04-26T18:25:06.301788+0000","last_unstale":"2024-04-26T18:27:57.984200+0000","last_undegraded":"2024-04-26T18:27:57.984200+0000","last_fullsized":"2024-04-26T18:27:57.984200+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:34:12.998583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591736+0000","last_change":"2024-04-26T18:24:20.955746+0000","last_active":"2024-04-26T18:27:57.591736+0000","last_peered":"2024-04-26T18:27:57.591736+0000","last_clean":"2024-04-26T18:27:57.591736+0000","last_became_active":"2024-04-26T18:24:20.953652+0000","last_became_peered":"2024-04-26T18:24:20.953652+0000","last_unstale":"2024-04-26T18:27:57.591736+0000","last_undegraded":"2024-04-26T18:27:57.591736+0000","last_fullsized":"2024-04-26T18:27:57.591736+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:09:31.414185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"94'8","reported_seq":207,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:01.595662+0000","last_change":"2024-04-26T18:25:06.309393+0000","last_active":"2024-04-26T18:28:01.595662+0000","last_peered":"2024-04-26T18:28:01.595662+0000","last_clean":"2024-04-26T18:28:01.595662+0000","last_became_active":"2024-04-26T18:25:06.302434+0000","last_became_peered":"2024-04-26T18:25:06.302434+0000","last_unstale":"2024-04-26T18:28:01.595662+0000","last_undegraded":"2024-04-26T18:28:01.595662+0000","last_fullsized":"2024-04-26T18:28:01.595662+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:58:54.730295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":184,"num_read_kb":4,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1561,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591717+0000","last_change":"2024-04-26T18:24:20.954238+0000","last_active":"2024-04-26T18:27:57.591717+0000","last_peered":"2024-04-26T18:27:57.591717+0000","last_clean":"2024-04-26T18:27:57.591717+0000","last_became_active":"2024-04-26T18:24:20.953883+0000","last_became_peered":"2024-04-26T18:24:20.953883+0000","last_unstale":"2024-04-26T18:27:57.591717+0000","last_undegraded":"2024-04-26T18:27:57.591717+0000","last_fullsized":"2024-04-26T18:27:57.591717+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:12:10.580835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"94'75","reported_seq":150,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:28:00.459215+0000","last_change":"2024-04-26T18:24:17.926901+0000","last_active":"2024-04-26T18:28:00.459215+0000","last_peered":"2024-04-26T18:28:00.459215+0000","last_clean":"2024-04-26T18:28:00.459215+0000","last_became_active":"2024-04-26T18:24:17.926291+0000","last_became_peered":"2024-04-26T18:24:17.926291+0000","last_unstale":"2024-04-26T18:28:00.459215+0000","last_undegraded":"2024-04-26T18:28:00.459215+0000","last_fullsized":"2024-04-26T18:28:00.459215+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:32:17.695198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":44,"num_write":84,"num_write_kb":1075,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591253+0000","last_change":"2024-04-26T18:24:20.950578+0000","last_active":"2024-04-26T18:27:57.591253+0000","last_peered":"2024-04-26T18:27:57.591253+0000","last_clean":"2024-04-26T18:27:57.591253+0000","last_became_active":"2024-04-26T18:24:20.950174+0000","last_became_peered":"2024-04-26T18:24:20.950174+0000","last_unstale":"2024-04-26T18:27:57.591253+0000","last_undegraded":"2024-04-26T18:27:57.591253+0000","last_fullsized":"2024-04-26T18:27:57.591253+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:52:22.332813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694486+0000","last_change":"2024-04-26T18:24:17.928054+0000","last_active":"2024-04-26T18:27:57.694486+0000","last_peered":"2024-04-26T18:27:57.694486+0000","last_clean":"2024-04-26T18:27:57.694486+0000","last_became_active":"2024-04-26T18:24:17.927715+0000","last_became_peered":"2024-04-26T18:24:17.927715+0000","last_unstale":"2024-04-26T18:27:57.694486+0000","last_undegraded":"2024-04-26T18:27:57.694486+0000","last_fullsized":"2024-04-26T18:27:57.694486+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:16.904627+0000","last_clean_scrub_stamp":"2024-04-26T18:24:16.904627+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:35:11.155436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.591822+0000","last_change":"2024-04-26T18:24:20.944512+0000","last_active":"2024-04-26T18:27:57.591822+0000","last_peered":"2024-04-26T18:27:57.591822+0000","last_clean":"2024-04-26T18:27:57.591822+0000","last_became_active":"2024-04-26T18:24:20.944123+0000","last_became_peered":"2024-04-26T18:24:20.944123+0000","last_unstale":"2024-04-26T18:27:57.591822+0000","last_undegraded":"2024-04-26T18:27:57.591822+0000","last_fullsized":"2024-04-26T18:27:57.591822+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:00:12.240206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984272+0000","last_change":"2024-04-26T18:24:20.959138+0000","last_active":"2024-04-26T18:27:57.984272+0000","last_peered":"2024-04-26T18:27:57.984272+0000","last_clean":"2024-04-26T18:27:57.984272+0000","last_became_active":"2024-04-26T18:24:20.958675+0000","last_became_peered":"2024-04-26T18:24:20.958675+0000","last_unstale":"2024-04-26T18:27:57.984272+0000","last_undegraded":"2024-04-26T18:27:57.984272+0000","last_fullsized":"2024-04-26T18:27:57.984272+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T19:34:32.103380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452173+0000","last_change":"2024-04-26T18:24:20.961689+0000","last_active":"2024-04-26T18:27:57.452173+0000","last_peered":"2024-04-26T18:27:57.452173+0000","last_clean":"2024-04-26T18:27:57.452173+0000","last_became_active":"2024-04-26T18:24:20.961374+0000","last_became_peered":"2024-04-26T18:24:20.961374+0000","last_unstale":"2024-04-26T18:27:57.452173+0000","last_undegraded":"2024-04-26T18:27:57.452173+0000","last_fullsized":"2024-04-26T18:27:57.452173+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:25:55.243887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452150+0000","last_change":"2024-04-26T18:24:20.950711+0000","last_active":"2024-04-26T18:27:57.452150+0000","last_peered":"2024-04-26T18:27:57.452150+0000","last_clean":"2024-04-26T18:27:57.452150+0000","last_became_active":"2024-04-26T18:24:20.950371+0000","last_became_peered":"2024-04-26T18:24:20.950371+0000","last_unstale":"2024-04-26T18:27:57.452150+0000","last_undegraded":"2024-04-26T18:27:57.452150+0000","last_fullsized":"2024-04-26T18:27:57.452150+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:22:35.307921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.697143+0000","last_change":"2024-04-26T18:24:20.959990+0000","last_active":"2024-04-26T18:27:57.697143+0000","last_peered":"2024-04-26T18:27:57.697143+0000","last_clean":"2024-04-26T18:27:57.697143+0000","last_became_active":"2024-04-26T18:24:20.959514+0000","last_became_peered":"2024-04-26T18:24:20.959514+0000","last_unstale":"2024-04-26T18:27:57.697143+0000","last_undegraded":"2024-04-26T18:27:57.697143+0000","last_fullsized":"2024-04-26T18:27:57.697143+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:42:00.184448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694549+0000","last_change":"2024-04-26T18:24:20.957338+0000","last_active":"2024-04-26T18:27:57.694549+0000","last_peered":"2024-04-26T18:27:57.694549+0000","last_clean":"2024-04-26T18:27:57.694549+0000","last_became_active":"2024-04-26T18:24:20.956999+0000","last_became_peered":"2024-04-26T18:24:20.956999+0000","last_unstale":"2024-04-26T18:27:57.694549+0000","last_undegraded":"2024-04-26T18:27:57.694549+0000","last_fullsized":"2024-04-26T18:27:57.694549+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T23:47:05.066902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":33,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984324+0000","last_change":"2024-04-26T18:24:20.953634+0000","last_active":"2024-04-26T18:27:57.984324+0000","last_peered":"2024-04-26T18:27:57.984324+0000","last_clean":"2024-04-26T18:27:57.984324+0000","last_became_active":"2024-04-26T18:24:20.953259+0000","last_became_peered":"2024-04-26T18:24:20.953259+0000","last_unstale":"2024-04-26T18:27:57.984324+0000","last_undegraded":"2024-04-26T18:27:57.984324+0000","last_fullsized":"2024-04-26T18:27:57.984324+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T01:26:00.095344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452650+0000","last_change":"2024-04-26T18:24:20.954997+0000","last_active":"2024-04-26T18:27:57.452650+0000","last_peered":"2024-04-26T18:27:57.452650+0000","last_clean":"2024-04-26T18:27:57.452650+0000","last_became_active":"2024-04-26T18:24:20.954223+0000","last_became_peered":"2024-04-26T18:24:20.954223+0000","last_unstale":"2024-04-26T18:27:57.452650+0000","last_undegraded":"2024-04-26T18:27:57.452650+0000","last_fullsized":"2024-04-26T18:27:57.452650+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:35:14.092455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":29,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452832+0000","last_change":"2024-04-26T18:24:20.957056+0000","last_active":"2024-04-26T18:27:57.452832+0000","last_peered":"2024-04-26T18:27:57.452832+0000","last_clean":"2024-04-26T18:27:57.452832+0000","last_became_active":"2024-04-26T18:24:20.956737+0000","last_became_peered":"2024-04-26T18:24:20.956737+0000","last_unstale":"2024-04-26T18:27:57.452832+0000","last_undegraded":"2024-04-26T18:27:57.452832+0000","last_fullsized":"2024-04-26T18:27:57.452832+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:44:48.358862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.550741+0000","last_change":"2024-04-26T18:25:06.350206+0000","last_active":"2024-04-26T18:27:57.550741+0000","last_peered":"2024-04-26T18:27:57.550741+0000","last_clean":"2024-04-26T18:27:57.550741+0000","last_became_active":"2024-04-26T18:25:06.349866+0000","last_became_peered":"2024-04-26T18:25:06.349866+0000","last_unstale":"2024-04-26T18:27:57.550741+0000","last_undegraded":"2024-04-26T18:27:57.550741+0000","last_fullsized":"2024-04-26T18:27:57.550741+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:41.909056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6],"acting":[5,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.452351+0000","last_change":"2024-04-26T18:24:20.956000+0000","last_active":"2024-04-26T18:27:57.452351+0000","last_peered":"2024-04-26T18:27:57.452351+0000","last_clean":"2024-04-26T18:27:57.452351+0000","last_became_active":"2024-04-26T18:24:20.955677+0000","last_became_peered":"2024-04-26T18:24:20.955677+0000","last_unstale":"2024-04-26T18:27:57.452351+0000","last_undegraded":"2024-04-26T18:27:57.452351+0000","last_fullsized":"2024-04-26T18:27:57.452351+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:16:20.895674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.589421+0000","last_change":"2024-04-26T18:24:20.959317+0000","last_active":"2024-04-26T18:27:57.589421+0000","last_peered":"2024-04-26T18:27:57.589421+0000","last_clean":"2024-04-26T18:27:57.589421+0000","last_became_active":"2024-04-26T18:24:20.958963+0000","last_became_peered":"2024-04-26T18:24:20.958963+0000","last_unstale":"2024-04-26T18:27:57.589421+0000","last_undegraded":"2024-04-26T18:27:57.589421+0000","last_fullsized":"2024-04-26T18:27:57.589421+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T22:47:08.231089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.694689+0000","last_change":"2024-04-26T18:24:20.954445+0000","last_active":"2024-04-26T18:27:57.694689+0000","last_peered":"2024-04-26T18:27:57.694689+0000","last_clean":"2024-04-26T18:27:57.694689+0000","last_became_active":"2024-04-26T18:24:20.953852+0000","last_became_peered":"2024-04-26T18:24:20.953852+0000","last_unstale":"2024-04-26T18:27:57.694689+0000","last_undegraded":"2024-04-26T18:27:57.694689+0000","last_fullsized":"2024-04-26T18:27:57.694689+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:16:05.636897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-26T18:27:57.984195+0000","last_change":"2024-04-26T18:24:20.958023+0000","last_active":"2024-04-26T18:27:57.984195+0000","last_peered":"2024-04-26T18:27:57.984195+0000","last_clean":"2024-04-26T18:27:57.984195+0000","last_became_active":"2024-04-26T18:24:20.957642+0000","last_became_peered":"2024-04-26T18:24:20.957642+0000","last_unstale":"2024-04-26T18:27:57.984195+0000","last_undegraded":"2024-04-26T18:27:57.984195+0000","last_fullsized":"2024-04-26T18:27:57.984195+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":86,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T18:24:19.927191+0000","last_clean_scrub_stamp":"2024-04-26T18:24:19.927191+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T21:51:46.332710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":106773,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":807456,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2531,"num_read_kb":1555,"num_write":570,"num_write_kb":2659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":1583,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":811008,"data_stored":2422368,"data_compressed":109023,"data_compressed_allocated":147456,"data_compressed_original":2125824,"omap_allocated":0,"internal_metadata":0},"log_size":426,"ondisk_log_size":426,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351035,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1036288,"data_stored":510598,"data_compressed":14494,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612672,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36560,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708592,"statfs":{"total":95995035648,"available":95957598208,"internally_reserved":0,"allocated":1245184,"data_stored":1505872,"data_compressed":67130,"data_compressed_allocated":90112,"data_compressed_original":946176,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776198,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":902799,"data_compressed":33790,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037835,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32224,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712928,"statfs":{"total":95995035648,"available":95962038272,"internally_reserved":0,"allocated":1073152,"data_stored":903155,"data_compressed":33790,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299471,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32256,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712896,"statfs":{"total":95995035648,"available":95962005504,"internally_reserved":0,"allocated":1105920,"data_stored":1104966,"data_compressed":50085,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462998,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":509691,"data_compressed":14494,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32272,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712880,"statfs":{"total":95995035648,"available":95961989120,"internally_reserved":0,"allocated":1122304,"data_stored":588482,"data_compressed":16337,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738483,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1060864,"data_stored":511562,"data_compressed":14494,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411434,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28116,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717036,"statfs":{"total":95995035648,"available":95966244864,"internally_reserved":0,"allocated":1126400,"data_stored":586362,"data_compressed":16337,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149796,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1126400,"data_stored":1230573,"data_compressed":48992,"data_compressed_allocated":65536,"data_compressed_original":741376,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":34,"seq":146028888160,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1040384,"data_stored":510334,"data_compressed":14494,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724633,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32320,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712832,"statfs":{"total":95995035648,"available":95961939968,"internally_reserved":0,"allocated":1163264,"data_stored":1431312,"data_compressed":65287,"data_compressed_allocated":86016,"data_compressed_original":937984,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35591,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":80021,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":77901,"data_compressed":1843,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34498,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":332366,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":394694,"data_compressed":19296,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19296,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":406926,"data_compressed":17045,"data_compressed_allocated":24576,"data_compressed_original":315392,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2137,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T18:28:09.247 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:28:09.489 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:09.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:09 smithi084 ceph-mon[29141]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 83 op/s 2024-04-26T18:28:09.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:09 smithi103 ceph-mon[32727]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 83 op/s 2024-04-26T18:28:09.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:09 smithi174 ceph-mon[32705]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 132 KiB/s wr, 83 op/s 2024-04-26T18:28:10.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:10 smithi084 ceph-mon[29141]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:28:10.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:10 smithi103 ceph-mon[32727]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:28:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:10 smithi174 ceph-mon[32705]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T18:28:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.211+0000 7f98fe592700 1 -- 172.21.15.84:0/1137398560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f8108390 msgr2=0x7f98f8108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.211+0000 7f98fe592700 1 --2- 172.21.15.84:0/1137398560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f8108390 0x7f98f8108770 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f98e8009990 tx=0x7f98e8009ca0 comp rx=0 tx=0).stop 2024-04-26T18:28:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.211+0000 7f98fe592700 1 -- 172.21.15.84:0/1137398560 shutdown_connections 2024-04-26T18:28:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.211+0000 7f98fe592700 1 --2- 172.21.15.84:0/1137398560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f8108390 0x7f98f8108770 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.211+0000 7f98fe592700 1 --2- 172.21.15.84:0/1137398560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98f8102ce0 0x7f98f810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.211+0000 7f98fe592700 1 --2- 172.21.15.84:0/1137398560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98f8102320 0x7f98f81027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.211+0000 7f98fe592700 1 -- 172.21.15.84:0/1137398560 >> 172.21.15.84:0/1137398560 conn(0x7f98f80792d0 msgr2=0x7f98f80796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.211+0000 7f98fe592700 1 -- 172.21.15.84:0/1137398560 shutdown_connections 2024-04-26T18:28:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.211+0000 7f98fe592700 1 -- 172.21.15.84:0/1137398560 wait complete. 2024-04-26T18:28:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.211+0000 7f98fe592700 1 Processor -- start 2024-04-26T18:28:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.211+0000 7f98fe592700 1 -- start start 2024-04-26T18:28:11.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98fe592700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f8102320 0x7f98f819f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98fe592700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98f8102ce0 0x7f98f819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98fe592700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98f8108390 0x7f98f8199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98fe592700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f98f8112560 con 0x7f98f8102320 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98fe592700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f98f81123e0 con 0x7f98f8102ce0 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98fe592700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f98f81126e0 con 0x7f98f8108390 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f8102320 0x7f98f819f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f8102320 0x7f98f819f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45680/0 (socket says 172.21.15.84:45680) 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f7fff700 1 -- 172.21.15.84:0/3896592590 learned_addr learned my addr 172.21.15.84:0/3896592590 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98fcb2f700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98f8108390 0x7f98f8199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f77fe700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98f8102ce0 0x7f98f819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f7fff700 1 -- 172.21.15.84:0/3896592590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98f8108390 msgr2=0x7f98f8199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f7fff700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98f8108390 0x7f98f8199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f7fff700 1 -- 172.21.15.84:0/3896592590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98f8102ce0 msgr2=0x7f98f819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:11.244 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f7fff700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98f8102ce0 0x7f98f819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f7fff700 1 -- 172.21.15.84:0/3896592590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98f819a0d0 con 0x7f98f8102320 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f77fe700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98f8102ce0 0x7f98f819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f7fff700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f8102320 0x7f98f819f7b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f98e8000c00 tx=0x7f98e80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f57fa700 1 -- 172.21.15.84:0/3896592590 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e8027070 con 0x7f98f8102320 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98fe592700 1 -- 172.21.15.84:0/3896592590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98f819a360 con 0x7f98f8102320 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f57fa700 1 -- 172.21.15.84:0/3896592590 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98e8003f20 con 0x7f98f8102320 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98f57fa700 1 -- 172.21.15.84:0/3896592590 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98e8022cf0 con 0x7f98f8102320 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.212+0000 7f98fe592700 1 -- 172.21.15.84:0/3896592590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98f81a6680 con 0x7f98f8102320 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.213+0000 7f98fe592700 1 -- 172.21.15.84:0/3896592590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98f806a110 con 0x7f98f8102320 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.214+0000 7f98f57fa700 1 -- 172.21.15.84:0/3896592590 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f98e802a070 con 0x7f98f8102320 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.215+0000 7f98f57fa700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f98e00a3f60 0x7f98e00a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.215+0000 7f98f57fa700 1 -- 172.21.15.84:0/3896592590 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f98e80d1c70 con 0x7f98f8102320 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.215+0000 7f98f77fe700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f98e00a3f60 0x7f98e00a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.215+0000 7f98f77fe700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f98e00a3f60 0x7f98e00a6420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f98f80ff420 tx=0x7f98ec00b2c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:11.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.219+0000 7f98f57fa700 1 -- 172.21.15.84:0/3896592590 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f98e8013070 con 0x7f98f8102320 2024-04-26T18:28:11.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.472+0000 7f98fe592700 1 -- 172.21.15.84:0/3896592590 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98f819aea0 con 0x7f98f8102320 2024-04-26T18:28:11.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.472+0000 7f98f57fa700 1 -- 172.21.15.84:0/3896592590 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f98e809e150 con 0x7f98f8102320 2024-04-26T18:28:11.474 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:28:11.474 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:28:11.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.475+0000 7f98fe592700 1 -- 172.21.15.84:0/3896592590 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f98e00a3f60 msgr2=0x7f98e00a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:11.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.475+0000 7f98fe592700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f98e00a3f60 0x7f98e00a6420 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f98f80ff420 tx=0x7f98ec00b2c0 comp rx=0 tx=0).stop 2024-04-26T18:28:11.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.475+0000 7f98fe592700 1 -- 172.21.15.84:0/3896592590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f8102320 msgr2=0x7f98f819f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:11.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.475+0000 7f98fe592700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f8102320 0x7f98f819f7b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f98e8000c00 tx=0x7f98e80047b0 comp rx=0 tx=0).stop 2024-04-26T18:28:11.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.475+0000 7f98fe592700 1 -- 172.21.15.84:0/3896592590 shutdown_connections 2024-04-26T18:28:11.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.475+0000 7f98fe592700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f98e00a3f60 0x7f98e00a6420 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:11.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.475+0000 7f98fe592700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f98f8102320 0x7f98f819f7b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:11.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.475+0000 7f98fe592700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f98f8108390 0x7f98f8199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:11.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.475+0000 7f98fe592700 1 --2- 172.21.15.84:0/3896592590 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f98f8102ce0 0x7f98f819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:11.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.475+0000 7f98fe592700 1 -- 172.21.15.84:0/3896592590 >> 172.21.15.84:0/3896592590 conn(0x7f98f80792d0 msgr2=0x7f98f80ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:11.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.475+0000 7f98fe592700 1 -- 172.21.15.84:0/3896592590 shutdown_connections 2024-04-26T18:28:11.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:11.475+0000 7f98fe592700 1 -- 172.21.15.84:0/3896592590 wait complete. 2024-04-26T18:28:11.477 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:28:11.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:11 smithi084 ceph-mon[29141]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 108 KiB/s wr, 72 op/s 2024-04-26T18:28:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:11 smithi103 ceph-mon[32727]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 108 KiB/s wr, 72 op/s 2024-04-26T18:28:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:11 smithi174 ceph-mon[32705]: pgmap v9: 97 pgs: 97 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 108 KiB/s wr, 72 op/s 2024-04-26T18:28:12.188 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:28:12.433 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:12.556 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:12 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3896592590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:12.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:12 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3896592590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:12.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:12 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3896592590' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.385+0000 7f36a5978640 1 -- 172.21.15.84:0/999858071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a0102940 msgr2=0x7f36a0102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.385+0000 7f36a5978640 1 --2- 172.21.15.84:0/999858071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a0102940 0x7f36a0102dc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3694009930 tx=0x7f369402a420 comp rx=0 tx=0).stop 2024-04-26T18:28:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.385+0000 7f36a5978640 1 -- 172.21.15.84:0/999858071 shutdown_connections 2024-04-26T18:28:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.385+0000 7f36a5978640 1 --2- 172.21.15.84:0/999858071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36a0103300 0x7f36a010f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.385+0000 7f36a5978640 1 --2- 172.21.15.84:0/999858071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a0102940 0x7f36a0102dc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.385+0000 7f36a5978640 1 --2- 172.21.15.84:0/999858071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36a0108950 0x7f36a0108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.385+0000 7f36a5978640 1 -- 172.21.15.84:0/999858071 >> 172.21.15.84:0/999858071 conn(0x7f36a0079dc0 msgr2=0x7f36a007a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.385+0000 7f36a5978640 1 -- 172.21.15.84:0/999858071 shutdown_connections 2024-04-26T18:28:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.385+0000 7f36a5978640 1 -- 172.21.15.84:0/999858071 wait complete. 2024-04-26T18:28:13.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.386+0000 7f36a5978640 1 Processor -- start 2024-04-26T18:28:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.386+0000 7f36a5978640 1 -- start start 2024-04-26T18:28:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.386+0000 7f36a5978640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a0102940 0x7f36a0199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.386+0000 7f36a5978640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36a0103300 0x7f36a0199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.386+0000 7f36a5978640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36a0108950 0x7f36a019d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.386+0000 7f36a5978640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0112be0 con 0x7f36a0102940 2024-04-26T18:28:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.386+0000 7f36a5978640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0112a60 con 0x7f36a0108950 2024-04-26T18:28:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.386+0000 7f36a5978640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36a0112d60 con 0x7f36a0103300 2024-04-26T18:28:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.386+0000 7f369effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a0102940 0x7f36a0199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.386+0000 7f369effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a0102940 0x7f36a0199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45712/0 (socket says 172.21.15.84:45712) 2024-04-26T18:28:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.386+0000 7f369effd640 1 -- 172.21.15.84:0/807558584 learned_addr learned my addr 172.21.15.84:0/807558584 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.386+0000 7f369f7fe640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36a0108950 0x7f36a019d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.387+0000 7f369e7fc640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36a0103300 0x7f36a0199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.387+0000 7f369effd640 1 -- 172.21.15.84:0/807558584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36a0103300 msgr2=0x7f36a0199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.387+0000 7f369effd640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36a0103300 0x7f36a0199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.387+0000 7f369effd640 1 -- 172.21.15.84:0/807558584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36a0108950 msgr2=0x7f36a019d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.387+0000 7f369effd640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36a0108950 0x7f36a019d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.387+0000 7f369effd640 1 -- 172.21.15.84:0/807558584 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a019e010 con 0x7f36a0102940 2024-04-26T18:28:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.387+0000 7f369effd640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a0102940 0x7f36a0199020 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f369000c920 tx=0x7f369000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.387+0000 7f36a4976640 1 -- 172.21.15.84:0/807558584 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690007c70 con 0x7f36a0102940 2024-04-26T18:28:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.387+0000 7f36a4976640 1 -- 172.21.15.84:0/807558584 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3690007e10 con 0x7f36a0102940 2024-04-26T18:28:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.387+0000 7f36a5978640 1 -- 172.21.15.84:0/807558584 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a019e300 con 0x7f36a0102940 2024-04-26T18:28:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.387+0000 7f36a4976640 1 -- 172.21.15.84:0/807558584 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3690018820 con 0x7f36a0102940 2024-04-26T18:28:13.390 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.387+0000 7f36a5978640 1 -- 172.21.15.84:0/807558584 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a01a5b90 con 0x7f36a0102940 2024-04-26T18:28:13.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.389+0000 7f36a4976640 1 -- 172.21.15.84:0/807558584 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f369000f450 con 0x7f36a0102940 2024-04-26T18:28:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.389+0000 7f36a876b740 1 Processor -- start 2024-04-26T18:28:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.389+0000 7f36a876b740 1 -- start start 2024-04-26T18:28:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f36a4976640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36880a3ff0 0x7f36880a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f36a876b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc5412e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f36a876b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc541820 0x5589dc54bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f36a876b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc54c0e0 0x5589dc54e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f36a876b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5589dc4a7460 con 0x5589dc540f00 2024-04-26T18:28:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f36a876b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5589dc461590 con 0x5589dc541820 2024-04-26T18:28:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f36a876b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5589dc4a3420 con 0x5589dc54c0e0 2024-04-26T18:28:13.398 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f369f7fe640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36880a3ff0 0x7f36880a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f36a4976640 1 -- 172.21.15.84:0/807558584 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f36900d0b40 con 0x7f36a0102940 2024-04-26T18:28:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f369effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc5412e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f369f7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc54c0e0 0x5589dc54e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f369effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc5412e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45714/0 (socket says 172.21.15.84:45714) 2024-04-26T18:28:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f369effd640 1 -- 172.21.15.84:0/45965118 learned_addr learned my addr 172.21.15.84:0/45965118 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f369e7fc640 1 --2- 172.21.15.84:0/45965118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc541820 0x5589dc54bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f369effd640 1 -- 172.21.15.84:0/45965118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc54c0e0 msgr2=0x5589dc54e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f369effd640 1 --2- 172.21.15.84:0/45965118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc54c0e0 0x5589dc54e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f369effd640 1 -- 172.21.15.84:0/45965118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc541820 msgr2=0x5589dc54bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:13.399 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f369effd640 1 --2- 172.21.15.84:0/45965118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc541820 0x5589dc54bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f369effd640 1 -- 172.21.15.84:0/45965118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589dc46e0d0 con 0x5589dc540f00 2024-04-26T18:28:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f369f7fe640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36880a3ff0 0x7f36880a64b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3698005fd0 tx=0x7f3698009b70 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f369effd640 1 --2- 172.21.15.84:0/45965118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc5412e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f369009c590 tx=0x7f369009c2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82d1201853e1e0aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.390+0000 7f3685ffb640 1 -- 172.21.15.84:0/45965118 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36900a0c40 con 0x5589dc540f00 2024-04-26T18:28:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.391+0000 7f3685ffb640 1 -- 172.21.15.84:0/45965118 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36900a5500 con 0x5589dc540f00 2024-04-26T18:28:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.391+0000 7f3685ffb640 1 -- 172.21.15.84:0/45965118 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369008f810 con 0x5589dc540f00 2024-04-26T18:28:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.391+0000 7f36a876b740 1 -- 172.21.15.84:0/45965118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 msgr2=0x5589dc5412e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.391+0000 7f36a876b740 1 --2- 172.21.15.84:0/45965118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc5412e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f369009c590 tx=0x7f369009c2a0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.391+0000 7f36a876b740 1 -- 172.21.15.84:0/45965118 shutdown_connections 2024-04-26T18:28:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.391+0000 7f36a876b740 1 --2- 172.21.15.84:0/45965118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc54c0e0 0x5589dc54e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.391+0000 7f36a876b740 1 --2- 172.21.15.84:0/45965118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc541820 0x5589dc54bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.391+0000 7f36a876b740 1 --2- 172.21.15.84:0/45965118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc5412e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.391+0000 7f36a876b740 1 -- 172.21.15.84:0/45965118 >> 172.21.15.84:0/45965118 conn(0x5589dc53d410 msgr2=0x5589dc53d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.391+0000 7f36a876b740 1 -- 172.21.15.84:0/45965118 shutdown_connections 2024-04-26T18:28:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.391+0000 7f36a876b740 1 -- 172.21.15.84:0/45965118 wait complete. 2024-04-26T18:28:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f36a876b740 1 -- start start 2024-04-26T18:28:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f36a876b740 1 Processor -- start 2024-04-26T18:28:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f36a876b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc549060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f36a876b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc541820 0x5589dc5495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f36a876b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc54c0e0 0x5589dc549ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f36a876b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5589dc54ed10 con 0x5589dc540f00 2024-04-26T18:28:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f36a876b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5589dc42eb60 con 0x5589dc54c0e0 2024-04-26T18:28:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f36a876b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5589dc54ee90 con 0x5589dc541820 2024-04-26T18:28:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f36a876b740 10 client.0 _ll_register_callbacks cb 0x5589dc4532b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:13.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f369effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc549060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f369effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc549060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45720/0 (socket says 172.21.15.84:45720) 2024-04-26T18:28:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f369effd640 1 -- 172.21.15.84:0/3449958253 learned_addr learned my addr 172.21.15.84:0/3449958253 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f369f7fe640 1 --2- 172.21.15.84:0/3449958253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc54c0e0 0x5589dc549ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f369e7fc640 1 --2- 172.21.15.84:0/3449958253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc541820 0x5589dc5495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f369effd640 1 -- 172.21.15.84:0/3449958253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc541820 msgr2=0x5589dc5495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f369effd640 1 --2- 172.21.15.84:0/3449958253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc541820 0x5589dc5495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f369effd640 1 -- 172.21.15.84:0/3449958253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc54c0e0 msgr2=0x5589dc549ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f369effd640 1 --2- 172.21.15.84:0/3449958253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc54c0e0 0x5589dc549ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.392+0000 7f369effd640 1 -- 172.21.15.84:0/3449958253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589dc1b9210 con 0x5589dc540f00 2024-04-26T18:28:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f369effd640 1 --2- 172.21.15.84:0/3449958253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc549060 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3690007590 tx=0x7f369009ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80240e337dda424 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f367effd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f367effd640 1 -- 172.21.15.84:0/3449958253 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36900a5270 con 0x5589dc540f00 2024-04-26T18:28:13.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f367effd640 1 -- 172.21.15.84:0/3449958253 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f369009dbc0 con 0x5589dc540f00 2024-04-26T18:28:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f36a876b740 1 -- 172.21.15.84:0/3449958253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36900a5500 con 0x5589dc540f00 2024-04-26T18:28:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f367effd640 1 -- 172.21.15.84:0/3449958253 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f369000fb50 con 0x5589dc540f00 2024-04-26T18:28:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f36a876b740 10 client.15384 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f36a876b740 1 -- 172.21.15.84:0/3449958253 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589dc285620 con 0x5589dc540f00 2024-04-26T18:28:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f367effd640 1 -- 172.21.15.84:0/3449958253 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36900a5500 con 0x5589dc540f00 2024-04-26T18:28:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f367effd640 1 -- 172.21.15.84:0/3449958253 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f369007f1a0 con 0x5589dc540f00 2024-04-26T18:28:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f36a876b740 10 client.15384 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f36a876b740 10 client.15384 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f36a876b740 20 client.15384 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f36a876b740 1 --2- 172.21.15.84:0/3449958253 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5589dc55c090 0x5589dc55e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f36a876b740 4 client.15384 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.393+0000 7f36a876b740 1 -- 172.21.15.84:0/3449958253 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589dc4b5070 con 0x5589dc55c090 2024-04-26T18:28:13.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.394+0000 7f369e7fc640 1 --2- 172.21.15.84:0/3449958253 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5589dc55c090 0x5589dc55e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.394+0000 7f369e7fc640 1 --2- 172.21.15.84:0/3449958253 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5589dc55c090 0x5589dc55e480 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f2f9e59e1cc35a8 server_cookie=1c5a4cade7579308 in_seq=0 out_seq=0 2024-04-26T18:28:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.394+0000 7f367effd640 10 client.15384 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.395+0000 7f367effd640 1 -- 172.21.15.84:0/3449958253 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5589dc4b5070 con 0x5589dc55c090 2024-04-26T18:28:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.395+0000 7f367effd640 10 client.15384 handle_command_reply: tid=0 2024-04-26T18:28:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.395+0000 7f36a876b740 1 client.15384 shutdown 2024-04-26T18:28:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.395+0000 7f36a876b740 1 -- 172.21.15.84:0/3449958253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 msgr2=0x5589dc549060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.395+0000 7f36a876b740 1 --2- 172.21.15.84:0/3449958253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc549060 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3690007590 tx=0x7f369009ab90 comp rx=0 tx=0).stop 2024-04-26T18:28:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.395+0000 7f36a876b740 1 -- 172.21.15.84:0/3449958253 shutdown_connections 2024-04-26T18:28:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.395+0000 7f36a876b740 1 --2- 172.21.15.84:0/3449958253 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5589dc55c090 0x5589dc55e480 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.395+0000 7f367effd640 0 client.15384 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.395+0000 7f36a876b740 1 --2- 172.21.15.84:0/3449958253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc54c0e0 0x5589dc549ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.395+0000 7f36a876b740 1 --2- 172.21.15.84:0/3449958253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc541820 0x5589dc5495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.395+0000 7f36a876b740 1 --2- 172.21.15.84:0/3449958253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc549060 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.395+0000 7f36a876b740 1 -- 172.21.15.84:0/3449958253 >> 172.21.15.84:0/3449958253 conn(0x5589dc53d410 msgr2=0x5589dc543190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.396+0000 7f36a876b740 1 -- 172.21.15.84:0/3449958253 shutdown_connections 2024-04-26T18:28:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.396+0000 7f36a876b740 1 -- 172.21.15.84:0/3449958253 wait complete. 2024-04-26T18:28:13.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.396+0000 7f36a876b740 20 client.15384 trim_cache size 0 max 16384 2024-04-26T18:28:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f36a876b740 1 Processor -- start 2024-04-26T18:28:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f36a876b740 1 -- start start 2024-04-26T18:28:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f36a876b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc54b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f36a876b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc541820 0x5589dc548960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.415 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f36a876b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc54c0e0 0x5589dc5614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f36a876b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01ae5a0 con 0x5589dc540f00 2024-04-26T18:28:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f36a876b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01b00e0 con 0x5589dc54c0e0 2024-04-26T18:28:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f36a876b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01aec70 con 0x5589dc541820 2024-04-26T18:28:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f369effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc54b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f369effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc54b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45736/0 (socket says 172.21.15.84:45736) 2024-04-26T18:28:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f369effd640 1 -- 172.21.15.84:0/2208132252 learned_addr learned my addr 172.21.15.84:0/2208132252 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f369e7fc640 1 --2- 172.21.15.84:0/2208132252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc541820 0x5589dc548960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f369effd640 1 -- 172.21.15.84:0/2208132252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc541820 msgr2=0x5589dc548960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:13.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f369effd640 1 --2- 172.21.15.84:0/2208132252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc541820 0x5589dc548960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f369effd640 1 -- 172.21.15.84:0/2208132252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc54c0e0 msgr2=0x5589dc5614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f369effd640 1 --2- 172.21.15.84:0/2208132252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc54c0e0 0x5589dc5614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f369effd640 1 -- 172.21.15.84:0/2208132252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589dc541650 con 0x5589dc540f00 2024-04-26T18:28:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f369effd640 1 --2- 172.21.15.84:0/2208132252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc54b360 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f369009c590 tx=0x7f369000fe60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b6825a377663f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f36857fa640 1 -- 172.21.15.84:0/2208132252 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369009c1f0 con 0x5589dc540f00 2024-04-26T18:28:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f36857fa640 1 -- 172.21.15.84:0/2208132252 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3690087280 con 0x5589dc540f00 2024-04-26T18:28:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.411+0000 7f36857fa640 1 -- 172.21.15.84:0/2208132252 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36900a52d0 con 0x5589dc540f00 2024-04-26T18:28:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.412+0000 7f36a876b740 1 -- 172.21.15.84:0/2208132252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 msgr2=0x5589dc54b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.412+0000 7f36a876b740 1 --2- 172.21.15.84:0/2208132252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc54b360 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f369009c590 tx=0x7f369000fe60 comp rx=0 tx=0).stop 2024-04-26T18:28:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.412+0000 7f36a876b740 1 -- 172.21.15.84:0/2208132252 shutdown_connections 2024-04-26T18:28:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.412+0000 7f36a876b740 1 --2- 172.21.15.84:0/2208132252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc54c0e0 0x5589dc5614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.412+0000 7f36a876b740 1 --2- 172.21.15.84:0/2208132252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc541820 0x5589dc548960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.412+0000 7f36a876b740 1 --2- 172.21.15.84:0/2208132252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc54b360 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.412+0000 7f36a876b740 1 -- 172.21.15.84:0/2208132252 >> 172.21.15.84:0/2208132252 conn(0x5589dc53d410 msgr2=0x5589dc557970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.412+0000 7f36a876b740 1 -- 172.21.15.84:0/2208132252 shutdown_connections 2024-04-26T18:28:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.412+0000 7f36a876b740 1 -- 172.21.15.84:0/2208132252 wait complete. 2024-04-26T18:28:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.412+0000 7f36a876b740 1 -- start start 2024-04-26T18:28:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f36a876b740 1 Processor -- start 2024-04-26T18:28:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f36a876b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc561620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f36a876b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc541820 0x5589dc55eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f36a876b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc54c0e0 0x5589dc55efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f36a876b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01afa10 con 0x5589dc540f00 2024-04-26T18:28:13.418 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f36a876b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01af340 con 0x5589dc541820 2024-04-26T18:28:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f36a876b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36a01ac580 con 0x5589dc54c0e0 2024-04-26T18:28:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f36a876b740 10 client.0 _ll_register_callbacks cb 0x7f36980093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f369effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc561620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f369effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc561620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45742/0 (socket says 172.21.15.84:45742) 2024-04-26T18:28:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f369effd640 1 -- 172.21.15.84:0/1886161222 learned_addr learned my addr 172.21.15.84:0/1886161222 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f369effd640 1 -- 172.21.15.84:0/1886161222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc54c0e0 msgr2=0x5589dc55efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f369effd640 1 --2- 172.21.15.84:0/1886161222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc54c0e0 0x5589dc55efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f369effd640 1 -- 172.21.15.84:0/1886161222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc541820 msgr2=0x5589dc55eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f369effd640 1 --2- 172.21.15.84:0/1886161222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc541820 0x5589dc55eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f369effd640 1 -- 172.21.15.84:0/1886161222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589dc55f5a0 con 0x5589dc540f00 2024-04-26T18:28:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f369effd640 1 --2- 172.21.15.84:0/1886161222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc561620 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3690098ca0 tx=0x7f36900a0d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8b2b33ad5676f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f367effd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:13.419 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f367effd640 1 -- 172.21.15.84:0/1886161222 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369008f8e0 con 0x5589dc540f00 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f367effd640 1 -- 172.21.15.84:0/1886161222 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f36900a5070 con 0x5589dc540f00 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.413+0000 7f36a876b740 1 -- 172.21.15.84:0/1886161222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3690087280 con 0x5589dc540f00 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f367effd640 1 -- 172.21.15.84:0/1886161222 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f36900a5270 con 0x5589dc540f00 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f36a876b740 10 client.15390 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f36a876b740 1 -- 172.21.15.84:0/1886161222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589dc55f5a0 con 0x5589dc540f00 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f367effd640 1 -- 172.21.15.84:0/1886161222 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3690087280 con 0x5589dc540f00 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f367effd640 1 -- 172.21.15.84:0/1886161222 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f369009b750 con 0x5589dc540f00 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f36a876b740 10 client.15390 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f36a876b740 10 client.15390 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f36a876b740 20 client.15390 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f36a876b740 1 --2- 172.21.15.84:0/1886161222 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5589dc55c090 0x5589dc5c4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f36a876b740 4 client.15390 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f36a876b740 1 -- 172.21.15.84:0/1886161222 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589dc4aeee0 con 0x5589dc55c090 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f369e7fc640 1 --2- 172.21.15.84:0/1886161222 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5589dc55c090 0x5589dc5c4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:13.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f367effd640 10 client.15390 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:13.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.414+0000 7f369e7fc640 1 --2- 172.21.15.84:0/1886161222 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5589dc55c090 0x5589dc5c4300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e803ffa49cfcd32d server_cookie=ca3a26ca36abf450 in_seq=0 out_seq=0 2024-04-26T18:28:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f367effd640 1 -- 172.21.15.84:0/1886161222 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589dc4aeee0 con 0x5589dc55c090 2024-04-26T18:28:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f367effd640 10 client.15390 handle_command_reply: tid=0 2024-04-26T18:28:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f36a876b740 1 client.15390 shutdown 2024-04-26T18:28:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f36a876b740 1 -- 172.21.15.84:0/1886161222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 msgr2=0x5589dc561620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f36a876b740 1 --2- 172.21.15.84:0/1886161222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc561620 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3690098ca0 tx=0x7f36900a0d60 comp rx=0 tx=0).stop 2024-04-26T18:28:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f36a876b740 1 -- 172.21.15.84:0/1886161222 shutdown_connections 2024-04-26T18:28:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f36a876b740 1 --2- 172.21.15.84:0/1886161222 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5589dc55c090 0x5589dc5c4300 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f36a876b740 1 --2- 172.21.15.84:0/1886161222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5589dc54c0e0 0x5589dc55efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f36a876b740 1 --2- 172.21.15.84:0/1886161222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5589dc541820 0x5589dc55eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f36a876b740 1 --2- 172.21.15.84:0/1886161222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5589dc540f00 0x5589dc561620 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f367effd640 0 client.15390 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:13.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f36a876b740 1 -- 172.21.15.84:0/1886161222 >> 172.21.15.84:0/1886161222 conn(0x5589dc53d410 msgr2=0x5589dc5428b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f36a876b740 1 -- 172.21.15.84:0/1886161222 shutdown_connections 2024-04-26T18:28:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f36a876b740 1 -- 172.21.15.84:0/1886161222 wait complete. 2024-04-26T18:28:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.427+0000 7f36a876b740 20 client.15390 trim_cache size 0 max 16384 2024-04-26T18:28:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:28:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:28:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:28:13.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:28:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.428+0000 7f36a5978640 1 -- 172.21.15.84:0/807558584 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36880a3ff0 msgr2=0x7f36880a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.428+0000 7f36a5978640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36880a3ff0 0x7f36880a64b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3698005fd0 tx=0x7f3698009b70 comp rx=0 tx=0).stop 2024-04-26T18:28:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.428+0000 7f36a5978640 1 -- 172.21.15.84:0/807558584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a0102940 msgr2=0x7f36a0199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.428+0000 7f36a5978640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a0102940 0x7f36a0199020 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f369000c920 tx=0x7f369000cde0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.430 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.428+0000 7f36a5978640 1 -- 172.21.15.84:0/807558584 shutdown_connections 2024-04-26T18:28:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.428+0000 7f36a5978640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36880a3ff0 0x7f36880a64b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.428+0000 7f36a5978640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36a0108950 0x7f36a019d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.428+0000 7f36a5978640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36a0103300 0x7f36a0199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.428+0000 7f36a5978640 1 --2- 172.21.15.84:0/807558584 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36a0102940 0x7f36a0199020 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.428+0000 7f36a5978640 1 -- 172.21.15.84:0/807558584 >> 172.21.15.84:0/807558584 conn(0x7f36a0079dc0 msgr2=0x7f36a0100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.428+0000 7f36a5978640 1 -- 172.21.15.84:0/807558584 shutdown_connections 2024-04-26T18:28:13.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:13.428+0000 7f36a5978640 1 -- 172.21.15.84:0/807558584 wait complete. 2024-04-26T18:28:13.581 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:13 smithi084 ceph-mon[29141]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-04-26T18:28:13.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:13 smithi103 ceph-mon[32727]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-04-26T18:28:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:13 smithi174 ceph-mon[32705]: pgmap v10: 97 pgs: 97 active+clean; 1.3 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 99 KiB/s wr, 66 op/s 2024-04-26T18:28:14.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.093+0000 7f8595520700 1 -- 172.21.15.84:0/2470095254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f859006bbd0 msgr2=0x7f859006c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:14.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.093+0000 7f8595520700 1 --2- 172.21.15.84:0/2470095254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f859006bbd0 0x7f859006c050 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8584009990 tx=0x7f8584009ca0 comp rx=0 tx=0).stop 2024-04-26T18:28:14.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.093+0000 7f8595520700 1 -- 172.21.15.84:0/2470095254 shutdown_connections 2024-04-26T18:28:14.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.093+0000 7f8595520700 1 --2- 172.21.15.84:0/2470095254 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f859006c590 0x7f8590110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:14.093 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.093+0000 7f8595520700 1 --2- 172.21.15.84:0/2470095254 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85901094d0 0x7f85901098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:14.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.093+0000 7f8595520700 1 --2- 172.21.15.84:0/2470095254 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f859006bbd0 0x7f859006c050 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:14.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.093+0000 7f8595520700 1 -- 172.21.15.84:0/2470095254 >> 172.21.15.84:0/2470095254 conn(0x7f85900793c0 msgr2=0x7f85900797d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:14.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.093+0000 7f8595520700 1 -- 172.21.15.84:0/2470095254 shutdown_connections 2024-04-26T18:28:14.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.093+0000 7f8595520700 1 -- 172.21.15.84:0/2470095254 wait complete. 2024-04-26T18:28:14.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.093+0000 7f8595520700 1 Processor -- start 2024-04-26T18:28:14.094 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.093+0000 7f8595520700 1 -- start start 2024-04-26T18:28:14.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f8595520700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f859006bbd0 0x7f859019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:14.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f8595520700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f859006c590 0x7f859019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:14.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f8595520700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85901094d0 0x7f85901998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:14.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f8595520700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8590113660 con 0x7f85901094d0 2024-04-26T18:28:14.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f8595520700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f85901134e0 con 0x7f859006c590 2024-04-26T18:28:14.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f8595520700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f85901137e0 con 0x7f859006bbd0 2024-04-26T18:28:14.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f858f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85901094d0 0x7f85901998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:14.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f858f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85901094d0 0x7f85901998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45762/0 (socket says 172.21.15.84:45762) 2024-04-26T18:28:14.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f858f7fe700 1 -- 172.21.15.84:0/2190838607 learned_addr learned my addr 172.21.15.84:0/2190838607 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:14.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f858effd700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f859006bbd0 0x7f859019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:14.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f858e7fc700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f859006c590 0x7f859019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:14.096 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f858f7fe700 1 -- 172.21.15.84:0/2190838607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f859006bbd0 msgr2=0x7f859019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:14.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f858f7fe700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f859006bbd0 0x7f859019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:14.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f858f7fe700 1 -- 172.21.15.84:0/2190838607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f859006c590 msgr2=0x7f859019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:14.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f858f7fe700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f859006c590 0x7f859019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:14.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f858f7fe700 1 -- 172.21.15.84:0/2190838607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f859019a040 con 0x7f85901094d0 2024-04-26T18:28:14.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f858f7fe700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85901094d0 0x7f85901998e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f858800b970 tx=0x7f858800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:14.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f8577fff700 1 -- 172.21.15.84:0/2190838607 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588019070 con 0x7f85901094d0 2024-04-26T18:28:14.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f8595520700 1 -- 172.21.15.84:0/2190838607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f859019a330 con 0x7f85901094d0 2024-04-26T18:28:14.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f8577fff700 1 -- 172.21.15.84:0/2190838607 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f858800cde0 con 0x7f85901094d0 2024-04-26T18:28:14.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.094+0000 7f8577fff700 1 -- 172.21.15.84:0/2190838607 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8588007370 con 0x7f85901094d0 2024-04-26T18:28:14.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.095+0000 7f8595520700 1 -- 172.21.15.84:0/2190838607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85901a65f0 con 0x7f85901094d0 2024-04-26T18:28:14.097 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.096+0000 7f8595520700 1 -- 172.21.15.84:0/2190838607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f859006a110 con 0x7f85901094d0 2024-04-26T18:28:14.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.097+0000 7f8577fff700 1 -- 172.21.15.84:0/2190838607 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f858801c070 con 0x7f85901094d0 2024-04-26T18:28:14.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.097+0000 7f8577fff700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f85780a3f60 0x7f85780a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:14.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.097+0000 7f858effd700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f85780a3f60 0x7f85780a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:14.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.097+0000 7f8577fff700 1 -- 172.21.15.84:0/2190838607 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f85880cfeb0 con 0x7f85901094d0 2024-04-26T18:28:14.098 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.098+0000 7f858effd700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f85780a3f60 0x7f85780a6420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f859019aa20 tx=0x7f858000a380 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:14.101 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.102+0000 7f8577fff700 1 -- 172.21.15.84:0/2190838607 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f858809c410 con 0x7f85901094d0 2024-04-26T18:28:14.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.345+0000 7f8595520700 1 -- 172.21.15.84:0/2190838607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f859019ae10 con 0x7f85901094d0 2024-04-26T18:28:14.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.345+0000 7f8577fff700 1 -- 172.21.15.84:0/2190838607 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f858809bd10 con 0x7f85901094d0 2024-04-26T18:28:14.346 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:28:14.346 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:28:14.348 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.348+0000 7f8595520700 1 -- 172.21.15.84:0/2190838607 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f85780a3f60 msgr2=0x7f85780a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:14.349 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.348+0000 7f8595520700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f85780a3f60 0x7f85780a6420 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f859019aa20 tx=0x7f858000a380 comp rx=0 tx=0).stop 2024-04-26T18:28:14.349 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.348+0000 7f8595520700 1 -- 172.21.15.84:0/2190838607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85901094d0 msgr2=0x7f85901998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:14.349 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.348+0000 7f8595520700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85901094d0 0x7f85901998e0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f858800b970 tx=0x7f858800bc80 comp rx=0 tx=0).stop 2024-04-26T18:28:14.349 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.349+0000 7f8595520700 1 -- 172.21.15.84:0/2190838607 shutdown_connections 2024-04-26T18:28:14.349 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.349+0000 7f8595520700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f85780a3f60 0x7f85780a6420 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:14.349 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.349+0000 7f8595520700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85901094d0 0x7f85901998e0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:14.349 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.349+0000 7f8595520700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f859006bbd0 0x7f859019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:14.349 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.349+0000 7f8595520700 1 --2- 172.21.15.84:0/2190838607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f859006c590 0x7f859019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:14.349 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.349+0000 7f8595520700 1 -- 172.21.15.84:0/2190838607 >> 172.21.15.84:0/2190838607 conn(0x7f85900793c0 msgr2=0x7f85901002d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:14.349 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.349+0000 7f8595520700 1 -- 172.21.15.84:0/2190838607 shutdown_connections 2024-04-26T18:28:14.349 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:14.349+0000 7f8595520700 1 -- 172.21.15.84:0/2190838607 wait complete. 2024-04-26T18:28:14.350 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:28:15.081 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:28:15.323 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:15.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:15 smithi084 ceph-mon[29141]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 100 KiB/s wr, 68 op/s 2024-04-26T18:28:15.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:15 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2190838607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:15.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:15 smithi103 ceph-mon[32727]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 100 KiB/s wr, 68 op/s 2024-04-26T18:28:15.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:15 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2190838607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:15.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:15 smithi174 ceph-mon[32705]: pgmap v11: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 100 KiB/s wr, 68 op/s 2024-04-26T18:28:15.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:15 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2190838607' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:16.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.995+0000 7f85480c9700 1 -- 172.21.15.84:0/135909058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f854006bb90 msgr2=0x7f854006c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:16.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.995+0000 7f85480c9700 1 --2- 172.21.15.84:0/135909058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f854006bb90 0x7f854006c010 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8530009990 tx=0x7f8530009ca0 comp rx=0 tx=0).stop 2024-04-26T18:28:16.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.995+0000 7f85480c9700 1 -- 172.21.15.84:0/135909058 shutdown_connections 2024-04-26T18:28:16.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.995+0000 7f85480c9700 1 --2- 172.21.15.84:0/135909058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f854006c550 0x7f85401102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:16.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.995+0000 7f85480c9700 1 --2- 172.21.15.84:0/135909058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f85401094b0 0x7f8540109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:16.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.995+0000 7f85480c9700 1 --2- 172.21.15.84:0/135909058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f854006bb90 0x7f854006c010 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:16.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.995+0000 7f85480c9700 1 -- 172.21.15.84:0/135909058 >> 172.21.15.84:0/135909058 conn(0x7f8540078370 msgr2=0x7f8540078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:16.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.995+0000 7f85480c9700 1 -- 172.21.15.84:0/135909058 shutdown_connections 2024-04-26T18:28:16.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.995+0000 7f85480c9700 1 -- 172.21.15.84:0/135909058 wait complete. 2024-04-26T18:28:16.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.995+0000 7f85480c9700 1 Processor -- start 2024-04-26T18:28:16.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.995+0000 7f85480c9700 1 -- start start 2024-04-26T18:28:16.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f85480c9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854006c550 0x7f8540199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:16.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f85480c9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85401094b0 0x7f854019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:16.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f85480c9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f854019e770 0x7f854019ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:16.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f85480c9700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f85401136c0 con 0x7f85401094b0 2024-04-26T18:28:16.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f85480c9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8540113540 con 0x7f854006c550 2024-04-26T18:28:16.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f85480c9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8540113840 con 0x7f854019e770 2024-04-26T18:28:16.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8545664700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85401094b0 0x7f854019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:16.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8545664700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85401094b0 0x7f854019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44290/0 (socket says 172.21.15.84:44290) 2024-04-26T18:28:16.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8545664700 1 -- 172.21.15.84:0/1581502032 learned_addr learned my addr 172.21.15.84:0/1581502032 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:16.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8545e65700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854006c550 0x7f8540199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:16.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8546666700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f854019e770 0x7f854019ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:16.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8545664700 1 -- 172.21.15.84:0/1581502032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f854019e770 msgr2=0x7f854019ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:16.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8545664700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f854019e770 0x7f854019ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:16.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8545664700 1 -- 172.21.15.84:0/1581502032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854006c550 msgr2=0x7f8540199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:16.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8545664700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854006c550 0x7f8540199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:16.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8545664700 1 -- 172.21.15.84:0/1581502032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f854019f290 con 0x7f85401094b0 2024-04-26T18:28:17.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8545e65700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854006c550 0x7f8540199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:17.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8545664700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85401094b0 0x7f854019a430 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8530006010 tx=0x7f8530003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:17.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8536ffd700 1 -- 172.21.15.84:0/1581502032 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8530026070 con 0x7f85401094b0 2024-04-26T18:28:17.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f8536ffd700 1 -- 172.21.15.84:0/1581502032 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8530004110 con 0x7f85401094b0 2024-04-26T18:28:17.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.996+0000 7f85480c9700 1 -- 172.21.15.84:0/1581502032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f854019f520 con 0x7f85401094b0 2024-04-26T18:28:17.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.997+0000 7f8536ffd700 1 -- 172.21.15.84:0/1581502032 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853000fd90 con 0x7f85401094b0 2024-04-26T18:28:17.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.997+0000 7f85480c9700 1 -- 172.21.15.84:0/1581502032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85401a6ca0 con 0x7f85401094b0 2024-04-26T18:28:17.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.997+0000 7f85480c9700 1 -- 172.21.15.84:0/1581502032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f854006a110 con 0x7f85401094b0 2024-04-26T18:28:17.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.999+0000 7f8536ffd700 1 -- 172.21.15.84:0/1581502032 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f8530029070 con 0x7f85401094b0 2024-04-26T18:28:17.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.999+0000 7f8536ffd700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f852c0a3fe0 0x7f852c0a64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:17.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.999+0000 7f8545e65700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f852c0a3fe0 0x7f852c0a64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:17.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:16.999+0000 7f8536ffd700 1 -- 172.21.15.84:0/1581502032 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f85300d1960 con 0x7f85401094b0 2024-04-26T18:28:17.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.000+0000 7f8545e65700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f852c0a3fe0 0x7f852c0a64a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f853c0099d0 tx=0x7f853c008040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:17.003 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.003+0000 7f8536ffd700 1 -- 172.21.15.84:0/1581502032 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8530030020 con 0x7f85401094b0 2024-04-26T18:28:17.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.209+0000 7f85480c9700 1 -- 172.21.15.84:0/1581502032 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f854002e650 con 0x7f85401094b0 2024-04-26T18:28:17.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.211+0000 7f8536ffd700 1 -- 172.21.15.84:0/1581502032 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+23103 (secure 0 0 0) 0x7f8530013070 con 0x7f85401094b0 2024-04-26T18:28:17.211 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:28:17.211 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":94,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:16:06.267694+0000","modified":"2024-04-26T18:27:57.444649+0000","last_up_change":"2024-04-26T18:23:19.906511+0000","last_in_change":"2024-04-26T18:22:59.541869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:19:31.174024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:24:16.533128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:24:19.742656+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1076912-986e-4825-b1d1-5fa9d69bc07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6803","nonce":140050011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6805","nonce":140050011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6809","nonce":140050011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6807","nonce":140050011}]},"public_addr":"172.21.15.84:6803/140050011","cluster_addr":"172.21.15.84:6805/140050011","heartbeat_back_addr":"172.21.15.84:6809/140050011","heartbeat_front_addr":"172.21.15.84:6807/140050011","state":["exists","up"]},{"osd":1,"uuid":"fa93293d-9058-4804-ba4b-7e1e674b0fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6801","nonce":2512857657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6803","nonce":2512857657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6807","nonce":2512857657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6805","nonce":2512857657}]},"public_addr":"172.21.15.103:6801/2512857657","cluster_addr":"172.21.15.103:6803/2512857657","heartbeat_back_addr":"172.21.15.103:6807/2512857657","heartbeat_front_addr":"172.21.15.103:6805/2512857657","state":["exists","up"]},{"osd":2,"uuid":"bf78a44f-80a2-4cd7-974f-5abdf771e650","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6801","nonce":76340924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6803","nonce":76340924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6807","nonce":76340924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6805","nonce":76340924}]},"public_addr":"172.21.15.174:6801/76340924","cluster_addr":"172.21.15.174:6803/76340924","heartbeat_back_addr":"172.21.15.174:6807/76340924","heartbeat_front_addr":"172.21.15.174:6805/76340924","state":["exists","up"]},{"osd":3,"uuid":"55fd0035-bc76-4235-b8b5-5cf5ef770139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6811","nonce":2352789256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6813","nonce":2352789256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6817","nonce":2352789256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6815","nonce":2352789256}]},"public_addr":"172.21.15.84:6811/2352789256","cluster_addr":"172.21.15.84:6813/2352789256","heartbeat_back_addr":"172.21.15.84:6817/2352789256","heartbeat_front_addr":"172.21.15.84:6815/2352789256","state":["exists","up"]},{"osd":4,"uuid":"c278a0f3-e436-4d1c-a009-f9165ebf3c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6809","nonce":1619258252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6811","nonce":1619258252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6815","nonce":1619258252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6813","nonce":1619258252}]},"public_addr":"172.21.15.103:6809/1619258252","cluster_addr":"172.21.15.103:6811/1619258252","heartbeat_back_addr":"172.21.15.103:6815/1619258252","heartbeat_front_addr":"172.21.15.103:6813/1619258252","state":["exists","up"]},{"osd":5,"uuid":"6505be9b-c1b0-4d8b-944e-9920f90cb88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6809","nonce":3168291768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6811","nonce":3168291768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6815","nonce":3168291768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6813","nonce":3168291768}]},"public_addr":"172.21.15.174:6809/3168291768","cluster_addr":"172.21.15.174:6811/3168291768","heartbeat_back_addr":"172.21.15.174:6815/3168291768","heartbeat_front_addr":"172.21.15.174:6813/3168291768","state":["exists","up"]},{"osd":6,"uuid":"a037b66a-7488-469d-b73a-14cc0d35cb5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6819","nonce":886635297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6821","nonce":886635297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6825","nonce":886635297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6823","nonce":886635297}]},"public_addr":"172.21.15.84:6819/886635297","cluster_addr":"172.21.15.84:6821/886635297","heartbeat_back_addr":"172.21.15.84:6825/886635297","heartbeat_front_addr":"172.21.15.84:6823/886635297","state":["exists","up"]},{"osd":7,"uuid":"c425a13f-8019-4484-86a0-a34fa4d8de81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6817","nonce":1738278079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6819","nonce":1738278079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6823","nonce":1738278079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6821","nonce":1738278079}]},"public_addr":"172.21.15.103:6817/1738278079","cluster_addr":"172.21.15.103:6819/1738278079","heartbeat_back_addr":"172.21.15.103:6823/1738278079","heartbeat_front_addr":"172.21.15.103:6821/1738278079","state":["exists","up"]},{"osd":8,"uuid":"687e6c08-bfa4-440c-b3bc-90a5748989d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6817","nonce":38410548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6819","nonce":38410548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6823","nonce":38410548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6821","nonce":38410548}]},"public_addr":"172.21.15.174:6817/38410548","cluster_addr":"172.21.15.174:6819/38410548","heartbeat_back_addr":"172.21.15.174:6823/38410548","heartbeat_front_addr":"172.21.15.174:6821/38410548","state":["exists","up"]},{"osd":9,"uuid":"059eb22d-6dfb-4dd0-b192-84ed42ee1fbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6827","nonce":2926393927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6829","nonce":2926393927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6833","nonce":2926393927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6831","nonce":2926393927}]},"public_addr":"172.21.15.84:6827/2926393927","cluster_addr":"172.21.15.84:6829/2926393927","heartbeat_back_addr":"172.21.15.84:6833/2926393927","heartbeat_front_addr":"172.21.15.84:6831/2926393927","state":["exists","up"]},{"osd":10,"uuid":"da31e305-5220-4a87-a131-005b7c7f76b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6825","nonce":2705726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6827","nonce":2705726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6831","nonce":2705726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6829","nonce":2705726915}]},"public_addr":"172.21.15.103:6825/2705726915","cluster_addr":"172.21.15.103:6827/2705726915","heartbeat_back_addr":"172.21.15.103:6831/2705726915","heartbeat_front_addr":"172.21.15.103:6829/2705726915","state":["exists","up"]},{"osd":11,"uuid":"3fde6c8f-ee02-41ee-a34f-73072b4761e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6825","nonce":1668672128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6827","nonce":1668672128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6831","nonce":1668672128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6829","nonce":1668672128}]},"public_addr":"172.21.15.174:6825/1668672128","cluster_addr":"172.21.15.174:6827/1668672128","heartbeat_back_addr":"172.21.15.174:6831/1668672128","heartbeat_front_addr":"172.21.15.174:6829/1668672128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:39.099188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:03.327608+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:27.714108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:52.506015+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:17.631670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:42.702489+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:07.606697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:34.123214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:59.534658+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:25.357720+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:51.543106+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:23:18.208230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/277477867":"2024-04-27T18:27:57.444610+0000","172.21.15.84:0/559313476":"2024-04-27T18:27:57.444610+0000","172.21.15.84:6800/2059208901":"2024-04-27T18:27:57.444610+0000","172.21.15.84:0/3834640430":"2024-04-27T18:27:57.444610+0000","172.21.15.84:0/2142936585":"2024-04-27T18:27:57.444610+0000","172.21.15.84:0/1604680842":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6800/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/2059208901":"2024-04-27T18:27:57.444610+0000","172.21.15.84:6801/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/538363975":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/952052990":"2024-04-27T18:17:05.083192+0000","172.21.15.84:0/1266942705":"2024-04-27T18:16:37.859669+0000","172.21.15.84:6800/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3970542149":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3607125049":"2024-04-27T18:17:05.083192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:28:17.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.213+0000 7f85480c9700 1 -- 172.21.15.84:0/1581502032 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f852c0a3fe0 msgr2=0x7f852c0a64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:17.213 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.213+0000 7f85480c9700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f852c0a3fe0 0x7f852c0a64a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f853c0099d0 tx=0x7f853c008040 comp rx=0 tx=0).stop 2024-04-26T18:28:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.213+0000 7f85480c9700 1 -- 172.21.15.84:0/1581502032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85401094b0 msgr2=0x7f854019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.213+0000 7f85480c9700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85401094b0 0x7f854019a430 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8530006010 tx=0x7f8530003730 comp rx=0 tx=0).stop 2024-04-26T18:28:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.213+0000 7f85480c9700 1 -- 172.21.15.84:0/1581502032 shutdown_connections 2024-04-26T18:28:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.213+0000 7f85480c9700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f852c0a3fe0 0x7f852c0a64a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.213+0000 7f85480c9700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f854019e770 0x7f854019ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.213+0000 7f85480c9700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f854006c550 0x7f8540199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.213+0000 7f85480c9700 1 --2- 172.21.15.84:0/1581502032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f85401094b0 0x7f854019a430 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.213+0000 7f85480c9700 1 -- 172.21.15.84:0/1581502032 >> 172.21.15.84:0/1581502032 conn(0x7f8540078370 msgr2=0x7f8540100600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.213+0000 7f85480c9700 1 -- 172.21.15.84:0/1581502032 shutdown_connections 2024-04-26T18:28:17.214 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:17.213+0000 7f85480c9700 1 -- 172.21.15.84:0/1581502032 wait complete. 2024-04-26T18:28:17.532 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:17 smithi084 ceph-mon[29141]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 68 op/s 2024-04-26T18:28:17.533 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:17 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1581502032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:28:17.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:17 smithi103 ceph-mon[32727]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 68 op/s 2024-04-26T18:28:17.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:17 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1581502032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:28:17.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:17 smithi174 ceph-mon[32705]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 68 op/s 2024-04-26T18:28:17.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:17 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1581502032' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:28:17.897 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-04-26T18:28:18.135 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:19.566 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:19 smithi084 ceph-mon[29141]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-26T18:28:19.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:19 smithi103 ceph-mon[32727]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-26T18:28:19.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:19 smithi174 ceph-mon[32705]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.0 KiB/s wr, 12 op/s 2024-04-26T18:28:20.777 INFO:teuthology.orchestra.run.smithi084.stdout:{"run_dir":"/var/run/ceph"} 2024-04-26T18:28:20.777 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-04-26T18:28:20.778 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 9496 2024-04-26T18:28:20.778 DEBUG:tasks.quiescer:Yielding 2024-04-26T18:28:20.778 INFO:teuthology.run_tasks:Running task workunit... 2024-04-26T18:28:20.793 INFO:tasks.workunit:Pulling workunits from ref bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:28:20.794 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T18:28:20.794 INFO:tasks.workunit:timeout=3h 2024-04-26T18:28:20.794 INFO:tasks.workunit:cleanup=True 2024-04-26T18:28:20.794 DEBUG:teuthology.orchestra.run.smithi084:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T18:28:20.795 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:28:20.811 INFO:teuthology.orchestra.run.smithi084.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T18:28:20.811 INFO:teuthology.orchestra.run.smithi084.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T18:28:20.811 INFO:teuthology.orchestra.run.smithi084.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-26T18:28:20.812 INFO:teuthology.orchestra.run.smithi084.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T18:28:20.812 INFO:teuthology.orchestra.run.smithi084.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T18:28:20.812 INFO:teuthology.orchestra.run.smithi084.stdout:Access: 2024-04-26 18:24:54.614785947 +0000 2024-04-26T18:28:20.812 INFO:teuthology.orchestra.run.smithi084.stdout:Modify: 2024-04-26 18:24:54.614785947 +0000 2024-04-26T18:28:20.812 INFO:teuthology.orchestra.run.smithi084.stdout:Change: 2024-04-26 18:27:55.422214672 +0000 2024-04-26T18:28:20.812 INFO:teuthology.orchestra.run.smithi084.stdout: Birth: 2024-04-26 18:24:54.614785947 +0000 2024-04-26T18:28:20.812 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T18:28:20.812 DEBUG:teuthology.orchestra.run.smithi084:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T18:28:20.884 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 2024-04-26T18:28:20.939 INFO:tasks.workunit.client.0.smithi084.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T18:28:21.045 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:21.450 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:21 smithi084 ceph-mon[29141]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 17 op/s 2024-04-26T18:28:21.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:21 smithi103 ceph-mon[32727]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 17 op/s 2024-04-26T18:28:21.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:21 smithi174 ceph-mon[32705]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.0 KiB/s wr, 17 op/s 2024-04-26T18:28:22.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.859+0000 7f41ca648700 1 -- 172.21.15.84:0/1312502266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41c4108390 msgr2=0x7f41c4108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:22.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.859+0000 7f41ca648700 1 --2- 172.21.15.84:0/1312502266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41c4108390 0x7f41c4108770 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f41ac009990 tx=0x7f41ac009ca0 comp rx=0 tx=0).stop 2024-04-26T18:28:22.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.859+0000 7f41ca648700 1 -- 172.21.15.84:0/1312502266 shutdown_connections 2024-04-26T18:28:22.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.859+0000 7f41ca648700 1 --2- 172.21.15.84:0/1312502266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41c4108390 0x7f41c4108770 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:22.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.859+0000 7f41ca648700 1 --2- 172.21.15.84:0/1312502266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c4102ce0 0x7f41c410f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:22.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.859+0000 7f41ca648700 1 --2- 172.21.15.84:0/1312502266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41c4102320 0x7f41c41027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:22.862 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.859+0000 7f41ca648700 1 -- 172.21.15.84:0/1312502266 >> 172.21.15.84:0/1312502266 conn(0x7f41c40792d0 msgr2=0x7f41c40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:22.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.859+0000 7f41ca648700 1 -- 172.21.15.84:0/1312502266 shutdown_connections 2024-04-26T18:28:22.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.859+0000 7f41ca648700 1 -- 172.21.15.84:0/1312502266 wait complete. 2024-04-26T18:28:22.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.860+0000 7f41ca648700 1 Processor -- start 2024-04-26T18:28:22.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.860+0000 7f41ca648700 1 -- start start 2024-04-26T18:28:22.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41ca648700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41c4102320 0x7f41c419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:22.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41ca648700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c4102ce0 0x7f41c419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:22.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41ca648700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41c4108390 0x7f41c4199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:22.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41ca648700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f41c41124e0 con 0x7f41c4102320 2024-04-26T18:28:22.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41ca648700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f41c4112360 con 0x7f41c4102ce0 2024-04-26T18:28:22.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41ca648700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f41c4112660 con 0x7f41c4108390 2024-04-26T18:28:22.863 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41c3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41c4102320 0x7f41c419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:22.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41c3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41c4102320 0x7f41c419f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44314/0 (socket says 172.21.15.84:44314) 2024-04-26T18:28:22.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41c3fff700 1 -- 172.21.15.84:0/3923529692 learned_addr learned my addr 172.21.15.84:0/3923529692 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:22.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41c8be5700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41c4108390 0x7f41c4199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:22.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41c37fe700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c4102ce0 0x7f41c419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:22.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41c3fff700 1 -- 172.21.15.84:0/3923529692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41c4108390 msgr2=0x7f41c4199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:22.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41c3fff700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41c4108390 0x7f41c4199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:22.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41c3fff700 1 -- 172.21.15.84:0/3923529692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c4102ce0 msgr2=0x7f41c419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:22.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41c3fff700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c4102ce0 0x7f41c419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:22.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41c3fff700 1 -- 172.21.15.84:0/3923529692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c419a0b0 con 0x7f41c4102320 2024-04-26T18:28:22.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41c8be5700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41c4108390 0x7f41c4199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:28:22.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.861+0000 7f41c3fff700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41c4102320 0x7f41c419f790 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f41ac000c00 tx=0x7f41ac0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:22.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.862+0000 7f41c17fa700 1 -- 172.21.15.84:0/3923529692 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac027070 con 0x7f41c4102320 2024-04-26T18:28:22.864 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.862+0000 7f41ca648700 1 -- 172.21.15.84:0/3923529692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c419a340 con 0x7f41c4102320 2024-04-26T18:28:22.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.862+0000 7f41c17fa700 1 -- 172.21.15.84:0/3923529692 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41ac003f20 con 0x7f41c4102320 2024-04-26T18:28:22.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.862+0000 7f41c17fa700 1 -- 172.21.15.84:0/3923529692 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41ac022ca0 con 0x7f41c4102320 2024-04-26T18:28:22.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.862+0000 7f41ca648700 1 -- 172.21.15.84:0/3923529692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c41a6660 con 0x7f41c4102320 2024-04-26T18:28:22.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.863+0000 7f41ca648700 1 -- 172.21.15.84:0/3923529692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c406a110 con 0x7f41c4102320 2024-04-26T18:28:22.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.864+0000 7f41c17fa700 1 -- 172.21.15.84:0/3923529692 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f41ac02a070 con 0x7f41c4102320 2024-04-26T18:28:22.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.865+0000 7f41c17fa700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f41b00a3f60 0x7f41b00a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:22.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.865+0000 7f41c37fe700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f41b00a3f60 0x7f41b00a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:22.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.865+0000 7f41c17fa700 1 -- 172.21.15.84:0/3923529692 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f41ac0d2020 con 0x7f41c4102320 2024-04-26T18:28:22.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.865+0000 7f41c37fe700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f41b00a3f60 0x7f41b00a6420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f41c40798d0 tx=0x7f41b400b3c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:22.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:22.868+0000 7f41c17fa700 1 -- 172.21.15.84:0/3923529692 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41ac031070 con 0x7f41c4102320 2024-04-26T18:28:23.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.116+0000 7f41ca648700 1 -- 172.21.15.84:0/3923529692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f41c41a6830 con 0x7f41c4102320 2024-04-26T18:28:23.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.117+0000 7f41c17fa700 1 -- 172.21.15.84:0/3923529692 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f41ac013070 con 0x7f41c4102320 2024-04-26T18:28:23.117 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:28:23.117 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:28:23.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.119+0000 7f41ca648700 1 -- 172.21.15.84:0/3923529692 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f41b00a3f60 msgr2=0x7f41b00a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:23.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.119+0000 7f41ca648700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f41b00a3f60 0x7f41b00a6420 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f41c40798d0 tx=0x7f41b400b3c0 comp rx=0 tx=0).stop 2024-04-26T18:28:23.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.119+0000 7f41ca648700 1 -- 172.21.15.84:0/3923529692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41c4102320 msgr2=0x7f41c419f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:23.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.119+0000 7f41ca648700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41c4102320 0x7f41c419f790 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f41ac000c00 tx=0x7f41ac0047b0 comp rx=0 tx=0).stop 2024-04-26T18:28:23.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.119+0000 7f41ca648700 1 -- 172.21.15.84:0/3923529692 shutdown_connections 2024-04-26T18:28:23.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.119+0000 7f41ca648700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f41b00a3f60 0x7f41b00a6420 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:23.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.119+0000 7f41ca648700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f41c4102320 0x7f41c419f790 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:23.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.119+0000 7f41ca648700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f41c4108390 0x7f41c4199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:23.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.119+0000 7f41ca648700 1 --2- 172.21.15.84:0/3923529692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f41c4102ce0 0x7f41c419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:23.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.119+0000 7f41ca648700 1 -- 172.21.15.84:0/3923529692 >> 172.21.15.84:0/3923529692 conn(0x7f41c40792d0 msgr2=0x7f41c4101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:23.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.119+0000 7f41ca648700 1 -- 172.21.15.84:0/3923529692 shutdown_connections 2024-04-26T18:28:23.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:23.119+0000 7f41ca648700 1 -- 172.21.15.84:0/3923529692 wait complete. 2024-04-26T18:28:23.120 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:28:23.501 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:23 smithi084 ceph-mon[29141]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-26T18:28:23.501 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:23 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3923529692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:23.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:23 smithi103 ceph-mon[32727]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-26T18:28:23.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:23 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3923529692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:23.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:23 smithi174 ceph-mon[32705]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1.9 KiB/s wr, 12 op/s 2024-04-26T18:28:23.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:23 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3923529692' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:23.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-26T18:24:22.810992+0000', 'modified': '2024-04-26T18:27:56.147537+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24434, 'mds_1': 24430, 'mds_2': 24422, 'mds_3': 24418, 'mds_4': 24407}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14598': {'gid': 14598, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6839/96876655', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6838', 'nonce': 96876655}, {'type': 'v1', 'addr': '172.21.15.84:6839', 'nonce': 96876655}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14610': {'gid': 14610, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.84:6841/2504248516', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.84:6840', 'nonce': 2504248516}, {'type': 'v1', 'addr': '172.21.15.84:6841', 'nonce': 2504248516}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24385': {'gid': 24385, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6833/280377862', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6832', 'nonce': 280377862}, {'type': 'v1', 'addr': '172.21.15.174:6833', 'nonce': 280377862}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24389': {'gid': 24389, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.103:6833/3711970972', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6832', 'nonce': 3711970972}, {'type': 'v1', 'addr': '172.21.15.103:6833', 'nonce': 3711970972}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24406': {'gid': 24406, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.174:6835/1310802824', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6834', 'nonce': 1310802824}, {'type': 'v1', 'addr': '172.21.15.174:6835', 'nonce': 1310802824}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24407': {'gid': 24407, 'name': 'e', 'rank': 4, 'incarnation': 25, 'state': 'up:active', 'state_seq': 14, 'addr': '172.21.15.103:6835/2529642104', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6834', 'nonce': 2529642104}, {'type': 'v1', 'addr': '172.21.15.103:6835', 'nonce': 2529642104}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24418': {'gid': 24418, 'name': 'c', 'rank': 3, 'incarnation': 23, 'state': 'up:active', 'state_seq': 13, 'addr': '172.21.15.174:6837/1532309720', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6836', 'nonce': 1532309720}, {'type': 'v1', 'addr': '172.21.15.174:6837', 'nonce': 1532309720}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24422': {'gid': 24422, 'name': 'h', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.103:6837/2271231481', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6836', 'nonce': 2271231481}, {'type': 'v1', 'addr': '172.21.15.103:6837', 'nonce': 2271231481}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24430': {'gid': 24430, 'name': 'i', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.174:6839/518809558', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.174:6838', 'nonce': 518809558}, {'type': 'v1', 'addr': '172.21.15.174:6839', 'nonce': 518809558}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24434': {'gid': 24434, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.103:6839/2465442393', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.103:6838', 'nonce': 2465442393}, {'type': 'v1', 'addr': '172.21.15.103:6839', 'nonce': 2465442393}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24434, 'qdb_cluster': [24434, 24430, 24422, 24407, 24418]} 2024-04-26T18:28:23.938 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-26T18:28:23.938 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.e status 2024-04-26T18:28:24.221 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:25.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:25 smithi084 ceph-mon[29141]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-04-26T18:28:25.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:25 smithi103 ceph-mon[32727]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-04-26T18:28:25.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:25 smithi174 ceph-mon[32705]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-04-26T18:28:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.038+0000 7fc0d935e700 1 -- 172.21.15.84:0/1391178307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0d406c510 msgr2=0x7fc0d41102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.038+0000 7fc0d935e700 1 --2- 172.21.15.84:0/1391178307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0d406c510 0x7fc0d41102e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc00b0a0 tx=0x7fc0cc00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.038+0000 7fc0d935e700 1 -- 172.21.15.84:0/1391178307 shutdown_connections 2024-04-26T18:28:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.038+0000 7fc0d935e700 1 --2- 172.21.15.84:0/1391178307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0d406c510 0x7fc0d41102e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.038+0000 7fc0d935e700 1 --2- 172.21.15.84:0/1391178307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d41094a0 0x7fc0d4109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.038+0000 7fc0d935e700 1 --2- 172.21.15.84:0/1391178307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d406bb70 0x7fc0d406bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.038+0000 7fc0d935e700 1 -- 172.21.15.84:0/1391178307 >> 172.21.15.84:0/1391178307 conn(0x7fc0d40792f0 msgr2=0x7fc0d4079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.038+0000 7fc0d935e700 1 -- 172.21.15.84:0/1391178307 shutdown_connections 2024-04-26T18:28:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.038+0000 7fc0d935e700 1 -- 172.21.15.84:0/1391178307 wait complete. 2024-04-26T18:28:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.038+0000 7fc0d935e700 1 Processor -- start 2024-04-26T18:28:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.038+0000 7fc0d935e700 1 -- start start 2024-04-26T18:28:26.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d935e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d406bb70 0x7fc0d419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d935e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0d406c510 0x7fc0d419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d935e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d41094a0 0x7fc0d41999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d935e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d4113620 con 0x7fc0d406c510 2024-04-26T18:28:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d935e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41134a0 con 0x7fc0d41094a0 2024-04-26T18:28:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d935e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc0d41137a0 con 0x7fc0d406bb70 2024-04-26T18:28:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d27fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0d406c510 0x7fc0d419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d27fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0d406c510 0x7fc0d419fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44332/0 (socket says 172.21.15.84:44332) 2024-04-26T18:28:26.040 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d27fc700 1 -- 172.21.15.84:0/2867798260 learned_addr learned my addr 172.21.15.84:0/2867798260 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:26.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d37fe700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d41094a0 0x7fc0d41999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d406bb70 0x7fc0d419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d27fc700 1 -- 172.21.15.84:0/2867798260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d406bb70 msgr2=0x7fc0d419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d406bb70 0x7fc0d419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d27fc700 1 -- 172.21.15.84:0/2867798260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d41094a0 msgr2=0x7fc0d41999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d41094a0 0x7fc0d41999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d27fc700 1 -- 172.21.15.84:0/2867798260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0d419a100 con 0x7fc0d406c510 2024-04-26T18:28:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d37fe700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d41094a0 0x7fc0d41999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0d406c510 0x7fc0d419fcd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400d950 tx=0x7fc0c400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0bbfff700 1 -- 172.21.15.84:0/2867798260 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4016070 con 0x7fc0d406c510 2024-04-26T18:28:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.039+0000 7fc0d935e700 1 -- 172.21.15.84:0/2867798260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0d419a3f0 con 0x7fc0d406c510 2024-04-26T18:28:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.040+0000 7fc0d935e700 1 -- 172.21.15.84:0/2867798260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0d41a6660 con 0x7fc0d406c510 2024-04-26T18:28:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.040+0000 7fc0bbfff700 1 -- 172.21.15.84:0/2867798260 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c4010460 con 0x7fc0d406c510 2024-04-26T18:28:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.040+0000 7fc0bbfff700 1 -- 172.21.15.84:0/2867798260 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c4020060 con 0x7fc0d406c510 2024-04-26T18:28:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.042+0000 7fc0bbfff700 1 -- 172.21.15.84:0/2867798260 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7fc0c4019070 con 0x7fc0d406c510 2024-04-26T18:28:26.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.042+0000 7fc0bbfff700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc0bc0a3f60 0x7fc0bc0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.042+0000 7fc0bbfff700 1 -- 172.21.15.84:0/2867798260 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fc0c40d0620 con 0x7fc0d406c510 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.042+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc0bc0a3f60 0x7fc0bc0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc0bc0a3f60 0x7fc0bc0a6420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8006fd0 tx=0x7fc0c8008040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0e3bf0b80 1 Processor -- start 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0e3bf0b80 1 -- start start 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0e3bf0b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 0x5556598bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0e3bf0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598be3f0 0x5556598b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0e3bf0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598b82d0 0x5556598b8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0e3bf0b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55565980a6f0 con 0x5556598bdad0 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0e3bf0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5556597a9590 con 0x5556598b82d0 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0e3bf0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5556598066b0 con 0x5556598be3f0 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0d2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 0x5556598bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0d2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 0x5556598bdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44336/0 (socket says 172.21.15.84:44336) 2024-04-26T18:28:26.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0d2ffd700 1 -- 172.21.15.84:0/210412349 learned_addr learned my addr 172.21.15.84:0/210412349 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/210412349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598be3f0 0x5556598b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0d37fe700 1 --2- 172.21.15.84:0/210412349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598b82d0 0x5556598b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0d2ffd700 1 -- 172.21.15.84:0/210412349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598be3f0 msgr2=0x5556598b7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.043+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/210412349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598be3f0 0x5556598b7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.044+0000 7fc0d2ffd700 1 -- 172.21.15.84:0/210412349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598b82d0 msgr2=0x5556598b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.044+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/210412349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598b82d0 0x5556598b8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.044+0000 7fc0d2ffd700 1 -- 172.21.15.84:0/210412349 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555659757880 con 0x5556598bdad0 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.044+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/210412349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598be3f0 0x5556598b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.044+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/210412349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 0x5556598bdeb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800d720 tx=0x7fc0c8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b76dba0e2e9df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.044+0000 7fc0b97fa700 1 -- 172.21.15.84:0/210412349 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8024070 con 0x5556598bdad0 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.044+0000 7fc0b97fa700 1 -- 172.21.15.84:0/210412349 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c8014af0 con 0x5556598bdad0 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.044+0000 7fc0b97fa700 1 -- 172.21.15.84:0/210412349 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c801fab0 con 0x5556598bdad0 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/210412349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 msgr2=0x5556598bdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/210412349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 0x5556598bdeb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800d720 tx=0x7fc0c8010000 comp rx=0 tx=0).stop 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/210412349 shutdown_connections 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/210412349 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 0x5556598bdeb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/210412349 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598b82d0 0x5556598b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/210412349 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598be3f0 0x5556598b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/210412349 >> 172.21.15.84:0/210412349 conn(0x5556597e4cf0 msgr2=0x5556598b31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/210412349 shutdown_connections 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/210412349 wait complete. 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 -- start start 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 Processor -- start 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598b82d0 0x5556598c4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 0x5556598c1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598be3f0 0x5556598c19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5556598c4f10 con 0x5556598bdad0 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555659770900 con 0x5556598be3f0 2024-04-26T18:28:26.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5556598c5090 con 0x5556598b82d0 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.045+0000 7fc0e3bf0b80 10 client.0 _ll_register_callbacks cb 0x55565979e530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0d27fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 0x5556598c1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0d27fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 0x5556598c1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44348/0 (socket says 172.21.15.84:44348) 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0d27fc700 1 -- 172.21.15.84:0/389734501 learned_addr learned my addr 172.21.15.84:0/389734501 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0d37fe700 1 --2- 172.21.15.84:0/389734501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598be3f0 0x5556598c19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/389734501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598b82d0 0x5556598c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0d27fc700 1 -- 172.21.15.84:0/389734501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598b82d0 msgr2=0x5556598c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/389734501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598b82d0 0x5556598c4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0d27fc700 1 -- 172.21.15.84:0/389734501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598be3f0 msgr2=0x5556598c19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/389734501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598be3f0 0x5556598c19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0d27fc700 1 -- 172.21.15.84:0/389734501 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556598c2010 con 0x5556598bdad0 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/389734501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 0x5556598c1460 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4005cd0 tx=0x7fc0c409caf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806a6554109d958c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:26.053 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0b6ff5700 1 -- 172.21.15.84:0/389734501 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40b5060 con 0x5556598bdad0 2024-04-26T18:28:26.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0b6ff5700 1 -- 172.21.15.84:0/389734501 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fc0c4009b40 con 0x5556598bdad0 2024-04-26T18:28:26.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/389734501 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0c8014af0 con 0x5556598bdad0 2024-04-26T18:28:26.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0b6ff5700 1 -- 172.21.15.84:0/389734501 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc0c40a0720 con 0x5556598bdad0 2024-04-26T18:28:26.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0e3bf0b80 10 client.15420 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:26.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/389734501 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5556598cf9c0 con 0x5556598bdad0 2024-04-26T18:28:26.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.046+0000 7fc0b6ff5700 1 -- 172.21.15.84:0/389734501 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c8014af0 con 0x5556598bdad0 2024-04-26T18:28:26.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.047+0000 7fc0b6ff5700 1 -- 172.21.15.84:0/389734501 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc0c4062080 con 0x5556598bdad0 2024-04-26T18:28:26.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.047+0000 7fc0e3bf0b80 10 client.15420 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:26.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.047+0000 7fc0e3bf0b80 10 client.15420 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T18:28:26.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.047+0000 7fc0e3bf0b80 20 client.15420 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:26.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.047+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/389734501 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5556598d1f90 0x5556598d3400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.047+0000 7fc0e3bf0b80 4 client.15420 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:26.054 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.047+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/389734501 --> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556598180d0 con 0x5556598d1f90 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.047+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/389734501 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5556598d1f90 0x5556598d3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.047+0000 7fc0b6ff5700 10 client.15420 ms_handle_connect on v2:172.21.15.103:6834/2529642104 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.047+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/389734501 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5556598d1f90 0x5556598d3400 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cfe1f6e4b8aaec6 server_cookie=7f43d2517e3c4ad8 in_seq=0 out_seq=0 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0b6ff5700 1 -- 172.21.15.84:0/389734501 <== mds.4 v2:172.21.15.103:6834/2529642104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556598180d0 con 0x5556598d1f90 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0b6ff5700 10 client.15420 handle_command_reply: tid=0 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0e3bf0b80 1 client.15420 shutdown 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/389734501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 msgr2=0x5556598c1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/389734501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 0x5556598c1460 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc0c4005cd0 tx=0x7fc0c409caf0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/389734501 shutdown_connections 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/389734501 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5556598d1f90 0x5556598d3400 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/389734501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598be3f0 0x5556598c19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/389734501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598b82d0 0x5556598c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/389734501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598bdad0 0x5556598c1460 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/389734501 >> 172.21.15.84:0/389734501 conn(0x5556597e4cf0 msgr2=0x5556598b2da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:26.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0b6ff5700 0 client.15420 ms_handle_reset on v2:172.21.15.103:6834/2529642104 2024-04-26T18:28:26.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/389734501 shutdown_connections 2024-04-26T18:28:26.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/389734501 wait complete. 2024-04-26T18:28:26.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.048+0000 7fc0e3bf0b80 20 client.15420 trim_cache size 0 max 16384 2024-04-26T18:28:26.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.068+0000 7fc0e3bf0b80 1 Processor -- start 2024-04-26T18:28:26.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.068+0000 7fc0e3bf0b80 1 -- start start 2024-04-26T18:28:26.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.068+0000 7fc0e3bf0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556597e4cf0 0x5556598b7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.068+0000 7fc0e3bf0b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 0x5556598c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.068+0000 7fc0e3bf0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598bdad0 0x5556598dc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.068+0000 7fc0e3bf0b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5556598d1740 con 0x5556598b82d0 2024-04-26T18:28:26.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0e3bf0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5556598d1e10 con 0x5556598bdad0 2024-04-26T18:28:26.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0e3bf0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5556598d24e0 con 0x5556597e4cf0 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0d27fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 0x5556598c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0d27fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 0x5556598c3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44360/0 (socket says 172.21.15.84:44360) 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0d27fc700 1 -- 172.21.15.84:0/1428062824 learned_addr learned my addr 172.21.15.84:0/1428062824 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0d37fe700 1 --2- 172.21.15.84:0/1428062824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598bdad0 0x5556598dc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/1428062824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556597e4cf0 0x5556598b7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0d27fc700 1 -- 172.21.15.84:0/1428062824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556597e4cf0 msgr2=0x5556598b7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/1428062824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556597e4cf0 0x5556598b7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0d27fc700 1 -- 172.21.15.84:0/1428062824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598bdad0 msgr2=0x5556598dc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/1428062824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598bdad0 0x5556598dc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0d27fc700 1 -- 172.21.15.84:0/1428062824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556598c2280 con 0x5556598b82d0 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0d37fe700 1 --2- 172.21.15.84:0/1428062824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598bdad0 0x5556598dc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/1428062824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556597e4cf0 0x5556598b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/1428062824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 0x5556598c3ca0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc0c409c070 tx=0x7fc0c409c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9364c17b2d84b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:26.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0b8ff9700 1 -- 172.21.15.84:0/1428062824 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40bb1a0 con 0x5556598b82d0 2024-04-26T18:28:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0b8ff9700 1 -- 172.21.15.84:0/1428062824 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c4098230 con 0x5556598b82d0 2024-04-26T18:28:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.069+0000 7fc0b8ff9700 1 -- 172.21.15.84:0/1428062824 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c409ba00 con 0x5556598b82d0 2024-04-26T18:28:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/1428062824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 msgr2=0x5556598c3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/1428062824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 0x5556598c3ca0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc0c409c070 tx=0x7fc0c409c4f0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/1428062824 shutdown_connections 2024-04-26T18:28:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/1428062824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556598bdad0 0x5556598dc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/1428062824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556597e4cf0 0x5556598b7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/1428062824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 0x5556598c3ca0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/1428062824 >> 172.21.15.84:0/1428062824 conn(0x5556598aeee0 msgr2=0x5556598bb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:26.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/1428062824 shutdown_connections 2024-04-26T18:28:26.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/1428062824 wait complete. 2024-04-26T18:28:26.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 -- start start 2024-04-26T18:28:26.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 Processor -- start 2024-04-26T18:28:26.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556597e4cf0 0x5556598dca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 0x5556598d7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598bdad0 0x5556598d7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5556598d3280 con 0x5556598b82d0 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5556598d2bb0 con 0x5556597e4cf0 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.070+0000 7fc0e3bf0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5556598d3950 con 0x5556598bdad0 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0e3bf0b80 10 client.0 _ll_register_callbacks cb 0x55565979e530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0d27fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 0x5556598d7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0d27fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 0x5556598d7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44362/0 (socket says 172.21.15.84:44362) 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0d27fc700 1 -- 172.21.15.84:0/4099952589 learned_addr learned my addr 172.21.15.84:0/4099952589 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556597e4cf0 0x5556598dca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0d37fe700 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598bdad0 0x5556598d7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0d27fc700 1 -- 172.21.15.84:0/4099952589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598bdad0 msgr2=0x5556598d7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598bdad0 0x5556598d7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0d27fc700 1 -- 172.21.15.84:0/4099952589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556597e4cf0 msgr2=0x5556598dca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.077 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556597e4cf0 0x5556598dca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0d27fc700 1 -- 172.21.15.84:0/4099952589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556598d82b0 con 0x5556598b82d0 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0d37fe700 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598bdad0 0x5556598d7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0d27fc700 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 0x5556598d7830 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc0c409c300 tx=0x7fc0c409cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f9db41956f40ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0b6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0b6ff5700 1 -- 172.21.15.84:0/4099952589 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0c40b9070 con 0x5556598b82d0 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0b6ff5700 1 -- 172.21.15.84:0/4099952589 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fc0c4009b40 con 0x5556598b82d0 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/4099952589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0c4098230 con 0x5556598b82d0 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0b6ff5700 1 -- 172.21.15.84:0/4099952589 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc0c40b5070 con 0x5556598b82d0 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0e3bf0b80 10 client.15426 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/4099952589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555659939d40 con 0x5556598b82d0 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.071+0000 7fc0b6ff5700 1 -- 172.21.15.84:0/4099952589 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0c4098230 con 0x5556598b82d0 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.072+0000 7fc0b6ff5700 1 -- 172.21.15.84:0/4099952589 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc0c4062070 con 0x5556598b82d0 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.072+0000 7fc0e3bf0b80 10 client.15426 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:26.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.072+0000 7fc0e3bf0b80 10 client.15426 resolve_mds: resolved name 'e' to daemon mds.e 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.072+0000 7fc0e3bf0b80 20 client.15426 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.072+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5556598be3f0 0x55565993d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.072+0000 7fc0e3bf0b80 4 client.15426 mds_command: new command op to 24407 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.072+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/4099952589 --> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] -- command(tid 0: {"prefix": "status"}) -- 0x555659812170 con 0x5556598be3f0 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.072+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5556598be3f0 0x55565993d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.072+0000 7fc0d2ffd700 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5556598be3f0 0x55565993d080 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=669680352b1ddfcb server_cookie=6c0335c1dd71871e in_seq=0 out_seq=0 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.072+0000 7fc0b6ff5700 10 client.15426 ms_handle_connect on v2:172.21.15.103:6834/2529642104 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0b6ff5700 1 -- 172.21.15.84:0/4099952589 <== mds.4 v2:172.21.15.103:6834/2529642104 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x555659812170 con 0x5556598be3f0 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0b6ff5700 10 client.15426 handle_command_reply: tid=0 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0e3bf0b80 1 client.15426 shutdown 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/4099952589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 msgr2=0x5556598d7830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 0x5556598d7830 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc0c409c300 tx=0x7fc0c409cbe0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/4099952589 shutdown_connections 2024-04-26T18:28:26.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.103:6834/2529642104,v1:172.21.15.103:6835/2529642104] conn(0x5556598be3f0 0x55565993d080 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5556598bdad0 0x5556598d7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5556597e4cf0 0x5556598dca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0b6ff5700 0 client.15426 ms_handle_reset on v2:172.21.15.103:6834/2529642104 2024-04-26T18:28:26.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0e3bf0b80 1 --2- 172.21.15.84:0/4099952589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5556598b82d0 0x5556598d7830 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/4099952589 >> 172.21.15.84:0/4099952589 conn(0x5556598aeee0 msgr2=0x5556598bfce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:26.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/4099952589 shutdown_connections 2024-04-26T18:28:26.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0e3bf0b80 1 -- 172.21.15.84:0/4099952589 wait complete. 2024-04-26T18:28:26.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0e3bf0b80 20 client.15426 trim_cache size 0 max 16384 2024-04-26T18:28:26.080 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:28:26.080 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:28:26.081 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 4, 2024-04-26T18:28:26.081 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24407, 2024-04-26T18:28:26.081 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:28:26.081 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:28:26.081 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:28:26.081 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 227.78050207800001, 2024-04-26T18:28:26.081 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 29, 2024-04-26T18:28:26.081 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 94, 2024-04-26T18:28:26.081 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 93, 2024-04-26T18:28:26.081 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 278.45241114599997 2024-04-26T18:28:26.081 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:28:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0d935e700 1 -- 172.21.15.84:0/2867798260 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc0bc0a3f60 msgr2=0x7fc0bc0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0d935e700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc0bc0a3f60 0x7fc0bc0a6420 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc0c8006fd0 tx=0x7fc0c8008040 comp rx=0 tx=0).stop 2024-04-26T18:28:26.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0d935e700 1 -- 172.21.15.84:0/2867798260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0d406c510 msgr2=0x7fc0d419fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0d935e700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0d406c510 0x7fc0d419fcd0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc0c400d950 tx=0x7fc0c400dd10 comp rx=0 tx=0).stop 2024-04-26T18:28:26.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0d935e700 1 -- 172.21.15.84:0/2867798260 shutdown_connections 2024-04-26T18:28:26.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0d935e700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc0bc0a3f60 0x7fc0bc0a6420 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0d935e700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc0d41094a0 0x7fc0d41999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0d935e700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc0d406bb70 0x7fc0d419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0d935e700 1 --2- 172.21.15.84:0/2867798260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc0d406c510 0x7fc0d419fcd0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.073+0000 7fc0d935e700 1 -- 172.21.15.84:0/2867798260 >> 172.21.15.84:0/2867798260 conn(0x7fc0d40792f0 msgr2=0x7fc0d4100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:26.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.074+0000 7fc0d935e700 1 -- 172.21.15.84:0/2867798260 shutdown_connections 2024-04-26T18:28:26.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:26.074+0000 7fc0d935e700 1 -- 172.21.15.84:0/2867798260 wait complete. 2024-04-26T18:28:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.516+0000 7f4f53652640 1 -- 172.21.15.84:0/1151547713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4c100e50 msgr2=0x7f4f4c1012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.516+0000 7f4f53652640 1 --2- 172.21.15.84:0/1151547713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4c100e50 0x7f4f4c1012d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009930 tx=0x7f4f4002a420 comp rx=0 tx=0).stop 2024-04-26T18:28:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.516+0000 7f4f53652640 1 -- 172.21.15.84:0/1151547713 shutdown_connections 2024-04-26T18:28:26.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.516+0000 7f4f53652640 1 --2- 172.21.15.84:0/1151547713 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f4c101810 0x7f4f4c10f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.516+0000 7f4f53652640 1 --2- 172.21.15.84:0/1151547713 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4c100e50 0x7f4f4c1012d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.516+0000 7f4f53652640 1 --2- 172.21.15.84:0/1151547713 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f4c107930 0x7f4f4c107d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.516+0000 7f4f53652640 1 -- 172.21.15.84:0/1151547713 >> 172.21.15.84:0/1151547713 conn(0x7f4f4c077dc0 msgr2=0x7f4f4c0ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.516+0000 7f4f53652640 1 -- 172.21.15.84:0/1151547713 shutdown_connections 2024-04-26T18:28:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.516+0000 7f4f53652640 1 -- 172.21.15.84:0/1151547713 wait complete. 2024-04-26T18:28:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.516+0000 7f4f53652640 1 Processor -- start 2024-04-26T18:28:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.516+0000 7f4f53652640 1 -- start start 2024-04-26T18:28:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.516+0000 7f4f53652640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4c100e50 0x7f4f4c199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.516+0000 7f4f53652640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f4c101810 0x7f4f4c1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f53652640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f4c107930 0x7f4f4c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f53652640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c112cf0 con 0x7f4f4c100e50 2024-04-26T18:28:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f53652640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c112b70 con 0x7f4f4c107930 2024-04-26T18:28:26.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f53652640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c112e70 con 0x7f4f4c101810 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f513ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4c100e50 0x7f4f4c199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f513ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4c100e50 0x7f4f4c199190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44374/0 (socket says 172.21.15.84:44374) 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f513ee640 1 -- 172.21.15.84:0/891380522 learned_addr learned my addr 172.21.15.84:0/891380522 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f51bef640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f4c107930 0x7f4f4c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f50bed640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f4c101810 0x7f4f4c1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f513ee640 1 -- 172.21.15.84:0/891380522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f4c101810 msgr2=0x7f4f4c1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f513ee640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f4c101810 0x7f4f4c1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f513ee640 1 -- 172.21.15.84:0/891380522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f4c107930 msgr2=0x7f4f4c19da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f513ee640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f4c107930 0x7f4f4c19da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f513ee640 1 -- 172.21.15.84:0/891380522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c19e130 con 0x7f4f4c100e50 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f51bef640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f4c107930 0x7f4f4c19da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f50bed640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f4c101810 0x7f4f4c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f513ee640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4c100e50 0x7f4f4c199190 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800d950 tx=0x7f4f4800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:26.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f3e7fc640 1 -- 172.21.15.84:0/891380522 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48017030 con 0x7f4f4c100e50 2024-04-26T18:28:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f53652640 1 -- 172.21.15.84:0/891380522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c19e420 con 0x7f4f4c100e50 2024-04-26T18:28:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f3e7fc640 1 -- 172.21.15.84:0/891380522 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f48004020 con 0x7f4f4c100e50 2024-04-26T18:28:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f3e7fc640 1 -- 172.21.15.84:0/891380522 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f480047a0 con 0x7f4f4c100e50 2024-04-26T18:28:26.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.517+0000 7f4f53652640 1 -- 172.21.15.84:0/891380522 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c1a5cb0 con 0x7f4f4c100e50 2024-04-26T18:28:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.520+0000 7f4f3e7fc640 1 -- 172.21.15.84:0/891380522 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f4f48009a30 con 0x7f4f4c100e50 2024-04-26T18:28:26.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.520+0000 7f4f5636f740 1 Processor -- start 2024-04-26T18:28:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.520+0000 7f4f5636f740 1 -- start start 2024-04-26T18:28:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f5636f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 0x558b4c8ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f5636f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8ad820 0x558b4c8b7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f5636f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8b80e0 0x558b4c8ba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f5636f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558b4c813460 con 0x558b4c8acf00 2024-04-26T18:28:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f5636f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558b4c7cd590 con 0x558b4c8ad820 2024-04-26T18:28:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f5636f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558b4c80f420 con 0x558b4c8b80e0 2024-04-26T18:28:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f51bef640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 0x558b4c8ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f3e7fc640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4f380a3f70 0x7f4f380a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f51bef640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 0x558b4c8ad2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44390/0 (socket says 172.21.15.84:44390) 2024-04-26T18:28:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f51bef640 1 -- 172.21.15.84:0/5595307 learned_addr learned my addr 172.21.15.84:0/5595307 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:26.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f50bed640 1 --2- 172.21.15.84:0/5595307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8b80e0 0x558b4c8ba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f51bef640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4f380a3f70 0x7f4f380a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f3e7fc640 1 -- 172.21.15.84:0/891380522 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f4f480d05e0 con 0x7f4f4c100e50 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f513ee640 1 --2- 172.21.15.84:0/5595307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8ad820 0x558b4c8b7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f51bef640 1 -- 172.21.15.84:0/5595307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8b80e0 msgr2=0x558b4c8ba4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f51bef640 1 --2- 172.21.15.84:0/5595307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8b80e0 0x558b4c8ba4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f51bef640 1 -- 172.21.15.84:0/5595307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8ad820 msgr2=0x558b4c8b7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f51bef640 1 --2- 172.21.15.84:0/5595307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8ad820 0x558b4c8b7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f51bef640 1 -- 172.21.15.84:0/5595307 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b4c7da0d0 con 0x558b4c8acf00 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f50bed640 1 --2- 172.21.15.84:0/5595307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8b80e0 0x558b4c8ba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f51bef640 1 --2- 172.21.15.84:0/5595307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 0x558b4c8ad2e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4f44014a90 tx=0x7f4f44014f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de4bcbed8b47314f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f377fe640 1 -- 172.21.15.84:0/5595307 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f44014530 con 0x558b4c8acf00 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f377fe640 1 -- 172.21.15.84:0/5595307 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f44023070 con 0x558b4c8acf00 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f51bef640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4f380a3f70 0x7f4f380a6430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4f44022000 tx=0x7f4f44020000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:26.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.521+0000 7f4f377fe640 1 -- 172.21.15.84:0/5595307 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4400cc90 con 0x558b4c8acf00 2024-04-26T18:28:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.522+0000 7f4f5636f740 1 -- 172.21.15.84:0/5595307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 msgr2=0x558b4c8ad2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.522+0000 7f4f5636f740 1 --2- 172.21.15.84:0/5595307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 0x558b4c8ad2e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4f44014a90 tx=0x7f4f44014f50 comp rx=0 tx=0).stop 2024-04-26T18:28:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.522+0000 7f4f5636f740 1 -- 172.21.15.84:0/5595307 shutdown_connections 2024-04-26T18:28:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.522+0000 7f4f5636f740 1 --2- 172.21.15.84:0/5595307 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8b80e0 0x558b4c8ba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.522+0000 7f4f5636f740 1 --2- 172.21.15.84:0/5595307 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8ad820 0x558b4c8b7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.522+0000 7f4f5636f740 1 --2- 172.21.15.84:0/5595307 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 0x558b4c8ad2e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.522+0000 7f4f5636f740 1 -- 172.21.15.84:0/5595307 >> 172.21.15.84:0/5595307 conn(0x558b4c8a9410 msgr2=0x558b4c8a97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.522+0000 7f4f5636f740 1 -- 172.21.15.84:0/5595307 shutdown_connections 2024-04-26T18:28:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.522+0000 7f4f5636f740 1 -- 172.21.15.84:0/5595307 wait complete. 2024-04-26T18:28:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.522+0000 7f4f5636f740 1 -- start start 2024-04-26T18:28:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f5636f740 1 Processor -- start 2024-04-26T18:28:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f5636f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8acf00 0x558b4c8b4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f5636f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 0x558b4c8b54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f5636f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8b80e0 0x558b4c8b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f5636f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558b4c8bad10 con 0x558b4c8ad820 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f5636f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558b4c79ab60 con 0x558b4c8b80e0 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f5636f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558b4c8bae90 con 0x558b4c8acf00 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f5636f740 10 client.0 _ll_register_callbacks cb 0x558b4c7bf2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f50bed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 0x558b4c8b54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f50bed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 0x558b4c8b54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44398/0 (socket says 172.21.15.84:44398) 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f50bed640 1 -- 172.21.15.84:0/2132257579 learned_addr learned my addr 172.21.15.84:0/2132257579 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f51bef640 1 --2- 172.21.15.84:0/2132257579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8b80e0 0x558b4c8b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f513ee640 1 --2- 172.21.15.84:0/2132257579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8acf00 0x558b4c8b4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f50bed640 1 -- 172.21.15.84:0/2132257579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8acf00 msgr2=0x558b4c8b4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f50bed640 1 --2- 172.21.15.84:0/2132257579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8acf00 0x558b4c8b4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f50bed640 1 -- 172.21.15.84:0/2132257579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8b80e0 msgr2=0x558b4c8b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.531 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f50bed640 1 --2- 172.21.15.84:0/2132257579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8b80e0 0x558b4c8b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.523+0000 7f4f50bed640 1 -- 172.21.15.84:0/2132257579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b4c525210 con 0x558b4c8ad820 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f50bed640 1 --2- 172.21.15.84:0/2132257579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 0x558b4c8b54e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009930 tx=0x7f4f40031750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6a1a6fd835b165 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f34ff9640 1 -- 172.21.15.84:0/2132257579 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40035030 con 0x558b4c8ad820 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f34ff9640 1 -- 172.21.15.84:0/2132257579 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f4f40004460 con 0x558b4c8ad820 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f5636f740 1 -- 172.21.15.84:0/2132257579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f44023070 con 0x558b4c8ad820 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f34ff9640 1 -- 172.21.15.84:0/2132257579 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f40005030 con 0x558b4c8ad820 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f5636f740 10 client.15438 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f5636f740 1 -- 172.21.15.84:0/2132257579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b4c5f1620 con 0x558b4c8ad820 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f34ff9640 1 -- 172.21.15.84:0/2132257579 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f44023070 con 0x558b4c8ad820 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f34ff9640 1 -- 172.21.15.84:0/2132257579 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4f40004c70 con 0x558b4c8ad820 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f5636f740 10 client.15438 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f5636f740 10 client.15438 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f5636f740 20 client.15438 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:26.532 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f5636f740 1 --2- 172.21.15.84:0/2132257579 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558b4c8c8150 0x558b4c8ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f5636f740 4 client.15438 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.524+0000 7f4f5636f740 1 -- 172.21.15.84:0/2132257579 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b4c821070 con 0x558b4c8c8150 2024-04-26T18:28:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.525+0000 7f4f513ee640 1 --2- 172.21.15.84:0/2132257579 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558b4c8c8150 0x558b4c8ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.525+0000 7f4f513ee640 1 --2- 172.21.15.84:0/2132257579 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558b4c8c8150 0x558b4c8ca540 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd374b019d60d5ec server_cookie=3424c6ca645bced2 in_seq=0 out_seq=0 2024-04-26T18:28:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.525+0000 7f4f34ff9640 10 client.15438 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f34ff9640 1 -- 172.21.15.84:0/2132257579 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b4c821070 con 0x558b4c8c8150 2024-04-26T18:28:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f34ff9640 10 client.15438 handle_command_reply: tid=0 2024-04-26T18:28:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f5636f740 1 client.15438 shutdown 2024-04-26T18:28:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f5636f740 1 -- 172.21.15.84:0/2132257579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 msgr2=0x558b4c8b54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f5636f740 1 --2- 172.21.15.84:0/2132257579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 0x558b4c8b54e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4f40009930 tx=0x7f4f40031750 comp rx=0 tx=0).stop 2024-04-26T18:28:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f5636f740 1 -- 172.21.15.84:0/2132257579 shutdown_connections 2024-04-26T18:28:26.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f5636f740 1 --2- 172.21.15.84:0/2132257579 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558b4c8c8150 0x558b4c8ca540 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f5636f740 1 --2- 172.21.15.84:0/2132257579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8b80e0 0x558b4c8b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f5636f740 1 --2- 172.21.15.84:0/2132257579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 0x558b4c8b54e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f34ff9640 0 client.15438 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f5636f740 1 --2- 172.21.15.84:0/2132257579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8acf00 0x558b4c8b4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f5636f740 1 -- 172.21.15.84:0/2132257579 >> 172.21.15.84:0/2132257579 conn(0x558b4c8a9410 msgr2=0x558b4c8af190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f5636f740 1 -- 172.21.15.84:0/2132257579 shutdown_connections 2024-04-26T18:28:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f5636f740 1 -- 172.21.15.84:0/2132257579 wait complete. 2024-04-26T18:28:26.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.526+0000 7f4f5636f740 20 client.15438 trim_cache size 0 max 16384 2024-04-26T18:28:26.545 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f5636f740 1 Processor -- start 2024-04-26T18:28:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f5636f740 1 -- start start 2024-04-26T18:28:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f5636f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 0x558b4c8af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f5636f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8ad820 0x558b4c8b6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f5636f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8b80e0 0x558b4c8cfe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f5636f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1ae530 con 0x558b4c8acf00 2024-04-26T18:28:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f5636f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1b0070 con 0x558b4c8b80e0 2024-04-26T18:28:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f5636f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1aec00 con 0x558b4c8ad820 2024-04-26T18:28:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f513ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 0x558b4c8af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f513ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 0x558b4c8af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44402/0 (socket says 172.21.15.84:44402) 2024-04-26T18:28:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f513ee640 1 -- 172.21.15.84:0/3541236360 learned_addr learned my addr 172.21.15.84:0/3541236360 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f50bed640 1 --2- 172.21.15.84:0/3541236360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8ad820 0x558b4c8b6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f513ee640 1 -- 172.21.15.84:0/3541236360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8ad820 msgr2=0x558b4c8b6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f51bef640 1 --2- 172.21.15.84:0/3541236360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8b80e0 0x558b4c8cfe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f513ee640 1 --2- 172.21.15.84:0/3541236360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8ad820 0x558b4c8b6ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f513ee640 1 -- 172.21.15.84:0/3541236360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8b80e0 msgr2=0x558b4c8cfe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f513ee640 1 --2- 172.21.15.84:0/3541236360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8b80e0 0x558b4c8cfe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f513ee640 1 -- 172.21.15.84:0/3541236360 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b4c8ac700 con 0x558b4c8acf00 2024-04-26T18:28:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f50bed640 1 --2- 172.21.15.84:0/3541236360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8ad820 0x558b4c8b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f51bef640 1 --2- 172.21.15.84:0/3541236360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8b80e0 0x558b4c8cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.541+0000 7f4f513ee640 1 --2- 172.21.15.84:0/3541236360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 0x558b4c8af8c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4f48098790 tx=0x7f4f480d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c3bd7bb68e2fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.542+0000 7f4f36ffd640 1 -- 172.21.15.84:0/3541236360 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f4809c330 con 0x558b4c8acf00 2024-04-26T18:28:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.542+0000 7f4f36ffd640 1 -- 172.21.15.84:0/3541236360 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f4809a3d0 con 0x558b4c8acf00 2024-04-26T18:28:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.542+0000 7f4f36ffd640 1 -- 172.21.15.84:0/3541236360 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f48098080 con 0x558b4c8acf00 2024-04-26T18:28:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.542+0000 7f4f5636f740 1 -- 172.21.15.84:0/3541236360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 msgr2=0x558b4c8af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.542+0000 7f4f5636f740 1 --2- 172.21.15.84:0/3541236360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 0x558b4c8af8c0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4f48098790 tx=0x7f4f480d0040 comp rx=0 tx=0).stop 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.542+0000 7f4f5636f740 1 -- 172.21.15.84:0/3541236360 shutdown_connections 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.542+0000 7f4f5636f740 1 --2- 172.21.15.84:0/3541236360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8b80e0 0x558b4c8cfe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.542+0000 7f4f5636f740 1 --2- 172.21.15.84:0/3541236360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8ad820 0x558b4c8b6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.542+0000 7f4f5636f740 1 --2- 172.21.15.84:0/3541236360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8acf00 0x558b4c8af8c0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.542+0000 7f4f5636f740 1 -- 172.21.15.84:0/3541236360 >> 172.21.15.84:0/3541236360 conn(0x558b4c8a9410 msgr2=0x558b4c8b67d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.542+0000 7f4f5636f740 1 -- 172.21.15.84:0/3541236360 shutdown_connections 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.542+0000 7f4f5636f740 1 -- 172.21.15.84:0/3541236360 wait complete. 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f5636f740 1 -- start start 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f5636f740 1 Processor -- start 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f5636f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8acf00 0x558b4c8b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f5636f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 0x558b4c8b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f5636f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8b80e0 0x558b4c92d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f5636f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1af9a0 con 0x558b4c8ad820 2024-04-26T18:28:26.548 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f5636f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1af2d0 con 0x558b4c8acf00 2024-04-26T18:28:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f5636f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c1ac510 con 0x558b4c8b80e0 2024-04-26T18:28:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f5636f740 10 client.0 _ll_register_callbacks cb 0x7f4f40009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f50bed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 0x558b4c8b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f50bed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 0x558b4c8b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44414/0 (socket says 172.21.15.84:44414) 2024-04-26T18:28:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f50bed640 1 -- 172.21.15.84:0/2695856563 learned_addr learned my addr 172.21.15.84:0/2695856563 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f513ee640 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8acf00 0x558b4c8b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f51bef640 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8b80e0 0x558b4c92d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f50bed640 1 -- 172.21.15.84:0/2695856563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8b80e0 msgr2=0x558b4c92d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f50bed640 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8b80e0 0x558b4c92d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f50bed640 1 -- 172.21.15.84:0/2695856563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8acf00 msgr2=0x558b4c8b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f50bed640 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8acf00 0x558b4c8b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.549 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f50bed640 1 -- 172.21.15.84:0/2695856563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b4c92dbe0 con 0x558b4c8ad820 2024-04-26T18:28:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.543+0000 7f4f513ee640 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8acf00 0x558b4c8b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f50bed640 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 0x558b4c8b28c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4f400035f0 tx=0x7f4f400319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9abd6599a5ecfa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f34ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f34ff9640 1 -- 172.21.15.84:0/2695856563 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f40000e60 con 0x558b4c8ad820 2024-04-26T18:28:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f34ff9640 1 -- 172.21.15.84:0/2695856563 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f4f40004460 con 0x558b4c8ad820 2024-04-26T18:28:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f5636f740 1 -- 172.21.15.84:0/2695856563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f4809a3d0 con 0x558b4c8ad820 2024-04-26T18:28:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f34ff9640 1 -- 172.21.15.84:0/2695856563 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f40035070 con 0x558b4c8ad820 2024-04-26T18:28:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f5636f740 10 client.15444 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f5636f740 1 -- 172.21.15.84:0/2695856563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b4c92ee10 con 0x558b4c8ad820 2024-04-26T18:28:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f34ff9640 1 -- 172.21.15.84:0/2695856563 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f4809a3d0 con 0x558b4c8ad820 2024-04-26T18:28:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f34ff9640 1 -- 172.21.15.84:0/2695856563 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4f40004aa0 con 0x558b4c8ad820 2024-04-26T18:28:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f5636f740 10 client.15444 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:26.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f5636f740 10 client.15444 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f5636f740 20 client.15444 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f5636f740 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558b4c92f5e0 0x558b4c931a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f5636f740 4 client.15444 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:28:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.544+0000 7f4f5636f740 1 -- 172.21.15.84:0/2695856563 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558b4c81aee0 con 0x558b4c92f5e0 2024-04-26T18:28:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.545+0000 7f4f513ee640 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558b4c92f5e0 0x558b4c931a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.545+0000 7f4f513ee640 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558b4c92f5e0 0x558b4c931a00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dd15d56a14a9b84 server_cookie=7b9c2492a1db9288 in_seq=0 out_seq=0 2024-04-26T18:28:26.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.545+0000 7f4f34ff9640 10 client.15444 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.552+0000 7f4f34ff9640 1 -- 172.21.15.84:0/2695856563 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558b4c81aee0 con 0x558b4c92f5e0 2024-04-26T18:28:26.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.552+0000 7f4f34ff9640 10 client.15444 handle_command_reply: tid=0 2024-04-26T18:28:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.553+0000 7f4f5636f740 1 client.15444 shutdown 2024-04-26T18:28:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.553+0000 7f4f5636f740 1 -- 172.21.15.84:0/2695856563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 msgr2=0x558b4c8b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.553+0000 7f4f5636f740 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 0x558b4c8b28c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4f400035f0 tx=0x7f4f400319f0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.553+0000 7f4f5636f740 1 -- 172.21.15.84:0/2695856563 shutdown_connections 2024-04-26T18:28:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.553+0000 7f4f5636f740 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558b4c92f5e0 0x558b4c931a00 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.553+0000 7f4f5636f740 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558b4c8b80e0 0x558b4c92d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.553+0000 7f4f5636f740 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558b4c8ad820 0x558b4c8b28c0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.553+0000 7f4f34ff9640 0 client.15444 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.553+0000 7f4f5636f740 1 --2- 172.21.15.84:0/2695856563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558b4c8acf00 0x558b4c8b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.553+0000 7f4f5636f740 1 -- 172.21.15.84:0/2695856563 >> 172.21.15.84:0/2695856563 conn(0x558b4c8a9410 msgr2=0x558b4c8a7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:26.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.553+0000 7f4f5636f740 1 -- 172.21.15.84:0/2695856563 shutdown_connections 2024-04-26T18:28:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.553+0000 7f4f5636f740 1 -- 172.21.15.84:0/2695856563 wait complete. 2024-04-26T18:28:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.553+0000 7f4f5636f740 20 client.15444 trim_cache size 0 max 16384 2024-04-26T18:28:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:28:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:28:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:28:26.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:28:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.554+0000 7f4f53652640 1 -- 172.21.15.84:0/891380522 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4f380a3f70 msgr2=0x7f4f380a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.554+0000 7f4f53652640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4f380a3f70 0x7f4f380a6430 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f4f44022000 tx=0x7f4f44020000 comp rx=0 tx=0).stop 2024-04-26T18:28:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.554+0000 7f4f53652640 1 -- 172.21.15.84:0/891380522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4c100e50 msgr2=0x7f4f4c199190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.554+0000 7f4f53652640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4c100e50 0x7f4f4c199190 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f4f4800d950 tx=0x7f4f4800de10 comp rx=0 tx=0).stop 2024-04-26T18:28:26.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.554+0000 7f4f53652640 1 -- 172.21.15.84:0/891380522 shutdown_connections 2024-04-26T18:28:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.554+0000 7f4f53652640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4f380a3f70 0x7f4f380a6430 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.554+0000 7f4f53652640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4f4c107930 0x7f4f4c19da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.554+0000 7f4f53652640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4f4c101810 0x7f4f4c1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.554+0000 7f4f53652640 1 --2- 172.21.15.84:0/891380522 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4f4c100e50 0x7f4f4c199190 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.554+0000 7f4f53652640 1 -- 172.21.15.84:0/891380522 >> 172.21.15.84:0/891380522 conn(0x7f4f4c077dc0 msgr2=0x7f4f4c105a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.554+0000 7f4f53652640 1 -- 172.21.15.84:0/891380522 shutdown_connections 2024-04-26T18:28:26.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:26.554+0000 7f4f53652640 1 -- 172.21.15.84:0/891380522 wait complete. 2024-04-26T18:28:26.945 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.c status 2024-04-26T18:28:27.181 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:27.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:27 smithi084 ceph-mon[29141]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-04-26T18:28:27.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:27 smithi103 ceph-mon[32727]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-04-26T18:28:27.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:27 smithi174 ceph-mon[32705]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-04-26T18:28:28.875 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.873+0000 7f8bc5549700 1 -- 172.21.15.84:0/1945639634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bc0102960 msgr2=0x7f8bc0102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.873+0000 7f8bc5549700 1 --2- 172.21.15.84:0/1945639634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bc0102960 0x7f8bc0102d40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80099c0 tx=0x7f8ba8009cd0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.874+0000 7f8bc5549700 1 -- 172.21.15.84:0/1945639634 shutdown_connections 2024-04-26T18:28:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.874+0000 7f8bc5549700 1 --2- 172.21.15.84:0/1945639634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bc0102960 0x7f8bc0102d40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.874+0000 7f8bc5549700 1 --2- 172.21.15.84:0/1945639634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bc01076b0 0x7f8bc010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.874+0000 7f8bc5549700 1 --2- 172.21.15.84:0/1945639634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc0103310 0x7f8bc0106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.874+0000 7f8bc5549700 1 -- 172.21.15.84:0/1945639634 >> 172.21.15.84:0/1945639634 conn(0x7f8bc00792d0 msgr2=0x7f8bc00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.874+0000 7f8bc5549700 1 -- 172.21.15.84:0/1945639634 shutdown_connections 2024-04-26T18:28:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.874+0000 7f8bc5549700 1 -- 172.21.15.84:0/1945639634 wait complete. 2024-04-26T18:28:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.874+0000 7f8bc5549700 1 Processor -- start 2024-04-26T18:28:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.874+0000 7f8bc5549700 1 -- start start 2024-04-26T18:28:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bc5549700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc0102960 0x7f8bc0104020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.876 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bc5549700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bc0103310 0x7f8bc0104560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bc5549700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bc01076b0 0x7f8bc0104aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bc5549700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0112510 con 0x7f8bc01076b0 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bc5549700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0112390 con 0x7f8bc0103310 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bc5549700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8bc0112690 con 0x7f8bc0102960 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bbf7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bc01076b0 0x7f8bc0104aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bbf7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bc01076b0 0x7f8bc0104aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43944/0 (socket says 172.21.15.84:43944) 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bbf7fe700 1 -- 172.21.15.84:0/552291058 learned_addr learned my addr 172.21.15.84:0/552291058 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bbe7fc700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bc0103310 0x7f8bc0104560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bbeffd700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc0102960 0x7f8bc0104020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bbf7fe700 1 -- 172.21.15.84:0/552291058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc0102960 msgr2=0x7f8bc0104020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bbf7fe700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc0102960 0x7f8bc0104020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bbf7fe700 1 -- 172.21.15.84:0/552291058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bc0103310 msgr2=0x7f8bc0104560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bbf7fe700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bc0103310 0x7f8bc0104560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bbf7fe700 1 -- 172.21.15.84:0/552291058 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bc01a80d0 con 0x7f8bc01076b0 2024-04-26T18:28:28.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bbf7fe700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bc01076b0 0x7f8bc0104aa0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8007ee0 tx=0x7f8bb800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:28.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bb7fff700 1 -- 172.21.15.84:0/552291058 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8004bb0 con 0x7f8bc01076b0 2024-04-26T18:28:28.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bc5549700 1 -- 172.21.15.84:0/552291058 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bc01a83c0 con 0x7f8bc01076b0 2024-04-26T18:28:28.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bb7fff700 1 -- 172.21.15.84:0/552291058 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb8004d50 con 0x7f8bc01076b0 2024-04-26T18:28:28.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bb7fff700 1 -- 172.21.15.84:0/552291058 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb8005920 con 0x7f8bc01076b0 2024-04-26T18:28:28.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.875+0000 7f8bc5549700 1 -- 172.21.15.84:0/552291058 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bc01a88b0 con 0x7f8bc01076b0 2024-04-26T18:28:28.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.878+0000 7f8bb7fff700 1 -- 172.21.15.84:0/552291058 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f8bb801a020 con 0x7f8bc01076b0 2024-04-26T18:28:28.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.879+0000 7f8bb7fff700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8bac0a3f60 0x7f8bac0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.879+0000 7f8bb7fff700 1 -- 172.21.15.84:0/552291058 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8bb80d0110 con 0x7f8bc01076b0 2024-04-26T18:28:28.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.879+0000 7f8bbeffd700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8bac0a3f60 0x7f8bac0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.879+0000 7f8bbeffd700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8bac0a3f60 0x7f8bac0a6420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009990 tx=0x7f8ba8009650 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.879+0000 7f8bcfddbb80 1 Processor -- start 2024-04-26T18:28:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.879+0000 7f8bcfddbb80 1 -- start start 2024-04-26T18:28:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bcfddbb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9be04a20 0x55dc9be04e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bcfddbb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdfed50 0x55dc9bdff1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bcfddbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9bdff6f0 0x55dc9be0b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bcfddbb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dc9bd517e0 con 0x55dc9bdfed50 2024-04-26T18:28:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bcfddbb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc9bcb78e0 con 0x55dc9be04a20 2024-04-26T18:28:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bcfddbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc9bd4d7a0 con 0x55dc9bdff6f0 2024-04-26T18:28:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bbe7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdfed50 0x55dc9bdff1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bbe7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdfed50 0x55dc9bdff1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43952/0 (socket says 172.21.15.84:43952) 2024-04-26T18:28:28.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bbe7fc700 1 -- 172.21.15.84:0/512086559 learned_addr learned my addr 172.21.15.84:0/512086559 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bbf7fe700 1 --2- 172.21.15.84:0/512086559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9bdff6f0 0x55dc9be0b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bbeffd700 1 --2- 172.21.15.84:0/512086559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9be04a20 0x55dc9be04e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bbe7fc700 1 -- 172.21.15.84:0/512086559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9bdff6f0 msgr2=0x55dc9be0b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bbe7fc700 1 --2- 172.21.15.84:0/512086559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9bdff6f0 0x55dc9be0b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bbe7fc700 1 -- 172.21.15.84:0/512086559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9be04a20 msgr2=0x55dc9be04e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bbe7fc700 1 --2- 172.21.15.84:0/512086559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9be04a20 0x55dc9be04e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bbe7fc700 1 -- 172.21.15.84:0/512086559 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc9bc9e880 con 0x55dc9bdfed50 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bbeffd700 1 --2- 172.21.15.84:0/512086559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9be04a20 0x55dc9be04e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.880+0000 7f8bbe7fc700 1 --2- 172.21.15.84:0/512086559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdfed50 0x55dc9bdff1b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000bc80 tx=0x7f8bb000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328c76ebfd75db5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.881+0000 7f8bb57fa700 1 -- 172.21.15.84:0/512086559 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0010ca0 con 0x55dc9bdfed50 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.881+0000 7f8bb57fa700 1 -- 172.21.15.84:0/512086559 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb0010e40 con 0x55dc9bdfed50 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.881+0000 7f8bb57fa700 1 -- 172.21.15.84:0/512086559 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0005280 con 0x55dc9bdfed50 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.881+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/512086559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdfed50 msgr2=0x55dc9bdff1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.881+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/512086559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdfed50 0x55dc9bdff1b0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000bc80 tx=0x7f8bb000bf90 comp rx=0 tx=0).stop 2024-04-26T18:28:28.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.881+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/512086559 shutdown_connections 2024-04-26T18:28:28.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.881+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/512086559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9bdff6f0 0x55dc9be0b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.881+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/512086559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9be04a20 0x55dc9be04e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.881+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/512086559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdfed50 0x55dc9bdff1b0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.881+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/512086559 >> 172.21.15.84:0/512086559 conn(0x55dc9bd2d570 msgr2=0x55dc9bdf9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:28.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.882+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/512086559 shutdown_connections 2024-04-26T18:28:28.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.882+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/512086559 wait complete. 2024-04-26T18:28:28.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.882+0000 7f8bcfddbb80 1 -- start start 2024-04-26T18:28:28.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.882+0000 7f8bcfddbb80 1 Processor -- start 2024-04-26T18:28:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.882+0000 7f8bcfddbb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdfed50 0x55dc9be0b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.882+0000 7f8bcfddbb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9bdff6f0 0x55dc9be086a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.882+0000 7f8bcfddbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9be04a20 0x55dc9be08be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.882+0000 7f8bcfddbb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dc9be0bcb0 con 0x55dc9bdfed50 2024-04-26T18:28:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.882+0000 7f8bcfddbb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc9bd0ba30 con 0x55dc9bdff6f0 2024-04-26T18:28:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.882+0000 7f8bcfddbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc9be0be30 con 0x55dc9be04a20 2024-04-26T18:28:28.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.882+0000 7f8bcfddbb80 10 client.0 _ll_register_callbacks cb 0x55dc9bce9900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8bbf7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9be04a20 0x55dc9be08be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8bbf7fe700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9be04a20 0x55dc9be08be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.84:49258/0 (socket says 172.21.15.84:49258) 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8bbf7fe700 1 -- 172.21.15.84:0/1871121747 learned_addr learned my addr 172.21.15.84:0/1871121747 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8bbe7fc700 1 --2- 172.21.15.84:0/1871121747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9bdff6f0 0x55dc9be086a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8bbf7fe700 1 -- 172.21.15.84:0/1871121747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9bdff6f0 msgr2=0x55dc9be086a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8bbf7fe700 1 --2- 172.21.15.84:0/1871121747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9bdff6f0 0x55dc9be086a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8bbf7fe700 1 -- 172.21.15.84:0/1871121747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdfed50 msgr2=0x55dc9be0b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8bbf7fe700 1 --2- 172.21.15.84:0/1871121747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdfed50 0x55dc9be0b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8bbf7fe700 1 -- 172.21.15.84:0/1871121747 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc9bd0f430 con 0x55dc9be04a20 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8bbf7fe700 1 --2- 172.21.15.84:0/1871121747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9be04a20 0x55dc9be08be0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bb809bdf0 tx=0x7f8bb8005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=232e8ec07e826c88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8ba67fc700 10 client.0 ms_handle_connect on v2:172.21.15.174:3300/0 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8ba67fc700 1 -- 172.21.15.84:0/1871121747 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb80a0210 con 0x55dc9be04a20 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1871121747 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb0010e40 con 0x55dc9be04a20 2024-04-26T18:28:28.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8ba67fc700 1 -- 172.21.15.84:0/1871121747 <== mon.1 v2:172.21.15.174:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8bb80b5220 con 0x55dc9be04a20 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8ba67fc700 1 -- 172.21.15.84:0/1871121747 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8bb80a4ad0 con 0x55dc9be04a20 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8bcfddbb80 10 client.24991 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.883+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1871121747 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc9be164e0 con 0x55dc9be04a20 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.884+0000 7f8ba67fc700 1 -- 172.21.15.84:0/1871121747 <== mon.1 v2:172.21.15.174:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb0010e40 con 0x55dc9be04a20 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.884+0000 7f8ba67fc700 1 -- 172.21.15.84:0/1871121747 <== mon.1 v2:172.21.15.174:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8bb80cd920 con 0x55dc9be04a20 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.884+0000 7f8bcfddbb80 10 client.24991 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.884+0000 7f8bcfddbb80 10 client.24991 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.884+0000 7f8bcfddbb80 20 client.24991 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.884+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/1871121747 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x55dc9be18b40 0x55dc9be19fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.884+0000 7f8bcfddbb80 4 client.24991 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.884+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1871121747 --> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc9bd5f1c0 con 0x55dc9be18b40 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.884+0000 7f8bbeffd700 1 --2- 172.21.15.84:0/1871121747 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x55dc9be18b40 0x55dc9be19fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8ba67fc700 10 client.24991 ms_handle_connect on v2:172.21.15.174:6836/1532309720 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8bbeffd700 1 --2- 172.21.15.84:0/1871121747 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x55dc9be18b40 0x55dc9be19fb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=addd394276425441 server_cookie=36ce22335318c48 in_seq=0 out_seq=0 2024-04-26T18:28:28.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8ba67fc700 1 -- 172.21.15.84:0/1871121747 <== mds.3 v2:172.21.15.174:6836/1532309720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dc9bd5f1c0 con 0x55dc9be18b40 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8ba67fc700 10 client.24991 handle_command_reply: tid=0 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8bcfddbb80 1 client.24991 shutdown 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1871121747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9be04a20 msgr2=0x55dc9be08be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/1871121747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9be04a20 0x55dc9be08be0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bb809bdf0 tx=0x7f8bb8005bb0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1871121747 shutdown_connections 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/1871121747 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x55dc9be18b40 0x55dc9be19fb0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/1871121747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdfed50 0x55dc9be0b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/1871121747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9be04a20 0x55dc9be08be0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/1871121747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9bdff6f0 0x55dc9be086a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8ba67fc700 0 client.24991 ms_handle_reset on v2:172.21.15.174:6836/1532309720 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.885+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1871121747 >> 172.21.15.84:0/1871121747 conn(0x55dc9bd2d570 msgr2=0x55dc9bdfdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.886+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1871121747 shutdown_connections 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.886+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1871121747 wait complete. 2024-04-26T18:28:28.890 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.886+0000 7f8bcfddbb80 20 client.24991 trim_cache size 0 max 16384 2024-04-26T18:28:28.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.906+0000 7f8bcfddbb80 1 Processor -- start 2024-04-26T18:28:28.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.906+0000 7f8bcfddbb80 1 -- start start 2024-04-26T18:28:28.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.906+0000 7f8bcfddbb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9bdfed50 0x55dc9bdfdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.906+0000 7f8bcfddbb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 0x55dc9be07c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.906+0000 7f8bcfddbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9be04a20 0x55dc9bdf94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.906+0000 7f8bcfddbb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dc9be182f0 con 0x55dc9bdff6f0 2024-04-26T18:28:28.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.906+0000 7f8bcfddbb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc9be189c0 con 0x55dc9bdfed50 2024-04-26T18:28:28.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.906+0000 7f8bcfddbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc9be19090 con 0x55dc9be04a20 2024-04-26T18:28:28.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.906+0000 7f8bbe7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 0x55dc9be07c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.906+0000 7f8bbe7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 0x55dc9be07c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43972/0 (socket says 172.21.15.84:43972) 2024-04-26T18:28:28.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.906+0000 7f8bbe7fc700 1 -- 172.21.15.84:0/4240070129 learned_addr learned my addr 172.21.15.84:0/4240070129 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:28.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.906+0000 7f8bbf7fe700 1 --2- 172.21.15.84:0/4240070129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9be04a20 0x55dc9bdf94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.906+0000 7f8bbeffd700 1 --2- 172.21.15.84:0/4240070129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9bdfed50 0x55dc9bdfdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.907+0000 7f8bbe7fc700 1 -- 172.21.15.84:0/4240070129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9be04a20 msgr2=0x55dc9bdf94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.907+0000 7f8bbe7fc700 1 --2- 172.21.15.84:0/4240070129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9be04a20 0x55dc9bdf94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.907+0000 7f8bbe7fc700 1 -- 172.21.15.84:0/4240070129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9bdfed50 msgr2=0x55dc9bdfdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.907+0000 7f8bbe7fc700 1 --2- 172.21.15.84:0/4240070129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9bdfed50 0x55dc9bdfdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.907+0000 7f8bbe7fc700 1 -- 172.21.15.84:0/4240070129 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc9be095b0 con 0x55dc9bdff6f0 2024-04-26T18:28:28.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.907+0000 7f8bbe7fc700 1 --2- 172.21.15.84:0/4240070129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 0x55dc9be07c40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000bc80 tx=0x7f8bb0005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c4069f3c10f14a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:28.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.907+0000 7f8bb4ff9700 1 -- 172.21.15.84:0/4240070129 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0010ca0 con 0x55dc9bdff6f0 2024-04-26T18:28:28.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.907+0000 7f8bb4ff9700 1 -- 172.21.15.84:0/4240070129 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb0010e40 con 0x55dc9bdff6f0 2024-04-26T18:28:28.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.907+0000 7f8bb4ff9700 1 -- 172.21.15.84:0/4240070129 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb00052e0 con 0x55dc9bdff6f0 2024-04-26T18:28:28.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.908+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/4240070129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 msgr2=0x55dc9be07c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.908+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/4240070129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 0x55dc9be07c40 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8bb000bc80 tx=0x7f8bb0005950 comp rx=0 tx=0).stop 2024-04-26T18:28:28.913 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.908+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/4240070129 shutdown_connections 2024-04-26T18:28:28.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.908+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/4240070129 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9be04a20 0x55dc9bdf94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.908+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/4240070129 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9bdfed50 0x55dc9bdfdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.908+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/4240070129 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 0x55dc9be07c40 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.908+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/4240070129 >> 172.21.15.84:0/4240070129 conn(0x55dc9bd2d570 msgr2=0x55dc9bdfe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:28.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.908+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/4240070129 shutdown_connections 2024-04-26T18:28:28.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.908+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/4240070129 wait complete. 2024-04-26T18:28:28.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.908+0000 7f8bcfddbb80 1 -- start start 2024-04-26T18:28:28.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bcfddbb80 1 Processor -- start 2024-04-26T18:28:28.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bcfddbb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9bdfed50 0x55dc9be03710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.914 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bcfddbb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 0x55dc9be03c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bcfddbb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9be04a20 0x55dc9be21ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bcfddbb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dc9be19e30 con 0x55dc9bdff6f0 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bcfddbb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dc9be19760 con 0x55dc9be04a20 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bcfddbb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dc9be1a500 con 0x55dc9bdfed50 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bcfddbb80 10 client.0 _ll_register_callbacks cb 0x55dc9bce9900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bbe7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 0x55dc9be03c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bbe7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 0x55dc9be03c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43978/0 (socket says 172.21.15.84:43978) 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bbe7fc700 1 -- 172.21.15.84:0/1997502992 learned_addr learned my addr 172.21.15.84:0/1997502992 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bbeffd700 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9bdfed50 0x55dc9be03710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bbf7fe700 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9be04a20 0x55dc9be21ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bbe7fc700 1 -- 172.21.15.84:0/1997502992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9bdfed50 msgr2=0x55dc9be03710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bbe7fc700 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9bdfed50 0x55dc9be03710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bbe7fc700 1 -- 172.21.15.84:0/1997502992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9be04a20 msgr2=0x55dc9be21ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bbe7fc700 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9be04a20 0x55dc9be21ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.915 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bbe7fc700 1 -- 172.21.15.84:0/1997502992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc9be220a0 con 0x55dc9bdff6f0 2024-04-26T18:28:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bbf7fe700 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9be04a20 0x55dc9be21ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bbeffd700 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9bdfed50 0x55dc9be03710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8bbe7fc700 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 0x55dc9be03c50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0010d50 tx=0x7f8bb0004f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ace70648331a05a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.909+0000 7f8ba67fc700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8ba67fc700 1 -- 172.21.15.84:0/1997502992 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0033020 con 0x55dc9bdff6f0 2024-04-26T18:28:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8ba67fc700 1 -- 172.21.15.84:0/1997502992 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8bb0003680 con 0x55dc9bdff6f0 2024-04-26T18:28:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1997502992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8bb0010e40 con 0x55dc9bdff6f0 2024-04-26T18:28:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8ba67fc700 1 -- 172.21.15.84:0/1997502992 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8bb0003880 con 0x55dc9bdff6f0 2024-04-26T18:28:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8bcfddbb80 10 client.15459 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1997502992 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dc9be80530 con 0x55dc9bdff6f0 2024-04-26T18:28:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8ba67fc700 1 -- 172.21.15.84:0/1997502992 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb0010e40 con 0x55dc9bdff6f0 2024-04-26T18:28:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8ba67fc700 1 -- 172.21.15.84:0/1997502992 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8bb002fbd0 con 0x55dc9bdff6f0 2024-04-26T18:28:28.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8bcfddbb80 10 client.15459 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8bcfddbb80 10 client.15459 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8bcfddbb80 20 client.15459 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x55dc9be849c0 0x55dc9be836d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8bcfddbb80 4 client.15459 mds_command: new command op to 24418 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.910+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1997502992 --> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] -- command(tid 0: {"prefix": "status"}) -- 0x55dc9bd59260 con 0x55dc9be849c0 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8bbeffd700 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x55dc9be849c0 0x55dc9be836d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8ba67fc700 10 client.15459 ms_handle_connect on v2:172.21.15.174:6836/1532309720 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8bbeffd700 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x55dc9be849c0 0x55dc9be836d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=999bc8b91ec15b3b server_cookie=c9329df70f7a6d44 in_seq=0 out_seq=0 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8ba67fc700 1 -- 172.21.15.84:0/1997502992 <== mds.3 v2:172.21.15.174:6836/1532309720 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55dc9bd59260 con 0x55dc9be849c0 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8ba67fc700 10 client.15459 handle_command_reply: tid=0 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8bcfddbb80 1 client.15459 shutdown 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1997502992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 msgr2=0x55dc9be03c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 0x55dc9be03c50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0010d50 tx=0x7f8bb0004f60 comp rx=0 tx=0).stop 2024-04-26T18:28:28.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1997502992 shutdown_connections 2024-04-26T18:28:28.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dc9be04a20 0x55dc9be21ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dc9bdfed50 0x55dc9be03710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.174:6836/1532309720,v1:172.21.15.174:6837/1532309720] conn(0x55dc9be849c0 0x55dc9be836d0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8ba67fc700 0 client.15459 ms_handle_reset on v2:172.21.15.174:6836/1532309720 2024-04-26T18:28:28.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8bcfddbb80 1 --2- 172.21.15.84:0/1997502992 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dc9bdff6f0 0x55dc9be03c50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1997502992 >> 172.21.15.84:0/1997502992 conn(0x55dc9bd2d570 msgr2=0x55dc9bdf94d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:28.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.911+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1997502992 shutdown_connections 2024-04-26T18:28:28.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bcfddbb80 1 -- 172.21.15.84:0/1997502992 wait complete. 2024-04-26T18:28:28.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bcfddbb80 20 client.15459 trim_cache size 0 max 16384 2024-04-26T18:28:28.918 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:28:28.918 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:28:28.919 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 3, 2024-04-26T18:28:28.919 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24418, 2024-04-26T18:28:28.919 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:28:28.919 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:28:28.919 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:28:28.919 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 231.62653083699999, 2024-04-26T18:28:28.919 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 29, 2024-04-26T18:28:28.919 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 94, 2024-04-26T18:28:28.919 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 93, 2024-04-26T18:28:28.919 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 275.77774326100001 2024-04-26T18:28:28.919 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:28:28.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bc5549700 1 -- 172.21.15.84:0/552291058 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8bac0a3f60 msgr2=0x7f8bac0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bc5549700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8bac0a3f60 0x7f8bac0a6420 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009990 tx=0x7f8ba8009650 comp rx=0 tx=0).stop 2024-04-26T18:28:28.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bc5549700 1 -- 172.21.15.84:0/552291058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bc01076b0 msgr2=0x7f8bc0104aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:28.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bc5549700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bc01076b0 0x7f8bc0104aa0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8007ee0 tx=0x7f8bb800d3b0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bc5549700 1 -- 172.21.15.84:0/552291058 shutdown_connections 2024-04-26T18:28:28.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bc5549700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8bac0a3f60 0x7f8bac0a6420 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bc5549700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8bc01076b0 0x7f8bc0104aa0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bc5549700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8bc0102960 0x7f8bc0104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bc5549700 1 --2- 172.21.15.84:0/552291058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8bc0103310 0x7f8bc0104560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:28.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bc5549700 1 -- 172.21.15.84:0/552291058 >> 172.21.15.84:0/552291058 conn(0x7f8bc00792d0 msgr2=0x7f8bc0106340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:28.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bc5549700 1 -- 172.21.15.84:0/552291058 shutdown_connections 2024-04-26T18:28:28.921 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:28.912+0000 7f8bc5549700 1 -- 172.21.15.84:0/552291058 wait complete. 2024-04-26T18:28:29.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:29 smithi084 ceph-mon[29141]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-26T18:28:29.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:29 smithi103 ceph-mon[32727]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-26T18:28:29.657 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.h status 2024-04-26T18:28:29.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:29 smithi174 ceph-mon[32705]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 938 B/s wr, 8 op/s 2024-04-26T18:28:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:31.522 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:31 smithi084 ceph-mon[29141]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2024-04-26T18:28:31.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:31 smithi103 ceph-mon[32727]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2024-04-26T18:28:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:31 smithi174 ceph-mon[32705]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.1 KiB/s wr, 15 op/s 2024-04-26T18:28:31.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.867+0000 7fd8f66dc700 1 -- 172.21.15.84:0/2584734351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f0109600 msgr2=0x7fd8f01099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:31.877 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.867+0000 7fd8f66dc700 1 --2- 172.21.15.84:0/2584734351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f0109600 0x7fd8f01099e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009990 tx=0x7fd8e0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.867+0000 7fd8f66dc700 1 -- 172.21.15.84:0/2584734351 shutdown_connections 2024-04-26T18:28:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.867+0000 7fd8f66dc700 1 --2- 172.21.15.84:0/2584734351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f0109600 0x7fd8f01099e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.867+0000 7fd8f66dc700 1 --2- 172.21.15.84:0/2584734351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f0102810 0x7fd8f0110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.867+0000 7fd8f66dc700 1 --2- 172.21.15.84:0/2584734351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8f0101e50 0x7fd8f01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.867+0000 7fd8f66dc700 1 -- 172.21.15.84:0/2584734351 >> 172.21.15.84:0/2584734351 conn(0x7fd8f0077590 msgr2=0x7fd8f00779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.867+0000 7fd8f66dc700 1 -- 172.21.15.84:0/2584734351 shutdown_connections 2024-04-26T18:28:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.867+0000 7fd8f66dc700 1 -- 172.21.15.84:0/2584734351 wait complete. 2024-04-26T18:28:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.868+0000 7fd8f66dc700 1 Processor -- start 2024-04-26T18:28:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.868+0000 7fd8f66dc700 1 -- start start 2024-04-26T18:28:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.868+0000 7fd8f66dc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8f0101e50 0x7fd8f019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.868+0000 7fd8f66dc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f0102810 0x7fd8f019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.868+0000 7fd8f66dc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f0109600 0x7fd8f0199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.878 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.868+0000 7fd8f66dc700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f0113680 con 0x7fd8f0102810 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.868+0000 7fd8f66dc700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f0113500 con 0x7fd8f0101e50 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.868+0000 7fd8f66dc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8f0113800 con 0x7fd8f0109600 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.868+0000 7fd8ef7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f0102810 0x7fd8f019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.868+0000 7fd8ef7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f0102810 0x7fd8f019fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43994/0 (socket says 172.21.15.84:43994) 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.868+0000 7fd8ef7fe700 1 -- 172.21.15.84:0/3836882944 learned_addr learned my addr 172.21.15.84:0/3836882944 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8effff700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8f0101e50 0x7fd8f019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8f4c79700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f0109600 0x7fd8f0199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8ef7fe700 1 -- 172.21.15.84:0/3836882944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f0109600 msgr2=0x7fd8f0199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8ef7fe700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f0109600 0x7fd8f0199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8ef7fe700 1 -- 172.21.15.84:0/3836882944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8f0101e50 msgr2=0x7fd8f019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8ef7fe700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8f0101e50 0x7fd8f019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8ef7fe700 1 -- 172.21.15.84:0/3836882944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8f019a0c0 con 0x7fd8f0102810 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8f4c79700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f0109600 0x7fd8f0199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:31.879 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8effff700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8f0101e50 0x7fd8f019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8ef7fe700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f0102810 0x7fd8f019fce0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400b530 tx=0x7fd8e400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8ed7fa700 1 -- 172.21.15.84:0/3836882944 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4015020 con 0x7fd8f0102810 2024-04-26T18:28:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8f66dc700 1 -- 172.21.15.84:0/3836882944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8f019a3b0 con 0x7fd8f0102810 2024-04-26T18:28:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8ed7fa700 1 -- 172.21.15.84:0/3836882944 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e4011ce0 con 0x7fd8f0102810 2024-04-26T18:28:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8ed7fa700 1 -- 172.21.15.84:0/3836882944 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e4010d20 con 0x7fd8f0102810 2024-04-26T18:28:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.869+0000 7fd8f66dc700 1 -- 172.21.15.84:0/3836882944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8f01a6670 con 0x7fd8f0102810 2024-04-26T18:28:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.871+0000 7fd8ed7fa700 1 -- 172.21.15.84:0/3836882944 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7fd8e4018020 con 0x7fd8f0102810 2024-04-26T18:28:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.872+0000 7fd8ed7fa700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd8d80a3f60 0x7fd8d80a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.872+0000 7fd8effff700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd8d80a3f60 0x7fd8d80a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.872+0000 7fd8ed7fa700 1 -- 172.21.15.84:0/3836882944 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fd8e40d01f0 con 0x7fd8f0102810 2024-04-26T18:28:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.872+0000 7fd8effff700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd8d80a3f60 0x7fd8d80a6420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009750 tx=0x7fd8e000b540 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:31.880 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd900f6eb80 1 Processor -- start 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd900f6eb80 1 -- start start 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd900f6eb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c8cc0 0x5621e34c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd900f6eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c95e0 0x5621e34c3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd900f6eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c3780 0x5621e34c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd900f6eb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5621e34117e0 con 0x5621e34c8cc0 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd900f6eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5621e33778e0 con 0x5621e34c95e0 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd900f6eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5621e340d7a0 con 0x5621e34c3780 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd8effff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c8cc0 0x5621e34c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd8effff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c8cc0 0x5621e34c90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44000/0 (socket says 172.21.15.84:44000) 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd8effff700 1 -- 172.21.15.84:0/1203612519 learned_addr learned my addr 172.21.15.84:0/1203612519 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd8ef7fe700 1 --2- 172.21.15.84:0/1203612519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c95e0 0x5621e34c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd8f4c79700 1 --2- 172.21.15.84:0/1203612519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c3780 0x5621e34c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd8effff700 1 -- 172.21.15.84:0/1203612519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c3780 msgr2=0x5621e34c3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd8effff700 1 --2- 172.21.15.84:0/1203612519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c3780 0x5621e34c3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd8effff700 1 -- 172.21.15.84:0/1203612519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c95e0 msgr2=0x5621e34c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd8effff700 1 --2- 172.21.15.84:0/1203612519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c95e0 0x5621e34c3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.873+0000 7fd8effff700 1 -- 172.21.15.84:0/1203612519 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e335e880 con 0x5621e34c8cc0 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.874+0000 7fd8effff700 1 --2- 172.21.15.84:0/1203612519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c8cc0 0x5621e34c90a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0015000 tx=0x7fd8e001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5377dbb6f615841d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.874+0000 7fd8d67fc700 1 -- 172.21.15.84:0/1203612519 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e0003920 con 0x5621e34c8cc0 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.874+0000 7fd8d67fc700 1 -- 172.21.15.84:0/1203612519 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e0005030 con 0x5621e34c8cc0 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.874+0000 7fd8d67fc700 1 -- 172.21.15.84:0/1203612519 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e002f6c0 con 0x5621e34c8cc0 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.874+0000 7fd900f6eb80 1 -- 172.21.15.84:0/1203612519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c8cc0 msgr2=0x5621e34c90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.874+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/1203612519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c8cc0 0x5621e34c90a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0015000 tx=0x7fd8e001f000 comp rx=0 tx=0).stop 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.874+0000 7fd900f6eb80 1 -- 172.21.15.84:0/1203612519 shutdown_connections 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.874+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/1203612519 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c8cc0 0x5621e34c90a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.874+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/1203612519 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c3780 0x5621e34c3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.874+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/1203612519 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c95e0 0x5621e34c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.874+0000 7fd900f6eb80 1 -- 172.21.15.84:0/1203612519 >> 172.21.15.84:0/1203612519 conn(0x5621e33ed570 msgr2=0x5621e34b9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.875+0000 7fd900f6eb80 1 -- 172.21.15.84:0/1203612519 shutdown_connections 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.875+0000 7fd900f6eb80 1 -- 172.21.15.84:0/1203612519 wait complete. 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.875+0000 7fd900f6eb80 1 -- start start 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.875+0000 7fd900f6eb80 1 Processor -- start 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.875+0000 7fd900f6eb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 0x5621e34cf390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.875+0000 7fd900f6eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c8cc0 0x5621e34cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.875+0000 7fd900f6eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c95e0 0x5621e34ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.875+0000 7fd900f6eb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5621e34d0060 con 0x5621e34c3780 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.875+0000 7fd900f6eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5621e33cba30 con 0x5621e34c95e0 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.875+0000 7fd900f6eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5621e34d01e0 con 0x5621e34c8cc0 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.875+0000 7fd900f6eb80 10 client.0 _ll_register_callbacks cb 0x5621e33a9900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.875+0000 7fd8effff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 0x5621e34cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8effff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 0x5621e34cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44002/0 (socket says 172.21.15.84:44002) 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8effff700 1 -- 172.21.15.84:0/3138257944 learned_addr learned my addr 172.21.15.84:0/3138257944 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:31.883 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8ef7fe700 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c8cc0 0x5621e34cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8f4c79700 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c95e0 0x5621e34ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8effff700 1 -- 172.21.15.84:0/3138257944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c8cc0 msgr2=0x5621e34cc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8effff700 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c8cc0 0x5621e34cc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8effff700 1 -- 172.21.15.84:0/3138257944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c95e0 msgr2=0x5621e34ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8effff700 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c95e0 0x5621e34ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8effff700 1 -- 172.21.15.84:0/3138257944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e33cf430 con 0x5621e34c3780 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8ef7fe700 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c8cc0 0x5621e34cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8effff700 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 0x5621e34cf390 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00098e0 tx=0x7fd8e0004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5be2d9ada8f4fc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8cf7fe700 1 -- 172.21.15.84:0/3138257944 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e003c060 con 0x5621e34c3780 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8cf7fe700 1 -- 172.21.15.84:0/3138257944 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fd8e002f670 con 0x5621e34c3780 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd900f6eb80 1 -- 172.21.15.84:0/3138257944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8e0005030 con 0x5621e34c3780 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd8cf7fe700 1 -- 172.21.15.84:0/3138257944 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8e0004e80 con 0x5621e34c3780 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd900f6eb80 10 client.15471 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:31.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.876+0000 7fd900f6eb80 1 -- 172.21.15.84:0/3138257944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5621e34da960 con 0x5621e34c3780 2024-04-26T18:28:31.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.877+0000 7fd8cf7fe700 1 -- 172.21.15.84:0/3138257944 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e0005030 con 0x5621e34c3780 2024-04-26T18:28:31.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.877+0000 7fd8cf7fe700 1 -- 172.21.15.84:0/3138257944 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd8e000f830 con 0x5621e34c3780 2024-04-26T18:28:31.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.877+0000 7fd900f6eb80 10 client.15471 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:31.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.877+0000 7fd900f6eb80 10 client.15471 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T18:28:31.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.877+0000 7fd900f6eb80 20 client.15471 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:31.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.877+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x5621e34dbcb0 0x5621e34ddd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.877+0000 7fd900f6eb80 4 client.15471 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:31.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.877+0000 7fd900f6eb80 1 -- 172.21.15.84:0/3138257944 --> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621e341f1c0 con 0x5621e34dbcb0 2024-04-26T18:28:31.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.877+0000 7fd8ef7fe700 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x5621e34dbcb0 0x5621e34ddd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.878+0000 7fd8ef7fe700 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x5621e34dbcb0 0x5621e34ddd90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a8afb9b2aef6575 server_cookie=50a694c80e4ab323 in_seq=0 out_seq=0 2024-04-26T18:28:31.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.878+0000 7fd8cf7fe700 10 client.15471 ms_handle_connect on v2:172.21.15.103:6836/2271231481 2024-04-26T18:28:31.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.878+0000 7fd8cf7fe700 1 -- 172.21.15.84:0/3138257944 <== mds.2 v2:172.21.15.103:6836/2271231481 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621e341f1c0 con 0x5621e34dbcb0 2024-04-26T18:28:31.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.878+0000 7fd8cf7fe700 10 client.15471 handle_command_reply: tid=0 2024-04-26T18:28:31.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.878+0000 7fd900f6eb80 1 client.15471 shutdown 2024-04-26T18:28:31.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.878+0000 7fd900f6eb80 1 -- 172.21.15.84:0/3138257944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 msgr2=0x5621e34cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:31.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.878+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 0x5621e34cf390 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd8e00098e0 tx=0x7fd8e0004be0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.879+0000 7fd900f6eb80 1 -- 172.21.15.84:0/3138257944 shutdown_connections 2024-04-26T18:28:31.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.879+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x5621e34dbcb0 0x5621e34ddd90 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.879+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 0x5621e34cf390 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.879+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c95e0 0x5621e34ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.879+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/3138257944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c8cc0 0x5621e34cc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.879+0000 7fd900f6eb80 1 -- 172.21.15.84:0/3138257944 >> 172.21.15.84:0/3138257944 conn(0x5621e33ed570 msgr2=0x5621e34bdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:31.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.879+0000 7fd8cf7fe700 0 client.15471 ms_handle_reset on v2:172.21.15.103:6836/2271231481 2024-04-26T18:28:31.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.879+0000 7fd900f6eb80 1 -- 172.21.15.84:0/3138257944 shutdown_connections 2024-04-26T18:28:31.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.879+0000 7fd900f6eb80 1 -- 172.21.15.84:0/3138257944 wait complete. 2024-04-26T18:28:31.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.879+0000 7fd900f6eb80 20 client.15471 trim_cache size 0 max 16384 2024-04-26T18:28:31.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd900f6eb80 1 Processor -- start 2024-04-26T18:28:31.899 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd900f6eb80 1 -- start start 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd900f6eb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 0x5621e34dd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd900f6eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c8cc0 0x5621e34b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd900f6eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c95e0 0x5621e34cb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd900f6eb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5621e34dc0d0 con 0x5621e34c3780 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd900f6eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5621e34dc7a0 con 0x5621e34c8cc0 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd900f6eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5621e34dce70 con 0x5621e34c95e0 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd8effff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 0x5621e34dd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd8effff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 0x5621e34dd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44010/0 (socket says 172.21.15.84:44010) 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd8effff700 1 -- 172.21.15.84:0/3426490559 learned_addr learned my addr 172.21.15.84:0/3426490559 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd8effff700 1 -- 172.21.15.84:0/3426490559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c95e0 msgr2=0x5621e34cb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd8effff700 1 --2- 172.21.15.84:0/3426490559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c95e0 0x5621e34cb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd8effff700 1 -- 172.21.15.84:0/3426490559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c8cc0 msgr2=0x5621e34b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd8effff700 1 --2- 172.21.15.84:0/3426490559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c8cc0 0x5621e34b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.900 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd8effff700 1 -- 172.21.15.84:0/3426490559 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621e34dbf00 con 0x5621e34c3780 2024-04-26T18:28:31.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd8effff700 1 --2- 172.21.15.84:0/3426490559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 0x5621e34dd6c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0000c00 tx=0x7fd8e000b850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=994221282417b7f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:31.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.899+0000 7fd8d5ffb700 1 -- 172.21.15.84:0/3426490559 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e000be60 con 0x5621e34c3780 2024-04-26T18:28:31.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.900+0000 7fd8d5ffb700 1 -- 172.21.15.84:0/3426490559 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e00055e0 con 0x5621e34c3780 2024-04-26T18:28:31.901 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.900+0000 7fd8d5ffb700 1 -- 172.21.15.84:0/3426490559 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e002f8d0 con 0x5621e34c3780 2024-04-26T18:28:31.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.900+0000 7fd900f6eb80 1 -- 172.21.15.84:0/3426490559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 msgr2=0x5621e34dd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:31.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.900+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/3426490559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 0x5621e34dd6c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0000c00 tx=0x7fd8e000b850 comp rx=0 tx=0).stop 2024-04-26T18:28:31.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.900+0000 7fd900f6eb80 1 -- 172.21.15.84:0/3426490559 shutdown_connections 2024-04-26T18:28:31.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.900+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/3426490559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c3780 0x5621e34dd6c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.900+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/3426490559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c95e0 0x5621e34cb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.900+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/3426490559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c8cc0 0x5621e34b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.900+0000 7fd900f6eb80 1 -- 172.21.15.84:0/3426490559 >> 172.21.15.84:0/3426490559 conn(0x5621e33ed570 msgr2=0x5621e34c3b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.900+0000 7fd900f6eb80 1 -- 172.21.15.84:0/3426490559 shutdown_connections 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.900+0000 7fd900f6eb80 1 -- 172.21.15.84:0/3426490559 wait complete. 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd900f6eb80 1 -- start start 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd900f6eb80 1 Processor -- start 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd900f6eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c3780 0x5621e34cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd900f6eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c8cc0 0x5621e34cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd900f6eb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c95e0 0x5621e34cca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd900f6eb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5621e34ddc10 con 0x5621e34c95e0 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd900f6eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5621e34dd540 con 0x5621e34c3780 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd900f6eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5621e34de2e0 con 0x5621e34c8cc0 2024-04-26T18:28:31.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd900f6eb80 10 client.0 _ll_register_callbacks cb 0x5621e33a9900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd8f4c79700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c95e0 0x5621e34cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd8f4c79700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c95e0 0x5621e34cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44026/0 (socket says 172.21.15.84:44026) 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd8f4c79700 1 -- 172.21.15.84:0/1299325942 learned_addr learned my addr 172.21.15.84:0/1299325942 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd8effff700 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c3780 0x5621e34cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.901+0000 7fd8ef7fe700 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c8cc0 0x5621e34cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd8f4c79700 1 -- 172.21.15.84:0/1299325942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c8cc0 msgr2=0x5621e34cc4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd8f4c79700 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c8cc0 0x5621e34cc4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd8f4c79700 1 -- 172.21.15.84:0/1299325942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c3780 msgr2=0x5621e34cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd8f4c79700 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c3780 0x5621e34cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd8f4c79700 1 -- 172.21.15.84:0/1299325942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621e34e5950 con 0x5621e34c95e0 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd8ef7fe700 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c8cc0 0x5621e34cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd8f4c79700 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c95e0 0x5621e34cca70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8004b10 tx=0x7fd8e8012530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8906da2c1ddeb38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd8cf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:31.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd8cf7fe700 1 -- 172.21.15.84:0/1299325942 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8e801a070 con 0x5621e34c95e0 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd8cf7fe700 1 -- 172.21.15.84:0/1299325942 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fd8e8012cd0 con 0x5621e34c95e0 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd900f6eb80 1 -- 172.21.15.84:0/1299325942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8e00055e0 con 0x5621e34c95e0 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd8cf7fe700 1 -- 172.21.15.84:0/1299325942 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8e8026050 con 0x5621e34c95e0 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd900f6eb80 10 client.15477 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd900f6eb80 1 -- 172.21.15.84:0/1299325942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5621e35440b0 con 0x5621e34c95e0 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd8cf7fe700 1 -- 172.21.15.84:0/1299325942 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8e00055e0 con 0x5621e34c95e0 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd8cf7fe700 1 -- 172.21.15.84:0/1299325942 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd8e80153d0 con 0x5621e34c95e0 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd900f6eb80 10 client.15477 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd900f6eb80 10 client.15477 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.902+0000 7fd900f6eb80 20 client.15477 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.903+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x5621e3548cd0 0x5621e35479e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.903+0000 7fd900f6eb80 4 client.15477 mds_command: new command op to 24422 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.903+0000 7fd900f6eb80 1 -- 172.21.15.84:0/1299325942 --> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] -- command(tid 0: {"prefix": "status"}) -- 0x5621e3419260 con 0x5621e3548cd0 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.903+0000 7fd8effff700 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x5621e3548cd0 0x5621e35479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:31.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.903+0000 7fd8effff700 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x5621e3548cd0 0x5621e35479e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e4d886ed73a64607 server_cookie=9695c6ffac22ec02 in_seq=0 out_seq=0 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.903+0000 7fd8cf7fe700 10 client.15477 ms_handle_connect on v2:172.21.15.103:6836/2271231481 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.903+0000 7fd8cf7fe700 1 -- 172.21.15.84:0/1299325942 <== mds.2 v2:172.21.15.103:6836/2271231481 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5621e3419260 con 0x5621e3548cd0 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.903+0000 7fd8cf7fe700 10 client.15477 handle_command_reply: tid=0 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.903+0000 7fd900f6eb80 1 client.15477 shutdown 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd900f6eb80 1 -- 172.21.15.84:0/1299325942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c95e0 msgr2=0x5621e34cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c95e0 0x5621e34cca70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fd8e8004b10 tx=0x7fd8e8012530 comp rx=0 tx=0).stop 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd900f6eb80 1 -- 172.21.15.84:0/1299325942 shutdown_connections 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.103:6836/2271231481,v1:172.21.15.103:6837/2271231481] conn(0x5621e3548cd0 0x5621e35479e0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5621e34c95e0 0x5621e34cca70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5621e34c3780 0x5621e34cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd900f6eb80 1 --2- 172.21.15.84:0/1299325942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5621e34c8cc0 0x5621e34cc4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd8cf7fe700 0 client.15477 ms_handle_reset on v2:172.21.15.103:6836/2271231481 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd900f6eb80 1 -- 172.21.15.84:0/1299325942 >> 172.21.15.84:0/1299325942 conn(0x5621e33ed570 msgr2=0x5621e34cac10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:31.908 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd900f6eb80 1 -- 172.21.15.84:0/1299325942 shutdown_connections 2024-04-26T18:28:31.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd900f6eb80 1 -- 172.21.15.84:0/1299325942 wait complete. 2024-04-26T18:28:31.909 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd900f6eb80 20 client.15477 trim_cache size 0 max 16384 2024-04-26T18:28:31.909 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:28:31.909 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:28:31.909 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 2, 2024-04-26T18:28:31.909 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24422, 2024-04-26T18:28:31.909 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:28:31.909 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:28:31.909 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:28:31.909 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 235.630324102, 2024-04-26T18:28:31.909 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 29, 2024-04-26T18:28:31.910 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 94, 2024-04-26T18:28:31.910 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 93, 2024-04-26T18:28:31.910 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 275.81001805800003 2024-04-26T18:28:31.910 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:28:31.910 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd8f66dc700 1 -- 172.21.15.84:0/3836882944 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd8d80a3f60 msgr2=0x7fd8d80a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd8f66dc700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd8d80a3f60 0x7fd8d80a6420 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd8e0009750 tx=0x7fd8e000b540 comp rx=0 tx=0).stop 2024-04-26T18:28:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd8f66dc700 1 -- 172.21.15.84:0/3836882944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f0102810 msgr2=0x7fd8f019fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd8f66dc700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f0102810 0x7fd8f019fce0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400b530 tx=0x7fd8e400b8f0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd8f66dc700 1 -- 172.21.15.84:0/3836882944 shutdown_connections 2024-04-26T18:28:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd8f66dc700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd8d80a3f60 0x7fd8d80a6420 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd8f66dc700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8f0109600 0x7fd8f0199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd8f66dc700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8f0101e50 0x7fd8f019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd8f66dc700 1 --2- 172.21.15.84:0/3836882944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8f0102810 0x7fd8f019fce0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd8f66dc700 1 -- 172.21.15.84:0/3836882944 >> 172.21.15.84:0/3836882944 conn(0x7fd8f0077590 msgr2=0x7fd8f0100aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd8f66dc700 1 -- 172.21.15.84:0/3836882944 shutdown_connections 2024-04-26T18:28:31.911 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:31.904+0000 7fd8f66dc700 1 -- 172.21.15.84:0/3836882944 wait complete. 2024-04-26T18:28:32.735 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.i status 2024-04-26T18:28:32.975 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:33.361 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:33 smithi084 ceph-mon[29141]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-04-26T18:28:33.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:33 smithi103 ceph-mon[32727]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-04-26T18:28:33.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:33 smithi174 ceph-mon[32705]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-04-26T18:28:34.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.679+0000 7f02fb1d1700 1 -- 172.21.15.84:0/3990222214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02f4102f60 msgr2=0x7f02f4106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:34.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.679+0000 7f02fb1d1700 1 --2- 172.21.15.84:0/3990222214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02f4102f60 0x7f02f4106f60 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f02e00099c0 tx=0x7f02e0009cd0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.679+0000 7f02fb1d1700 1 -- 172.21.15.84:0/3990222214 shutdown_connections 2024-04-26T18:28:34.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.679+0000 7f02fb1d1700 1 --2- 172.21.15.84:0/3990222214 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02f4107640 0x7f02f410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.679+0000 7f02fb1d1700 1 --2- 172.21.15.84:0/3990222214 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02f41025b0 0x7f02f4102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.689 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.679+0000 7f02fb1d1700 1 --2- 172.21.15.84:0/3990222214 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02f4102f60 0x7f02f4106f60 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.679+0000 7f02fb1d1700 1 -- 172.21.15.84:0/3990222214 >> 172.21.15.84:0/3990222214 conn(0x7f02f40792d0 msgr2=0x7f02f40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.679+0000 7f02fb1d1700 1 -- 172.21.15.84:0/3990222214 shutdown_connections 2024-04-26T18:28:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.679+0000 7f02fb1d1700 1 -- 172.21.15.84:0/3990222214 wait complete. 2024-04-26T18:28:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02fb1d1700 1 Processor -- start 2024-04-26T18:28:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02fb1d1700 1 -- start start 2024-04-26T18:28:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02fb1d1700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02f41025b0 0x7f02f4104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02fb1d1700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02f4102f60 0x7f02f4104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02fb1d1700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02f4107640 0x7f02f4104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02fb1d1700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f02f41124d0 con 0x7f02f41025b0 2024-04-26T18:28:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02fb1d1700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4112350 con 0x7f02f4107640 2024-04-26T18:28:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02fb1d1700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f02f4112650 con 0x7f02f4102f60 2024-04-26T18:28:34.690 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02f8f6d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02f41025b0 0x7f02f4104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02f8f6d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02f41025b0 0x7f02f4104000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44052/0 (socket says 172.21.15.84:44052) 2024-04-26T18:28:34.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02f8f6d700 1 -- 172.21.15.84:0/751644121 learned_addr learned my addr 172.21.15.84:0/751644121 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:34.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02f976e700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02f4107640 0x7f02f4104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02f3fff700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02f4102f60 0x7f02f4104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02f8f6d700 1 -- 172.21.15.84:0/751644121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02f4102f60 msgr2=0x7f02f4104540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:34.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02f8f6d700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02f4102f60 0x7f02f4104540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02f8f6d700 1 -- 172.21.15.84:0/751644121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02f4107640 msgr2=0x7f02f4104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:34.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02f8f6d700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02f4107640 0x7f02f4104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.680+0000 7f02f8f6d700 1 -- 172.21.15.84:0/751644121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02f41a80a0 con 0x7f02f41025b0 2024-04-26T18:28:34.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.681+0000 7f02f3fff700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02f4102f60 0x7f02f4104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:34.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.681+0000 7f02f8f6d700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02f41025b0 0x7f02f4104000 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f02e800d950 tx=0x7f02e800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:34.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.681+0000 7f02f1ffb700 1 -- 172.21.15.84:0/751644121 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8016070 con 0x7f02f41025b0 2024-04-26T18:28:34.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.681+0000 7f02f1ffb700 1 -- 172.21.15.84:0/751644121 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e8010460 con 0x7f02f41025b0 2024-04-26T18:28:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.681+0000 7f02fb1d1700 1 -- 172.21.15.84:0/751644121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02f41a8390 con 0x7f02f41025b0 2024-04-26T18:28:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.681+0000 7f02f1ffb700 1 -- 172.21.15.84:0/751644121 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e800f620 con 0x7f02f41025b0 2024-04-26T18:28:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.681+0000 7f02fb1d1700 1 -- 172.21.15.84:0/751644121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02f41a88d0 con 0x7f02f41025b0 2024-04-26T18:28:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.683+0000 7f02f1ffb700 1 -- 172.21.15.84:0/751644121 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f02e8019070 con 0x7f02f41025b0 2024-04-26T18:28:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.683+0000 7f02f1ffb700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f02e40a3f60 0x7f02e40a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.684+0000 7f02f3fff700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f02e40a3f60 0x7f02e40a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.684+0000 7f02f1ffb700 1 -- 172.21.15.84:0/751644121 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f02e80d0770 con 0x7f02f41025b0 2024-04-26T18:28:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.684+0000 7f02f3fff700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f02e40a3f60 0x7f02e40a6420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f02e0000c00 tx=0x7f02e0005e80 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.684+0000 7f0305a63b80 1 Processor -- start 2024-04-26T18:28:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.684+0000 7f0305a63b80 1 -- start start 2024-04-26T18:28:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f0305a63b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa30fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f0305a63b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30fc3f0 0x558fa30f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f0305a63b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa30f6710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f0305a63b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558fa30486f0 con 0x558fa30f62d0 2024-04-26T18:28:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f0305a63b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fa2fe7590 con 0x558fa30fc3f0 2024-04-26T18:28:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f0305a63b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558fa30446b0 con 0x558fa30fbad0 2024-04-26T18:28:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02f976e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa30f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02f976e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa30f6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44056/0 (socket says 172.21.15.84:44056) 2024-04-26T18:28:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02f976e700 1 -- 172.21.15.84:0/2168343380 learned_addr learned my addr 172.21.15.84:0/2168343380 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02f8f6d700 1 --2- 172.21.15.84:0/2168343380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa30fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02f3fff700 1 --2- 172.21.15.84:0/2168343380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30fc3f0 0x558fa30f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02f976e700 1 -- 172.21.15.84:0/2168343380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 msgr2=0x558fa30fbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02f976e700 1 --2- 172.21.15.84:0/2168343380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa30fbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02f976e700 1 -- 172.21.15.84:0/2168343380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30fc3f0 msgr2=0x558fa30f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02f976e700 1 --2- 172.21.15.84:0/2168343380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30fc3f0 0x558fa30f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02f976e700 1 -- 172.21.15.84:0/2168343380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa2f95880 con 0x558fa30f62d0 2024-04-26T18:28:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02f976e700 1 --2- 172.21.15.84:0/2168343380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa30f6710 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009440 tx=0x7f02ec009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af640947e3ad47e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02deffd700 1 -- 172.21.15.84:0/2168343380 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec013750 con 0x558fa30f62d0 2024-04-26T18:28:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02deffd700 1 -- 172.21.15.84:0/2168343380 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02ec013da0 con 0x558fa30f62d0 2024-04-26T18:28:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.685+0000 7f02deffd700 1 -- 172.21.15.84:0/2168343380 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02ec015e30 con 0x558fa30f62d0 2024-04-26T18:28:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.686+0000 7f0305a63b80 1 -- 172.21.15.84:0/2168343380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 msgr2=0x558fa30f6710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.686+0000 7f0305a63b80 1 --2- 172.21.15.84:0/2168343380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa30f6710 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f02ec009440 tx=0x7f02ec009920 comp rx=0 tx=0).stop 2024-04-26T18:28:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.686+0000 7f0305a63b80 1 -- 172.21.15.84:0/2168343380 shutdown_connections 2024-04-26T18:28:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.686+0000 7f0305a63b80 1 --2- 172.21.15.84:0/2168343380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa30f6710 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.686+0000 7f0305a63b80 1 --2- 172.21.15.84:0/2168343380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa30fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.686+0000 7f0305a63b80 1 --2- 172.21.15.84:0/2168343380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30fc3f0 0x558fa30f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.686+0000 7f0305a63b80 1 -- 172.21.15.84:0/2168343380 >> 172.21.15.84:0/2168343380 conn(0x558fa3022cf0 msgr2=0x558fa30f11b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.686+0000 7f0305a63b80 1 -- 172.21.15.84:0/2168343380 shutdown_connections 2024-04-26T18:28:34.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.686+0000 7f0305a63b80 1 -- 172.21.15.84:0/2168343380 wait complete. 2024-04-26T18:28:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.686+0000 7f0305a63b80 1 -- start start 2024-04-26T18:28:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.686+0000 7f0305a63b80 1 Processor -- start 2024-04-26T18:28:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f0305a63b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa3102120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f0305a63b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa30ff420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f0305a63b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30fc3f0 0x558fa30ff960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f0305a63b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558fa3102f10 con 0x558fa30f62d0 2024-04-26T18:28:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f0305a63b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fa2fae900 con 0x558fa30fc3f0 2024-04-26T18:28:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f0305a63b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558fa3103090 con 0x558fa30fbad0 2024-04-26T18:28:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f0305a63b80 10 client.0 _ll_register_callbacks cb 0x558fa2fdc530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f02f8f6d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa3102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f02f8f6d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa3102120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44066/0 (socket says 172.21.15.84:44066) 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f02f8f6d700 1 -- 172.21.15.84:0/3786350244 learned_addr learned my addr 172.21.15.84:0/3786350244 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f02f3fff700 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa30ff420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f02f976e700 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30fc3f0 0x558fa30ff960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f02f8f6d700 1 -- 172.21.15.84:0/3786350244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 msgr2=0x558fa30ff420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f02f8f6d700 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa30ff420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f02f8f6d700 1 -- 172.21.15.84:0/3786350244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30fc3f0 msgr2=0x558fa30ff960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f02f8f6d700 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30fc3f0 0x558fa30ff960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f02f8f6d700 1 -- 172.21.15.84:0/3786350244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa30fff70 con 0x558fa30f62d0 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f02f976e700 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30fc3f0 0x558fa30ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f02f8f6d700 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa3102120 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f02e8000c00 tx=0x7f02e809ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6acec95030cbd6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.687+0000 7f02d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f02d3fff700 1 -- 172.21.15.84:0/3786350244 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e80b5060 con 0x558fa30f62d0 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f02d3fff700 1 -- 172.21.15.84:0/3786350244 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f02e809c920 con 0x558fa30f62d0 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f0305a63b80 1 -- 172.21.15.84:0/3786350244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02ec013da0 con 0x558fa30f62d0 2024-04-26T18:28:34.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f02d3fff700 1 -- 172.21.15.84:0/3786350244 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02e8020040 con 0x558fa30f62d0 2024-04-26T18:28:34.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f0305a63b80 10 client.15489 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:34.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f0305a63b80 1 -- 172.21.15.84:0/3786350244 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558fa310d920 con 0x558fa30f62d0 2024-04-26T18:28:34.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f02d3fff700 1 -- 172.21.15.84:0/3786350244 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02ec013da0 con 0x558fa30f62d0 2024-04-26T18:28:34.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f02d3fff700 1 -- 172.21.15.84:0/3786350244 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f02e809b910 con 0x558fa30f62d0 2024-04-26T18:28:34.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f0305a63b80 10 client.15489 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:34.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f0305a63b80 10 client.15489 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:28:34.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f0305a63b80 20 client.15489 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:34.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f0305a63b80 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x558fa310fe80 0x558fa31112f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f0305a63b80 4 client.15489 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:34.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.688+0000 7f0305a63b80 1 -- 172.21.15.84:0/3786350244 --> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fa30560d0 con 0x558fa310fe80 2024-04-26T18:28:34.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.689+0000 7f02f3fff700 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x558fa310fe80 0x558fa31112f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.689+0000 7f02d3fff700 10 client.15489 ms_handle_connect on v2:172.21.15.174:6838/518809558 2024-04-26T18:28:34.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.689+0000 7f02f3fff700 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x558fa310fe80 0x558fa31112f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4a39c5fc05048194 server_cookie=2de0cc484b84d8d1 in_seq=0 out_seq=0 2024-04-26T18:28:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f02d3fff700 1 -- 172.21.15.84:0/3786350244 <== mds.1 v2:172.21.15.174:6838/518809558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558fa30560d0 con 0x558fa310fe80 2024-04-26T18:28:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f02d3fff700 10 client.15489 handle_command_reply: tid=0 2024-04-26T18:28:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f0305a63b80 1 client.15489 shutdown 2024-04-26T18:28:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f0305a63b80 1 -- 172.21.15.84:0/3786350244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 msgr2=0x558fa3102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f0305a63b80 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa3102120 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f02e8000c00 tx=0x7f02e809ccf0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f0305a63b80 1 -- 172.21.15.84:0/3786350244 shutdown_connections 2024-04-26T18:28:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f0305a63b80 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x558fa310fe80 0x558fa31112f0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f0305a63b80 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa3102120 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f0305a63b80 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30fc3f0 0x558fa30ff960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f0305a63b80 1 --2- 172.21.15.84:0/3786350244 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa30ff420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f0305a63b80 1 -- 172.21.15.84:0/3786350244 >> 172.21.15.84:0/3786350244 conn(0x558fa3022cf0 msgr2=0x558fa30f0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f02d3fff700 0 client.15489 ms_handle_reset on v2:172.21.15.174:6838/518809558 2024-04-26T18:28:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f0305a63b80 1 -- 172.21.15.84:0/3786350244 shutdown_connections 2024-04-26T18:28:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f0305a63b80 1 -- 172.21.15.84:0/3786350244 wait complete. 2024-04-26T18:28:34.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.690+0000 7f0305a63b80 20 client.15489 trim_cache size 0 max 16384 2024-04-26T18:28:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.710+0000 7f0305a63b80 1 Processor -- start 2024-04-26T18:28:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.710+0000 7f0305a63b80 1 -- start start 2024-04-26T18:28:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.710+0000 7f0305a63b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa3022cf0 0x558fa30f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.710+0000 7f0305a63b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa30ecb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.710+0000 7f0305a63b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa3101c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.710+0000 7f0305a63b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558fa310f630 con 0x558fa30f62d0 2024-04-26T18:28:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.710+0000 7f0305a63b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fa310fd00 con 0x558fa3022cf0 2024-04-26T18:28:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.710+0000 7f0305a63b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558fa31103d0 con 0x558fa30fbad0 2024-04-26T18:28:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02f3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa30ecb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02f3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa30ecb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44072/0 (socket says 172.21.15.84:44072) 2024-04-26T18:28:34.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02f3fff700 1 -- 172.21.15.84:0/1560754276 learned_addr learned my addr 172.21.15.84:0/1560754276 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02f8f6d700 1 --2- 172.21.15.84:0/1560754276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa3022cf0 0x558fa30f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02f976e700 1 --2- 172.21.15.84:0/1560754276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa3101c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02f3fff700 1 -- 172.21.15.84:0/1560754276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 msgr2=0x558fa3101c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02f3fff700 1 --2- 172.21.15.84:0/1560754276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa3101c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02f3fff700 1 -- 172.21.15.84:0/1560754276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa3022cf0 msgr2=0x558fa30f5cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02f3fff700 1 --2- 172.21.15.84:0/1560754276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa3022cf0 0x558fa30f5cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02f3fff700 1 -- 172.21.15.84:0/1560754276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fa30fd9f0 con 0x558fa30f62d0 2024-04-26T18:28:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02f8f6d700 1 --2- 172.21.15.84:0/1560754276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa3022cf0 0x558fa30f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02f976e700 1 --2- 172.21.15.84:0/1560754276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa3101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02f3fff700 1 --2- 172.21.15.84:0/1560754276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa30ecb70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f02e000a010 tx=0x7f02e0021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf19751d4456414 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02de7fc700 1 -- 172.21.15.84:0/1560754276 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0003c20 con 0x558fa30f62d0 2024-04-26T18:28:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02de7fc700 1 -- 172.21.15.84:0/1560754276 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e00328f0 con 0x558fa30f62d0 2024-04-26T18:28:34.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.711+0000 7f02de7fc700 1 -- 172.21.15.84:0/1560754276 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e0025420 con 0x558fa30f62d0 2024-04-26T18:28:34.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.712+0000 7f0305a63b80 1 -- 172.21.15.84:0/1560754276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 msgr2=0x558fa30ecb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:34.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.712+0000 7f0305a63b80 1 --2- 172.21.15.84:0/1560754276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa30ecb70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f02e000a010 tx=0x7f02e0021000 comp rx=0 tx=0).stop 2024-04-26T18:28:34.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.712+0000 7f0305a63b80 1 -- 172.21.15.84:0/1560754276 shutdown_connections 2024-04-26T18:28:34.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.712+0000 7f0305a63b80 1 --2- 172.21.15.84:0/1560754276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa3101c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.712+0000 7f0305a63b80 1 --2- 172.21.15.84:0/1560754276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa3022cf0 0x558fa30f5cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.712+0000 7f0305a63b80 1 --2- 172.21.15.84:0/1560754276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa30f62d0 0x558fa30ecb70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.712+0000 7f0305a63b80 1 -- 172.21.15.84:0/1560754276 >> 172.21.15.84:0/1560754276 conn(0x558fa30f6a20 msgr2=0x558fa30fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.712+0000 7f0305a63b80 1 -- 172.21.15.84:0/1560754276 shutdown_connections 2024-04-26T18:28:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.712+0000 7f0305a63b80 1 -- 172.21.15.84:0/1560754276 wait complete. 2024-04-26T18:28:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.713+0000 7f0305a63b80 1 -- start start 2024-04-26T18:28:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.713+0000 7f0305a63b80 1 Processor -- start 2024-04-26T18:28:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.713+0000 7f0305a63b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa3022cf0 0x558fa31017d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.713+0000 7f0305a63b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30f62d0 0x558fa3101d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.713+0000 7f0305a63b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa3118e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.713+0000 7f0305a63b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558fa3111170 con 0x558fa3022cf0 2024-04-26T18:28:34.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.713+0000 7f0305a63b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558fa3110aa0 con 0x558fa30f62d0 2024-04-26T18:28:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f0305a63b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558fa3111840 con 0x558fa30fbad0 2024-04-26T18:28:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02f8f6d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa3022cf0 0x558fa31017d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f0305a63b80 10 client.0 _ll_register_callbacks cb 0x558fa2fdc530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02f8f6d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa3022cf0 0x558fa31017d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44074/0 (socket says 172.21.15.84:44074) 2024-04-26T18:28:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02f8f6d700 1 -- 172.21.15.84:0/2781031183 learned_addr learned my addr 172.21.15.84:0/2781031183 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02f976e700 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa3118e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02f3fff700 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30f62d0 0x558fa3101d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02f8f6d700 1 -- 172.21.15.84:0/2781031183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 msgr2=0x558fa3118e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02f8f6d700 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa3118e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02f8f6d700 1 -- 172.21.15.84:0/2781031183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30f62d0 msgr2=0x558fa3101d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:34.719 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02f8f6d700 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30f62d0 0x558fa3101d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02f8f6d700 1 -- 172.21.15.84:0/2781031183 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fa31194b0 con 0x558fa3022cf0 2024-04-26T18:28:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02f3fff700 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30f62d0 0x558fa3101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02f8f6d700 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa3022cf0 0x558fa31017d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f02e8098e90 tx=0x7f02e8009650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1186f3841f9f43c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02d3fff700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02d3fff700 1 -- 172.21.15.84:0/2781031183 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e80b5040 con 0x558fa3022cf0 2024-04-26T18:28:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02d3fff700 1 -- 172.21.15.84:0/2781031183 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f02e80a4ea0 con 0x558fa3022cf0 2024-04-26T18:28:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f0305a63b80 1 -- 172.21.15.84:0/2781031183 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e00328f0 con 0x558fa3022cf0 2024-04-26T18:28:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f02d3fff700 1 -- 172.21.15.84:0/2781031183 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02e809c720 con 0x558fa3022cf0 2024-04-26T18:28:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f0305a63b80 10 client.15495 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.714+0000 7f0305a63b80 1 -- 172.21.15.84:0/2781031183 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558fa3177830 con 0x558fa3022cf0 2024-04-26T18:28:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.715+0000 7f02d3fff700 1 -- 172.21.15.84:0/2781031183 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e00328f0 con 0x558fa3022cf0 2024-04-26T18:28:34.720 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.715+0000 7f02d3fff700 1 -- 172.21.15.84:0/2781031183 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f02e8098210 con 0x558fa3022cf0 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.715+0000 7f0305a63b80 10 client.15495 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.715+0000 7f0305a63b80 10 client.15495 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.715+0000 7f0305a63b80 20 client.15495 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.715+0000 7f0305a63b80 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x558fa30fc3f0 0x558fa317b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.715+0000 7f0305a63b80 4 client.15495 mds_command: new command op to 24430 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.715+0000 7f0305a63b80 1 -- 172.21.15.84:0/2781031183 --> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] -- command(tid 0: {"prefix": "status"}) -- 0x558fa3050170 con 0x558fa30fc3f0 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.715+0000 7f02f3fff700 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x558fa30fc3f0 0x558fa317b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f02f3fff700 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x558fa30fc3f0 0x558fa317b180 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7efe4993552989a5 server_cookie=28037adbd8e0824a in_seq=0 out_seq=0 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f02d3fff700 10 client.15495 ms_handle_connect on v2:172.21.15.174:6838/518809558 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f02d3fff700 1 -- 172.21.15.84:0/2781031183 <== mds.1 v2:172.21.15.174:6838/518809558 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558fa3050170 con 0x558fa30fc3f0 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f02d3fff700 10 client.15495 handle_command_reply: tid=0 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f0305a63b80 1 client.15495 shutdown 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f0305a63b80 1 -- 172.21.15.84:0/2781031183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa3022cf0 msgr2=0x558fa31017d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:34.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f0305a63b80 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa3022cf0 0x558fa31017d0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f02e8098e90 tx=0x7f02e8009650 comp rx=0 tx=0).stop 2024-04-26T18:28:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f0305a63b80 1 -- 172.21.15.84:0/2781031183 shutdown_connections 2024-04-26T18:28:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f0305a63b80 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.174:6838/518809558,v1:172.21.15.174:6839/518809558] conn(0x558fa30fc3f0 0x558fa317b180 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f0305a63b80 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558fa3022cf0 0x558fa31017d0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f02d3fff700 0 client.15495 ms_handle_reset on v2:172.21.15.174:6838/518809558 2024-04-26T18:28:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f0305a63b80 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558fa30fbad0 0x558fa3118e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f0305a63b80 1 --2- 172.21.15.84:0/2781031183 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558fa30f62d0 0x558fa3101d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f0305a63b80 1 -- 172.21.15.84:0/2781031183 >> 172.21.15.84:0/2781031183 conn(0x558fa30f6a20 msgr2=0x558fa30fc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f0305a63b80 1 -- 172.21.15.84:0/2781031183 shutdown_connections 2024-04-26T18:28:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.716+0000 7f0305a63b80 1 -- 172.21.15.84:0/2781031183 wait complete. 2024-04-26T18:28:34.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.717+0000 7f0305a63b80 20 client.15495 trim_cache size 0 max 16384 2024-04-26T18:28:34.723 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:28:34.723 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:28:34.723 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 1, 2024-04-26T18:28:34.723 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24430, 2024-04-26T18:28:34.723 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:28:34.723 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:28:34.723 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:28:34.723 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 239.454863163, 2024-04-26T18:28:34.723 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 29, 2024-04-26T18:28:34.723 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 94, 2024-04-26T18:28:34.723 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 93, 2024-04-26T18:28:34.723 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 275.85243719800002 2024-04-26T18:28:34.723 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:28:34.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.717+0000 7f02fb1d1700 1 -- 172.21.15.84:0/751644121 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f02e40a3f60 msgr2=0x7f02e40a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:34.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.717+0000 7f02fb1d1700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f02e40a3f60 0x7f02e40a6420 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f02e0000c00 tx=0x7f02e0005e80 comp rx=0 tx=0).stop 2024-04-26T18:28:34.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.717+0000 7f02fb1d1700 1 -- 172.21.15.84:0/751644121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02f41025b0 msgr2=0x7f02f4104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:34.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.717+0000 7f02fb1d1700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02f41025b0 0x7f02f4104000 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f02e800d950 tx=0x7f02e800dd10 comp rx=0 tx=0).stop 2024-04-26T18:28:34.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.717+0000 7f02fb1d1700 1 -- 172.21.15.84:0/751644121 shutdown_connections 2024-04-26T18:28:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.717+0000 7f02fb1d1700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f02e40a3f60 0x7f02e40a6420 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.717+0000 7f02fb1d1700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f02f41025b0 0x7f02f4104000 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.717+0000 7f02fb1d1700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f02f4107640 0x7f02f4104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.717+0000 7f02fb1d1700 1 --2- 172.21.15.84:0/751644121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f02f4102f60 0x7f02f4104540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.717+0000 7f02fb1d1700 1 -- 172.21.15.84:0/751644121 >> 172.21.15.84:0/751644121 conn(0x7f02f40792d0 msgr2=0x7f02f406bed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.717+0000 7f02fb1d1700 1 -- 172.21.15.84:0/751644121 shutdown_connections 2024-04-26T18:28:34.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:34.717+0000 7f02fb1d1700 1 -- 172.21.15.84:0/751644121 wait complete. 2024-04-26T18:28:35.500 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:35 smithi084 ceph-mon[29141]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-26T18:28:35.530 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph tell mds.b status 2024-04-26T18:28:35.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:35 smithi103 ceph-mon[32727]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-26T18:28:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:35 smithi174 ceph-mon[32705]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 2.1 KiB/s wr, 12 op/s 2024-04-26T18:28:35.769 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:37.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.483+0000 7f978f5cf700 1 -- 172.21.15.84:0/3143333795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9788102610 msgr2=0x7f97881029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:37.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.483+0000 7f978f5cf700 1 --2- 172.21.15.84:0/3143333795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9788102610 0x7f97881029f0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f97840099c0 tx=0x7f9784009cd0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.483+0000 7f978f5cf700 1 -- 172.21.15.84:0/3143333795 shutdown_connections 2024-04-26T18:28:37.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.483+0000 7f978f5cf700 1 --2- 172.21.15.84:0/3143333795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9788102610 0x7f97881029f0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.483+0000 7f978f5cf700 1 --2- 172.21.15.84:0/3143333795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9788107660 0x7f978810f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.483+0000 7f978f5cf700 1 --2- 172.21.15.84:0/3143333795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9788102fc0 0x7f9788106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.483+0000 7f978f5cf700 1 -- 172.21.15.84:0/3143333795 >> 172.21.15.84:0/3143333795 conn(0x7f9788078370 msgr2=0x7f9788078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:37.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.483+0000 7f978f5cf700 1 -- 172.21.15.84:0/3143333795 shutdown_connections 2024-04-26T18:28:37.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.483+0000 7f978f5cf700 1 -- 172.21.15.84:0/3143333795 wait complete. 2024-04-26T18:28:37.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978f5cf700 1 Processor -- start 2024-04-26T18:28:37.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978f5cf700 1 -- start start 2024-04-26T18:28:37.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978f5cf700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9788102610 0x7f978819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978f5cf700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9788102fc0 0x7f978819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978f5cf700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9788107660 0x7f9788199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978f5cf700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9788112510 con 0x7f9788102610 2024-04-26T18:28:37.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978f5cf700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9788112390 con 0x7f9788107660 2024-04-26T18:28:37.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978f5cf700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9788112690 con 0x7f9788102fc0 2024-04-26T18:28:37.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978d36b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9788102610 0x7f978819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978d36b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9788102610 0x7f978819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39770/0 (socket says 172.21.15.84:39770) 2024-04-26T18:28:37.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978d36b700 1 -- 172.21.15.84:0/4112624979 learned_addr learned my addr 172.21.15.84:0/4112624979 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:37.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978cb6a700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9788102fc0 0x7f978819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978d36b700 1 -- 172.21.15.84:0/4112624979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9788102fc0 msgr2=0x7f978819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:37.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978d36b700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9788102fc0 0x7f978819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978db6c700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9788107660 0x7f9788199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978d36b700 1 -- 172.21.15.84:0/4112624979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9788107660 msgr2=0x7f9788199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:37.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978d36b700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9788107660 0x7f9788199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978d36b700 1 -- 172.21.15.84:0/4112624979 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f978819a080 con 0x7f9788102610 2024-04-26T18:28:37.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.484+0000 7f978d36b700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9788102610 0x7f978819f760 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f978400b580 tx=0x7f97840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:37.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.485+0000 7f977e7fc700 1 -- 172.21.15.84:0/4112624979 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9784027070 con 0x7f9788102610 2024-04-26T18:28:37.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.485+0000 7f977e7fc700 1 -- 172.21.15.84:0/4112624979 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9784004020 con 0x7f9788102610 2024-04-26T18:28:37.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.485+0000 7f978f5cf700 1 -- 172.21.15.84:0/4112624979 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f978819a310 con 0x7f9788102610 2024-04-26T18:28:37.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.485+0000 7f977e7fc700 1 -- 172.21.15.84:0/4112624979 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9784022e30 con 0x7f9788102610 2024-04-26T18:28:37.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.485+0000 7f978f5cf700 1 -- 172.21.15.84:0/4112624979 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97881a6630 con 0x7f9788102610 2024-04-26T18:28:37.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.487+0000 7f977e7fc700 1 -- 172.21.15.84:0/4112624979 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f9784022590 con 0x7f9788102610 2024-04-26T18:28:37.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.487+0000 7f977e7fc700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97740a3f60 0x7f97740a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.487+0000 7f977e7fc700 1 -- 172.21.15.84:0/4112624979 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f97840d17c0 con 0x7f9788102610 2024-04-26T18:28:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.487+0000 7f978cb6a700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97740a3f60 0x7f97740a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.488+0000 7f978cb6a700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97740a3f60 0x7f97740a6420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9778005950 tx=0x7f97780058e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f9799e61b80 1 Processor -- start 2024-04-26T18:28:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f9799e61b80 1 -- start start 2024-04-26T18:28:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f9799e61b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 0x56405238b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f9799e61b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238b5e0 0x564052385160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f9799e61b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564052385780 0x564052385be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f9799e61b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5640522d37e0 con 0x56405238b5e0 2024-04-26T18:28:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f9799e61b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5640522398e0 con 0x564052385780 2024-04-26T18:28:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f9799e61b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5640522cf7a0 con 0x56405238acc0 2024-04-26T18:28:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f978cb6a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238b5e0 0x564052385160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f978cb6a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238b5e0 0x564052385160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39778/0 (socket says 172.21.15.84:39778) 2024-04-26T18:28:37.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f978cb6a700 1 -- 172.21.15.84:0/1769888963 learned_addr learned my addr 172.21.15.84:0/1769888963 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f978db6c700 1 --2- 172.21.15.84:0/1769888963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564052385780 0x564052385be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f978d36b700 1 --2- 172.21.15.84:0/1769888963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 0x56405238b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f978cb6a700 1 -- 172.21.15.84:0/1769888963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 msgr2=0x56405238b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f978cb6a700 1 --2- 172.21.15.84:0/1769888963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 0x56405238b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f978cb6a700 1 -- 172.21.15.84:0/1769888963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564052385780 msgr2=0x564052385be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f978cb6a700 1 --2- 172.21.15.84:0/1769888963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564052385780 0x564052385be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f978cb6a700 1 -- 172.21.15.84:0/1769888963 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564052220880 con 0x56405238b5e0 2024-04-26T18:28:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f978db6c700 1 --2- 172.21.15.84:0/1769888963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564052385780 0x564052385be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.489+0000 7f978cb6a700 1 --2- 172.21.15.84:0/1769888963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238b5e0 0x564052385160 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f977800fd10 tx=0x7f9778012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62200f36c50ef626 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.490+0000 7f97737fe700 1 -- 172.21.15.84:0/1769888963 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778022070 con 0x56405238b5e0 2024-04-26T18:28:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.490+0000 7f97737fe700 1 -- 172.21.15.84:0/1769888963 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9778011b80 con 0x56405238b5e0 2024-04-26T18:28:37.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.490+0000 7f97737fe700 1 -- 172.21.15.84:0/1769888963 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f977801d610 con 0x56405238b5e0 2024-04-26T18:28:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.490+0000 7f9799e61b80 1 -- 172.21.15.84:0/1769888963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238b5e0 msgr2=0x564052385160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.490+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1769888963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238b5e0 0x564052385160 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f977800fd10 tx=0x7f9778012000 comp rx=0 tx=0).stop 2024-04-26T18:28:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.490+0000 7f9799e61b80 1 -- 172.21.15.84:0/1769888963 shutdown_connections 2024-04-26T18:28:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.490+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1769888963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564052385780 0x564052385be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.490+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1769888963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 0x56405238b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.490+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1769888963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238b5e0 0x564052385160 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.490+0000 7f9799e61b80 1 -- 172.21.15.84:0/1769888963 >> 172.21.15.84:0/1769888963 conn(0x5640522af570 msgr2=0x56405237b900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f9799e61b80 1 -- 172.21.15.84:0/1769888963 shutdown_connections 2024-04-26T18:28:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f9799e61b80 1 -- 172.21.15.84:0/1769888963 wait complete. 2024-04-26T18:28:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f9799e61b80 1 -- start start 2024-04-26T18:28:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f9799e61b80 1 Processor -- start 2024-04-26T18:28:37.500 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f9799e61b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 0x5640523912d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f9799e61b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 0x56405238e670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f9799e61b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56405238b5e0 0x56405238ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f9799e61b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564052392110 con 0x564052385780 2024-04-26T18:28:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f9799e61b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56405228da30 con 0x56405238b5e0 2024-04-26T18:28:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f9799e61b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564052392290 con 0x56405238acc0 2024-04-26T18:28:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f9799e61b80 10 client.0 _ll_register_callbacks cb 0x56405226b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f978d36b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 0x5640523912d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f978d36b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 0x5640523912d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39794/0 (socket says 172.21.15.84:39794) 2024-04-26T18:28:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.491+0000 7f978d36b700 1 -- 172.21.15.84:0/1724361534 learned_addr learned my addr 172.21.15.84:0/1724361534 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f978cb6a700 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 0x56405238e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f978db6c700 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56405238b5e0 0x56405238ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f978d36b700 1 -- 172.21.15.84:0/1724361534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 msgr2=0x56405238e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:37.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f978d36b700 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 0x56405238e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f978d36b700 1 -- 172.21.15.84:0/1724361534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56405238b5e0 msgr2=0x56405238ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f978d36b700 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56405238b5e0 0x56405238ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f978d36b700 1 -- 172.21.15.84:0/1724361534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564052291430 con 0x564052385780 2024-04-26T18:28:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f978db6c700 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56405238b5e0 0x56405238ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f978d36b700 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 0x5640523912d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f978409d2a0 tx=0x7f978409d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d9ac75e82ffc11 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f9770ff9700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f9770ff9700 1 -- 172.21.15.84:0/1724361534 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f978409d5a0 con 0x564052385780 2024-04-26T18:28:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f9770ff9700 1 -- 172.21.15.84:0/1724361534 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f9784003b70 con 0x564052385780 2024-04-26T18:28:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f9799e61b80 1 -- 172.21.15.84:0/1724361534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9778011b80 con 0x564052385780 2024-04-26T18:28:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f9770ff9700 1 -- 172.21.15.84:0/1724361534 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97840a1950 con 0x564052385780 2024-04-26T18:28:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f9799e61b80 10 client.15507 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f9799e61b80 1 -- 172.21.15.84:0/1724361534 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56405239c8c0 con 0x564052385780 2024-04-26T18:28:37.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.492+0000 7f9770ff9700 1 -- 172.21.15.84:0/1724361534 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9778011b80 con 0x564052385780 2024-04-26T18:28:37.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.493+0000 7f9770ff9700 1 -- 172.21.15.84:0/1724361534 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f97840cefd0 con 0x564052385780 2024-04-26T18:28:37.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.493+0000 7f9799e61b80 10 client.15507 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:37.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.493+0000 7f9799e61b80 10 client.15507 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:28:37.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.493+0000 7f9799e61b80 20 client.15507 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:37.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.493+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56405239ef30 0x5640523a03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.493+0000 7f9799e61b80 4 client.15507 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:37.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.493+0000 7f9799e61b80 1 -- 172.21.15.84:0/1724361534 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640522e11c0 con 0x56405239ef30 2024-04-26T18:28:37.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.493+0000 7f978cb6a700 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56405239ef30 0x5640523a03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.494+0000 7f978cb6a700 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56405239ef30 0x5640523a03a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba39398a9cff8f64 server_cookie=5be9e8d4c3c3b173 in_seq=0 out_seq=0 2024-04-26T18:28:37.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.494+0000 7f9770ff9700 10 client.15507 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:37.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.494+0000 7f9770ff9700 1 -- 172.21.15.84:0/1724361534 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640522e11c0 con 0x56405239ef30 2024-04-26T18:28:37.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.494+0000 7f9770ff9700 10 client.15507 handle_command_reply: tid=0 2024-04-26T18:28:37.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.494+0000 7f9799e61b80 1 client.15507 shutdown 2024-04-26T18:28:37.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.495+0000 7f9799e61b80 1 -- 172.21.15.84:0/1724361534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 msgr2=0x5640523912d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:37.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.495+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 0x5640523912d0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f978409d2a0 tx=0x7f978409d030 comp rx=0 tx=0).stop 2024-04-26T18:28:37.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.495+0000 7f9799e61b80 1 -- 172.21.15.84:0/1724361534 shutdown_connections 2024-04-26T18:28:37.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.495+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56405239ef30 0x5640523a03a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.495+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 0x5640523912d0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.495+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56405238b5e0 0x56405238ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.495+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1724361534 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 0x56405238e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.495+0000 7f9799e61b80 1 -- 172.21.15.84:0/1724361534 >> 172.21.15.84:0/1724361534 conn(0x5640522af570 msgr2=0x56405237fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:37.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.495+0000 7f9770ff9700 0 client.15507 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:37.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.495+0000 7f9799e61b80 1 -- 172.21.15.84:0/1724361534 shutdown_connections 2024-04-26T18:28:37.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.495+0000 7f9799e61b80 1 -- 172.21.15.84:0/1724361534 wait complete. 2024-04-26T18:28:37.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.495+0000 7f9799e61b80 20 client.15507 trim_cache size 0 max 16384 2024-04-26T18:28:37.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.515+0000 7f9799e61b80 1 Processor -- start 2024-04-26T18:28:37.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.515+0000 7f9799e61b80 1 -- start start 2024-04-26T18:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f9799e61b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 0x56405237b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f9799e61b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 0x5640523a9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f9799e61b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56405238b5e0 0x5640523a9d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f9799e61b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56405239e6e0 con 0x564052385780 2024-04-26T18:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f9799e61b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56405239edb0 con 0x56405238b5e0 2024-04-26T18:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f9799e61b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56405239f480 con 0x56405238acc0 2024-04-26T18:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f978d36b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 0x56405237b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f978d36b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 0x56405237b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39804/0 (socket says 172.21.15.84:39804) 2024-04-26T18:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f978d36b700 1 -- 172.21.15.84:0/4177198898 learned_addr learned my addr 172.21.15.84:0/4177198898 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f978cb6a700 1 --2- 172.21.15.84:0/4177198898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 0x5640523a9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f978d36b700 1 -- 172.21.15.84:0/4177198898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 msgr2=0x5640523a9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f978d36b700 1 --2- 172.21.15.84:0/4177198898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 0x5640523a9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f978d36b700 1 -- 172.21.15.84:0/4177198898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56405238b5e0 msgr2=0x5640523a9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:37.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f978d36b700 1 --2- 172.21.15.84:0/4177198898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56405238b5e0 0x5640523a9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f978d36b700 1 -- 172.21.15.84:0/4177198898 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564052390f10 con 0x564052385780 2024-04-26T18:28:37.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f978d36b700 1 --2- 172.21.15.84:0/4177198898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 0x56405237b4d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9784099f70 tx=0x7f9784009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be98f3aedcebce48 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:37.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f9772ffd700 1 -- 172.21.15.84:0/4177198898 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97840a6180 con 0x564052385780 2024-04-26T18:28:37.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f9772ffd700 1 -- 172.21.15.84:0/4177198898 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97840cefd0 con 0x564052385780 2024-04-26T18:28:37.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.516+0000 7f9772ffd700 1 -- 172.21.15.84:0/4177198898 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97840bc2e0 con 0x564052385780 2024-04-26T18:28:37.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.517+0000 7f9799e61b80 1 -- 172.21.15.84:0/4177198898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 msgr2=0x56405237b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:37.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.517+0000 7f9799e61b80 1 --2- 172.21.15.84:0/4177198898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 0x56405237b4d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9784099f70 tx=0x7f9784009310 comp rx=0 tx=0).stop 2024-04-26T18:28:37.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.517+0000 7f9799e61b80 1 -- 172.21.15.84:0/4177198898 shutdown_connections 2024-04-26T18:28:37.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.517+0000 7f9799e61b80 1 --2- 172.21.15.84:0/4177198898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564052385780 0x56405237b4d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.517+0000 7f9799e61b80 1 --2- 172.21.15.84:0/4177198898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56405238b5e0 0x5640523a9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.517+0000 7f9799e61b80 1 --2- 172.21.15.84:0/4177198898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238acc0 0x5640523a9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.517+0000 7f9799e61b80 1 -- 172.21.15.84:0/4177198898 >> 172.21.15.84:0/4177198898 conn(0x5640522af570 msgr2=0x564052385b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:37.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.517+0000 7f9799e61b80 1 -- 172.21.15.84:0/4177198898 shutdown_connections 2024-04-26T18:28:37.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.517+0000 7f9799e61b80 1 -- 172.21.15.84:0/4177198898 wait complete. 2024-04-26T18:28:37.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.517+0000 7f9799e61b80 1 -- start start 2024-04-26T18:28:37.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.517+0000 7f9799e61b80 1 Processor -- start 2024-04-26T18:28:37.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f9799e61b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564052385780 0x56405238df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f9799e61b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238acc0 0x56405238e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f9799e61b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238b5e0 0x5640523a7c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f9799e61b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5640523a0220 con 0x56405238acc0 2024-04-26T18:28:37.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f9799e61b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56405239fb50 con 0x564052385780 2024-04-26T18:28:37.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f9799e61b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5640523a08f0 con 0x56405238b5e0 2024-04-26T18:28:37.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f9799e61b80 10 client.0 _ll_register_callbacks cb 0x56405226b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:37.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f978cb6a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238acc0 0x56405238e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f978cb6a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238acc0 0x56405238e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39818/0 (socket says 172.21.15.84:39818) 2024-04-26T18:28:37.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f978cb6a700 1 -- 172.21.15.84:0/1040116199 learned_addr learned my addr 172.21.15.84:0/1040116199 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:37.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f978db6c700 1 --2- 172.21.15.84:0/1040116199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238b5e0 0x5640523a7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f978d36b700 1 --2- 172.21.15.84:0/1040116199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564052385780 0x56405238df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f978cb6a700 1 -- 172.21.15.84:0/1040116199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238b5e0 msgr2=0x5640523a7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:37.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f978cb6a700 1 --2- 172.21.15.84:0/1040116199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238b5e0 0x5640523a7c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f978cb6a700 1 -- 172.21.15.84:0/1040116199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564052385780 msgr2=0x56405238df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f978cb6a700 1 --2- 172.21.15.84:0/1040116199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564052385780 0x56405238df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f978cb6a700 1 -- 172.21.15.84:0/1040116199 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640523a81d0 con 0x56405238acc0 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f978cb6a700 1 --2- 172.21.15.84:0/1040116199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238acc0 0x56405238e460 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9778014040 tx=0x7f9778005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c8917cb66e783d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f9770ff9700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.518+0000 7f9770ff9700 1 -- 172.21.15.84:0/1040116199 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9778022040 con 0x56405238acc0 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f9770ff9700 1 -- 172.21.15.84:0/1040116199 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f9778011b80 con 0x56405238acc0 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f9799e61b80 1 -- 172.21.15.84:0/1040116199 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97840cefd0 con 0x56405238acc0 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f9770ff9700 1 -- 172.21.15.84:0/1040116199 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9778045050 con 0x56405238acc0 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f9799e61b80 10 client.15513 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f9799e61b80 1 -- 172.21.15.84:0/1040116199 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564052406680 con 0x56405238acc0 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f9770ff9700 1 -- 172.21.15.84:0/1040116199 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97840cefd0 con 0x56405238acc0 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f9770ff9700 1 -- 172.21.15.84:0/1040116199 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9778004970 con 0x56405238acc0 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f9799e61b80 10 client.15513 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:37.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f9799e61b80 10 client.15513 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f9799e61b80 20 client.15513 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1040116199 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56405240af30 0x564052409c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f9799e61b80 4 client.15513 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f9799e61b80 1 -- 172.21.15.84:0/1040116199 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "status"}) -- 0x5640522db260 con 0x56405240af30 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.519+0000 7f978d36b700 1 --2- 172.21.15.84:0/1040116199 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56405240af30 0x564052409c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f978d36b700 1 --2- 172.21.15.84:0/1040116199 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56405240af30 0x564052409c40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=785c0d8be58c522c server_cookie=a8db2564737df6f2 in_seq=0 out_seq=0 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9770ff9700 10 client.15513 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9770ff9700 1 -- 172.21.15.84:0/1040116199 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5640522db260 con 0x56405240af30 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9770ff9700 10 client.15513 handle_command_reply: tid=0 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9799e61b80 1 client.15513 shutdown 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9799e61b80 1 -- 172.21.15.84:0/1040116199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238acc0 msgr2=0x56405238e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1040116199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238acc0 0x56405238e460 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9778014040 tx=0x7f9778005fd0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9799e61b80 1 -- 172.21.15.84:0/1040116199 shutdown_connections 2024-04-26T18:28:37.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1040116199 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56405240af30 0x564052409c40 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1040116199 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56405238b5e0 0x5640523a7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1040116199 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564052385780 0x56405238df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9799e61b80 1 --2- 172.21.15.84:0/1040116199 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56405238acc0 0x56405238e460 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9799e61b80 1 -- 172.21.15.84:0/1040116199 >> 172.21.15.84:0/1040116199 conn(0x5640522af570 msgr2=0x56405237b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:37.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9770ff9700 0 client.15513 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:37.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9799e61b80 1 -- 172.21.15.84:0/1040116199 shutdown_connections 2024-04-26T18:28:37.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9799e61b80 1 -- 172.21.15.84:0/1040116199 wait complete. 2024-04-26T18:28:37.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.520+0000 7f9799e61b80 20 client.15513 trim_cache size 0 max 16384 2024-04-26T18:28:37.526 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:28:37.526 INFO:teuthology.orchestra.run.smithi084.stdout: "cluster_fsid": "df3f3816-03f8-11ef-bc93-c7b262605968", 2024-04-26T18:28:37.527 INFO:teuthology.orchestra.run.smithi084.stdout: "whoami": 0, 2024-04-26T18:28:37.527 INFO:teuthology.orchestra.run.smithi084.stdout: "id": 24434, 2024-04-26T18:28:37.527 INFO:teuthology.orchestra.run.smithi084.stdout: "want_state": "up:active", 2024-04-26T18:28:37.527 INFO:teuthology.orchestra.run.smithi084.stdout: "state": "up:active", 2024-04-26T18:28:37.527 INFO:teuthology.orchestra.run.smithi084.stdout: "fs_name": "cephfs", 2024-04-26T18:28:37.527 INFO:teuthology.orchestra.run.smithi084.stdout: "rank_uptime": 254.56982684600001, 2024-04-26T18:28:37.527 INFO:teuthology.orchestra.run.smithi084.stdout: "mdsmap_epoch": 29, 2024-04-26T18:28:37.527 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch": 94, 2024-04-26T18:28:37.527 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_epoch_barrier": 93, 2024-04-26T18:28:37.527 INFO:teuthology.orchestra.run.smithi084.stdout: "uptime": 268.82708093899998 2024-04-26T18:28:37.527 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:28:37.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.521+0000 7f978f5cf700 1 -- 172.21.15.84:0/4112624979 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97740a3f60 msgr2=0x7f97740a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:37.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.521+0000 7f978f5cf700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97740a3f60 0x7f97740a6420 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9778005950 tx=0x7f97780058e0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.521+0000 7f978f5cf700 1 -- 172.21.15.84:0/4112624979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9788102610 msgr2=0x7f978819f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:37.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.521+0000 7f978f5cf700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9788102610 0x7f978819f760 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f978400b580 tx=0x7f97840047b0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.521+0000 7f978f5cf700 1 -- 172.21.15.84:0/4112624979 shutdown_connections 2024-04-26T18:28:37.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.521+0000 7f978f5cf700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97740a3f60 0x7f97740a6420 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.521+0000 7f978f5cf700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9788102610 0x7f978819f760 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.521+0000 7f978f5cf700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9788107660 0x7f9788199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.521+0000 7f978f5cf700 1 --2- 172.21.15.84:0/4112624979 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9788102fc0 0x7f978819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:37.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.521+0000 7f978f5cf700 1 -- 172.21.15.84:0/4112624979 >> 172.21.15.84:0/4112624979 conn(0x7f9788078370 msgr2=0x7f97881065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:37.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.521+0000 7f978f5cf700 1 -- 172.21.15.84:0/4112624979 shutdown_connections 2024-04-26T18:28:37.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:37.521+0000 7f978f5cf700 1 -- 172.21.15.84:0/4112624979 wait complete. 2024-04-26T18:28:37.531 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:37 smithi084 ceph-mon[29141]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-04-26T18:28:37.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:37 smithi103 ceph-mon[32727]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-04-26T18:28:37.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:37 smithi174 ceph-mon[32705]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 2.1 KiB/s wr, 14 op/s 2024-04-26T18:28:38.463 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-04-26T18:28:39.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:39 smithi084 ceph-mon[29141]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-26T18:28:39.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:39 smithi103 ceph-mon[32727]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-26T18:28:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.650+0000 7f91b2358640 1 -- 172.21.15.84:0/2077092858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ac102220 msgr2=0x7f91ac10e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.650+0000 7f91b2358640 1 --2- 172.21.15.84:0/2077092858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ac102220 0x7f91ac10e6e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f91a4009cd0 tx=0x7f91a402a8e0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.650+0000 7f91b2358640 1 -- 172.21.15.84:0/2077092858 shutdown_connections 2024-04-26T18:28:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.650+0000 7f91b2358640 1 --2- 172.21.15.84:0/2077092858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ac102220 0x7f91ac10e6e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.650+0000 7f91b2358640 1 --2- 172.21.15.84:0/2077092858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91ac101860 0x7f91ac101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.650+0000 7f91b2358640 1 --2- 172.21.15.84:0/2077092858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91ac107820 0x7f91ac107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.650+0000 7f91b2358640 1 -- 172.21.15.84:0/2077092858 >> 172.21.15.84:0/2077092858 conn(0x7f91ac0789b0 msgr2=0x7f91ac078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.650+0000 7f91b2358640 1 -- 172.21.15.84:0/2077092858 shutdown_connections 2024-04-26T18:28:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.650+0000 7f91b2358640 1 -- 172.21.15.84:0/2077092858 wait complete. 2024-04-26T18:28:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.651+0000 7f91b2358640 1 Processor -- start 2024-04-26T18:28:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.651+0000 7f91b2358640 1 -- start start 2024-04-26T18:28:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b2358640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91ac101860 0x7f91ac199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b2358640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91ac102220 0x7f91ac19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b2358640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ac107820 0x7f91ac19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b2358640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac111aa0 con 0x7f91ac107820 2024-04-26T18:28:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b2358640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac111920 con 0x7f91ac101860 2024-04-26T18:28:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b2358640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac111c20 con 0x7f91ac102220 2024-04-26T18:28:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b08f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ac107820 0x7f91ac19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b08f5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ac107820 0x7f91ac19a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39830/0 (socket says 172.21.15.84:39830) 2024-04-26T18:28:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b08f5640 1 -- 172.21.15.84:0/1999900917 learned_addr learned my addr 172.21.15.84:0/1999900917 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91ac102220 0x7f91ac19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91abfff640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91ac101860 0x7f91ac199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b08f5640 1 -- 172.21.15.84:0/1999900917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91ac102220 msgr2=0x7f91ac19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b08f5640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91ac102220 0x7f91ac19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b08f5640 1 -- 172.21.15.84:0/1999900917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91ac101860 msgr2=0x7f91ac199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b08f5640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91ac101860 0x7f91ac199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b08f5640 1 -- 172.21.15.84:0/1999900917 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91ac1a52c0 con 0x7f91ac107820 2024-04-26T18:28:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b08f5640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ac107820 0x7f91ac19a980 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f91a4033040 tx=0x7f91a4009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91a97fa640 1 -- 172.21.15.84:0/1999900917 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4004040 con 0x7f91ac107820 2024-04-26T18:28:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91a97fa640 1 -- 172.21.15.84:0/1999900917 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91a4004690 con 0x7f91ac107820 2024-04-26T18:28:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b2358640 1 -- 172.21.15.84:0/1999900917 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91ac1a5550 con 0x7f91ac107820 2024-04-26T18:28:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91a97fa640 1 -- 172.21.15.84:0/1999900917 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a4030d90 con 0x7f91ac107820 2024-04-26T18:28:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.652+0000 7f91b2358640 1 -- 172.21.15.84:0/1999900917 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91ac1a5aa0 con 0x7f91ac107820 2024-04-26T18:28:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91a97fa640 1 -- 172.21.15.84:0/1999900917 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f91a40041e0 con 0x7f91ac107820 2024-04-26T18:28:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91b5072740 1 Processor -- start 2024-04-26T18:28:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91b5072740 1 -- start start 2024-04-26T18:28:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91b5072740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349ef00 0x56341349f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91b5072740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 0x5634134a9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91b5072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91b5072740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563413405460 con 0x56341349f820 2024-04-26T18:28:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91b5072740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5634133bf590 con 0x56341349ef00 2024-04-26T18:28:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91b5072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563413401420 con 0x5634134aa0e0 2024-04-26T18:28:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 0x5634134a9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91a97fa640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f91940a3f70 0x7f91940a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 0x5634134a9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39836/0 (socket says 172.21.15.84:39836) 2024-04-26T18:28:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91abfff640 1 -- 172.21.15.84:0/3596680702 learned_addr learned my addr 172.21.15.84:0/3596680702 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/3596680702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91b08f5640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f91940a3f70 0x7f91940a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.655+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/3596680702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349ef00 0x56341349f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.656+0000 7f91abfff640 1 -- 172.21.15.84:0/3596680702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 msgr2=0x5634134ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.656+0000 7f91abfff640 1 --2- 172.21.15.84:0/3596680702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134ac4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.656+0000 7f91abfff640 1 -- 172.21.15.84:0/3596680702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349ef00 msgr2=0x56341349f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.656+0000 7f91abfff640 1 --2- 172.21.15.84:0/3596680702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349ef00 0x56341349f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.656+0000 7f91abfff640 1 -- 172.21.15.84:0/3596680702 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634133cc0d0 con 0x56341349f820 2024-04-26T18:28:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.656+0000 7f91a97fa640 1 -- 172.21.15.84:0/1999900917 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f91a402e030 con 0x7f91ac107820 2024-04-26T18:28:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.656+0000 7f91abfff640 1 --2- 172.21.15.84:0/3596680702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 0x5634134a9ba0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f919c009c00 tx=0x7f919c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbc90f482b16f77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.656+0000 7f91b08f5640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f91940a3f70 0x7f91940a6430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f91a40ef070 tx=0x7f91a40e8970 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.656+0000 7f91927fc640 1 -- 172.21.15.84:0/3596680702 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c012450 con 0x56341349f820 2024-04-26T18:28:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.656+0000 7f91927fc640 1 -- 172.21.15.84:0/3596680702 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f919c012aa0 con 0x56341349f820 2024-04-26T18:28:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.656+0000 7f91927fc640 1 -- 172.21.15.84:0/3596680702 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c00fa40 con 0x56341349f820 2024-04-26T18:28:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.657+0000 7f91b5072740 1 -- 172.21.15.84:0/3596680702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 msgr2=0x5634134a9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.657+0000 7f91b5072740 1 --2- 172.21.15.84:0/3596680702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 0x5634134a9ba0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f919c009c00 tx=0x7f919c0117b0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.657+0000 7f91b5072740 1 -- 172.21.15.84:0/3596680702 shutdown_connections 2024-04-26T18:28:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.657+0000 7f91b5072740 1 --2- 172.21.15.84:0/3596680702 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.657+0000 7f91b5072740 1 --2- 172.21.15.84:0/3596680702 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 0x5634134a9ba0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.657+0000 7f91b5072740 1 --2- 172.21.15.84:0/3596680702 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349ef00 0x56341349f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.657+0000 7f91b5072740 1 -- 172.21.15.84:0/3596680702 >> 172.21.15.84:0/3596680702 conn(0x56341349b410 msgr2=0x56341349b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.657+0000 7f91b5072740 1 -- 172.21.15.84:0/3596680702 shutdown_connections 2024-04-26T18:28:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.657+0000 7f91b5072740 1 -- 172.21.15.84:0/3596680702 wait complete. 2024-04-26T18:28:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.657+0000 7f91b5072740 1 -- start start 2024-04-26T18:28:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.657+0000 7f91b5072740 1 Processor -- start 2024-04-26T18:28:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91b5072740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349ef00 0x5634134a7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91b5072740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 0x5634134a7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91b5072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91b5072740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5634134acd10 con 0x56341349f820 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91b5072740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56341338cb60 con 0x56341349ef00 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91b5072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5634134ace90 con 0x5634134aa0e0 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91b5072740 10 client.0 _ll_register_callbacks cb 0x5634133b12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 0x5634134a7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 0x5634134a7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39842/0 (socket says 172.21.15.84:39842) 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91abfff640 1 -- 172.21.15.84:0/3007663222 learned_addr learned my addr 172.21.15.84:0/3007663222 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349ef00 0x5634134a7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91abfff640 1 -- 172.21.15.84:0/3007663222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 msgr2=0x5634134a7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91abfff640 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134a7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91abfff640 1 -- 172.21.15.84:0/3007663222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349ef00 msgr2=0x5634134a7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91abfff640 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349ef00 0x5634134a7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91abfff640 1 -- 172.21.15.84:0/3007663222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563413117210 con 0x56341349f820 2024-04-26T18:28:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349ef00 0x5634134a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f91abfff640 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 0x5634134a7570 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f919c000c00 tx=0x7f919c009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ae087abca6d47a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f918b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.658+0000 7f918b7fe640 1 -- 172.21.15.84:0/3007663222 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919c012450 con 0x56341349f820 2024-04-26T18:28:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f918b7fe640 1 -- 172.21.15.84:0/3007663222 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f919c0125f0 con 0x56341349f820 2024-04-26T18:28:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f91b5072740 1 -- 172.21.15.84:0/3007663222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f919c012aa0 con 0x56341349f820 2024-04-26T18:28:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f918b7fe640 1 -- 172.21.15.84:0/3007663222 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f919c032050 con 0x56341349f820 2024-04-26T18:28:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f91b5072740 10 client.15525 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f91b5072740 1 -- 172.21.15.84:0/3007663222 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5634131e3620 con 0x56341349f820 2024-04-26T18:28:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f918b7fe640 1 -- 172.21.15.84:0/3007663222 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f919c012aa0 con 0x56341349f820 2024-04-26T18:28:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f918b7fe640 1 -- 172.21.15.84:0/3007663222 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f919c0035f0 con 0x56341349f820 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f91b5072740 10 client.15525 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f91b5072740 10 client.15525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f91b5072740 20 client.15525 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f91b5072740 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5634134ba170 0x5634134bc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f91b5072740 4 client.15525 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f91b5072740 1 -- 172.21.15.84:0/3007663222 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563413413070 con 0x5634134ba170 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.659+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5634134ba170 0x5634134bc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.660+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5634134ba170 0x5634134bc560 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78e7c3158bd428fa server_cookie=c2f14f79d570431 in_seq=0 out_seq=0 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.660+0000 7f918b7fe640 10 client.15525 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.660+0000 7f918b7fe640 1 -- 172.21.15.84:0/3007663222 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563413413070 con 0x5634134ba170 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.660+0000 7f918b7fe640 10 client.15525 handle_command_reply: tid=0 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.660+0000 7f91b5072740 1 client.15525 shutdown 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.661+0000 7f91b5072740 1 -- 172.21.15.84:0/3007663222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 msgr2=0x5634134a7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.661+0000 7f91b5072740 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 0x5634134a7570 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f919c000c00 tx=0x7f919c009f20 comp rx=0 tx=0).stop 2024-04-26T18:28:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.661+0000 7f91b5072740 1 -- 172.21.15.84:0/3007663222 shutdown_connections 2024-04-26T18:28:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.661+0000 7f91b5072740 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5634134ba170 0x5634134bc560 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.661+0000 7f91b5072740 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.661+0000 7f91b5072740 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349f820 0x5634134a7570 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.661+0000 7f918b7fe640 0 client.15525 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.661+0000 7f91b5072740 1 --2- 172.21.15.84:0/3007663222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349ef00 0x5634134a7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.661+0000 7f91b5072740 1 -- 172.21.15.84:0/3007663222 >> 172.21.15.84:0/3007663222 conn(0x56341349b410 msgr2=0x5634134a1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.661+0000 7f91b5072740 1 -- 172.21.15.84:0/3007663222 shutdown_connections 2024-04-26T18:28:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.661+0000 7f91b5072740 1 -- 172.21.15.84:0/3007663222 wait complete. 2024-04-26T18:28:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.661+0000 7f91b5072740 20 client.15525 trim_cache size 0 max 16384 2024-04-26T18:28:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.675+0000 7f91b5072740 1 Processor -- start 2024-04-26T18:28:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.675+0000 7f91b5072740 1 -- start start 2024-04-26T18:28:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91b5072740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 0x5634134b5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91b5072740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349f820 0x5634134a7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91b5072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134c1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91b5072740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1ae470 con 0x56341349ef00 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91b5072740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1affb0 con 0x56341349f820 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91b5072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1aeb40 con 0x5634134aa0e0 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91ab7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 0x5634134b5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91ab7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 0x5634134b5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39858/0 (socket says 172.21.15.84:39858) 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91ab7fe640 1 -- 172.21.15.84:0/2990118648 learned_addr learned my addr 172.21.15.84:0/2990118648 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91ab7fe640 1 -- 172.21.15.84:0/2990118648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 msgr2=0x5634134c1be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/2990118648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134c1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91ab7fe640 1 -- 172.21.15.84:0/2990118648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349f820 msgr2=0x5634134a7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/2990118648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349f820 0x5634134a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91ab7fe640 1 -- 172.21.15.84:0/2990118648 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634134a1830 con 0x56341349ef00 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/2990118648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 0x5634134b5bd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009730 tx=0x7f91a00096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1124729a9b2e8365 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f9191ffb640 1 -- 172.21.15.84:0/2990118648 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0029070 con 0x56341349ef00 2024-04-26T18:28:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f9191ffb640 1 -- 172.21.15.84:0/2990118648 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91a000fd80 con 0x56341349ef00 2024-04-26T18:28:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.676+0000 7f9191ffb640 1 -- 172.21.15.84:0/2990118648 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0024800 con 0x56341349ef00 2024-04-26T18:28:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.677+0000 7f91b5072740 1 -- 172.21.15.84:0/2990118648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 msgr2=0x5634134b5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.677+0000 7f91b5072740 1 --2- 172.21.15.84:0/2990118648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 0x5634134b5bd0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009730 tx=0x7f91a00096c0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.677+0000 7f91b5072740 1 -- 172.21.15.84:0/2990118648 shutdown_connections 2024-04-26T18:28:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.677+0000 7f91b5072740 1 --2- 172.21.15.84:0/2990118648 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134c1be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.677+0000 7f91b5072740 1 --2- 172.21.15.84:0/2990118648 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349f820 0x5634134a7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.677+0000 7f91b5072740 1 --2- 172.21.15.84:0/2990118648 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 0x5634134b5bd0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.677+0000 7f91b5072740 1 -- 172.21.15.84:0/2990118648 >> 172.21.15.84:0/2990118648 conn(0x56341349b410 msgr2=0x5634134a6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.677+0000 7f91b5072740 1 -- 172.21.15.84:0/2990118648 shutdown_connections 2024-04-26T18:28:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.677+0000 7f91b5072740 1 -- 172.21.15.84:0/2990118648 wait complete. 2024-04-26T18:28:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.677+0000 7f91b5072740 1 -- start start 2024-04-26T18:28:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.677+0000 7f91b5072740 1 Processor -- start 2024-04-26T18:28:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91b5072740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 0x5634134b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91b5072740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349f820 0x5634134b2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91b5072740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134af0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91b5072740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1af8e0 con 0x56341349ef00 2024-04-26T18:28:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91b5072740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1af210 con 0x56341349f820 2024-04-26T18:28:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91b5072740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f91ac1ac450 con 0x5634134aa0e0 2024-04-26T18:28:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91b5072740 10 client.0 _ll_register_callbacks cb 0x7f91a0009a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91ab7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 0x5634134b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91ab7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 0x5634134b20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39868/0 (socket says 172.21.15.84:39868) 2024-04-26T18:28:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91ab7fe640 1 -- 172.21.15.84:0/288149134 learned_addr learned my addr 172.21.15.84:0/288149134 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91ab7fe640 1 -- 172.21.15.84:0/288149134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 msgr2=0x5634134af0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/288149134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134af0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91ab7fe640 1 -- 172.21.15.84:0/288149134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349f820 msgr2=0x5634134b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/288149134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349f820 0x5634134b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91ab7fe640 1 -- 172.21.15.84:0/288149134 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634134af8b0 con 0x56341349ef00 2024-04-26T18:28:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91ab7fe640 1 --2- 172.21.15.84:0/288149134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 0x5634134b20e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f91a00098a0 tx=0x7f91a0024970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5746b25467e94c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f918b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f918b7fe640 1 -- 172.21.15.84:0/288149134 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a0029070 con 0x56341349ef00 2024-04-26T18:28:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f918b7fe640 1 -- 172.21.15.84:0/288149134 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f91a000fa10 con 0x56341349ef00 2024-04-26T18:28:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f91b5072740 1 -- 172.21.15.84:0/288149134 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91a000fd80 con 0x56341349ef00 2024-04-26T18:28:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.678+0000 7f918b7fe640 1 -- 172.21.15.84:0/288149134 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f91a00240d0 con 0x56341349ef00 2024-04-26T18:28:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.679+0000 7f91b5072740 10 client.15531 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.679+0000 7f91b5072740 1 -- 172.21.15.84:0/288149134 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563413520e90 con 0x56341349ef00 2024-04-26T18:28:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.679+0000 7f918b7fe640 1 -- 172.21.15.84:0/288149134 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91a000fd80 con 0x56341349ef00 2024-04-26T18:28:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.679+0000 7f918b7fe640 1 -- 172.21.15.84:0/288149134 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f91a0020040 con 0x56341349ef00 2024-04-26T18:28:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.679+0000 7f91b5072740 10 client.15531 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.679+0000 7f91b5072740 10 client.15531 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.679+0000 7f91b5072740 20 client.15531 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.679+0000 7f91b5072740 1 --2- 172.21.15.84:0/288149134 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563413521550 0x563413523970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.679+0000 7f91b5072740 4 client.15531 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:28:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.679+0000 7f91b5072740 1 -- 172.21.15.84:0/288149134 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56341340cee0 con 0x563413521550 2024-04-26T18:28:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.679+0000 7f91abfff640 1 --2- 172.21.15.84:0/288149134 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563413521550 0x563413523970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.680+0000 7f918b7fe640 10 client.15531 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.680+0000 7f91abfff640 1 --2- 172.21.15.84:0/288149134 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563413521550 0x563413523970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=762a0bc7ac308144 server_cookie=fc7c31ae89567d10 in_seq=0 out_seq=0 2024-04-26T18:28:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.683+0000 7f918b7fe640 1 -- 172.21.15.84:0/288149134 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56341340cee0 con 0x563413521550 2024-04-26T18:28:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.683+0000 7f918b7fe640 10 client.15531 handle_command_reply: tid=0 2024-04-26T18:28:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.683+0000 7f91b5072740 1 client.15531 shutdown 2024-04-26T18:28:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b5072740 1 -- 172.21.15.84:0/288149134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 msgr2=0x5634134b20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b5072740 1 --2- 172.21.15.84:0/288149134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 0x5634134b20e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f91a00098a0 tx=0x7f91a0024970 comp rx=0 tx=0).stop 2024-04-26T18:28:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b5072740 1 -- 172.21.15.84:0/288149134 shutdown_connections 2024-04-26T18:28:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b5072740 1 --2- 172.21.15.84:0/288149134 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563413521550 0x563413523970 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b5072740 1 --2- 172.21.15.84:0/288149134 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5634134aa0e0 0x5634134af0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b5072740 1 --2- 172.21.15.84:0/288149134 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56341349f820 0x5634134b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b5072740 1 --2- 172.21.15.84:0/288149134 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56341349ef00 0x5634134b20e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f918b7fe640 0 client.15531 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b5072740 1 -- 172.21.15.84:0/288149134 >> 172.21.15.84:0/288149134 conn(0x56341349b410 msgr2=0x5634134a10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b5072740 1 -- 172.21.15.84:0/288149134 shutdown_connections 2024-04-26T18:28:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b5072740 1 -- 172.21.15.84:0/288149134 wait complete. 2024-04-26T18:28:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b5072740 20 client.15531 trim_cache size 0 max 16384 2024-04-26T18:28:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:28:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:28:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:28:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:28:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b2358640 1 -- 172.21.15.84:0/1999900917 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f91940a3f70 msgr2=0x7f91940a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b2358640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f91940a3f70 0x7f91940a6430 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f91a40ef070 tx=0x7f91a40e8970 comp rx=0 tx=0).stop 2024-04-26T18:28:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b2358640 1 -- 172.21.15.84:0/1999900917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ac107820 msgr2=0x7f91ac19a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b2358640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ac107820 0x7f91ac19a980 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f91a4033040 tx=0x7f91a4009360 comp rx=0 tx=0).stop 2024-04-26T18:28:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b2358640 1 -- 172.21.15.84:0/1999900917 shutdown_connections 2024-04-26T18:28:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b2358640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f91940a3f70 0x7f91940a6430 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.684+0000 7f91b2358640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f91ac107820 0x7f91ac19a980 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.685+0000 7f91b2358640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f91ac102220 0x7f91ac19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.685+0000 7f91b2358640 1 --2- 172.21.15.84:0/1999900917 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f91ac101860 0x7f91ac199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.685+0000 7f91b2358640 1 -- 172.21.15.84:0/1999900917 >> 172.21.15.84:0/1999900917 conn(0x7f91ac0789b0 msgr2=0x7f91ac1042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.685+0000 7f91b2358640 1 -- 172.21.15.84:0/1999900917 shutdown_connections 2024-04-26T18:28:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:39.685+0000 7f91b2358640 1 -- 172.21.15.84:0/1999900917 wait complete. 2024-04-26T18:28:39.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:39 smithi174 ceph-mon[32705]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-26T18:28:41.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:41 smithi084 ceph-mon[29141]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-26T18:28:41.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:41 smithi103 ceph-mon[32727]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-26T18:28:41.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:41 smithi174 ceph-mon[32705]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-04-26T18:28:43.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:43 smithi084 ceph-mon[29141]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:28:43.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:43 smithi103 ceph-mon[32727]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:28:43.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:43 smithi174 ceph-mon[32705]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:28:45.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:45 smithi084 ceph-mon[29141]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T18:28:45.626 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:45 smithi103 ceph-mon[32727]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T18:28:45.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:45 smithi174 ceph-mon[32705]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T18:28:47.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:47 smithi084 ceph-mon[29141]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-26T18:28:47.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:47 smithi103 ceph-mon[32727]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-26T18:28:47.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:47 smithi174 ceph-mon[32705]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s 2024-04-26T18:28:49.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:49 smithi103 ceph-mon[32727]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:28:49.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:49 smithi174 ceph-mon[32705]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:28:49.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:49 smithi084 ceph-mon[29141]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:28:51.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:51 smithi103 ceph-mon[32727]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-26T18:28:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:51 smithi174 ceph-mon[32705]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-26T18:28:51.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:51 smithi084 ceph-mon[29141]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-26T18:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.780+0000 7f9b55cdc640 1 -- 172.21.15.84:0/1452390819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b50102940 msgr2=0x7f9b50102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.780+0000 7f9b55cdc640 1 --2- 172.21.15.84:0/1452390819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b50102940 0x7f9b50102dc0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9b44009930 tx=0x7f9b4402a420 comp rx=0 tx=0).stop 2024-04-26T18:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.780+0000 7f9b55cdc640 1 -- 172.21.15.84:0/1452390819 shutdown_connections 2024-04-26T18:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.780+0000 7f9b55cdc640 1 --2- 172.21.15.84:0/1452390819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b50103300 0x7f9b5010f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.780+0000 7f9b55cdc640 1 --2- 172.21.15.84:0/1452390819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b50102940 0x7f9b50102dc0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.780+0000 7f9b55cdc640 1 --2- 172.21.15.84:0/1452390819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b50108950 0x7f9b50108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.780+0000 7f9b55cdc640 1 -- 172.21.15.84:0/1452390819 >> 172.21.15.84:0/1452390819 conn(0x7f9b50079dc0 msgr2=0x7f9b5007a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:52.781 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.780+0000 7f9b55cdc640 1 -- 172.21.15.84:0/1452390819 shutdown_connections 2024-04-26T18:28:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.781+0000 7f9b55cdc640 1 -- 172.21.15.84:0/1452390819 wait complete. 2024-04-26T18:28:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.781+0000 7f9b55cdc640 1 Processor -- start 2024-04-26T18:28:52.782 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.781+0000 7f9b55cdc640 1 -- start start 2024-04-26T18:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.781+0000 7f9b55cdc640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b50102940 0x7f9b50199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.781+0000 7f9b55cdc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b50103300 0x7f9b50199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.781+0000 7f9b55cdc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b50108950 0x7f9b5019d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.781+0000 7f9b55cdc640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9b50112c10 con 0x7f9b50103300 2024-04-26T18:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.781+0000 7f9b55cdc640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b50112a90 con 0x7f9b50108950 2024-04-26T18:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.781+0000 7f9b55cdc640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b50112d90 con 0x7f9b50102940 2024-04-26T18:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b50103300 0x7f9b50199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b50103300 0x7f9b50199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44418/0 (socket says 172.21.15.84:44418) 2024-04-26T18:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4effd640 1 -- 172.21.15.84:0/3123227061 learned_addr learned my addr 172.21.15.84:0/3123227061 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4f7fe640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b50102940 0x7f9b50199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4ffff640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b50108950 0x7f9b5019d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4effd640 1 -- 172.21.15.84:0/3123227061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b50102940 msgr2=0x7f9b50199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4effd640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b50102940 0x7f9b50199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4effd640 1 -- 172.21.15.84:0/3123227061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b50108950 msgr2=0x7f9b5019d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4effd640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b50108950 0x7f9b5019d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4effd640 1 -- 172.21.15.84:0/3123227061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5019e020 con 0x7f9b50103300 2024-04-26T18:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4ffff640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b50108950 0x7f9b5019d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4effd640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b50103300 0x7f9b50199570 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9b44004770 tx=0x7f9b440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4cff9640 1 -- 172.21.15.84:0/3123227061 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4403f070 con 0x7f9b50103300 2024-04-26T18:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4cff9640 1 -- 172.21.15.84:0/3123227061 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b4403b460 con 0x7f9b50103300 2024-04-26T18:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b55cdc640 1 -- 172.21.15.84:0/3123227061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b5019e2b0 con 0x7f9b50103300 2024-04-26T18:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b4cff9640 1 -- 172.21.15.84:0/3123227061 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b44035070 con 0x7f9b50103300 2024-04-26T18:28:52.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.782+0000 7f9b55cdc640 1 -- 172.21.15.84:0/3123227061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b501a5ba0 con 0x7f9b50103300 2024-04-26T18:28:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.785+0000 7f9b4cff9640 1 -- 172.21.15.84:0/3123227061 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f9b4403b8d0 con 0x7f9b50103300 2024-04-26T18:28:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.785+0000 7f9b589f4740 1 Processor -- start 2024-04-26T18:28:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.785+0000 7f9b589f4740 1 -- start start 2024-04-26T18:28:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.785+0000 7f9b589f4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 0x56084b9321c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.785+0000 7f9b589f4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b932700 0x56084b940fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.785+0000 7f9b589f4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b9414f0 0x56084b9438e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.785+0000 7f9b589f4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56084b898460 con 0x56084b932700 2024-04-26T18:28:52.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.785+0000 7f9b589f4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56084b852590 con 0x56084b931de0 2024-04-26T18:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.785+0000 7f9b589f4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56084b894420 con 0x56084b9414f0 2024-04-26T18:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b932700 0x56084b940fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b932700 0x56084b940fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44422/0 (socket says 172.21.15.84:44422) 2024-04-26T18:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4f7fe640 1 -- 172.21.15.84:0/2495954423 learned_addr learned my addr 172.21.15.84:0/2495954423 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4ffff640 1 --2- 172.21.15.84:0/2495954423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 0x56084b9321c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4cff9640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9b380a3f70 0x7f9b380a6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4effd640 1 --2- 172.21.15.84:0/2495954423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b9414f0 0x56084b9438e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4f7fe640 1 -- 172.21.15.84:0/2495954423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b9414f0 msgr2=0x56084b9438e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4f7fe640 1 --2- 172.21.15.84:0/2495954423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b9414f0 0x56084b9438e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4ffff640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9b380a3f70 0x7f9b380a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4f7fe640 1 -- 172.21.15.84:0/2495954423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 msgr2=0x56084b9321c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:52.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4f7fe640 1 --2- 172.21.15.84:0/2495954423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 0x56084b9321c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4f7fe640 1 -- 172.21.15.84:0/2495954423 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084b85f0d0 con 0x56084b932700 2024-04-26T18:28:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4cff9640 1 -- 172.21.15.84:0/3123227061 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f9b440f0ee0 con 0x7f9b50103300 2024-04-26T18:28:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4f7fe640 1 --2- 172.21.15.84:0/2495954423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b932700 0x56084b940fb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000cc10 tx=0x7f9b4000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da9b6ad7389c9985 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b35ffb640 1 -- 172.21.15.84:0/2495954423 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40005220 con 0x56084b932700 2024-04-26T18:28:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b4ffff640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9b380a3f70 0x7f9b380a6430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9b48005fd0 tx=0x7f9b48004ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b35ffb640 1 -- 172.21.15.84:0/2495954423 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b40004c50 con 0x56084b932700 2024-04-26T18:28:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.786+0000 7f9b35ffb640 1 -- 172.21.15.84:0/2495954423 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40010720 con 0x56084b932700 2024-04-26T18:28:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.787+0000 7f9b589f4740 1 -- 172.21.15.84:0/2495954423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b932700 msgr2=0x56084b940fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.787+0000 7f9b589f4740 1 --2- 172.21.15.84:0/2495954423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b932700 0x56084b940fb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000cc10 tx=0x7f9b4000e720 comp rx=0 tx=0).stop 2024-04-26T18:28:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.787+0000 7f9b589f4740 1 -- 172.21.15.84:0/2495954423 shutdown_connections 2024-04-26T18:28:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.787+0000 7f9b589f4740 1 --2- 172.21.15.84:0/2495954423 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b9414f0 0x56084b9438e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.787+0000 7f9b589f4740 1 --2- 172.21.15.84:0/2495954423 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b932700 0x56084b940fb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.787+0000 7f9b589f4740 1 --2- 172.21.15.84:0/2495954423 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 0x56084b9321c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.787+0000 7f9b589f4740 1 -- 172.21.15.84:0/2495954423 >> 172.21.15.84:0/2495954423 conn(0x56084b92e410 msgr2=0x56084b92e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.787+0000 7f9b589f4740 1 -- 172.21.15.84:0/2495954423 shutdown_connections 2024-04-26T18:28:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.787+0000 7f9b589f4740 1 -- 172.21.15.84:0/2495954423 wait complete. 2024-04-26T18:28:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b589f4740 1 -- start start 2024-04-26T18:28:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b589f4740 1 Processor -- start 2024-04-26T18:28:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b589f4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 0x56084b93e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b589f4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b932700 0x56084b93e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b589f4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 0x56084b93ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b589f4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56084b944120 con 0x56084b9414f0 2024-04-26T18:28:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b589f4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56084b81fb60 con 0x56084b931de0 2024-04-26T18:28:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b589f4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56084b9442a0 con 0x56084b932700 2024-04-26T18:28:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b589f4740 10 client.0 _ll_register_callbacks cb 0x56084b8442b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:52.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b4ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 0x56084b93ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b4ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 0x56084b93ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44436/0 (socket says 172.21.15.84:44436) 2024-04-26T18:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b4ffff640 1 -- 172.21.15.84:0/899752766 learned_addr learned my addr 172.21.15.84:0/899752766 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b4f7fe640 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 0x56084b93e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b4effd640 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b932700 0x56084b93e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b4ffff640 1 -- 172.21.15.84:0/899752766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b932700 msgr2=0x56084b93e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b4ffff640 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b932700 0x56084b93e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b4ffff640 1 -- 172.21.15.84:0/899752766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 msgr2=0x56084b93e3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b4ffff640 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 0x56084b93e3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.788+0000 7f9b4ffff640 1 -- 172.21.15.84:0/899752766 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084b5aa210 con 0x56084b9414f0 2024-04-26T18:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b4effd640 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b932700 0x56084b93e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b4ffff640 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 0x56084b93ee50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9b480119b0 tx=0x7f9b48009720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9de49dd7a1184f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b2effd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:52.796 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b2effd640 1 -- 172.21.15.84:0/899752766 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48010de0 con 0x56084b9414f0 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b2effd640 1 -- 172.21.15.84:0/899752766 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f9b4801d420 con 0x56084b9414f0 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b589f4740 1 -- 172.21.15.84:0/899752766 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b40004c50 con 0x56084b9414f0 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b2effd640 1 -- 172.21.15.84:0/899752766 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9b4801d620 con 0x56084b9414f0 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b589f4740 10 client.15543 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b589f4740 1 -- 172.21.15.84:0/899752766 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56084b676620 con 0x56084b9414f0 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b2effd640 1 -- 172.21.15.84:0/899752766 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b40004c50 con 0x56084b9414f0 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b2effd640 1 -- 172.21.15.84:0/899752766 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9b480278b0 con 0x56084b9414f0 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b589f4740 10 client.15543 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b589f4740 10 client.15543 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b589f4740 20 client.15543 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b589f4740 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56084b951690 0x56084b953a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b589f4740 4 client.15543 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.789+0000 7f9b589f4740 1 -- 172.21.15.84:0/899752766 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56084b8a6070 con 0x56084b951690 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.790+0000 7f9b4f7fe640 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56084b951690 0x56084b953a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.797 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.790+0000 7f9b4f7fe640 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56084b951690 0x56084b953a80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f813222d80221bd server_cookie=3d1e855cc911e5e0 in_seq=0 out_seq=0 2024-04-26T18:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.790+0000 7f9b2effd640 10 client.15543 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b2effd640 1 -- 172.21.15.84:0/899752766 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56084b8a6070 con 0x56084b951690 2024-04-26T18:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b2effd640 10 client.15543 handle_command_reply: tid=0 2024-04-26T18:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b589f4740 1 client.15543 shutdown 2024-04-26T18:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b589f4740 1 -- 172.21.15.84:0/899752766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 msgr2=0x56084b93ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b589f4740 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 0x56084b93ee50 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9b480119b0 tx=0x7f9b48009720 comp rx=0 tx=0).stop 2024-04-26T18:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b589f4740 1 -- 172.21.15.84:0/899752766 shutdown_connections 2024-04-26T18:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b589f4740 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56084b951690 0x56084b953a80 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b589f4740 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 0x56084b93ee50 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b589f4740 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b932700 0x56084b93e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b2effd640 0 client.15543 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b589f4740 1 --2- 172.21.15.84:0/899752766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 0x56084b93e3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b589f4740 1 -- 172.21.15.84:0/899752766 >> 172.21.15.84:0/899752766 conn(0x56084b92e410 msgr2=0x56084b934220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b589f4740 1 -- 172.21.15.84:0/899752766 shutdown_connections 2024-04-26T18:28:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b589f4740 1 -- 172.21.15.84:0/899752766 wait complete. 2024-04-26T18:28:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.791+0000 7f9b589f4740 20 client.15543 trim_cache size 0 max 16384 2024-04-26T18:28:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b589f4740 1 Processor -- start 2024-04-26T18:28:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b589f4740 1 -- start start 2024-04-26T18:28:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b589f4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b931de0 0x56084b94d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b589f4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b932700 0x56084b92d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b589f4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b9414f0 0x56084b959390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b589f4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501ac310 con 0x56084b931de0 2024-04-26T18:28:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b589f4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501ade50 con 0x56084b932700 2024-04-26T18:28:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b589f4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501ac9e0 con 0x56084b9414f0 2024-04-26T18:28:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b4f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b931de0 0x56084b94d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b4f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b931de0 0x56084b94d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44438/0 (socket says 172.21.15.84:44438) 2024-04-26T18:28:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b4f7fe640 1 -- 172.21.15.84:0/2968866877 learned_addr learned my addr 172.21.15.84:0/2968866877 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b4effd640 1 --2- 172.21.15.84:0/2968866877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b932700 0x56084b92d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b4f7fe640 1 -- 172.21.15.84:0/2968866877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b9414f0 msgr2=0x56084b959390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b4ffff640 1 --2- 172.21.15.84:0/2968866877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b9414f0 0x56084b959390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b4f7fe640 1 --2- 172.21.15.84:0/2968866877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b9414f0 0x56084b959390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b4f7fe640 1 -- 172.21.15.84:0/2968866877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b932700 msgr2=0x56084b92d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b4f7fe640 1 --2- 172.21.15.84:0/2968866877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b932700 0x56084b92d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b4f7fe640 1 -- 172.21.15.84:0/2968866877 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56084b932530 con 0x56084b931de0 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.806+0000 7f9b4effd640 1 --2- 172.21.15.84:0/2968866877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b932700 0x56084b92d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.807+0000 7f9b4f7fe640 1 --2- 172.21.15.84:0/2968866877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b931de0 0x56084b94d020 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000f010 tx=0x7f9b4000c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee57e8bbe1a550 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.807+0000 7f9b357fa640 1 -- 172.21.15.84:0/2968866877 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b4002e070 con 0x56084b931de0 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.807+0000 7f9b357fa640 1 -- 172.21.15.84:0/2968866877 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b40028780 con 0x56084b931de0 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.807+0000 7f9b357fa640 1 -- 172.21.15.84:0/2968866877 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b40026be0 con 0x56084b931de0 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.807+0000 7f9b589f4740 1 -- 172.21.15.84:0/2968866877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b931de0 msgr2=0x56084b94d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.807+0000 7f9b589f4740 1 --2- 172.21.15.84:0/2968866877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b931de0 0x56084b94d020 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9b4000f010 tx=0x7f9b4000c4d0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.807+0000 7f9b589f4740 1 -- 172.21.15.84:0/2968866877 shutdown_connections 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.807+0000 7f9b589f4740 1 --2- 172.21.15.84:0/2968866877 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b9414f0 0x56084b959390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.807+0000 7f9b589f4740 1 --2- 172.21.15.84:0/2968866877 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b932700 0x56084b92d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.807+0000 7f9b589f4740 1 --2- 172.21.15.84:0/2968866877 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b931de0 0x56084b94d020 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.813 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.807+0000 7f9b589f4740 1 -- 172.21.15.84:0/2968866877 >> 172.21.15.84:0/2968866877 conn(0x56084b92e410 msgr2=0x56084b92cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b589f4740 1 -- 172.21.15.84:0/2968866877 shutdown_connections 2024-04-26T18:28:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b589f4740 1 -- 172.21.15.84:0/2968866877 wait complete. 2024-04-26T18:28:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b589f4740 1 -- start start 2024-04-26T18:28:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b589f4740 1 Processor -- start 2024-04-26T18:28:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b589f4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 0x56084b93af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b589f4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b932700 0x56084b93b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b589f4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 0x56084b93ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b589f4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501ad780 con 0x56084b9414f0 2024-04-26T18:28:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b589f4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501ad0b0 con 0x56084b931de0 2024-04-26T18:28:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b589f4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9b501aa310 con 0x56084b932700 2024-04-26T18:28:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b589f4740 10 client.0 _ll_register_callbacks cb 0x7f9b48009ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:52.814 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b4ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 0x56084b93ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b4ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 0x56084b93ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44446/0 (socket says 172.21.15.84:44446) 2024-04-26T18:28:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b4ffff640 1 -- 172.21.15.84:0/4043563791 learned_addr learned my addr 172.21.15.84:0/4043563791 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.808+0000 7f9b4f7fe640 1 --2- 172.21.15.84:0/4043563791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 0x56084b93af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b4ffff640 1 -- 172.21.15.84:0/4043563791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b932700 msgr2=0x56084b93b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b4effd640 1 --2- 172.21.15.84:0/4043563791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b932700 0x56084b93b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b4ffff640 1 --2- 172.21.15.84:0/4043563791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b932700 0x56084b93b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b4ffff640 1 -- 172.21.15.84:0/4043563791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 msgr2=0x56084b93af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b4ffff640 1 --2- 172.21.15.84:0/4043563791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 0x56084b93af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b4ffff640 1 -- 172.21.15.84:0/4043563791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56084b9b6c00 con 0x56084b9414f0 2024-04-26T18:28:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b4ffff640 1 --2- 172.21.15.84:0/4043563791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 0x56084b93ba00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009bb0 tx=0x7f9b48014f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fed5bf098a8f2e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b2effd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b2effd640 1 -- 172.21.15.84:0/4043563791 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b48033040 con 0x56084b9414f0 2024-04-26T18:28:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b589f4740 1 -- 172.21.15.84:0/4043563791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b40028780 con 0x56084b9414f0 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b2effd640 1 -- 172.21.15.84:0/4043563791 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f9b48013040 con 0x56084b9414f0 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b2effd640 1 -- 172.21.15.84:0/4043563791 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9b48045050 con 0x56084b9414f0 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b589f4740 10 client.15549 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b589f4740 1 -- 172.21.15.84:0/4043563791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56084b9b7d60 con 0x56084b9414f0 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b2effd640 1 -- 172.21.15.84:0/4043563791 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b40028780 con 0x56084b9414f0 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.809+0000 7f9b2effd640 1 -- 172.21.15.84:0/4043563791 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9b48036070 con 0x56084b9414f0 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.810+0000 7f9b589f4740 10 client.15549 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.810+0000 7f9b589f4740 10 client.15549 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.810+0000 7f9b589f4740 20 client.15549 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.810+0000 7f9b589f4740 1 --2- 172.21.15.84:0/4043563791 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56084b9b84d0 0x56084b9ba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.810+0000 7f9b589f4740 4 client.15549 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.810+0000 7f9b589f4740 1 -- 172.21.15.84:0/4043563791 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56084b89fee0 con 0x56084b9b84d0 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.810+0000 7f9b4f7fe640 1 --2- 172.21.15.84:0/4043563791 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56084b9b84d0 0x56084b9ba8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.810+0000 7f9b2effd640 10 client.15549 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.810+0000 7f9b4f7fe640 1 --2- 172.21.15.84:0/4043563791 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56084b9b84d0 0x56084b9ba8f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cbb4802e3fe220e server_cookie=54daccdd7e15ee2a in_seq=0 out_seq=0 2024-04-26T18:28:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b2effd640 1 -- 172.21.15.84:0/4043563791 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56084b89fee0 con 0x56084b9b84d0 2024-04-26T18:28:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b2effd640 10 client.15549 handle_command_reply: tid=0 2024-04-26T18:28:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b589f4740 1 client.15549 shutdown 2024-04-26T18:28:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b589f4740 1 -- 172.21.15.84:0/4043563791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 msgr2=0x56084b93ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b589f4740 1 --2- 172.21.15.84:0/4043563791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 0x56084b93ba00 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009bb0 tx=0x7f9b48014f00 comp rx=0 tx=0).stop 2024-04-26T18:28:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b589f4740 1 -- 172.21.15.84:0/4043563791 shutdown_connections 2024-04-26T18:28:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b589f4740 1 --2- 172.21.15.84:0/4043563791 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56084b9b84d0 0x56084b9ba8f0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b589f4740 1 --2- 172.21.15.84:0/4043563791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56084b9414f0 0x56084b93ba00 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b589f4740 1 --2- 172.21.15.84:0/4043563791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56084b932700 0x56084b93b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b2effd640 0 client.15549 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b589f4740 1 --2- 172.21.15.84:0/4043563791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56084b931de0 0x56084b93af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b589f4740 1 -- 172.21.15.84:0/4043563791 >> 172.21.15.84:0/4043563791 conn(0x56084b92e410 msgr2=0x56084b93ec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b589f4740 1 -- 172.21.15.84:0/4043563791 shutdown_connections 2024-04-26T18:28:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b589f4740 1 -- 172.21.15.84:0/4043563791 wait complete. 2024-04-26T18:28:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.814+0000 7f9b589f4740 20 client.15549 trim_cache size 0 max 16384 2024-04-26T18:28:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:28:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:28:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:28:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:28:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.815+0000 7f9b55cdc640 1 -- 172.21.15.84:0/3123227061 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9b380a3f70 msgr2=0x7f9b380a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.815+0000 7f9b55cdc640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9b380a3f70 0x7f9b380a6430 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9b48005fd0 tx=0x7f9b48004ab0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.815+0000 7f9b55cdc640 1 -- 172.21.15.84:0/3123227061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b50103300 msgr2=0x7f9b50199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.815+0000 7f9b55cdc640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b50103300 0x7f9b50199570 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9b44004770 tx=0x7f9b440042f0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.815+0000 7f9b55cdc640 1 -- 172.21.15.84:0/3123227061 shutdown_connections 2024-04-26T18:28:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.815+0000 7f9b55cdc640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9b380a3f70 0x7f9b380a6430 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.815+0000 7f9b55cdc640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9b50108950 0x7f9b5019d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.815+0000 7f9b55cdc640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9b50103300 0x7f9b50199570 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.815+0000 7f9b55cdc640 1 --2- 172.21.15.84:0/3123227061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9b50102940 0x7f9b50199030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.815+0000 7f9b55cdc640 1 -- 172.21.15.84:0/3123227061 >> 172.21.15.84:0/3123227061 conn(0x7f9b50079dc0 msgr2=0x7f9b50100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.815+0000 7f9b55cdc640 1 -- 172.21.15.84:0/3123227061 shutdown_connections 2024-04-26T18:28:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:28:52.815+0000 7f9b55cdc640 1 -- 172.21.15.84:0/3123227061 wait complete. 2024-04-26T18:28:53.464 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-26T18:28:53.464 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-26T18:28:53.465 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:28:53.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:53 smithi103 ceph-mon[32727]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:28:53.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:53 smithi174 ceph-mon[32705]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:28:53.714 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:53.836 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:53 smithi084 ceph-mon[29141]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:28:55.523 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:55 smithi084 ceph-mon[29141]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-26T18:28:55.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.625+0000 7f6558fa2700 1 -- 172.21.15.84:0/2714704824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6554103e30 msgr2=0x7f6554110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:55.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.625+0000 7f6558fa2700 1 --2- 172.21.15.84:0/2714704824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6554103e30 0x7f6554110290 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f654c00b0a0 tx=0x7f654c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:28:55.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.625+0000 7f6558fa2700 1 -- 172.21.15.84:0/2714704824 shutdown_connections 2024-04-26T18:28:55.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.625+0000 7f6558fa2700 1 --2- 172.21.15.84:0/2714704824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6554103e30 0x7f6554110290 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:55.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.625+0000 7f6558fa2700 1 --2- 172.21.15.84:0/2714704824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65541094c0 0x7f65541098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:55.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.625+0000 7f6558fa2700 1 --2- 172.21.15.84:0/2714704824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6554103470 0x7f65541038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:55.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.625+0000 7f6558fa2700 1 -- 172.21.15.84:0/2714704824 >> 172.21.15.84:0/2714704824 conn(0x7f655407a580 msgr2=0x7f655407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:55.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.625+0000 7f6558fa2700 1 -- 172.21.15.84:0/2714704824 shutdown_connections 2024-04-26T18:28:55.626 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.625+0000 7f6558fa2700 1 -- 172.21.15.84:0/2714704824 wait complete. 2024-04-26T18:28:55.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:55 smithi103 ceph-mon[32727]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-26T18:28:55.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.626+0000 7f6558fa2700 1 Processor -- start 2024-04-26T18:28:55.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.626+0000 7f6558fa2700 1 -- start start 2024-04-26T18:28:55.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.626+0000 7f6558fa2700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6554103470 0x7f655419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:55.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.626+0000 7f6558fa2700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6554103e30 0x7f655419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:55.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.626+0000 7f6558fa2700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65541094c0 0x7f65541998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:55.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.626+0000 7f6558fa2700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f65541135c0 con 0x7f65541094c0 2024-04-26T18:28:55.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.626+0000 7f6558fa2700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6554113440 con 0x7f6554103e30 2024-04-26T18:28:55.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.626+0000 7f6558fa2700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6554113740 con 0x7f6554103470 2024-04-26T18:28:55.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.626+0000 7f6552d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65541094c0 0x7f65541998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:55.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.626+0000 7f6552d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65541094c0 0x7f65541998a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44468/0 (socket says 172.21.15.84:44468) 2024-04-26T18:28:55.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.626+0000 7f6552d9d700 1 -- 172.21.15.84:0/2438826546 learned_addr learned my addr 172.21.15.84:0/2438826546 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.626+0000 7f6551d9b700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6554103e30 0x7f655419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.626+0000 7f655259c700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6554103470 0x7f655419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.627+0000 7f6552d9d700 1 -- 172.21.15.84:0/2438826546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6554103470 msgr2=0x7f655419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.627+0000 7f6552d9d700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6554103470 0x7f655419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.627+0000 7f6552d9d700 1 -- 172.21.15.84:0/2438826546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6554103e30 msgr2=0x7f655419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.627+0000 7f6552d9d700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6554103e30 0x7f655419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.627+0000 7f6552d9d700 1 -- 172.21.15.84:0/2438826546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655419a000 con 0x7f65541094c0 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.627+0000 7f655259c700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6554103470 0x7f655419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.627+0000 7f6552d9d700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65541094c0 0x7f65541998a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f654c00b830 tx=0x7f654c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.627+0000 7f65437fe700 1 -- 172.21.15.84:0/2438826546 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c021070 con 0x7f65541094c0 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.627+0000 7f65437fe700 1 -- 172.21.15.84:0/2438826546 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f654c007e30 con 0x7f65541094c0 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.627+0000 7f6558fa2700 1 -- 172.21.15.84:0/2438826546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655419a290 con 0x7f65541094c0 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.627+0000 7f65437fe700 1 -- 172.21.15.84:0/2438826546 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c0047d0 con 0x7f65541094c0 2024-04-26T18:28:55.630 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.627+0000 7f6558fa2700 1 -- 172.21.15.84:0/2438826546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65541a65b0 con 0x7f65541094c0 2024-04-26T18:28:55.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.628+0000 7f6558fa2700 1 -- 172.21.15.84:0/2438826546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f655406a110 con 0x7f65541094c0 2024-04-26T18:28:55.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.629+0000 7f65437fe700 1 -- 172.21.15.84:0/2438826546 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f654c007980 con 0x7f65541094c0 2024-04-26T18:28:55.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.630+0000 7f65437fe700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f653c0a3f60 0x7f653c0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:55.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.630+0000 7f655259c700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f653c0a3f60 0x7f653c0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:55.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.630+0000 7f65437fe700 1 -- 172.21.15.84:0/2438826546 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f654c00f020 con 0x7f65541094c0 2024-04-26T18:28:55.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.630+0000 7f655259c700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f653c0a3f60 0x7f653c0a6420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f655419a9e0 tx=0x7f6548005d80 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:55.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.634+0000 7f65437fe700 1 -- 172.21.15.84:0/2438826546 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f654c02a020 con 0x7f65541094c0 2024-04-26T18:28:55.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:55 smithi174 ceph-mon[32705]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 511 B/s wr, 12 op/s 2024-04-26T18:28:55.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.881+0000 7f6558fa2700 1 -- 172.21.15.84:0/2438826546 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f655419add0 con 0x7f65541094c0 2024-04-26T18:28:55.882 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.882+0000 7f65437fe700 1 -- 172.21.15.84:0/2438826546 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f654c09e910 con 0x7f65541094c0 2024-04-26T18:28:55.882 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:28:55.882 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:28:55.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.884+0000 7f6558fa2700 1 -- 172.21.15.84:0/2438826546 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f653c0a3f60 msgr2=0x7f653c0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:55.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.884+0000 7f6558fa2700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f653c0a3f60 0x7f653c0a6420 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f655419a9e0 tx=0x7f6548005d80 comp rx=0 tx=0).stop 2024-04-26T18:28:55.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.884+0000 7f6558fa2700 1 -- 172.21.15.84:0/2438826546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65541094c0 msgr2=0x7f65541998a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:55.884 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.884+0000 7f6558fa2700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65541094c0 0x7f65541998a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f654c00b830 tx=0x7f654c004ce0 comp rx=0 tx=0).stop 2024-04-26T18:28:55.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.884+0000 7f6558fa2700 1 -- 172.21.15.84:0/2438826546 shutdown_connections 2024-04-26T18:28:55.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.884+0000 7f6558fa2700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f653c0a3f60 0x7f653c0a6420 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:55.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.884+0000 7f6558fa2700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65541094c0 0x7f65541998a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:55.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.884+0000 7f6558fa2700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6554103470 0x7f655419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:55.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.884+0000 7f6558fa2700 1 --2- 172.21.15.84:0/2438826546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6554103e30 0x7f655419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:55.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.884+0000 7f6558fa2700 1 -- 172.21.15.84:0/2438826546 >> 172.21.15.84:0/2438826546 conn(0x7f655407a580 msgr2=0x7f65541020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:55.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.884+0000 7f6558fa2700 1 -- 172.21.15.84:0/2438826546 shutdown_connections 2024-04-26T18:28:55.885 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:55.884+0000 7f6558fa2700 1 -- 172.21.15.84:0/2438826546 wait complete. 2024-04-26T18:28:55.885 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:28:56.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:56 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2438826546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:56.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:56 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2438826546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:56.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:56 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2438826546' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:28:56.691 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-26T18:28:56.692 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-26T18:28:56.930 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:28:57.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:57 smithi084 ceph-mon[29141]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-26T18:28:57.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:57 smithi103 ceph-mon[32727]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-26T18:28:57.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:57 smithi174 ceph-mon[32705]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 511 B/s wr, 11 op/s 2024-04-26T18:28:58.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.678+0000 7f8341fa0700 1 -- 172.21.15.84:0/1245789501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833c102f30 msgr2=0x7f833c106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.678+0000 7f8341fa0700 1 --2- 172.21.15.84:0/1245789501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833c102f30 0x7f833c106f70 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f83300099c0 tx=0x7f8330009cd0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.678+0000 7f8341fa0700 1 -- 172.21.15.84:0/1245789501 shutdown_connections 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.678+0000 7f8341fa0700 1 --2- 172.21.15.84:0/1245789501 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833c107650 0x7f833c10f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.678+0000 7f8341fa0700 1 --2- 172.21.15.84:0/1245789501 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f833c102580 0x7f833c102960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.678+0000 7f8341fa0700 1 --2- 172.21.15.84:0/1245789501 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833c102f30 0x7f833c106f70 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.678+0000 7f8341fa0700 1 -- 172.21.15.84:0/1245789501 >> 172.21.15.84:0/1245789501 conn(0x7f833c0792c0 msgr2=0x7f833c0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.678+0000 7f8341fa0700 1 -- 172.21.15.84:0/1245789501 shutdown_connections 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.678+0000 7f8341fa0700 1 -- 172.21.15.84:0/1245789501 wait complete. 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.679+0000 7f8341fa0700 1 Processor -- start 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.679+0000 7f8341fa0700 1 -- start start 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.679+0000 7f8341fa0700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f833c102580 0x7f833c19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.679+0000 7f8341fa0700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833c102f30 0x7f833c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.679+0000 7f8341fa0700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833c107650 0x7f833c1a5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.679+0000 7f8341fa0700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1124c0 con 0x7f833c107650 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.679+0000 7f8341fa0700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f833c112340 con 0x7f833c102580 2024-04-26T18:28:58.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.679+0000 7f8341fa0700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f833c112640 con 0x7f833c102f30 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.679+0000 7f833bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833c107650 0x7f833c1a5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.679+0000 7f833bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833c107650 0x7f833c1a5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45190/0 (socket says 172.21.15.84:45190) 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.679+0000 7f833bfff700 1 -- 172.21.15.84:0/2053455726 learned_addr learned my addr 172.21.15.84:0/2053455726 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.680+0000 7f833b7fe700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f833c102580 0x7f833c19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.680+0000 7f833affd700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833c102f30 0x7f833c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.680+0000 7f833bfff700 1 -- 172.21.15.84:0/2053455726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833c102f30 msgr2=0x7f833c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.680+0000 7f833bfff700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833c102f30 0x7f833c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.680+0000 7f833bfff700 1 -- 172.21.15.84:0/2053455726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f833c102580 msgr2=0x7f833c19a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.680+0000 7f833bfff700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f833c102580 0x7f833c19a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.680+0000 7f833bfff700 1 -- 172.21.15.84:0/2053455726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f833c1a5e40 con 0x7f833c107650 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.680+0000 7f833bfff700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833c107650 0x7f833c1a5900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8334007ee0 tx=0x7f833400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.680+0000 7f8338ff9700 1 -- 172.21.15.84:0/2053455726 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833400dbd0 con 0x7f833c107650 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.680+0000 7f8341fa0700 1 -- 172.21.15.84:0/2053455726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f833c1a6070 con 0x7f833c107650 2024-04-26T18:28:58.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.680+0000 7f8338ff9700 1 -- 172.21.15.84:0/2053455726 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f833400dd70 con 0x7f833c107650 2024-04-26T18:28:58.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.680+0000 7f8338ff9700 1 -- 172.21.15.84:0/2053455726 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8334005990 con 0x7f833c107650 2024-04-26T18:28:58.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.680+0000 7f8341fa0700 1 -- 172.21.15.84:0/2053455726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f833c1a6580 con 0x7f833c107650 2024-04-26T18:28:58.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.682+0000 7f8338ff9700 1 -- 172.21.15.84:0/2053455726 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f833401a020 con 0x7f833c107650 2024-04-26T18:28:58.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.683+0000 7f8338ff9700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f83240a4000 0x7f83240a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.683+0000 7f833b7fe700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f83240a4000 0x7f83240a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.683+0000 7f8338ff9700 1 -- 172.21.15.84:0/2053455726 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f83340d0260 con 0x7f833c107650 2024-04-26T18:28:58.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.683+0000 7f833b7fe700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f83240a4000 0x7f83240a64c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f832c009c60 tx=0x7f832c009260 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:58.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f834c832b80 1 Processor -- start 2024-04-26T18:28:58.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f834c832b80 1 -- start start 2024-04-26T18:28:58.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f834c832b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bb7230 0x558ce4bb7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f834c832b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 0x558ce4bc24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f834c832b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bc2a00 0x558ce4bc4df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.685 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f834c832b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558ce4b0a8d0 con 0x558ce4bc2a00 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f834c832b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558ce4aa9770 con 0x558ce4bb7230 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f834c832b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558ce4b06890 con 0x558ce4bb7b50 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f833bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bc2a00 0x558ce4bc4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f833bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bc2a00 0x558ce4bc4df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45192/0 (socket says 172.21.15.84:45192) 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f833bfff700 1 -- 172.21.15.84:0/1665510499 learned_addr learned my addr 172.21.15.84:0/1665510499 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f833b7fe700 1 --2- 172.21.15.84:0/1665510499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bb7230 0x558ce4bb7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f833affd700 1 --2- 172.21.15.84:0/1665510499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 0x558ce4bc24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f833bfff700 1 -- 172.21.15.84:0/1665510499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 msgr2=0x558ce4bc24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f833bfff700 1 --2- 172.21.15.84:0/1665510499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 0x558ce4bc24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f833bfff700 1 -- 172.21.15.84:0/1665510499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bb7230 msgr2=0x558ce4bb7610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f833bfff700 1 --2- 172.21.15.84:0/1665510499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bb7230 0x558ce4bb7610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f833bfff700 1 -- 172.21.15.84:0/1665510499 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce4a57a60 con 0x558ce4bc2a00 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f833b7fe700 1 --2- 172.21.15.84:0/1665510499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bb7230 0x558ce4bb7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:58.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.684+0000 7f833affd700 1 --2- 172.21.15.84:0/1665510499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 0x558ce4bc24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:28:58.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.685+0000 7f833bfff700 1 --2- 172.21.15.84:0/1665510499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bc2a00 0x558ce4bc4df0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8334098450 tx=0x7f833409bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4dc60992d2591a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:58.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.685+0000 7f8321ffb700 1 -- 172.21.15.84:0/1665510499 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83340b5070 con 0x558ce4bc2a00 2024-04-26T18:28:58.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.685+0000 7f8321ffb700 1 -- 172.21.15.84:0/1665510499 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83340a03a0 con 0x558ce4bc2a00 2024-04-26T18:28:58.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.685+0000 7f8321ffb700 1 -- 172.21.15.84:0/1665510499 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f833409c250 con 0x558ce4bc2a00 2024-04-26T18:28:58.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.685+0000 7f834c832b80 1 -- 172.21.15.84:0/1665510499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bc2a00 msgr2=0x558ce4bc4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.685+0000 7f834c832b80 1 --2- 172.21.15.84:0/1665510499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bc2a00 0x558ce4bc4df0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8334098450 tx=0x7f833409bc30 comp rx=0 tx=0).stop 2024-04-26T18:28:58.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.685+0000 7f834c832b80 1 -- 172.21.15.84:0/1665510499 shutdown_connections 2024-04-26T18:28:58.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.685+0000 7f834c832b80 1 --2- 172.21.15.84:0/1665510499 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bc2a00 0x558ce4bc4df0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.685+0000 7f834c832b80 1 --2- 172.21.15.84:0/1665510499 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bb7230 0x558ce4bb7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.685+0000 7f834c832b80 1 --2- 172.21.15.84:0/1665510499 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 0x558ce4bc24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.685+0000 7f834c832b80 1 -- 172.21.15.84:0/1665510499 >> 172.21.15.84:0/1665510499 conn(0x558ce4ae4ed0 msgr2=0x558ce4bb3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.685+0000 7f834c832b80 1 -- 172.21.15.84:0/1665510499 shutdown_connections 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.686+0000 7f834c832b80 1 -- 172.21.15.84:0/1665510499 wait complete. 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.686+0000 7f834c832b80 1 -- start start 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.686+0000 7f834c832b80 1 Processor -- start 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.686+0000 7f834c832b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bc2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.686+0000 7f834c832b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 0x558ce4bbf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.686+0000 7f834c832b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bc2a00 0x558ce4bbf970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.686+0000 7f834c832b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558ce4bc55d0 con 0x558ce4bb7230 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.686+0000 7f834c832b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558ce4a70ae0 con 0x558ce4bc2a00 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.686+0000 7f834c832b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558ce4bc5750 con 0x558ce4bb7b50 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.686+0000 7f834c832b80 10 client.0 _ll_register_callbacks cb 0x558ce4a9e710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.686+0000 7f833b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bc2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.686+0000 7f833b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bc2130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45204/0 (socket says 172.21.15.84:45204) 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.686+0000 7f833b7fe700 1 -- 172.21.15.84:0/4186067541 learned_addr learned my addr 172.21.15.84:0/4186067541 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:58.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f833affd700 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 0x558ce4bbf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f833bfff700 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bc2a00 0x558ce4bbf970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f833b7fe700 1 -- 172.21.15.84:0/4186067541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 msgr2=0x558ce4bbf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f833b7fe700 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 0x558ce4bbf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f833b7fe700 1 -- 172.21.15.84:0/4186067541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bc2a00 msgr2=0x558ce4bbf970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f833b7fe700 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bc2a00 0x558ce4bbf970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f833b7fe700 1 -- 172.21.15.84:0/4186067541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce4bbffb0 con 0x558ce4bb7230 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f833bfff700 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bc2a00 0x558ce4bbf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f833b7fe700 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bc2130 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f832c01dbf0 tx=0x7f832c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a7c6d6ef3f81df7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f831affd700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f831affd700 1 -- 172.21.15.84:0/4186067541 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c00ed00 con 0x558ce4bb7230 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f831affd700 1 -- 172.21.15.84:0/4186067541 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f832c01b400 con 0x558ce4bb7230 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f834c832b80 1 -- 172.21.15.84:0/4186067541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83340a03a0 con 0x558ce4bb7230 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f831affd700 1 -- 172.21.15.84:0/4186067541 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f832c01b600 con 0x558ce4bb7230 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f834c832b80 10 client.15567 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:58.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f834c832b80 1 -- 172.21.15.84:0/4186067541 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ce4bd0020 con 0x558ce4bb7230 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.687+0000 7f831affd700 1 -- 172.21.15.84:0/4186067541 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83340a03a0 con 0x558ce4bb7230 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.688+0000 7f831affd700 1 -- 172.21.15.84:0/4186067541 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f832c02f830 con 0x558ce4bb7230 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.688+0000 7f834c832b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.688+0000 7f834c832b80 10 client.15567 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.688+0000 7f834c832b80 20 client.15567 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.688+0000 7f834c832b80 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558ce4bd2660 0x558ce4bd4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.688+0000 7f834c832b80 4 client.15567 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.688+0000 7f834c832b80 1 -- 172.21.15.84:0/4186067541 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ce4b182b0 con 0x558ce4bd2660 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.688+0000 7f833affd700 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558ce4bd2660 0x558ce4bd4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.688+0000 7f831affd700 10 client.15567 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.688+0000 7f833affd700 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558ce4bd2660 0x558ce4bd4a50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98b1ec642f682ab9 server_cookie=418c803d583d9277 in_seq=0 out_seq=0 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.689+0000 7f831affd700 1 -- 172.21.15.84:0/4186067541 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ce4b182b0 con 0x558ce4bd2660 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.689+0000 7f831affd700 10 client.15567 handle_command_reply: tid=0 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.689+0000 7f834c832b80 1 client.15567 shutdown 2024-04-26T18:28:58.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.689+0000 7f834c832b80 1 -- 172.21.15.84:0/4186067541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 msgr2=0x558ce4bc2130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.689+0000 7f834c832b80 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bc2130 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f832c01dbf0 tx=0x7f832c0047c0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.689+0000 7f834c832b80 1 -- 172.21.15.84:0/4186067541 shutdown_connections 2024-04-26T18:28:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.689+0000 7f834c832b80 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558ce4bd2660 0x558ce4bd4a50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.689+0000 7f834c832b80 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bc2130 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.689+0000 7f831affd700 0 client.15567 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.689+0000 7f834c832b80 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bc2a00 0x558ce4bbf970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.689+0000 7f834c832b80 1 --2- 172.21.15.84:0/4186067541 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 0x558ce4bbf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.689+0000 7f834c832b80 1 -- 172.21.15.84:0/4186067541 >> 172.21.15.84:0/4186067541 conn(0x558ce4ae4ed0 msgr2=0x558ce4bb3350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.690+0000 7f834c832b80 1 -- 172.21.15.84:0/4186067541 shutdown_connections 2024-04-26T18:28:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.690+0000 7f834c832b80 1 -- 172.21.15.84:0/4186067541 wait complete. 2024-04-26T18:28:58.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.690+0000 7f834c832b80 20 client.15567 trim_cache size 0 max 16384 2024-04-26T18:28:58.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f834c832b80 1 Processor -- start 2024-04-26T18:28:58.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f834c832b80 1 -- start start 2024-04-26T18:28:58.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f834c832b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bb38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f834c832b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bb7b50 0x558ce4bb98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f834c832b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bc2a00 0x558ce4bce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f834c832b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1adc90 con 0x558ce4bb7230 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f834c832b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1af7d0 con 0x558ce4bb7b50 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f834c832b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1ae360 con 0x558ce4bc2a00 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f833b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bb38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f833b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bb38f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45218/0 (socket says 172.21.15.84:45218) 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f833b7fe700 1 -- 172.21.15.84:0/2809452933 learned_addr learned my addr 172.21.15.84:0/2809452933 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f833b7fe700 1 -- 172.21.15.84:0/2809452933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bc2a00 msgr2=0x558ce4bce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f833b7fe700 1 --2- 172.21.15.84:0/2809452933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bc2a00 0x558ce4bce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f833b7fe700 1 -- 172.21.15.84:0/2809452933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bb7b50 msgr2=0x558ce4bb98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f833b7fe700 1 --2- 172.21.15.84:0/2809452933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bb7b50 0x558ce4bb98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f833b7fe700 1 -- 172.21.15.84:0/2809452933 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ce4bd5030 con 0x558ce4bb7230 2024-04-26T18:28:58.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f833b7fe700 1 --2- 172.21.15.84:0/2809452933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bb38f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f832c005a60 tx=0x7f832c005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40545de1c46ed044 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:58.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.709+0000 7f83217fa700 1 -- 172.21.15.84:0/2809452933 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c00eb00 con 0x558ce4bb7230 2024-04-26T18:28:58.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.710+0000 7f83217fa700 1 -- 172.21.15.84:0/2809452933 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f832c0044f0 con 0x558ce4bb7230 2024-04-26T18:28:58.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.710+0000 7f83217fa700 1 -- 172.21.15.84:0/2809452933 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c02fa10 con 0x558ce4bb7230 2024-04-26T18:28:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.710+0000 7f834c832b80 1 -- 172.21.15.84:0/2809452933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 msgr2=0x558ce4bb38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.710+0000 7f834c832b80 1 --2- 172.21.15.84:0/2809452933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bb38f0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f832c005a60 tx=0x7f832c005c20 comp rx=0 tx=0).stop 2024-04-26T18:28:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.710+0000 7f834c832b80 1 -- 172.21.15.84:0/2809452933 shutdown_connections 2024-04-26T18:28:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.710+0000 7f834c832b80 1 --2- 172.21.15.84:0/2809452933 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bb38f0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.710+0000 7f834c832b80 1 --2- 172.21.15.84:0/2809452933 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bc2a00 0x558ce4bce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.710+0000 7f834c832b80 1 --2- 172.21.15.84:0/2809452933 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bb7b50 0x558ce4bb98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.710+0000 7f834c832b80 1 -- 172.21.15.84:0/2809452933 >> 172.21.15.84:0/2809452933 conn(0x558ce4ae4ed0 msgr2=0x558ce4bb91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.710+0000 7f834c832b80 1 -- 172.21.15.84:0/2809452933 shutdown_connections 2024-04-26T18:28:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.710+0000 7f834c832b80 1 -- 172.21.15.84:0/2809452933 wait complete. 2024-04-26T18:28:58.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f834c832b80 1 -- start start 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f834c832b80 1 Processor -- start 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f834c832b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bcaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f834c832b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 0x558ce4bc79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f834c832b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bc2a00 0x558ce4bc7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f834c832b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1af100 con 0x558ce4bb7230 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f834c832b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1aea30 con 0x558ce4bc2a00 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f834c832b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f833c1abc30 con 0x558ce4bb7b50 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f834c832b80 10 client.0 _ll_register_callbacks cb 0x558ce4a9e710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f833b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bcaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f833b7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bcaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45220/0 (socket says 172.21.15.84:45220) 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f833b7fe700 1 -- 172.21.15.84:0/3784762947 learned_addr learned my addr 172.21.15.84:0/3784762947 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f833bfff700 1 --2- 172.21.15.84:0/3784762947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bc2a00 0x558ce4bc7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.711+0000 7f833affd700 1 --2- 172.21.15.84:0/3784762947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 0x558ce4bc79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f833b7fe700 1 -- 172.21.15.84:0/3784762947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 msgr2=0x558ce4bc79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f833b7fe700 1 --2- 172.21.15.84:0/3784762947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 0x558ce4bc79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f833b7fe700 1 -- 172.21.15.84:0/3784762947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bc2a00 msgr2=0x558ce4bc7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f833b7fe700 1 --2- 172.21.15.84:0/3784762947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bc2a00 0x558ce4bc7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f833b7fe700 1 -- 172.21.15.84:0/3784762947 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ce4bc8770 con 0x558ce4bb7230 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f833b7fe700 1 --2- 172.21.15.84:0/3784762947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bcaeb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f832c009b20 tx=0x7f832c01b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d47fb2c8c57642b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f831affd700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f831affd700 1 -- 172.21.15.84:0/3784762947 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f832c02f690 con 0x558ce4bb7230 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f831affd700 1 -- 172.21.15.84:0/3784762947 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f832c02f830 con 0x558ce4bb7230 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f834c832b80 1 -- 172.21.15.84:0/3784762947 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f832c0044f0 con 0x558ce4bb7230 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f831affd700 1 -- 172.21.15.84:0/3784762947 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f832c046050 con 0x558ce4bb7230 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f834c832b80 10 client.15573 fetch_fsmap learned FSMap version 29 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f834c832b80 1 -- 172.21.15.84:0/3784762947 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ce4c39bd0 con 0x558ce4bb7230 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.712+0000 7f831affd700 1 -- 172.21.15.84:0/3784762947 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f832c0044f0 con 0x558ce4bb7230 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.713+0000 7f831affd700 1 -- 172.21.15.84:0/3784762947 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f832c01bb70 con 0x558ce4bb7230 2024-04-26T18:28:58.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.713+0000 7f834c832b80 10 client.15573 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:28:58.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.713+0000 7f834c832b80 10 client.15573 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:28:58.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.713+0000 7f834c832b80 20 client.15573 populate_metadata read hostname 'smithi084' 2024-04-26T18:28:58.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.713+0000 7f834c832b80 1 --2- 172.21.15.84:0/3784762947 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558ce4c3a260 0x558ce4c3c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:28:58.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.713+0000 7f834c832b80 4 client.15573 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-26T18:28:58.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.713+0000 7f834c832b80 1 -- 172.21.15.84:0/3784762947 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x558ce4b12350 con 0x558ce4c3a260 2024-04-26T18:28:58.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.713+0000 7f833affd700 1 --2- 172.21.15.84:0/3784762947 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558ce4c3a260 0x558ce4c3c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:28:58.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.714+0000 7f831affd700 10 client.15573 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:58.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.714+0000 7f833affd700 1 --2- 172.21.15.84:0/3784762947 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558ce4c3a260 0x558ce4c3c680 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b5fd207922d3c75 server_cookie=e40142db0d972051 in_seq=0 out_seq=0 2024-04-26T18:28:58.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f831affd700 1 -- 172.21.15.84:0/3784762947 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x558ce4b12350 con 0x558ce4c3a260 2024-04-26T18:28:58.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f831affd700 10 client.15573 handle_command_reply: tid=0 2024-04-26T18:28:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f834c832b80 1 client.15573 shutdown 2024-04-26T18:28:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f834c832b80 1 -- 172.21.15.84:0/3784762947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 msgr2=0x558ce4bcaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f834c832b80 1 --2- 172.21.15.84:0/3784762947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bcaeb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f832c009b20 tx=0x7f832c01b400 comp rx=0 tx=0).stop 2024-04-26T18:28:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f834c832b80 1 -- 172.21.15.84:0/3784762947 shutdown_connections 2024-04-26T18:28:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f834c832b80 1 --2- 172.21.15.84:0/3784762947 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558ce4c3a260 0x558ce4c3c680 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f834c832b80 1 --2- 172.21.15.84:0/3784762947 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558ce4bb7230 0x558ce4bcaeb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f834c832b80 1 --2- 172.21.15.84:0/3784762947 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558ce4bc2a00 0x558ce4bc7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f831affd700 0 client.15573 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:28:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f834c832b80 1 --2- 172.21.15.84:0/3784762947 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558ce4bb7b50 0x558ce4bc79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f834c832b80 1 -- 172.21.15.84:0/3784762947 >> 172.21.15.84:0/3784762947 conn(0x558ce4ae4ed0 msgr2=0x558ce4bb9a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f834c832b80 1 -- 172.21.15.84:0/3784762947 shutdown_connections 2024-04-26T18:28:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f834c832b80 1 -- 172.21.15.84:0/3784762947 wait complete. 2024-04-26T18:28:58.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.726+0000 7f834c832b80 20 client.15573 trim_cache size 0 max 16384 2024-04-26T18:28:58.728 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:28:58.728 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:28:58.728 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:28:58.728 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 2, 2024-04-26T18:28:58.728 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:28:58.728 INFO:teuthology.orchestra.run.smithi084.stdout: "f5df7d40": { 2024-04-26T18:28:58.728 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 2, 2024-04-26T18:28:58.728 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:28:58.728 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:28:58.728 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:28:58.728 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:28:58.728 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 90.0, 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:28:58.729 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:28:58.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.727+0000 7f8341fa0700 1 -- 172.21.15.84:0/2053455726 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f83240a4000 msgr2=0x7f83240a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.727+0000 7f8341fa0700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f83240a4000 0x7f83240a64c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f832c009c60 tx=0x7f832c009260 comp rx=0 tx=0).stop 2024-04-26T18:28:58.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.727+0000 7f8341fa0700 1 -- 172.21.15.84:0/2053455726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833c107650 msgr2=0x7f833c1a5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:28:58.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.727+0000 7f8341fa0700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833c107650 0x7f833c1a5900 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8334007ee0 tx=0x7f833400d3b0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.727+0000 7f8341fa0700 1 -- 172.21.15.84:0/2053455726 shutdown_connections 2024-04-26T18:28:58.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.727+0000 7f8341fa0700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f83240a4000 0x7f83240a64c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.727+0000 7f8341fa0700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f833c107650 0x7f833c1a5900 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.727+0000 7f8341fa0700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f833c102580 0x7f833c19a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.727+0000 7f8341fa0700 1 --2- 172.21.15.84:0/2053455726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f833c102f30 0x7f833c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:28:58.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.727+0000 7f8341fa0700 1 -- 172.21.15.84:0/2053455726 >> 172.21.15.84:0/2053455726 conn(0x7f833c0792c0 msgr2=0x7f833c100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:28:58.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.727+0000 7f8341fa0700 1 -- 172.21.15.84:0/2053455726 shutdown_connections 2024-04-26T18:28:58.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:28:58.727+0000 7f8341fa0700 1 -- 172.21.15.84:0/2053455726 wait complete. 2024-04-26T18:28:59.557 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:28:59 smithi084 ceph-mon[29141]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:28:59.584 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f5df7d40', quiesce took 6.1 seconds. Will release after: 30.0 2024-04-26T18:28:59.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:28:59 smithi103 ceph-mon[32727]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:28:59.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:28:59 smithi174 ceph-mon[32705]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:29:01.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:01 smithi103 ceph-mon[32727]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-26T18:29:01.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:01 smithi174 ceph-mon[32705]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-26T18:29:01.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:01 smithi084 ceph-mon[29141]: pgmap v34: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-26T18:29:03.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:03 smithi103 ceph-mon[32727]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-26T18:29:03.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:03 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:29:03.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:03 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:29:03.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:03 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:29:03.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:03 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:29:03.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:03 smithi174 ceph-mon[32705]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-26T18:29:03.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:03 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:29:03.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:03 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:29:03.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:03 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:29:03.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:03 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:29:03.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:03 smithi084 ceph-mon[29141]: pgmap v35: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-26T18:29:03.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:03 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:29:03.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:03 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:29:03.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:03 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:29:03.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:03 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:29:05.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:05 smithi103 ceph-mon[32727]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-04-26T18:29:05.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:05 smithi174 ceph-mon[32705]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-04-26T18:29:05.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:05 smithi084 ceph-mon[29141]: pgmap v36: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 28 op/s 2024-04-26T18:29:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.901+0000 7f8d30c7b640 1 -- 172.21.15.84:0/2250991654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d2c101860 msgr2=0x7f8d2c101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.901+0000 7f8d30c7b640 1 --2- 172.21.15.84:0/2250991654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d2c101860 0x7f8d2c101ce0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009930 tx=0x7f8d1402a420 comp rx=0 tx=0).stop 2024-04-26T18:29:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.902+0000 7f8d30c7b640 1 -- 172.21.15.84:0/2250991654 shutdown_connections 2024-04-26T18:29:05.902 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.902+0000 7f8d30c7b640 1 --2- 172.21.15.84:0/2250991654 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d2c102220 0x7f8d2c10e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.902+0000 7f8d30c7b640 1 --2- 172.21.15.84:0/2250991654 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d2c101860 0x7f8d2c101ce0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.902+0000 7f8d30c7b640 1 --2- 172.21.15.84:0/2250991654 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d2c107820 0x7f8d2c107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.902+0000 7f8d30c7b640 1 -- 172.21.15.84:0/2250991654 >> 172.21.15.84:0/2250991654 conn(0x7f8d2c0789b0 msgr2=0x7f8d2c078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.902+0000 7f8d30c7b640 1 -- 172.21.15.84:0/2250991654 shutdown_connections 2024-04-26T18:29:05.903 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.902+0000 7f8d30c7b640 1 -- 172.21.15.84:0/2250991654 wait complete. 2024-04-26T18:29:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.902+0000 7f8d30c7b640 1 Processor -- start 2024-04-26T18:29:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.902+0000 7f8d30c7b640 1 -- start start 2024-04-26T18:29:05.905 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d30c7b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d2c101860 0x7f8d2c199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d30c7b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d2c102220 0x7f8d2c19a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d30c7b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d2c107820 0x7f8d2c1a4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d30c7b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c111ae0 con 0x7f8d2c101860 2024-04-26T18:29:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d30c7b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c111960 con 0x7f8d2c107820 2024-04-26T18:29:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d30c7b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c111c60 con 0x7f8d2c102220 2024-04-26T18:29:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d2a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d2c101860 0x7f8d2c199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d2a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d2c101860 0x7f8d2c199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45236/0 (socket says 172.21.15.84:45236) 2024-04-26T18:29:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d2a59c640 1 -- 172.21.15.84:0/1481255845 learned_addr learned my addr 172.21.15.84:0/1481255845 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d2c107820 0x7f8d2c1a4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d23fff640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d2c102220 0x7f8d2c19a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d2a59c640 1 -- 172.21.15.84:0/1481255845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d2c102220 msgr2=0x7f8d2c19a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:05.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d2a59c640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d2c102220 0x7f8d2c19a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d2a59c640 1 -- 172.21.15.84:0/1481255845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d2c107820 msgr2=0x7f8d2c1a4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d2a59c640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d2c107820 0x7f8d2c1a4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d2a59c640 1 -- 172.21.15.84:0/1481255845 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d2c1a5440 con 0x7f8d2c101860 2024-04-26T18:29:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d2a59c640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d2c101860 0x7f8d2c199e80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00d770 tx=0x7f8d1c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d237fe640 1 -- 172.21.15.84:0/1481255845 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c017070 con 0x7f8d2c101860 2024-04-26T18:29:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d237fe640 1 -- 172.21.15.84:0/1481255845 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d1c0054d0 con 0x7f8d2c101860 2024-04-26T18:29:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d237fe640 1 -- 172.21.15.84:0/1481255845 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c004c30 con 0x7f8d2c101860 2024-04-26T18:29:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.903+0000 7f8d30c7b640 1 -- 172.21.15.84:0/1481255845 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d2c1a5610 con 0x7f8d2c101860 2024-04-26T18:29:05.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.904+0000 7f8d30c7b640 1 -- 172.21.15.84:0/1481255845 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d2c1a5b30 con 0x7f8d2c101860 2024-04-26T18:29:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.906+0000 7f8d33994740 1 Processor -- start 2024-04-26T18:29:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.906+0000 7f8d33994740 1 -- start start 2024-04-26T18:29:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d33994740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c35f00 0x563488c362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d33994740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 0x563488c40ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d33994740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 0x563488c434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d33994740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563488b9c460 con 0x563488c36820 2024-04-26T18:29:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d33994740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563488b56590 con 0x563488c410e0 2024-04-26T18:29:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d33994740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563488b98420 con 0x563488c35f00 2024-04-26T18:29:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d237fe640 1 -- 172.21.15.84:0/1481255845 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f8d1c004df0 con 0x7f8d2c101860 2024-04-26T18:29:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d2a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 0x563488c40ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d2ad9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c35f00 0x563488c362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d2a59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 0x563488c40ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45240/0 (socket says 172.21.15.84:45240) 2024-04-26T18:29:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d2a59c640 1 -- 172.21.15.84:0/1025717345 learned_addr learned my addr 172.21.15.84:0/1025717345 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d23fff640 1 --2- 172.21.15.84:0/1025717345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 0x563488c434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d2ad9d640 1 -- 172.21.15.84:0/1025717345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 msgr2=0x563488c434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/1025717345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 0x563488c434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d2ad9d640 1 -- 172.21.15.84:0/1025717345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 msgr2=0x563488c40ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/1025717345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 0x563488c40ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d2ad9d640 1 -- 172.21.15.84:0/1025717345 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563488b630d0 con 0x563488c35f00 2024-04-26T18:29:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d2a59c640 1 --2- 172.21.15.84:0/1025717345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 0x563488c40ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.907+0000 7f8d237fe640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8d180a3ff0 0x7f8d180a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.908+0000 7f8d2a59c640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8d180a3ff0 0x7f8d180a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.908+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/1025717345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c35f00 0x563488c362e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8d24007e40 tx=0x7f8d2400f790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=188b2cc9d026ab38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.908+0000 7f8d237fe640 1 -- 172.21.15.84:0/1481255845 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8d1c0d0430 con 0x7f8d2c101860 2024-04-26T18:29:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.908+0000 7f8d20ff9640 1 -- 172.21.15.84:0/1025717345 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24003b70 con 0x563488c35f00 2024-04-26T18:29:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.908+0000 7f8d20ff9640 1 -- 172.21.15.84:0/1025717345 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d24003d10 con 0x563488c35f00 2024-04-26T18:29:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.908+0000 7f8d20ff9640 1 -- 172.21.15.84:0/1025717345 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24004420 con 0x563488c35f00 2024-04-26T18:29:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 -- 172.21.15.84:0/1025717345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c35f00 msgr2=0x563488c362e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:05.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d2a59c640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8d180a3ff0 0x7f8d180a64b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c09c1c0 tx=0x7f8d1c0d5040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 --2- 172.21.15.84:0/1025717345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c35f00 0x563488c362e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8d24007e40 tx=0x7f8d2400f790 comp rx=0 tx=0).stop 2024-04-26T18:29:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 -- 172.21.15.84:0/1025717345 shutdown_connections 2024-04-26T18:29:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 --2- 172.21.15.84:0/1025717345 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 0x563488c434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 --2- 172.21.15.84:0/1025717345 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 0x563488c40ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 --2- 172.21.15.84:0/1025717345 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c35f00 0x563488c362e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 -- 172.21.15.84:0/1025717345 >> 172.21.15.84:0/1025717345 conn(0x563488c32410 msgr2=0x563488c327f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 -- 172.21.15.84:0/1025717345 shutdown_connections 2024-04-26T18:29:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 -- 172.21.15.84:0/1025717345 wait complete. 2024-04-26T18:29:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 -- start start 2024-04-26T18:29:05.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 Processor -- start 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c35f00 0x563488c3df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 0x563488c3e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 0x563488c3e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563488c43d10 con 0x563488c36820 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563488b23b60 con 0x563488c410e0 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563488c43e90 con 0x563488c35f00 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d33994740 10 client.0 _ll_register_callbacks cb 0x563488b482b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d23fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 0x563488c3e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d23fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 0x563488c3e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45242/0 (socket says 172.21.15.84:45242) 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.909+0000 7f8d23fff640 1 -- 172.21.15.84:0/1423981744 learned_addr learned my addr 172.21.15.84:0/1423981744 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c35f00 0x563488c3df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 0x563488c3e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d23fff640 1 -- 172.21.15.84:0/1423981744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c35f00 msgr2=0x563488c3df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:05.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d23fff640 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c35f00 0x563488c3df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d23fff640 1 -- 172.21.15.84:0/1423981744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 msgr2=0x563488c3e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d23fff640 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 0x563488c3e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d23fff640 1 -- 172.21.15.84:0/1423981744 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634888ae210 con 0x563488c36820 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 0x563488c3e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d23fff640 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 0x563488c3e4a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009930 tx=0x7f8d140314c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c887769848db6e1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d11ffb640 1 -- 172.21.15.84:0/1423981744 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d14035040 con 0x563488c36820 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d11ffb640 1 -- 172.21.15.84:0/1423981744 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8d14004040 con 0x563488c36820 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d33994740 1 -- 172.21.15.84:0/1423981744 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d24003d10 con 0x563488c36820 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d11ffb640 1 -- 172.21.15.84:0/1423981744 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d140318f0 con 0x563488c36820 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d33994740 10 client.15585 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.910+0000 7f8d33994740 1 -- 172.21.15.84:0/1423981744 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56348897a620 con 0x563488c36820 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.911+0000 7f8d11ffb640 1 -- 172.21.15.84:0/1423981744 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d24003d10 con 0x563488c36820 2024-04-26T18:29:05.920 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.911+0000 7f8d11ffb640 1 -- 172.21.15.84:0/1423981744 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8d14046480 con 0x563488c36820 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.911+0000 7f8d33994740 10 client.15585 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.911+0000 7f8d33994740 10 client.15585 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.911+0000 7f8d33994740 20 client.15585 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.911+0000 7f8d33994740 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563488c51200 0x563488c53610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.911+0000 7f8d33994740 4 client.15585 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.911+0000 7f8d33994740 1 -- 172.21.15.84:0/1423981744 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563488baa070 con 0x563488c51200 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.911+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563488c51200 0x563488c53610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563488c51200 0x563488c53610 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27777c6a8d352211 server_cookie=17e42c652aad2000 in_seq=0 out_seq=0 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d11ffb640 10 client.15585 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d11ffb640 1 -- 172.21.15.84:0/1423981744 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563488baa070 con 0x563488c51200 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d11ffb640 10 client.15585 handle_command_reply: tid=0 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d33994740 1 client.15585 shutdown 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d33994740 1 -- 172.21.15.84:0/1423981744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 msgr2=0x563488c3e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:05.921 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d33994740 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 0x563488c3e4a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8d14009930 tx=0x7f8d140314c0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d33994740 1 -- 172.21.15.84:0/1423981744 shutdown_connections 2024-04-26T18:29:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d33994740 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563488c51200 0x563488c53610 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d33994740 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 0x563488c3e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d33994740 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c36820 0x563488c3e4a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d11ffb640 0 client.15585 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d33994740 1 --2- 172.21.15.84:0/1423981744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c35f00 0x563488c3df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.912+0000 7f8d33994740 1 -- 172.21.15.84:0/1423981744 >> 172.21.15.84:0/1423981744 conn(0x563488c32410 msgr2=0x563488c38190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.913+0000 7f8d33994740 1 -- 172.21.15.84:0/1423981744 shutdown_connections 2024-04-26T18:29:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.913+0000 7f8d33994740 1 -- 172.21.15.84:0/1423981744 wait complete. 2024-04-26T18:29:05.922 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.913+0000 7f8d33994740 20 client.15585 trim_cache size 0 max 16384 2024-04-26T18:29:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d33994740 1 Processor -- start 2024-04-26T18:29:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d33994740 1 -- start start 2024-04-26T18:29:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d33994740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 0x563488c3e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d33994740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c36820 0x563488c3fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d33994740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 0x563488c58ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d33994740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1ae460 con 0x563488c35f00 2024-04-26T18:29:05.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d33994740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1affa0 con 0x563488c410e0 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d33994740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1aeb30 con 0x563488c36820 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d2ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 0x563488c3e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d2ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 0x563488c3e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45258/0 (socket says 172.21.15.84:45258) 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d2ad9d640 1 -- 172.21.15.84:0/3888466476 learned_addr learned my addr 172.21.15.84:0/3888466476 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d2ad9d640 1 -- 172.21.15.84:0/3888466476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c36820 msgr2=0x563488c3fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/3888466476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c36820 0x563488c3fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d2ad9d640 1 -- 172.21.15.84:0/3888466476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 msgr2=0x563488c58ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/3888466476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 0x563488c58ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d2ad9d640 1 -- 172.21.15.84:0/3888466476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563488c36650 con 0x563488c35f00 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/3888466476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 0x563488c3e800 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8d24007e40 tx=0x7f8d24007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f23c26a2cd3c86a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.930+0000 7f8d13fff640 1 -- 172.21.15.84:0/3888466476 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24003c40 con 0x563488c35f00 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.931+0000 7f8d13fff640 1 -- 172.21.15.84:0/3888466476 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d24003de0 con 0x563488c35f00 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.931+0000 7f8d13fff640 1 -- 172.21.15.84:0/3888466476 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d240109d0 con 0x563488c35f00 2024-04-26T18:29:05.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.931+0000 7f8d33994740 1 -- 172.21.15.84:0/3888466476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 msgr2=0x563488c3e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.931+0000 7f8d33994740 1 --2- 172.21.15.84:0/3888466476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 0x563488c3e800 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f8d24007e40 tx=0x7f8d24007f90 comp rx=0 tx=0).stop 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.931+0000 7f8d33994740 1 -- 172.21.15.84:0/3888466476 shutdown_connections 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.931+0000 7f8d33994740 1 --2- 172.21.15.84:0/3888466476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c410e0 0x563488c58ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.931+0000 7f8d33994740 1 --2- 172.21.15.84:0/3888466476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c36820 0x563488c3fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.931+0000 7f8d33994740 1 --2- 172.21.15.84:0/3888466476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 0x563488c3e800 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.931+0000 7f8d33994740 1 -- 172.21.15.84:0/3888466476 >> 172.21.15.84:0/3888466476 conn(0x563488c32410 msgr2=0x563488c3f790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.931+0000 7f8d33994740 1 -- 172.21.15.84:0/3888466476 shutdown_connections 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.931+0000 7f8d33994740 1 -- 172.21.15.84:0/3888466476 wait complete. 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.931+0000 7f8d33994740 1 -- start start 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.931+0000 7f8d33994740 1 Processor -- start 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d33994740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 0x563488c3ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d33994740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c36820 0x563488c3b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d33994740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c410e0 0x563488c3b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d33994740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1af8d0 con 0x563488c35f00 2024-04-26T18:29:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d33994740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1af200 con 0x563488c36820 2024-04-26T18:29:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d33994740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8d2c1ac440 con 0x563488c410e0 2024-04-26T18:29:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d33994740 10 client.0 _ll_register_callbacks cb 0x7f8d14031cb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d2ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 0x563488c3ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d2ad9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 0x563488c3ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45270/0 (socket says 172.21.15.84:45270) 2024-04-26T18:29:05.938 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d2ad9d640 1 -- 172.21.15.84:0/2570203886 learned_addr learned my addr 172.21.15.84:0/2570203886 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d23fff640 1 --2- 172.21.15.84:0/2570203886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c36820 0x563488c3b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d2ad9d640 1 -- 172.21.15.84:0/2570203886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c410e0 msgr2=0x563488c3b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/2570203886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c410e0 0x563488c3b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d2ad9d640 1 -- 172.21.15.84:0/2570203886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c36820 msgr2=0x563488c3b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/2570203886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c36820 0x563488c3b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d2ad9d640 1 -- 172.21.15.84:0/2570203886 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563488cb6820 con 0x563488c35f00 2024-04-26T18:29:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d23fff640 1 --2- 172.21.15.84:0/2570203886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c36820 0x563488c3b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:29:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d2ad9d640 1 --2- 172.21.15.84:0/2570203886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 0x563488c3ad20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8d24005020 tx=0x7f8d2400fda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=775bebd60a427089 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d11ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:29:05.939 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d11ffb640 1 -- 172.21.15.84:0/2570203886 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d24033030 con 0x563488c35f00 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d11ffb640 1 -- 172.21.15.84:0/2570203886 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8d24005530 con 0x563488c35f00 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d33994740 1 -- 172.21.15.84:0/2570203886 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d24003de0 con 0x563488c35f00 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.932+0000 7f8d11ffb640 1 -- 172.21.15.84:0/2570203886 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d2401d040 con 0x563488c35f00 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.933+0000 7f8d33994740 10 client.15591 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.933+0000 7f8d33994740 1 -- 172.21.15.84:0/2570203886 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563488cb7b30 con 0x563488c35f00 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.933+0000 7f8d11ffb640 1 -- 172.21.15.84:0/2570203886 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d24003de0 con 0x563488c35f00 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.933+0000 7f8d11ffb640 1 -- 172.21.15.84:0/2570203886 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8d240404f0 con 0x563488c35f00 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.933+0000 7f8d33994740 10 client.15591 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.933+0000 7f8d33994740 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.933+0000 7f8d33994740 20 client.15591 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.933+0000 7f8d33994740 1 --2- 172.21.15.84:0/2570203886 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563488cb7fa0 0x563488cba390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.933+0000 7f8d33994740 4 client.15591 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.933+0000 7f8d33994740 1 -- 172.21.15.84:0/2570203886 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563488ba3ee0 con 0x563488cb7fa0 2024-04-26T18:29:05.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.933+0000 7f8d23fff640 1 --2- 172.21.15.84:0/2570203886 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563488cb7fa0 0x563488cba390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.933+0000 7f8d23fff640 1 --2- 172.21.15.84:0/2570203886 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563488cb7fa0 0x563488cba390 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45608114702a38 server_cookie=dd3594365c582ea6 in_seq=0 out_seq=0 2024-04-26T18:29:05.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.933+0000 7f8d11ffb640 10 client.15591 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.941+0000 7f8d11ffb640 1 -- 172.21.15.84:0/2570203886 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563488ba3ee0 con 0x563488cb7fa0 2024-04-26T18:29:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.941+0000 7f8d11ffb640 10 client.15591 handle_command_reply: tid=0 2024-04-26T18:29:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d33994740 1 client.15591 shutdown 2024-04-26T18:29:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d33994740 1 -- 172.21.15.84:0/2570203886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 msgr2=0x563488c3ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d33994740 1 --2- 172.21.15.84:0/2570203886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 0x563488c3ad20 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8d24005020 tx=0x7f8d2400fda0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d33994740 1 -- 172.21.15.84:0/2570203886 shutdown_connections 2024-04-26T18:29:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d33994740 1 --2- 172.21.15.84:0/2570203886 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563488cb7fa0 0x563488cba390 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d33994740 1 --2- 172.21.15.84:0/2570203886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563488c410e0 0x563488c3b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d33994740 1 --2- 172.21.15.84:0/2570203886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563488c36820 0x563488c3b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d11ffb640 0 client.15591 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d33994740 1 --2- 172.21.15.84:0/2570203886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563488c35f00 0x563488c3ad20 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d33994740 1 -- 172.21.15.84:0/2570203886 >> 172.21.15.84:0/2570203886 conn(0x563488c32410 msgr2=0x563488c40020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d33994740 1 -- 172.21.15.84:0/2570203886 shutdown_connections 2024-04-26T18:29:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d33994740 1 -- 172.21.15.84:0/2570203886 wait complete. 2024-04-26T18:29:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d33994740 20 client.15591 trim_cache size 0 max 16384 2024-04-26T18:29:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:29:05.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:29:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:29:05.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:29:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d30c7b640 1 -- 172.21.15.84:0/1481255845 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8d180a3ff0 msgr2=0x7f8d180a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d30c7b640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8d180a3ff0 0x7f8d180a64b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c09c1c0 tx=0x7f8d1c0d5040 comp rx=0 tx=0).stop 2024-04-26T18:29:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d30c7b640 1 -- 172.21.15.84:0/1481255845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d2c101860 msgr2=0x7f8d2c199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d30c7b640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d2c101860 0x7f8d2c199e80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00d770 tx=0x7f8d1c00dc30 comp rx=0 tx=0).stop 2024-04-26T18:29:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d30c7b640 1 -- 172.21.15.84:0/1481255845 shutdown_connections 2024-04-26T18:29:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d30c7b640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8d180a3ff0 0x7f8d180a64b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d30c7b640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8d2c107820 0x7f8d2c1a4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d30c7b640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8d2c102220 0x7f8d2c19a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d30c7b640 1 --2- 172.21.15.84:0/1481255845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8d2c101860 0x7f8d2c199e80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.942+0000 7f8d30c7b640 1 -- 172.21.15.84:0/1481255845 >> 172.21.15.84:0/1481255845 conn(0x7f8d2c0789b0 msgr2=0x7f8d2c1043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:05.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.943+0000 7f8d30c7b640 1 -- 172.21.15.84:0/1481255845 shutdown_connections 2024-04-26T18:29:05.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:05.943+0000 7f8d30c7b640 1 -- 172.21.15.84:0/1481255845 wait complete. 2024-04-26T18:29:07.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:07 smithi103 ceph-mon[32727]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-26T18:29:07.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:07 smithi174 ceph-mon[32705]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-26T18:29:07.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:07 smithi084 ceph-mon[29141]: pgmap v37: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 24 op/s 2024-04-26T18:29:09.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:09 smithi174 ceph-mon[32705]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:29:09.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:09 smithi084 ceph-mon[29141]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:29:09.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:09 smithi103 ceph-mon[32727]: pgmap v38: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 23 KiB/s wr, 23 op/s 2024-04-26T18:29:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:11 smithi174 ceph-mon[32705]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-26T18:29:11.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:11 smithi084 ceph-mon[29141]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-26T18:29:11.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:11 smithi103 ceph-mon[32727]: pgmap v39: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-04-26T18:29:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:13 smithi174 ceph-mon[32705]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-26T18:29:13.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:13 smithi084 ceph-mon[29141]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-26T18:29:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:13 smithi103 ceph-mon[32727]: pgmap v40: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-04-26T18:29:15.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:15 smithi174 ceph-mon[32705]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-26T18:29:15.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:15 smithi084 ceph-mon[29141]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-26T18:29:15.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:15 smithi103 ceph-mon[32727]: pgmap v41: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-04-26T18:29:17.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:17 smithi103 ceph-mon[32727]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:17.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:17 smithi174 ceph-mon[32705]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:17.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:17 smithi084 ceph-mon[29141]: pgmap v42: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.076+0000 7f1de5ef1640 1 -- 172.21.15.84:0/3088093945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1de0108950 msgr2=0x7f1de0108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.076+0000 7f1de5ef1640 1 --2- 172.21.15.84:0/3088093945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1de0108950 0x7f1de0108d30 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009930 tx=0x7f1dd002a420 comp rx=0 tx=0).stop 2024-04-26T18:29:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.076+0000 7f1de5ef1640 1 -- 172.21.15.84:0/3088093945 shutdown_connections 2024-04-26T18:29:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.076+0000 7f1de5ef1640 1 --2- 172.21.15.84:0/3088093945 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de0103300 0x7f1de010f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.076+0000 7f1de5ef1640 1 --2- 172.21.15.84:0/3088093945 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1de0102940 0x7f1de0102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.076+0000 7f1de5ef1640 1 --2- 172.21.15.84:0/3088093945 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1de0108950 0x7f1de0108d30 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.076+0000 7f1de5ef1640 1 -- 172.21.15.84:0/3088093945 >> 172.21.15.84:0/3088093945 conn(0x7f1de00fe430 msgr2=0x7f1de0100850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.076+0000 7f1de5ef1640 1 -- 172.21.15.84:0/3088093945 shutdown_connections 2024-04-26T18:29:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.076+0000 7f1de5ef1640 1 -- 172.21.15.84:0/3088093945 wait complete. 2024-04-26T18:29:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.077+0000 7f1de5ef1640 1 Processor -- start 2024-04-26T18:29:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.077+0000 7f1de5ef1640 1 -- start start 2024-04-26T18:29:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.077+0000 7f1de5ef1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de0102940 0x7f1de0196f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.077+0000 7f1de5ef1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1de0103300 0x7f1de0197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.077+0000 7f1de5ef1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1de0108950 0x7f1de019b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.077+0000 7f1ddffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1de0108950 0x7f1de019b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.077+0000 7f1ddffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1de0108950 0x7f1de019b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52960/0 (socket says 172.21.15.84:52960) 2024-04-26T18:29:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.077+0000 7f1ddffff640 1 -- 172.21.15.84:0/4179381964 learned_addr learned my addr 172.21.15.84:0/4179381964 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.077+0000 7f1de5ef1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1de0110a50 con 0x7f1de0108950 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.077+0000 7f1de5ef1640 1 -- 172.21.15.84:0/4179381964 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01108d0 con 0x7f1de0102940 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.077+0000 7f1de5ef1640 1 -- 172.21.15.84:0/4179381964 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1de0110bd0 con 0x7f1de0103300 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.078+0000 7f1ddf7fe640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de0102940 0x7f1de0196f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.078+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1de0103300 0x7f1de0197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.078+0000 7f1ddffff640 1 -- 172.21.15.84:0/4179381964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1de0103300 msgr2=0x7f1de0197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.078+0000 7f1ddffff640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1de0103300 0x7f1de0197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.078+0000 7f1ddffff640 1 -- 172.21.15.84:0/4179381964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de0102940 msgr2=0x7f1de0196f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.078+0000 7f1ddffff640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de0102940 0x7f1de0196f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.078+0000 7f1ddffff640 1 -- 172.21.15.84:0/4179381964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de019bed0 con 0x7f1de0108950 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.078+0000 7f1ddffff640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1de0108950 0x7f1de019b7a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800eb10 tx=0x7f1dd800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.078+0000 7f1ddcff9640 1 -- 172.21.15.84:0/4179381964 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8004b20 con 0x7f1de0108950 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.078+0000 7f1ddcff9640 1 -- 172.21.15.84:0/4179381964 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd8004cc0 con 0x7f1de0108950 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.078+0000 7f1ddcff9640 1 -- 172.21.15.84:0/4179381964 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8010670 con 0x7f1de0108950 2024-04-26T18:29:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.078+0000 7f1de5ef1640 1 -- 172.21.15.84:0/4179381964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de019c1c0 con 0x7f1de0108950 2024-04-26T18:29:19.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.078+0000 7f1de5ef1640 1 -- 172.21.15.84:0/4179381964 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de01a3a50 con 0x7f1de0108950 2024-04-26T18:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.082+0000 7f1de8c06740 1 Processor -- start 2024-04-26T18:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.082+0000 7f1ddcff9640 1 -- 172.21.15.84:0/4179381964 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 24) ==== 138930+0+0 (secure 0 0 0) 0x7f1dd8010830 con 0x7f1de0108950 2024-04-26T18:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.082+0000 7f1de8c06740 1 -- start start 2024-04-26T18:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.082+0000 7f1de8c06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41dde710 0x558a41ddeaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.082+0000 7f1de8c06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de93b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.082+0000 7f1de8c06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 0x558a41debce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.082+0000 7f1de8c06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558a41d3c460 con 0x558a41ddf030 2024-04-26T18:29:19.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.082+0000 7f1de8c06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a41cf6590 con 0x558a41dde710 2024-04-26T18:29:19.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.082+0000 7f1de8c06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a41d38420 con 0x558a41de98f0 2024-04-26T18:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.082+0000 7f1ddf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.082+0000 7f1ddf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52968/0 (socket says 172.21.15.84:52968) 2024-04-26T18:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.082+0000 7f1ddf7fe640 1 -- 172.21.15.84:0/1470082364 learned_addr learned my addr 172.21.15.84:0/1470082364 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/1470082364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 0x558a41debce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/1470082364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41dde710 0x558a41ddeaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1ddcff9640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1dc80a3ff0 0x7f1dc80a64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1ddeffd640 1 -- 172.21.15.84:0/1470082364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41dde710 msgr2=0x558a41ddeaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/1470082364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41dde710 0x558a41ddeaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1ddeffd640 1 -- 172.21.15.84:0/1470082364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 msgr2=0x558a41de93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/1470082364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1ddeffd640 1 -- 172.21.15.84:0/1470082364 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a41d030d0 con 0x558a41de98f0 2024-04-26T18:29:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1ddf7fe640 1 --2- 172.21.15.84:0/1470082364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1ddffff640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1dc80a3ff0 0x7f1dc80a64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1ddcff9640 1 -- 172.21.15.84:0/4179381964 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f1dd80d0fa0 con 0x7f1de0108950 2024-04-26T18:29:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/1470082364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 0x558a41debce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400feb0 tx=0x7f1dd4005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c55dbef085d19df5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1dc5ffb640 1 -- 172.21.15.84:0/1470082364 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4005220 con 0x558a41de98f0 2024-04-26T18:29:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1dc5ffb640 1 -- 172.21.15.84:0/1470082364 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd4004c50 con 0x558a41de98f0 2024-04-26T18:29:19.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.083+0000 7f1dc5ffb640 1 -- 172.21.15.84:0/1470082364 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd401d650 con 0x558a41de98f0 2024-04-26T18:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.084+0000 7f1de8c06740 1 -- 172.21.15.84:0/1470082364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 msgr2=0x558a41debce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.084+0000 7f1ddffff640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1dc80a3ff0 0x7f1dc80a64b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80ce500 tx=0x7f1dd800cee0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.084+0000 7f1de8c06740 1 --2- 172.21.15.84:0/1470082364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 0x558a41debce0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1dd400feb0 tx=0x7f1dd4005f90 comp rx=0 tx=0).stop 2024-04-26T18:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.084+0000 7f1de8c06740 1 -- 172.21.15.84:0/1470082364 shutdown_connections 2024-04-26T18:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.084+0000 7f1de8c06740 1 --2- 172.21.15.84:0/1470082364 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 0x558a41debce0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.084+0000 7f1de8c06740 1 --2- 172.21.15.84:0/1470082364 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.084+0000 7f1de8c06740 1 --2- 172.21.15.84:0/1470082364 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41dde710 0x558a41ddeaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.084+0000 7f1de8c06740 1 -- 172.21.15.84:0/1470082364 >> 172.21.15.84:0/1470082364 conn(0x558a41ddac20 msgr2=0x558a41ddb000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:19.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.085+0000 7f1de8c06740 1 -- 172.21.15.84:0/1470082364 shutdown_connections 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.085+0000 7f1de8c06740 1 -- 172.21.15.84:0/1470082364 wait complete. 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.085+0000 7f1de8c06740 1 -- start start 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.085+0000 7f1de8c06740 1 Processor -- start 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1de8c06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41dde710 0x558a41de6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1de8c06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1ddf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1ddf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de6eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52976/0 (socket says 172.21.15.84:52976) 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1ddf7fe640 1 -- 172.21.15.84:0/3848817849 learned_addr learned my addr 172.21.15.84:0/3848817849 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1de8c06740 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41de98f0 0x558a41de73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1de8c06740 1 -- 172.21.15.84:0/3848817849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558a41dec4c0 con 0x558a41ddf030 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1de8c06740 1 -- 172.21.15.84:0/3848817849 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a41cc3b60 con 0x558a41de98f0 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1de8c06740 1 -- 172.21.15.84:0/3848817849 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a41dec640 con 0x558a41dde710 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41dde710 0x558a41de6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1de8c06740 10 client.0 _ll_register_callbacks cb 0x558a41ce82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:19.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1ddf7fe640 1 -- 172.21.15.84:0/3848817849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41dde710 msgr2=0x558a41de6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1ddf7fe640 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41dde710 0x558a41de6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1ddf7fe640 1 -- 172.21.15.84:0/3848817849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41de98f0 msgr2=0x558a41de73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1ddf7fe640 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41de98f0 0x558a41de73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1ddf7fe640 1 -- 172.21.15.84:0/3848817849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a41a4e210 con 0x558a41ddf030 2024-04-26T18:29:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.086+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41dde710 0x558a41de6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.087+0000 7f1ddf7fe640 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de6eb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009230 tx=0x7f1dd0004440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9cf0b73e8e9d4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.087+0000 7f1dbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:29:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.087+0000 7f1dbeffd640 1 -- 172.21.15.84:0/3848817849 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0035040 con 0x558a41ddf030 2024-04-26T18:29:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.087+0000 7f1dbeffd640 1 -- 172.21.15.84:0/3848817849 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f1dd0005450 con 0x558a41ddf030 2024-04-26T18:29:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.087+0000 7f1de8c06740 1 -- 172.21.15.84:0/3848817849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dd4004c50 con 0x558a41ddf030 2024-04-26T18:29:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.087+0000 7f1dbeffd640 1 -- 172.21.15.84:0/3848817849 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1dd0005650 con 0x558a41ddf030 2024-04-26T18:29:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.087+0000 7f1de8c06740 10 client.15603 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:19.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.087+0000 7f1de8c06740 1 -- 172.21.15.84:0/3848817849 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a41b1a620 con 0x558a41ddf030 2024-04-26T18:29:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.087+0000 7f1dbeffd640 1 -- 172.21.15.84:0/3848817849 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd4004c50 con 0x558a41ddf030 2024-04-26T18:29:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.087+0000 7f1dbeffd640 1 -- 172.21.15.84:0/3848817849 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1dd0003b90 con 0x558a41ddf030 2024-04-26T18:29:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.088+0000 7f1de8c06740 10 client.15603 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.088+0000 7f1de8c06740 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.088+0000 7f1de8c06740 20 client.15603 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.088+0000 7f1de8c06740 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a41df9fc0 0x558a41dfc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.088+0000 7f1de8c06740 4 client.15603 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.088+0000 7f1de8c06740 1 -- 172.21.15.84:0/3848817849 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a41d4a070 con 0x558a41df9fc0 2024-04-26T18:29:19.091 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.088+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a41df9fc0 0x558a41dfc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.088+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a41df9fc0 0x558a41dfc460 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4d2524884ea3cd6 server_cookie=afb0bce939c497c1 in_seq=0 out_seq=0 2024-04-26T18:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.088+0000 7f1dbeffd640 10 client.15603 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.089+0000 7f1dbeffd640 1 -- 172.21.15.84:0/3848817849 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a41d4a070 con 0x558a41df9fc0 2024-04-26T18:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.089+0000 7f1dbeffd640 10 client.15603 handle_command_reply: tid=0 2024-04-26T18:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.089+0000 7f1de8c06740 1 client.15603 shutdown 2024-04-26T18:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.089+0000 7f1de8c06740 1 -- 172.21.15.84:0/3848817849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 msgr2=0x558a41de6eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.089+0000 7f1de8c06740 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de6eb0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0009230 tx=0x7f1dd0004440 comp rx=0 tx=0).stop 2024-04-26T18:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.090+0000 7f1de8c06740 1 -- 172.21.15.84:0/3848817849 shutdown_connections 2024-04-26T18:29:19.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.090+0000 7f1de8c06740 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a41df9fc0 0x558a41dfc460 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.090+0000 7f1de8c06740 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41de98f0 0x558a41de73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.090+0000 7f1dbeffd640 0 client.15603 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.090+0000 7f1de8c06740 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de6eb0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.090+0000 7f1de8c06740 1 --2- 172.21.15.84:0/3848817849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41dde710 0x558a41de6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.090+0000 7f1de8c06740 1 -- 172.21.15.84:0/3848817849 >> 172.21.15.84:0/3848817849 conn(0x558a41ddac20 msgr2=0x558a41dd15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.090+0000 7f1de8c06740 1 -- 172.21.15.84:0/3848817849 shutdown_connections 2024-04-26T18:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.090+0000 7f1de8c06740 1 -- 172.21.15.84:0/3848817849 wait complete. 2024-04-26T18:29:19.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.090+0000 7f1de8c06740 20 client.15603 trim_cache size 0 max 16384 2024-04-26T18:29:19.120 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.119+0000 7f1de8c06740 1 Processor -- start 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.119+0000 7f1de8c06740 1 -- start start 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.119+0000 7f1de8c06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41dde710 0x558a41df5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.119+0000 7f1de8c06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de1120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1de8c06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 0x558a41dd1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1ddf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1ddf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52988/0 (socket says 172.21.15.84:52988) 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1ddf7fe640 1 -- 172.21.15.84:0/3118660729 learned_addr learned my addr 172.21.15.84:0/3118660729 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1de8c06740 1 -- 172.21.15.84:0/3118660729 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01ac370 con 0x558a41ddf030 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1de8c06740 1 -- 172.21.15.84:0/3118660729 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01adeb0 con 0x558a41dde710 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1de8c06740 1 -- 172.21.15.84:0/3118660729 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01aca40 con 0x558a41de98f0 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/3118660729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41dde710 0x558a41df5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/3118660729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 0x558a41dd1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1ddf7fe640 1 -- 172.21.15.84:0/3118660729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 msgr2=0x558a41dd1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:19.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1ddf7fe640 1 --2- 172.21.15.84:0/3118660729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 0x558a41dd1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1ddf7fe640 1 -- 172.21.15.84:0/3118660729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41dde710 msgr2=0x558a41df5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1ddf7fe640 1 --2- 172.21.15.84:0/3118660729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41dde710 0x558a41df5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1ddf7fe640 1 -- 172.21.15.84:0/3118660729 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a41ddee50 con 0x558a41ddf030 2024-04-26T18:29:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/3118660729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41dde710 0x558a41df5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.120+0000 7f1ddf7fe640 1 --2- 172.21.15.84:0/3118660729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de1120 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0006010 tx=0x7f1dd002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1eed314650fe941 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.121+0000 7f1dc57fa640 1 -- 172.21.15.84:0/3118660729 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0035070 con 0x558a41ddf030 2024-04-26T18:29:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.121+0000 7f1dc57fa640 1 -- 172.21.15.84:0/3118660729 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd0041070 con 0x558a41ddf030 2024-04-26T18:29:19.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.121+0000 7f1dc57fa640 1 -- 172.21.15.84:0/3118660729 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0053070 con 0x558a41ddf030 2024-04-26T18:29:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.122+0000 7f1de8c06740 1 -- 172.21.15.84:0/3118660729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 msgr2=0x558a41de1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.122+0000 7f1de8c06740 1 --2- 172.21.15.84:0/3118660729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de1120 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0006010 tx=0x7f1dd002a420 comp rx=0 tx=0).stop 2024-04-26T18:29:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.122+0000 7f1de8c06740 1 -- 172.21.15.84:0/3118660729 shutdown_connections 2024-04-26T18:29:19.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.122+0000 7f1de8c06740 1 --2- 172.21.15.84:0/3118660729 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 0x558a41dd1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.122+0000 7f1de8c06740 1 --2- 172.21.15.84:0/3118660729 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41ddf030 0x558a41de1120 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.122+0000 7f1de8c06740 1 --2- 172.21.15.84:0/3118660729 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41dde710 0x558a41df5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.122+0000 7f1de8c06740 1 -- 172.21.15.84:0/3118660729 >> 172.21.15.84:0/3118660729 conn(0x558a41ddac20 msgr2=0x558a41dd0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.122+0000 7f1de8c06740 1 -- 172.21.15.84:0/3118660729 shutdown_connections 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.122+0000 7f1de8c06740 1 -- 172.21.15.84:0/3118660729 wait complete. 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.122+0000 7f1de8c06740 1 -- start start 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.123+0000 7f1de8c06740 1 Processor -- start 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.123+0000 7f1de8c06740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41dde710 0x558a41df1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.123+0000 7f1de8c06740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41ddf030 0x558a41de4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.123+0000 7f1de8c06740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 0x558a41de2be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.123+0000 7f1de8c06740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01ad7e0 con 0x558a41dde710 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.123+0000 7f1de8c06740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01ad110 con 0x558a41ddf030 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.123+0000 7f1de8c06740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1de01aa350 con 0x558a41de98f0 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1ddeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41dde710 0x558a41df1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1ddeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41dde710 0x558a41df1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52992/0 (socket says 172.21.15.84:52992) 2024-04-26T18:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1ddeffd640 1 -- 172.21.15.84:0/4097827509 learned_addr learned my addr 172.21.15.84:0/4097827509 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1de8c06740 10 client.0 _ll_register_callbacks cb 0x7f1dd0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1ddf7fe640 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41ddf030 0x558a41de4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 0x558a41de2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1ddeffd640 1 -- 172.21.15.84:0/4097827509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 msgr2=0x558a41de2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 0x558a41de2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1ddeffd640 1 -- 172.21.15.84:0/4097827509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41ddf030 msgr2=0x558a41de4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41ddf030 0x558a41de4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1ddeffd640 1 -- 172.21.15.84:0/4097827509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a41de3120 con 0x558a41dde710 2024-04-26T18:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1ddf7fe640 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41ddf030 0x558a41de4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1ddeffd640 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41dde710 0x558a41df1dd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4004310 tx=0x7f1dd4004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e4f8aa0b729b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1dbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:29:19.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1dbeffd640 1 -- 172.21.15.84:0/4097827509 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4034040 con 0x558a41dde710 2024-04-26T18:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1dbeffd640 1 -- 172.21.15.84:0/4097827509 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f1dd402b340 con 0x558a41dde710 2024-04-26T18:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.124+0000 7f1de8c06740 1 -- 172.21.15.84:0/4097827509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dd0041070 con 0x558a41dde710 2024-04-26T18:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.125+0000 7f1dbeffd640 1 -- 172.21.15.84:0/4097827509 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1dd402b540 con 0x558a41dde710 2024-04-26T18:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.125+0000 7f1de8c06740 10 client.15609 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.125+0000 7f1de8c06740 1 -- 172.21.15.84:0/4097827509 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a41e5fb80 con 0x558a41dde710 2024-04-26T18:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.125+0000 7f1dbeffd640 1 -- 172.21.15.84:0/4097827509 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd0041070 con 0x558a41dde710 2024-04-26T18:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.125+0000 7f1dbeffd640 1 -- 172.21.15.84:0/4097827509 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1dd401d430 con 0x558a41dde710 2024-04-26T18:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.125+0000 7f1de8c06740 10 client.15609 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.125+0000 7f1de8c06740 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.125+0000 7f1de8c06740 20 client.15609 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.125+0000 7f1de8c06740 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a41df9fc0 0x558a41e61f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.125+0000 7f1de8c06740 4 client.15609 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:29:19.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.125+0000 7f1de8c06740 1 -- 172.21.15.84:0/4097827509 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a41d43ee0 con 0x558a41df9fc0 2024-04-26T18:29:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.126+0000 7f1ddf7fe640 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a41df9fc0 0x558a41e61f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.126+0000 7f1dbeffd640 10 client.15609 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:19.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.126+0000 7f1ddf7fe640 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a41df9fc0 0x558a41e61f60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cce91f18064fa79d server_cookie=1deb0a846c533824 in_seq=0 out_seq=0 2024-04-26T18:29:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.130+0000 7f1dbeffd640 1 -- 172.21.15.84:0/4097827509 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a41d43ee0 con 0x558a41df9fc0 2024-04-26T18:29:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.130+0000 7f1dbeffd640 10 client.15609 handle_command_reply: tid=0 2024-04-26T18:29:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.130+0000 7f1de8c06740 1 client.15609 shutdown 2024-04-26T18:29:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.130+0000 7f1de8c06740 1 -- 172.21.15.84:0/4097827509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41dde710 msgr2=0x558a41df1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.130+0000 7f1de8c06740 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41dde710 0x558a41df1dd0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4004310 tx=0x7f1dd4004470 comp rx=0 tx=0).stop 2024-04-26T18:29:19.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.130+0000 7f1de8c06740 1 -- 172.21.15.84:0/4097827509 shutdown_connections 2024-04-26T18:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.130+0000 7f1de8c06740 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a41df9fc0 0x558a41e61f60 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.130+0000 7f1de8c06740 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a41de98f0 0x558a41de2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.130+0000 7f1dbeffd640 0 client.15609 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.130+0000 7f1de8c06740 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a41ddf030 0x558a41de4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.130+0000 7f1de8c06740 1 --2- 172.21.15.84:0/4097827509 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a41dde710 0x558a41df1dd0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.130+0000 7f1de8c06740 1 -- 172.21.15.84:0/4097827509 >> 172.21.15.84:0/4097827509 conn(0x558a41ddac20 msgr2=0x558a41dd15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.131+0000 7f1de8c06740 1 -- 172.21.15.84:0/4097827509 shutdown_connections 2024-04-26T18:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.131+0000 7f1de8c06740 1 -- 172.21.15.84:0/4097827509 wait complete. 2024-04-26T18:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.131+0000 7f1de8c06740 20 client.15609 trim_cache size 0 max 16384 2024-04-26T18:29:19.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:29:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:29:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:29:19.133 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.131+0000 7f1de5ef1640 1 -- 172.21.15.84:0/4179381964 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1dc80a3ff0 msgr2=0x7f1dc80a64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.131+0000 7f1de5ef1640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1dc80a3ff0 0x7f1dc80a64b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1dd80ce500 tx=0x7f1dd800cee0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.131+0000 7f1de5ef1640 1 -- 172.21.15.84:0/4179381964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1de0108950 msgr2=0x7f1de019b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.131+0000 7f1de5ef1640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1de0108950 0x7f1de019b7a0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800eb10 tx=0x7f1dd800c2d0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.132+0000 7f1de5ef1640 1 -- 172.21.15.84:0/4179381964 shutdown_connections 2024-04-26T18:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.132+0000 7f1de5ef1640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1dc80a3ff0 0x7f1dc80a64b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.132+0000 7f1de5ef1640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1de0108950 0x7f1de019b7a0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.132+0000 7f1de5ef1640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1de0103300 0x7f1de0197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.132+0000 7f1de5ef1640 1 --2- 172.21.15.84:0/4179381964 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1de0102940 0x7f1de0196f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.132+0000 7f1de5ef1640 1 -- 172.21.15.84:0/4179381964 >> 172.21.15.84:0/4179381964 conn(0x7f1de00fe430 msgr2=0x7f1de00fea50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.132+0000 7f1de5ef1640 1 -- 172.21.15.84:0/4179381964 shutdown_connections 2024-04-26T18:29:19.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:19.132+0000 7f1de5ef1640 1 -- 172.21.15.84:0/4179381964 wait complete. 2024-04-26T18:29:19.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:19 smithi174 ceph-mon[32705]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:19.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:19 smithi084 ceph-mon[29141]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:19 smithi103 ceph-mon[32727]: pgmap v43: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:21.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:21 smithi174 ceph-mon[32705]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-26T18:29:21.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:21 smithi084 ceph-mon[29141]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-26T18:29:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:21 smithi103 ceph-mon[32727]: pgmap v44: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-26T18:29:23.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:23 smithi174 ceph-mon[32705]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:29:23.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:23 smithi084 ceph-mon[29141]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:29:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:23 smithi103 ceph-mon[32727]: pgmap v45: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:29:25.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:25 smithi174 ceph-mon[32705]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-26T18:29:25.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:25 smithi084 ceph-mon[29141]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-26T18:29:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:25 smithi103 ceph-mon[32727]: pgmap v46: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-26T18:29:27.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:27 smithi174 ceph-mon[32705]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:29:27.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:27 smithi084 ceph-mon[29141]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:29:27.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:27 smithi103 ceph-mon[32727]: pgmap v47: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:29:29.585 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:29:29.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:29 smithi174 ceph-mon[32705]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:29:29.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:29 smithi084 ceph-mon[29141]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:29:29.867 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:29:29.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:29 smithi103 ceph-mon[32727]: pgmap v48: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:29:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:31 smithi174 ceph-mon[32705]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-26T18:29:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:31 smithi174 ceph-mon[32705]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-04-26T18:29:31.718 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:31 smithi084 ceph-mon[29141]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-26T18:29:31.718 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:31 smithi084 ceph-mon[29141]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-04-26T18:29:31.865 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.865+0000 7fa935a5f700 1 -- 172.21.15.84:0/3418655103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9301034d0 msgr2=0x7fa930103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:31.866 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.865+0000 7fa935a5f700 1 --2- 172.21.15.84:0/3418655103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9301034d0 0x7fa930103950 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa9240099c0 tx=0x7fa924009cd0 comp rx=0 tx=0).stop 2024-04-26T18:29:31.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.865+0000 7fa935a5f700 1 -- 172.21.15.84:0/3418655103 shutdown_connections 2024-04-26T18:29:31.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.865+0000 7fa935a5f700 1 --2- 172.21.15.84:0/3418655103 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa930103e90 0x7fa930110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:31.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.865+0000 7fa935a5f700 1 --2- 172.21.15.84:0/3418655103 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa9301083b0 0x7fa930108790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:31.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.865+0000 7fa935a5f700 1 --2- 172.21.15.84:0/3418655103 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9301034d0 0x7fa930103950 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:31.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.865+0000 7fa935a5f700 1 -- 172.21.15.84:0/3418655103 >> 172.21.15.84:0/3418655103 conn(0x7fa93007a580 msgr2=0x7fa93007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:31.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.865+0000 7fa935a5f700 1 -- 172.21.15.84:0/3418655103 shutdown_connections 2024-04-26T18:29:31.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.866+0000 7fa935a5f700 1 -- 172.21.15.84:0/3418655103 wait complete. 2024-04-26T18:29:31.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.866+0000 7fa935a5f700 1 Processor -- start 2024-04-26T18:29:31.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.867+0000 7fa935a5f700 1 -- start start 2024-04-26T18:29:31.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.867+0000 7fa935a5f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa930103e90 0x7fa930075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:31.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.867+0000 7fa935a5f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9301083b0 0x7fa930075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:31.867 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.867+0000 7fa92e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9301083b0 0x7fa930075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:31.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.867+0000 7fa92e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9301083b0 0x7fa930075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36032/0 (socket says 172.21.15.84:36032) 2024-04-26T18:29:31.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.867+0000 7fa92e7fc700 1 -- 172.21.15.84:0/2811471726 learned_addr learned my addr 172.21.15.84:0/2811471726 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:31.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.867+0000 7fa92effd700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa930103e90 0x7fa930075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:31.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.867+0000 7fa935a5f700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9300761a0 0x7fa9301a7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:31.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.867+0000 7fa935a5f700 1 -- 172.21.15.84:0/2811471726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa930113620 con 0x7fa9301083b0 2024-04-26T18:29:31.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.867+0000 7fa935a5f700 1 -- 172.21.15.84:0/2811471726 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa9301134a0 con 0x7fa930103e90 2024-04-26T18:29:31.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.867+0000 7fa935a5f700 1 -- 172.21.15.84:0/2811471726 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa9301137a0 con 0x7fa9300761a0 2024-04-26T18:29:31.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.868+0000 7fa92f7fe700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9300761a0 0x7fa9301a7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:31.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.868+0000 7fa92e7fc700 1 -- 172.21.15.84:0/2811471726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9300761a0 msgr2=0x7fa9301a7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:31.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.868+0000 7fa92e7fc700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9300761a0 0x7fa9301a7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:31.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.868+0000 7fa92e7fc700 1 -- 172.21.15.84:0/2811471726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa930103e90 msgr2=0x7fa930075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:31.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.868+0000 7fa92e7fc700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa930103e90 0x7fa930075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:31.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.868+0000 7fa92e7fc700 1 -- 172.21.15.84:0/2811471726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9301a85f0 con 0x7fa9301083b0 2024-04-26T18:29:31.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.868+0000 7fa92f7fe700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9300761a0 0x7fa9301a7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:31.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.868+0000 7fa92e7fc700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9301083b0 0x7fa930075c60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa924009f50 tx=0x7fa924003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:31.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.869+0000 7fa934a5d700 1 -- 172.21.15.84:0/2811471726 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924027070 con 0x7fa9301083b0 2024-04-26T18:29:31.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.869+0000 7fa934a5d700 1 -- 172.21.15.84:0/2811471726 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa924003680 con 0x7fa9301083b0 2024-04-26T18:29:31.870 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.869+0000 7fa934a5d700 1 -- 172.21.15.84:0/2811471726 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa924022af0 con 0x7fa9301083b0 2024-04-26T18:29:31.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.869+0000 7fa935a5f700 1 -- 172.21.15.84:0/2811471726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9301a8880 con 0x7fa9301083b0 2024-04-26T18:29:31.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.869+0000 7fa935a5f700 1 -- 172.21.15.84:0/2811471726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9301a8c80 con 0x7fa9301083b0 2024-04-26T18:29:31.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.870+0000 7fa935a5f700 1 -- 172.21.15.84:0/2811471726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa93006a110 con 0x7fa9301083b0 2024-04-26T18:29:31.871 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.871+0000 7fa934a5d700 1 -- 172.21.15.84:0/2811471726 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa92400f460 con 0x7fa9301083b0 2024-04-26T18:29:31.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.872+0000 7fa934a5d700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa9180a3eb0 0x7fa9180a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:31.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.872+0000 7fa934a5d700 1 -- 172.21.15.84:0/2811471726 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fa9240d1c10 con 0x7fa9301083b0 2024-04-26T18:29:31.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.872+0000 7fa92effd700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa9180a3eb0 0x7fa9180a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:31.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.873+0000 7fa92effd700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa9180a3eb0 0x7fa9180a6370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa920005950 tx=0x7fa92000b410 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:31.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:31 smithi103 ceph-mon[32727]: pgmap v49: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-26T18:29:31.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:31 smithi103 ceph-mon[32727]: mgrmap e25: x(active, since 92s), standbys: y, z 2024-04-26T18:29:31.881 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:31.881+0000 7fa934a5d700 1 -- 172.21.15.84:0/2811471726 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa92405dff0 con 0x7fa9301083b0 2024-04-26T18:29:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.268+0000 7f52b0474640 1 -- 172.21.15.84:0/562688228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52a8101f40 msgr2=0x7f52a81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.268+0000 7f52b0474640 1 --2- 172.21.15.84:0/562688228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52a8101f40 0x7f52a81023c0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5298009930 tx=0x7f529802a420 comp rx=0 tx=0).stop 2024-04-26T18:29:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.268+0000 7f52b0474640 1 -- 172.21.15.84:0/562688228 shutdown_connections 2024-04-26T18:29:32.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.268+0000 7f52b0474640 1 --2- 172.21.15.84:0/562688228 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52a8102900 0x7f52a810e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.268+0000 7f52b0474640 1 --2- 172.21.15.84:0/562688228 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52a8101f40 0x7f52a81023c0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.268+0000 7f52b0474640 1 --2- 172.21.15.84:0/562688228 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52a8107a30 0x7f52a8107e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.268+0000 7f52b0474640 1 -- 172.21.15.84:0/562688228 >> 172.21.15.84:0/562688228 conn(0x7f52a8077dc0 msgr2=0x7f52a80ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.268+0000 7f52b0474640 1 -- 172.21.15.84:0/562688228 shutdown_connections 2024-04-26T18:29:32.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.268+0000 7f52b0474640 1 -- 172.21.15.84:0/562688228 wait complete. 2024-04-26T18:29:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.269+0000 7f52b0474640 1 Processor -- start 2024-04-26T18:29:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.269+0000 7f52b0474640 1 -- start start 2024-04-26T18:29:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.269+0000 7f52b0474640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52a8101f40 0x7f52a81970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.269+0000 7f52b0474640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52a8102900 0x7f52a81975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.269+0000 7f52ae210640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52a8101f40 0x7f52a81970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.270+0000 7f52ae210640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52a8101f40 0x7f52a81970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36040/0 (socket says 172.21.15.84:36040) 2024-04-26T18:29:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.270+0000 7f52ae210640 1 -- 172.21.15.84:0/552416763 learned_addr learned my addr 172.21.15.84:0/552416763 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.270+0000 7f52b0474640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52a8107a30 0x7f52a819b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.270+0000 7f52b0474640 1 -- 172.21.15.84:0/552416763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8110c30 con 0x7f52a8101f40 2024-04-26T18:29:32.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.270+0000 7f52b0474640 1 -- 172.21.15.84:0/552416763 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8110ab0 con 0x7f52a8107a30 2024-04-26T18:29:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.270+0000 7f52b0474640 1 -- 172.21.15.84:0/552416763 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f52a8110db0 con 0x7f52a8102900 2024-04-26T18:29:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.270+0000 7f52ae210640 1 -- 172.21.15.84:0/552416763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52a8102900 msgr2=0x7f52a81975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.270+0000 7f52ae210640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52a8102900 0x7f52a81975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.270+0000 7f52ae210640 1 -- 172.21.15.84:0/552416763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52a8107a30 msgr2=0x7f52a819b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.270+0000 7f52aea11640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52a8107a30 0x7f52a819b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.270+0000 7f52ae210640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52a8107a30 0x7f52a819b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.270+0000 7f52ae210640 1 -- 172.21.15.84:0/552416763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52a819c0a0 con 0x7f52a8101f40 2024-04-26T18:29:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.270+0000 7f52ae210640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52a8101f40 0x7f52a81970b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f52a4004650 tx=0x7f52a4003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.271+0000 7f529f7fe640 1 -- 172.21.15.84:0/552416763 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4009880 con 0x7f52a8101f40 2024-04-26T18:29:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.271+0000 7f52b0474640 1 -- 172.21.15.84:0/552416763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52a819c390 con 0x7f52a8101f40 2024-04-26T18:29:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.271+0000 7f529f7fe640 1 -- 172.21.15.84:0/552416763 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a4005020 con 0x7f52a8101f40 2024-04-26T18:29:32.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.271+0000 7f52b0474640 1 -- 172.21.15.84:0/552416763 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52a81a3c20 con 0x7f52a8101f40 2024-04-26T18:29:32.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.271+0000 7f529f7fe640 1 -- 172.21.15.84:0/552416763 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4004b60 con 0x7f52a8101f40 2024-04-26T18:29:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.274+0000 7f52b318a740 1 Processor -- start 2024-04-26T18:29:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.274+0000 7f529f7fe640 1 -- 172.21.15.84:0/552416763 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f52a401d020 con 0x7f52a8101f40 2024-04-26T18:29:32.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.275+0000 7f52b318a740 1 -- start start 2024-04-26T18:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.275+0000 7f52b318a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064396bf00 0x56064396c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.275+0000 7f52b318a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396c820 0x560643976ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.275+0000 7f52b318a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606439770e0 0x5606439794d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.275+0000 7f52b318a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5606438d2460 con 0x56064396c820 2024-04-26T18:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.275+0000 7f52b318a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56064388c590 con 0x5606439770e0 2024-04-26T18:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.275+0000 7f52b318a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5606438ce420 con 0x56064396bf00 2024-04-26T18:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.275+0000 7f52ada0f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606439770e0 0x5606439794d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.275+0000 7f52ada0f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606439770e0 0x5606439794d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:39188/0 (socket says 172.21.15.84:39188) 2024-04-26T18:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.275+0000 7f52ada0f640 1 -- 172.21.15.84:0/121498 learned_addr learned my addr 172.21.15.84:0/121498 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.275+0000 7f52ae210640 1 --2- 172.21.15.84:0/121498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396c820 0x560643976ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.275+0000 7f52aea11640 1 --2- 172.21.15.84:0/121498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064396bf00 0x56064396c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.276+0000 7f529f7fe640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f52940a3f20 0x7f52940a63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.276+0000 7f52ada0f640 1 -- 172.21.15.84:0/121498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064396bf00 msgr2=0x56064396c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.276+0000 7f52ada0f640 1 --2- 172.21.15.84:0/121498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064396bf00 0x56064396c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.276+0000 7f52ada0f640 1 -- 172.21.15.84:0/121498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396c820 msgr2=0x560643976ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.276+0000 7f52aea11640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f52940a3f20 0x7f52940a63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.276+0000 7f529f7fe640 1 -- 172.21.15.84:0/552416763 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f52a40d1e20 con 0x7f52a8101f40 2024-04-26T18:29:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.276+0000 7f52ada0f640 1 --2- 172.21.15.84:0/121498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396c820 0x560643976ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.276+0000 7f52ada0f640 1 -- 172.21.15.84:0/121498 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606438990d0 con 0x5606439770e0 2024-04-26T18:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.276+0000 7f52ae210640 1 --2- 172.21.15.84:0/121498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396c820 0x560643976ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.276+0000 7f52ada0f640 1 --2- 172.21.15.84:0/121498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606439770e0 0x5606439794d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f529802a940 tx=0x7f5298000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7afde8177b2ae3b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.276+0000 7f52aea11640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f52940a3f20 0x7f52940a63e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f52a0005fd0 tx=0x7f52a0004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.277+0000 7f529cbfe640 1 -- 172.21.15.84:0/121498 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529803e030 con 0x5606439770e0 2024-04-26T18:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.277+0000 7f529cbfe640 1 -- 172.21.15.84:0/121498 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5298004200 con 0x5606439770e0 2024-04-26T18:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.277+0000 7f529cbfe640 1 -- 172.21.15.84:0/121498 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5298035040 con 0x5606439770e0 2024-04-26T18:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.278+0000 7f52b318a740 1 -- 172.21.15.84:0/121498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606439770e0 msgr2=0x5606439794d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.278+0000 7f52b318a740 1 --2- 172.21.15.84:0/121498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606439770e0 0x5606439794d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f529802a940 tx=0x7f5298000bc0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.278+0000 7f52b318a740 1 -- 172.21.15.84:0/121498 shutdown_connections 2024-04-26T18:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.278+0000 7f52b318a740 1 --2- 172.21.15.84:0/121498 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5606439770e0 0x5606439794d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.278+0000 7f52b318a740 1 --2- 172.21.15.84:0/121498 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396c820 0x560643976ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.278+0000 7f52b318a740 1 --2- 172.21.15.84:0/121498 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064396bf00 0x56064396c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.278+0000 7f52b318a740 1 -- 172.21.15.84:0/121498 >> 172.21.15.84:0/121498 conn(0x560643968410 msgr2=0x5606439687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.278+0000 7f52b318a740 1 -- 172.21.15.84:0/121498 shutdown_connections 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.278+0000 7f52b318a740 1 -- 172.21.15.84:0/121498 wait complete. 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.278+0000 7f52b318a740 1 -- start start 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.278+0000 7f52b318a740 1 Processor -- start 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.279+0000 7f52b318a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396bf00 0x560643973f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.279+0000 7f52b318a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396c820 0x5606439744c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.279+0000 7f52b318a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606439770e0 0x560643974a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.279+0000 7f52ada0f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396c820 0x5606439744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.279+0000 7f52ada0f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396c820 0x5606439744c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36050/0 (socket says 172.21.15.84:36050) 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.279+0000 7f52ae210640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396bf00 0x560643973f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.279+0000 7f52ada0f640 1 -- 172.21.15.84:0/3774008051 learned_addr learned my addr 172.21.15.84:0/3774008051 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.279+0000 7f52ae210640 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396bf00 0x560643973f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:39200/0 (socket says 172.21.15.84:39200) 2024-04-26T18:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.279+0000 7f52b318a740 1 -- 172.21.15.84:0/3774008051 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56064397e0c0 con 0x56064396c820 2024-04-26T18:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.279+0000 7f52b318a740 1 -- 172.21.15.84:0/3774008051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560643859b60 con 0x56064396bf00 2024-04-26T18:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.279+0000 7f52b318a740 1 -- 172.21.15.84:0/3774008051 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56064397e240 con 0x5606439770e0 2024-04-26T18:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.279+0000 7f52b318a740 10 client.0 _ll_register_callbacks cb 0x56064387e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f52aea11640 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606439770e0 0x560643974a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f52ae210640 1 -- 172.21.15.84:0/3774008051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606439770e0 msgr2=0x560643974a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f52ae210640 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606439770e0 0x560643974a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f52ae210640 1 -- 172.21.15.84:0/3774008051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396c820 msgr2=0x5606439744c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f52ae210640 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396c820 0x5606439744c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f52ae210640 1 -- 172.21.15.84:0/3774008051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606435e4210 con 0x56064396bf00 2024-04-26T18:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f52aea11640 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606439770e0 0x560643974a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f52ada0f640 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396c820 0x5606439744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f52b318a740 1 -- 172.21.15.84:0/3774008051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5298004200 con 0x56064396bf00 2024-04-26T18:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f52ae210640 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396bf00 0x560643973f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f52a40cf560 tx=0x7f52a409f500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f2d3c3918e4dc7c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f5291ffb640 10 client.25085 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-26T18:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f5291ffb640 1 -- 172.21.15.84:0/3774008051 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a409ee00 con 0x56064396bf00 2024-04-26T18:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f5291ffb640 1 -- 172.21.15.84:0/3774008051 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f52a409b470 con 0x56064396bf00 2024-04-26T18:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f5291ffb640 1 -- 172.21.15.84:0/3774008051 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52a40a31b0 con 0x56064396bf00 2024-04-26T18:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f52b318a740 10 client.25085 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.280+0000 7f52b318a740 1 -- 172.21.15.84:0/3774008051 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5606436b0620 con 0x56064396bf00 2024-04-26T18:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.281+0000 7f5291ffb640 1 -- 172.21.15.84:0/3774008051 <== mon.2 v2:172.21.15.103:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f52a40d4040 con 0x56064396bf00 2024-04-26T18:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.281+0000 7f52b318a740 10 client.25085 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.281+0000 7f52b318a740 10 client.25085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.281+0000 7f52b318a740 20 client.25085 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.281+0000 7f52b318a740 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56064398ba10 0x56064398ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.281+0000 7f52b318a740 4 client.25085 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.281+0000 7f52b318a740 1 -- 172.21.15.84:0/3774008051 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606438e0070 con 0x56064398ba10 2024-04-26T18:29:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.281+0000 7f52ada0f640 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56064398ba10 0x56064398ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.282+0000 7f52ada0f640 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56064398ba10 0x56064398ded0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=318d9933c8549b29 server_cookie=2fb7d100db19b56 in_seq=0 out_seq=0 2024-04-26T18:29:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.282+0000 7f5291ffb640 10 client.25085 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.282+0000 7f5291ffb640 1 -- 172.21.15.84:0/3774008051 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606438e0070 con 0x56064398ba10 2024-04-26T18:29:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.282+0000 7f5291ffb640 10 client.25085 handle_command_reply: tid=0 2024-04-26T18:29:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.282+0000 7f52b318a740 1 client.25085 shutdown 2024-04-26T18:29:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.283+0000 7f52b318a740 1 -- 172.21.15.84:0/3774008051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396bf00 msgr2=0x560643973f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.283+0000 7f52b318a740 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396bf00 0x560643973f80 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f52a40cf560 tx=0x7f52a409f500 comp rx=0 tx=0).stop 2024-04-26T18:29:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.283+0000 7f52b318a740 1 -- 172.21.15.84:0/3774008051 shutdown_connections 2024-04-26T18:29:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.283+0000 7f52b318a740 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56064398ba10 0x56064398ded0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.283+0000 7f52b318a740 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606439770e0 0x560643974a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.283+0000 7f5291ffb640 0 client.25085 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.283+0000 7f52b318a740 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396c820 0x5606439744c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.283+0000 7f52b318a740 1 --2- 172.21.15.84:0/3774008051 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396bf00 0x560643973f80 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.283+0000 7f52b318a740 1 -- 172.21.15.84:0/3774008051 >> 172.21.15.84:0/3774008051 conn(0x560643968410 msgr2=0x56064396e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.283+0000 7f52b318a740 1 -- 172.21.15.84:0/3774008051 shutdown_connections 2024-04-26T18:29:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.283+0000 7f52b318a740 1 -- 172.21.15.84:0/3774008051 wait complete. 2024-04-26T18:29:32.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.283+0000 7f52b318a740 20 client.25085 trim_cache size 0 max 16384 2024-04-26T18:29:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.311+0000 7f52b318a740 1 Processor -- start 2024-04-26T18:29:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.312+0000 7f52b318a740 1 -- start start 2024-04-26T18:29:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.312+0000 7f52b318a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396bf00 0x560643986fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.312+0000 7f52b318a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396c820 0x56064396e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.312+0000 7f52b318a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606439770e0 0x5606439730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.312+0000 7f52ada0f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396c820 0x56064396e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.312+0000 7f52ada0f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396c820 0x56064396e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:39210/0 (socket says 172.21.15.84:39210) 2024-04-26T18:29:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.312+0000 7f52ada0f640 1 -- 172.21.15.84:0/1382993525 learned_addr learned my addr 172.21.15.84:0/1382993525 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.313+0000 7f52ae210640 1 --2- 172.21.15.84:0/1382993525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396bf00 0x560643986fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.313+0000 7f52aea11640 1 --2- 172.21.15.84:0/1382993525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606439770e0 0x5606439730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.313+0000 7f52b318a740 1 -- 172.21.15.84:0/1382993525 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f528c00e6c0 con 0x56064396bf00 2024-04-26T18:29:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.313+0000 7f52b318a740 1 -- 172.21.15.84:0/1382993525 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f528c010200 con 0x56064396c820 2024-04-26T18:29:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.313+0000 7f52b318a740 1 -- 172.21.15.84:0/1382993525 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f528c00ed90 con 0x5606439770e0 2024-04-26T18:29:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.313+0000 7f52ae210640 1 -- 172.21.15.84:0/1382993525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606439770e0 msgr2=0x5606439730e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.313+0000 7f52ae210640 1 --2- 172.21.15.84:0/1382993525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606439770e0 0x5606439730e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.313+0000 7f52ae210640 1 -- 172.21.15.84:0/1382993525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396c820 msgr2=0x56064396e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.313+0000 7f52ae210640 1 --2- 172.21.15.84:0/1382993525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396c820 0x56064396e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.313+0000 7f52ae210640 1 -- 172.21.15.84:0/1382993525 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56064396c650 con 0x56064396bf00 2024-04-26T18:29:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.313+0000 7f52aea11640 1 --2- 172.21.15.84:0/1382993525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606439770e0 0x5606439730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.313+0000 7f52ada0f640 1 --2- 172.21.15.84:0/1382993525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396c820 0x56064396e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.314+0000 7f52ae210640 1 --2- 172.21.15.84:0/1382993525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396bf00 0x560643986fb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f52a409d0b0 tx=0x7f52a40d1000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e8280c179a7109 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.314+0000 7f5293fff640 1 -- 172.21.15.84:0/1382993525 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a40a7ac0 con 0x56064396bf00 2024-04-26T18:29:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.314+0000 7f5293fff640 1 -- 172.21.15.84:0/1382993525 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5298004200 con 0x56064396bf00 2024-04-26T18:29:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.314+0000 7f5293fff640 1 -- 172.21.15.84:0/1382993525 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a409dcc0 con 0x56064396bf00 2024-04-26T18:29:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.315+0000 7f52b318a740 1 -- 172.21.15.84:0/1382993525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396bf00 msgr2=0x560643986fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.315+0000 7f52b318a740 1 --2- 172.21.15.84:0/1382993525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396bf00 0x560643986fb0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f52a409d0b0 tx=0x7f52a40d1000 comp rx=0 tx=0).stop 2024-04-26T18:29:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.315+0000 7f52b318a740 1 -- 172.21.15.84:0/1382993525 shutdown_connections 2024-04-26T18:29:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.315+0000 7f52b318a740 1 --2- 172.21.15.84:0/1382993525 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5606439770e0 0x5606439730e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.315+0000 7f52b318a740 1 --2- 172.21.15.84:0/1382993525 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396c820 0x56064396e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.315+0000 7f52b318a740 1 --2- 172.21.15.84:0/1382993525 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56064396bf00 0x560643986fb0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.315+0000 7f52b318a740 1 -- 172.21.15.84:0/1382993525 >> 172.21.15.84:0/1382993525 conn(0x560643968410 msgr2=0x560643966d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.315+0000 7f52b318a740 1 -- 172.21.15.84:0/1382993525 shutdown_connections 2024-04-26T18:29:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.315+0000 7f52b318a740 1 -- 172.21.15.84:0/1382993525 wait complete. 2024-04-26T18:29:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.315+0000 7f52b318a740 1 -- start start 2024-04-26T18:29:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.316+0000 7f52b318a740 1 Processor -- start 2024-04-26T18:29:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.316+0000 7f52b318a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064396bf00 0x56064397e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.316+0000 7f52b318a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396c820 0x56064397ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.316+0000 7f52b318a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5606439770e0 0x5606439f0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.316+0000 7f52b318a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f528c00fb30 con 0x5606439770e0 2024-04-26T18:29:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.316+0000 7f52aea11640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5606439770e0 0x5606439f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.316+0000 7f52ada0f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396c820 0x56064397ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.316+0000 7f52aea11640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5606439770e0 0x5606439f0000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36056/0 (socket says 172.21.15.84:36056) 2024-04-26T18:29:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.316+0000 7f52ada0f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396c820 0x56064397ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:39216/0 (socket says 172.21.15.84:39216) 2024-04-26T18:29:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.316+0000 7f52aea11640 1 -- 172.21.15.84:0/1015320026 learned_addr learned my addr 172.21.15.84:0/1015320026 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.316+0000 7f52ae210640 1 --2- 172.21.15.84:0/1015320026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064396bf00 0x56064397e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.316+0000 7f52b318a740 1 -- 172.21.15.84:0/1015320026 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f528c00f460 con 0x56064396c820 2024-04-26T18:29:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.316+0000 7f52b318a740 1 -- 172.21.15.84:0/1015320026 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f528c00c6a0 con 0x56064396bf00 2024-04-26T18:29:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.317+0000 7f52aea11640 1 -- 172.21.15.84:0/1015320026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064396bf00 msgr2=0x56064397e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.317+0000 7f52aea11640 1 --2- 172.21.15.84:0/1015320026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064396bf00 0x56064397e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.317+0000 7f52aea11640 1 -- 172.21.15.84:0/1015320026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396c820 msgr2=0x56064397ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.317+0000 7f52aea11640 1 --2- 172.21.15.84:0/1015320026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396c820 0x56064397ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.317+0000 7f52aea11640 1 -- 172.21.15.84:0/1015320026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606439f05f0 con 0x5606439770e0 2024-04-26T18:29:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.317+0000 7f52ae210640 1 --2- 172.21.15.84:0/1015320026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064396bf00 0x56064397e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.317+0000 7f52aea11640 1 --2- 172.21.15.84:0/1015320026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5606439770e0 0x5606439f0000 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009ee0 tx=0x7f52a0009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d830adcfe7071c5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.317+0000 7f5291ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:29:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.317+0000 7f52b318a740 10 client.0 _ll_register_callbacks cb 0x7f52a409e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.317+0000 7f52b318a740 1 -- 172.21.15.84:0/1015320026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5298004200 con 0x5606439770e0 2024-04-26T18:29:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.317+0000 7f5291ffb640 1 -- 172.21.15.84:0/1015320026 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a001e770 con 0x5606439770e0 2024-04-26T18:29:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.317+0000 7f5291ffb640 1 -- 172.21.15.84:0/1015320026 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f52a001e910 con 0x5606439770e0 2024-04-26T18:29:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.318+0000 7f5291ffb640 1 -- 172.21.15.84:0/1015320026 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f52a001eb10 con 0x5606439770e0 2024-04-26T18:29:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.318+0000 7f52b318a740 10 client.15633 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.318+0000 7f52b318a740 1 -- 172.21.15.84:0/1015320026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({fsmap=29}) -- 0x5606439f1780 con 0x5606439770e0 2024-04-26T18:29:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.318+0000 7f5291ffb640 1 -- 172.21.15.84:0/1015320026 <== mon.0 v2:172.21.15.84:3300/0 4 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f52a001ecb0 con 0x5606439770e0 2024-04-26T18:29:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.318+0000 7f52b318a740 10 client.15633 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.318+0000 7f52b318a740 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.318+0000 7f52b318a740 20 client.15633 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.318+0000 7f52b318a740 1 --2- 172.21.15.84:0/1015320026 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56064398ba10 0x5606439f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.318+0000 7f52b318a740 4 client.15633 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:29:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.318+0000 7f52b318a740 1 -- 172.21.15.84:0/1015320026 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f5298004200 con 0x56064398ba10 2024-04-26T18:29:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.318+0000 7f52ae210640 1 --2- 172.21.15.84:0/1015320026 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56064398ba10 0x5606439f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.319+0000 7f52ae210640 1 --2- 172.21.15.84:0/1015320026 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56064398ba10 0x5606439f3e90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d4bd84687995a0d server_cookie=98407e58034df527 in_seq=0 out_seq=0 2024-04-26T18:29:32.323 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.319+0000 7f5291ffb640 10 client.15633 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.324+0000 7f5291ffb640 1 -- 172.21.15.84:0/1015320026 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f5298004200 con 0x56064398ba10 2024-04-26T18:29:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.324+0000 7f5291ffb640 10 client.15633 handle_command_reply: tid=0 2024-04-26T18:29:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.324+0000 7f52b318a740 1 client.15633 shutdown 2024-04-26T18:29:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.325+0000 7f52b318a740 1 -- 172.21.15.84:0/1015320026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5606439770e0 msgr2=0x5606439f0000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.325+0000 7f52b318a740 1 --2- 172.21.15.84:0/1015320026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5606439770e0 0x5606439f0000 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f52a0009ee0 tx=0x7f52a0009500 comp rx=0 tx=0).stop 2024-04-26T18:29:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.325+0000 7f52b318a740 1 -- 172.21.15.84:0/1015320026 shutdown_connections 2024-04-26T18:29:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.325+0000 7f52b318a740 1 --2- 172.21.15.84:0/1015320026 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56064398ba10 0x5606439f3e90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.325+0000 7f52b318a740 1 --2- 172.21.15.84:0/1015320026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5606439770e0 0x5606439f0000 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.325 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.325+0000 7f52b318a740 1 --2- 172.21.15.84:0/1015320026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56064396c820 0x56064397ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.325+0000 7f52b318a740 1 --2- 172.21.15.84:0/1015320026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56064396bf00 0x56064397e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.325+0000 7f52b318a740 1 -- 172.21.15.84:0/1015320026 >> 172.21.15.84:0/1015320026 conn(0x560643968410 msgr2=0x560643966d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.325+0000 7f5291ffb640 0 client.15633 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.325+0000 7f52b318a740 1 -- 172.21.15.84:0/1015320026 shutdown_connections 2024-04-26T18:29:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.325+0000 7f52b318a740 1 -- 172.21.15.84:0/1015320026 wait complete. 2024-04-26T18:29:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.325+0000 7f52b318a740 20 client.15633 trim_cache size 0 max 16384 2024-04-26T18:29:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:29:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:29:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:29:32.326 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:29:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.326+0000 7f52b0474640 1 -- 172.21.15.84:0/552416763 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f52940a3f20 msgr2=0x7f52940a63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.326+0000 7f52b0474640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f52940a3f20 0x7f52940a63e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f52a0005fd0 tx=0x7f52a0004a40 comp rx=0 tx=0).stop 2024-04-26T18:29:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.326+0000 7f52b0474640 1 -- 172.21.15.84:0/552416763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52a8101f40 msgr2=0x7f52a81970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.326+0000 7f52b0474640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52a8101f40 0x7f52a81970b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f52a4004650 tx=0x7f52a4003aa0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.326+0000 7f52b0474640 1 -- 172.21.15.84:0/552416763 shutdown_connections 2024-04-26T18:29:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.326+0000 7f52b0474640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f52940a3f20 0x7f52940a63e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.326+0000 7f52b0474640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f52a8107a30 0x7f52a819b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.326+0000 7f52b0474640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f52a8102900 0x7f52a81975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.327+0000 7f52b0474640 1 --2- 172.21.15.84:0/552416763 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f52a8101f40 0x7f52a81970b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.327+0000 7f52b0474640 1 -- 172.21.15.84:0/552416763 >> 172.21.15.84:0/552416763 conn(0x7f52a8077dc0 msgr2=0x7f52a81048d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.327+0000 7f52b0474640 1 -- 172.21.15.84:0/552416763 shutdown_connections 2024-04-26T18:29:32.329 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:32.327+0000 7f52b0474640 1 -- 172.21.15.84:0/552416763 wait complete. 2024-04-26T18:29:32.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.386+0000 7fa935a5f700 1 -- 172.21.15.84:0/2811471726 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa93002e650 con 0x7fa9301083b0 2024-04-26T18:29:32.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.387+0000 7fa934a5d700 1 -- 172.21.15.84:0/2811471726 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7fa924013070 con 0x7fa9301083b0 2024-04-26T18:29:32.387 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:29:32.387 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:29:32.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.390+0000 7fa935a5f700 1 -- 172.21.15.84:0/2811471726 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa9180a3eb0 msgr2=0x7fa9180a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:32.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.390+0000 7fa935a5f700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa9180a3eb0 0x7fa9180a6370 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa920005950 tx=0x7fa92000b410 comp rx=0 tx=0).stop 2024-04-26T18:29:32.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.390+0000 7fa935a5f700 1 -- 172.21.15.84:0/2811471726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9301083b0 msgr2=0x7fa930075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:32.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.390+0000 7fa935a5f700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9301083b0 0x7fa930075c60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa924009f50 tx=0x7fa924003a90 comp rx=0 tx=0).stop 2024-04-26T18:29:32.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.390+0000 7fa935a5f700 1 -- 172.21.15.84:0/2811471726 shutdown_connections 2024-04-26T18:29:32.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.390+0000 7fa935a5f700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa9180a3eb0 0x7fa9180a6370 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.390+0000 7fa935a5f700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa9300761a0 0x7fa9301a7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.390+0000 7fa935a5f700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa930103e90 0x7fa930075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.390+0000 7fa935a5f700 1 --2- 172.21.15.84:0/2811471726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa9301083b0 0x7fa930075c60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:32.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.390+0000 7fa935a5f700 1 -- 172.21.15.84:0/2811471726 >> 172.21.15.84:0/2811471726 conn(0x7fa93007a580 msgr2=0x7fa9301015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:32.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.391+0000 7fa935a5f700 1 -- 172.21.15.84:0/2811471726 shutdown_connections 2024-04-26T18:29:32.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:32.391+0000 7fa935a5f700 1 -- 172.21.15.84:0/2811471726 wait complete. 2024-04-26T18:29:32.392 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:29:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:32 smithi174 ceph-mon[32705]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:32.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:32 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2811471726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:29:32.698 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:32 smithi084 ceph-mon[29141]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:32.698 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:32 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2811471726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:29:32.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:32 smithi103 ceph-mon[32727]: pgmap v50: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:32.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:32 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2811471726' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:29:33.151 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id f5df7d40 --release --await' 2024-04-26T18:29:33.151 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db --set-id f5df7d40 --release --await 2024-04-26T18:29:33.418 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:29:34.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:34 smithi084 ceph-mon[29141]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-04-26T18:29:34.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:34 smithi103 ceph-mon[32727]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-04-26T18:29:34.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:34 smithi174 ceph-mon[32705]: pgmap v51: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 1.6 KiB/s wr, 15 op/s 2024-04-26T18:29:35.290 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.290+0000 7f614f7a5700 1 -- 172.21.15.84:0/3098124445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148102320 msgr2=0x7f61481027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:35.290 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.290+0000 7f614f7a5700 1 --2- 172.21.15.84:0/3098124445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148102320 0x7f61481027a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f6138009990 tx=0x7f6138009ca0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.290+0000 7f614f7a5700 1 -- 172.21.15.84:0/3098124445 shutdown_connections 2024-04-26T18:29:35.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.290+0000 7f614f7a5700 1 --2- 172.21.15.84:0/3098124445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6148102ce0 0x7f614810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.290+0000 7f614f7a5700 1 --2- 172.21.15.84:0/3098124445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6148108390 0x7f6148108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.290+0000 7f614f7a5700 1 --2- 172.21.15.84:0/3098124445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148102320 0x7f61481027a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.290+0000 7f614f7a5700 1 -- 172.21.15.84:0/3098124445 >> 172.21.15.84:0/3098124445 conn(0x7f61480792d0 msgr2=0x7f61480796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:35.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.290+0000 7f614f7a5700 1 -- 172.21.15.84:0/3098124445 shutdown_connections 2024-04-26T18:29:35.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.290+0000 7f614f7a5700 1 -- 172.21.15.84:0/3098124445 wait complete. 2024-04-26T18:29:35.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.291+0000 7f614f7a5700 1 Processor -- start 2024-04-26T18:29:35.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.291+0000 7f614f7a5700 1 -- start start 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.291+0000 7f614f7a5700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6148102320 0x7f614819e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.291+0000 7f614f7a5700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6148102ce0 0x7f614819ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.291+0000 7f614f7a5700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148108390 0x7f6148198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.291+0000 7f614f7a5700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61481124e0 con 0x7f6148108390 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.291+0000 7f614f7a5700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6148112360 con 0x7f6148102320 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.291+0000 7f614f7a5700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6148112660 con 0x7f6148102ce0 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.292+0000 7f614cd40700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6148102ce0 0x7f614819ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.292+0000 7f614dd42700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148108390 0x7f6148198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.292+0000 7f614d541700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6148102320 0x7f614819e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.292+0000 7f614d541700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6148102320 0x7f614819e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:39244/0 (socket says 172.21.15.84:39244) 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.292+0000 7f614d541700 1 -- 172.21.15.84:0/2523929465 learned_addr learned my addr 172.21.15.84:0/2523929465 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.292+0000 7f614d541700 1 -- 172.21.15.84:0/2523929465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6148102ce0 msgr2=0x7f614819ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.292+0000 7f614d541700 1 --2- 172.21.15.84:0/2523929465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6148102ce0 0x7f614819ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.292+0000 7f614d541700 1 -- 172.21.15.84:0/2523929465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148108390 msgr2=0x7f6148198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:35.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.292+0000 7f614d541700 1 --2- 172.21.15.84:0/2523929465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148108390 0x7f6148198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.292+0000 7f614d541700 1 -- 172.21.15.84:0/2523929465 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6148199020 con 0x7f6148102320 2024-04-26T18:29:35.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.292+0000 7f614dd42700 1 --2- 172.21.15.84:0/2523929465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148108390 0x7f6148198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:35.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.293+0000 7f614d541700 1 --2- 172.21.15.84:0/2523929465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6148102320 0x7f614819e670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f614400c9b0 tx=0x7f614400ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:35.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.293+0000 7f613e7fc700 1 -- 172.21.15.84:0/2523929465 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144007b00 con 0x7f6148102320 2024-04-26T18:29:35.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.293+0000 7f614f7a5700 1 -- 172.21.15.84:0/2523929465 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6148199310 con 0x7f6148102320 2024-04-26T18:29:35.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.293+0000 7f613e7fc700 1 -- 172.21.15.84:0/2523929465 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6144007ca0 con 0x7f6148102320 2024-04-26T18:29:35.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.293+0000 7f613e7fc700 1 -- 172.21.15.84:0/2523929465 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6144018660 con 0x7f6148102320 2024-04-26T18:29:35.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.293+0000 7f614f7a5700 1 -- 172.21.15.84:0/2523929465 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614806b100 con 0x7f6148102320 2024-04-26T18:29:35.295 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.295+0000 7f613e7fc700 1 -- 172.21.15.84:0/2523929465 <== mon.2 v2:172.21.15.103:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f614400f450 con 0x7f6148102320 2024-04-26T18:29:35.296 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.296+0000 7f613e7fc700 1 --2- 172.21.15.84:0/2523929465 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f61340a8240 0x7f61340aa700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.296 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.297+0000 7f613e7fc700 1 -- 172.21.15.84:0/2523929465 <== mon.2 v2:172.21.15.103:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f61440d0760 con 0x7f6148102320 2024-04-26T18:29:35.297 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.297+0000 7f614cd40700 1 --2- 172.21.15.84:0/2523929465 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f61340a8240 0x7f61340aa700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.297 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.297+0000 7f614cd40700 1 --2- 172.21.15.84:0/2523929465 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f61340a8240 0x7f61340aa700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6138006010 tx=0x7f613801a040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:35.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.298+0000 7f615a037b80 1 Processor -- start 2024-04-26T18:29:35.298 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.298+0000 7f615a037b80 1 -- start start 2024-04-26T18:29:35.299 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.298+0000 7f615a037b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f8411a0 0x56480f841580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.299 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f615a037b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f84c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.299 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f615a037b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f84c970 0x56480f84ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.299 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f615a037b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56480f794840 con 0x56480f84c970 2024-04-26T18:29:35.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f614cd40700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f84c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f614cd40700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f84c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.84:39410/0 (socket says 172.21.15.84:39410) 2024-04-26T18:29:35.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f614cd40700 1 -- 172.21.15.84:0/30664308 learned_addr learned my addr 172.21.15.84:0/30664308 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:35.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f615a037b80 1 -- 172.21.15.84:0/30664308 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56480f7336e0 con 0x56480f8411a0 2024-04-26T18:29:35.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f615a037b80 1 -- 172.21.15.84:0/30664308 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56480f790800 con 0x56480f841ac0 2024-04-26T18:29:35.300 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f614dd42700 1 --2- 172.21.15.84:0/30664308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f84c970 0x56480f84ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f614d541700 1 --2- 172.21.15.84:0/30664308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f8411a0 0x56480f841580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f614cd40700 1 -- 172.21.15.84:0/30664308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f8411a0 msgr2=0x56480f841580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:35.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f614cd40700 1 --2- 172.21.15.84:0/30664308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f8411a0 0x56480f841580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f614cd40700 1 -- 172.21.15.84:0/30664308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f84c970 msgr2=0x56480f84ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:35.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f614cd40700 1 --2- 172.21.15.84:0/30664308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f84c970 0x56480f84ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.299+0000 7f614cd40700 1 -- 172.21.15.84:0/30664308 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56480f6e19d0 con 0x56480f841ac0 2024-04-26T18:29:35.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.300+0000 7f614dd42700 1 --2- 172.21.15.84:0/30664308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f84c970 0x56480f84ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:35.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.300+0000 7f614cd40700 1 --2- 172.21.15.84:0/30664308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f84c430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f613800ba80 tx=0x7f613801e000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25d6c858589b9b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:35.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.300+0000 7f61337fe700 1 -- 172.21.15.84:0/30664308 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f613800f8f0 con 0x56480f841ac0 2024-04-26T18:29:35.301 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.300+0000 7f61337fe700 1 -- 172.21.15.84:0/30664308 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6138032070 con 0x56480f841ac0 2024-04-26T18:29:35.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.301+0000 7f61337fe700 1 -- 172.21.15.84:0/30664308 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6138023e60 con 0x56480f841ac0 2024-04-26T18:29:35.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.301+0000 7f615a037b80 1 -- 172.21.15.84:0/30664308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 msgr2=0x56480f84c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:35.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.301+0000 7f615a037b80 1 --2- 172.21.15.84:0/30664308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f84c430 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f613800ba80 tx=0x7f613801e000 comp rx=0 tx=0).stop 2024-04-26T18:29:35.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.301+0000 7f615a037b80 1 -- 172.21.15.84:0/30664308 shutdown_connections 2024-04-26T18:29:35.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.301+0000 7f615a037b80 1 --2- 172.21.15.84:0/30664308 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f84c970 0x56480f84ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.301+0000 7f615a037b80 1 --2- 172.21.15.84:0/30664308 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f8411a0 0x56480f841580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.301+0000 7f615a037b80 1 --2- 172.21.15.84:0/30664308 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f84c430 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.301+0000 7f615a037b80 1 -- 172.21.15.84:0/30664308 >> 172.21.15.84:0/30664308 conn(0x56480f76ee40 msgr2=0x56480f83d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:35.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.301+0000 7f615a037b80 1 -- 172.21.15.84:0/30664308 shutdown_connections 2024-04-26T18:29:35.305 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.301+0000 7f615a037b80 1 -- 172.21.15.84:0/30664308 wait complete. 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.302+0000 7f615a037b80 1 -- start start 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.302+0000 7f615a037b80 1 Processor -- start 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.302+0000 7f615a037b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 0x56480f84c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.302+0000 7f615a037b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f8493b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.302+0000 7f615a037b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f84c970 0x56480f8498f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f615a037b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56480f84f580 con 0x56480f8411a0 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f615a037b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56480f6faa50 con 0x56480f84c970 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f615a037b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56480f84f700 con 0x56480f841ac0 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f614d541700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 0x56480f84c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f614d541700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 0x56480f84c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36102/0 (socket says 172.21.15.84:36102) 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f614d541700 1 -- 172.21.15.84:0/432878068 learned_addr learned my addr 172.21.15.84:0/432878068 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f615a037b80 10 client.0 _ll_register_callbacks cb 0x56480f728680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f614dd42700 1 --2- 172.21.15.84:0/432878068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f84c970 0x56480f8498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.306 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f614cd40700 1 --2- 172.21.15.84:0/432878068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f8493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f614d541700 1 -- 172.21.15.84:0/432878068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 msgr2=0x56480f8493b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f614d541700 1 --2- 172.21.15.84:0/432878068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f8493b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f614d541700 1 -- 172.21.15.84:0/432878068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f84c970 msgr2=0x56480f8498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f614d541700 1 --2- 172.21.15.84:0/432878068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f84c970 0x56480f8498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f614d541700 1 -- 172.21.15.84:0/432878068 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56480f849f60 con 0x56480f8411a0 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f614d541700 1 --2- 172.21.15.84:0/432878068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 0x56480f84c0b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f614400afd0 tx=0x7f614409c2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe2f4a0dacc8fae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.303+0000 7f6130ff9700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.304+0000 7f6130ff9700 1 -- 172.21.15.84:0/432878068 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f614409c690 con 0x56480f8411a0 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.304+0000 7f6130ff9700 1 -- 172.21.15.84:0/432878068 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f614409c8f0 con 0x56480f8411a0 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.304+0000 7f615a037b80 1 -- 172.21.15.84:0/432878068 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6138032070 con 0x56480f8411a0 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.304+0000 7f6130ff9700 1 -- 172.21.15.84:0/432878068 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f61440a0a40 con 0x56480f8411a0 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.304+0000 7f615a037b80 10 client.15645 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.304+0000 7f615a037b80 1 -- 172.21.15.84:0/432878068 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56480f85a050 con 0x56480f8411a0 2024-04-26T18:29:35.307 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.304+0000 7f6130ff9700 1 -- 172.21.15.84:0/432878068 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61440a5300 con 0x56480f8411a0 2024-04-26T18:29:35.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.304+0000 7f6130ff9700 1 -- 172.21.15.84:0/432878068 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f61440a4e70 con 0x56480f8411a0 2024-04-26T18:29:35.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.305+0000 7f615a037b80 10 client.15645 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:35.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.305+0000 7f615a037b80 10 client.15645 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:29:35.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.305+0000 7f615a037b80 20 client.15645 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:35.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.305+0000 7f615a037b80 1 --2- 172.21.15.84:0/432878068 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56480f85ca40 0x56480f85ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.305+0000 7f615a037b80 4 client.15645 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:35.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.305+0000 7f615a037b80 1 -- 172.21.15.84:0/432878068 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6138032070 con 0x56480f85ca40 2024-04-26T18:29:35.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.305+0000 7f614cd40700 1 --2- 172.21.15.84:0/432878068 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56480f85ca40 0x56480f85ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.305+0000 7f614cd40700 1 --2- 172.21.15.84:0/432878068 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56480f85ca40 0x56480f85ee80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55e201dc7a49bdee server_cookie=ba03ee1cc7fe7761 in_seq=0 out_seq=0 2024-04-26T18:29:35.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.305+0000 7f6130ff9700 10 client.15645 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:35.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.306+0000 7f6130ff9700 1 -- 172.21.15.84:0/432878068 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f6138032070 con 0x56480f85ca40 2024-04-26T18:29:35.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.306+0000 7f6130ff9700 10 client.15645 handle_command_reply: tid=0 2024-04-26T18:29:35.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.306+0000 7f615a037b80 1 client.15645 shutdown 2024-04-26T18:29:35.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.306+0000 7f615a037b80 1 -- 172.21.15.84:0/432878068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 msgr2=0x56480f84c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:35.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.306+0000 7f615a037b80 1 --2- 172.21.15.84:0/432878068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 0x56480f84c0b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f614400afd0 tx=0x7f614409c2c0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.306+0000 7f615a037b80 1 -- 172.21.15.84:0/432878068 shutdown_connections 2024-04-26T18:29:35.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.306+0000 7f615a037b80 1 --2- 172.21.15.84:0/432878068 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56480f85ca40 0x56480f85ee80 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.306+0000 7f615a037b80 1 --2- 172.21.15.84:0/432878068 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 0x56480f84c0b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.306+0000 7f615a037b80 1 --2- 172.21.15.84:0/432878068 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f84c970 0x56480f8498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.306+0000 7f615a037b80 1 --2- 172.21.15.84:0/432878068 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f8493b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.306+0000 7f6130ff9700 0 client.15645 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:35.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.306+0000 7f615a037b80 1 -- 172.21.15.84:0/432878068 >> 172.21.15.84:0/432878068 conn(0x56480f76ee40 msgr2=0x56480f83d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:35.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.307+0000 7f615a037b80 1 -- 172.21.15.84:0/432878068 shutdown_connections 2024-04-26T18:29:35.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.307+0000 7f615a037b80 1 -- 172.21.15.84:0/432878068 wait complete. 2024-04-26T18:29:35.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.307+0000 7f615a037b80 20 client.15645 trim_cache size 0 max 16384 2024-04-26T18:29:35.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.344+0000 7f615a037b80 1 Processor -- start 2024-04-26T18:29:35.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.344+0000 7f615a037b80 1 -- start start 2024-04-26T18:29:35.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.344+0000 7f615a037b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f8411a0 0x56480f8437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.344+0000 7f615a037b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f84bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.344+0000 7f615a037b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f84c970 0x56480f864790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f615a037b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61481ab9c0 con 0x56480f84c970 2024-04-26T18:29:35.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f615a037b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f61481ad500 con 0x56480f8411a0 2024-04-26T18:29:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f615a037b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61481ac090 con 0x56480f841ac0 2024-04-26T18:29:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f614dd42700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f84c970 0x56480f864790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f614dd42700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f84c970 0x56480f864790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36110/0 (socket says 172.21.15.84:36110) 2024-04-26T18:29:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f614dd42700 1 -- 172.21.15.84:0/2715013952 learned_addr learned my addr 172.21.15.84:0/2715013952 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f614d541700 1 --2- 172.21.15.84:0/2715013952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f8411a0 0x56480f8437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f614cd40700 1 --2- 172.21.15.84:0/2715013952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f84bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f614dd42700 1 -- 172.21.15.84:0/2715013952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 msgr2=0x56480f84bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f614dd42700 1 --2- 172.21.15.84:0/2715013952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f84bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f614dd42700 1 -- 172.21.15.84:0/2715013952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f8411a0 msgr2=0x56480f8437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f614dd42700 1 --2- 172.21.15.84:0/2715013952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f8411a0 0x56480f8437b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f614dd42700 1 -- 172.21.15.84:0/2715013952 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56480f83eb30 con 0x56480f84c970 2024-04-26T18:29:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f614cd40700 1 --2- 172.21.15.84:0/2715013952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f84bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:35.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.345+0000 7f614dd42700 1 --2- 172.21.15.84:0/2715013952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f84c970 0x56480f864790 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6140003e30 tx=0x7f6140003e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=527658d465abc61c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:35.348 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.346+0000 7f6132ffd700 1 -- 172.21.15.84:0/2715013952 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140020470 con 0x56480f84c970 2024-04-26T18:29:35.348 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.346+0000 7f6132ffd700 1 -- 172.21.15.84:0/2715013952 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f614001b070 con 0x56480f84c970 2024-04-26T18:29:35.348 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.346+0000 7f6132ffd700 1 -- 172.21.15.84:0/2715013952 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140017970 con 0x56480f84c970 2024-04-26T18:29:35.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.347+0000 7f615a037b80 1 -- 172.21.15.84:0/2715013952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f84c970 msgr2=0x56480f864790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:35.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.347+0000 7f615a037b80 1 --2- 172.21.15.84:0/2715013952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f84c970 0x56480f864790 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f6140003e30 tx=0x7f6140003e60 comp rx=0 tx=0).stop 2024-04-26T18:29:35.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.347+0000 7f615a037b80 1 -- 172.21.15.84:0/2715013952 shutdown_connections 2024-04-26T18:29:35.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.347+0000 7f615a037b80 1 --2- 172.21.15.84:0/2715013952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f84c970 0x56480f864790 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.347+0000 7f615a037b80 1 --2- 172.21.15.84:0/2715013952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f8411a0 0x56480f8437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.347+0000 7f615a037b80 1 --2- 172.21.15.84:0/2715013952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f84bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.347+0000 7f615a037b80 1 -- 172.21.15.84:0/2715013952 >> 172.21.15.84:0/2715013952 conn(0x56480f76ee40 msgr2=0x56480f8430a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:35.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.347+0000 7f615a037b80 1 -- 172.21.15.84:0/2715013952 shutdown_connections 2024-04-26T18:29:35.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.347+0000 7f615a037b80 1 -- 172.21.15.84:0/2715013952 wait complete. 2024-04-26T18:29:35.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.347+0000 7f615a037b80 1 -- start start 2024-04-26T18:29:35.353 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.347+0000 7f615a037b80 1 Processor -- start 2024-04-26T18:29:35.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.348+0000 7f615a037b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 0x56480f8467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.348+0000 7f614d541700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 0x56480f8467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.348+0000 7f614d541700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 0x56480f8467e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36126/0 (socket says 172.21.15.84:36126) 2024-04-26T18:29:35.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.348+0000 7f615a037b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f846d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.348+0000 7f615a037b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f84c970 0x56480f8c2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.348+0000 7f615a037b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61481ace30 con 0x56480f8411a0 2024-04-26T18:29:35.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.348+0000 7f615a037b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f61481ac760 con 0x56480f84c970 2024-04-26T18:29:35.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.348+0000 7f615a037b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61481a9960 con 0x56480f841ac0 2024-04-26T18:29:35.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.348+0000 7f614d541700 1 -- 172.21.15.84:0/974441823 learned_addr learned my addr 172.21.15.84:0/974441823 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:35.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.348+0000 7f615a037b80 10 client.0 _ll_register_callbacks cb 0x56480f728680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:35.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.348+0000 7f614dd42700 1 --2- 172.21.15.84:0/974441823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f84c970 0x56480f8c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.349+0000 7f614d541700 1 -- 172.21.15.84:0/974441823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 msgr2=0x56480f846d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:35.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.349+0000 7f614d541700 1 --2- 172.21.15.84:0/974441823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f846d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.349+0000 7f614d541700 1 -- 172.21.15.84:0/974441823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f84c970 msgr2=0x56480f8c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:35.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.349+0000 7f614d541700 1 --2- 172.21.15.84:0/974441823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f84c970 0x56480f8c2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.349+0000 7f614d541700 1 -- 172.21.15.84:0/974441823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56480f8c2540 con 0x56480f8411a0 2024-04-26T18:29:35.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.349+0000 7f614d541700 1 --2- 172.21.15.84:0/974441823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 0x56480f8467e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6144087010 tx=0x7f614409aba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58f85956cd82ba02 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:35.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.349+0000 7f6130ff9700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:29:35.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.349+0000 7f6130ff9700 1 -- 172.21.15.84:0/974441823 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61440a5300 con 0x56480f8411a0 2024-04-26T18:29:35.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.349+0000 7f6130ff9700 1 -- 172.21.15.84:0/974441823 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f614409bcf0 con 0x56480f8411a0 2024-04-26T18:29:35.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.349+0000 7f615a037b80 1 -- 172.21.15.84:0/974441823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f614001b070 con 0x56480f8411a0 2024-04-26T18:29:35.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.349+0000 7f6130ff9700 1 -- 172.21.15.84:0/974441823 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f614409c690 con 0x56480f8411a0 2024-04-26T18:29:35.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.349+0000 7f615a037b80 10 client.15651 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.349+0000 7f615a037b80 1 -- 172.21.15.84:0/974441823 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56480f8c3850 con 0x56480f8411a0 2024-04-26T18:29:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.350+0000 7f6130ff9700 1 -- 172.21.15.84:0/974441823 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6144082070 con 0x56480f8411a0 2024-04-26T18:29:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.350+0000 7f6130ff9700 1 -- 172.21.15.84:0/974441823 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f614409b550 con 0x56480f8411a0 2024-04-26T18:29:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.350+0000 7f615a037b80 10 client.15651 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.350+0000 7f615a037b80 10 client.15651 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:29:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.350+0000 7f615a037b80 20 client.15651 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.350+0000 7f615a037b80 1 --2- 172.21.15.84:0/974441823 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56480f8c3da0 0x56480f8c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.350+0000 7f615a037b80 4 client.15651 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f5df7d40", "release": true, "await": true}] 2024-04-26T18:29:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.350+0000 7f615a037b80 1 -- 172.21.15.84:0/974441823 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f5df7d40", "release": true, "await": true}) -- 0x7f614001b070 con 0x56480f8c3da0 2024-04-26T18:29:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.350+0000 7f614cd40700 1 --2- 172.21.15.84:0/974441823 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56480f8c3da0 0x56480f8c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.351+0000 7f6130ff9700 10 client.15651 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:35.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.351+0000 7f614cd40700 1 --2- 172.21.15.84:0/974441823 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56480f8c3da0 0x56480f8c61c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cc7521670a0f5d4 server_cookie=80a3da6f8826823a in_seq=0 out_seq=0 2024-04-26T18:29:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.353+0000 7f6130ff9700 1 -- 172.21.15.84:0/974441823 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x7f614001b070 con 0x56480f8c3da0 2024-04-26T18:29:35.358 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.353+0000 7f6130ff9700 10 client.15651 handle_command_reply: tid=0 2024-04-26T18:29:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.353+0000 7f615a037b80 1 client.15651 shutdown 2024-04-26T18:29:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.354+0000 7f615a037b80 1 -- 172.21.15.84:0/974441823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 msgr2=0x56480f8467e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.354+0000 7f615a037b80 1 --2- 172.21.15.84:0/974441823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 0x56480f8467e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6144087010 tx=0x7f614409aba0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.354+0000 7f615a037b80 1 -- 172.21.15.84:0/974441823 shutdown_connections 2024-04-26T18:29:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.354+0000 7f615a037b80 1 --2- 172.21.15.84:0/974441823 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56480f8c3da0 0x56480f8c61c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.354+0000 7f6130ff9700 0 client.15651 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.354+0000 7f615a037b80 1 --2- 172.21.15.84:0/974441823 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56480f8411a0 0x56480f8467e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.354+0000 7f615a037b80 1 --2- 172.21.15.84:0/974441823 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56480f84c970 0x56480f8c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.354+0000 7f615a037b80 1 --2- 172.21.15.84:0/974441823 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56480f841ac0 0x56480f846d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.354+0000 7f615a037b80 1 -- 172.21.15.84:0/974441823 >> 172.21.15.84:0/974441823 conn(0x56480f76ee40 msgr2=0x56480f843930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.354+0000 7f615a037b80 1 -- 172.21.15.84:0/974441823 shutdown_connections 2024-04-26T18:29:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.354+0000 7f615a037b80 1 -- 172.21.15.84:0/974441823 wait complete. 2024-04-26T18:29:35.359 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.354+0000 7f615a037b80 20 client.15651 trim_cache size 0 max 16384 2024-04-26T18:29:35.360 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:29:35.360 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:29:35.360 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:29:35.360 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 4, 2024-04-26T18:29:35.360 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:29:35.360 INFO:teuthology.orchestra.run.smithi084.stdout: "f5df7d40": { 2024-04-26T18:29:35.361 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 4, 2024-04-26T18:29:35.361 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:29:35.361 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:29:35.361 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:29:35.361 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:29:35.361 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:29:35.361 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:29:35.361 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 90.0, 2024-04-26T18:29:35.361 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:29:35.362 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:29:35.362 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:29:35.362 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:29:35.362 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:29:35.362 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:29:35.362 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:29:35.362 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:29:35.362 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:29:35.362 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:29:35.363 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:29:35.363 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:29:35.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.355+0000 7f614f7a5700 1 -- 172.21.15.84:0/2523929465 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f61340a8240 msgr2=0x7f61340aa700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:35.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.355+0000 7f614f7a5700 1 --2- 172.21.15.84:0/2523929465 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f61340a8240 0x7f61340aa700 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6138006010 tx=0x7f613801a040 comp rx=0 tx=0).stop 2024-04-26T18:29:35.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.355+0000 7f614f7a5700 1 -- 172.21.15.84:0/2523929465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6148102320 msgr2=0x7f614819e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:35.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.355+0000 7f614f7a5700 1 --2- 172.21.15.84:0/2523929465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6148102320 0x7f614819e670 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f614400c9b0 tx=0x7f614400ccc0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.355+0000 7f614f7a5700 1 -- 172.21.15.84:0/2523929465 shutdown_connections 2024-04-26T18:29:35.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.355+0000 7f614f7a5700 1 --2- 172.21.15.84:0/2523929465 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f61340a8240 0x7f61340aa700 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.355+0000 7f614f7a5700 1 --2- 172.21.15.84:0/2523929465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6148108390 0x7f6148198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.355+0000 7f614f7a5700 1 --2- 172.21.15.84:0/2523929465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6148102320 0x7f614819e670 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.355+0000 7f614f7a5700 1 --2- 172.21.15.84:0/2523929465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6148102ce0 0x7f614819ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:35.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.355+0000 7f614f7a5700 1 -- 172.21.15.84:0/2523929465 >> 172.21.15.84:0/2523929465 conn(0x7f61480792d0 msgr2=0x7f6148100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:35.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.355+0000 7f614f7a5700 1 -- 172.21.15.84:0/2523929465 shutdown_connections 2024-04-26T18:29:35.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:35.355+0000 7f614f7a5700 1 -- 172.21.15.84:0/2523929465 wait complete. 2024-04-26T18:29:36.185 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f5df7d40', seconds elapsed: 42.7 2024-04-26T18:29:36.185 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-04-26T18:29:36.186 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:29:36.432 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:29:36.812 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:36 smithi084 ceph-mon[29141]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:36.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:36 smithi103 ceph-mon[32727]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:36.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:36 smithi174 ceph-mon[32705]: pgmap v52: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.235+0000 7f46f2d6b700 1 -- 172.21.15.84:0/3499547064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46ec101a10 msgr2=0x7f46ec101e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.235+0000 7f46f2d6b700 1 --2- 172.21.15.84:0/3499547064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46ec101a10 0x7f46ec101e70 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009990 tx=0x7f46d8009ca0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.235+0000 7f46f2d6b700 1 -- 172.21.15.84:0/3499547064 shutdown_connections 2024-04-26T18:29:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.235+0000 7f46f2d6b700 1 --2- 172.21.15.84:0/3499547064 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46ec1023b0 0x7f46ec1102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.235+0000 7f46f2d6b700 1 --2- 172.21.15.84:0/3499547064 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46ec1094b0 0x7f46ec109890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.235+0000 7f46f2d6b700 1 --2- 172.21.15.84:0/3499547064 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46ec101a10 0x7f46ec101e70 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.235+0000 7f46f2d6b700 1 -- 172.21.15.84:0/3499547064 >> 172.21.15.84:0/3499547064 conn(0x7f46ec078990 msgr2=0x7f46ec0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.235+0000 7f46f2d6b700 1 -- 172.21.15.84:0/3499547064 shutdown_connections 2024-04-26T18:29:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.235+0000 7f46f2d6b700 1 -- 172.21.15.84:0/3499547064 wait complete. 2024-04-26T18:29:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.236+0000 7f46f2d6b700 1 Processor -- start 2024-04-26T18:29:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.237+0000 7f46f2d6b700 1 -- start start 2024-04-26T18:29:38.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.237+0000 7f46f2d6b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46ec101a10 0x7f46ec19f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.237+0000 7f46f2d6b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46ec1023b0 0x7f46ec19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.237+0000 7f46f2d6b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46ec1094b0 0x7f46ec199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.237+0000 7f46f0b07700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46ec101a10 0x7f46ec19f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.237+0000 7f46f0b07700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46ec101a10 0x7f46ec19f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51224/0 (socket says 172.21.15.84:51224) 2024-04-26T18:29:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.237+0000 7f46f0b07700 1 -- 172.21.15.84:0/1210293016 learned_addr learned my addr 172.21.15.84:0/1210293016 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.237+0000 7f46f2d6b700 1 -- 172.21.15.84:0/1210293016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec1135c0 con 0x7f46ec101a10 2024-04-26T18:29:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.237+0000 7f46f2d6b700 1 -- 172.21.15.84:0/1210293016 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec113440 con 0x7f46ec1023b0 2024-04-26T18:29:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.237+0000 7f46f2d6b700 1 -- 172.21.15.84:0/1210293016 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46ec113740 con 0x7f46ec1094b0 2024-04-26T18:29:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.237+0000 7f46ebfff700 1 --2- 172.21.15.84:0/1210293016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46ec1023b0 0x7f46ec19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.238+0000 7f46f0b07700 1 -- 172.21.15.84:0/1210293016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46ec1094b0 msgr2=0x7f46ec199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.238+0000 7f46f0b07700 1 --2- 172.21.15.84:0/1210293016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46ec1094b0 0x7f46ec199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.238+0000 7f46f0b07700 1 -- 172.21.15.84:0/1210293016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46ec1023b0 msgr2=0x7f46ec19fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.238+0000 7f46f0b07700 1 --2- 172.21.15.84:0/1210293016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46ec1023b0 0x7f46ec19fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.238+0000 7f46f0b07700 1 -- 172.21.15.84:0/1210293016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46ec19a0a0 con 0x7f46ec101a10 2024-04-26T18:29:38.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.238+0000 7f46f0b07700 1 --2- 172.21.15.84:0/1210293016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46ec101a10 0x7f46ec19f780 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f46e000c970 tx=0x7f46e000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:38.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.238+0000 7f46e9ffb700 1 -- 172.21.15.84:0/1210293016 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0007aa0 con 0x7f46ec101a10 2024-04-26T18:29:38.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.238+0000 7f46e9ffb700 1 -- 172.21.15.84:0/1210293016 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46e0004ce0 con 0x7f46ec101a10 2024-04-26T18:29:38.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.238+0000 7f46e9ffb700 1 -- 172.21.15.84:0/1210293016 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46e0005510 con 0x7f46ec101a10 2024-04-26T18:29:38.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.238+0000 7f46f2d6b700 1 -- 172.21.15.84:0/1210293016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46ec19a390 con 0x7f46ec101a10 2024-04-26T18:29:38.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.238+0000 7f46f2d6b700 1 -- 172.21.15.84:0/1210293016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46ec1a6650 con 0x7f46ec101a10 2024-04-26T18:29:38.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.240+0000 7f46f2d6b700 1 -- 172.21.15.84:0/1210293016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46ec104d30 con 0x7f46ec101a10 2024-04-26T18:29:38.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.241+0000 7f46e9ffb700 1 -- 172.21.15.84:0/1210293016 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f46e0007c40 con 0x7f46ec101a10 2024-04-26T18:29:38.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.242+0000 7f46e9ffb700 1 --2- 172.21.15.84:0/1210293016 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46dc0a3eb0 0x7f46dc0a6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:38.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.242+0000 7f46e9ffb700 1 -- 172.21.15.84:0/1210293016 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f46e00d07a0 con 0x7f46ec101a10 2024-04-26T18:29:38.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.242+0000 7f46ebfff700 1 --2- 172.21.15.84:0/1210293016 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46dc0a3eb0 0x7f46dc0a6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:38.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.242+0000 7f46ebfff700 1 --2- 172.21.15.84:0/1210293016 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46dc0a3eb0 0x7f46dc0a6370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009750 tx=0x7f46d800b560 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:38.250 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.250+0000 7f46e9ffb700 1 -- 172.21.15.84:0/1210293016 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46e009cef0 con 0x7f46ec101a10 2024-04-26T18:29:38.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.511+0000 7f46f2d6b700 1 -- 172.21.15.84:0/1210293016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46ec02e490 con 0x7f46ec101a10 2024-04-26T18:29:38.512 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:29:38.512 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:29:38.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.512+0000 7f46e9ffb700 1 -- 172.21.15.84:0/1210293016 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f46e009c7f0 con 0x7f46ec101a10 2024-04-26T18:29:38.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.515+0000 7f46f2d6b700 1 -- 172.21.15.84:0/1210293016 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46dc0a3eb0 msgr2=0x7f46dc0a6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.515+0000 7f46f2d6b700 1 --2- 172.21.15.84:0/1210293016 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46dc0a3eb0 0x7f46dc0a6370 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f46d8009750 tx=0x7f46d800b560 comp rx=0 tx=0).stop 2024-04-26T18:29:38.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.515+0000 7f46f2d6b700 1 -- 172.21.15.84:0/1210293016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46ec101a10 msgr2=0x7f46ec19f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:38.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.515+0000 7f46f2d6b700 1 --2- 172.21.15.84:0/1210293016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46ec101a10 0x7f46ec19f780 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f46e000c970 tx=0x7f46e000cc80 comp rx=0 tx=0).stop 2024-04-26T18:29:38.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.515+0000 7f46f2d6b700 1 -- 172.21.15.84:0/1210293016 shutdown_connections 2024-04-26T18:29:38.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.515+0000 7f46f2d6b700 1 --2- 172.21.15.84:0/1210293016 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46dc0a3eb0 0x7f46dc0a6370 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.515+0000 7f46f2d6b700 1 --2- 172.21.15.84:0/1210293016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46ec101a10 0x7f46ec19f780 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.515+0000 7f46f2d6b700 1 --2- 172.21.15.84:0/1210293016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46ec1094b0 0x7f46ec199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.515+0000 7f46f2d6b700 1 --2- 172.21.15.84:0/1210293016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46ec1023b0 0x7f46ec19fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:38.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.515+0000 7f46f2d6b700 1 -- 172.21.15.84:0/1210293016 >> 172.21.15.84:0/1210293016 conn(0x7f46ec078990 msgr2=0x7f46ec1063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:38.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.515+0000 7f46f2d6b700 1 -- 172.21.15.84:0/1210293016 shutdown_connections 2024-04-26T18:29:38.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:38.515+0000 7f46f2d6b700 1 -- 172.21.15.84:0/1210293016 wait complete. 2024-04-26T18:29:38.517 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:29:38.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:38 smithi084 ceph-mon[29141]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:38.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:38 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1210293016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:29:38.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:38 smithi103 ceph-mon[32727]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:38.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:38 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1210293016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:29:38.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:38 smithi174 ceph-mon[32705]: pgmap v53: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.3 KiB/s wr, 10 op/s 2024-04-26T18:29:38.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:38 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1210293016' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:29:39.345 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-04-26T18:29:39.345 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-04-26T18:29:39.621 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:29:40.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:40 smithi084 ceph-mon[29141]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-26T18:29:40.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:40 smithi103 ceph-mon[32727]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-26T18:29:40.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:40 smithi174 ceph-mon[32705]: pgmap v54: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.3 KiB/s wr, 14 op/s 2024-04-26T18:29:41.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.473+0000 7f2c93af3700 1 -- 172.21.15.84:0/797768565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c8c103470 msgr2=0x7f2c8c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:41.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.473+0000 7f2c93af3700 1 --2- 172.21.15.84:0/797768565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c8c103470 0x7f2c8c1038f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f2c80009990 tx=0x7f2c80009ca0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.473+0000 7f2c93af3700 1 -- 172.21.15.84:0/797768565 shutdown_connections 2024-04-26T18:29:41.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.473+0000 7f2c93af3700 1 --2- 172.21.15.84:0/797768565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c8c103e30 0x7f2c8c1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.473+0000 7f2c93af3700 1 --2- 172.21.15.84:0/797768565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c8c1083c0 0x7f2c8c1087a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.473+0000 7f2c93af3700 1 --2- 172.21.15.84:0/797768565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c8c103470 0x7f2c8c1038f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.473+0000 7f2c93af3700 1 -- 172.21.15.84:0/797768565 >> 172.21.15.84:0/797768565 conn(0x7f2c8c07a5d0 msgr2=0x7f2c8c07a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:41.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.473+0000 7f2c93af3700 1 -- 172.21.15.84:0/797768565 shutdown_connections 2024-04-26T18:29:41.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.474+0000 7f2c93af3700 1 -- 172.21.15.84:0/797768565 wait complete. 2024-04-26T18:29:41.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.474+0000 7f2c93af3700 1 Processor -- start 2024-04-26T18:29:41.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.474+0000 7f2c93af3700 1 -- start start 2024-04-26T18:29:41.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.474+0000 7f2c93af3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c8c103470 0x7f2c8c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.474+0000 7f2c93af3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c8c103e30 0x7f2c8c19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.474+0000 7f2c93af3700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c8c1083c0 0x7f2c8c1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.474+0000 7f2c93af3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c113670 con 0x7f2c8c103e30 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.474+0000 7f2c93af3700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c1134f0 con 0x7f2c8c1083c0 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.474+0000 7f2c93af3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c1137f0 con 0x7f2c8c103470 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.474+0000 7f2c9108e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c8c103e30 0x7f2c8c19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c9108e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c8c103e30 0x7f2c8c19fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51246/0 (socket says 172.21.15.84:51246) 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c9108e700 1 -- 172.21.15.84:0/1703913196 learned_addr learned my addr 172.21.15.84:0/1703913196 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c9188f700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c8c103470 0x7f2c8c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c92090700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c8c1083c0 0x7f2c8c1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c9108e700 1 -- 172.21.15.84:0/1703913196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c8c103470 msgr2=0x7f2c8c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c9108e700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c8c103470 0x7f2c8c19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c9108e700 1 -- 172.21.15.84:0/1703913196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c8c1083c0 msgr2=0x7f2c8c1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c9108e700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c8c1083c0 0x7f2c8c1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c9108e700 1 -- 172.21.15.84:0/1703913196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c8c19a010 con 0x7f2c8c103e30 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c92090700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c8c1083c0 0x7f2c8c1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:41.477 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c9108e700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c8c103e30 0x7f2c8c19fc30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2c80000c00 tx=0x7f2c800047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:41.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c8affd700 1 -- 172.21.15.84:0/1703913196 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80027070 con 0x7f2c8c103e30 2024-04-26T18:29:41.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c8affd700 1 -- 172.21.15.84:0/1703913196 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c80003f20 con 0x7f2c8c103e30 2024-04-26T18:29:41.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c93af3700 1 -- 172.21.15.84:0/1703913196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c8c19a2a0 con 0x7f2c8c103e30 2024-04-26T18:29:41.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c8affd700 1 -- 172.21.15.84:0/1703913196 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80022cf0 con 0x7f2c8c103e30 2024-04-26T18:29:41.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.475+0000 7f2c93af3700 1 -- 172.21.15.84:0/1703913196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c8c1a65c0 con 0x7f2c8c103e30 2024-04-26T18:29:41.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.477+0000 7f2c8affd700 1 -- 172.21.15.84:0/1703913196 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f2c8002a070 con 0x7f2c8c103e30 2024-04-26T18:29:41.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.478+0000 7f2c8affd700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f2c740a3e50 0x7f2c740a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.478+0000 7f2c9188f700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f2c740a3e50 0x7f2c740a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.479+0000 7f2c8affd700 1 -- 172.21.15.84:0/1703913196 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f2c800d15c0 con 0x7f2c8c103e30 2024-04-26T18:29:41.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.479+0000 7f2c9188f700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f2c740a3e50 0x7f2c740a6310 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009550 tx=0x7f2c840091b0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:41.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.479+0000 7f2c9e385b80 1 Processor -- start 2024-04-26T18:29:41.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.479+0000 7f2c9e385b80 1 -- start start 2024-04-26T18:29:41.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.479+0000 7f2c9e385b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 0x55ddf773e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.479+0000 7f2c9e385b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf77386f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9e385b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 0x55ddf7739170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9e385b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ddf76869c0 con 0x55ddf773ecb0 2024-04-26T18:29:41.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9e385b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ddf75ecac0 con 0x55ddf773e390 2024-04-26T18:29:41.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9e385b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ddf7682980 con 0x55ddf7738d10 2024-04-26T18:29:41.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9108e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf77386f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9108e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf77386f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51258/0 (socket says 172.21.15.84:51258) 2024-04-26T18:29:41.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9108e700 1 -- 172.21.15.84:0/2196733734 learned_addr learned my addr 172.21.15.84:0/2196733734 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:41.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c92090700 1 --2- 172.21.15.84:0/2196733734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 0x55ddf7739170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9108e700 1 -- 172.21.15.84:0/2196733734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 msgr2=0x55ddf7739170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:41.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9188f700 1 --2- 172.21.15.84:0/2196733734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 0x55ddf773e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9108e700 1 --2- 172.21.15.84:0/2196733734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 0x55ddf7739170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9108e700 1 -- 172.21.15.84:0/2196733734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 msgr2=0x55ddf773e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:41.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9108e700 1 --2- 172.21.15.84:0/2196733734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 0x55ddf773e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9108e700 1 -- 172.21.15.84:0/2196733734 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf75d3a60 con 0x55ddf773ecb0 2024-04-26T18:29:41.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c92090700 1 --2- 172.21.15.84:0/2196733734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 0x55ddf7739170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:41.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c9108e700 1 --2- 172.21.15.84:0/2196733734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf77386f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2c80099a40 tx=0x7f2c8009dd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db569e5aba21115 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:41.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c73fff700 1 -- 172.21.15.84:0/2196733734 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c800a61c0 con 0x55ddf773ecb0 2024-04-26T18:29:41.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c73fff700 1 -- 172.21.15.84:0/2196733734 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c80022470 con 0x55ddf773ecb0 2024-04-26T18:29:41.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.480+0000 7f2c73fff700 1 -- 172.21.15.84:0/2196733734 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c80003a70 con 0x55ddf773ecb0 2024-04-26T18:29:41.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.481+0000 7f2c9e385b80 1 -- 172.21.15.84:0/2196733734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 msgr2=0x55ddf77386f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:41.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.481+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/2196733734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf77386f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f2c80099a40 tx=0x7f2c8009dd60 comp rx=0 tx=0).stop 2024-04-26T18:29:41.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.481+0000 7f2c9e385b80 1 -- 172.21.15.84:0/2196733734 shutdown_connections 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.481+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/2196733734 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 0x55ddf7739170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.481+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/2196733734 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 0x55ddf773e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.481+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/2196733734 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf77386f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.481+0000 7f2c9e385b80 1 -- 172.21.15.84:0/2196733734 >> 172.21.15.84:0/2196733734 conn(0x55ddf7662750 msgr2=0x55ddf772f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.481+0000 7f2c9e385b80 1 -- 172.21.15.84:0/2196733734 shutdown_connections 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.481+0000 7f2c9e385b80 1 -- 172.21.15.84:0/2196733734 wait complete. 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.482+0000 7f2c9e385b80 1 -- start start 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.482+0000 7f2c9e385b80 1 Processor -- start 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.482+0000 7f2c9e385b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 0x55ddf7744a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.482+0000 7f2c9e385b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 0x55ddf7741e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.482+0000 7f2c9e385b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf7742350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.482+0000 7f2c9e385b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ddf76869c0 con 0x55ddf773ecb0 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.482+0000 7f2c9e385b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ddf7640c10 con 0x55ddf773e390 2024-04-26T18:29:41.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.482+0000 7f2c9e385b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ddf7745500 con 0x55ddf7738d10 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.482+0000 7f2c92090700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf7742350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.482+0000 7f2c92090700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf7742350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51274/0 (socket says 172.21.15.84:51274) 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.482+0000 7f2c92090700 1 -- 172.21.15.84:0/3364100067 learned_addr learned my addr 172.21.15.84:0/3364100067 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.482+0000 7f2c9e385b80 10 client.0 _ll_register_callbacks cb 0x55ddf761eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.482+0000 7f2c9108e700 1 --2- 172.21.15.84:0/3364100067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 0x55ddf7741e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c9188f700 1 --2- 172.21.15.84:0/3364100067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 0x55ddf7744a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c92090700 1 -- 172.21.15.84:0/3364100067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 msgr2=0x55ddf7744a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c92090700 1 --2- 172.21.15.84:0/3364100067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 0x55ddf7744a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c92090700 1 -- 172.21.15.84:0/3364100067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 msgr2=0x55ddf7741e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c92090700 1 --2- 172.21.15.84:0/3364100067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 0x55ddf7741e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c92090700 1 -- 172.21.15.84:0/3364100067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf7644610 con 0x55ddf773ecb0 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c92090700 1 --2- 172.21.15.84:0/3364100067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf7742350 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c007f60 tx=0x7f2c7c007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec536fe282630385 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c717fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:29:41.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c717fa700 1 -- 172.21.15.84:0/3364100067 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c018020 con 0x55ddf773ecb0 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c717fa700 1 -- 172.21.15.84:0/3364100067 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f2c7c014460 con 0x55ddf773ecb0 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c9e385b80 1 -- 172.21.15.84:0/3364100067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c80022470 con 0x55ddf773ecb0 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c717fa700 1 -- 172.21.15.84:0/3364100067 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c7c023050 con 0x55ddf773ecb0 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c9e385b80 10 client.15669 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.483+0000 7f2c9e385b80 1 -- 172.21.15.84:0/3364100067 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ddf774fc70 con 0x55ddf773ecb0 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.484+0000 7f2c717fa700 1 -- 172.21.15.84:0/3364100067 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c80022470 con 0x55ddf773ecb0 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.484+0000 7f2c717fa700 1 -- 172.21.15.84:0/3364100067 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2c7c013690 con 0x55ddf773ecb0 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.484+0000 7f2c9e385b80 10 client.15669 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.484+0000 7f2c9e385b80 10 client.15669 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.484+0000 7f2c9e385b80 20 client.15669 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.484+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/3364100067 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ddf7752300 0x55ddf77546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.484+0000 7f2c9e385b80 4 client.15669 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.484+0000 7f2c9e385b80 1 -- 172.21.15.84:0/3364100067 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddf76943a0 con 0x55ddf7752300 2024-04-26T18:29:41.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.484+0000 7f2c9188f700 1 --2- 172.21.15.84:0/3364100067 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ddf7752300 0x55ddf77546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.485+0000 7f2c717fa700 10 client.15669 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:41.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.485+0000 7f2c9188f700 1 --2- 172.21.15.84:0/3364100067 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ddf7752300 0x55ddf77546f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=def5f6a6d0543677 server_cookie=4b11eb47de952e6a in_seq=0 out_seq=0 2024-04-26T18:29:41.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.485+0000 7f2c717fa700 1 -- 172.21.15.84:0/3364100067 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ddf76943a0 con 0x55ddf7752300 2024-04-26T18:29:41.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.485+0000 7f2c717fa700 10 client.15669 handle_command_reply: tid=0 2024-04-26T18:29:41.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.485+0000 7f2c9e385b80 1 client.15669 shutdown 2024-04-26T18:29:41.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.485+0000 7f2c9e385b80 1 -- 172.21.15.84:0/3364100067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 msgr2=0x55ddf7742350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:41.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.485+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/3364100067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf7742350 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c007f60 tx=0x7f2c7c007f90 comp rx=0 tx=0).stop 2024-04-26T18:29:41.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.486+0000 7f2c9e385b80 1 -- 172.21.15.84:0/3364100067 shutdown_connections 2024-04-26T18:29:41.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.486+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/3364100067 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ddf7752300 0x55ddf77546f0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.486+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/3364100067 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf7742350 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.486+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/3364100067 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 0x55ddf7744a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.490 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.486+0000 7f2c717fa700 0 client.15669 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:41.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.486+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/3364100067 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 0x55ddf7741e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.486+0000 7f2c9e385b80 1 -- 172.21.15.84:0/3364100067 >> 172.21.15.84:0/3364100067 conn(0x55ddf7662750 msgr2=0x55ddf77332d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:41.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.486+0000 7f2c9e385b80 1 -- 172.21.15.84:0/3364100067 shutdown_connections 2024-04-26T18:29:41.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.486+0000 7f2c9e385b80 1 -- 172.21.15.84:0/3364100067 wait complete. 2024-04-26T18:29:41.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.486+0000 7f2c9e385b80 20 client.15669 trim_cache size 0 max 16384 2024-04-26T18:29:41.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.516+0000 7f2c9e385b80 1 Processor -- start 2024-04-26T18:29:41.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.516+0000 7f2c9e385b80 1 -- start start 2024-04-26T18:29:41.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.516+0000 7f2c9e385b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 0x55ddf773d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.516+0000 7f2c9e385b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 0x55ddf7742380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.516+0000 7f2c9e385b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf774e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.516+0000 7f2c9e385b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c1aba30 con 0x55ddf773ecb0 2024-04-26T18:29:41.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.516+0000 7f2c9e385b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c1ad570 con 0x55ddf773e390 2024-04-26T18:29:41.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.516+0000 7f2c9e385b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c1ac100 con 0x55ddf7738d10 2024-04-26T18:29:41.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.516+0000 7f2c92090700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf774e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.516+0000 7f2c92090700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf774e240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51288/0 (socket says 172.21.15.84:51288) 2024-04-26T18:29:41.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.516+0000 7f2c92090700 1 -- 172.21.15.84:0/3099663635 learned_addr learned my addr 172.21.15.84:0/3099663635 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:41.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.517+0000 7f2c9188f700 1 --2- 172.21.15.84:0/3099663635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 0x55ddf773d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.517+0000 7f2c9108e700 1 --2- 172.21.15.84:0/3099663635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 0x55ddf7742380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.517+0000 7f2c92090700 1 -- 172.21.15.84:0/3099663635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 msgr2=0x55ddf773d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:41.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.517+0000 7f2c92090700 1 --2- 172.21.15.84:0/3099663635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 0x55ddf773d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.517+0000 7f2c92090700 1 -- 172.21.15.84:0/3099663635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 msgr2=0x55ddf7742380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:41.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.517+0000 7f2c92090700 1 --2- 172.21.15.84:0/3099663635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 0x55ddf7742380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.517+0000 7f2c92090700 1 -- 172.21.15.84:0/3099663635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddf77428c0 con 0x55ddf773ecb0 2024-04-26T18:29:41.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.517+0000 7f2c92090700 1 --2- 172.21.15.84:0/3099663635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf774e240 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0048a0 tx=0x7f2c7c00f7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1a9e58cab96f9b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:41.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.517+0000 7f2c737fe700 1 -- 172.21.15.84:0/3099663635 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c02b070 con 0x55ddf773ecb0 2024-04-26T18:29:41.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.517+0000 7f2c737fe700 1 -- 172.21.15.84:0/3099663635 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7c013870 con 0x55ddf773ecb0 2024-04-26T18:29:41.519 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.517+0000 7f2c737fe700 1 -- 172.21.15.84:0/3099663635 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c018050 con 0x55ddf773ecb0 2024-04-26T18:29:41.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.518+0000 7f2c9e385b80 1 -- 172.21.15.84:0/3099663635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 msgr2=0x55ddf774e240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.518+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/3099663635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf774e240 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c0048a0 tx=0x7f2c7c00f7a0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.518+0000 7f2c9e385b80 1 -- 172.21.15.84:0/3099663635 shutdown_connections 2024-04-26T18:29:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.518+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/3099663635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf774e240 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.518+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/3099663635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf7738d10 0x55ddf773d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.518+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/3099663635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf773e390 0x55ddf7742380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.518+0000 7f2c9e385b80 1 -- 172.21.15.84:0/3099663635 >> 172.21.15.84:0/3099663635 conn(0x55ddf7662750 msgr2=0x55ddf7728700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.518+0000 7f2c9e385b80 1 -- 172.21.15.84:0/3099663635 shutdown_connections 2024-04-26T18:29:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.518+0000 7f2c9e385b80 1 -- 172.21.15.84:0/3099663635 wait complete. 2024-04-26T18:29:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c9e385b80 1 -- start start 2024-04-26T18:29:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c9e385b80 1 Processor -- start 2024-04-26T18:29:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c9e385b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf7738d10 0x55ddf7747720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c9e385b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf773e390 0x55ddf7747c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c9e385b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf77481a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c9e385b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c1acea0 con 0x55ddf773ecb0 2024-04-26T18:29:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c92090700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf77481a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c9188f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf7738d10 0x55ddf7747720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c9e385b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c1ac7d0 con 0x55ddf7738d10 2024-04-26T18:29:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c9e385b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2c8c1a99d0 con 0x55ddf773e390 2024-04-26T18:29:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c92090700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf77481a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51290/0 (socket says 172.21.15.84:51290) 2024-04-26T18:29:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c9188f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf7738d10 0x55ddf7747720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:41580/0 (socket says 172.21.15.84:41580) 2024-04-26T18:29:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c92090700 1 -- 172.21.15.84:0/2140057177 learned_addr learned my addr 172.21.15.84:0/2140057177 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.519+0000 7f2c9e385b80 10 client.0 _ll_register_callbacks cb 0x55ddf761eae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c9108e700 1 --2- 172.21.15.84:0/2140057177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf773e390 0x55ddf7747c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c9188f700 1 -- 172.21.15.84:0/2140057177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf773e390 msgr2=0x55ddf7747c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c9188f700 1 --2- 172.21.15.84:0/2140057177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf773e390 0x55ddf7747c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c9188f700 1 -- 172.21.15.84:0/2140057177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 msgr2=0x55ddf77481a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c9188f700 1 --2- 172.21.15.84:0/2140057177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf77481a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c9188f700 1 -- 172.21.15.84:0/2140057177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddf77b87f0 con 0x55ddf7738d10 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c92090700 1 --2- 172.21.15.84:0/2140057177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf77481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c9188f700 1 --2- 172.21.15.84:0/2140057177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf7738d10 0x55ddf7747720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55ddf7754eb0 tx=0x7f2c8400b050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed10ecedb025ca75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c717fa700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c717fa700 1 -- 172.21.15.84:0/2140057177 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c8401c040 con 0x55ddf7738d10 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c9e385b80 1 -- 172.21.15.84:0/2140057177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c7c013870 con 0x55ddf7738d10 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c717fa700 1 -- 172.21.15.84:0/2140057177 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f2c84004db0 con 0x55ddf7738d10 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c717fa700 1 -- 172.21.15.84:0/2140057177 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2c8402d480 con 0x55ddf7738d10 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c9e385b80 10 client.25124 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.520+0000 7f2c9e385b80 1 -- 172.21.15.84:0/2140057177 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ddf77b9a20 con 0x55ddf7738d10 2024-04-26T18:29:41.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.521+0000 7f2c717fa700 1 -- 172.21.15.84:0/2140057177 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c7c013870 con 0x55ddf7738d10 2024-04-26T18:29:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.521+0000 7f2c717fa700 1 -- 172.21.15.84:0/2140057177 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2c840154d0 con 0x55ddf7738d10 2024-04-26T18:29:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.521+0000 7f2c9e385b80 10 client.25124 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.521+0000 7f2c9e385b80 10 client.25124 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:29:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.521+0000 7f2c9e385b80 20 client.25124 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.521+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/2140057177 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ddf77ba140 0x55ddf77bc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.521+0000 7f2c9e385b80 4 client.25124 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-04-26T18:29:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.521+0000 7f2c9e385b80 1 -- 172.21.15.84:0/2140057177 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x55ddf768e440 con 0x55ddf77ba140 2024-04-26T18:29:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.521+0000 7f2c9108e700 1 --2- 172.21.15.84:0/2140057177 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ddf77ba140 0x55ddf77bc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.521+0000 7f2c9108e700 1 --2- 172.21.15.84:0/2140057177 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ddf77ba140 0x55ddf77bc530 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf1c40c6b3535fbf server_cookie=56e5532cf7db71bd in_seq=0 out_seq=0 2024-04-26T18:29:41.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.521+0000 7f2c717fa700 10 client.25124 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c717fa700 1 -- 172.21.15.84:0/2140057177 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55ddf768e440 con 0x55ddf77ba140 2024-04-26T18:29:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c717fa700 10 client.25124 handle_command_reply: tid=0 2024-04-26T18:29:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c9e385b80 1 client.25124 shutdown 2024-04-26T18:29:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c9e385b80 1 -- 172.21.15.84:0/2140057177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf7738d10 msgr2=0x55ddf7747720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/2140057177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf7738d10 0x55ddf7747720 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55ddf7754eb0 tx=0x7f2c8400b050 comp rx=0 tx=0).stop 2024-04-26T18:29:41.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c9e385b80 1 -- 172.21.15.84:0/2140057177 shutdown_connections 2024-04-26T18:29:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/2140057177 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ddf77ba140 0x55ddf77bc530 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/2140057177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ddf773ecb0 0x55ddf77481a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c717fa700 0 client.25124 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/2140057177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ddf7738d10 0x55ddf7747720 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c9e385b80 1 --2- 172.21.15.84:0/2140057177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ddf773e390 0x55ddf7747c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c9e385b80 1 -- 172.21.15.84:0/2140057177 >> 172.21.15.84:0/2140057177 conn(0x55ddf7662750 msgr2=0x55ddf773d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c9e385b80 1 -- 172.21.15.84:0/2140057177 shutdown_connections 2024-04-26T18:29:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c9e385b80 1 -- 172.21.15.84:0/2140057177 wait complete. 2024-04-26T18:29:41.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.532+0000 7f2c9e385b80 20 client.25124 trim_cache size 0 max 16384 2024-04-26T18:29:41.533 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:29:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:29:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:29:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 6, 2024-04-26T18:29:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:29:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "4ccfe41a": { 2024-04-26T18:29:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 6, 2024-04-26T18:29:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:29:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:29:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:29:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:29:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:29:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:29:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 90.0, 2024-04-26T18:29:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:29:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:29:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:29:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:29:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:29:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:29:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:29:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:29:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:29:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:29:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:29:41.536 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:29:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.533+0000 7f2c93af3700 1 -- 172.21.15.84:0/1703913196 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f2c740a3e50 msgr2=0x7f2c740a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.533+0000 7f2c93af3700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f2c740a3e50 0x7f2c740a6310 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2c84009550 tx=0x7f2c840091b0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.533+0000 7f2c93af3700 1 -- 172.21.15.84:0/1703913196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c8c103e30 msgr2=0x7f2c8c19fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.533+0000 7f2c93af3700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c8c103e30 0x7f2c8c19fc30 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f2c80000c00 tx=0x7f2c800047b0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.533+0000 7f2c93af3700 1 -- 172.21.15.84:0/1703913196 shutdown_connections 2024-04-26T18:29:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.533+0000 7f2c93af3700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f2c740a3e50 0x7f2c740a6310 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.533+0000 7f2c93af3700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2c8c1083c0 0x7f2c8c1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.533+0000 7f2c93af3700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2c8c103470 0x7f2c8c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.533+0000 7f2c93af3700 1 --2- 172.21.15.84:0/1703913196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2c8c103e30 0x7f2c8c19fc30 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.533+0000 7f2c93af3700 1 -- 172.21.15.84:0/1703913196 >> 172.21.15.84:0/1703913196 conn(0x7f2c8c07a5d0 msgr2=0x7f2c8c101480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.533+0000 7f2c93af3700 1 -- 172.21.15.84:0/1703913196 shutdown_connections 2024-04-26T18:29:41.539 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:29:41.533+0000 7f2c93af3700 1 -- 172.21.15.84:0/1703913196 wait complete. 2024-04-26T18:29:42.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:42 smithi084 ceph-mon[29141]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:29:42.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:42 smithi103 ceph-mon[32727]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:29:42.887 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '4ccfe41a', quiesce took 6.7 seconds. Will release after: 30.0 2024-04-26T18:29:42.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:42 smithi174 ceph-mon[32705]: pgmap v55: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-26T18:29:44.527 INFO:tasks.workunit.client.0.smithi084.stderr:Note: switching to 'bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1'. 2024-04-26T18:29:44.528 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-04-26T18:29:44.528 INFO:tasks.workunit.client.0.smithi084.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T18:29:44.528 INFO:tasks.workunit.client.0.smithi084.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T18:29:44.528 INFO:tasks.workunit.client.0.smithi084.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T18:29:44.528 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-04-26T18:29:44.528 INFO:tasks.workunit.client.0.smithi084.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T18:29:44.528 INFO:tasks.workunit.client.0.smithi084.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T18:29:44.528 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-04-26T18:29:44.528 INFO:tasks.workunit.client.0.smithi084.stderr: git switch -c 2024-04-26T18:29:44.528 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-04-26T18:29:44.528 INFO:tasks.workunit.client.0.smithi084.stderr:Or undo this operation with: 2024-04-26T18:29:44.528 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-04-26T18:29:44.529 INFO:tasks.workunit.client.0.smithi084.stderr: git switch - 2024-04-26T18:29:44.529 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-04-26T18:29:44.529 INFO:tasks.workunit.client.0.smithi084.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T18:29:44.529 INFO:tasks.workunit.client.0.smithi084.stderr: 2024-04-26T18:29:44.529 INFO:tasks.workunit.client.0.smithi084.stderr:HEAD is now at bc94a2a924f Merge commit 'refs/pull/57099/head' of github.com:ceph/ceph into quiescer 2024-04-26T18:29:44.534 DEBUG:teuthology.orchestra.run.smithi084:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T18:29:44.554 INFO:tasks.workunit.client.0.smithi084.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T18:29:44.556 INFO:tasks.workunit.client.0.smithi084.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T18:29:44.556 INFO:tasks.workunit.client.0.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T18:29:44.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:44 smithi084 ceph-mon[29141]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T18:29:44.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:44 smithi103 ceph-mon[32727]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T18:29:44.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:44 smithi174 ceph-mon[32705]: pgmap v56: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-26T18:29:45.194 INFO:tasks.workunit.client.0.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T18:29:45.251 INFO:tasks.workunit.client.0.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T18:29:45.291 INFO:tasks.workunit.client.0.smithi084.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T18:29:45.293 INFO:tasks.workunit.client.0.smithi084.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T18:29:45.293 INFO:tasks.workunit.client.0.smithi084.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T18:29:45.332 INFO:tasks.workunit.client.0.smithi084.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T18:29:45.335 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:29:45.335 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T18:29:45.391 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-26T18:29:45.392 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-26T18:29:45.392 DEBUG:teuthology.orchestra.run.smithi084:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-26T18:29:45.420 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.418+0000 7fabfbb89640 1 -- 172.21.15.84:0/429669730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf4103bc0 msgr2=0x7fabf4103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.418+0000 7fabfbb89640 1 --2- 172.21.15.84:0/429669730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf4103bc0 0x7fabf4103fa0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fabf0009ca0 tx=0x7fabf002a9e0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.418+0000 7fabfbb89640 1 -- 172.21.15.84:0/429669730 shutdown_connections 2024-04-26T18:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.418+0000 7fabfbb89640 1 --2- 172.21.15.84:0/429669730 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabf410cf00 0x7fabf410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.418+0000 7fabfbb89640 1 --2- 172.21.15.84:0/429669730 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf41044e0 0x7fabf410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.418+0000 7fabfbb89640 1 --2- 172.21.15.84:0/429669730 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf4103bc0 0x7fabf4103fa0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.418+0000 7fabfbb89640 1 -- 172.21.15.84:0/429669730 >> 172.21.15.84:0/429669730 conn(0x7fabf40789b0 msgr2=0x7fabf4078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.418+0000 7fabfbb89640 1 -- 172.21.15.84:0/429669730 shutdown_connections 2024-04-26T18:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.418+0000 7fabfbb89640 1 -- 172.21.15.84:0/429669730 wait complete. 2024-04-26T18:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.418+0000 7fabfbb89640 1 Processor -- start 2024-04-26T18:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.418+0000 7fabfbb89640 1 -- start start 2024-04-26T18:29:45.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabfbb89640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabf4103bc0 0x7fabf4199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabfbb89640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf41044e0 0x7fabf4199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabfbb89640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf410cf00 0x7fabf419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabfbb89640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fabf4112bb0 con 0x7fabf41044e0 2024-04-26T18:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabfbb89640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabf4112a30 con 0x7fabf4103bc0 2024-04-26T18:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabfbb89640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabf4112d30 con 0x7fabf410cf00 2024-04-26T18:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabf9124640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf41044e0 0x7fabf4199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabf9124640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf41044e0 0x7fabf4199670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51308/0 (socket says 172.21.15.84:51308) 2024-04-26T18:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabf9124640 1 -- 172.21.15.84:0/2663891196 learned_addr learned my addr 172.21.15.84:0/2663891196 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabf9925640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabf4103bc0 0x7fabf4199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabfa126640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf410cf00 0x7fabf419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabf9124640 1 -- 172.21.15.84:0/2663891196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf410cf00 msgr2=0x7fabf419d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabf9124640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf410cf00 0x7fabf419d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabf9124640 1 -- 172.21.15.84:0/2663891196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabf4103bc0 msgr2=0x7fabf4199130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabf9124640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabf4103bc0 0x7fabf4199130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabf9124640 1 -- 172.21.15.84:0/2663891196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabf419e0d0 con 0x7fabf41044e0 2024-04-26T18:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabfa126640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf410cf00 0x7fabf419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabf9124640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf41044e0 0x7fabf4199670 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fabe400d950 tx=0x7fabe400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabeaffd640 1 -- 172.21.15.84:0/2663891196 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4017070 con 0x7fabf41044e0 2024-04-26T18:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabfbb89640 1 -- 172.21.15.84:0/2663891196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabf419e3c0 con 0x7fabf41044e0 2024-04-26T18:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabeaffd640 1 -- 172.21.15.84:0/2663891196 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabe4009c30 con 0x7fabf41044e0 2024-04-26T18:29:45.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabeaffd640 1 -- 172.21.15.84:0/2663891196 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe400f620 con 0x7fabf41044e0 2024-04-26T18:29:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.419+0000 7fabfbb89640 1 -- 172.21.15.84:0/2663891196 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabf41a5c50 con 0x7fabf41044e0 2024-04-26T18:29:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.421+0000 7fabeaffd640 1 -- 172.21.15.84:0/2663891196 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fabe4010460 con 0x7fabf41044e0 2024-04-26T18:29:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.422+0000 7fabfe97c740 1 Processor -- start 2024-04-26T18:29:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.422+0000 7fabfe97c740 1 -- start start 2024-04-26T18:29:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.422+0000 7fabeaffd640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fabe00a3dc0 0x7fabe00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.422+0000 7fabfe97c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 0x560e5464b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.422+0000 7fabfe97c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464b820 0x560e54655ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.422+0000 7fabfe97c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e546584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.422+0000 7fabfe97c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560e545b1460 con 0x560e546560e0 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.422+0000 7fabfe97c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560e5456b590 con 0x560e5464af00 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.422+0000 7fabfe97c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e545ad420 con 0x560e5464b820 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabfa126640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fabe00a3dc0 0x7fabe00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabeaffd640 1 -- 172.21.15.84:0/2663891196 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fabe40d0470 con 0x7fabf41044e0 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabfa126640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e546584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabfa126640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e546584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51312/0 (socket says 172.21.15.84:51312) 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabfa126640 1 -- 172.21.15.84:0/2144095098 learned_addr learned my addr 172.21.15.84:0/2144095098 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabf9925640 1 --2- 172.21.15.84:0/2144095098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 0x560e5464b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabf9124640 1 --2- 172.21.15.84:0/2144095098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464b820 0x560e54655ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabf9925640 1 -- 172.21.15.84:0/2144095098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464b820 msgr2=0x560e54655ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabf9925640 1 --2- 172.21.15.84:0/2144095098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464b820 0x560e54655ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabf9925640 1 -- 172.21.15.84:0/2144095098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 msgr2=0x560e546584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabf9925640 1 --2- 172.21.15.84:0/2144095098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e546584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabf9925640 1 -- 172.21.15.84:0/2144095098 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e545780d0 con 0x560e5464af00 2024-04-26T18:29:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabfa126640 1 --2- 172.21.15.84:0/2144095098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e546584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:29:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabfa126640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fabe00a3dc0 0x7fabe00a6280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fabec013800 tx=0x7fabec00ba50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabf9925640 1 --2- 172.21.15.84:0/2144095098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 0x560e5464b2e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fabf0000f80 tx=0x7fabf0005fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c59d99a6c54bdf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.423+0000 7fabdffff640 1 -- 172.21.15.84:0/2144095098 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf0033040 con 0x560e5464af00 2024-04-26T18:29:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.424+0000 7fabdffff640 1 -- 172.21.15.84:0/2144095098 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabf0003e20 con 0x560e5464af00 2024-04-26T18:29:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.424+0000 7fabdffff640 1 -- 172.21.15.84:0/2144095098 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf002f8a0 con 0x560e5464af00 2024-04-26T18:29:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.424+0000 7fabfe97c740 1 -- 172.21.15.84:0/2144095098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 msgr2=0x560e5464b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.424+0000 7fabfe97c740 1 --2- 172.21.15.84:0/2144095098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 0x560e5464b2e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fabf0000f80 tx=0x7fabf0005fa0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.424+0000 7fabfe97c740 1 -- 172.21.15.84:0/2144095098 shutdown_connections 2024-04-26T18:29:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.424+0000 7fabfe97c740 1 --2- 172.21.15.84:0/2144095098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e546584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.424+0000 7fabfe97c740 1 --2- 172.21.15.84:0/2144095098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464b820 0x560e54655ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.424+0000 7fabfe97c740 1 --2- 172.21.15.84:0/2144095098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 0x560e5464b2e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.424+0000 7fabfe97c740 1 -- 172.21.15.84:0/2144095098 >> 172.21.15.84:0/2144095098 conn(0x560e54647410 msgr2=0x560e546477f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.424+0000 7fabfe97c740 1 -- 172.21.15.84:0/2144095098 shutdown_connections 2024-04-26T18:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.424+0000 7fabfe97c740 1 -- 172.21.15.84:0/2144095098 wait complete. 2024-04-26T18:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabfe97c740 1 -- start start 2024-04-26T18:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabfe97c740 1 Processor -- start 2024-04-26T18:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabfe97c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 0x560e54653090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabfe97c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464b820 0x560e546535d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabfe97c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e54653b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabfe97c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560e54658cb0 con 0x560e546560e0 2024-04-26T18:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabfe97c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560e54538b60 con 0x560e5464af00 2024-04-26T18:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabfe97c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e54658e30 con 0x560e5464b820 2024-04-26T18:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabfe97c740 10 client.0 _ll_register_callbacks cb 0x560e5455d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:45.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabfa126640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e54653b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabfa126640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e54653b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51314/0 (socket says 172.21.15.84:51314) 2024-04-26T18:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabfa126640 1 -- 172.21.15.84:0/1637685701 learned_addr learned my addr 172.21.15.84:0/1637685701 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabf9925640 1 --2- 172.21.15.84:0/1637685701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 0x560e54653090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.425+0000 7fabf9124640 1 --2- 172.21.15.84:0/1637685701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464b820 0x560e546535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabfa126640 1 -- 172.21.15.84:0/1637685701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464b820 msgr2=0x560e546535d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabfa126640 1 --2- 172.21.15.84:0/1637685701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464b820 0x560e546535d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabfa126640 1 -- 172.21.15.84:0/1637685701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 msgr2=0x560e54653090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabfa126640 1 --2- 172.21.15.84:0/1637685701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 0x560e54653090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabfa126640 1 -- 172.21.15.84:0/1637685701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e542c3210 con 0x560e546560e0 2024-04-26T18:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabfa126640 1 --2- 172.21.15.84:0/1637685701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e54653b10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fabec010c50 tx=0x7fabec005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a904277dfcbb0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:29:45.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabdd7fa640 1 -- 172.21.15.84:0/1637685701 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec010c80 con 0x560e546560e0 2024-04-26T18:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabdd7fa640 1 -- 172.21.15.84:0/1637685701 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fabec010e20 con 0x560e546560e0 2024-04-26T18:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabfe97c740 1 -- 172.21.15.84:0/1637685701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabf0003e20 con 0x560e546560e0 2024-04-26T18:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabdd7fa640 1 -- 172.21.15.84:0/1637685701 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fabec02a410 con 0x560e546560e0 2024-04-26T18:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabfe97c740 10 client.15687 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabfe97c740 1 -- 172.21.15.84:0/1637685701 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e5438f620 con 0x560e546560e0 2024-04-26T18:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.426+0000 7fabdd7fa640 1 -- 172.21.15.84:0/1637685701 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabf0003e20 con 0x560e546560e0 2024-04-26T18:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.427+0000 7fabdd7fa640 1 -- 172.21.15.84:0/1637685701 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fabec00f040 con 0x560e546560e0 2024-04-26T18:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.427+0000 7fabfe97c740 10 client.15687 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.427+0000 7fabfe97c740 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.427+0000 7fabfe97c740 20 client.15687 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:45.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.427+0000 7fabfe97c740 1 --2- 172.21.15.84:0/1637685701 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e54666190 0x560e54668580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.427+0000 7fabfe97c740 4 client.15687 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.427+0000 7fabfe97c740 1 -- 172.21.15.84:0/1637685701 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e545bf070 con 0x560e54666190 2024-04-26T18:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.427+0000 7fabf9925640 1 --2- 172.21.15.84:0/1637685701 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e54666190 0x560e54668580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.427+0000 7fabdd7fa640 10 client.15687 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.427+0000 7fabf9925640 1 --2- 172.21.15.84:0/1637685701 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e54666190 0x560e54668580 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63bc7cf248a6d22 server_cookie=1488c66e6d143e28 in_seq=0 out_seq=0 2024-04-26T18:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabdd7fa640 1 -- 172.21.15.84:0/1637685701 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e545bf070 con 0x560e54666190 2024-04-26T18:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabdd7fa640 10 client.15687 handle_command_reply: tid=0 2024-04-26T18:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabfe97c740 1 client.15687 shutdown 2024-04-26T18:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabfe97c740 1 -- 172.21.15.84:0/1637685701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 msgr2=0x560e54653b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabfe97c740 1 --2- 172.21.15.84:0/1637685701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e54653b10 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fabec010c50 tx=0x7fabec005850 comp rx=0 tx=0).stop 2024-04-26T18:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabfe97c740 1 -- 172.21.15.84:0/1637685701 shutdown_connections 2024-04-26T18:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabfe97c740 1 --2- 172.21.15.84:0/1637685701 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e54666190 0x560e54668580 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabfe97c740 1 --2- 172.21.15.84:0/1637685701 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e54653b10 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabfe97c740 1 --2- 172.21.15.84:0/1637685701 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464b820 0x560e546535d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabfe97c740 1 --2- 172.21.15.84:0/1637685701 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 0x560e54653090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabdd7fa640 0 client.15687 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabfe97c740 1 -- 172.21.15.84:0/1637685701 >> 172.21.15.84:0/1637685701 conn(0x560e54647410 msgr2=0x560e5464d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabfe97c740 1 -- 172.21.15.84:0/1637685701 shutdown_connections 2024-04-26T18:29:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.428+0000 7fabfe97c740 1 -- 172.21.15.84:0/1637685701 wait complete. 2024-04-26T18:29:45.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.429+0000 7fabfe97c740 20 client.15687 trim_cache size 0 max 16384 2024-04-26T18:29:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.443+0000 7fabfe97c740 1 Processor -- start 2024-04-26T18:29:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.443+0000 7fabfe97c740 1 -- start start 2024-04-26T18:29:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.443+0000 7fabfe97c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 0x560e54661b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.443+0000 7fabfe97c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5464b820 0x560e5464d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabfe97c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e546560e0 0x560e5466de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabfe97c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41ae620 con 0x560e5464b820 2024-04-26T18:29:45.449 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabfe97c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41b0160 con 0x560e5464af00 2024-04-26T18:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabfe97c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41aecf0 con 0x560e546560e0 2024-04-26T18:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabf9124640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5464b820 0x560e5464d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabf9124640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5464b820 0x560e5464d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51330/0 (socket says 172.21.15.84:51330) 2024-04-26T18:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabf9124640 1 -- 172.21.15.84:0/4080253927 learned_addr learned my addr 172.21.15.84:0/4080253927 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabf9925640 1 --2- 172.21.15.84:0/4080253927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 0x560e54661b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabfa126640 1 --2- 172.21.15.84:0/4080253927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e546560e0 0x560e5466de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabf9124640 1 -- 172.21.15.84:0/4080253927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e546560e0 msgr2=0x560e5466de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabf9124640 1 --2- 172.21.15.84:0/4080253927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e546560e0 0x560e5466de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabf9124640 1 -- 172.21.15.84:0/4080253927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 msgr2=0x560e54661b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabf9124640 1 --2- 172.21.15.84:0/4080253927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 0x560e54661b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabf9124640 1 -- 172.21.15.84:0/4080253927 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e54658840 con 0x560e5464b820 2024-04-26T18:29:45.450 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabf9124640 1 --2- 172.21.15.84:0/4080253927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5464b820 0x560e5464d8c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fabe40cdc60 tx=0x7fabe4098740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9029136ab0f97ce server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabdf7fe640 1 -- 172.21.15.84:0/4080253927 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4020040 con 0x560e5464b820 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabdf7fe640 1 -- 172.21.15.84:0/4080253927 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabe40a07b0 con 0x560e5464b820 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.444+0000 7fabdf7fe640 1 -- 172.21.15.84:0/4080253927 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabe4098c00 con 0x560e5464b820 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.445+0000 7fabfe97c740 1 -- 172.21.15.84:0/4080253927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5464b820 msgr2=0x560e5464d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.445+0000 7fabfe97c740 1 --2- 172.21.15.84:0/4080253927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5464b820 0x560e5464d8c0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fabe40cdc60 tx=0x7fabe4098740 comp rx=0 tx=0).stop 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.445+0000 7fabfe97c740 1 -- 172.21.15.84:0/4080253927 shutdown_connections 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.445+0000 7fabfe97c740 1 --2- 172.21.15.84:0/4080253927 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e546560e0 0x560e5466de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.445+0000 7fabfe97c740 1 --2- 172.21.15.84:0/4080253927 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5464b820 0x560e5464d8c0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.445+0000 7fabfe97c740 1 --2- 172.21.15.84:0/4080253927 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464af00 0x560e54661b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.445+0000 7fabfe97c740 1 -- 172.21.15.84:0/4080253927 >> 172.21.15.84:0/4080253927 conn(0x560e54647410 msgr2=0x560e546465b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.445+0000 7fabfe97c740 1 -- 172.21.15.84:0/4080253927 shutdown_connections 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.445+0000 7fabfe97c740 1 -- 172.21.15.84:0/4080253927 wait complete. 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.445+0000 7fabfe97c740 1 -- start start 2024-04-26T18:29:45.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.445+0000 7fabfe97c740 1 Processor -- start 2024-04-26T18:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfe97c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464af00 0x560e546504d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfe97c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464b820 0x560e54650a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfe97c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e546cb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfe97c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41afa90 con 0x560e546560e0 2024-04-26T18:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfe97c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41af3c0 con 0x560e5464b820 2024-04-26T18:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfe97c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fabf41ac600 con 0x560e5464af00 2024-04-26T18:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfe97c740 10 client.0 _ll_register_callbacks cb 0x7fabec00b890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfa126640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e546cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfa126640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e546cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51334/0 (socket says 172.21.15.84:51334) 2024-04-26T18:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfa126640 1 -- 172.21.15.84:0/2211141665 learned_addr learned my addr 172.21.15.84:0/2211141665 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabf9925640 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464af00 0x560e546504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabf9124640 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464b820 0x560e54650a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfa126640 1 -- 172.21.15.84:0/2211141665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464af00 msgr2=0x560e546504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfa126640 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464af00 0x560e546504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfa126640 1 -- 172.21.15.84:0/2211141665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464b820 msgr2=0x560e54650a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfa126640 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464b820 0x560e54650a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfa126640 1 -- 172.21.15.84:0/2211141665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e546688f0 con 0x560e546560e0 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabf9124640 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464b820 0x560e54650a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfa126640 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e546cb700 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fabec005940 tx=0x7fabec00bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e478746bae686610 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabdd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabdd7fa640 1 -- 172.21.15.84:0/2211141665 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabec037070 con 0x560e546560e0 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabdd7fa640 1 -- 172.21.15.84:0/2211141665 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fabec013240 con 0x560e546560e0 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.446+0000 7fabfe97c740 1 -- 172.21.15.84:0/2211141665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabe40a07b0 con 0x560e546560e0 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.447+0000 7fabdd7fa640 1 -- 172.21.15.84:0/2211141665 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fabec043050 con 0x560e546560e0 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.447+0000 7fabfe97c740 10 client.15693 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:45.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.447+0000 7fabfe97c740 1 -- 172.21.15.84:0/2211141665 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e546ccd20 con 0x560e546560e0 2024-04-26T18:29:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.447+0000 7fabdd7fa640 1 -- 172.21.15.84:0/2211141665 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabe40a07b0 con 0x560e546560e0 2024-04-26T18:29:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.447+0000 7fabdd7fa640 1 -- 172.21.15.84:0/2211141665 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fabec033440 con 0x560e546560e0 2024-04-26T18:29:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.447+0000 7fabfe97c740 10 client.15693 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.447+0000 7fabfe97c740 10 client.15693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.447+0000 7fabfe97c740 20 client.15693 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.447+0000 7fabfe97c740 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e546cd440 0x560e546cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.447+0000 7fabfe97c740 4 client.15693 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:29:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.447+0000 7fabfe97c740 1 -- 172.21.15.84:0/2211141665 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560e545b8ee0 con 0x560e546cd440 2024-04-26T18:29:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.447+0000 7fabf9925640 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e546cd440 0x560e546cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.448+0000 7fabf9925640 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e546cd440 0x560e546cf860 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49b960852bfb47aa server_cookie=961fd42baec83e41 in_seq=0 out_seq=0 2024-04-26T18:29:45.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.448+0000 7fabdd7fa640 10 client.15693 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:45.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabdd7fa640 1 -- 172.21.15.84:0/2211141665 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560e545b8ee0 con 0x560e546cd440 2024-04-26T18:29:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabdd7fa640 10 client.15693 handle_command_reply: tid=0 2024-04-26T18:29:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabfe97c740 1 client.15693 shutdown 2024-04-26T18:29:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabfe97c740 1 -- 172.21.15.84:0/2211141665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 msgr2=0x560e546cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabfe97c740 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e546cb700 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fabec005940 tx=0x7fabec00bc60 comp rx=0 tx=0).stop 2024-04-26T18:29:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabfe97c740 1 -- 172.21.15.84:0/2211141665 shutdown_connections 2024-04-26T18:29:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabfe97c740 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e546cd440 0x560e546cf860 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabfe97c740 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e546560e0 0x560e546cb700 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabdd7fa640 0 client.15693 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabfe97c740 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5464b820 0x560e54650a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabfe97c740 1 --2- 172.21.15.84:0/2211141665 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5464af00 0x560e546504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabfe97c740 1 -- 172.21.15.84:0/2211141665 >> 172.21.15.84:0/2211141665 conn(0x560e54647410 msgr2=0x560e54653050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabfe97c740 1 -- 172.21.15.84:0/2211141665 shutdown_connections 2024-04-26T18:29:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabfe97c740 1 -- 172.21.15.84:0/2211141665 wait complete. 2024-04-26T18:29:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.453+0000 7fabfe97c740 20 client.15693 trim_cache size 0 max 16384 2024-04-26T18:29:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:29:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:29:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:29:45.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:29:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.454+0000 7fabfbb89640 1 -- 172.21.15.84:0/2663891196 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fabe00a3dc0 msgr2=0x7fabe00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:45.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.454+0000 7fabfbb89640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fabe00a3dc0 0x7fabe00a6280 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fabec013800 tx=0x7fabec00ba50 comp rx=0 tx=0).stop 2024-04-26T18:29:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.454+0000 7fabfbb89640 1 -- 172.21.15.84:0/2663891196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf41044e0 msgr2=0x7fabf4199670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.454+0000 7fabfbb89640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf41044e0 0x7fabf4199670 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fabe400d950 tx=0x7fabe400de10 comp rx=0 tx=0).stop 2024-04-26T18:29:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.454+0000 7fabfbb89640 1 -- 172.21.15.84:0/2663891196 shutdown_connections 2024-04-26T18:29:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.454+0000 7fabfbb89640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fabe00a3dc0 0x7fabe00a6280 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.454+0000 7fabfbb89640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fabf410cf00 0x7fabf419d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.454+0000 7fabfbb89640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fabf41044e0 0x7fabf4199670 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.454+0000 7fabfbb89640 1 --2- 172.21.15.84:0/2663891196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fabf4103bc0 0x7fabf4199130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.454+0000 7fabfbb89640 1 -- 172.21.15.84:0/2663891196 >> 172.21.15.84:0/2663891196 conn(0x7fabf40789b0 msgr2=0x7fabf40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.454+0000 7fabfbb89640 1 -- 172.21.15.84:0/2663891196 shutdown_connections 2024-04-26T18:29:45.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:45.454+0000 7fabfbb89640 1 -- 172.21.15.84:0/2663891196 wait complete. 2024-04-26T18:29:46.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:46 smithi084 ceph-mon[29141]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T18:29:46.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:46 smithi103 ceph-mon[32727]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T18:29:46.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:46 smithi174 ceph-mon[32705]: pgmap v57: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T18:29:48.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:48 smithi084 ceph-mon[29141]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T18:29:48.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:48 smithi103 ceph-mon[32727]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T18:29:48.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:48 smithi174 ceph-mon[32705]: pgmap v58: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 0 B/s wr, 8 op/s 2024-04-26T18:29:50.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:50 smithi084 ceph-mon[29141]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-26T18:29:50.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:50 smithi103 ceph-mon[32727]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-26T18:29:50.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:50 smithi174 ceph-mon[32705]: pgmap v59: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-26T18:29:52.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:52 smithi084 ceph-mon[29141]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T18:29:52.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:52 smithi103 ceph-mon[32727]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T18:29:52.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:52 smithi174 ceph-mon[32705]: pgmap v60: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 9 op/s 2024-04-26T18:29:54.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:54 smithi084 ceph-mon[29141]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-26T18:29:54.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:54 smithi103 ceph-mon[32727]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-26T18:29:54.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:54 smithi174 ceph-mon[32705]: pgmap v61: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-26T18:29:56.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:56 smithi084 ceph-mon[29141]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T18:29:56.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:56 smithi103 ceph-mon[32727]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T18:29:56.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:56 smithi174 ceph-mon[32705]: pgmap v62: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T18:29:58.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.551+0000 7f7a8de09640 1 -- 172.21.15.84:0/2301435184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a8806bb20 msgr2=0x7f7a8810f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.551+0000 7f7a8de09640 1 --2- 172.21.15.84:0/2301435184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a8806bb20 0x7f7a8810f7d0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009ca0 tx=0x7f7a8002a950 comp rx=0 tx=0).stop 2024-04-26T18:29:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.551+0000 7f7a8de09640 1 -- 172.21.15.84:0/2301435184 shutdown_connections 2024-04-26T18:29:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.551+0000 7f7a8de09640 1 --2- 172.21.15.84:0/2301435184 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a8806bb20 0x7f7a8810f7d0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.551+0000 7f7a8de09640 1 --2- 172.21.15.84:0/2301435184 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8806b160 0x7f7a8806b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.551+0000 7f7a8de09640 1 --2- 172.21.15.84:0/2301435184 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a88108b30 0x7f7a88108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.551+0000 7f7a8de09640 1 -- 172.21.15.84:0/2301435184 >> 172.21.15.84:0/2301435184 conn(0x7f7a88077630 msgr2=0x7f7a88077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.551+0000 7f7a8de09640 1 -- 172.21.15.84:0/2301435184 shutdown_connections 2024-04-26T18:29:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.551+0000 7f7a8de09640 1 -- 172.21.15.84:0/2301435184 wait complete. 2024-04-26T18:29:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.552+0000 7f7a8de09640 1 Processor -- start 2024-04-26T18:29:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.552+0000 7f7a8de09640 1 -- start start 2024-04-26T18:29:58.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.552+0000 7f7a8de09640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a8806b160 0x7f7a88199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.552+0000 7f7a8de09640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8806bb20 0x7f7a88199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.552+0000 7f7a8de09640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a88108b30 0x7f7a8819d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.552+0000 7f7a8de09640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7a88112c00 con 0x7f7a8806b160 2024-04-26T18:29:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.552+0000 7f7a8de09640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a88112a80 con 0x7f7a8806bb20 2024-04-26T18:29:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.552+0000 7f7a8de09640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a88112d80 con 0x7f7a88108b30 2024-04-26T18:29:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.552+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a8806b160 0x7f7a88199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a877fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a8806b160 0x7f7a88199040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53638/0 (socket says 172.21.15.84:53638) 2024-04-26T18:29:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a877fe640 1 -- 172.21.15.84:0/871120388 learned_addr learned my addr 172.21.15.84:0/871120388 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a87fff640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a88108b30 0x7f7a8819d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a86ffd640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8806bb20 0x7f7a88199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a877fe640 1 -- 172.21.15.84:0/871120388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a88108b30 msgr2=0x7f7a8819d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a877fe640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a88108b30 0x7f7a8819d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a877fe640 1 -- 172.21.15.84:0/871120388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8806bb20 msgr2=0x7f7a88199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a877fe640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8806bb20 0x7f7a88199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a877fe640 1 -- 172.21.15.84:0/871120388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8819dfe0 con 0x7f7a8806b160 2024-04-26T18:29:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a877fe640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a8806b160 0x7f7a88199040 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800b4a0 tx=0x7f7a7800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a84ff9640 1 -- 172.21.15.84:0/871120388 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78014070 con 0x7f7a8806b160 2024-04-26T18:29:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a8de09640 1 -- 172.21.15.84:0/871120388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8819e2d0 con 0x7f7a8806b160 2024-04-26T18:29:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a84ff9640 1 -- 172.21.15.84:0/871120388 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a78010ac0 con 0x7f7a8806b160 2024-04-26T18:29:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a84ff9640 1 -- 172.21.15.84:0/871120388 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a78017c30 con 0x7f7a8806b160 2024-04-26T18:29:58.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.553+0000 7f7a8de09640 1 -- 172.21.15.84:0/871120388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a881a5b60 con 0x7f7a8806b160 2024-04-26T18:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.555+0000 7f7a84ff9640 1 -- 172.21.15.84:0/871120388 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f7a78017070 con 0x7f7a8806b160 2024-04-26T18:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a90c02740 1 Processor -- start 2024-04-26T18:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a90c02740 1 -- start start 2024-04-26T18:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a90c02740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01df00 0x55d26c01e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a84ff9640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7a700a3e40 0x7f7a700a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a90c02740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01e820 0x55d26c028ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a90c02740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c0290e0 0x55d26c02b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a90c02740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d26bf84460 con 0x55d26c0290e0 2024-04-26T18:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a90c02740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d26bf3e590 con 0x55d26c01e820 2024-04-26T18:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a90c02740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d26bf80420 con 0x55d26c01df00 2024-04-26T18:29:58.563 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a877fe640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7a700a3e40 0x7f7a700a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c0290e0 0x55d26c02b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a86ffd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01e820 0x55d26c028ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c0290e0 0x55d26c02b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53644/0 (socket says 172.21.15.84:53644) 2024-04-26T18:29:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a87fff640 1 -- 172.21.15.84:0/2452378153 learned_addr learned my addr 172.21.15.84:0/2452378153 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a84ff9640 1 -- 172.21.15.84:0/871120388 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f7a780cfb00 con 0x7f7a8806b160 2024-04-26T18:29:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.556+0000 7f7a87fff640 1 --2- 172.21.15.84:0/2452378153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01df00 0x55d26c01e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.557+0000 7f7a86ffd640 1 -- 172.21.15.84:0/2452378153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01df00 msgr2=0x55d26c01e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.557+0000 7f7a86ffd640 1 --2- 172.21.15.84:0/2452378153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01df00 0x55d26c01e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.557+0000 7f7a86ffd640 1 -- 172.21.15.84:0/2452378153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c0290e0 msgr2=0x55d26c02b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.557+0000 7f7a86ffd640 1 --2- 172.21.15.84:0/2452378153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c0290e0 0x55d26c02b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.557+0000 7f7a86ffd640 1 -- 172.21.15.84:0/2452378153 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d26bf4b0d0 con 0x55d26c01e820 2024-04-26T18:29:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.557+0000 7f7a877fe640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7a700a3e40 0x7f7a700a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7a7809bb30 tx=0x7f7a78097e40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:58.564 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.557+0000 7f7a86ffd640 1 --2- 172.21.15.84:0/2452378153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01e820 0x55d26c028ba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0058c0 tx=0x7f7a7c00e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dff82d50bcf830d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.557+0000 7f7a6dffb640 1 -- 172.21.15.84:0/2452378153 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c00ee30 con 0x55d26c01e820 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.557+0000 7f7a6dffb640 1 -- 172.21.15.84:0/2452378153 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a7c0035f0 con 0x55d26c01e820 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.557+0000 7f7a6dffb640 1 -- 172.21.15.84:0/2452378153 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c010690 con 0x55d26c01e820 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.558+0000 7f7a90c02740 1 -- 172.21.15.84:0/2452378153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01e820 msgr2=0x55d26c028ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.558+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2452378153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01e820 0x55d26c028ba0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0058c0 tx=0x7f7a7c00e350 comp rx=0 tx=0).stop 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.558+0000 7f7a90c02740 1 -- 172.21.15.84:0/2452378153 shutdown_connections 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.558+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2452378153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c0290e0 0x55d26c02b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.558+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2452378153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01e820 0x55d26c028ba0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.558+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2452378153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01df00 0x55d26c01e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.558+0000 7f7a90c02740 1 -- 172.21.15.84:0/2452378153 >> 172.21.15.84:0/2452378153 conn(0x55d26c01a410 msgr2=0x55d26c01a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.558+0000 7f7a90c02740 1 -- 172.21.15.84:0/2452378153 shutdown_connections 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.558+0000 7f7a90c02740 1 -- 172.21.15.84:0/2452378153 wait complete. 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.558+0000 7f7a90c02740 1 -- start start 2024-04-26T18:29:58.565 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.558+0000 7f7a90c02740 1 Processor -- start 2024-04-26T18:29:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a90c02740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01df00 0x55d26c026020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a90c02740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 0x55d26c026560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a90c02740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c0290e0 0x55d26c026aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a90c02740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d26c02bd10 con 0x55d26c01e820 2024-04-26T18:29:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a90c02740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d26bf0bb60 con 0x55d26c01df00 2024-04-26T18:29:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a90c02740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d26c02be90 con 0x55d26c0290e0 2024-04-26T18:29:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a86ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 0x55d26c026560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a90c02740 10 client.0 _ll_register_callbacks cb 0x55d26bf302b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a86ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 0x55d26c026560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53648/0 (socket says 172.21.15.84:53648) 2024-04-26T18:29:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a86ffd640 1 -- 172.21.15.84:0/2288071545 learned_addr learned my addr 172.21.15.84:0/2288071545 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a87fff640 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c0290e0 0x55d26c026aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a87fff640 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01df00 0x55d26c026020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.566 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a86ffd640 1 -- 172.21.15.84:0/2288071545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c0290e0 msgr2=0x55d26c026aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a86ffd640 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c0290e0 0x55d26c026aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a86ffd640 1 -- 172.21.15.84:0/2288071545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01df00 msgr2=0x55d26c026020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a86ffd640 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01df00 0x55d26c026020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a86ffd640 1 -- 172.21.15.84:0/2288071545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d26bc96210 con 0x55d26c01e820 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a87fff640 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01df00 0x55d26c026020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a86ffd640 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 0x55d26c026560 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c003e10 tx=0x7f7a7c0039d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc550ee1f75ec2e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a66ffd640 1 -- 172.21.15.84:0/2288071545 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c029060 con 0x55d26c01e820 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a66ffd640 1 -- 172.21.15.84:0/2288071545 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f7a7c004120 con 0x55d26c01e820 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.559+0000 7f7a90c02740 1 -- 172.21.15.84:0/2288071545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a7c0035f0 con 0x55d26c01e820 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.560+0000 7f7a66ffd640 1 -- 172.21.15.84:0/2288071545 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7a7c004320 con 0x55d26c01e820 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.560+0000 7f7a90c02740 10 client.15705 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.560+0000 7f7a90c02740 1 -- 172.21.15.84:0/2288071545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d26bd62620 con 0x55d26c01e820 2024-04-26T18:29:58.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.560+0000 7f7a66ffd640 1 -- 172.21.15.84:0/2288071545 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a7c0035f0 con 0x55d26c01e820 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.560+0000 7f7a66ffd640 1 -- 172.21.15.84:0/2288071545 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7a7c004920 con 0x55d26c01e820 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.560+0000 7f7a90c02740 10 client.15705 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.560+0000 7f7a90c02740 10 client.15705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.560+0000 7f7a90c02740 20 client.15705 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.560+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d26c0390c0 0x55d26c03b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.560+0000 7f7a90c02740 4 client.15705 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.560+0000 7f7a90c02740 1 -- 172.21.15.84:0/2288071545 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d26bf92070 con 0x55d26c0390c0 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.560+0000 7f7a87fff640 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d26c0390c0 0x55d26c03b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.561+0000 7f7a87fff640 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d26c0390c0 0x55d26c03b4b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde8c5532245e6b1 server_cookie=c7ea6cbbbf6abe40 in_seq=0 out_seq=0 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.561+0000 7f7a66ffd640 10 client.15705 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.561+0000 7f7a66ffd640 1 -- 172.21.15.84:0/2288071545 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d26bf92070 con 0x55d26c0390c0 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.561+0000 7f7a66ffd640 10 client.15705 handle_command_reply: tid=0 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.561+0000 7f7a90c02740 1 client.15705 shutdown 2024-04-26T18:29:58.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.562+0000 7f7a90c02740 1 -- 172.21.15.84:0/2288071545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 msgr2=0x55d26c026560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.562+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 0x55d26c026560 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c003e10 tx=0x7f7a7c0039d0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.562+0000 7f7a90c02740 1 -- 172.21.15.84:0/2288071545 shutdown_connections 2024-04-26T18:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.562+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d26c0390c0 0x55d26c03b4b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.562+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c0290e0 0x55d26c026aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.562+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 0x55d26c026560 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.562+0000 7f7a66ffd640 0 client.15705 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.562+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2288071545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01df00 0x55d26c026020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.562+0000 7f7a90c02740 1 -- 172.21.15.84:0/2288071545 >> 172.21.15.84:0/2288071545 conn(0x55d26c01a410 msgr2=0x55d26c020140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.562+0000 7f7a90c02740 1 -- 172.21.15.84:0/2288071545 shutdown_connections 2024-04-26T18:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.562+0000 7f7a90c02740 1 -- 172.21.15.84:0/2288071545 wait complete. 2024-04-26T18:29:58.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.562+0000 7f7a90c02740 20 client.15705 trim_cache size 0 max 16384 2024-04-26T18:29:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.577+0000 7f7a90c02740 1 Processor -- start 2024-04-26T18:29:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.577+0000 7f7a90c02740 1 -- start start 2024-04-26T18:29:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.577+0000 7f7a90c02740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01df00 0x55d26c034bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.577+0000 7f7a90c02740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 0x55d26c01ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.577+0000 7f7a90c02740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c0290e0 0x55d26c040ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.577+0000 7f7a90c02740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881ae5c0 con 0x55d26c01e820 2024-04-26T18:29:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.577+0000 7f7a90c02740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881b0100 con 0x55d26c0290e0 2024-04-26T18:29:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.577+0000 7f7a90c02740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881aec90 con 0x55d26c01df00 2024-04-26T18:29:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a86ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 0x55d26c01ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a86ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 0x55d26c01ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53652/0 (socket says 172.21.15.84:53652) 2024-04-26T18:29:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a86ffd640 1 -- 172.21.15.84:0/974532008 learned_addr learned my addr 172.21.15.84:0/974532008 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a87fff640 1 --2- 172.21.15.84:0/974532008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01df00 0x55d26c034bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a87fff640 1 --2- 172.21.15.84:0/974532008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c0290e0 0x55d26c040ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a86ffd640 1 -- 172.21.15.84:0/974532008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01df00 msgr2=0x55d26c034bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a86ffd640 1 --2- 172.21.15.84:0/974532008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01df00 0x55d26c034bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a86ffd640 1 -- 172.21.15.84:0/974532008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c0290e0 msgr2=0x55d26c040ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:58.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a86ffd640 1 --2- 172.21.15.84:0/974532008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c0290e0 0x55d26c040ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a86ffd640 1 -- 172.21.15.84:0/974532008 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d26c01e650 con 0x55d26c01e820 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a86ffd640 1 --2- 172.21.15.84:0/974532008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 0x55d26c01ffc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c004fd0 tx=0x7f7a7c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc5dad95e2ebc45c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a6d7fa640 1 -- 172.21.15.84:0/974532008 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c01e070 con 0x55d26c01e820 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a6d7fa640 1 -- 172.21.15.84:0/974532008 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a7c00c3b0 con 0x55d26c01e820 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.578+0000 7f7a6d7fa640 1 -- 172.21.15.84:0/974532008 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0043a0 con 0x55d26c01e820 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.579+0000 7f7a90c02740 1 -- 172.21.15.84:0/974532008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 msgr2=0x55d26c01ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.579+0000 7f7a90c02740 1 --2- 172.21.15.84:0/974532008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 0x55d26c01ffc0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c004fd0 tx=0x7f7a7c005ce0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.579+0000 7f7a90c02740 1 -- 172.21.15.84:0/974532008 shutdown_connections 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.579+0000 7f7a90c02740 1 --2- 172.21.15.84:0/974532008 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c0290e0 0x55d26c040ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.579+0000 7f7a90c02740 1 --2- 172.21.15.84:0/974532008 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c01e820 0x55d26c01ffc0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.579+0000 7f7a90c02740 1 --2- 172.21.15.84:0/974532008 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01df00 0x55d26c034bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.579+0000 7f7a90c02740 1 -- 172.21.15.84:0/974532008 >> 172.21.15.84:0/974532008 conn(0x55d26c01a410 msgr2=0x55d26c01f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.579+0000 7f7a90c02740 1 -- 172.21.15.84:0/974532008 shutdown_connections 2024-04-26T18:29:58.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.579+0000 7f7a90c02740 1 -- 172.21.15.84:0/974532008 wait complete. 2024-04-26T18:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.579+0000 7f7a90c02740 1 -- start start 2024-04-26T18:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a90c02740 1 Processor -- start 2024-04-26T18:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a90c02740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01df00 0x55d26c0230a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a90c02740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01e820 0x55d26c0235e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a90c02740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c0290e0 0x55d26c023b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a90c02740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881afa30 con 0x55d26c0290e0 2024-04-26T18:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a90c02740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881af360 con 0x55d26c01df00 2024-04-26T18:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a90c02740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7a881ac5a0 con 0x55d26c01e820 2024-04-26T18:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a90c02740 10 client.0 _ll_register_callbacks cb 0x7f7a7c014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c0290e0 0x55d26c023b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a87fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c0290e0 0x55d26c023b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:53660/0 (socket says 172.21.15.84:53660) 2024-04-26T18:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a87fff640 1 -- 172.21.15.84:0/2825526249 learned_addr learned my addr 172.21.15.84:0/2825526249 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:29:58.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a87fff640 1 --2- 172.21.15.84:0/2825526249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01df00 0x55d26c0230a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a86ffd640 1 --2- 172.21.15.84:0/2825526249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01e820 0x55d26c0235e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a87fff640 1 -- 172.21.15.84:0/2825526249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01e820 msgr2=0x55d26c0235e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a87fff640 1 --2- 172.21.15.84:0/2825526249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01e820 0x55d26c0235e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a87fff640 1 -- 172.21.15.84:0/2825526249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01df00 msgr2=0x55d26c0230a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a87fff640 1 --2- 172.21.15.84:0/2825526249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01df00 0x55d26c0230a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a87fff640 1 -- 172.21.15.84:0/2825526249 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d26c09e450 con 0x55d26c0290e0 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a87fff640 1 --2- 172.21.15.84:0/2825526249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c0290e0 0x55d26c023b20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7a8002ae70 tx=0x7f7a7c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5518f8ea6099512c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.580+0000 7f7a66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a66ffd640 1 -- 172.21.15.84:0/2825526249 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8004fe60 con 0x55d26c0290e0 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a66ffd640 1 -- 172.21.15.84:0/2825526249 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f7a80003970 con 0x55d26c0290e0 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a90c02740 1 -- 172.21.15.84:0/2825526249 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7a7c00c3b0 con 0x55d26c0290e0 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a66ffd640 1 -- 172.21.15.84:0/2825526249 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7a80034040 con 0x55d26c0290e0 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a90c02740 10 client.15711 fetch_fsmap learned FSMap version 29 2024-04-26T18:29:58.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a90c02740 1 -- 172.21.15.84:0/2825526249 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d26c09f7a0 con 0x55d26c0290e0 2024-04-26T18:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a66ffd640 1 -- 172.21.15.84:0/2825526249 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a7c00c3b0 con 0x55d26c0290e0 2024-04-26T18:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a66ffd640 1 -- 172.21.15.84:0/2825526249 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7a8004e460 con 0x55d26c0290e0 2024-04-26T18:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a90c02740 10 client.15711 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a90c02740 10 client.15711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a90c02740 20 client.15711 populate_metadata read hostname 'smithi084' 2024-04-26T18:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2825526249 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d26c09ff50 0x55d26c0a2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a90c02740 4 client.15711 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a90c02740 1 -- 172.21.15.84:0/2825526249 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d26bf8bee0 con 0x55d26c09ff50 2024-04-26T18:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.581+0000 7f7a86ffd640 1 --2- 172.21.15.84:0/2825526249 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d26c09ff50 0x55d26c0a2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.582+0000 7f7a86ffd640 1 --2- 172.21.15.84:0/2825526249 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d26c09ff50 0x55d26c0a2370 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e587bd5ecf3a4e server_cookie=bb71b5d5deeddf52 in_seq=0 out_seq=0 2024-04-26T18:29:58.587 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.582+0000 7f7a66ffd640 10 client.15711 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:58.588 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.585+0000 7f7a66ffd640 1 -- 172.21.15.84:0/2825526249 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d26bf8bee0 con 0x55d26c09ff50 2024-04-26T18:29:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.585+0000 7f7a66ffd640 10 client.15711 handle_command_reply: tid=0 2024-04-26T18:29:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.585+0000 7f7a90c02740 1 client.15711 shutdown 2024-04-26T18:29:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a90c02740 1 -- 172.21.15.84:0/2825526249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c0290e0 msgr2=0x55d26c023b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2825526249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c0290e0 0x55d26c023b20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7a8002ae70 tx=0x7f7a7c00ef40 comp rx=0 tx=0).stop 2024-04-26T18:29:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a90c02740 1 -- 172.21.15.84:0/2825526249 shutdown_connections 2024-04-26T18:29:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2825526249 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d26c09ff50 0x55d26c0a2370 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2825526249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d26c0290e0 0x55d26c023b20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2825526249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d26c01e820 0x55d26c0235e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a66ffd640 0 client.15711 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:29:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a90c02740 1 --2- 172.21.15.84:0/2825526249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d26c01df00 0x55d26c0230a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a90c02740 1 -- 172.21.15.84:0/2825526249 >> 172.21.15.84:0/2825526249 conn(0x55d26c01a410 msgr2=0x55d26c020140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a90c02740 1 -- 172.21.15.84:0/2825526249 shutdown_connections 2024-04-26T18:29:58.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a90c02740 1 -- 172.21.15.84:0/2825526249 wait complete. 2024-04-26T18:29:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a90c02740 20 client.15711 trim_cache size 0 max 16384 2024-04-26T18:29:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:29:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:29:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:29:58.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a8de09640 1 -- 172.21.15.84:0/871120388 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7a700a3e40 msgr2=0x7f7a700a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a8de09640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7a700a3e40 0x7f7a700a6300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7a7809bb30 tx=0x7f7a78097e40 comp rx=0 tx=0).stop 2024-04-26T18:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a8de09640 1 -- 172.21.15.84:0/871120388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a8806b160 msgr2=0x7f7a88199040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a8de09640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a8806b160 0x7f7a88199040 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7a7800b4a0 tx=0x7f7a7800b960 comp rx=0 tx=0).stop 2024-04-26T18:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a8de09640 1 -- 172.21.15.84:0/871120388 shutdown_connections 2024-04-26T18:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a8de09640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7a700a3e40 0x7f7a700a6300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a8de09640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7a88108b30 0x7f7a8819d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a8de09640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7a8806bb20 0x7f7a88199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a8de09640 1 --2- 172.21.15.84:0/871120388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7a8806b160 0x7f7a88199040 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:29:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.586+0000 7f7a8de09640 1 -- 172.21.15.84:0/871120388 >> 172.21.15.84:0/871120388 conn(0x7f7a88077630 msgr2=0x7f7a881006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:29:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.587+0000 7f7a8de09640 1 -- 172.21.15.84:0/871120388 shutdown_connections 2024-04-26T18:29:58.592 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:29:58.587+0000 7f7a8de09640 1 -- 172.21.15.84:0/871120388 wait complete. 2024-04-26T18:29:58.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:29:58 smithi084 ceph-mon[29141]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T18:29:58.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:29:58 smithi103 ceph-mon[32727]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T18:29:58.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:29:58 smithi174 ceph-mon[32705]: pgmap v63: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T18:30:00.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:00 smithi084 ceph-mon[29141]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-26T18:30:00.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:00 smithi084 ceph-mon[29141]: overall HEALTH_OK 2024-04-26T18:30:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:00 smithi103 ceph-mon[32727]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-26T18:30:00.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:00 smithi103 ceph-mon[32727]: overall HEALTH_OK 2024-04-26T18:30:00.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:00 smithi174 ceph-mon[32705]: pgmap v64: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 1023 B/s wr, 13 op/s 2024-04-26T18:30:00.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:00 smithi174 ceph-mon[32705]: overall HEALTH_OK 2024-04-26T18:30:02.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:02 smithi084 ceph-mon[29141]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:30:02.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:02 smithi103 ceph-mon[32727]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:30:02.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:02 smithi174 ceph-mon[32705]: pgmap v65: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:30:03.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:03 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:30:03.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:03 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:30:03.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:03 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:30:04.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:04 smithi084 ceph-mon[29141]: pgmap v66: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T18:30:04.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:04 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:30:04.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:04 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:30:04.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:04 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:30:04.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:04 smithi103 ceph-mon[32727]: pgmap v66: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T18:30:04.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:04 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:30:04.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:04 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:30:04.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:04 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:30:04.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:04 smithi174 ceph-mon[32705]: pgmap v66: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T18:30:04.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:04 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:30:04.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:04 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:30:04.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:04 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:30:06.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:06 smithi084 ceph-mon[29141]: pgmap v67: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T18:30:06.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:06 smithi103 ceph-mon[32727]: pgmap v67: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T18:30:06.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:06 smithi174 ceph-mon[32705]: pgmap v67: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T18:30:08.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:08 smithi084 ceph-mon[29141]: pgmap v68: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T18:30:08.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:08 smithi103 ceph-mon[32727]: pgmap v68: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T18:30:08.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:08 smithi174 ceph-mon[32705]: pgmap v68: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T18:30:10.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:10 smithi084 ceph-mon[29141]: pgmap v69: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T18:30:10.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:10 smithi103 ceph-mon[32727]: pgmap v69: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T18:30:10.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:10 smithi174 ceph-mon[32705]: pgmap v69: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 KiB/s wr, 27 op/s 2024-04-26T18:30:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.681+0000 7f3df614d640 1 -- 172.21.15.84:0/3369912182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df0103be0 msgr2=0x7f3df0103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.681+0000 7f3df614d640 1 --2- 172.21.15.84:0/3369912182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df0103be0 0x7f3df0103fc0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3de0009900 tx=0x7f3de002a480 comp rx=0 tx=0).stop 2024-04-26T18:30:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.682+0000 7f3df614d640 1 -- 172.21.15.84:0/3369912182 shutdown_connections 2024-04-26T18:30:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.682+0000 7f3df614d640 1 --2- 172.21.15.84:0/3369912182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3df010cfb0 0x7f3df010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.682+0000 7f3df614d640 1 --2- 172.21.15.84:0/3369912182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df0104500 0x7f3df010c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.682+0000 7f3df614d640 1 --2- 172.21.15.84:0/3369912182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df0103be0 0x7f3df0103fc0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.682+0000 7f3df614d640 1 -- 172.21.15.84:0/3369912182 >> 172.21.15.84:0/3369912182 conn(0x7f3df0077bb0 msgr2=0x7f3df0077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:11.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.682+0000 7f3df614d640 1 -- 172.21.15.84:0/3369912182 shutdown_connections 2024-04-26T18:30:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.682+0000 7f3df614d640 1 -- 172.21.15.84:0/3369912182 wait complete. 2024-04-26T18:30:11.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.682+0000 7f3df614d640 1 Processor -- start 2024-04-26T18:30:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.682+0000 7f3df614d640 1 -- start start 2024-04-26T18:30:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3df614d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3df0103be0 0x7f3df0101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3df614d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df0104500 0x7f3df0101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3df614d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df010cfb0 0x7f3df01022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3df614d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0112bf0 con 0x7f3df010cfb0 2024-04-26T18:30:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3df614d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0112a70 con 0x7f3df0104500 2024-04-26T18:30:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3df614d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3df0112d70 con 0x7f3df0103be0 2024-04-26T18:30:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3deffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df010cfb0 0x7f3df01022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3deffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df010cfb0 0x7f3df01022b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46476/0 (socket says 172.21.15.84:46476) 2024-04-26T18:30:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3deffff640 1 -- 172.21.15.84:0/1631744412 learned_addr learned my addr 172.21.15.84:0/1631744412 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3def7fe640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3df0103be0 0x7f3df0101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.685 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3deeffd640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df0104500 0x7f3df0101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3deffff640 1 -- 172.21.15.84:0/1631744412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3df0103be0 msgr2=0x7f3df0101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3deffff640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3df0103be0 0x7f3df0101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3deffff640 1 -- 172.21.15.84:0/1631744412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df0104500 msgr2=0x7f3df0101d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3deffff640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df0104500 0x7f3df0101d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3deffff640 1 -- 172.21.15.84:0/1631744412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3df01a75b0 con 0x7f3df010cfb0 2024-04-26T18:30:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3def7fe640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3df0103be0 0x7f3df0101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3deffff640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df010cfb0 0x7f3df01022b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3de800cab0 tx=0x7f3de800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3decff9640 1 -- 172.21.15.84:0/1631744412 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de800ae20 con 0x7f3df010cfb0 2024-04-26T18:30:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3df614d640 1 -- 172.21.15.84:0/1631744412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3df01a78a0 con 0x7f3df010cfb0 2024-04-26T18:30:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3decff9640 1 -- 172.21.15.84:0/1631744412 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de800f900 con 0x7f3df010cfb0 2024-04-26T18:30:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.683+0000 7f3decff9640 1 -- 172.21.15.84:0/1631744412 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de8018a50 con 0x7f3df010cfb0 2024-04-26T18:30:11.686 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.684+0000 7f3df614d640 1 -- 172.21.15.84:0/1631744412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3df01a7e40 con 0x7f3df010cfb0 2024-04-26T18:30:11.692 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.686+0000 7f3decff9640 1 -- 172.21.15.84:0/1631744412 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3de800f450 con 0x7f3df010cfb0 2024-04-26T18:30:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.686+0000 7f3df8e61740 1 Processor -- start 2024-04-26T18:30:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.686+0000 7f3df8e61740 1 -- start start 2024-04-26T18:30:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.686+0000 7f3df8e61740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409d72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.686+0000 7f3df8e61740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d7820 0x559b409e1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.686+0000 7f3df8e61740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 0x559b409e44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.686+0000 7f3df8e61740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559b4093d460 con 0x559b409d6f00 2024-04-26T18:30:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.686+0000 7f3df8e61740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559b408f7590 con 0x559b409e20e0 2024-04-26T18:30:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.686+0000 7f3df8e61740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b40939420 con 0x559b409d7820 2024-04-26T18:30:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.686+0000 7f3deeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409d72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.686+0000 7f3deeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409d72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46478/0 (socket says 172.21.15.84:46478) 2024-04-26T18:30:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.686+0000 7f3deeffd640 1 -- 172.21.15.84:0/828494414 learned_addr learned my addr 172.21.15.84:0/828494414 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.686+0000 7f3def7fe640 1 --2- 172.21.15.84:0/828494414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d7820 0x559b409e1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.687+0000 7f3decff9640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3dd80a3dc0 0x7f3dd80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.687+0000 7f3deeffd640 1 -- 172.21.15.84:0/828494414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d7820 msgr2=0x559b409e1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.687+0000 7f3deeffd640 1 --2- 172.21.15.84:0/828494414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d7820 0x559b409e1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.687+0000 7f3deeffd640 1 -- 172.21.15.84:0/828494414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 msgr2=0x559b409e44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.687+0000 7f3deeffd640 1 --2- 172.21.15.84:0/828494414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 0x559b409e44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.687+0000 7f3deeffd640 1 -- 172.21.15.84:0/828494414 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b409040d0 con 0x559b409d6f00 2024-04-26T18:30:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.687+0000 7f3deffff640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3dd80a3dc0 0x7f3dd80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.687+0000 7f3decff9640 1 -- 172.21.15.84:0/1631744412 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f3de80d0bb0 con 0x7f3df010cfb0 2024-04-26T18:30:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.687+0000 7f3deeffd640 1 --2- 172.21.15.84:0/828494414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409d72e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3de40058c0 tx=0x7f3de4011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f16d462ab4975c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.687+0000 7f3dd5ffb640 1 -- 172.21.15.84:0/828494414 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de401a400 con 0x559b409d6f00 2024-04-26T18:30:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.687+0000 7f3dd5ffb640 1 -- 172.21.15.84:0/828494414 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de401aa50 con 0x559b409d6f00 2024-04-26T18:30:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.687+0000 7f3dd5ffb640 1 -- 172.21.15.84:0/828494414 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4012a90 con 0x559b409d6f00 2024-04-26T18:30:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.688+0000 7f3df8e61740 1 -- 172.21.15.84:0/828494414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 msgr2=0x559b409d72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:11.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.688+0000 7f3deffff640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3dd80a3dc0 0x7f3dd80a6280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3de8098fa0 tx=0x7f3de806e580 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.688+0000 7f3df8e61740 1 --2- 172.21.15.84:0/828494414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409d72e0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f3de40058c0 tx=0x7f3de4011750 comp rx=0 tx=0).stop 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.688+0000 7f3df8e61740 1 -- 172.21.15.84:0/828494414 shutdown_connections 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.688+0000 7f3df8e61740 1 --2- 172.21.15.84:0/828494414 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 0x559b409e44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.688+0000 7f3df8e61740 1 --2- 172.21.15.84:0/828494414 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d7820 0x559b409e1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.688+0000 7f3df8e61740 1 --2- 172.21.15.84:0/828494414 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409d72e0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.688+0000 7f3df8e61740 1 -- 172.21.15.84:0/828494414 >> 172.21.15.84:0/828494414 conn(0x559b409d3410 msgr2=0x559b409d37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.688+0000 7f3df8e61740 1 -- 172.21.15.84:0/828494414 shutdown_connections 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.688+0000 7f3df8e61740 1 -- 172.21.15.84:0/828494414 wait complete. 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.688+0000 7f3df8e61740 1 -- start start 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.688+0000 7f3df8e61740 1 Processor -- start 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3df8e61740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409df020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3df8e61740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409d7820 0x559b409df560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3df8e61740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409e20e0 0x559b409dfaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3df8e61740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559b409e4d10 con 0x559b409d6f00 2024-04-26T18:30:11.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3df8e61740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559b408c4b60 con 0x559b409d7820 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3df8e61740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559b409e4e90 con 0x559b409e20e0 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3df8e61740 10 client.0 _ll_register_callbacks cb 0x559b408e92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3deeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409df020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3deeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409df020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46494/0 (socket says 172.21.15.84:46494) 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3deeffd640 1 -- 172.21.15.84:0/2396687542 learned_addr learned my addr 172.21.15.84:0/2396687542 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3deeffd640 1 -- 172.21.15.84:0/2396687542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409e20e0 msgr2=0x559b409dfaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3deeffd640 1 --2- 172.21.15.84:0/2396687542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409e20e0 0x559b409dfaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3deeffd640 1 -- 172.21.15.84:0/2396687542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409d7820 msgr2=0x559b409df560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3deeffd640 1 --2- 172.21.15.84:0/2396687542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409d7820 0x559b409df560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3deeffd640 1 -- 172.21.15.84:0/2396687542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b4064f210 con 0x559b409d6f00 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3deeffd640 1 --2- 172.21.15.84:0/2396687542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409df020 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3de4012690 tx=0x7f3de400bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e406f7037ac081 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3dceffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3dceffd640 1 -- 172.21.15.84:0/2396687542 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4020070 con 0x559b409d6f00 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3dceffd640 1 -- 172.21.15.84:0/2396687542 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f3de401a600 con 0x559b409d6f00 2024-04-26T18:30:11.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.689+0000 7f3df8e61740 1 -- 172.21.15.84:0/2396687542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3de401aa50 con 0x559b409d6f00 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.690+0000 7f3dceffd640 1 -- 172.21.15.84:0/2396687542 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3de401a800 con 0x559b409d6f00 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.690+0000 7f3df8e61740 10 client.15723 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.690+0000 7f3df8e61740 1 -- 172.21.15.84:0/2396687542 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b4071b620 con 0x559b409d6f00 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.690+0000 7f3dceffd640 1 -- 172.21.15.84:0/2396687542 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de401aa50 con 0x559b409d6f00 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.690+0000 7f3dceffd640 1 -- 172.21.15.84:0/2396687542 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3de402f990 con 0x559b409d6f00 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.690+0000 7f3df8e61740 10 client.15723 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.690+0000 7f3df8e61740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.690+0000 7f3df8e61740 20 client.15723 populate_metadata read hostname 'smithi084' 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.690+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2396687542 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559b409f20c0 0x559b409f44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.690+0000 7f3df8e61740 4 client.15723 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.690+0000 7f3df8e61740 1 -- 172.21.15.84:0/2396687542 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559b4094b070 con 0x559b409f20c0 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.690+0000 7f3def7fe640 1 --2- 172.21.15.84:0/2396687542 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559b409f20c0 0x559b409f44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.691+0000 7f3def7fe640 1 --2- 172.21.15.84:0/2396687542 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559b409f20c0 0x559b409f44b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df8a8b22834296e8 server_cookie=9354658e465fa95c in_seq=0 out_seq=0 2024-04-26T18:30:11.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.691+0000 7f3dceffd640 10 client.15723 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3dceffd640 1 -- 172.21.15.84:0/2396687542 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559b4094b070 con 0x559b409f20c0 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3dceffd640 10 client.15723 handle_command_reply: tid=0 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3df8e61740 1 client.15723 shutdown 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3df8e61740 1 -- 172.21.15.84:0/2396687542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 msgr2=0x559b409df020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2396687542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409df020 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f3de4012690 tx=0x7f3de400bad0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3df8e61740 1 -- 172.21.15.84:0/2396687542 shutdown_connections 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2396687542 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559b409f20c0 0x559b409f44b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2396687542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409e20e0 0x559b409dfaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2396687542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409d7820 0x559b409df560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3dceffd640 0 client.15723 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2396687542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409df020 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3df8e61740 1 -- 172.21.15.84:0/2396687542 >> 172.21.15.84:0/2396687542 conn(0x559b409d3410 msgr2=0x559b409d9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3df8e61740 1 -- 172.21.15.84:0/2396687542 shutdown_connections 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3df8e61740 1 -- 172.21.15.84:0/2396687542 wait complete. 2024-04-26T18:30:11.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.692+0000 7f3df8e61740 20 client.15723 trim_cache size 0 max 16384 2024-04-26T18:30:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3df8e61740 1 Processor -- start 2024-04-26T18:30:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3df8e61740 1 -- start start 2024-04-26T18:30:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3df8e61740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d6f00 0x559b409edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3df8e61740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d7820 0x559b409d9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3df8e61740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 0x559b409de900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3df8e61740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b0810 con 0x559b409d7820 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3df8e61740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b2350 con 0x559b409e20e0 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3df8e61740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b0ee0 con 0x559b409d6f00 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3def7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d7820 0x559b409d9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3def7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d7820 0x559b409d9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46504/0 (socket says 172.21.15.84:46504) 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3def7fe640 1 -- 172.21.15.84:0/2853768807 learned_addr learned my addr 172.21.15.84:0/2853768807 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3deeffd640 1 --2- 172.21.15.84:0/2853768807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d6f00 0x559b409edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3def7fe640 1 -- 172.21.15.84:0/2853768807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d6f00 msgr2=0x559b409edbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3def7fe640 1 --2- 172.21.15.84:0/2853768807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d6f00 0x559b409edbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3deeffd640 1 --2- 172.21.15.84:0/2853768807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 0x559b409de900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3def7fe640 1 -- 172.21.15.84:0/2853768807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 msgr2=0x559b409de900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3def7fe640 1 --2- 172.21.15.84:0/2853768807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 0x559b409de900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.707+0000 7f3def7fe640 1 -- 172.21.15.84:0/2853768807 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559b409d41a0 con 0x559b409d7820 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.708+0000 7f3def7fe640 1 --2- 172.21.15.84:0/2853768807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d7820 0x559b409d9380 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005940 tx=0x7f3de0009880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329f0e438647613c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.708+0000 7f3dd57fa640 1 -- 172.21.15.84:0/2853768807 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0035040 con 0x559b409d7820 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.708+0000 7f3dd57fa640 1 -- 172.21.15.84:0/2853768807 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de0004040 con 0x559b409d7820 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.708+0000 7f3dd57fa640 1 -- 172.21.15.84:0/2853768807 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de0043250 con 0x559b409d7820 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.708+0000 7f3df8e61740 1 -- 172.21.15.84:0/2853768807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d7820 msgr2=0x559b409d9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.708+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2853768807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d7820 0x559b409d9380 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f3de0005940 tx=0x7f3de0009880 comp rx=0 tx=0).stop 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.708+0000 7f3df8e61740 1 -- 172.21.15.84:0/2853768807 shutdown_connections 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.708+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2853768807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 0x559b409de900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.708+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2853768807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d7820 0x559b409d9380 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.708+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2853768807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d6f00 0x559b409edbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.708+0000 7f3df8e61740 1 -- 172.21.15.84:0/2853768807 >> 172.21.15.84:0/2853768807 conn(0x559b409d3410 msgr2=0x559b409df870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.708+0000 7f3df8e61740 1 -- 172.21.15.84:0/2853768807 shutdown_connections 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.708+0000 7f3df8e61740 1 -- 172.21.15.84:0/2853768807 wait complete. 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.709+0000 7f3df8e61740 1 -- start start 2024-04-26T18:30:11.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.709+0000 7f3df8e61740 1 Processor -- start 2024-04-26T18:30:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.709+0000 7f3df8e61740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409dc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.709+0000 7f3df8e61740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d7820 0x559b409dc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.709+0000 7f3df8e61740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 0x559b40a57320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.709+0000 7f3df8e61740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b1c80 con 0x559b409d6f00 2024-04-26T18:30:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.709+0000 7f3df8e61740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01b15b0 con 0x559b409e20e0 2024-04-26T18:30:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.709+0000 7f3df8e61740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3df01ae7f0 con 0x559b409d7820 2024-04-26T18:30:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.709+0000 7f3df8e61740 10 client.0 _ll_register_callbacks cb 0x7f3de0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.709+0000 7f3deeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409dc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.709+0000 7f3deeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409dc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46506/0 (socket says 172.21.15.84:46506) 2024-04-26T18:30:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.709+0000 7f3deeffd640 1 -- 172.21.15.84:0/2850392362 learned_addr learned my addr 172.21.15.84:0/2850392362 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.709+0000 7f3deeffd640 1 --2- 172.21.15.84:0/2850392362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 0x559b40a57320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3deeffd640 1 -- 172.21.15.84:0/2850392362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d7820 msgr2=0x559b409dc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:11.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3deeffd640 1 --2- 172.21.15.84:0/2850392362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d7820 0x559b409dc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3deeffd640 1 -- 172.21.15.84:0/2850392362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 msgr2=0x559b40a57320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3deeffd640 1 --2- 172.21.15.84:0/2850392362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 0x559b40a57320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3deeffd640 1 -- 172.21.15.84:0/2850392362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559b40a57860 con 0x559b409d6f00 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3deeffd640 1 --2- 172.21.15.84:0/2850392362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409dc090 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3de402ff00 tx=0x7f3de4011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbf7d6422872844 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3dceffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3dceffd640 1 -- 172.21.15.84:0/2850392362 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3de4020040 con 0x559b409d6f00 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3dceffd640 1 -- 172.21.15.84:0/2850392362 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f3de40052f0 con 0x559b409d6f00 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3df8e61740 1 -- 172.21.15.84:0/2850392362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3de0004040 con 0x559b409d6f00 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3dceffd640 1 -- 172.21.15.84:0/2850392362 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3de40054f0 con 0x559b409d6f00 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3df8e61740 10 client.15729 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3df8e61740 1 -- 172.21.15.84:0/2850392362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559b40a58a00 con 0x559b409d6f00 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3dceffd640 1 -- 172.21.15.84:0/2850392362 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3de0004040 con 0x559b409d6f00 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3dceffd640 1 -- 172.21.15.84:0/2850392362 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3de402f840 con 0x559b409d6f00 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3df8e61740 10 client.15729 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:11.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3df8e61740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3df8e61740 20 client.15729 populate_metadata read hostname 'smithi084' 2024-04-26T18:30:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2850392362 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559b40a59160 0x559b40a5b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3df8e61740 4 client.15729 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:30:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.710+0000 7f3df8e61740 1 -- 172.21.15.84:0/2850392362 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559b40944ee0 con 0x559b40a59160 2024-04-26T18:30:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.711+0000 7f3def7fe640 1 --2- 172.21.15.84:0/2850392362 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559b40a59160 0x559b40a5b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.711+0000 7f3dceffd640 10 client.15729 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:11.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.711+0000 7f3def7fe640 1 --2- 172.21.15.84:0/2850392362 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559b40a59160 0x559b40a5b580 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfa08c7fac5140d1 server_cookie=e178e6858aecac81 in_seq=0 out_seq=0 2024-04-26T18:30:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.717+0000 7f3dceffd640 1 -- 172.21.15.84:0/2850392362 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559b40944ee0 con 0x559b40a59160 2024-04-26T18:30:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.717+0000 7f3dceffd640 10 client.15729 handle_command_reply: tid=0 2024-04-26T18:30:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.717+0000 7f3df8e61740 1 client.15729 shutdown 2024-04-26T18:30:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.717+0000 7f3df8e61740 1 -- 172.21.15.84:0/2850392362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 msgr2=0x559b409dc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.717+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2850392362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409dc090 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3de402ff00 tx=0x7f3de4011750 comp rx=0 tx=0).stop 2024-04-26T18:30:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.717+0000 7f3df8e61740 1 -- 172.21.15.84:0/2850392362 shutdown_connections 2024-04-26T18:30:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.717+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2850392362 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559b40a59160 0x559b40a5b580 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.717+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2850392362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559b409e20e0 0x559b40a57320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.717+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2850392362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559b409d7820 0x559b409dc5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.717+0000 7f3dceffd640 0 client.15729 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.717+0000 7f3df8e61740 1 --2- 172.21.15.84:0/2850392362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559b409d6f00 0x559b409dc090 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.717+0000 7f3df8e61740 1 -- 172.21.15.84:0/2850392362 >> 172.21.15.84:0/2850392362 conn(0x559b409d3410 msgr2=0x559b409defe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df8e61740 1 -- 172.21.15.84:0/2850392362 shutdown_connections 2024-04-26T18:30:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df8e61740 1 -- 172.21.15.84:0/2850392362 wait complete. 2024-04-26T18:30:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df8e61740 20 client.15729 trim_cache size 0 max 16384 2024-04-26T18:30:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:30:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:30:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:30:11.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:30:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df614d640 1 -- 172.21.15.84:0/1631744412 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3dd80a3dc0 msgr2=0x7f3dd80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df614d640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3dd80a3dc0 0x7f3dd80a6280 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3de8098fa0 tx=0x7f3de806e580 comp rx=0 tx=0).stop 2024-04-26T18:30:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df614d640 1 -- 172.21.15.84:0/1631744412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df010cfb0 msgr2=0x7f3df01022b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df614d640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df010cfb0 0x7f3df01022b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f3de800cab0 tx=0x7f3de800cf70 comp rx=0 tx=0).stop 2024-04-26T18:30:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df614d640 1 -- 172.21.15.84:0/1631744412 shutdown_connections 2024-04-26T18:30:11.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df614d640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3dd80a3dc0 0x7f3dd80a6280 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df614d640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3df010cfb0 0x7f3df01022b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df614d640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3df0104500 0x7f3df0101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df614d640 1 --2- 172.21.15.84:0/1631744412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3df0103be0 0x7f3df0101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df614d640 1 -- 172.21.15.84:0/1631744412 >> 172.21.15.84:0/1631744412 conn(0x7f3df0077bb0 msgr2=0x7f3df00fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df614d640 1 -- 172.21.15.84:0/1631744412 shutdown_connections 2024-04-26T18:30:11.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:11.718+0000 7f3df614d640 1 -- 172.21.15.84:0/1631744412 wait complete. 2024-04-26T18:30:12.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:12 smithi103 ceph-mon[32727]: pgmap v70: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T18:30:12.888 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:30:12.915 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:12 smithi084 ceph-mon[29141]: pgmap v70: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T18:30:12.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:12 smithi174 ceph-mon[32705]: pgmap v70: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-04-26T18:30:13.144 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:30:14.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:14 smithi103 ceph-mon[32727]: pgmap v71: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-26T18:30:14.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:14 smithi174 ceph-mon[32705]: pgmap v71: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-26T18:30:15.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:14 smithi084 ceph-mon[29141]: pgmap v71: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 28 op/s 2024-04-26T18:30:16.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:16 smithi103 ceph-mon[32727]: pgmap v72: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-26T18:30:16.905 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:16 smithi084 ceph-mon[29141]: pgmap v72: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-26T18:30:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:16 smithi174 ceph-mon[32705]: pgmap v72: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-26T18:30:17.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.281+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3457911463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e94101a10 msgr2=0x7f6e94101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.281+0000 7f6e9ba30700 1 --2- 172.21.15.84:0/3457911463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e94101a10 0x7f6e94101e90 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f6e84009990 tx=0x7f6e84009ca0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.282+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3457911463 shutdown_connections 2024-04-26T18:30:17.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.282+0000 7f6e9ba30700 1 --2- 172.21.15.84:0/3457911463 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e941024b0 0x7f6e941121a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.282+0000 7f6e9ba30700 1 --2- 172.21.15.84:0/3457911463 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e94104330 0x7f6e94104710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.282+0000 7f6e9ba30700 1 --2- 172.21.15.84:0/3457911463 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e94101a10 0x7f6e94101e90 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.282+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3457911463 >> 172.21.15.84:0/3457911463 conn(0x7f6e94078990 msgr2=0x7f6e940ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:17.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.282+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3457911463 shutdown_connections 2024-04-26T18:30:17.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.282+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3457911463 wait complete. 2024-04-26T18:30:17.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.282+0000 7f6e9ba30700 1 Processor -- start 2024-04-26T18:30:17.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e9ba30700 1 -- start start 2024-04-26T18:30:17.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e9ba30700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e94101a10 0x7f6e941a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e9ba30700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e941024b0 0x7f6e941a1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e9ba30700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e94104330 0x7f6e9419bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e9ba30700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e941157e0 con 0x7f6e941024b0 2024-04-26T18:30:17.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e9ba30700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e94115660 con 0x7f6e94101a10 2024-04-26T18:30:17.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e9ba30700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e94115960 con 0x7f6e94104330 2024-04-26T18:30:17.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e98fcb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e941024b0 0x7f6e941a1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e98fcb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e941024b0 0x7f6e941a1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34518/0 (socket says 172.21.15.84:34518) 2024-04-26T18:30:17.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e98fcb700 1 -- 172.21.15.84:0/3887708146 learned_addr learned my addr 172.21.15.84:0/3887708146 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:17.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e99fcd700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e94104330 0x7f6e9419bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e997cc700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e94101a10 0x7f6e941a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e98fcb700 1 -- 172.21.15.84:0/3887708146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e94104330 msgr2=0x7f6e9419bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e98fcb700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e94104330 0x7f6e9419bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e98fcb700 1 -- 172.21.15.84:0/3887708146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e94101a10 msgr2=0x7f6e941a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e98fcb700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e94101a10 0x7f6e941a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e98fcb700 1 -- 172.21.15.84:0/3887708146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e9419c260 con 0x7f6e941024b0 2024-04-26T18:30:17.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e997cc700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e94101a10 0x7f6e941a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:30:17.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e99fcd700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e94104330 0x7f6e9419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:17.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.283+0000 7f6e98fcb700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e941024b0 0x7f6e941a1e80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6e84000c00 tx=0x7f6e840047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:17.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.284+0000 7f6e8a7fc700 1 -- 172.21.15.84:0/3887708146 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84027070 con 0x7f6e941024b0 2024-04-26T18:30:17.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.284+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3887708146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e9419c4f0 con 0x7f6e941024b0 2024-04-26T18:30:17.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.284+0000 7f6e8a7fc700 1 -- 172.21.15.84:0/3887708146 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e84003fa0 con 0x7f6e941024b0 2024-04-26T18:30:17.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.284+0000 7f6e8a7fc700 1 -- 172.21.15.84:0/3887708146 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84022db0 con 0x7f6e941024b0 2024-04-26T18:30:17.287 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.284+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3887708146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e941a8810 con 0x7f6e941024b0 2024-04-26T18:30:17.288 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.285+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3887708146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e94102a20 con 0x7f6e941024b0 2024-04-26T18:30:17.288 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.286+0000 7f6e8a7fc700 1 -- 172.21.15.84:0/3887708146 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f6e8402a070 con 0x7f6e941024b0 2024-04-26T18:30:17.288 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.287+0000 7f6e8a7fc700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e800a3db0 0x7f6e800a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:17.288 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.287+0000 7f6e8a7fc700 1 -- 172.21.15.84:0/3887708146 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f6e840d1970 con 0x7f6e941024b0 2024-04-26T18:30:17.288 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.287+0000 7f6e997cc700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e800a3db0 0x7f6e800a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:17.288 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.288+0000 7f6e997cc700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e800a3db0 0x7f6e800a6270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e90005fd0 tx=0x7f6e90005d40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:17.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.291+0000 7f6e8a7fc700 1 -- 172.21.15.84:0/3887708146 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e84013070 con 0x7f6e941024b0 2024-04-26T18:30:17.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.558+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3887708146 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e9419cff0 con 0x7f6e941024b0 2024-04-26T18:30:17.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.558+0000 7f6e8a7fc700 1 -- 172.21.15.84:0/3887708146 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f6e8409e030 con 0x7f6e941024b0 2024-04-26T18:30:17.559 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:30:17.559 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:30:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.561+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3887708146 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e800a3db0 msgr2=0x7f6e800a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.561+0000 7f6e9ba30700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e800a3db0 0x7f6e800a6270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6e90005fd0 tx=0x7f6e90005d40 comp rx=0 tx=0).stop 2024-04-26T18:30:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.561+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3887708146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e941024b0 msgr2=0x7f6e941a1e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.561+0000 7f6e9ba30700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e941024b0 0x7f6e941a1e80 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f6e84000c00 tx=0x7f6e840047b0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.561+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3887708146 shutdown_connections 2024-04-26T18:30:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.561+0000 7f6e9ba30700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e800a3db0 0x7f6e800a6270 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.561+0000 7f6e9ba30700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e94104330 0x7f6e9419bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.561+0000 7f6e9ba30700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e94101a10 0x7f6e941a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.561+0000 7f6e9ba30700 1 --2- 172.21.15.84:0/3887708146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e941024b0 0x7f6e941a1e80 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:17.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.561+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3887708146 >> 172.21.15.84:0/3887708146 conn(0x7f6e94078990 msgr2=0x7f6e940ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:17.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.561+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3887708146 shutdown_connections 2024-04-26T18:30:17.562 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:17.561+0000 7f6e9ba30700 1 -- 172.21.15.84:0/3887708146 wait complete. 2024-04-26T18:30:17.562 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:30:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:17 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3887708146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:30:17.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:17 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3887708146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:30:17.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:17 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3887708146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:30:18.343 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id 4ccfe41a --release --await' 2024-04-26T18:30:18.344 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db --set-id 4ccfe41a --release --await 2024-04-26T18:30:18.627 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:30:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:18 smithi103 ceph-mon[32727]: pgmap v73: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-26T18:30:18.923 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:18 smithi084 ceph-mon[29141]: pgmap v73: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-26T18:30:18.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:18 smithi174 ceph-mon[32705]: pgmap v73: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-26T18:30:20.354 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.352+0000 7f1dd144b700 1 -- 172.21.15.84:0/3820115655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dcc103e30 msgr2=0x7f1dcc110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.352+0000 7f1dd144b700 1 --2- 172.21.15.84:0/3820115655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dcc103e30 0x7f1dcc110290 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f1dc400b0a0 tx=0x7f1dc400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.352+0000 7f1dd144b700 1 -- 172.21.15.84:0/3820115655 shutdown_connections 2024-04-26T18:30:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.352+0000 7f1dd144b700 1 --2- 172.21.15.84:0/3820115655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dcc103e30 0x7f1dcc110290 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.352+0000 7f1dd144b700 1 --2- 172.21.15.84:0/3820115655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dcc1094c0 0x7f1dcc1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.352+0000 7f1dd144b700 1 --2- 172.21.15.84:0/3820115655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dcc103470 0x7f1dcc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.352+0000 7f1dd144b700 1 -- 172.21.15.84:0/3820115655 >> 172.21.15.84:0/3820115655 conn(0x7f1dcc07a580 msgr2=0x7f1dcc07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.352+0000 7f1dd144b700 1 -- 172.21.15.84:0/3820115655 shutdown_connections 2024-04-26T18:30:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.353+0000 7f1dd144b700 1 -- 172.21.15.84:0/3820115655 wait complete. 2024-04-26T18:30:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.353+0000 7f1dd144b700 1 Processor -- start 2024-04-26T18:30:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.353+0000 7f1dd144b700 1 -- start start 2024-04-26T18:30:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.353+0000 7f1dd144b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dcc103470 0x7f1dcc19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.353+0000 7f1dd144b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dcc103e30 0x7f1dcc19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.355 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.353+0000 7f1dd144b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dcc1094c0 0x7f1dcc199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.353+0000 7f1dd144b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc113640 con 0x7f1dcc1094c0 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.353+0000 7f1dd144b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc1134c0 con 0x7f1dcc103470 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.353+0000 7f1dd144b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc1137c0 con 0x7f1dcc103e30 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dcb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dcc1094c0 0x7f1dcc199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dcb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dcc1094c0 0x7f1dcc199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34544/0 (socket says 172.21.15.84:34544) 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dcb7fe700 1 -- 172.21.15.84:0/4261650303 learned_addr learned my addr 172.21.15.84:0/4261650303 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dcc103470 0x7f1dcc19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dca7fc700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dcc103e30 0x7f1dcc19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dcb7fe700 1 -- 172.21.15.84:0/4261650303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dcc103e30 msgr2=0x7f1dcc19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dcb7fe700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dcc103e30 0x7f1dcc19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dcb7fe700 1 -- 172.21.15.84:0/4261650303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dcc103470 msgr2=0x7f1dcc19f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dcb7fe700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dcc103470 0x7f1dcc19f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dcb7fe700 1 -- 172.21.15.84:0/4261650303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1dcc19a0c0 con 0x7f1dcc1094c0 2024-04-26T18:30:20.356 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dcc103470 0x7f1dcc19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:20.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dcb7fe700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dcc1094c0 0x7f1dcc199960 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40062a0 tx=0x7f1dc4004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:20.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1db3fff700 1 -- 172.21.15.84:0/4261650303 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4021070 con 0x7f1dcc1094c0 2024-04-26T18:30:20.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1db3fff700 1 -- 172.21.15.84:0/4261650303 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dc4003c00 con 0x7f1dcc1094c0 2024-04-26T18:30:20.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dd144b700 1 -- 172.21.15.84:0/4261650303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1dcc19a350 con 0x7f1dcc1094c0 2024-04-26T18:30:20.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1db3fff700 1 -- 172.21.15.84:0/4261650303 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dc4004830 con 0x7f1dcc1094c0 2024-04-26T18:30:20.357 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.354+0000 7f1dd144b700 1 -- 172.21.15.84:0/4261650303 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1dcc1a6620 con 0x7f1dcc1094c0 2024-04-26T18:30:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.356+0000 7f1db3fff700 1 -- 172.21.15.84:0/4261650303 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1dc4007a00 con 0x7f1dcc1094c0 2024-04-26T18:30:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.357+0000 7f1db3fff700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1db40a3e30 0x7f1db40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.357+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1db40a3e30 0x7f1db40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.357+0000 7f1db3fff700 1 -- 172.21.15.84:0/4261650303 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f1dc400f020 con 0x7f1dcc1094c0 2024-04-26T18:30:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.357+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1db40a3e30 0x7f1db40a62f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc101aa0 tx=0x7f1dbc009260 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1ddbcddb80 1 Processor -- start 2024-04-26T18:30:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1ddbcddb80 1 -- start start 2024-04-26T18:30:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1ddbcddb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f8301fd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1ddbcddb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8301fdac0 0x55f830208430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1ddbcddb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f830208970 0x55f83020ad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1ddbcddb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f830150840 con 0x55f8301fd1a0 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1ddbcddb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f8300ef6e0 con 0x55f8301fdac0 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1ddbcddb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f83014c800 con 0x55f830208970 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1dcaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f8301fd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1dcaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f8301fd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34548/0 (socket says 172.21.15.84:34548) 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1dcaffd700 1 -- 172.21.15.84:0/3513544277 learned_addr learned my addr 172.21.15.84:0/3513544277 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1dcb7fe700 1 --2- 172.21.15.84:0/3513544277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f830208970 0x55f83020ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1dca7fc700 1 --2- 172.21.15.84:0/3513544277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8301fdac0 0x55f830208430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1dcaffd700 1 -- 172.21.15.84:0/3513544277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f830208970 msgr2=0x55f83020ad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/3513544277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f830208970 0x55f83020ad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1dcaffd700 1 -- 172.21.15.84:0/3513544277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8301fdac0 msgr2=0x55f830208430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/3513544277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8301fdac0 0x55f830208430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1dcaffd700 1 -- 172.21.15.84:0/3513544277 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f83009d9d0 con 0x55f8301fd1a0 2024-04-26T18:30:20.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.358+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/3513544277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f8301fd580 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc011ed0 tx=0x7f1dbc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb3bbe0831b95ef server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.359+0000 7f1db17fa700 1 -- 172.21.15.84:0/3513544277 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc012dc0 con 0x55f8301fd1a0 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.359+0000 7f1db17fa700 1 -- 172.21.15.84:0/3513544277 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dbc01f8d0 con 0x55f8301fd1a0 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.359+0000 7f1db17fa700 1 -- 172.21.15.84:0/3513544277 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc01d710 con 0x55f8301fd1a0 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.359+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/3513544277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 msgr2=0x55f8301fd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.359+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/3513544277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f8301fd580 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc011ed0 tx=0x7f1dbc014000 comp rx=0 tx=0).stop 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.359+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/3513544277 shutdown_connections 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.359+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/3513544277 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f8301fd580 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.359+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/3513544277 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f830208970 0x55f83020ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.359+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/3513544277 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8301fdac0 0x55f830208430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.359+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/3513544277 >> 172.21.15.84:0/3513544277 conn(0x55f83012ae40 msgr2=0x55f8301f9690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.359+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/3513544277 shutdown_connections 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.359+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/3513544277 wait complete. 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1ddbcddb80 1 -- start start 2024-04-26T18:30:20.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1ddbcddb80 1 Processor -- start 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1ddbcddb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f8302080b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1ddbcddb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8301fdac0 0x55f8302053b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1ddbcddb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f830208970 0x55f8302058f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1ddbcddb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f83020b540 con 0x55f8301fd1a0 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1ddbcddb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f8300b6a50 con 0x55f830208970 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1ddbcddb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f83020b6c0 con 0x55f8301fdac0 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1ddbcddb80 10 client.0 _ll_register_callbacks cb 0x55f8300e4680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1dcaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f8302080b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1dcaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f8302080b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34564/0 (socket says 172.21.15.84:34564) 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1dcaffd700 1 -- 172.21.15.84:0/1960753197 learned_addr learned my addr 172.21.15.84:0/1960753197 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1dcb7fe700 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f830208970 0x55f8302058f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1dca7fc700 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8301fdac0 0x55f8302053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1dcaffd700 1 -- 172.21.15.84:0/1960753197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8301fdac0 msgr2=0x55f8302053b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:20.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8301fdac0 0x55f8302053b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1dcaffd700 1 -- 172.21.15.84:0/1960753197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f830208970 msgr2=0x55f8302058f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f830208970 0x55f8302058f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1dcaffd700 1 -- 172.21.15.84:0/1960753197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f830205f60 con 0x55f8301fd1a0 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.360+0000 7f1dcb7fe700 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f830208970 0x55f8302058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f8302080b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc021040 tx=0x7f1dbc01f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4147aad38b22e532 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1daeff5700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1daeff5700 1 -- 172.21.15.84:0/1960753197 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc037060 con 0x55f8301fd1a0 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1daeff5700 1 -- 172.21.15.84:0/1960753197 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f1dbc01fc90 con 0x55f8301fd1a0 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/1960753197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dbc01f8d0 con 0x55f8301fd1a0 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1daeff5700 1 -- 172.21.15.84:0/1960753197 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1dbc005040 con 0x55f8301fd1a0 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1ddbcddb80 10 client.15747 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/1960753197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f830215f90 con 0x55f8301fd1a0 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1daeff5700 1 -- 172.21.15.84:0/1960753197 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dbc01f8d0 con 0x55f8301fd1a0 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1daeff5700 1 -- 172.21.15.84:0/1960753197 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1dbc032410 con 0x55f8301fd1a0 2024-04-26T18:30:20.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1ddbcddb80 10 client.15747 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:20.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1ddbcddb80 10 client.15747 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:30:20.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1ddbcddb80 20 client.15747 populate_metadata read hostname 'smithi084' 2024-04-26T18:30:20.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f830218510 0x55f83021a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1ddbcddb80 4 client.15747 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:30:20.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.361+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/1960753197 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f83015e220 con 0x55f830218510 2024-04-26T18:30:20.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.362+0000 7f1dca7fc700 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f830218510 0x55f83021a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.362+0000 7f1dca7fc700 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f830218510 0x55f83021a900 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f29f38111bbab2eb server_cookie=f21c08075439f326 in_seq=0 out_seq=0 2024-04-26T18:30:20.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.362+0000 7f1daeff5700 10 client.15747 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:20.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1daeff5700 1 -- 172.21.15.84:0/1960753197 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f83015e220 con 0x55f830218510 2024-04-26T18:30:20.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1daeff5700 10 client.15747 handle_command_reply: tid=0 2024-04-26T18:30:20.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1ddbcddb80 1 client.15747 shutdown 2024-04-26T18:30:20.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/1960753197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 msgr2=0x55f8302080b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f8302080b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc021040 tx=0x7f1dbc01f6a0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/1960753197 shutdown_connections 2024-04-26T18:30:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f830218510 0x55f83021a900 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f8302080b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f830208970 0x55f8302058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1daeff5700 0 client.15747 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/1960753197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8301fdac0 0x55f8302053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/1960753197 >> 172.21.15.84:0/1960753197 conn(0x55f83012ae40 msgr2=0x55f8301f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/1960753197 shutdown_connections 2024-04-26T18:30:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/1960753197 wait complete. 2024-04-26T18:30:20.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.363+0000 7f1ddbcddb80 20 client.15747 trim_cache size 0 max 16384 2024-04-26T18:30:20.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1ddbcddb80 1 Processor -- start 2024-04-26T18:30:20.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1ddbcddb80 1 -- start start 2024-04-26T18:30:20.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1ddbcddb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f830200850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1ddbcddb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8301fdac0 0x55f830210d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1ddbcddb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f830208970 0x55f830214440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1ddbcddb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc1add50 con 0x55f8301fd1a0 2024-04-26T18:30:20.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1ddbcddb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc1af890 con 0x55f830208970 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1ddbcddb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc1ae420 con 0x55f8301fdac0 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1dcaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f830200850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1dcaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f830200850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34574/0 (socket says 172.21.15.84:34574) 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1dcaffd700 1 -- 172.21.15.84:0/3929995204 learned_addr learned my addr 172.21.15.84:0/3929995204 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1dcb7fe700 1 --2- 172.21.15.84:0/3929995204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f830208970 0x55f830214440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1dca7fc700 1 --2- 172.21.15.84:0/3929995204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8301fdac0 0x55f830210d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1dcaffd700 1 -- 172.21.15.84:0/3929995204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8301fdac0 msgr2=0x55f830210d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.382+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/3929995204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8301fdac0 0x55f830210d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1dcaffd700 1 -- 172.21.15.84:0/3929995204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f830208970 msgr2=0x55f830214440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/3929995204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f830208970 0x55f830214440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1dcaffd700 1 -- 172.21.15.84:0/3929995204 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8301f0e90 con 0x55f8301fd1a0 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1dcb7fe700 1 --2- 172.21.15.84:0/3929995204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f830208970 0x55f830214440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1dca7fc700 1 --2- 172.21.15.84:0/3929995204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8301fdac0 0x55f830210d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:30:20.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/3929995204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f830200850 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc012a90 tx=0x7f1dbc009d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0c372139be2bd3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:20.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1db0ff9700 1 -- 172.21.15.84:0/3929995204 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc037070 con 0x55f8301fd1a0 2024-04-26T18:30:20.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1db0ff9700 1 -- 172.21.15.84:0/3929995204 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dbc01f8c0 con 0x55f8301fd1a0 2024-04-26T18:30:20.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1db0ff9700 1 -- 172.21.15.84:0/3929995204 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc0046d0 con 0x55f8301fd1a0 2024-04-26T18:30:20.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/3929995204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 msgr2=0x55f830200850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:20.385 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/3929995204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f830200850 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc012a90 tx=0x7f1dbc009d60 comp rx=0 tx=0).stop 2024-04-26T18:30:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/3929995204 shutdown_connections 2024-04-26T18:30:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/3929995204 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f830200850 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/3929995204 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f830208970 0x55f830214440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/3929995204 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f8301fdac0 0x55f830210d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.383+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/3929995204 >> 172.21.15.84:0/3929995204 conn(0x55f83012ae40 msgr2=0x55f830204440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/3929995204 shutdown_connections 2024-04-26T18:30:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/3929995204 wait complete. 2024-04-26T18:30:20.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1ddbcddb80 1 -- start start 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1ddbcddb80 1 Processor -- start 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1ddbcddb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f830202ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1ddbcddb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8301fdac0 0x55f830213b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1ddbcddb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f830208970 0x55f830214070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1ddbcddb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc1af1c0 con 0x55f8301fd1a0 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1ddbcddb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc1aeaf0 con 0x55f8301fdac0 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1ddbcddb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1dcc1abcf0 con 0x55f830208970 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1ddbcddb80 10 client.0 _ll_register_callbacks cb 0x55f8300e4680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1dcaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f830202ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1dcaffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f830202ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34584/0 (socket says 172.21.15.84:34584) 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1dcaffd700 1 -- 172.21.15.84:0/2836684266 learned_addr learned my addr 172.21.15.84:0/2836684266 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1dca7fc700 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8301fdac0 0x55f830213b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.384+0000 7f1dcb7fe700 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f830208970 0x55f830214070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1dcaffd700 1 -- 172.21.15.84:0/2836684266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f830208970 msgr2=0x55f830214070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:20.389 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f830208970 0x55f830214070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1dcaffd700 1 -- 172.21.15.84:0/2836684266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8301fdac0 msgr2=0x55f830213b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8301fdac0 0x55f830213b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1dcaffd700 1 -- 172.21.15.84:0/2836684266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f830214860 con 0x55f8301fd1a0 2024-04-26T18:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1dca7fc700 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8301fdac0 0x55f830213b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1dcaffd700 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f830202ef0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009a30 tx=0x7f1dbc032710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dabd36e70dcd84a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1daeff5700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1daeff5700 1 -- 172.21.15.84:0/2836684266 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dbc037040 con 0x55f8301fd1a0 2024-04-26T18:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1daeff5700 1 -- 172.21.15.84:0/2836684266 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f1dbc0053f0 con 0x55f8301fd1a0 2024-04-26T18:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/2836684266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f8301582c0 con 0x55f8301fd1a0 2024-04-26T18:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1daeff5700 1 -- 172.21.15.84:0/2836684266 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1dbc053050 con 0x55f8301fd1a0 2024-04-26T18:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1ddbcddb80 10 client.15753 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/2836684266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f8302826f0 con 0x55f8301fd1a0 2024-04-26T18:30:20.390 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1daeff5700 1 -- 172.21.15.84:0/2836684266 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f8301582c0 con 0x55f8301fd1a0 2024-04-26T18:30:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1daeff5700 1 -- 172.21.15.84:0/2836684266 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1dbc043070 con 0x55f8301fd1a0 2024-04-26T18:30:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1ddbcddb80 10 client.15753 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1ddbcddb80 10 client.15753 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:30:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.385+0000 7f1ddbcddb80 20 client.15753 populate_metadata read hostname 'smithi084' 2024-04-26T18:30:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.386+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f830282ce0 0x55f830285100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.386+0000 7f1ddbcddb80 4 client.15753 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "4ccfe41a", "release": true, "await": true}] 2024-04-26T18:30:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.386+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/2836684266 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "set_id": "4ccfe41a", "release": true, "await": true}) -- 0x55f82ffd0350 con 0x55f830282ce0 2024-04-26T18:30:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.386+0000 7f1dca7fc700 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f830282ce0 0x55f830285100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.386+0000 7f1daeff5700 10 client.15753 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:20.391 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.386+0000 7f1dca7fc700 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f830282ce0 0x55f830285100 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a20eba6361a53d09 server_cookie=e767000c98d0289c in_seq=0 out_seq=0 2024-04-26T18:30:20.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1daeff5700 1 -- 172.21.15.84:0/2836684266 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55f82ffd0350 con 0x55f830282ce0 2024-04-26T18:30:20.392 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1daeff5700 10 client.15753 handle_command_reply: tid=0 2024-04-26T18:30:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1ddbcddb80 1 client.15753 shutdown 2024-04-26T18:30:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/2836684266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 msgr2=0x55f830202ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f830202ef0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009a30 tx=0x7f1dbc032710 comp rx=0 tx=0).stop 2024-04-26T18:30:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/2836684266 shutdown_connections 2024-04-26T18:30:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f830282ce0 0x55f830285100 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f8301fd1a0 0x55f830202ef0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f830208970 0x55f830214070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1ddbcddb80 1 --2- 172.21.15.84:0/2836684266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f8301fdac0 0x55f830213b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/2836684266 >> 172.21.15.84:0/2836684266 conn(0x55f83012ae40 msgr2=0x55f8301ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1daeff5700 0 client.15753 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/2836684266 shutdown_connections 2024-04-26T18:30:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1ddbcddb80 1 -- 172.21.15.84:0/2836684266 wait complete. 2024-04-26T18:30:20.393 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.390+0000 7f1ddbcddb80 20 client.15753 trim_cache size 0 max 16384 2024-04-26T18:30:20.394 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:30:20.394 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:30:20.394 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:30:20.394 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 8, 2024-04-26T18:30:20.394 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:30:20.394 INFO:teuthology.orchestra.run.smithi084.stdout: "4ccfe41a": { 2024-04-26T18:30:20.394 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 8, 2024-04-26T18:30:20.394 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:30:20.394 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:30:20.394 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:30:20.394 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 90.0, 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:30:20.395 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:30:20.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.391+0000 7f1dd144b700 1 -- 172.21.15.84:0/4261650303 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1db40a3e30 msgr2=0x7f1db40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:20.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.391+0000 7f1dd144b700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1db40a3e30 0x7f1db40a62f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc101aa0 tx=0x7f1dbc009260 comp rx=0 tx=0).stop 2024-04-26T18:30:20.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.391+0000 7f1dd144b700 1 -- 172.21.15.84:0/4261650303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dcc1094c0 msgr2=0x7f1dcc199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:20.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.391+0000 7f1dd144b700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dcc1094c0 0x7f1dcc199960 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f1dc40062a0 tx=0x7f1dc4004d30 comp rx=0 tx=0).stop 2024-04-26T18:30:20.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.391+0000 7f1dd144b700 1 -- 172.21.15.84:0/4261650303 shutdown_connections 2024-04-26T18:30:20.396 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.391+0000 7f1dd144b700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1db40a3e30 0x7f1db40a62f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.391+0000 7f1dd144b700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1dcc1094c0 0x7f1dcc199960 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.391+0000 7f1dd144b700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1dcc103470 0x7f1dcc19f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.391+0000 7f1dd144b700 1 --2- 172.21.15.84:0/4261650303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1dcc103e30 0x7f1dcc19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:20.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.391+0000 7f1dd144b700 1 -- 172.21.15.84:0/4261650303 >> 172.21.15.84:0/4261650303 conn(0x7f1dcc07a580 msgr2=0x7f1dcc1021c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:20.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.391+0000 7f1dd144b700 1 -- 172.21.15.84:0/4261650303 shutdown_connections 2024-04-26T18:30:20.397 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:30:20.391+0000 7f1dd144b700 1 -- 172.21.15.84:0/4261650303 wait complete. 2024-04-26T18:30:20.397 INFO:tasks.workunit.client.0.smithi084.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-26T18:30:20.397 INFO:tasks.workunit.client.0.smithi084.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-26T18:30:20.398 INFO:tasks.workunit.client.0.smithi084.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-26T18:30:20.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:20 smithi084 ceph-mon[29141]: pgmap v74: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-26T18:30:20.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:20 smithi103 ceph-mon[32727]: pgmap v74: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-26T18:30:20.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:20 smithi174 ceph-mon[32705]: pgmap v74: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-26T18:30:21.096 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '4ccfe41a', seconds elapsed: 44.9 2024-04-26T18:30:22.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:22 smithi103 ceph-mon[32727]: pgmap v75: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-26T18:30:22.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:22 smithi174 ceph-mon[32705]: pgmap v75: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-26T18:30:23.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:22 smithi084 ceph-mon[29141]: pgmap v75: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 9 op/s 2024-04-26T18:30:24.816 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.814+0000 7f4c85324640 1 -- 172.21.15.84:0/2633704987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c8010cf00 msgr2=0x7f4c8010f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.814+0000 7f4c85324640 1 --2- 172.21.15.84:0/2633704987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c8010cf00 0x7f4c8010f3a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f4c78009ca0 tx=0x7f4c7802a950 comp rx=0 tx=0).stop 2024-04-26T18:30:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.814+0000 7f4c85324640 1 -- 172.21.15.84:0/2633704987 shutdown_connections 2024-04-26T18:30:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.814+0000 7f4c85324640 1 --2- 172.21.15.84:0/2633704987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c8010cf00 0x7f4c8010f3a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.814+0000 7f4c85324640 1 --2- 172.21.15.84:0/2633704987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c80104400 0x7f4c8010c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.814+0000 7f4c85324640 1 --2- 172.21.15.84:0/2633704987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c80103ae0 0x7f4c80103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.814+0000 7f4c85324640 1 -- 172.21.15.84:0/2633704987 >> 172.21.15.84:0/2633704987 conn(0x7f4c800fd300 msgr2=0x7f4c800ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.814+0000 7f4c85324640 1 -- 172.21.15.84:0/2633704987 shutdown_connections 2024-04-26T18:30:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.814+0000 7f4c85324640 1 -- 172.21.15.84:0/2633704987 wait complete. 2024-04-26T18:30:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.815+0000 7f4c85324640 1 Processor -- start 2024-04-26T18:30:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.815+0000 7f4c85324640 1 -- start start 2024-04-26T18:30:24.817 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.815+0000 7f4c85324640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c80103ae0 0x7f4c80101730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.815+0000 7f4c85324640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c80104400 0x7f4c80101c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.815+0000 7f4c85324640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c8010cf00 0x7f4c801021b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.815+0000 7f4c85324640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80112bb0 con 0x7f4c8010cf00 2024-04-26T18:30:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.815+0000 7f4c85324640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80112a30 con 0x7f4c80103ae0 2024-04-26T18:30:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.815+0000 7f4c85324640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c80112d30 con 0x7f4c80104400 2024-04-26T18:30:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.815+0000 7f4c7f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c8010cf00 0x7f4c801021b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.815+0000 7f4c7f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c8010cf00 0x7f4c801021b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34600/0 (socket says 172.21.15.84:34600) 2024-04-26T18:30:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.815+0000 7f4c7f7fe640 1 -- 172.21.15.84:0/3774396114 learned_addr learned my addr 172.21.15.84:0/3774396114 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c7effd640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c80103ae0 0x7f4c80101730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c80104400 0x7f4c80101c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c7f7fe640 1 -- 172.21.15.84:0/3774396114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c80104400 msgr2=0x7f4c80101c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:24.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c7f7fe640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c80104400 0x7f4c80101c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c7f7fe640 1 -- 172.21.15.84:0/3774396114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c80103ae0 msgr2=0x7f4c80101730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c7f7fe640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c80103ae0 0x7f4c80101730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c7f7fe640 1 -- 172.21.15.84:0/3774396114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c801a74d0 con 0x7f4c8010cf00 2024-04-26T18:30:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c80104400 0x7f4c80101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c7effd640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c80103ae0 0x7f4c80101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c7f7fe640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c8010cf00 0x7f4c801021b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4c7802ae70 tx=0x7f4c7802f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c77fff640 1 -- 172.21.15.84:0/3774396114 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c78034040 con 0x7f4c8010cf00 2024-04-26T18:30:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c77fff640 1 -- 172.21.15.84:0/3774396114 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c780308f0 con 0x7f4c8010cf00 2024-04-26T18:30:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c85324640 1 -- 172.21.15.84:0/3774396114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c801a7760 con 0x7f4c8010cf00 2024-04-26T18:30:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c77fff640 1 -- 172.21.15.84:0/3774396114 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c780047f0 con 0x7f4c8010cf00 2024-04-26T18:30:24.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.816+0000 7f4c85324640 1 -- 172.21.15.84:0/3774396114 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c801a7ca0 con 0x7f4c8010cf00 2024-04-26T18:30:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.819+0000 7f4c77fff640 1 -- 172.21.15.84:0/3774396114 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4c78037020 con 0x7f4c8010cf00 2024-04-26T18:30:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.819+0000 7f4c88041740 1 Processor -- start 2024-04-26T18:30:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.819+0000 7f4c88041740 1 -- start start 2024-04-26T18:30:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c88041740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12df00 0x563b1e12e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c88041740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e12e820 0x563b1e138ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c88041740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e1390e0 0x563b1e13b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.826 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c88041740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563b1e094460 con 0x563b1e12df00 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c88041740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563b1e04e590 con 0x563b1e12e820 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c88041740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b1e090420 con 0x563b1e1390e0 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c7e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12df00 0x563b1e12e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c7e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12df00 0x563b1e12e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34606/0 (socket says 172.21.15.84:34606) 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c7e7fc640 1 -- 172.21.15.84:0/3779035773 learned_addr learned my addr 172.21.15.84:0/3779035773 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c77fff640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c6c0a3e70 0x7f4c6c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c7effd640 1 --2- 172.21.15.84:0/3779035773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e12e820 0x563b1e138ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c7f7fe640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c6c0a3e70 0x7f4c6c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/3779035773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e1390e0 0x563b1e13b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c7e7fc640 1 -- 172.21.15.84:0/3779035773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e1390e0 msgr2=0x563b1e13b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/3779035773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e1390e0 0x563b1e13b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c7e7fc640 1 -- 172.21.15.84:0/3779035773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e12e820 msgr2=0x563b1e138ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/3779035773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e12e820 0x563b1e138ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.827 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c7e7fc640 1 -- 172.21.15.84:0/3779035773 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b1e05b0d0 con 0x563b1e12df00 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c7effd640 1 --2- 172.21.15.84:0/3779035773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e12e820 0x563b1e138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c77fff640 1 -- 172.21.15.84:0/3774396114 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f4c780f16f0 con 0x7f4c8010cf00 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.820+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/3779035773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12df00 0x563b1e12e2e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800be90 tx=0x7f4c6800b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80425e61897d7bca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.821+0000 7f4c7f7fe640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c6c0a3e70 0x7f4c6c0a6330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4c7804d000 tx=0x7f4c780e88a0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.821+0000 7f4c757fa640 1 -- 172.21.15.84:0/3779035773 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c680110d0 con 0x563b1e12df00 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.821+0000 7f4c757fa640 1 -- 172.21.15.84:0/3779035773 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c68017030 con 0x563b1e12df00 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.821+0000 7f4c757fa640 1 -- 172.21.15.84:0/3779035773 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c68004e40 con 0x563b1e12df00 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.821+0000 7f4c88041740 1 -- 172.21.15.84:0/3779035773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12df00 msgr2=0x563b1e12e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.821+0000 7f4c88041740 1 --2- 172.21.15.84:0/3779035773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12df00 0x563b1e12e2e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800be90 tx=0x7f4c6800b640 comp rx=0 tx=0).stop 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 -- 172.21.15.84:0/3779035773 shutdown_connections 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 --2- 172.21.15.84:0/3779035773 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e1390e0 0x563b1e13b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 --2- 172.21.15.84:0/3779035773 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e12e820 0x563b1e138ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 --2- 172.21.15.84:0/3779035773 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12df00 0x563b1e12e2e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.828 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 -- 172.21.15.84:0/3779035773 >> 172.21.15.84:0/3779035773 conn(0x563b1e12a410 msgr2=0x563b1e12a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 -- 172.21.15.84:0/3779035773 shutdown_connections 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 -- 172.21.15.84:0/3779035773 wait complete. 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 -- start start 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 Processor -- start 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e135fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e12e820 0x563b1e136520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e1390e0 0x563b1e136a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563b1e13bd10 con 0x563b1e1390e0 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563b1e01bb60 con 0x563b1e12e820 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563b1e13be90 con 0x563b1e12df00 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c88041740 10 client.0 _ll_register_callbacks cb 0x563b1e0402b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c7e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e1390e0 0x563b1e136a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c7e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e1390e0 0x563b1e136a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34610/0 (socket says 172.21.15.84:34610) 2024-04-26T18:30:24.829 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.822+0000 7f4c7e7fc640 1 -- 172.21.15.84:0/1764802407 learned_addr learned my addr 172.21.15.84:0/1764802407 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c7effd640 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e12e820 0x563b1e136520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e135fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c7e7fc640 1 -- 172.21.15.84:0/1764802407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 msgr2=0x563b1e135fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e135fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c7e7fc640 1 -- 172.21.15.84:0/1764802407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e12e820 msgr2=0x563b1e136520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e12e820 0x563b1e136520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c7e7fc640 1 -- 172.21.15.84:0/1764802407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b1dda6210 con 0x563b1e1390e0 2024-04-26T18:30:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c7effd640 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e12e820 0x563b1e136520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:30:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e1390e0 0x563b1e136a60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4c68005fd0 tx=0x7f4c68004aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c964d620d0bd2e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c667fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:30:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c667fc640 1 -- 172.21.15.84:0/1764802407 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6801c670 con 0x563b1e1390e0 2024-04-26T18:30:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c667fc640 1 -- 172.21.15.84:0/1764802407 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f4c6801c810 con 0x563b1e1390e0 2024-04-26T18:30:24.830 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c88041740 1 -- 172.21.15.84:0/1764802407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c68017030 con 0x563b1e1390e0 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c667fc640 1 -- 172.21.15.84:0/1764802407 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c6801ca30 con 0x563b1e1390e0 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c88041740 10 client.15765 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c88041740 1 -- 172.21.15.84:0/1764802407 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b1de72620 con 0x563b1e1390e0 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.823+0000 7f4c667fc640 1 -- 172.21.15.84:0/1764802407 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c68017030 con 0x563b1e1390e0 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.824+0000 7f4c667fc640 1 -- 172.21.15.84:0/1764802407 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4c68028070 con 0x563b1e1390e0 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.824+0000 7f4c88041740 10 client.15765 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.824+0000 7f4c88041740 10 client.15765 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.824+0000 7f4c88041740 20 client.15765 populate_metadata read hostname 'smithi084' 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.824+0000 7f4c88041740 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563b1e149070 0x563b1e14b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.824+0000 7f4c88041740 4 client.15765 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.824+0000 7f4c88041740 1 -- 172.21.15.84:0/1764802407 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b1e0a2070 con 0x563b1e149070 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.824+0000 7f4c7effd640 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563b1e149070 0x563b1e14b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c7effd640 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563b1e149070 0x563b1e14b460 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a56a9ef16031ad0 server_cookie=6a158059aae69b6e in_seq=0 out_seq=0 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c667fc640 10 client.15765 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c667fc640 1 -- 172.21.15.84:0/1764802407 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b1e0a2070 con 0x563b1e149070 2024-04-26T18:30:24.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c667fc640 10 client.15765 handle_command_reply: tid=0 2024-04-26T18:30:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c88041740 1 client.15765 shutdown 2024-04-26T18:30:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c88041740 1 -- 172.21.15.84:0/1764802407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e1390e0 msgr2=0x563b1e136a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c88041740 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e1390e0 0x563b1e136a60 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f4c68005fd0 tx=0x7f4c68004aa0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c88041740 1 -- 172.21.15.84:0/1764802407 shutdown_connections 2024-04-26T18:30:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c88041740 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563b1e149070 0x563b1e14b460 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c88041740 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e1390e0 0x563b1e136a60 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c88041740 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e12e820 0x563b1e136520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c667fc640 0 client.15765 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c88041740 1 --2- 172.21.15.84:0/1764802407 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e135fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.825+0000 7f4c88041740 1 -- 172.21.15.84:0/1764802407 >> 172.21.15.84:0/1764802407 conn(0x563b1e12a410 msgr2=0x563b1e130190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.826+0000 7f4c88041740 1 -- 172.21.15.84:0/1764802407 shutdown_connections 2024-04-26T18:30:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.826+0000 7f4c88041740 1 -- 172.21.15.84:0/1764802407 wait complete. 2024-04-26T18:30:24.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.826+0000 7f4c88041740 20 client.15765 trim_cache size 0 max 16384 2024-04-26T18:30:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.840+0000 7f4c88041740 1 Processor -- start 2024-04-26T18:30:24.840 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.840+0000 7f4c88041740 1 -- start start 2024-04-26T18:30:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.840+0000 7f4c88041740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e144bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.840+0000 7f4c88041740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 0x563b1e136830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.840+0000 7f4c88041740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e1390e0 0x563b1e150ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.840+0000 7f4c88041740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4c801b06b0 con 0x563b1e12e820 2024-04-26T18:30:24.844 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.840+0000 7f4c88041740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c801b21f0 con 0x563b1e1390e0 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.840+0000 7f4c88041740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c801b0d80 con 0x563b1e12df00 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c7effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 0x563b1e136830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c7effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 0x563b1e136830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34616/0 (socket says 172.21.15.84:34616) 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c7effd640 1 -- 172.21.15.84:0/182512914 learned_addr learned my addr 172.21.15.84:0/182512914 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/182512914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e144bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/182512914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e1390e0 0x563b1e150ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c7effd640 1 -- 172.21.15.84:0/182512914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 msgr2=0x563b1e144bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c7effd640 1 --2- 172.21.15.84:0/182512914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e144bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c7effd640 1 -- 172.21.15.84:0/182512914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e1390e0 msgr2=0x563b1e150ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c7effd640 1 --2- 172.21.15.84:0/182512914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e1390e0 0x563b1e150ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c7effd640 1 -- 172.21.15.84:0/182512914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b1e134f90 con 0x563b1e12e820 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/182512914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c7effd640 1 --2- 172.21.15.84:0/182512914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 0x563b1e136830 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000b370 tx=0x7f4c7000b3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65505d1768df5dd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:24.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c74ff9640 1 -- 172.21.15.84:0/182512914 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c70004030 con 0x563b1e12e820 2024-04-26T18:30:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c74ff9640 1 -- 172.21.15.84:0/182512914 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c7001a7e0 con 0x563b1e12e820 2024-04-26T18:30:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.841+0000 7f4c74ff9640 1 -- 172.21.15.84:0/182512914 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c700038e0 con 0x563b1e12e820 2024-04-26T18:30:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 -- 172.21.15.84:0/182512914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 msgr2=0x563b1e136830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 --2- 172.21.15.84:0/182512914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 0x563b1e136830 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4c7000b370 tx=0x7f4c7000b3a0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 -- 172.21.15.84:0/182512914 shutdown_connections 2024-04-26T18:30:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 --2- 172.21.15.84:0/182512914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e1390e0 0x563b1e150ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 --2- 172.21.15.84:0/182512914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 0x563b1e136830 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 --2- 172.21.15.84:0/182512914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e144bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 -- 172.21.15.84:0/182512914 >> 172.21.15.84:0/182512914 conn(0x563b1e12a410 msgr2=0x563b1e12a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 -- 172.21.15.84:0/182512914 shutdown_connections 2024-04-26T18:30:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 -- 172.21.15.84:0/182512914 wait complete. 2024-04-26T18:30:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 -- start start 2024-04-26T18:30:24.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 Processor -- start 2024-04-26T18:30:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e13ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 0x563b1e13d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e1390e0 0x563b1e13d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4c801b1b20 con 0x563b1e12e820 2024-04-26T18:30:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c801b1450 con 0x563b1e1390e0 2024-04-26T18:30:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.842+0000 7f4c88041740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c801ae690 con 0x563b1e12df00 2024-04-26T18:30:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c88041740 10 client.0 _ll_register_callbacks cb 0x7f4c6800bcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c7effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 0x563b1e13d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c7effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 0x563b1e13d200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34628/0 (socket says 172.21.15.84:34628) 2024-04-26T18:30:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c7effd640 1 -- 172.21.15.84:0/879819296 learned_addr learned my addr 172.21.15.84:0/879819296 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/879819296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e13ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c7effd640 1 -- 172.21.15.84:0/879819296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 msgr2=0x563b1e13ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:24.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c7effd640 1 --2- 172.21.15.84:0/879819296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e13ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c7effd640 1 -- 172.21.15.84:0/879819296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e1390e0 msgr2=0x563b1e13d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c7effd640 1 --2- 172.21.15.84:0/879819296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e1390e0 0x563b1e13d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c7effd640 1 -- 172.21.15.84:0/879819296 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b1e1b6d30 con 0x563b1e12e820 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/879819296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e13ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c7effd640 1 --2- 172.21.15.84:0/879819296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 0x563b1e13d200 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4c70015010 tx=0x7f4c7001aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97aa90d21fa35f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c667fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c667fc640 1 -- 172.21.15.84:0/879819296 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7001ad80 con 0x563b1e12e820 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c667fc640 1 -- 172.21.15.84:0/879819296 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f4c7001a330 con 0x563b1e12e820 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c88041740 1 -- 172.21.15.84:0/879819296 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4c7001a7e0 con 0x563b1e12e820 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c667fc640 1 -- 172.21.15.84:0/879819296 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4c7001a570 con 0x563b1e12e820 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c88041740 10 client.15771 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.843+0000 7f4c88041740 1 -- 172.21.15.84:0/879819296 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b1e1b8000 con 0x563b1e12e820 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.844+0000 7f4c667fc640 1 -- 172.21.15.84:0/879819296 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c7001a7e0 con 0x563b1e12e820 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.844+0000 7f4c667fc640 1 -- 172.21.15.84:0/879819296 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4c7002d4c0 con 0x563b1e12e820 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.844+0000 7f4c88041740 10 client.15771 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:24.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.844+0000 7f4c88041740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.844+0000 7f4c88041740 20 client.15771 populate_metadata read hostname 'smithi084' 2024-04-26T18:30:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.844+0000 7f4c88041740 1 --2- 172.21.15.84:0/879819296 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563b1e1b86f0 0x563b1e1bab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.844+0000 7f4c88041740 4 client.15771 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:30:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.844+0000 7f4c88041740 1 -- 172.21.15.84:0/879819296 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b1e09bee0 con 0x563b1e1b86f0 2024-04-26T18:30:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.844+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/879819296 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563b1e1b86f0 0x563b1e1bab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.844+0000 7f4c667fc640 10 client.15771 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:24.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.844+0000 7f4c7e7fc640 1 --2- 172.21.15.84:0/879819296 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563b1e1b86f0 0x563b1e1bab10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=325210690a1f3044 server_cookie=7063f83ca4ee7284 in_seq=0 out_seq=0 2024-04-26T18:30:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.853+0000 7f4c667fc640 1 -- 172.21.15.84:0/879819296 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b1e09bee0 con 0x563b1e1b86f0 2024-04-26T18:30:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.853+0000 7f4c667fc640 10 client.15771 handle_command_reply: tid=0 2024-04-26T18:30:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.853+0000 7f4c88041740 1 client.15771 shutdown 2024-04-26T18:30:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.853+0000 7f4c88041740 1 -- 172.21.15.84:0/879819296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 msgr2=0x563b1e13d200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.853+0000 7f4c88041740 1 --2- 172.21.15.84:0/879819296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 0x563b1e13d200 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4c70015010 tx=0x7f4c7001aaf0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.853+0000 7f4c88041740 1 -- 172.21.15.84:0/879819296 shutdown_connections 2024-04-26T18:30:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.853+0000 7f4c88041740 1 --2- 172.21.15.84:0/879819296 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563b1e1b86f0 0x563b1e1bab10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.853+0000 7f4c88041740 1 --2- 172.21.15.84:0/879819296 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563b1e1390e0 0x563b1e13d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.853+0000 7f4c88041740 1 --2- 172.21.15.84:0/879819296 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563b1e12e820 0x563b1e13d200 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.853+0000 7f4c667fc640 0 client.15771 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:24.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.853+0000 7f4c88041740 1 --2- 172.21.15.84:0/879819296 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563b1e12df00 0x563b1e13ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.853+0000 7f4c88041740 1 -- 172.21.15.84:0/879819296 >> 172.21.15.84:0/879819296 conn(0x563b1e12a410 msgr2=0x563b1e130140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.853+0000 7f4c88041740 1 -- 172.21.15.84:0/879819296 shutdown_connections 2024-04-26T18:30:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c88041740 1 -- 172.21.15.84:0/879819296 wait complete. 2024-04-26T18:30:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c88041740 20 client.15771 trim_cache size 0 max 16384 2024-04-26T18:30:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:30:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:30:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:30:24.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:30:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c85324640 1 -- 172.21.15.84:0/3774396114 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c6c0a3e70 msgr2=0x7f4c6c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c85324640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c6c0a3e70 0x7f4c6c0a6330 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4c7804d000 tx=0x7f4c780e88a0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c85324640 1 -- 172.21.15.84:0/3774396114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c8010cf00 msgr2=0x7f4c801021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c85324640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c8010cf00 0x7f4c801021b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f4c7802ae70 tx=0x7f4c7802f5d0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c85324640 1 -- 172.21.15.84:0/3774396114 shutdown_connections 2024-04-26T18:30:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c85324640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c6c0a3e70 0x7f4c6c0a6330 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c85324640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c8010cf00 0x7f4c801021b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c85324640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c80104400 0x7f4c80101c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c85324640 1 --2- 172.21.15.84:0/3774396114 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c80103ae0 0x7f4c80101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c85324640 1 -- 172.21.15.84:0/3774396114 >> 172.21.15.84:0/3774396114 conn(0x7f4c800fd300 msgr2=0x7f4c800fea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c85324640 1 -- 172.21.15.84:0/3774396114 shutdown_connections 2024-04-26T18:30:24.857 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:24.854+0000 7f4c85324640 1 -- 172.21.15.84:0/3774396114 wait complete. 2024-04-26T18:30:24.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:24 smithi103 ceph-mon[32727]: pgmap v76: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-04-26T18:30:24.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:24 smithi174 ceph-mon[32705]: pgmap v76: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-04-26T18:30:25.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:24 smithi084 ceph-mon[29141]: pgmap v76: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 14 op/s 2024-04-26T18:30:26.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:26 smithi103 ceph-mon[32727]: pgmap v77: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-04-26T18:30:26.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:26 smithi174 ceph-mon[32705]: pgmap v77: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-04-26T18:30:27.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:26 smithi084 ceph-mon[29141]: pgmap v77: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-04-26T18:30:28.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:28 smithi084 ceph-mon[29141]: pgmap v78: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-04-26T18:30:28.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:28 smithi103 ceph-mon[32727]: pgmap v78: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-04-26T18:30:28.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:28 smithi174 ceph-mon[32705]: pgmap v78: 97 pgs: 97 active+clean; 1.4 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 6.1 KiB/s wr, 9 op/s 2024-04-26T18:30:30.924 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:30 smithi084 ceph-mon[29141]: pgmap v79: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-26T18:30:30.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:30 smithi174 ceph-mon[32705]: pgmap v79: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-26T18:30:31.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:30 smithi103 ceph-mon[32727]: pgmap v79: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 18 op/s 2024-04-26T18:30:32.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:32 smithi174 ceph-mon[32705]: pgmap v80: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-04-26T18:30:33.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:32 smithi084 ceph-mon[29141]: pgmap v80: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-04-26T18:30:33.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:32 smithi103 ceph-mon[32727]: pgmap v80: 97 pgs: 97 active+clean; 1.5 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 16 KiB/s wr, 14 op/s 2024-04-26T18:30:34.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:34 smithi174 ceph-mon[32705]: pgmap v81: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:30:35.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:34 smithi084 ceph-mon[29141]: pgmap v81: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:30:35.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:34 smithi103 ceph-mon[32727]: pgmap v81: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 20 op/s 2024-04-26T18:30:36.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:36 smithi174 ceph-mon[32705]: pgmap v82: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-26T18:30:37.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:36 smithi084 ceph-mon[29141]: pgmap v82: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-26T18:30:37.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:36 smithi103 ceph-mon[32727]: pgmap v82: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-26T18:30:37.952 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.950+0000 7fc404fe4640 1 -- 172.21.15.84:0/4144499886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc400101b80 msgr2=0x7fc400102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.950+0000 7fc404fe4640 1 --2- 172.21.15.84:0/4144499886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc400101b80 0x7fc400102000 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009900 tx=0x7fc3f402a410 comp rx=0 tx=0).stop 2024-04-26T18:30:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.950+0000 7fc404fe4640 1 -- 172.21.15.84:0/4144499886 shutdown_connections 2024-04-26T18:30:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.950+0000 7fc404fe4640 1 --2- 172.21.15.84:0/4144499886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc400102540 0x7fc40010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.950+0000 7fc404fe4640 1 --2- 172.21.15.84:0/4144499886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc400101b80 0x7fc400102000 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.950+0000 7fc404fe4640 1 --2- 172.21.15.84:0/4144499886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4001067d0 0x7fc400106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.950+0000 7fc404fe4640 1 -- 172.21.15.84:0/4144499886 >> 172.21.15.84:0/4144499886 conn(0x7fc400077630 msgr2=0x7fc400077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.950+0000 7fc404fe4640 1 -- 172.21.15.84:0/4144499886 shutdown_connections 2024-04-26T18:30:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.950+0000 7fc404fe4640 1 -- 172.21.15.84:0/4144499886 wait complete. 2024-04-26T18:30:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.951+0000 7fc404fe4640 1 Processor -- start 2024-04-26T18:30:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.951+0000 7fc404fe4640 1 -- start start 2024-04-26T18:30:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.951+0000 7fc404fe4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc400101b80 0x7fc400199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.953 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.951+0000 7fc404fe4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc400102540 0x7fc400199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.951+0000 7fc404fe4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4001067d0 0x7fc40019d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.951+0000 7fc404fe4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc400111ab0 con 0x7fc400101b80 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.951+0000 7fc404fe4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc400111930 con 0x7fc400102540 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.951+0000 7fc404fe4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc400111c30 con 0x7fc4001067d0 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.951+0000 7fc3fe59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc400101b80 0x7fc400199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.951+0000 7fc3fe59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc400101b80 0x7fc400199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39806/0 (socket says 172.21.15.84:39806) 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.951+0000 7fc3fe59c640 1 -- 172.21.15.84:0/1403307353 learned_addr learned my addr 172.21.15.84:0/1403307353 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.951+0000 7fc3fdd9b640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc400102540 0x7fc400199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.952+0000 7fc3fed9d640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4001067d0 0x7fc40019d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.952+0000 7fc3fe59c640 1 -- 172.21.15.84:0/1403307353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4001067d0 msgr2=0x7fc40019d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.952+0000 7fc3fe59c640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4001067d0 0x7fc40019d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.952+0000 7fc3fe59c640 1 -- 172.21.15.84:0/1403307353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc400102540 msgr2=0x7fc400199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.952+0000 7fc3fe59c640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc400102540 0x7fc400199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.954 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.952+0000 7fc3fe59c640 1 -- 172.21.15.84:0/1403307353 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc40019dff0 con 0x7fc400101b80 2024-04-26T18:30:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.952+0000 7fc3fe59c640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc400101b80 0x7fc400199000 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000d770 tx=0x7fc3f000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.952+0000 7fc3ef7fe640 1 -- 172.21.15.84:0/1403307353 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0017070 con 0x7fc400101b80 2024-04-26T18:30:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.952+0000 7fc3ef7fe640 1 -- 172.21.15.84:0/1403307353 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3f00054d0 con 0x7fc400101b80 2024-04-26T18:30:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.952+0000 7fc404fe4640 1 -- 172.21.15.84:0/1403307353 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc40019e2e0 con 0x7fc400101b80 2024-04-26T18:30:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.952+0000 7fc3ef7fe640 1 -- 172.21.15.84:0/1403307353 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f0004c30 con 0x7fc400101b80 2024-04-26T18:30:37.955 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.952+0000 7fc404fe4640 1 -- 172.21.15.84:0/1403307353 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4001a5b70 con 0x7fc400101b80 2024-04-26T18:30:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.954+0000 7fc3ef7fe640 1 -- 172.21.15.84:0/1403307353 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc3f0004e10 con 0x7fc400101b80 2024-04-26T18:30:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.954+0000 7fc407dd1740 1 Processor -- start 2024-04-26T18:30:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc407dd1740 1 -- start start 2024-04-26T18:30:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc407dd1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d2df00 0x55b859d2e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc407dd1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2e820 0x55b859d38ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc407dd1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d390e0 0x55b859d3b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc407dd1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b859c94460 con 0x55b859d390e0 2024-04-26T18:30:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc407dd1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b859c4e590 con 0x55b859d2df00 2024-04-26T18:30:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc407dd1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b859c90420 con 0x55b859d2e820 2024-04-26T18:30:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc3ef7fe640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc3e80a3dc0 0x7fc3e80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc3fdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d390e0 0x55b859d3b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc3fed9d640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc3e80a3dc0 0x7fc3e80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc3fdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d390e0 0x55b859d3b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39818/0 (socket says 172.21.15.84:39818) 2024-04-26T18:30:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/1371248500 learned_addr learned my addr 172.21.15.84:0/1371248500 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc3ef7fe640 1 -- 172.21.15.84:0/1403307353 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fc3f00d02e0 con 0x7fc400101b80 2024-04-26T18:30:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc3fe59c640 1 --2- 172.21.15.84:0/1371248500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2e820 0x55b859d38ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.955+0000 7fc3fed9d640 1 --2- 172.21.15.84:0/1371248500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d2df00 0x55b859d2e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.956+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/1371248500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2e820 msgr2=0x55b859d38ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.956+0000 7fc3fdd9b640 1 --2- 172.21.15.84:0/1371248500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2e820 0x55b859d38ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.956+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/1371248500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d2df00 msgr2=0x55b859d2e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.956+0000 7fc3fdd9b640 1 --2- 172.21.15.84:0/1371248500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d2df00 0x55b859d2e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.956+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/1371248500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b859c5b0d0 con 0x55b859d390e0 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.956+0000 7fc3fed9d640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc3e80a3dc0 0x7fc3e80a6280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009270 tx=0x7fc3f8012c60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.956+0000 7fc3fdd9b640 1 --2- 172.21.15.84:0/1371248500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d390e0 0x55b859d3b4d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009900 tx=0x7fc3f4032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb41a54c98ae5c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.956+0000 7fc3ecff9640 1 -- 172.21.15.84:0/1371248500 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4040070 con 0x55b859d390e0 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.956+0000 7fc3ecff9640 1 -- 172.21.15.84:0/1371248500 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3f4004030 con 0x55b859d390e0 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.956+0000 7fc3ecff9640 1 -- 172.21.15.84:0/1371248500 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4035040 con 0x55b859d390e0 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 -- 172.21.15.84:0/1371248500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d390e0 msgr2=0x55b859d3b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 --2- 172.21.15.84:0/1371248500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d390e0 0x55b859d3b4d0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009900 tx=0x7fc3f4032bd0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 -- 172.21.15.84:0/1371248500 shutdown_connections 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 --2- 172.21.15.84:0/1371248500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d390e0 0x55b859d3b4d0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 --2- 172.21.15.84:0/1371248500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2e820 0x55b859d38ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 --2- 172.21.15.84:0/1371248500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d2df00 0x55b859d2e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 -- 172.21.15.84:0/1371248500 >> 172.21.15.84:0/1371248500 conn(0x55b859d2a410 msgr2=0x55b859d2a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 -- 172.21.15.84:0/1371248500 shutdown_connections 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 -- 172.21.15.84:0/1371248500 wait complete. 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 -- start start 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 Processor -- start 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d36000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d36540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 0x55b859d36a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b859d3bcb0 con 0x55b859d2e820 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b859c1bb60 con 0x55b859d390e0 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.957+0000 7fc407dd1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b859d3be30 con 0x55b859d2df00 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc407dd1740 10 client.0 _ll_register_callbacks cb 0x55b859c402b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3fdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d36540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3fdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d36540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39832/0 (socket says 172.21.15.84:39832) 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/730783267 learned_addr learned my addr 172.21.15.84:0/730783267 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3fe59c640 1 --2- 172.21.15.84:0/730783267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d36000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/730783267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 msgr2=0x55b859d36000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3fdd9b640 1 --2- 172.21.15.84:0/730783267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d36000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/730783267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 msgr2=0x55b859d36a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3fdd9b640 1 --2- 172.21.15.84:0/730783267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 0x55b859d36a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/730783267 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b8599a6210 con 0x55b859d2e820 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3fe59c640 1 --2- 172.21.15.84:0/730783267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d36000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3fdd9b640 1 --2- 172.21.15.84:0/730783267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d36540 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009630 tx=0x7fc3f402a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6ee461c2ec35083 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3e5ffb640 1 -- 172.21.15.84:0/730783267 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4040070 con 0x55b859d2e820 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3e5ffb640 1 -- 172.21.15.84:0/730783267 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fc3f4005480 con 0x55b859d2e820 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc407dd1740 1 -- 172.21.15.84:0/730783267 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3f4004030 con 0x55b859d2e820 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc3e5ffb640 1 -- 172.21.15.84:0/730783267 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc3f4056050 con 0x55b859d2e820 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc407dd1740 10 client.15783 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.958+0000 7fc407dd1740 1 -- 172.21.15.84:0/730783267 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b859a72620 con 0x55b859d2e820 2024-04-26T18:30:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.959+0000 7fc3e5ffb640 1 -- 172.21.15.84:0/730783267 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3f4004030 con 0x55b859d2e820 2024-04-26T18:30:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.959+0000 7fc3e5ffb640 1 -- 172.21.15.84:0/730783267 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc3f40035f0 con 0x55b859d2e820 2024-04-26T18:30:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.959+0000 7fc407dd1740 10 client.15783 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.959+0000 7fc407dd1740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.959+0000 7fc407dd1740 20 client.15783 populate_metadata read hostname 'smithi084' 2024-04-26T18:30:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.959+0000 7fc407dd1740 1 --2- 172.21.15.84:0/730783267 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b859d48ff0 0x55b859d4b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.959+0000 7fc407dd1740 4 client.15783 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:30:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.959+0000 7fc407dd1740 1 -- 172.21.15.84:0/730783267 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b859ca2070 con 0x55b859d48ff0 2024-04-26T18:30:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.959+0000 7fc3fe59c640 1 --2- 172.21.15.84:0/730783267 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b859d48ff0 0x55b859d4b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc3fe59c640 1 --2- 172.21.15.84:0/730783267 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b859d48ff0 0x55b859d4b3e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba83d3655f4ad859 server_cookie=8528dc41e6878390 in_seq=0 out_seq=0 2024-04-26T18:30:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc3e5ffb640 10 client.15783 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc3e5ffb640 1 -- 172.21.15.84:0/730783267 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b859ca2070 con 0x55b859d48ff0 2024-04-26T18:30:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc3e5ffb640 10 client.15783 handle_command_reply: tid=0 2024-04-26T18:30:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc407dd1740 1 client.15783 shutdown 2024-04-26T18:30:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc407dd1740 1 -- 172.21.15.84:0/730783267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 msgr2=0x55b859d36540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc407dd1740 1 --2- 172.21.15.84:0/730783267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d36540 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009630 tx=0x7fc3f402a410 comp rx=0 tx=0).stop 2024-04-26T18:30:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc407dd1740 1 -- 172.21.15.84:0/730783267 shutdown_connections 2024-04-26T18:30:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc407dd1740 1 --2- 172.21.15.84:0/730783267 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b859d48ff0 0x55b859d4b3e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc407dd1740 1 --2- 172.21.15.84:0/730783267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 0x55b859d36a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc407dd1740 1 --2- 172.21.15.84:0/730783267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d36540 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc3e5ffb640 0 client.15783 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc407dd1740 1 --2- 172.21.15.84:0/730783267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d36000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.960+0000 7fc407dd1740 1 -- 172.21.15.84:0/730783267 >> 172.21.15.84:0/730783267 conn(0x55b859d2a410 msgr2=0x55b859d30140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.961+0000 7fc407dd1740 1 -- 172.21.15.84:0/730783267 shutdown_connections 2024-04-26T18:30:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.961+0000 7fc407dd1740 1 -- 172.21.15.84:0/730783267 wait complete. 2024-04-26T18:30:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.961+0000 7fc407dd1740 20 client.15783 trim_cache size 0 max 16384 2024-04-26T18:30:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.975+0000 7fc407dd1740 1 Processor -- start 2024-04-26T18:30:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.975+0000 7fc407dd1740 1 -- start start 2024-04-26T18:30:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc407dd1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d44b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc407dd1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d509b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc407dd1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 0x55b859d50ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc407dd1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001ae480 con 0x55b859d2e820 2024-04-26T18:30:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc407dd1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001affc0 con 0x55b859d390e0 2024-04-26T18:30:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc407dd1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001aeb50 con 0x55b859d2df00 2024-04-26T18:30:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3fdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3fdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39846/0 (socket says 172.21.15.84:39846) 2024-04-26T18:30:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/3017350574 learned_addr learned my addr 172.21.15.84:0/3017350574 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3fe59c640 1 --2- 172.21.15.84:0/3017350574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d44b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/3017350574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 msgr2=0x55b859d44b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3fdd9b640 1 --2- 172.21.15.84:0/3017350574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d44b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3fed9d640 1 --2- 172.21.15.84:0/3017350574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 0x55b859d50ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/3017350574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 msgr2=0x55b859d50ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3fdd9b640 1 --2- 172.21.15.84:0/3017350574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 0x55b859d50ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/3017350574 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b859d25d60 con 0x55b859d2e820 2024-04-26T18:30:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3fe59c640 1 --2- 172.21.15.84:0/3017350574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d44b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:30:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3fdd9b640 1 --2- 172.21.15.84:0/3017350574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d509b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc3f402a930 tx=0x7fc3f40097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2b18d0682a487a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3e7fff640 1 -- 172.21.15.84:0/3017350574 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4040040 con 0x55b859d2e820 2024-04-26T18:30:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3e7fff640 1 -- 172.21.15.84:0/3017350574 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3f404d070 con 0x55b859d2e820 2024-04-26T18:30:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.976+0000 7fc3e7fff640 1 -- 172.21.15.84:0/3017350574 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4035040 con 0x55b859d2e820 2024-04-26T18:30:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.977+0000 7fc407dd1740 1 -- 172.21.15.84:0/3017350574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 msgr2=0x55b859d509b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.977+0000 7fc407dd1740 1 --2- 172.21.15.84:0/3017350574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d509b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fc3f402a930 tx=0x7fc3f40097c0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.977+0000 7fc407dd1740 1 -- 172.21.15.84:0/3017350574 shutdown_connections 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.977+0000 7fc407dd1740 1 --2- 172.21.15.84:0/3017350574 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 0x55b859d50ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.977+0000 7fc407dd1740 1 --2- 172.21.15.84:0/3017350574 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d509b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.977+0000 7fc407dd1740 1 --2- 172.21.15.84:0/3017350574 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d44b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.977+0000 7fc407dd1740 1 -- 172.21.15.84:0/3017350574 >> 172.21.15.84:0/3017350574 conn(0x55b859d2a410 msgr2=0x55b859d36850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.977+0000 7fc407dd1740 1 -- 172.21.15.84:0/3017350574 shutdown_connections 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.977+0000 7fc407dd1740 1 -- 172.21.15.84:0/3017350574 wait complete. 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.977+0000 7fc407dd1740 1 -- start start 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.977+0000 7fc407dd1740 1 Processor -- start 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc407dd1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d41080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc407dd1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc407dd1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 0x55b859d3e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc407dd1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001af8f0 con 0x55b859d2e820 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc407dd1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001af220 con 0x55b859d390e0 2024-04-26T18:30:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc407dd1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc4001ac460 con 0x55b859d2df00 2024-04-26T18:30:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc407dd1740 10 client.0 _ll_register_callbacks cb 0x7fc3f0009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3fdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3fdd9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d415c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39858/0 (socket says 172.21.15.84:39858) 2024-04-26T18:30:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/2333424014 learned_addr learned my addr 172.21.15.84:0/2333424014 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3fe59c640 1 --2- 172.21.15.84:0/2333424014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d41080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3fed9d640 1 --2- 172.21.15.84:0/2333424014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 0x55b859d3e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/2333424014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 msgr2=0x55b859d41080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3fdd9b640 1 --2- 172.21.15.84:0/2333424014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d41080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/2333424014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 msgr2=0x55b859d3e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3fdd9b640 1 --2- 172.21.15.84:0/2333424014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 0x55b859d3e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3fdd9b640 1 -- 172.21.15.84:0/2333424014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b859d3e6c0 con 0x55b859d2e820 2024-04-26T18:30:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3fdd9b640 1 --2- 172.21.15.84:0/2333424014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d415c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4004330 tx=0x7fc3f4004010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9cb23c3281919a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3e5ffb640 1 -- 172.21.15.84:0/2333424014 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f4035070 con 0x55b859d2e820 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3e5ffb640 1 -- 172.21.15.84:0/2333424014 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fc3f4005350 con 0x55b859d2e820 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc407dd1740 1 -- 172.21.15.84:0/2333424014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3f404d070 con 0x55b859d2e820 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc3e5ffb640 1 -- 172.21.15.84:0/2333424014 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc3f4048040 con 0x55b859d2e820 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc407dd1740 10 client.15789 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.978+0000 7fc407dd1740 1 -- 172.21.15.84:0/2333424014 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b859dafba0 con 0x55b859d2e820 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.979+0000 7fc3e5ffb640 1 -- 172.21.15.84:0/2333424014 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3f404d070 con 0x55b859d2e820 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.979+0000 7fc3e5ffb640 1 -- 172.21.15.84:0/2333424014 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc3f40314a0 con 0x55b859d2e820 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.979+0000 7fc407dd1740 10 client.15789 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.979+0000 7fc407dd1740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.979+0000 7fc407dd1740 20 client.15789 populate_metadata read hostname 'smithi084' 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.979+0000 7fc407dd1740 1 --2- 172.21.15.84:0/2333424014 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b859db0210 0x55b859db2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.979+0000 7fc407dd1740 4 client.15789 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:30:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.979+0000 7fc407dd1740 1 -- 172.21.15.84:0/2333424014 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b859c9bee0 con 0x55b859db0210 2024-04-26T18:30:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.979+0000 7fc3fe59c640 1 --2- 172.21.15.84:0/2333424014 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b859db0210 0x55b859db2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.979+0000 7fc3fe59c640 1 --2- 172.21.15.84:0/2333424014 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b859db0210 0x55b859db2630 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6054d4a2415607e2 server_cookie=8d91cbe8877f4777 in_seq=0 out_seq=0 2024-04-26T18:30:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.979+0000 7fc3e5ffb640 10 client.15789 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.992+0000 7fc3e5ffb640 1 -- 172.21.15.84:0/2333424014 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b859c9bee0 con 0x55b859db0210 2024-04-26T18:30:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.992+0000 7fc3e5ffb640 10 client.15789 handle_command_reply: tid=0 2024-04-26T18:30:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.992+0000 7fc407dd1740 1 client.15789 shutdown 2024-04-26T18:30:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.992+0000 7fc407dd1740 1 -- 172.21.15.84:0/2333424014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 msgr2=0x55b859d415c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.992+0000 7fc407dd1740 1 --2- 172.21.15.84:0/2333424014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d415c0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4004330 tx=0x7fc3f4004010 comp rx=0 tx=0).stop 2024-04-26T18:30:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.992+0000 7fc407dd1740 1 -- 172.21.15.84:0/2333424014 shutdown_connections 2024-04-26T18:30:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.992+0000 7fc407dd1740 1 --2- 172.21.15.84:0/2333424014 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b859db0210 0x55b859db2630 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.992+0000 7fc407dd1740 1 --2- 172.21.15.84:0/2333424014 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b859d390e0 0x55b859d3e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc3e5ffb640 0 client.15789 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc407dd1740 1 --2- 172.21.15.84:0/2333424014 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b859d2e820 0x55b859d415c0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc407dd1740 1 --2- 172.21.15.84:0/2333424014 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b859d2df00 0x55b859d41080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc407dd1740 1 -- 172.21.15.84:0/2333424014 >> 172.21.15.84:0/2333424014 conn(0x55b859d2a410 msgr2=0x55b859d2f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:37.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc407dd1740 1 -- 172.21.15.84:0/2333424014 shutdown_connections 2024-04-26T18:30:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc407dd1740 1 -- 172.21.15.84:0/2333424014 wait complete. 2024-04-26T18:30:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc407dd1740 20 client.15789 trim_cache size 0 max 16384 2024-04-26T18:30:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:30:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:30:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:30:37.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:30:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc404fe4640 1 -- 172.21.15.84:0/1403307353 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc3e80a3dc0 msgr2=0x7fc3e80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc404fe4640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc3e80a3dc0 0x7fc3e80a6280 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009270 tx=0x7fc3f8012c60 comp rx=0 tx=0).stop 2024-04-26T18:30:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc404fe4640 1 -- 172.21.15.84:0/1403307353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc400101b80 msgr2=0x7fc400199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc404fe4640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc400101b80 0x7fc400199000 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc3f000d770 tx=0x7fc3f000dc30 comp rx=0 tx=0).stop 2024-04-26T18:30:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc404fe4640 1 -- 172.21.15.84:0/1403307353 shutdown_connections 2024-04-26T18:30:37.995 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc404fe4640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc3e80a3dc0 0x7fc3e80a6280 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc404fe4640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc4001067d0 0x7fc40019d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc404fe4640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc400102540 0x7fc400199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc404fe4640 1 --2- 172.21.15.84:0/1403307353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc400101b80 0x7fc400199000 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc404fe4640 1 -- 172.21.15.84:0/1403307353 >> 172.21.15.84:0/1403307353 conn(0x7fc400077630 msgr2=0x7fc40006b440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc404fe4640 1 -- 172.21.15.84:0/1403307353 shutdown_connections 2024-04-26T18:30:37.996 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:37.993+0000 7fc404fe4640 1 -- 172.21.15.84:0/1403307353 wait complete. 2024-04-26T18:30:38.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:38 smithi174 ceph-mon[32705]: pgmap v83: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-26T18:30:39.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:38 smithi084 ceph-mon[29141]: pgmap v83: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-26T18:30:39.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:38 smithi103 ceph-mon[32727]: pgmap v83: 97 pgs: 97 active+clean; 1.6 MiB data, 374 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 15 op/s 2024-04-26T18:30:40.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:40 smithi174 ceph-mon[32705]: pgmap v84: 97 pgs: 97 active+clean; 1.7 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-04-26T18:30:41.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:40 smithi084 ceph-mon[29141]: pgmap v84: 97 pgs: 97 active+clean; 1.7 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-04-26T18:30:41.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:40 smithi103 ceph-mon[32727]: pgmap v84: 97 pgs: 97 active+clean; 1.7 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 29 KiB/s wr, 24 op/s 2024-04-26T18:30:42.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:42 smithi174 ceph-mon[32705]: pgmap v85: 97 pgs: 97 active+clean; 1.7 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-26T18:30:43.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:42 smithi084 ceph-mon[29141]: pgmap v85: 97 pgs: 97 active+clean; 1.7 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-26T18:30:43.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:42 smithi103 ceph-mon[32727]: pgmap v85: 97 pgs: 97 active+clean; 1.7 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 19 KiB/s wr, 15 op/s 2024-04-26T18:30:44.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:44 smithi174 ceph-mon[32705]: pgmap v86: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-26T18:30:45.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:44 smithi084 ceph-mon[29141]: pgmap v86: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-26T18:30:45.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:44 smithi103 ceph-mon[32727]: pgmap v86: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 21 op/s 2024-04-26T18:30:46.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:46 smithi174 ceph-mon[32705]: pgmap v87: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:30:47.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:46 smithi084 ceph-mon[29141]: pgmap v87: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:30:47.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:46 smithi103 ceph-mon[32727]: pgmap v87: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:30:48.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:48 smithi103 ceph-mon[32727]: pgmap v88: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:30:48.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:48 smithi174 ceph-mon[32705]: pgmap v88: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:30:49.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:48 smithi084 ceph-mon[29141]: pgmap v88: 97 pgs: 97 active+clean; 1.8 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:30:50.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:50 smithi174 ceph-mon[32705]: pgmap v89: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 22 op/s 2024-04-26T18:30:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.089+0000 7ff72586d640 1 -- 172.21.15.84:0/289640192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720102930 msgr2=0x7ff720102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.089+0000 7ff72586d640 1 --2- 172.21.15.84:0/289640192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720102930 0x7ff720102db0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7ff714009930 tx=0x7ff71402a420 comp rx=0 tx=0).stop 2024-04-26T18:30:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.089+0000 7ff72586d640 1 -- 172.21.15.84:0/289640192 shutdown_connections 2024-04-26T18:30:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.089+0000 7ff72586d640 1 --2- 172.21.15.84:0/289640192 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7201032f0 0x7ff72010f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.089+0000 7ff72586d640 1 --2- 172.21.15.84:0/289640192 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720102930 0x7ff720102db0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.089+0000 7ff72586d640 1 --2- 172.21.15.84:0/289640192 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff720108940 0x7ff720108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.092 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.089+0000 7ff72586d640 1 -- 172.21.15.84:0/289640192 >> 172.21.15.84:0/289640192 conn(0x7ff7200fe400 msgr2=0x7ff720100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.089+0000 7ff72586d640 1 -- 172.21.15.84:0/289640192 shutdown_connections 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.089+0000 7ff72586d640 1 -- 172.21.15.84:0/289640192 wait complete. 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.090+0000 7ff72586d640 1 Processor -- start 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.090+0000 7ff72586d640 1 -- start start 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.090+0000 7ff72586d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff720102930 0x7ff7201990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.090+0000 7ff72586d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7201032f0 0x7ff720199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.090+0000 7ff72586d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720108940 0x7ff72019d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.090+0000 7ff72586d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff720112c00 con 0x7ff720108940 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.090+0000 7ff72586d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff720112a80 con 0x7ff7201032f0 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.090+0000 7ff72586d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff720112d80 con 0x7ff720102930 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff71f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720108940 0x7ff72019d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff71f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720108940 0x7ff72019d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38682/0 (socket says 172.21.15.84:38682) 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff71f7fe640 1 -- 172.21.15.84:0/1661985054 learned_addr learned my addr 172.21.15.84:0/1661985054 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:51.093 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff71effd640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff720102930 0x7ff7201990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff71e7fc640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7201032f0 0x7ff720199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff71f7fe640 1 -- 172.21.15.84:0/1661985054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff720102930 msgr2=0x7ff7201990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff71f7fe640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff720102930 0x7ff7201990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff71f7fe640 1 -- 172.21.15.84:0/1661985054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7201032f0 msgr2=0x7ff720199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff71f7fe640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7201032f0 0x7ff720199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff71f7fe640 1 -- 172.21.15.84:0/1661985054 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72019e0b0 con 0x7ff720108940 2024-04-26T18:30:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff71f7fe640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720108940 0x7ff72019d980 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff71800d750 tx=0x7ff71800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff72486b640 1 -- 172.21.15.84:0/1661985054 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff718017070 con 0x7ff720108940 2024-04-26T18:30:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff72486b640 1 -- 172.21.15.84:0/1661985054 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7180054d0 con 0x7ff720108940 2024-04-26T18:30:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff72586d640 1 -- 172.21.15.84:0/1661985054 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72019e3a0 con 0x7ff720108940 2024-04-26T18:30:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff72486b640 1 -- 172.21.15.84:0/1661985054 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff718004c60 con 0x7ff720108940 2024-04-26T18:30:51.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.091+0000 7ff72586d640 1 -- 172.21.15.84:0/1661985054 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7201a5c30 con 0x7ff720108940 2024-04-26T18:30:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.093+0000 7ff72486b640 1 -- 172.21.15.84:0/1661985054 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff718009bd0 con 0x7ff720108940 2024-04-26T18:30:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff728584740 1 Processor -- start 2024-04-26T18:30:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff728584740 1 -- start start 2024-04-26T18:30:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff72486b640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff7080a3dc0 0x7ff7080a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff728584740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f18310 0x563176f186f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff728584740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f18c30 0x563176f22fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff728584740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f234f0 0x563176f258e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff728584740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563176e7a460 con 0x563176f234f0 2024-04-26T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff728584740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563176e34590 con 0x563176f18310 2024-04-26T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff728584740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563176e76420 con 0x563176f18c30 2024-04-26T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff71effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f18310 0x563176f186f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff71effd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f18310 0x563176f186f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:50136/0 (socket says 172.21.15.84:50136) 2024-04-26T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff71effd640 1 -- 172.21.15.84:0/2025317910 learned_addr learned my addr 172.21.15.84:0/2025317910 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff71e7fc640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff7080a3dc0 0x7ff7080a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff72486b640 1 -- 172.21.15.84:0/1661985054 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7ff7180d01e0 con 0x7ff720108940 2024-04-26T18:30:51.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:50 smithi084 ceph-mon[29141]: pgmap v89: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 22 op/s 2024-04-26T18:30:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.094+0000 7ff71f7fe640 1 --2- 172.21.15.84:0/2025317910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f234f0 0x563176f258e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.095+0000 7ff71e7fc640 1 --2- 172.21.15.84:0/2025317910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f18c30 0x563176f22fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.095+0000 7ff71effd640 1 -- 172.21.15.84:0/2025317910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f18c30 msgr2=0x563176f22fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:51.103 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.095+0000 7ff71effd640 1 --2- 172.21.15.84:0/2025317910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f18c30 0x563176f22fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.095+0000 7ff71effd640 1 -- 172.21.15.84:0/2025317910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f234f0 msgr2=0x563176f258e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.095+0000 7ff71effd640 1 --2- 172.21.15.84:0/2025317910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f234f0 0x563176f258e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.095+0000 7ff71effd640 1 -- 172.21.15.84:0/2025317910 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563176e410d0 con 0x563176f18310 2024-04-26T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.095+0000 7ff71e7fc640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff7080a3dc0 0x7ff7080a6280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff714006010 tx=0x7ff714032e30 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.095+0000 7ff71effd640 1 --2- 172.21.15.84:0/2025317910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f18310 0x563176f186f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff710010370 tx=0x7ff710010830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=223eea90187113 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.095+0000 7ff705ffb640 1 -- 172.21.15.84:0/2025317910 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710016070 con 0x563176f18310 2024-04-26T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.095+0000 7ff705ffb640 1 -- 172.21.15.84:0/2025317910 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff710004d20 con 0x563176f18310 2024-04-26T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.095+0000 7ff705ffb640 1 -- 172.21.15.84:0/2025317910 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7100053a0 con 0x563176f18310 2024-04-26T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.096+0000 7ff728584740 1 -- 172.21.15.84:0/2025317910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f18310 msgr2=0x563176f186f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.096+0000 7ff728584740 1 --2- 172.21.15.84:0/2025317910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f18310 0x563176f186f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff710010370 tx=0x7ff710010830 comp rx=0 tx=0).stop 2024-04-26T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.096+0000 7ff728584740 1 -- 172.21.15.84:0/2025317910 shutdown_connections 2024-04-26T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.096+0000 7ff728584740 1 --2- 172.21.15.84:0/2025317910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f234f0 0x563176f258e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.096+0000 7ff728584740 1 --2- 172.21.15.84:0/2025317910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f18c30 0x563176f22fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.096+0000 7ff728584740 1 --2- 172.21.15.84:0/2025317910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f18310 0x563176f186f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.096+0000 7ff728584740 1 -- 172.21.15.84:0/2025317910 >> 172.21.15.84:0/2025317910 conn(0x563176f14820 msgr2=0x563176f14c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.096+0000 7ff728584740 1 -- 172.21.15.84:0/2025317910 shutdown_connections 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.096+0000 7ff728584740 1 -- 172.21.15.84:0/2025317910 wait complete. 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.096+0000 7ff728584740 1 -- start start 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff728584740 1 Processor -- start 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff728584740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18310 0x563176f203d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff728584740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f18c30 0x563176f20910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff728584740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f234f0 0x563176f20e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff728584740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563176f26120 con 0x563176f18310 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff728584740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563176e01b60 con 0x563176f18c30 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff728584740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563176f262a0 con 0x563176f234f0 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff728584740 10 client.0 _ll_register_callbacks cb 0x563176e262b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:51.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff71effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18310 0x563176f203d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff71e7fc640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f18c30 0x563176f20910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff71effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18310 0x563176f203d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38710/0 (socket says 172.21.15.84:38710) 2024-04-26T18:30:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff71effd640 1 -- 172.21.15.84:0/3577933989 learned_addr learned my addr 172.21.15.84:0/3577933989 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff71f7fe640 1 --2- 172.21.15.84:0/3577933989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f234f0 0x563176f20e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff71effd640 1 -- 172.21.15.84:0/3577933989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f234f0 msgr2=0x563176f20e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff71effd640 1 --2- 172.21.15.84:0/3577933989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f234f0 0x563176f20e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff71effd640 1 -- 172.21.15.84:0/3577933989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f18c30 msgr2=0x563176f20910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff71effd640 1 --2- 172.21.15.84:0/3577933989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f18c30 0x563176f20910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff71effd640 1 -- 172.21.15.84:0/3577933989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563176b8c210 con 0x563176f18310 2024-04-26T18:30:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff71e7fc640 1 --2- 172.21.15.84:0/3577933989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f18c30 0x563176f20910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:30:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff71effd640 1 --2- 172.21.15.84:0/3577933989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18310 0x563176f203d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff710010fe0 tx=0x7ff710004c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31dec52f694d04af server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.097+0000 7ff6feffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:30:51.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff6feffd640 1 -- 172.21.15.84:0/3577933989 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710016070 con 0x563176f18310 2024-04-26T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff6feffd640 1 -- 172.21.15.84:0/3577933989 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7ff710003cc0 con 0x563176f18310 2024-04-26T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff728584740 1 -- 172.21.15.84:0/3577933989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff710004d20 con 0x563176f18310 2024-04-26T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff6feffd640 1 -- 172.21.15.84:0/3577933989 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff710004040 con 0x563176f18310 2024-04-26T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff728584740 10 client.15801 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff728584740 1 -- 172.21.15.84:0/3577933989 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563176c58620 con 0x563176f18310 2024-04-26T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff6feffd640 1 -- 172.21.15.84:0/3577933989 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff710004d20 con 0x563176f18310 2024-04-26T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff6feffd640 1 -- 172.21.15.84:0/3577933989 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff71000f040 con 0x563176f18310 2024-04-26T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff728584740 10 client.15801 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff728584740 10 client.15801 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff728584740 20 client.15801 populate_metadata read hostname 'smithi084' 2024-04-26T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff728584740 1 --2- 172.21.15.84:0/3577933989 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563176f334f0 0x563176f358e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff728584740 4 client.15801 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:30:51.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff728584740 1 -- 172.21.15.84:0/3577933989 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563176e88070 con 0x563176f334f0 2024-04-26T18:30:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.098+0000 7ff71e7fc640 1 --2- 172.21.15.84:0/3577933989 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563176f334f0 0x563176f358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.099+0000 7ff71e7fc640 1 --2- 172.21.15.84:0/3577933989 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563176f334f0 0x563176f358e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c0bc897a30f28f4 server_cookie=6644e9985375ff15 in_seq=0 out_seq=0 2024-04-26T18:30:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.099+0000 7ff6feffd640 10 client.15801 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff6feffd640 1 -- 172.21.15.84:0/3577933989 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563176e88070 con 0x563176f334f0 2024-04-26T18:30:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff6feffd640 10 client.15801 handle_command_reply: tid=0 2024-04-26T18:30:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff728584740 1 client.15801 shutdown 2024-04-26T18:30:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff728584740 1 -- 172.21.15.84:0/3577933989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18310 msgr2=0x563176f203d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff728584740 1 --2- 172.21.15.84:0/3577933989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18310 0x563176f203d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7ff710010fe0 tx=0x7ff710004c70 comp rx=0 tx=0).stop 2024-04-26T18:30:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff728584740 1 -- 172.21.15.84:0/3577933989 shutdown_connections 2024-04-26T18:30:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff728584740 1 --2- 172.21.15.84:0/3577933989 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563176f334f0 0x563176f358e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff728584740 1 --2- 172.21.15.84:0/3577933989 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f234f0 0x563176f20e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff6feffd640 0 client.15801 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:51.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff728584740 1 --2- 172.21.15.84:0/3577933989 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f18c30 0x563176f20910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff728584740 1 --2- 172.21.15.84:0/3577933989 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18310 0x563176f203d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff728584740 1 -- 172.21.15.84:0/3577933989 >> 172.21.15.84:0/3577933989 conn(0x563176f14820 msgr2=0x563176f0f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff728584740 1 -- 172.21.15.84:0/3577933989 shutdown_connections 2024-04-26T18:30:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff728584740 1 -- 172.21.15.84:0/3577933989 wait complete. 2024-04-26T18:30:51.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.100+0000 7ff728584740 20 client.15801 trim_cache size 0 max 16384 2024-04-26T18:30:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.115+0000 7ff728584740 1 Processor -- start 2024-04-26T18:30:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.115+0000 7ff728584740 1 -- start start 2024-04-26T18:30:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.115+0000 7ff728584740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 0x563176f2efe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.115+0000 7ff728584740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f234f0 0x563176f358e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.115+0000 7ff728584740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f3b0b0 0x563176f3b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.115+0000 7ff728584740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201ac3c0 con 0x563176f18c30 2024-04-26T18:30:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.115+0000 7ff728584740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201adf00 con 0x563176f234f0 2024-04-26T18:30:51.120 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.115+0000 7ff728584740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201aca90 con 0x563176f3b0b0 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.115+0000 7ff71effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 0x563176f2efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.116+0000 7ff71effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 0x563176f2efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38714/0 (socket says 172.21.15.84:38714) 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.116+0000 7ff71effd640 1 -- 172.21.15.84:0/1082098400 learned_addr learned my addr 172.21.15.84:0/1082098400 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.116+0000 7ff71e7fc640 1 --2- 172.21.15.84:0/1082098400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f234f0 0x563176f358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.116+0000 7ff71effd640 1 -- 172.21.15.84:0/1082098400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f3b0b0 msgr2=0x563176f3b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.116+0000 7ff71effd640 1 --2- 172.21.15.84:0/1082098400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f3b0b0 0x563176f3b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.116+0000 7ff71effd640 1 -- 172.21.15.84:0/1082098400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f234f0 msgr2=0x563176f358e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.116+0000 7ff71effd640 1 --2- 172.21.15.84:0/1082098400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f234f0 0x563176f358e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.116+0000 7ff71effd640 1 -- 172.21.15.84:0/1082098400 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563176f18a60 con 0x563176f18c30 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.116+0000 7ff71e7fc640 1 --2- 172.21.15.84:0/1082098400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f234f0 0x563176f358e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.116+0000 7ff71effd640 1 --2- 172.21.15.84:0/1082098400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 0x563176f2efe0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff710011010 tx=0x7ff7100098a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6203c1b24135c03c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.116+0000 7ff7057fa640 1 -- 172.21.15.84:0/1082098400 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710016040 con 0x563176f18c30 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.116+0000 7ff7057fa640 1 -- 172.21.15.84:0/1082098400 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff710009bf0 con 0x563176f18c30 2024-04-26T18:30:51.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.116+0000 7ff7057fa640 1 -- 172.21.15.84:0/1082098400 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710012510 con 0x563176f18c30 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.117+0000 7ff728584740 1 -- 172.21.15.84:0/1082098400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 msgr2=0x563176f2efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.117+0000 7ff728584740 1 --2- 172.21.15.84:0/1082098400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 0x563176f2efe0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff710011010 tx=0x7ff7100098a0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.117+0000 7ff728584740 1 -- 172.21.15.84:0/1082098400 shutdown_connections 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.117+0000 7ff728584740 1 --2- 172.21.15.84:0/1082098400 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f3b0b0 0x563176f3b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.117+0000 7ff728584740 1 --2- 172.21.15.84:0/1082098400 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f234f0 0x563176f358e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.117+0000 7ff728584740 1 --2- 172.21.15.84:0/1082098400 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 0x563176f2efe0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.117+0000 7ff728584740 1 -- 172.21.15.84:0/1082098400 >> 172.21.15.84:0/1082098400 conn(0x563176f14820 msgr2=0x563176f14c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.117+0000 7ff728584740 1 -- 172.21.15.84:0/1082098400 shutdown_connections 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.117+0000 7ff728584740 1 -- 172.21.15.84:0/1082098400 wait complete. 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.117+0000 7ff728584740 1 -- start start 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.117+0000 7ff728584740 1 Processor -- start 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff728584740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 0x563176f2b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff728584740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f234f0 0x563176f26ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff728584740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f3b0b0 0x563176f26ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff728584740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201ad830 con 0x563176f18c30 2024-04-26T18:30:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff728584740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201ad160 con 0x563176f234f0 2024-04-26T18:30:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff728584740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff7201aa3a0 con 0x563176f3b0b0 2024-04-26T18:30:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff728584740 10 client.0 _ll_register_callbacks cb 0x7ff7180046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:30:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff71effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 0x563176f2b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff71effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 0x563176f2b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38728/0 (socket says 172.21.15.84:38728) 2024-04-26T18:30:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff71effd640 1 -- 172.21.15.84:0/1958320363 learned_addr learned my addr 172.21.15.84:0/1958320363 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:30:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff71f7fe640 1 --2- 172.21.15.84:0/1958320363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f3b0b0 0x563176f26ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff71effd640 1 -- 172.21.15.84:0/1958320363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f3b0b0 msgr2=0x563176f26ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff71effd640 1 --2- 172.21.15.84:0/1958320363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f3b0b0 0x563176f26ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff71effd640 1 -- 172.21.15.84:0/1958320363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f234f0 msgr2=0x563176f26ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:30:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff71effd640 1 --2- 172.21.15.84:0/1958320363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f234f0 0x563176f26ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff71effd640 1 -- 172.21.15.84:0/1958320363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563176f275b0 con 0x563176f18c30 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff71effd640 1 --2- 172.21.15.84:0/1958320363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 0x563176f2b640 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff710010f10 tx=0x7ff710012e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3941c1113308b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff6feffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff6feffd640 1 -- 172.21.15.84:0/1958320363 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff710016040 con 0x563176f18c30 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff6feffd640 1 -- 172.21.15.84:0/1958320363 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7ff710012720 con 0x563176f18c30 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff728584740 1 -- 172.21.15.84:0/1958320363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff710009bf0 con 0x563176f18c30 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff6feffd640 1 -- 172.21.15.84:0/1958320363 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff7100041e0 con 0x563176f18c30 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff728584740 10 client.15807 fetch_fsmap learned FSMap version 29 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.118+0000 7ff728584740 1 -- 172.21.15.84:0/1958320363 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563176f99670 con 0x563176f18c30 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.119+0000 7ff6feffd640 1 -- 172.21.15.84:0/1958320363 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff710009bf0 con 0x563176f18c30 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.119+0000 7ff6feffd640 1 -- 172.21.15.84:0/1958320363 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff710034020 con 0x563176f18c30 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.119+0000 7ff728584740 10 client.15807 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.119+0000 7ff728584740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.119+0000 7ff728584740 20 client.15807 populate_metadata read hostname 'smithi084' 2024-04-26T18:30:51.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.119+0000 7ff728584740 1 --2- 172.21.15.84:0/1958320363 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563176f99b20 0x563176f9bf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:30:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.119+0000 7ff728584740 4 client.15807 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:30:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.119+0000 7ff728584740 1 -- 172.21.15.84:0/1958320363 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563176e81ee0 con 0x563176f99b20 2024-04-26T18:30:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.119+0000 7ff71e7fc640 1 --2- 172.21.15.84:0/1958320363 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563176f99b20 0x563176f9bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:30:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.119+0000 7ff6feffd640 10 client.15807 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:51.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.119+0000 7ff71e7fc640 1 --2- 172.21.15.84:0/1958320363 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563176f99b20 0x563176f9bf40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc676e1ba3419aaa server_cookie=b815c1c55874dd9b in_seq=0 out_seq=0 2024-04-26T18:30:51.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:50 smithi103 ceph-mon[32727]: pgmap v89: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 22 op/s 2024-04-26T18:30:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff6feffd640 1 -- 172.21.15.84:0/1958320363 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563176e81ee0 con 0x563176f99b20 2024-04-26T18:30:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff6feffd640 10 client.15807 handle_command_reply: tid=0 2024-04-26T18:30:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff728584740 1 client.15807 shutdown 2024-04-26T18:30:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff728584740 1 -- 172.21.15.84:0/1958320363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 msgr2=0x563176f2b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff728584740 1 --2- 172.21.15.84:0/1958320363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 0x563176f2b640 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff710010f10 tx=0x7ff710012e60 comp rx=0 tx=0).stop 2024-04-26T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff728584740 1 -- 172.21.15.84:0/1958320363 shutdown_connections 2024-04-26T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff728584740 1 --2- 172.21.15.84:0/1958320363 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563176f99b20 0x563176f9bf40 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff728584740 1 --2- 172.21.15.84:0/1958320363 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563176f3b0b0 0x563176f26ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff6feffd640 0 client.15807 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff728584740 1 --2- 172.21.15.84:0/1958320363 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563176f234f0 0x563176f26ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff728584740 1 --2- 172.21.15.84:0/1958320363 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563176f18c30 0x563176f2b640 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff728584740 1 -- 172.21.15.84:0/1958320363 >> 172.21.15.84:0/1958320363 conn(0x563176f14820 msgr2=0x563176f14c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff728584740 1 -- 172.21.15.84:0/1958320363 shutdown_connections 2024-04-26T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff728584740 1 -- 172.21.15.84:0/1958320363 wait complete. 2024-04-26T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.134+0000 7ff728584740 20 client.15807 trim_cache size 0 max 16384 2024-04-26T18:30:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:30:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:30:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:30:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:30:51.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.135+0000 7ff72586d640 1 -- 172.21.15.84:0/1661985054 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff7080a3dc0 msgr2=0x7ff7080a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.135+0000 7ff72586d640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff7080a3dc0 0x7ff7080a6280 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff714006010 tx=0x7ff714032e30 comp rx=0 tx=0).stop 2024-04-26T18:30:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.135+0000 7ff72586d640 1 -- 172.21.15.84:0/1661985054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720108940 msgr2=0x7ff72019d980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:30:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.135+0000 7ff72586d640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720108940 0x7ff72019d980 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7ff71800d750 tx=0x7ff71800dc10 comp rx=0 tx=0).stop 2024-04-26T18:30:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.135+0000 7ff72586d640 1 -- 172.21.15.84:0/1661985054 shutdown_connections 2024-04-26T18:30:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.135+0000 7ff72586d640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff7080a3dc0 0x7ff7080a6280 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.135+0000 7ff72586d640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff720108940 0x7ff72019d980 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.135+0000 7ff72586d640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff7201032f0 0x7ff720199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.135+0000 7ff72586d640 1 --2- 172.21.15.84:0/1661985054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff720102930 0x7ff7201990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:30:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.135+0000 7ff72586d640 1 -- 172.21.15.84:0/1661985054 >> 172.21.15.84:0/1661985054 conn(0x7ff7200fe400 msgr2=0x7ff7200feb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:30:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.135+0000 7ff72586d640 1 -- 172.21.15.84:0/1661985054 shutdown_connections 2024-04-26T18:30:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:30:51.135+0000 7ff72586d640 1 -- 172.21.15.84:0/1661985054 wait complete. 2024-04-26T18:30:52.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:52 smithi174 ceph-mon[32705]: pgmap v90: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-26T18:30:53.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:52 smithi084 ceph-mon[29141]: pgmap v90: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-26T18:30:53.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:52 smithi103 ceph-mon[32727]: pgmap v90: 97 pgs: 97 active+clean; 1.9 MiB data, 379 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-26T18:30:54.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:54 smithi174 ceph-mon[32705]: pgmap v91: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-04-26T18:30:55.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:54 smithi084 ceph-mon[29141]: pgmap v91: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-04-26T18:30:55.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:54 smithi103 ceph-mon[32727]: pgmap v91: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 21 op/s 2024-04-26T18:30:56.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:56 smithi174 ceph-mon[32705]: pgmap v92: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:30:57.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:56 smithi084 ceph-mon[29141]: pgmap v92: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:30:57.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:56 smithi103 ceph-mon[32727]: pgmap v92: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:30:58.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:30:58 smithi174 ceph-mon[32705]: pgmap v93: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:30:59.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:30:58 smithi084 ceph-mon[29141]: pgmap v93: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:30:59.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:30:58 smithi103 ceph-mon[32727]: pgmap v93: 97 pgs: 97 active+clean; 2.0 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:31:00.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:00 smithi174 ceph-mon[32705]: pgmap v94: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-26T18:31:01.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:00 smithi084 ceph-mon[29141]: pgmap v94: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-26T18:31:01.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:00 smithi103 ceph-mon[32727]: pgmap v94: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-26T18:31:02.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:02 smithi174 ceph-mon[32705]: pgmap v95: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:31:03.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:02 smithi084 ceph-mon[29141]: pgmap v95: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:31:03.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:02 smithi103 ceph-mon[32727]: pgmap v95: 97 pgs: 97 active+clean; 2.0 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-26T18:31:04.225 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.222+0000 7feeddb4e640 1 -- 172.21.15.84:0/2309295053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed8107820 msgr2=0x7feed8107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.222+0000 7feeddb4e640 1 --2- 172.21.15.84:0/2309295053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed8107820 0x7feed8107c00 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7feecc009930 tx=0x7feecc02a420 comp rx=0 tx=0).stop 2024-04-26T18:31:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.222+0000 7feeddb4e640 1 -- 172.21.15.84:0/2309295053 shutdown_connections 2024-04-26T18:31:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.222+0000 7feeddb4e640 1 --2- 172.21.15.84:0/2309295053 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed8102220 0x7feed810e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.222+0000 7feeddb4e640 1 --2- 172.21.15.84:0/2309295053 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8101860 0x7feed8101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.222+0000 7feeddb4e640 1 --2- 172.21.15.84:0/2309295053 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed8107820 0x7feed8107c00 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.222+0000 7feeddb4e640 1 -- 172.21.15.84:0/2309295053 >> 172.21.15.84:0/2309295053 conn(0x7feed80789b0 msgr2=0x7feed8078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.222+0000 7feeddb4e640 1 -- 172.21.15.84:0/2309295053 shutdown_connections 2024-04-26T18:31:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.222+0000 7feeddb4e640 1 -- 172.21.15.84:0/2309295053 wait complete. 2024-04-26T18:31:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.222+0000 7feeddb4e640 1 Processor -- start 2024-04-26T18:31:04.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feeddb4e640 1 -- start start 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feeddb4e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed8101860 0x7feed80fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feeddb4e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed8102220 0x7feed80fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feeddb4e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8107820 0x7feed80ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feeddb4e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7feed8111ae0 con 0x7feed8102220 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feeddb4e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feed8111960 con 0x7feed8101860 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feeddb4e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feed8111c60 con 0x7feed8107820 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feed6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed8102220 0x7feed80fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feed6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed8102220 0x7feed80fed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41590/0 (socket says 172.21.15.84:41590) 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feed6ffd640 1 -- 172.21.15.84:0/2788819694 learned_addr learned my addr 172.21.15.84:0/2788819694 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feed77fe640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed8101860 0x7feed80fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feed7fff640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8107820 0x7feed80ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feed6ffd640 1 -- 172.21.15.84:0/2788819694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8107820 msgr2=0x7feed80ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feed6ffd640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8107820 0x7feed80ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feed6ffd640 1 -- 172.21.15.84:0/2788819694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed8101860 msgr2=0x7feed80fe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feed6ffd640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed8101860 0x7feed80fe7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feed6ffd640 1 -- 172.21.15.84:0/2788819694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feed81a75d0 con 0x7feed8102220 2024-04-26T18:31:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feed77fe640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed8101860 0x7feed80fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:31:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.223+0000 7feed6ffd640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed8102220 0x7feed80fed30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7feec800e9f0 tx=0x7feec800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.224+0000 7feed4ff9640 1 -- 172.21.15.84:0/2788819694 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec800cd80 con 0x7feed8102220 2024-04-26T18:31:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.224+0000 7feeddb4e640 1 -- 172.21.15.84:0/2788819694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feed81a78c0 con 0x7feed8102220 2024-04-26T18:31:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.224+0000 7feed4ff9640 1 -- 172.21.15.84:0/2788819694 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec80054d0 con 0x7feed8102220 2024-04-26T18:31:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.224+0000 7feed4ff9640 1 -- 172.21.15.84:0/2788819694 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec8010690 con 0x7feed8102220 2024-04-26T18:31:04.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.224+0000 7feeddb4e640 1 -- 172.21.15.84:0/2788819694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feed81a7e10 con 0x7feed8102220 2024-04-26T18:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.226+0000 7feed4ff9640 1 -- 172.21.15.84:0/2788819694 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7feec8004e90 con 0x7feed8102220 2024-04-26T18:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.226+0000 7feee0860740 1 Processor -- start 2024-04-26T18:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.226+0000 7feee0860740 1 -- start start 2024-04-26T18:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.226+0000 7feee0860740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff13f00 0x5555dff142e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.226+0000 7feee0860740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 0x5555dff1eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.226+0000 7feee0860740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff1f0e0 0x5555dff214d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.226+0000 7feee0860740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5555dfe7a460 con 0x5555dff14820 2024-04-26T18:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.226+0000 7feee0860740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5555dfe34590 con 0x5555dff13f00 2024-04-26T18:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.226+0000 7feee0860740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5555dfe76420 con 0x5555dff1f0e0 2024-04-26T18:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 0x5555dff1eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed6ffd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff1f0e0 0x5555dff214d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed7fff640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff13f00 0x5555dff142e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 0x5555dff1eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41600/0 (socket says 172.21.15.84:41600) 2024-04-26T18:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed77fe640 1 -- 172.21.15.84:0/962347090 learned_addr learned my addr 172.21.15.84:0/962347090 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed4ff9640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7feec00a3dc0 0x7feec00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed77fe640 1 -- 172.21.15.84:0/962347090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff1f0e0 msgr2=0x5555dff214d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed77fe640 1 --2- 172.21.15.84:0/962347090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff1f0e0 0x5555dff214d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed77fe640 1 -- 172.21.15.84:0/962347090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff13f00 msgr2=0x5555dff142e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed77fe640 1 --2- 172.21.15.84:0/962347090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff13f00 0x5555dff142e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed77fe640 1 -- 172.21.15.84:0/962347090 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555dfe410d0 con 0x5555dff14820 2024-04-26T18:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed7fff640 1 --2- 172.21.15.84:0/962347090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff13f00 0x5555dff142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:31:04.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed7fff640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7feec00a3dc0 0x7feec00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed4ff9640 1 -- 172.21.15.84:0/2788819694 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7feec80d0c60 con 0x7feed8102220 2024-04-26T18:31:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed77fe640 1 --2- 172.21.15.84:0/962347090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 0x5555dff1eba0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feecc006010 tx=0x7feecc032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6669e8a4ccab1e58 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feed7fff640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7feec00a3dc0 0x7feec00a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feed0009fd0 tx=0x7feed0004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feebdffb640 1 -- 172.21.15.84:0/962347090 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc035040 con 0x5555dff14820 2024-04-26T18:31:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.227+0000 7feebdffb640 1 -- 172.21.15.84:0/962347090 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feecc031860 con 0x5555dff14820 2024-04-26T18:31:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.228+0000 7feebdffb640 1 -- 172.21.15.84:0/962347090 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc004a00 con 0x5555dff14820 2024-04-26T18:31:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.228+0000 7feee0860740 1 -- 172.21.15.84:0/962347090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 msgr2=0x5555dff1eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.228+0000 7feee0860740 1 --2- 172.21.15.84:0/962347090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 0x5555dff1eba0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7feecc006010 tx=0x7feecc032ed0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.228+0000 7feee0860740 1 -- 172.21.15.84:0/962347090 shutdown_connections 2024-04-26T18:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.228+0000 7feee0860740 1 --2- 172.21.15.84:0/962347090 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff1f0e0 0x5555dff214d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.228+0000 7feee0860740 1 --2- 172.21.15.84:0/962347090 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 0x5555dff1eba0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.228+0000 7feee0860740 1 --2- 172.21.15.84:0/962347090 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff13f00 0x5555dff142e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.228+0000 7feee0860740 1 -- 172.21.15.84:0/962347090 >> 172.21.15.84:0/962347090 conn(0x5555dff10410 msgr2=0x5555dff107f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.228+0000 7feee0860740 1 -- 172.21.15.84:0/962347090 shutdown_connections 2024-04-26T18:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.228+0000 7feee0860740 1 -- 172.21.15.84:0/962347090 wait complete. 2024-04-26T18:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feee0860740 1 -- start start 2024-04-26T18:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feee0860740 1 Processor -- start 2024-04-26T18:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feee0860740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 0x5555dff1c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feee0860740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff14820 0x5555dff1c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feee0860740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff1f0e0 0x5555dff1cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feee0860740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5555dff21cb0 con 0x5555dff13f00 2024-04-26T18:31:04.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feee0860740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5555dfe01b60 con 0x5555dff1f0e0 2024-04-26T18:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feee0860740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5555dff21e30 con 0x5555dff14820 2024-04-26T18:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feee0860740 10 client.0 _ll_register_callbacks cb 0x5555dfe262b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feed77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 0x5555dff1c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feed77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 0x5555dff1c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41610/0 (socket says 172.21.15.84:41610) 2024-04-26T18:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feed77fe640 1 -- 172.21.15.84:0/1532584746 learned_addr learned my addr 172.21.15.84:0/1532584746 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feed6ffd640 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff14820 0x5555dff1c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feed7fff640 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff1f0e0 0x5555dff1cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feed77fe640 1 -- 172.21.15.84:0/1532584746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff14820 msgr2=0x5555dff1c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:04.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feed77fe640 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff14820 0x5555dff1c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feed77fe640 1 -- 172.21.15.84:0/1532584746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff1f0e0 msgr2=0x5555dff1cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.229+0000 7feed77fe640 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff1f0e0 0x5555dff1cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feed77fe640 1 -- 172.21.15.84:0/1532584746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555dfb8c210 con 0x5555dff13f00 2024-04-26T18:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feed6ffd640 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff14820 0x5555dff1c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feed77fe640 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 0x5555dff1c040 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7feecc02a460 tx=0x7feecc047710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41329f5dc0303300 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feeb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feeb6ffd640 1 -- 172.21.15.84:0/1532584746 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc04e030 con 0x5555dff13f00 2024-04-26T18:31:04.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feeb6ffd640 1 -- 172.21.15.84:0/1532584746 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7feecc031660 con 0x5555dff13f00 2024-04-26T18:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feee0860740 1 -- 172.21.15.84:0/1532584746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feecc031860 con 0x5555dff13f00 2024-04-26T18:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feeb6ffd640 1 -- 172.21.15.84:0/1532584746 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feecc035040 con 0x5555dff13f00 2024-04-26T18:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feee0860740 10 client.15819 fetch_fsmap learned FSMap version 29 2024-04-26T18:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feee0860740 1 -- 172.21.15.84:0/1532584746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5555dfc58620 con 0x5555dff13f00 2024-04-26T18:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feeb6ffd640 1 -- 172.21.15.84:0/1532584746 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feecc031860 con 0x5555dff13f00 2024-04-26T18:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feeb6ffd640 1 -- 172.21.15.84:0/1532584746 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7feecc03ca40 con 0x5555dff13f00 2024-04-26T18:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feee0860740 10 client.15819 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feee0860740 10 client.15819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feee0860740 20 client.15819 populate_metadata read hostname 'smithi084' 2024-04-26T18:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.230+0000 7feee0860740 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5555dff2f010 0x5555dff31400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.231+0000 7feee0860740 4 client.15819 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.231+0000 7feee0860740 1 -- 172.21.15.84:0/1532584746 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555dfe88070 con 0x5555dff2f010 2024-04-26T18:31:04.241 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.231+0000 7feed6ffd640 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5555dff2f010 0x5555dff31400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.231+0000 7feed6ffd640 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5555dff2f010 0x5555dff31400 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c58ce134b1fa51cd server_cookie=cac79ecfc3fb76f8 in_seq=0 out_seq=0 2024-04-26T18:31:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.231+0000 7feeb6ffd640 10 client.15819 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feeb6ffd640 1 -- 172.21.15.84:0/1532584746 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5555dfe88070 con 0x5555dff2f010 2024-04-26T18:31:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feeb6ffd640 10 client.15819 handle_command_reply: tid=0 2024-04-26T18:31:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feee0860740 1 client.15819 shutdown 2024-04-26T18:31:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feee0860740 1 -- 172.21.15.84:0/1532584746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 msgr2=0x5555dff1c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feee0860740 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 0x5555dff1c040 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7feecc02a460 tx=0x7feecc047710 comp rx=0 tx=0).stop 2024-04-26T18:31:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feee0860740 1 -- 172.21.15.84:0/1532584746 shutdown_connections 2024-04-26T18:31:04.242 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feee0860740 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5555dff2f010 0x5555dff31400 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feee0860740 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff1f0e0 0x5555dff1cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feee0860740 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff14820 0x5555dff1c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feeb6ffd640 0 client.15819 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feee0860740 1 --2- 172.21.15.84:0/1532584746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 0x5555dff1c040 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feee0860740 1 -- 172.21.15.84:0/1532584746 >> 172.21.15.84:0/1532584746 conn(0x5555dff10410 msgr2=0x5555dff16190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feee0860740 1 -- 172.21.15.84:0/1532584746 shutdown_connections 2024-04-26T18:31:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feee0860740 1 -- 172.21.15.84:0/1532584746 wait complete. 2024-04-26T18:31:04.243 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.232+0000 7feee0860740 20 client.15819 trim_cache size 0 max 16384 2024-04-26T18:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feee0860740 1 Processor -- start 2024-04-26T18:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feee0860740 1 -- start start 2024-04-26T18:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feee0860740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff13f00 0x5555dff2ab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feee0860740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 0x5555dff1c710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.253 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feee0860740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff1f0e0 0x5555dff36a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feee0860740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7feed81b0720 con 0x5555dff14820 2024-04-26T18:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feee0860740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feed81b2260 con 0x5555dff1f0e0 2024-04-26T18:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feee0860740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feed81b0df0 con 0x5555dff13f00 2024-04-26T18:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feed6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 0x5555dff1c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feed6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 0x5555dff1c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41626/0 (socket says 172.21.15.84:41626) 2024-04-26T18:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feed6ffd640 1 -- 172.21.15.84:0/2703448329 learned_addr learned my addr 172.21.15.84:0/2703448329 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feed7fff640 1 --2- 172.21.15.84:0/2703448329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff1f0e0 0x5555dff36a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feed77fe640 1 --2- 172.21.15.84:0/2703448329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff13f00 0x5555dff2ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feed6ffd640 1 -- 172.21.15.84:0/2703448329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff13f00 msgr2=0x5555dff2ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feed6ffd640 1 --2- 172.21.15.84:0/2703448329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff13f00 0x5555dff2ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feed6ffd640 1 -- 172.21.15.84:0/2703448329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff1f0e0 msgr2=0x5555dff36a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feed6ffd640 1 --2- 172.21.15.84:0/2703448329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff1f0e0 0x5555dff36a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.247+0000 7feed6ffd640 1 -- 172.21.15.84:0/2703448329 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555dff10240 con 0x5555dff14820 2024-04-26T18:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.248+0000 7feed6ffd640 1 --2- 172.21.15.84:0/2703448329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 0x5555dff1c710 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7feec809c890 tx=0x7feec80bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7551718e97f248 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.248+0000 7feebd7fa640 1 -- 172.21.15.84:0/2703448329 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec80b5020 con 0x5555dff14820 2024-04-26T18:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.248+0000 7feebd7fa640 1 -- 172.21.15.84:0/2703448329 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec80d4560 con 0x5555dff14820 2024-04-26T18:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.248+0000 7feebd7fa640 1 -- 172.21.15.84:0/2703448329 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec809cb90 con 0x5555dff14820 2024-04-26T18:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.248+0000 7feee0860740 1 -- 172.21.15.84:0/2703448329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 msgr2=0x5555dff1c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.248+0000 7feee0860740 1 --2- 172.21.15.84:0/2703448329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 0x5555dff1c710 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7feec809c890 tx=0x7feec80bd040 comp rx=0 tx=0).stop 2024-04-26T18:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 -- 172.21.15.84:0/2703448329 shutdown_connections 2024-04-26T18:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 --2- 172.21.15.84:0/2703448329 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff1f0e0 0x5555dff36a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 --2- 172.21.15.84:0/2703448329 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff14820 0x5555dff1c710 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 --2- 172.21.15.84:0/2703448329 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff13f00 0x5555dff2ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 -- 172.21.15.84:0/2703448329 >> 172.21.15.84:0/2703448329 conn(0x5555dff10410 msgr2=0x5555dff1c000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 -- 172.21.15.84:0/2703448329 shutdown_connections 2024-04-26T18:31:04.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 -- 172.21.15.84:0/2703448329 wait complete. 2024-04-26T18:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 -- start start 2024-04-26T18:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 Processor -- start 2024-04-26T18:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 0x5555dff22e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff14820 0x5555dff23350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff1f0e0 0x5555dff94300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7feed81b1b90 con 0x5555dff13f00 2024-04-26T18:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7feed81b14c0 con 0x5555dff14820 2024-04-26T18:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7feed81ae700 con 0x5555dff1f0e0 2024-04-26T18:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feee0860740 10 client.0 _ll_register_callbacks cb 0x7feecc040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.249+0000 7feed77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 0x5555dff22e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feed77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 0x5555dff22e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41630/0 (socket says 172.21.15.84:41630) 2024-04-26T18:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feed77fe640 1 -- 172.21.15.84:0/3541732909 learned_addr learned my addr 172.21.15.84:0/3541732909 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:04.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feed6ffd640 1 --2- 172.21.15.84:0/3541732909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff14820 0x5555dff23350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feed7fff640 1 --2- 172.21.15.84:0/3541732909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff1f0e0 0x5555dff94300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feed77fe640 1 -- 172.21.15.84:0/3541732909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff1f0e0 msgr2=0x5555dff94300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feed77fe640 1 --2- 172.21.15.84:0/3541732909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff1f0e0 0x5555dff94300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feed77fe640 1 -- 172.21.15.84:0/3541732909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff14820 msgr2=0x5555dff23350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feed77fe640 1 --2- 172.21.15.84:0/3541732909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff14820 0x5555dff23350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feed77fe640 1 -- 172.21.15.84:0/3541732909 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555dff23c50 con 0x5555dff13f00 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feed77fe640 1 --2- 172.21.15.84:0/3541732909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 0x5555dff22e10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feecc03cde0 tx=0x7feecc032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24729fbf30e192e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feeb6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feeb6ffd640 1 -- 172.21.15.84:0/3541732909 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feecc05f070 con 0x5555dff13f00 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feeb6ffd640 1 -- 172.21.15.84:0/3541732909 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7feecc057020 con 0x5555dff13f00 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feee0860740 1 -- 172.21.15.84:0/3541732909 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec80d4560 con 0x5555dff13f00 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feeb6ffd640 1 -- 172.21.15.84:0/3541732909 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7feecc06b050 con 0x5555dff13f00 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feee0860740 10 client.15825 fetch_fsmap learned FSMap version 29 2024-04-26T18:31:04.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feee0860740 1 -- 172.21.15.84:0/3541732909 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5555dff95980 con 0x5555dff13f00 2024-04-26T18:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.250+0000 7feeb6ffd640 1 -- 172.21.15.84:0/3541732909 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec80d4560 con 0x5555dff13f00 2024-04-26T18:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.251+0000 7feeb6ffd640 1 -- 172.21.15.84:0/3541732909 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7feecc031660 con 0x5555dff13f00 2024-04-26T18:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.251+0000 7feee0860740 10 client.15825 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.251+0000 7feee0860740 10 client.15825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.251+0000 7feee0860740 20 client.15825 populate_metadata read hostname 'smithi084' 2024-04-26T18:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.251+0000 7feee0860740 1 --2- 172.21.15.84:0/3541732909 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5555dff96070 0x5555dff98490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.251+0000 7feee0860740 4 client.15825 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.251+0000 7feee0860740 1 -- 172.21.15.84:0/3541732909 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5555dfe81ee0 con 0x5555dff96070 2024-04-26T18:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.251+0000 7feed6ffd640 1 --2- 172.21.15.84:0/3541732909 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5555dff96070 0x5555dff98490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.251+0000 7feeb6ffd640 10 client.15825 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:04.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.251+0000 7feed6ffd640 1 --2- 172.21.15.84:0/3541732909 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5555dff96070 0x5555dff98490 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e06a3ce33bf7ca6 server_cookie=150c9a7988b12add in_seq=0 out_seq=0 2024-04-26T18:31:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feeb6ffd640 1 -- 172.21.15.84:0/3541732909 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5555dfe81ee0 con 0x5555dff96070 2024-04-26T18:31:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feeb6ffd640 10 client.15825 handle_command_reply: tid=0 2024-04-26T18:31:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feee0860740 1 client.15825 shutdown 2024-04-26T18:31:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feee0860740 1 -- 172.21.15.84:0/3541732909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 msgr2=0x5555dff22e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feee0860740 1 --2- 172.21.15.84:0/3541732909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 0x5555dff22e10 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7feecc03cde0 tx=0x7feecc032fc0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feee0860740 1 -- 172.21.15.84:0/3541732909 shutdown_connections 2024-04-26T18:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feee0860740 1 --2- 172.21.15.84:0/3541732909 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5555dff96070 0x5555dff98490 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feee0860740 1 --2- 172.21.15.84:0/3541732909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5555dff1f0e0 0x5555dff94300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feee0860740 1 --2- 172.21.15.84:0/3541732909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5555dff14820 0x5555dff23350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feeb6ffd640 0 client.15825 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feee0860740 1 --2- 172.21.15.84:0/3541732909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5555dff13f00 0x5555dff22e10 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feee0860740 1 -- 172.21.15.84:0/3541732909 >> 172.21.15.84:0/3541732909 conn(0x5555dff10410 msgr2=0x5555dff0f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feee0860740 1 -- 172.21.15.84:0/3541732909 shutdown_connections 2024-04-26T18:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.264+0000 7feee0860740 1 -- 172.21.15.84:0/3541732909 wait complete. 2024-04-26T18:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.265+0000 7feee0860740 20 client.15825 trim_cache size 0 max 16384 2024-04-26T18:31:04.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:31:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:31:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:31:04.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:31:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.265+0000 7feeddb4e640 1 -- 172.21.15.84:0/2788819694 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7feec00a3dc0 msgr2=0x7feec00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.265+0000 7feeddb4e640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7feec00a3dc0 0x7feec00a6280 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7feed0009fd0 tx=0x7feed0004a40 comp rx=0 tx=0).stop 2024-04-26T18:31:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.265+0000 7feeddb4e640 1 -- 172.21.15.84:0/2788819694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed8102220 msgr2=0x7feed80fed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.265+0000 7feeddb4e640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed8102220 0x7feed80fed30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7feec800e9f0 tx=0x7feec800edf0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.265+0000 7feeddb4e640 1 -- 172.21.15.84:0/2788819694 shutdown_connections 2024-04-26T18:31:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.265+0000 7feeddb4e640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7feec00a3dc0 0x7feec00a6280 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.265+0000 7feeddb4e640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7feed8107820 0x7feed80ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.265+0000 7feeddb4e640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7feed8102220 0x7feed80fed30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.265+0000 7feeddb4e640 1 --2- 172.21.15.84:0/2788819694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7feed8101860 0x7feed80fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.265+0000 7feeddb4e640 1 -- 172.21.15.84:0/2788819694 >> 172.21.15.84:0/2788819694 conn(0x7feed80789b0 msgr2=0x7feed806a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:04.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.265+0000 7feeddb4e640 1 -- 172.21.15.84:0/2788819694 shutdown_connections 2024-04-26T18:31:04.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:04.265+0000 7feeddb4e640 1 -- 172.21.15.84:0/2788819694 wait complete. 2024-04-26T18:31:04.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:04 smithi174 ceph-mon[32705]: pgmap v96: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 52 KiB/s wr, 37 op/s 2024-04-26T18:31:04.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:04 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:31:04.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:04 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:31:04.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:04 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:31:04.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:04 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:31:05.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:04 smithi084 ceph-mon[29141]: pgmap v96: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 52 KiB/s wr, 37 op/s 2024-04-26T18:31:05.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:04 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:31:05.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:04 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:31:05.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:04 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:31:05.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:04 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:31:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:04 smithi103 ceph-mon[32727]: pgmap v96: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 52 KiB/s wr, 37 op/s 2024-04-26T18:31:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:04 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:31:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:04 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:31:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:04 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:31:05.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:04 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:31:06.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:06 smithi174 ceph-mon[32705]: pgmap v97: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-26T18:31:07.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:06 smithi084 ceph-mon[29141]: pgmap v97: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-26T18:31:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:06 smithi103 ceph-mon[32727]: pgmap v97: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-26T18:31:09.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:08 smithi084 ceph-mon[29141]: pgmap v98: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-26T18:31:09.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:08 smithi103 ceph-mon[32727]: pgmap v98: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-26T18:31:09.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:08 smithi174 ceph-mon[32705]: pgmap v98: 97 pgs: 97 active+clean; 2.2 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-26T18:31:11.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:10 smithi084 ceph-mon[29141]: pgmap v99: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 37 op/s 2024-04-26T18:31:11.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:10 smithi103 ceph-mon[32727]: pgmap v99: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 37 op/s 2024-04-26T18:31:11.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:10 smithi174 ceph-mon[32705]: pgmap v99: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 54 KiB/s wr, 37 op/s 2024-04-26T18:31:13.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:12 smithi084 ceph-mon[29141]: pgmap v100: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-04-26T18:31:13.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:12 smithi103 ceph-mon[32727]: pgmap v100: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-04-26T18:31:13.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:12 smithi174 ceph-mon[32705]: pgmap v100: 97 pgs: 97 active+clean; 2.3 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 46 KiB/s wr, 30 op/s 2024-04-26T18:31:15.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:14 smithi084 ceph-mon[29141]: pgmap v101: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 56 KiB/s wr, 37 op/s 2024-04-26T18:31:15.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:14 smithi103 ceph-mon[32727]: pgmap v101: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 56 KiB/s wr, 37 op/s 2024-04-26T18:31:15.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:14 smithi174 ceph-mon[32705]: pgmap v101: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 56 KiB/s wr, 37 op/s 2024-04-26T18:31:17.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:16 smithi084 ceph-mon[29141]: pgmap v102: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:17.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:16 smithi103 ceph-mon[32727]: pgmap v102: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:17.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:16 smithi174 ceph-mon[32705]: pgmap v102: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.361+0000 7f1b77cd1640 1 -- 172.21.15.84:0/3973235092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b7006bb00 msgr2=0x7f1b7010f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.361+0000 7f1b77cd1640 1 --2- 172.21.15.84:0/3973235092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b7006bb00 0x7f1b7010f770 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f1b68009ca0 tx=0x7f1b6802a8c0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.361+0000 7f1b77cd1640 1 -- 172.21.15.84:0/3973235092 shutdown_connections 2024-04-26T18:31:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.361+0000 7f1b77cd1640 1 --2- 172.21.15.84:0/3973235092 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b7006bb00 0x7f1b7010f770 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.365 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.361+0000 7f1b77cd1640 1 --2- 172.21.15.84:0/3973235092 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b7006b160 0x7f1b7006b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.361+0000 7f1b77cd1640 1 --2- 172.21.15.84:0/3973235092 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b70108920 0x7f1b70108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.361+0000 7f1b77cd1640 1 -- 172.21.15.84:0/3973235092 >> 172.21.15.84:0/3973235092 conn(0x7f1b70077560 msgr2=0x7f1b70077970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.362+0000 7f1b77cd1640 1 -- 172.21.15.84:0/3973235092 shutdown_connections 2024-04-26T18:31:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.362+0000 7f1b77cd1640 1 -- 172.21.15.84:0/3973235092 wait complete. 2024-04-26T18:31:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.362+0000 7f1b77cd1640 1 Processor -- start 2024-04-26T18:31:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.362+0000 7f1b77cd1640 1 -- start start 2024-04-26T18:31:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.362+0000 7f1b77cd1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b7006b160 0x7f1b70198fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.362+0000 7f1b77cd1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b7006bb00 0x7f1b70199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.362+0000 7f1b77cd1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b70108920 0x7f1b7019d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.366 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.362+0000 7f1b77cd1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70112bc0 con 0x7f1b7006b160 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.362+0000 7f1b77cd1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70112a40 con 0x7f1b70108920 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.362+0000 7f1b77cd1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1b70112d40 con 0x7f1b7006bb00 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.362+0000 7f1b75a6d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b7006b160 0x7f1b70198fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b75a6d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b7006b160 0x7f1b70198fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55074/0 (socket says 172.21.15.84:55074) 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b75a6d640 1 -- 172.21.15.84:0/3297705845 learned_addr learned my addr 172.21.15.84:0/3297705845 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b7526c640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b7006bb00 0x7f1b70199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b7626e640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b70108920 0x7f1b7019d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b75a6d640 1 -- 172.21.15.84:0/3297705845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b7006bb00 msgr2=0x7f1b70199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b75a6d640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b7006bb00 0x7f1b70199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b75a6d640 1 -- 172.21.15.84:0/3297705845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b70108920 msgr2=0x7f1b7019d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b75a6d640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b70108920 0x7f1b7019d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b75a6d640 1 -- 172.21.15.84:0/3297705845 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7019dfb0 con 0x7f1b7006b160 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b7626e640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b70108920 0x7f1b7019d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:31:17.367 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b75a6d640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b7006b160 0x7f1b70198fc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000e9f0 tx=0x7f1b6000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b66ffd640 1 -- 172.21.15.84:0/3297705845 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6000cd80 con 0x7f1b7006b160 2024-04-26T18:31:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b66ffd640 1 -- 172.21.15.84:0/3297705845 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b600054d0 con 0x7f1b7006b160 2024-04-26T18:31:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b77cd1640 1 -- 172.21.15.84:0/3297705845 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7019e2a0 con 0x7f1b7006b160 2024-04-26T18:31:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b66ffd640 1 -- 172.21.15.84:0/3297705845 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b60010690 con 0x7f1b7006b160 2024-04-26T18:31:17.368 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.363+0000 7f1b77cd1640 1 -- 172.21.15.84:0/3297705845 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b701a5b30 con 0x7f1b7006b160 2024-04-26T18:31:17.375 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.366+0000 7f1b66ffd640 1 -- 172.21.15.84:0/3297705845 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1b60004e90 con 0x7f1b7006b160 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.366+0000 7f1b7a9ed740 1 Processor -- start 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7a9ed740 1 -- start start 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7a9ed740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 0x56178ffe82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7a9ed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178ffe8820 0x56178fff2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7a9ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178fff30e0 0x56178fff54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7a9ed740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56178ff4e460 con 0x56178ffe7f00 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7a9ed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56178ff08590 con 0x56178ffe8820 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7a9ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56178ff4a420 con 0x56178fff30e0 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7626e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 0x56178ffe82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7626e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 0x56178ffe82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55076/0 (socket says 172.21.15.84:55076) 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7626e640 1 -- 172.21.15.84:0/537032432 learned_addr learned my addr 172.21.15.84:0/537032432 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b66ffd640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1b5c0a3dc0 0x7f1b5c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7526c640 1 --2- 172.21.15.84:0/537032432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178fff30e0 0x56178fff54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b75a6d640 1 --2- 172.21.15.84:0/537032432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178ffe8820 0x56178fff2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7626e640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1b5c0a3dc0 0x7f1b5c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b66ffd640 1 -- 172.21.15.84:0/3297705845 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f1b600d0c40 con 0x7f1b7006b160 2024-04-26T18:31:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7626e640 1 -- 172.21.15.84:0/537032432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178fff30e0 msgr2=0x56178fff54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7626e640 1 --2- 172.21.15.84:0/537032432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178fff30e0 0x56178fff54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7626e640 1 -- 172.21.15.84:0/537032432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178ffe8820 msgr2=0x56178fff2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.367+0000 7f1b7626e640 1 --2- 172.21.15.84:0/537032432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178ffe8820 0x56178fff2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.368+0000 7f1b7626e640 1 -- 172.21.15.84:0/537032432 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56178ff150d0 con 0x56178ffe7f00 2024-04-26T18:31:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.368+0000 7f1b75a6d640 1 --2- 172.21.15.84:0/537032432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178ffe8820 0x56178fff2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:31:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.368+0000 7f1b7626e640 1 --2- 172.21.15.84:0/537032432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 0x56178ffe82e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1b68004af0 tx=0x7f1b68031de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae98f86f3f5b5d5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.368+0000 7f1b5bfff640 1 -- 172.21.15.84:0/537032432 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68004d40 con 0x56178ffe7f00 2024-04-26T18:31:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.368+0000 7f1b5bfff640 1 -- 172.21.15.84:0/537032432 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b6804d070 con 0x56178ffe7f00 2024-04-26T18:31:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.368+0000 7f1b7626e640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1b5c0a3dc0 0x7f1b5c0a6280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1b6804c000 tx=0x7f1b6804a000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:17.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.368+0000 7f1b5bfff640 1 -- 172.21.15.84:0/537032432 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68004680 con 0x56178ffe7f00 2024-04-26T18:31:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.369+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/537032432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 msgr2=0x56178ffe82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.369+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/537032432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 0x56178ffe82e0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1b68004af0 tx=0x7f1b68031de0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.369+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/537032432 shutdown_connections 2024-04-26T18:31:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.369+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/537032432 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178fff30e0 0x56178fff54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.369+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/537032432 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178ffe8820 0x56178fff2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.369+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/537032432 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 0x56178ffe82e0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.369+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/537032432 >> 172.21.15.84:0/537032432 conn(0x56178ffe4410 msgr2=0x56178ffe47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.369+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/537032432 shutdown_connections 2024-04-26T18:31:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.369+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/537032432 wait complete. 2024-04-26T18:31:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.369+0000 7f1b7a9ed740 1 -- start start 2024-04-26T18:31:17.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.369+0000 7f1b7a9ed740 1 Processor -- start 2024-04-26T18:31:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7a9ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe7f00 0x56178fff0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7a9ed740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe8820 0x56178fff0670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7a9ed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178fff30e0 0x56178fff0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7a9ed740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56178fff5d10 con 0x56178ffe8820 2024-04-26T18:31:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7a9ed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56178fed5b60 con 0x56178fff30e0 2024-04-26T18:31:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7a9ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56178fff5e90 con 0x56178ffe7f00 2024-04-26T18:31:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7a9ed740 10 client.0 _ll_register_callbacks cb 0x56178fefa2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7526c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe8820 0x56178fff0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7526c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe8820 0x56178fff0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55088/0 (socket says 172.21.15.84:55088) 2024-04-26T18:31:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7526c640 1 -- 172.21.15.84:0/2645845378 learned_addr learned my addr 172.21.15.84:0/2645845378 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b75a6d640 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe7f00 0x56178fff0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7626e640 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178fff30e0 0x56178fff0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7526c640 1 -- 172.21.15.84:0/2645845378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe7f00 msgr2=0x56178fff0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7526c640 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe7f00 0x56178fff0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7526c640 1 -- 172.21.15.84:0/2645845378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178fff30e0 msgr2=0x56178fff0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7526c640 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178fff30e0 0x56178fff0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7526c640 1 -- 172.21.15.84:0/2645845378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56178fc60210 con 0x56178ffe8820 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7626e640 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178fff30e0 0x56178fff0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b7526c640 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe8820 0x56178fff0670 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00bad0 tx=0x7f1b6c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10be1051194a7d3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b597fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b597fa640 1 -- 172.21.15.84:0/2645845378 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c003e00 con 0x56178ffe8820 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.370+0000 7f1b597fa640 1 -- 172.21.15.84:0/2645845378 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f1b6c011440 con 0x56178ffe8820 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.371+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/2645845378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b6804d070 con 0x56178ffe8820 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.371+0000 7f1b597fa640 1 -- 172.21.15.84:0/2645845378 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b6c011680 con 0x56178ffe8820 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.371+0000 7f1b7a9ed740 10 client.15837 fetch_fsmap learned FSMap version 29 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.371+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/2645845378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56178fd2c620 con 0x56178ffe8820 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.371+0000 7f1b597fa640 1 -- 172.21.15.84:0/2645845378 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b6804d070 con 0x56178ffe8820 2024-04-26T18:31:17.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.371+0000 7f1b597fa640 1 -- 172.21.15.84:0/2645845378 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1b6c01ba40 con 0x56178ffe8820 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.371+0000 7f1b7a9ed740 10 client.15837 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.371+0000 7f1b7a9ed740 10 client.15837 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.371+0000 7f1b7a9ed740 20 client.15837 populate_metadata read hostname 'smithi084' 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.371+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5617900033e0 0x561790005810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.371+0000 7f1b7a9ed740 4 client.15837 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.371+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/2645845378 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56178ff5c070 con 0x5617900033e0 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.371+0000 7f1b75a6d640 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5617900033e0 0x561790005810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.372+0000 7f1b75a6d640 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5617900033e0 0x561790005810 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd83af9d2af23235 server_cookie=cc9691b0b016d0dd in_seq=0 out_seq=0 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.372+0000 7f1b597fa640 10 client.15837 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.372+0000 7f1b597fa640 1 -- 172.21.15.84:0/2645845378 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56178ff5c070 con 0x5617900033e0 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.372+0000 7f1b597fa640 10 client.15837 handle_command_reply: tid=0 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.372+0000 7f1b7a9ed740 1 client.15837 shutdown 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.373+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/2645845378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe8820 msgr2=0x56178fff0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:17.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.373+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe8820 0x56178fff0670 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c00bad0 tx=0x7f1b6c00bf90 comp rx=0 tx=0).stop 2024-04-26T18:31:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.373+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/2645845378 shutdown_connections 2024-04-26T18:31:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.373+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5617900033e0 0x561790005810 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.373+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178fff30e0 0x56178fff0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.373+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe8820 0x56178fff0670 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.373+0000 7f1b597fa640 0 client.15837 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.373+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/2645845378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe7f00 0x56178fff0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.373+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/2645845378 >> 172.21.15.84:0/2645845378 conn(0x56178ffe4410 msgr2=0x56178ffea190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.373+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/2645845378 shutdown_connections 2024-04-26T18:31:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.373+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/2645845378 wait complete. 2024-04-26T18:31:17.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.373+0000 7f1b7a9ed740 20 client.15837 trim_cache size 0 max 16384 2024-04-26T18:31:17.392 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7a9ed740 1 Processor -- start 2024-04-26T18:31:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7a9ed740 1 -- start start 2024-04-26T18:31:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7a9ed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178ffe7f00 0x56178fffebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7a9ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe8820 0x56178ffea910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7a9ed740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178fff30e0 0x56178ffe3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7a9ed740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701ae520 con 0x56178fff30e0 2024-04-26T18:31:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7a9ed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701b0060 con 0x56178ffe7f00 2024-04-26T18:31:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7a9ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701aebf0 con 0x56178ffe8820 2024-04-26T18:31:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7626e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178fff30e0 0x56178ffe3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7626e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178fff30e0 0x56178ffe3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55092/0 (socket says 172.21.15.84:55092) 2024-04-26T18:31:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7626e640 1 -- 172.21.15.84:0/595039602 learned_addr learned my addr 172.21.15.84:0/595039602 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b75a6d640 1 --2- 172.21.15.84:0/595039602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178ffe7f00 0x56178fffebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7526c640 1 --2- 172.21.15.84:0/595039602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe8820 0x56178ffea910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.393 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7626e640 1 -- 172.21.15.84:0/595039602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe8820 msgr2=0x56178ffea910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7626e640 1 --2- 172.21.15.84:0/595039602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe8820 0x56178ffea910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7626e640 1 -- 172.21.15.84:0/595039602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178ffe7f00 msgr2=0x56178fffebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7626e640 1 --2- 172.21.15.84:0/595039602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178ffe7f00 0x56178fffebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7626e640 1 -- 172.21.15.84:0/595039602 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56178fff54d0 con 0x56178fff30e0 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b75a6d640 1 --2- 172.21.15.84:0/595039602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178ffe7f00 0x56178fffebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.388+0000 7f1b7626e640 1 --2- 172.21.15.84:0/595039602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178fff30e0 0x56178ffe3430 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1b68031e50 tx=0x7f1b68031c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab69c8c22e520b18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.389+0000 7f1b5b7fe640 1 -- 172.21.15.84:0/595039602 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68058020 con 0x56178fff30e0 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.389+0000 7f1b5b7fe640 1 -- 172.21.15.84:0/595039602 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b6804d070 con 0x56178fff30e0 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.389+0000 7f1b5b7fe640 1 -- 172.21.15.84:0/595039602 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b68004360 con 0x56178fff30e0 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.389+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/595039602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178fff30e0 msgr2=0x56178ffe3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.389+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/595039602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178fff30e0 0x56178ffe3430 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1b68031e50 tx=0x7f1b68031c80 comp rx=0 tx=0).stop 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.389+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/595039602 shutdown_connections 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.389+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/595039602 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178fff30e0 0x56178ffe3430 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.389+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/595039602 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe8820 0x56178ffea910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.389+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/595039602 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178ffe7f00 0x56178fffebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.389+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/595039602 >> 172.21.15.84:0/595039602 conn(0x56178ffe4410 msgr2=0x56178ffe2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.389+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/595039602 shutdown_connections 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.389+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/595039602 wait complete. 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b7a9ed740 1 -- start start 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b7a9ed740 1 Processor -- start 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b7a9ed740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 0x56178fffb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b7a9ed740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe8820 0x56178ffecb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b7a9ed740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178fff30e0 0x56178ffed0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b7a9ed740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701af990 con 0x56178ffe7f00 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b7a9ed740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701af2c0 con 0x56178fff30e0 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b7a9ed740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1b701ac500 con 0x56178ffe8820 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b7a9ed740 10 client.0 _ll_register_callbacks cb 0x7f1b6c004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:17.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b75a6d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 0x56178fffb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b75a6d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 0x56178fffb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55094/0 (socket says 172.21.15.84:55094) 2024-04-26T18:31:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b75a6d640 1 -- 172.21.15.84:0/784143252 learned_addr learned my addr 172.21.15.84:0/784143252 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b7526c640 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe8820 0x56178ffecb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b7626e640 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178fff30e0 0x56178ffed0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b75a6d640 1 -- 172.21.15.84:0/784143252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe8820 msgr2=0x56178ffecb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b75a6d640 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe8820 0x56178ffecb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b75a6d640 1 -- 172.21.15.84:0/784143252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178fff30e0 msgr2=0x56178ffed0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b75a6d640 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178fff30e0 0x56178ffed0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b75a6d640 1 -- 172.21.15.84:0/784143252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56178ffed8d0 con 0x56178ffe7f00 2024-04-26T18:31:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.390+0000 7f1b7626e640 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178fff30e0 0x56178ffed0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:31:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b75a6d640 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 0x56178fffb540 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1b6009cb60 tx=0x7f1b60005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bc6673e0e2ff6a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b597fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:31:17.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b597fa640 1 -- 172.21.15.84:0/784143252 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b600a0f80 con 0x56178ffe7f00 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b597fa640 1 -- 172.21.15.84:0/784143252 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f1b600ce600 con 0x56178ffe7f00 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/784143252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b6804d070 con 0x56178ffe7f00 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b597fa640 1 -- 172.21.15.84:0/784143252 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b600a5840 con 0x56178ffe7f00 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b7a9ed740 10 client.15843 fetch_fsmap learned FSMap version 29 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/784143252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561790069aa0 con 0x56178ffe7f00 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b597fa640 1 -- 172.21.15.84:0/784143252 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b6804d070 con 0x56178ffe7f00 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b597fa640 1 -- 172.21.15.84:0/784143252 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f1b600a53b0 con 0x56178ffe7f00 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b7a9ed740 10 client.15843 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b7a9ed740 10 client.15843 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b7a9ed740 20 client.15843 populate_metadata read hostname 'smithi084' 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56179006a170 0x56179006c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b7a9ed740 4 client.15843 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.391+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/784143252 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56178ff55ee0 con 0x56179006a170 2024-04-26T18:31:17.397 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.392+0000 7f1b7526c640 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56179006a170 0x56179006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.392+0000 7f1b7526c640 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56179006a170 0x56179006c590 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdf59fbc99c1ba45 server_cookie=8d5bb102c817cec3 in_seq=0 out_seq=0 2024-04-26T18:31:17.398 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.392+0000 7f1b597fa640 10 client.15843 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b597fa640 1 -- 172.21.15.84:0/784143252 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56178ff55ee0 con 0x56179006a170 2024-04-26T18:31:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b597fa640 10 client.15843 handle_command_reply: tid=0 2024-04-26T18:31:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b7a9ed740 1 client.15843 shutdown 2024-04-26T18:31:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/784143252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 msgr2=0x56178fffb540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 0x56178fffb540 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1b6009cb60 tx=0x7f1b60005240 comp rx=0 tx=0).stop 2024-04-26T18:31:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/784143252 shutdown_connections 2024-04-26T18:31:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56179006a170 0x56179006c590 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56178fff30e0 0x56178ffed0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56178ffe8820 0x56178ffecb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b597fa640 0 client.15843 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b7a9ed740 1 --2- 172.21.15.84:0/784143252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56178ffe7f00 0x56178fffb540 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/784143252 >> 172.21.15.84:0/784143252 conn(0x56178ffe4410 msgr2=0x56178ffe35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:17.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/784143252 shutdown_connections 2024-04-26T18:31:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b7a9ed740 1 -- 172.21.15.84:0/784143252 wait complete. 2024-04-26T18:31:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.407+0000 7f1b7a9ed740 20 client.15843 trim_cache size 0 max 16384 2024-04-26T18:31:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:31:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:31:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:31:17.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:31:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.408+0000 7f1b77cd1640 1 -- 172.21.15.84:0/3297705845 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1b5c0a3dc0 msgr2=0x7f1b5c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.408+0000 7f1b77cd1640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1b5c0a3dc0 0x7f1b5c0a6280 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1b6804c000 tx=0x7f1b6804a000 comp rx=0 tx=0).stop 2024-04-26T18:31:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.408+0000 7f1b77cd1640 1 -- 172.21.15.84:0/3297705845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b7006b160 msgr2=0x7f1b70198fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.408+0000 7f1b77cd1640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b7006b160 0x7f1b70198fc0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000e9f0 tx=0x7f1b6000edf0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.408+0000 7f1b77cd1640 1 -- 172.21.15.84:0/3297705845 shutdown_connections 2024-04-26T18:31:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.408+0000 7f1b77cd1640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f1b5c0a3dc0 0x7f1b5c0a6280 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.408+0000 7f1b77cd1640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1b70108920 0x7f1b7019d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.408+0000 7f1b77cd1640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1b7006bb00 0x7f1b70199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.408+0000 7f1b77cd1640 1 --2- 172.21.15.84:0/3297705845 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1b7006b160 0x7f1b70198fc0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.408+0000 7f1b77cd1640 1 -- 172.21.15.84:0/3297705845 >> 172.21.15.84:0/3297705845 conn(0x7f1b70077560 msgr2=0x7f1b70100690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.408+0000 7f1b77cd1640 1 -- 172.21.15.84:0/3297705845 shutdown_connections 2024-04-26T18:31:17.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:17.408+0000 7f1b77cd1640 1 -- 172.21.15.84:0/3297705845 wait complete. 2024-04-26T18:31:18.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:18 smithi174 ceph-mon[32705]: pgmap v103: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:19.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:18 smithi084 ceph-mon[29141]: pgmap v103: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:19.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:18 smithi103 ceph-mon[32727]: pgmap v103: 97 pgs: 97 active+clean; 2.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:21.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:20 smithi084 ceph-mon[29141]: pgmap v104: 97 pgs: 97 active+clean; 2.5 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-04-26T18:31:21.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:20 smithi103 ceph-mon[32727]: pgmap v104: 97 pgs: 97 active+clean; 2.5 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-04-26T18:31:21.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:20 smithi174 ceph-mon[32705]: pgmap v104: 97 pgs: 97 active+clean; 2.5 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-04-26T18:31:23.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:22 smithi084 ceph-mon[29141]: pgmap v105: 97 pgs: 97 active+clean; 2.5 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:23.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:22 smithi103 ceph-mon[32727]: pgmap v105: 97 pgs: 97 active+clean; 2.5 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:23.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:22 smithi174 ceph-mon[32705]: pgmap v105: 97 pgs: 97 active+clean; 2.5 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:23.889 INFO:tasks.workunit.client.0.smithi084.stdout:1/600: open fd = 3 2024-04-26T18:31:23.889 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:2/600: open fd = 3 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:3/600: open fd = 3 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:4/600: open fd = 3 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.890 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:5/600: open fd = 3 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:6/600: open fd = 3 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:7/600: open fd = 3 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:8/600: open fd = 3 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.891 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.892 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.892 INFO:tasks.workunit.client.0.smithi084.stdout:9/600: open fd = 3 2024-04-26T18:31:23.892 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.892 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.892 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.892 INFO:tasks.workunit.client.0.smithi084.stdout:10/600: open fd = 3 2024-04-26T18:31:23.892 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.892 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.892 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.892 INFO:tasks.workunit.client.0.smithi084.stdout:11/600: open fd = 3 2024-04-26T18:31:23.892 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.892 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.892 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:12/600: open fd = 3 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:13/600: open fd = 3 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:14/600: open fd = 3 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:15/600: open fd = 3 2024-04-26T18:31:23.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi084.stdout:16/600: open fd = 3 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi084.stdout:17/600: open fd = 3 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi084.stdout:18/600: open fd = 3 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.894 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:19/600: open fd = 3 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:20/600: open fd = 3 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:21/600: open fd = 3 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:22/600: open fd = 3 2024-04-26T18:31:23.895 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:23/600: open fd = 3 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:24/600: open fd = 3 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:25/600: open fd = 3 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.896 INFO:tasks.workunit.client.0.smithi084.stdout:26/600: open fd = 3 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:27/600: open fd = 3 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:28/600: open fd = 3 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:29/600: open fd = 3 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.897 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:30/600: open fd = 3 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:31/600: open fd = 3 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:32/600: open fd = 3 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:33/600: open fd = 3 2024-04-26T18:31:23.898 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:34/600: open fd = 3 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:35/600: open fd = 3 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:36/600: open fd = 3 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:37/600: open fd = 3 2024-04-26T18:31:23.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:38/600: open fd = 3 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:39/600: open fd = 3 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:40/600: open fd = 3 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.900 INFO:tasks.workunit.client.0.smithi084.stdout:41/600: open fd = 3 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:42/600: open fd = 3 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:43/600: open fd = 3 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:44/600: open fd = 3 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:45/600: open fd = 3 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:46/600: open fd = 3 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:47/600: open fd = 3 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:48/600: open fd = 3 2024-04-26T18:31:23.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:49/600: open fd = 3 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:50/600: open fd = 3 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:51/600: open fd = 3 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.903 INFO:tasks.workunit.client.0.smithi084.stdout:52/600: open fd = 3 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:53/600: open fd = 3 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:54/600: open fd = 3 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:55/600: open fd = 3 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:56/600: open fd = 3 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:57/600: open fd = 3 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:58/600: open fd = 3 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:59/600: open fd = 3 2024-04-26T18:31:23.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:60/600: open fd = 3 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:61/600: open fd = 3 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:62/600: open fd = 3 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:23.906 INFO:tasks.workunit.client.0.smithi084.stdout:63/600: open fd = 3 2024-04-26T18:31:23.907 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.907 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:31:23.907 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:31:25.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:24 smithi084 ceph-mon[29141]: pgmap v106: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-04-26T18:31:25.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:24 smithi103 ceph-mon[32727]: pgmap v106: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-04-26T18:31:25.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:24 smithi174 ceph-mon[32705]: pgmap v106: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-04-26T18:31:27.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:26 smithi084 ceph-mon[29141]: pgmap v107: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:27.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:26 smithi103 ceph-mon[32727]: pgmap v107: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:27.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:26 smithi174 ceph-mon[32705]: pgmap v107: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:29.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:28 smithi084 ceph-mon[29141]: pgmap v108: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:29.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:28 smithi103 ceph-mon[32727]: pgmap v108: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:29.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:28 smithi174 ceph-mon[32705]: pgmap v108: 97 pgs: 97 active+clean; 2.6 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.504+0000 7fe7862a2640 1 -- 172.21.15.84:0/351202983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe780103be0 msgr2=0x7fe780103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.504+0000 7fe7862a2640 1 --2- 172.21.15.84:0/351202983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe780103be0 0x7fe780103fc0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fe770009900 tx=0x7fe77002a480 comp rx=0 tx=0).stop 2024-04-26T18:31:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.504+0000 7fe7862a2640 1 -- 172.21.15.84:0/351202983 shutdown_connections 2024-04-26T18:31:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.504+0000 7fe7862a2640 1 --2- 172.21.15.84:0/351202983 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe78010cfb0 0x7fe78010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.504+0000 7fe7862a2640 1 --2- 172.21.15.84:0/351202983 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe780104500 0x7fe78010c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.504+0000 7fe7862a2640 1 --2- 172.21.15.84:0/351202983 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe780103be0 0x7fe780103fc0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.505+0000 7fe7862a2640 1 -- 172.21.15.84:0/351202983 >> 172.21.15.84:0/351202983 conn(0x7fe780077bb0 msgr2=0x7fe780077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.505+0000 7fe7862a2640 1 -- 172.21.15.84:0/351202983 shutdown_connections 2024-04-26T18:31:30.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.505+0000 7fe7862a2640 1 -- 172.21.15.84:0/351202983 wait complete. 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.505+0000 7fe7862a2640 1 Processor -- start 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.505+0000 7fe7862a2640 1 -- start start 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.505+0000 7fe7862a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe780103be0 0x7fe7801990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.505+0000 7fe7862a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe780104500 0x7fe7801995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.505+0000 7fe7862a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe78010cfb0 0x7fe78019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.505+0000 7fe7862a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe780112c50 con 0x7fe78010cfb0 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.505+0000 7fe7862a2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe780112ad0 con 0x7fe780103be0 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.505+0000 7fe7862a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe780112dd0 con 0x7fe780104500 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.505+0000 7fe78483f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe78010cfb0 0x7fe78019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe78483f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe78010cfb0 0x7fe78019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38586/0 (socket says 172.21.15.84:38586) 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe78483f640 1 -- 172.21.15.84:0/1319330495 learned_addr learned my addr 172.21.15.84:0/1319330495 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe777fff640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe780104500 0x7fe7801995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe77ffff640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe780103be0 0x7fe7801990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe78483f640 1 -- 172.21.15.84:0/1319330495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe780104500 msgr2=0x7fe7801995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe78483f640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe780104500 0x7fe7801995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe78483f640 1 -- 172.21.15.84:0/1319330495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe780103be0 msgr2=0x7fe7801990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe78483f640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe780103be0 0x7fe7801990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe78483f640 1 -- 172.21.15.84:0/1319330495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe78019e0a0 con 0x7fe78010cfb0 2024-04-26T18:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe77ffff640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe780103be0 0x7fe7801990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe777fff640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe780104500 0x7fe7801995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe78483f640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe78010cfb0 0x7fe78019d970 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe77800ccb0 tx=0x7fe77800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe77dffb640 1 -- 172.21.15.84:0/1319330495 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77800f450 con 0x7fe78010cfb0 2024-04-26T18:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe7862a2640 1 -- 172.21.15.84:0/1319330495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe78019e390 con 0x7fe78010cfb0 2024-04-26T18:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe77dffb640 1 -- 172.21.15.84:0/1319330495 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe77800faa0 con 0x7fe78010cfb0 2024-04-26T18:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe77dffb640 1 -- 172.21.15.84:0/1319330495 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe778018b00 con 0x7fe78010cfb0 2024-04-26T18:31:30.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.506+0000 7fe7862a2640 1 -- 172.21.15.84:0/1319330495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7801a5c20 con 0x7fe78010cfb0 2024-04-26T18:31:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe77dffb640 1 -- 172.21.15.84:0/1319330495 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fe77800fc50 con 0x7fe78010cfb0 2024-04-26T18:31:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe788fbd740 1 Processor -- start 2024-04-26T18:31:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe788fbd740 1 -- start start 2024-04-26T18:31:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe77dffb640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe76c0a3dc0 0x7fe76c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe788fbd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85354f00 0x556a853552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe788fbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85355820 0x556a8535fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe788fbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a853600e0 0x556a853624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe788fbd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556a852bb460 con 0x556a85354f00 2024-04-26T18:31:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe788fbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556a85275590 con 0x556a853600e0 2024-04-26T18:31:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe788fbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556a852b7420 con 0x556a85355820 2024-04-26T18:31:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe77ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85354f00 0x556a853552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe77dffb640 1 -- 172.21.15.84:0/1319330495 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fe7780d1030 con 0x7fe78010cfb0 2024-04-26T18:31:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe777fff640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe76c0a3dc0 0x7fe76c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe77ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85354f00 0x556a853552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38598/0 (socket says 172.21.15.84:38598) 2024-04-26T18:31:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.509+0000 7fe77ffff640 1 -- 172.21.15.84:0/3782347926 learned_addr learned my addr 172.21.15.84:0/3782347926 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.510+0000 7fe78483f640 1 --2- 172.21.15.84:0/3782347926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a853600e0 0x556a853624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.510+0000 7fe777fff640 1 --2- 172.21.15.84:0/3782347926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85355820 0x556a8535fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.510+0000 7fe77ffff640 1 -- 172.21.15.84:0/3782347926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85355820 msgr2=0x556a8535fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.510+0000 7fe77ffff640 1 --2- 172.21.15.84:0/3782347926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85355820 0x556a8535fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.510+0000 7fe77ffff640 1 -- 172.21.15.84:0/3782347926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a853600e0 msgr2=0x556a853624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.510+0000 7fe77ffff640 1 --2- 172.21.15.84:0/3782347926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a853600e0 0x556a853624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.510+0000 7fe77ffff640 1 -- 172.21.15.84:0/3782347926 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a852820d0 con 0x556a85354f00 2024-04-26T18:31:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.510+0000 7fe777fff640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe76c0a3dc0 0x7fe76c0a6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe78010eb20 tx=0x7fe768009e70 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.510+0000 7fe77ffff640 1 --2- 172.21.15.84:0/3782347926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85354f00 0x556a853552e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe77002a9a0 tx=0x7fe770031e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24672e2ad273c06c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.510+0000 7fe7767fc640 1 -- 172.21.15.84:0/3782347926 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe77003f070 con 0x556a85354f00 2024-04-26T18:31:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.510+0000 7fe7767fc640 1 -- 172.21.15.84:0/3782347926 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7700316a0 con 0x556a85354f00 2024-04-26T18:31:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.510+0000 7fe7767fc640 1 -- 172.21.15.84:0/3782347926 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe770035070 con 0x556a85354f00 2024-04-26T18:31:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.511+0000 7fe788fbd740 1 -- 172.21.15.84:0/3782347926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85354f00 msgr2=0x556a853552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.511+0000 7fe788fbd740 1 --2- 172.21.15.84:0/3782347926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85354f00 0x556a853552e0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fe77002a9a0 tx=0x7fe770031e60 comp rx=0 tx=0).stop 2024-04-26T18:31:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.511+0000 7fe788fbd740 1 -- 172.21.15.84:0/3782347926 shutdown_connections 2024-04-26T18:31:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.511+0000 7fe788fbd740 1 --2- 172.21.15.84:0/3782347926 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a853600e0 0x556a853624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.511+0000 7fe788fbd740 1 --2- 172.21.15.84:0/3782347926 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85355820 0x556a8535fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.511+0000 7fe788fbd740 1 --2- 172.21.15.84:0/3782347926 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85354f00 0x556a853552e0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.511+0000 7fe788fbd740 1 -- 172.21.15.84:0/3782347926 >> 172.21.15.84:0/3782347926 conn(0x556a85351410 msgr2=0x556a853517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.511+0000 7fe788fbd740 1 -- 172.21.15.84:0/3782347926 shutdown_connections 2024-04-26T18:31:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.511+0000 7fe788fbd740 1 -- 172.21.15.84:0/3782347926 wait complete. 2024-04-26T18:31:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.511+0000 7fe788fbd740 1 -- start start 2024-04-26T18:31:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe788fbd740 1 Processor -- start 2024-04-26T18:31:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe788fbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85354f00 0x556a8535d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe788fbd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85355820 0x556a8535d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe788fbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a853600e0 0x556a8535dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe788fbd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556a85362cb0 con 0x556a85355820 2024-04-26T18:31:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe788fbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556a85242b60 con 0x556a85354f00 2024-04-26T18:31:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe788fbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556a85362e30 con 0x556a853600e0 2024-04-26T18:31:30.521 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe788fbd740 10 client.0 _ll_register_callbacks cb 0x556a852672b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe777fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85355820 0x556a8535d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe777fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85355820 0x556a8535d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38600/0 (socket says 172.21.15.84:38600) 2024-04-26T18:31:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe777fff640 1 -- 172.21.15.84:0/4224716999 learned_addr learned my addr 172.21.15.84:0/4224716999 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe77ffff640 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85354f00 0x556a8535d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe78483f640 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a853600e0 0x556a8535dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe777fff640 1 -- 172.21.15.84:0/4224716999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a853600e0 msgr2=0x556a8535dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe777fff640 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a853600e0 0x556a8535dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe777fff640 1 -- 172.21.15.84:0/4224716999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85354f00 msgr2=0x556a8535d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe777fff640 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85354f00 0x556a8535d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe777fff640 1 -- 172.21.15.84:0/4224716999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a84fcd210 con 0x556a85355820 2024-04-26T18:31:30.522 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe77ffff640 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85354f00 0x556a8535d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:31:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe777fff640 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85355820 0x556a8535d570 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe7680176b0 tx=0x7fe7680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c3c5977eb3b997c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.512+0000 7fe75f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:31:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe75f7fe640 1 -- 172.21.15.84:0/4224716999 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe768027060 con 0x556a85355820 2024-04-26T18:31:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe75f7fe640 1 -- 172.21.15.84:0/4224716999 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fe768022420 con 0x556a85355820 2024-04-26T18:31:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe788fbd740 1 -- 172.21.15.84:0/4224716999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7700316a0 con 0x556a85355820 2024-04-26T18:31:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe75f7fe640 1 -- 172.21.15.84:0/4224716999 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe768022620 con 0x556a85355820 2024-04-26T18:31:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe788fbd740 10 client.15855 fetch_fsmap learned FSMap version 29 2024-04-26T18:31:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe788fbd740 1 -- 172.21.15.84:0/4224716999 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556a85099620 con 0x556a85355820 2024-04-26T18:31:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe75f7fe640 1 -- 172.21.15.84:0/4224716999 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7700316a0 con 0x556a85355820 2024-04-26T18:31:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe75f7fe640 1 -- 172.21.15.84:0/4224716999 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe768015940 con 0x556a85355820 2024-04-26T18:31:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe788fbd740 10 client.15855 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:31:30.523 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe788fbd740 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe788fbd740 20 client.15855 populate_metadata read hostname 'smithi084' 2024-04-26T18:31:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe788fbd740 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556a85370170 0x556a853725c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe788fbd740 4 client.15855 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:31:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.513+0000 7fe788fbd740 1 -- 172.21.15.84:0/4224716999 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a852c9070 con 0x556a85370170 2024-04-26T18:31:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.514+0000 7fe77ffff640 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556a85370170 0x556a853725c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.514+0000 7fe75f7fe640 10 client.15855 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.514+0000 7fe77ffff640 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556a85370170 0x556a853725c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b61efec90159491a server_cookie=23c2b2d3f7cad543 in_seq=0 out_seq=0 2024-04-26T18:31:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe75f7fe640 1 -- 172.21.15.84:0/4224716999 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a852c9070 con 0x556a85370170 2024-04-26T18:31:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe75f7fe640 10 client.15855 handle_command_reply: tid=0 2024-04-26T18:31:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe788fbd740 1 client.15855 shutdown 2024-04-26T18:31:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe788fbd740 1 -- 172.21.15.84:0/4224716999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85355820 msgr2=0x556a8535d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:30.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe788fbd740 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85355820 0x556a8535d570 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fe7680176b0 tx=0x7fe7680046c0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe788fbd740 1 -- 172.21.15.84:0/4224716999 shutdown_connections 2024-04-26T18:31:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe788fbd740 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556a85370170 0x556a853725c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe788fbd740 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a853600e0 0x556a8535dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe788fbd740 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a85355820 0x556a8535d570 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe75f7fe640 0 client.15855 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe788fbd740 1 --2- 172.21.15.84:0/4224716999 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85354f00 0x556a8535d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe788fbd740 1 -- 172.21.15.84:0/4224716999 >> 172.21.15.84:0/4224716999 conn(0x556a85351410 msgr2=0x556a85357140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe788fbd740 1 -- 172.21.15.84:0/4224716999 shutdown_connections 2024-04-26T18:31:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe788fbd740 1 -- 172.21.15.84:0/4224716999 wait complete. 2024-04-26T18:31:30.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.515+0000 7fe788fbd740 20 client.15855 trim_cache size 0 max 16384 2024-04-26T18:31:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.530+0000 7fe788fbd740 1 Processor -- start 2024-04-26T18:31:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.530+0000 7fe788fbd740 1 -- start start 2024-04-26T18:31:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.530+0000 7fe788fbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85354f00 0x556a8536bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.530+0000 7fe788fbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85355820 0x556a853578c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.530+0000 7fe788fbd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 0x556a8535d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.530+0000 7fe788fbd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe7801ae470 con 0x556a853600e0 2024-04-26T18:31:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.530+0000 7fe788fbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7801affb0 con 0x556a85354f00 2024-04-26T18:31:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.530+0000 7fe788fbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe7801aeb40 con 0x556a85355820 2024-04-26T18:31:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.530+0000 7fe78483f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 0x556a8535d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.530+0000 7fe78483f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 0x556a8535d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38608/0 (socket says 172.21.15.84:38608) 2024-04-26T18:31:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.530+0000 7fe78483f640 1 -- 172.21.15.84:0/888954117 learned_addr learned my addr 172.21.15.84:0/888954117 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.530+0000 7fe77ffff640 1 --2- 172.21.15.84:0/888954117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85354f00 0x556a8536bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.530+0000 7fe777fff640 1 --2- 172.21.15.84:0/888954117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85355820 0x556a853578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.531+0000 7fe78483f640 1 -- 172.21.15.84:0/888954117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85355820 msgr2=0x556a853578c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.531+0000 7fe78483f640 1 --2- 172.21.15.84:0/888954117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85355820 0x556a853578c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.531+0000 7fe78483f640 1 -- 172.21.15.84:0/888954117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85354f00 msgr2=0x556a8536bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:30.536 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.531+0000 7fe78483f640 1 --2- 172.21.15.84:0/888954117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85354f00 0x556a8536bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.531+0000 7fe78483f640 1 -- 172.21.15.84:0/888954117 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a8534c440 con 0x556a853600e0 2024-04-26T18:31:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.531+0000 7fe78483f640 1 --2- 172.21.15.84:0/888954117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 0x556a8535d700 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe778099990 tx=0x7fe77809d780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c344af6e97b7593 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.531+0000 7fe775ffb640 1 -- 172.21.15.84:0/888954117 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7780a1370 con 0x556a853600e0 2024-04-26T18:31:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.531+0000 7fe775ffb640 1 -- 172.21.15.84:0/888954117 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7780bd040 con 0x556a853600e0 2024-04-26T18:31:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.531+0000 7fe775ffb640 1 -- 172.21.15.84:0/888954117 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7780a57a0 con 0x556a853600e0 2024-04-26T18:31:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.531+0000 7fe788fbd740 1 -- 172.21.15.84:0/888954117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 msgr2=0x556a8535d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.531+0000 7fe788fbd740 1 --2- 172.21.15.84:0/888954117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 0x556a8535d700 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fe778099990 tx=0x7fe77809d780 comp rx=0 tx=0).stop 2024-04-26T18:31:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.531+0000 7fe788fbd740 1 -- 172.21.15.84:0/888954117 shutdown_connections 2024-04-26T18:31:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 --2- 172.21.15.84:0/888954117 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 0x556a8535d700 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 --2- 172.21.15.84:0/888954117 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85355820 0x556a853578c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.537 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 --2- 172.21.15.84:0/888954117 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85354f00 0x556a8536bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 -- 172.21.15.84:0/888954117 >> 172.21.15.84:0/888954117 conn(0x556a85351410 msgr2=0x556a8535cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 -- 172.21.15.84:0/888954117 shutdown_connections 2024-04-26T18:31:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 -- 172.21.15.84:0/888954117 wait complete. 2024-04-26T18:31:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 -- start start 2024-04-26T18:31:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 Processor -- start 2024-04-26T18:31:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85354f00 0x556a853684a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85355820 0x556a85363810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 0x556a85363d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe7801af8e0 con 0x556a853600e0 2024-04-26T18:31:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe7801af210 con 0x556a85355820 2024-04-26T18:31:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe7801ac450 con 0x556a85354f00 2024-04-26T18:31:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe788fbd740 10 client.0 _ll_register_callbacks cb 0x7fe7680097d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:30.538 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe78483f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 0x556a85363d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe78483f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 0x556a85363d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38610/0 (socket says 172.21.15.84:38610) 2024-04-26T18:31:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.532+0000 7fe78483f640 1 -- 172.21.15.84:0/3958136319 learned_addr learned my addr 172.21.15.84:0/3958136319 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe77ffff640 1 --2- 172.21.15.84:0/3958136319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85354f00 0x556a853684a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe777fff640 1 --2- 172.21.15.84:0/3958136319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85355820 0x556a85363810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe78483f640 1 -- 172.21.15.84:0/3958136319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85354f00 msgr2=0x556a853684a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe78483f640 1 --2- 172.21.15.84:0/3958136319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85354f00 0x556a853684a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe78483f640 1 -- 172.21.15.84:0/3958136319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85355820 msgr2=0x556a85363810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe78483f640 1 --2- 172.21.15.84:0/3958136319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85355820 0x556a85363810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe78483f640 1 -- 172.21.15.84:0/3958136319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a853645f0 con 0x556a853600e0 2024-04-26T18:31:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe78483f640 1 --2- 172.21.15.84:0/3958136319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 0x556a85363d70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe77809cf00 tx=0x7fe77809cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b8fcf632edb536 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:30.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe75f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:31:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe75f7fe640 1 -- 172.21.15.84:0/3958136319 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7780a1370 con 0x556a853600e0 2024-04-26T18:31:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe75f7fe640 1 -- 172.21.15.84:0/3958136319 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fe7780a57a0 con 0x556a853600e0 2024-04-26T18:31:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe788fbd740 1 -- 172.21.15.84:0/3958136319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe7780bd040 con 0x556a853600e0 2024-04-26T18:31:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe75f7fe640 1 -- 172.21.15.84:0/3958136319 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe7780a5c30 con 0x556a853600e0 2024-04-26T18:31:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe788fbd740 10 client.15861 fetch_fsmap learned FSMap version 29 2024-04-26T18:31:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe788fbd740 1 -- 172.21.15.84:0/3958136319 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556a853d67c0 con 0x556a853600e0 2024-04-26T18:31:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe75f7fe640 1 -- 172.21.15.84:0/3958136319 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7780bd040 con 0x556a853600e0 2024-04-26T18:31:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe75f7fe640 1 -- 172.21.15.84:0/3958136319 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe77809f040 con 0x556a853600e0 2024-04-26T18:31:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe788fbd740 10 client.15861 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:31:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe788fbd740 10 client.15861 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:30.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.533+0000 7fe788fbd740 20 client.15861 populate_metadata read hostname 'smithi084' 2024-04-26T18:31:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.534+0000 7fe788fbd740 1 --2- 172.21.15.84:0/3958136319 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556a853d6dd0 0x556a853d91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.534+0000 7fe788fbd740 4 client.15861 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:31:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.534+0000 7fe788fbd740 1 -- 172.21.15.84:0/3958136319 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556a852c2ee0 con 0x556a853d6dd0 2024-04-26T18:31:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.534+0000 7fe77ffff640 1 --2- 172.21.15.84:0/3958136319 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556a853d6dd0 0x556a853d91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.534+0000 7fe75f7fe640 10 client.15861 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:30.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.534+0000 7fe77ffff640 1 --2- 172.21.15.84:0/3958136319 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556a853d6dd0 0x556a853d91c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90acda02c588edd0 server_cookie=7179094f3cc21c1f in_seq=0 out_seq=0 2024-04-26T18:31:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe75f7fe640 1 -- 172.21.15.84:0/3958136319 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556a852c2ee0 con 0x556a853d6dd0 2024-04-26T18:31:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe75f7fe640 10 client.15861 handle_command_reply: tid=0 2024-04-26T18:31:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe788fbd740 1 client.15861 shutdown 2024-04-26T18:31:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe788fbd740 1 -- 172.21.15.84:0/3958136319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 msgr2=0x556a85363d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe788fbd740 1 --2- 172.21.15.84:0/3958136319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 0x556a85363d70 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fe77809cf00 tx=0x7fe77809cb70 comp rx=0 tx=0).stop 2024-04-26T18:31:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe788fbd740 1 -- 172.21.15.84:0/3958136319 shutdown_connections 2024-04-26T18:31:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe788fbd740 1 --2- 172.21.15.84:0/3958136319 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556a853d6dd0 0x556a853d91c0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.550 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe788fbd740 1 --2- 172.21.15.84:0/3958136319 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556a853600e0 0x556a85363d70 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe788fbd740 1 --2- 172.21.15.84:0/3958136319 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556a85355820 0x556a85363810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe788fbd740 1 --2- 172.21.15.84:0/3958136319 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556a85354f00 0x556a853684a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe75f7fe640 0 client.15861 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe788fbd740 1 -- 172.21.15.84:0/3958136319 >> 172.21.15.84:0/3958136319 conn(0x556a85351410 msgr2=0x556a8535d880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe788fbd740 1 -- 172.21.15.84:0/3958136319 shutdown_connections 2024-04-26T18:31:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe788fbd740 1 -- 172.21.15.84:0/3958136319 wait complete. 2024-04-26T18:31:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.549+0000 7fe788fbd740 20 client.15861 trim_cache size 0 max 16384 2024-04-26T18:31:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:31:30.551 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:31:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:31:30.552 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:31:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.550+0000 7fe7862a2640 1 -- 172.21.15.84:0/1319330495 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe76c0a3dc0 msgr2=0x7fe76c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.550+0000 7fe7862a2640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe76c0a3dc0 0x7fe76c0a6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe78010eb20 tx=0x7fe768009e70 comp rx=0 tx=0).stop 2024-04-26T18:31:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.550+0000 7fe7862a2640 1 -- 172.21.15.84:0/1319330495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe78010cfb0 msgr2=0x7fe78019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.550+0000 7fe7862a2640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe78010cfb0 0x7fe78019d970 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fe77800ccb0 tx=0x7fe77800a620 comp rx=0 tx=0).stop 2024-04-26T18:31:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.550+0000 7fe7862a2640 1 -- 172.21.15.84:0/1319330495 shutdown_connections 2024-04-26T18:31:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.550+0000 7fe7862a2640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe76c0a3dc0 0x7fe76c0a6280 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.550+0000 7fe7862a2640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe78010cfb0 0x7fe78019d970 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.550+0000 7fe7862a2640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe780104500 0x7fe7801995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.550+0000 7fe7862a2640 1 --2- 172.21.15.84:0/1319330495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe780103be0 0x7fe7801990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.550+0000 7fe7862a2640 1 -- 172.21.15.84:0/1319330495 >> 172.21.15.84:0/1319330495 conn(0x7fe780077bb0 msgr2=0x7fe7800fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.550+0000 7fe7862a2640 1 -- 172.21.15.84:0/1319330495 shutdown_connections 2024-04-26T18:31:30.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:30.550+0000 7fe7862a2640 1 -- 172.21.15.84:0/1319330495 wait complete. 2024-04-26T18:31:31.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:30 smithi084 ceph-mon[29141]: pgmap v109: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:31:31.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:30 smithi103 ceph-mon[32727]: pgmap v109: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:31:31.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:30 smithi174 ceph-mon[32705]: pgmap v109: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:31:33.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:32 smithi084 ceph-mon[29141]: pgmap v110: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-26T18:31:33.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:32 smithi103 ceph-mon[32727]: pgmap v110: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-26T18:31:33.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:32 smithi174 ceph-mon[32705]: pgmap v110: 97 pgs: 97 active+clean; 2.7 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-26T18:31:35.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:34 smithi084 ceph-mon[29141]: pgmap v111: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:31:35.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:34 smithi103 ceph-mon[32727]: pgmap v111: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:31:35.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:34 smithi174 ceph-mon[32705]: pgmap v111: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:31:37.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:36 smithi084 ceph-mon[29141]: pgmap v112: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-26T18:31:37.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:36 smithi103 ceph-mon[32727]: pgmap v112: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-26T18:31:37.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:36 smithi174 ceph-mon[32705]: pgmap v112: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-26T18:31:39.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:38 smithi084 ceph-mon[29141]: pgmap v113: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-26T18:31:39.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:38 smithi103 ceph-mon[32727]: pgmap v113: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-26T18:31:39.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:38 smithi174 ceph-mon[32705]: pgmap v113: 97 pgs: 97 active+clean; 2.9 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-26T18:31:41.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:40 smithi084 ceph-mon[29141]: pgmap v114: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:31:41.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:40 smithi103 ceph-mon[32727]: pgmap v114: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:31:41.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:40 smithi174 ceph-mon[32705]: pgmap v114: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:31:43.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:42 smithi084 ceph-mon[29141]: pgmap v115: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:43.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:42 smithi103 ceph-mon[32727]: pgmap v115: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:43.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:42 smithi174 ceph-mon[32705]: pgmap v115: 97 pgs: 97 active+clean; 3.0 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.646+0000 7f8adccd1640 1 -- 172.21.15.84:0/774584848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8102210 msgr2=0x7f8ad810e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.646+0000 7f8adccd1640 1 --2- 172.21.15.84:0/774584848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8102210 0x7f8ad810e720 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009cd0 tx=0x7f8ad002a8e0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.646+0000 7f8adccd1640 1 -- 172.21.15.84:0/774584848 shutdown_connections 2024-04-26T18:31:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.646+0000 7f8adccd1640 1 --2- 172.21.15.84:0/774584848 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8102210 0x7f8ad810e720 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.646+0000 7f8adccd1640 1 --2- 172.21.15.84:0/774584848 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ad8101850 0x7f8ad8101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.646+0000 7f8adccd1640 1 --2- 172.21.15.84:0/774584848 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ad8107860 0x7f8ad8107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.646+0000 7f8adccd1640 1 -- 172.21.15.84:0/774584848 >> 172.21.15.84:0/774584848 conn(0x7f8ad8077bb0 msgr2=0x7f8ad8077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.646+0000 7f8adccd1640 1 -- 172.21.15.84:0/774584848 shutdown_connections 2024-04-26T18:31:43.647 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.646+0000 7f8adccd1640 1 -- 172.21.15.84:0/774584848 wait complete. 2024-04-26T18:31:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.647+0000 7f8adccd1640 1 Processor -- start 2024-04-26T18:31:43.648 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.647+0000 7f8adccd1640 1 -- start start 2024-04-26T18:31:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.647+0000 7f8adccd1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ad8101850 0x7f8ad8198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.647+0000 7f8adccd1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8102210 0x7f8ad81994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.647+0000 7f8adccd1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ad8107860 0x7f8ad819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.647+0000 7f8adccd1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8111b20 con 0x7f8ad8102210 2024-04-26T18:31:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.647+0000 7f8adccd1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81119a0 con 0x7f8ad8101850 2024-04-26T18:31:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.647+0000 7f8adccd1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad8111ca0 con 0x7f8ad8107860 2024-04-26T18:31:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.647+0000 7f8acffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8102210 0x7f8ad81994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8acffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8102210 0x7f8ad81994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45656/0 (socket says 172.21.15.84:45656) 2024-04-26T18:31:43.650 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8acffff640 1 -- 172.21.15.84:0/2424981911 learned_addr learned my addr 172.21.15.84:0/2424981911 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8ad6d9d640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ad8107860 0x7f8ad819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8ad659c640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ad8101850 0x7f8ad8198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8acffff640 1 -- 172.21.15.84:0/2424981911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ad8107860 msgr2=0x7f8ad819d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8acffff640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ad8107860 0x7f8ad819d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8acffff640 1 -- 172.21.15.84:0/2424981911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ad8101850 msgr2=0x7f8ad8198f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8acffff640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ad8101850 0x7f8ad8198f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8acffff640 1 -- 172.21.15.84:0/2424981911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ad819df80 con 0x7f8ad8102210 2024-04-26T18:31:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8acffff640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8102210 0x7f8ad81994d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000c920 tx=0x7f8ac000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8acf7fe640 1 -- 172.21.15.84:0/2424981911 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0007c70 con 0x7f8ad8102210 2024-04-26T18:31:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8acf7fe640 1 -- 172.21.15.84:0/2424981911 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ac0007e10 con 0x7f8ad8102210 2024-04-26T18:31:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8adccd1640 1 -- 172.21.15.84:0/2424981911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ad819e270 con 0x7f8ad8102210 2024-04-26T18:31:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8acf7fe640 1 -- 172.21.15.84:0/2424981911 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0018820 con 0x7f8ad8102210 2024-04-26T18:31:43.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.648+0000 7f8adccd1640 1 -- 172.21.15.84:0/2424981911 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ad81a5b00 con 0x7f8ad8102210 2024-04-26T18:31:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.650+0000 7f8acf7fe640 1 -- 172.21.15.84:0/2424981911 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8ac000f450 con 0x7f8ad8102210 2024-04-26T18:31:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.651+0000 7f8adf9eb740 1 Processor -- start 2024-04-26T18:31:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.651+0000 7f8adf9eb740 1 -- start start 2024-04-26T18:31:43.658 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.651+0000 7f8adf9eb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebae7f00 0x55d2ebae82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.651+0000 7f8adf9eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 0x55d2ebaf2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.651+0000 7f8adf9eb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.651+0000 7f8adf9eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d2eba4e460 con 0x55d2ebae8820 2024-04-26T18:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.651+0000 7f8adf9eb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d2eba08590 con 0x55d2ebaf30e0 2024-04-26T18:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.651+0000 7f8adf9eb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d2eba4a420 con 0x55d2ebae7f00 2024-04-26T18:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.651+0000 7f8acf7fe640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8ac40a3dc0 0x7f8ac40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.651+0000 7f8ad659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 0x55d2ebaf2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.651+0000 7f8ad6d9d640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8ac40a3dc0 0x7f8ac40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.651+0000 7f8ad659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 0x55d2ebaf2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45658/0 (socket says 172.21.15.84:45658) 2024-04-26T18:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.651+0000 7f8ad659c640 1 -- 172.21.15.84:0/3279675997 learned_addr learned my addr 172.21.15.84:0/3279675997 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.652+0000 7f8acf7fe640 1 -- 172.21.15.84:0/2424981911 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8ac00d0890 con 0x7f8ad8102210 2024-04-26T18:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.652+0000 7f8acffff640 1 --2- 172.21.15.84:0/3279675997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.659 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.652+0000 7f8ad6d9d640 1 --2- 172.21.15.84:0/3279675997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebae7f00 0x55d2ebae82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.652+0000 7f8ad659c640 1 -- 172.21.15.84:0/3279675997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebae7f00 msgr2=0x55d2ebae82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.652+0000 7f8ad659c640 1 --2- 172.21.15.84:0/3279675997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebae7f00 0x55d2ebae82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.652+0000 7f8ad659c640 1 -- 172.21.15.84:0/3279675997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebaf30e0 msgr2=0x55d2ebaf54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.652+0000 7f8ad659c640 1 --2- 172.21.15.84:0/3279675997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.652+0000 7f8ad659c640 1 -- 172.21.15.84:0/3279675997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2eba150d0 con 0x55d2ebae8820 2024-04-26T18:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.652+0000 7f8ad6d9d640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8ac40a3dc0 0x7f8ac40a6280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0004af0 tx=0x7f8ad00316e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.652+0000 7f8ad659c640 1 --2- 172.21.15.84:0/3279675997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 0x55d2ebaf2ba0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800bc90 tx=0x7f8ac80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba7414614c04fe6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.652+0000 7f8accff9640 1 -- 172.21.15.84:0/3279675997 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac801c480 con 0x55d2ebae8820 2024-04-26T18:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.652+0000 7f8accff9640 1 -- 172.21.15.84:0/3279675997 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ac801cad0 con 0x55d2ebae8820 2024-04-26T18:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.652+0000 7f8accff9640 1 -- 172.21.15.84:0/3279675997 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8011a80 con 0x55d2ebae8820 2024-04-26T18:31:43.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.653+0000 7f8adf9eb740 1 -- 172.21.15.84:0/3279675997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 msgr2=0x55d2ebaf2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.653+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/3279675997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 0x55d2ebaf2ba0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8ac800bc90 tx=0x7f8ac80107a0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.653+0000 7f8adf9eb740 1 -- 172.21.15.84:0/3279675997 shutdown_connections 2024-04-26T18:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.653+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/3279675997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.653+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/3279675997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 0x55d2ebaf2ba0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.653+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/3279675997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebae7f00 0x55d2ebae82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.653+0000 7f8adf9eb740 1 -- 172.21.15.84:0/3279675997 >> 172.21.15.84:0/3279675997 conn(0x55d2ebae4410 msgr2=0x55d2ebae47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.653+0000 7f8adf9eb740 1 -- 172.21.15.84:0/3279675997 shutdown_connections 2024-04-26T18:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.653+0000 7f8adf9eb740 1 -- 172.21.15.84:0/3279675997 wait complete. 2024-04-26T18:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.653+0000 7f8adf9eb740 1 -- start start 2024-04-26T18:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.653+0000 7f8adf9eb740 1 Processor -- start 2024-04-26T18:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8adf9eb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae7f00 0x55d2ebaf0050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8adf9eb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebae8820 0x55d2ebaf0590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8adf9eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8adf9eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d2ebaf5d10 con 0x55d2ebaf30e0 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8adf9eb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d2eb9d5b60 con 0x55d2ebae7f00 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8adf9eb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d2ebaf5e90 con 0x55d2ebae8820 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8adf9eb740 10 client.0 _ll_register_callbacks cb 0x55d2eb9fa2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8ad6d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8ad6d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45662/0 (socket says 172.21.15.84:45662) 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8ad6d9d640 1 -- 172.21.15.84:0/2307303679 learned_addr learned my addr 172.21.15.84:0/2307303679 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8acffff640 1 --2- 172.21.15.84:0/2307303679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebae8820 0x55d2ebaf0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8ad659c640 1 --2- 172.21.15.84:0/2307303679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae7f00 0x55d2ebaf0050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8ad6d9d640 1 -- 172.21.15.84:0/2307303679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebae8820 msgr2=0x55d2ebaf0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8ad6d9d640 1 --2- 172.21.15.84:0/2307303679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebae8820 0x55d2ebaf0590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8ad6d9d640 1 -- 172.21.15.84:0/2307303679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae7f00 msgr2=0x55d2ebaf0050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:43.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8ad6d9d640 1 --2- 172.21.15.84:0/2307303679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae7f00 0x55d2ebaf0050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8ad6d9d640 1 -- 172.21.15.84:0/2307303679 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2eb760210 con 0x55d2ebaf30e0 2024-04-26T18:31:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8ad6d9d640 1 --2- 172.21.15.84:0/2307303679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf0ad0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00317f0 tx=0x7f8ad0005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82420644f04d2e74 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8abdffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:31:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8abdffb640 1 -- 172.21.15.84:0/2307303679 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0004250 con 0x55d2ebaf30e0 2024-04-26T18:31:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8abdffb640 1 -- 172.21.15.84:0/2307303679 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8ad00043f0 con 0x55d2ebaf30e0 2024-04-26T18:31:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.654+0000 7f8adf9eb740 1 -- 172.21.15.84:0/2307303679 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac801cad0 con 0x55d2ebaf30e0 2024-04-26T18:31:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.655+0000 7f8abdffb640 1 -- 172.21.15.84:0/2307303679 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ad00045f0 con 0x55d2ebaf30e0 2024-04-26T18:31:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.655+0000 7f8adf9eb740 10 client.15873 fetch_fsmap learned FSMap version 29 2024-04-26T18:31:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.655+0000 7f8adf9eb740 1 -- 172.21.15.84:0/2307303679 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d2eb82c620 con 0x55d2ebaf30e0 2024-04-26T18:31:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.655+0000 7f8abdffb640 1 -- 172.21.15.84:0/2307303679 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ac801cad0 con 0x55d2ebaf30e0 2024-04-26T18:31:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.655+0000 7f8abdffb640 1 -- 172.21.15.84:0/2307303679 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8ad0049070 con 0x55d2ebaf30e0 2024-04-26T18:31:43.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.655+0000 7f8adf9eb740 10 client.15873 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.655+0000 7f8adf9eb740 10 client.15873 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.655+0000 7f8adf9eb740 20 client.15873 populate_metadata read hostname 'smithi084' 2024-04-26T18:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.655+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/2307303679 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d2ebb031b0 0x55d2ebb055c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.655+0000 7f8adf9eb740 4 client.15873 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.655+0000 7f8adf9eb740 1 -- 172.21.15.84:0/2307303679 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2eba5c070 con 0x55d2ebb031b0 2024-04-26T18:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.655+0000 7f8ad659c640 1 --2- 172.21.15.84:0/2307303679 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d2ebb031b0 0x55d2ebb055c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.656+0000 7f8ad659c640 1 --2- 172.21.15.84:0/2307303679 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d2ebb031b0 0x55d2ebb055c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef48289417a33e5c server_cookie=f246b994396c6d14 in_seq=0 out_seq=0 2024-04-26T18:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.656+0000 7f8abdffb640 10 client.15873 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.656+0000 7f8abdffb640 1 -- 172.21.15.84:0/2307303679 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d2eba5c070 con 0x55d2ebb031b0 2024-04-26T18:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.656+0000 7f8abdffb640 10 client.15873 handle_command_reply: tid=0 2024-04-26T18:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.657+0000 7f8adf9eb740 1 client.15873 shutdown 2024-04-26T18:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.657+0000 7f8adf9eb740 1 -- 172.21.15.84:0/2307303679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebaf30e0 msgr2=0x55d2ebaf0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:43.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.657+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/2307303679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf0ad0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00317f0 tx=0x7f8ad0005da0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.657+0000 7f8adf9eb740 1 -- 172.21.15.84:0/2307303679 shutdown_connections 2024-04-26T18:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.657+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/2307303679 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d2ebb031b0 0x55d2ebb055c0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.657+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/2307303679 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf0ad0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.657+0000 7f8abdffb640 0 client.15873 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.657+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/2307303679 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebae8820 0x55d2ebaf0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.657+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/2307303679 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae7f00 0x55d2ebaf0050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.657+0000 7f8adf9eb740 1 -- 172.21.15.84:0/2307303679 >> 172.21.15.84:0/2307303679 conn(0x55d2ebae4410 msgr2=0x55d2ebae35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.657+0000 7f8adf9eb740 1 -- 172.21.15.84:0/2307303679 shutdown_connections 2024-04-26T18:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.657+0000 7f8adf9eb740 1 -- 172.21.15.84:0/2307303679 wait complete. 2024-04-26T18:31:43.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.657+0000 7f8adf9eb740 20 client.15873 trim_cache size 0 max 16384 2024-04-26T18:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8adf9eb740 1 Processor -- start 2024-04-26T18:31:43.676 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8adf9eb740 1 -- start start 2024-04-26T18:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8adf9eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae7f00 0x55d2ebafebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8adf9eb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae8820 0x55d2ebb0aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8adf9eb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebaf30e0 0x55d2ebb0b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8adf9eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81ae4a0 con 0x55d2ebae7f00 2024-04-26T18:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8adf9eb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81affe0 con 0x55d2ebae8820 2024-04-26T18:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8adf9eb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81aeb70 con 0x55d2ebaf30e0 2024-04-26T18:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8ad659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae7f00 0x55d2ebafebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8ad659c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae7f00 0x55d2ebafebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45676/0 (socket says 172.21.15.84:45676) 2024-04-26T18:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8ad659c640 1 -- 172.21.15.84:0/941673061 learned_addr learned my addr 172.21.15.84:0/941673061 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8ad659c640 1 -- 172.21.15.84:0/941673061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebaf30e0 msgr2=0x55d2ebb0b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8ad659c640 1 --2- 172.21.15.84:0/941673061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebaf30e0 0x55d2ebb0b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8ad659c640 1 -- 172.21.15.84:0/941673061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae8820 msgr2=0x55d2ebb0aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:43.677 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8ad659c640 1 --2- 172.21.15.84:0/941673061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae8820 0x55d2ebb0aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8ad659c640 1 -- 172.21.15.84:0/941673061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2ebae8d60 con 0x55d2ebae7f00 2024-04-26T18:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8ad659c640 1 --2- 172.21.15.84:0/941673061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae7f00 0x55d2ebafebd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55d2ebae94a0 tx=0x7f8ac8011680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f00827e19137e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8abffff640 1 -- 172.21.15.84:0/941673061 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8023040 con 0x55d2ebae7f00 2024-04-26T18:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8abffff640 1 -- 172.21.15.84:0/941673061 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ac80116f0 con 0x55d2ebae7f00 2024-04-26T18:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.672+0000 7f8abffff640 1 -- 172.21.15.84:0/941673061 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac8028ae0 con 0x55d2ebae7f00 2024-04-26T18:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.673+0000 7f8adf9eb740 1 -- 172.21.15.84:0/941673061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae7f00 msgr2=0x55d2ebafebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.673+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/941673061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae7f00 0x55d2ebafebd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x55d2ebae94a0 tx=0x7f8ac8011680 comp rx=0 tx=0).stop 2024-04-26T18:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.673+0000 7f8adf9eb740 1 -- 172.21.15.84:0/941673061 shutdown_connections 2024-04-26T18:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.673+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/941673061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebaf30e0 0x55d2ebb0b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.673+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/941673061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae8820 0x55d2ebb0aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.673+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/941673061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae7f00 0x55d2ebafebd0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.673+0000 7f8adf9eb740 1 -- 172.21.15.84:0/941673061 >> 172.21.15.84:0/941673061 conn(0x55d2ebae4410 msgr2=0x55d2ebaeb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:43.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.673+0000 7f8adf9eb740 1 -- 172.21.15.84:0/941673061 shutdown_connections 2024-04-26T18:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.673+0000 7f8adf9eb740 1 -- 172.21.15.84:0/941673061 wait complete. 2024-04-26T18:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.673+0000 7f8adf9eb740 1 -- start start 2024-04-26T18:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.673+0000 7f8adf9eb740 1 Processor -- start 2024-04-26T18:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8adf9eb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae7f00 0x55d2ebafb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8adf9eb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 0x55d2ebafb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8adf9eb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8adf9eb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81af910 con 0x55d2ebae8820 2024-04-26T18:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8adf9eb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81af240 con 0x55d2ebae7f00 2024-04-26T18:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8adf9eb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ad81ac480 con 0x55d2ebaf30e0 2024-04-26T18:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8adf9eb740 10 client.0 _ll_register_callbacks cb 0x7f8ad0009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8acffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 0x55d2ebafb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8acffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 0x55d2ebafb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45680/0 (socket says 172.21.15.84:45680) 2024-04-26T18:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8acffff640 1 -- 172.21.15.84:0/4039978455 learned_addr learned my addr 172.21.15.84:0/4039978455 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8ad659c640 1 --2- 172.21.15.84:0/4039978455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae7f00 0x55d2ebafb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8ad6d9d640 1 --2- 172.21.15.84:0/4039978455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8acffff640 1 -- 172.21.15.84:0/4039978455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebaf30e0 msgr2=0x55d2ebaf80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8acffff640 1 --2- 172.21.15.84:0/4039978455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8acffff640 1 -- 172.21.15.84:0/4039978455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae7f00 msgr2=0x55d2ebafb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8acffff640 1 --2- 172.21.15.84:0/4039978455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae7f00 0x55d2ebafb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8acffff640 1 -- 172.21.15.84:0/4039978455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2ebaf86a0 con 0x55d2ebae8820 2024-04-26T18:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8acffff640 1 --2- 172.21.15.84:0/4039978455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 0x55d2ebafb620 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8ac009c7b0 tx=0x7f8ac009c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1ffca24b8638dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8abdffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8abdffb640 1 -- 172.21.15.84:0/4039978455 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ac00a0bd0 con 0x55d2ebae8820 2024-04-26T18:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8abdffb640 1 -- 172.21.15.84:0/4039978455 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8ac000f740 con 0x55d2ebae8820 2024-04-26T18:31:43.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.674+0000 7f8adf9eb740 1 -- 172.21.15.84:0/4039978455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ac80116f0 con 0x55d2ebae8820 2024-04-26T18:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8abdffb640 1 -- 172.21.15.84:0/4039978455 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ac00a20e0 con 0x55d2ebae8820 2024-04-26T18:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8adf9eb740 10 client.15879 fetch_fsmap learned FSMap version 29 2024-04-26T18:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8adf9eb740 1 -- 172.21.15.84:0/4039978455 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d2ebb6a040 con 0x55d2ebae8820 2024-04-26T18:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8abdffb640 1 -- 172.21.15.84:0/4039978455 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ac80116f0 con 0x55d2ebae8820 2024-04-26T18:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8abdffb640 1 -- 172.21.15.84:0/4039978455 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8ac009ca80 con 0x55d2ebae8820 2024-04-26T18:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8adf9eb740 10 client.15879 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8adf9eb740 10 client.15879 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8adf9eb740 20 client.15879 populate_metadata read hostname 'smithi084' 2024-04-26T18:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/4039978455 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d2ebb031b0 0x55d2ebb6c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8adf9eb740 4 client.15879 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8adf9eb740 1 -- 172.21.15.84:0/4039978455 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2eba55ee0 con 0x55d2ebb031b0 2024-04-26T18:31:43.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8ad659c640 1 --2- 172.21.15.84:0/4039978455 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d2ebb031b0 0x55d2ebb6c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8ad659c640 1 --2- 172.21.15.84:0/4039978455 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d2ebb031b0 0x55d2ebb6c850 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=776e93581b61346 server_cookie=6cd7f059f66d0b8 in_seq=0 out_seq=0 2024-04-26T18:31:43.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.675+0000 7f8abdffb640 10 client.15879 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8abdffb640 1 -- 172.21.15.84:0/4039978455 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2eba55ee0 con 0x55d2ebb031b0 2024-04-26T18:31:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8abdffb640 10 client.15879 handle_command_reply: tid=0 2024-04-26T18:31:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adf9eb740 1 client.15879 shutdown 2024-04-26T18:31:43.691 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adf9eb740 1 -- 172.21.15.84:0/4039978455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 msgr2=0x55d2ebafb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/4039978455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 0x55d2ebafb620 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f8ac009c7b0 tx=0x7f8ac009c240 comp rx=0 tx=0).stop 2024-04-26T18:31:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adf9eb740 1 -- 172.21.15.84:0/4039978455 shutdown_connections 2024-04-26T18:31:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/4039978455 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d2ebb031b0 0x55d2ebb6c850 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/4039978455 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d2ebaf30e0 0x55d2ebaf80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8abdffb640 0 client.15879 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/4039978455 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d2ebae8820 0x55d2ebafb620 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adf9eb740 1 --2- 172.21.15.84:0/4039978455 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d2ebae7f00 0x55d2ebafb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adf9eb740 1 -- 172.21.15.84:0/4039978455 >> 172.21.15.84:0/4039978455 conn(0x55d2ebae4410 msgr2=0x55d2ebae2d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adf9eb740 1 -- 172.21.15.84:0/4039978455 shutdown_connections 2024-04-26T18:31:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adf9eb740 1 -- 172.21.15.84:0/4039978455 wait complete. 2024-04-26T18:31:43.692 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adf9eb740 20 client.15879 trim_cache size 0 max 16384 2024-04-26T18:31:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:31:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:31:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:31:43.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:31:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adccd1640 1 -- 172.21.15.84:0/2424981911 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8ac40a3dc0 msgr2=0x7f8ac40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.691+0000 7f8adccd1640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8ac40a3dc0 0x7f8ac40a6280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0004af0 tx=0x7f8ad00316e0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.692+0000 7f8adccd1640 1 -- 172.21.15.84:0/2424981911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8102210 msgr2=0x7f8ad81994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.692+0000 7f8adccd1640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8102210 0x7f8ad81994d0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8ac000c920 tx=0x7f8ac000cde0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.692+0000 7f8adccd1640 1 -- 172.21.15.84:0/2424981911 shutdown_connections 2024-04-26T18:31:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.692+0000 7f8adccd1640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8ac40a3dc0 0x7f8ac40a6280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.692+0000 7f8adccd1640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ad8107860 0x7f8ad819d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.692+0000 7f8adccd1640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ad8102210 0x7f8ad81994d0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.692+0000 7f8adccd1640 1 --2- 172.21.15.84:0/2424981911 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ad8101850 0x7f8ad8198f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:43.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.692+0000 7f8adccd1640 1 -- 172.21.15.84:0/2424981911 >> 172.21.15.84:0/2424981911 conn(0x7f8ad8077bb0 msgr2=0x7f8ad806b340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.692+0000 7f8adccd1640 1 -- 172.21.15.84:0/2424981911 shutdown_connections 2024-04-26T18:31:43.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:43.692+0000 7f8adccd1640 1 -- 172.21.15.84:0/2424981911 wait complete. 2024-04-26T18:31:45.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:44 smithi084 ceph-mon[29141]: pgmap v116: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-04-26T18:31:45.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:44 smithi103 ceph-mon[32727]: pgmap v116: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-04-26T18:31:45.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:44 smithi174 ceph-mon[32705]: pgmap v116: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-04-26T18:31:47.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:46 smithi084 ceph-mon[29141]: pgmap v117: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-26T18:31:47.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:46 smithi103 ceph-mon[32727]: pgmap v117: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-26T18:31:47.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:46 smithi174 ceph-mon[32705]: pgmap v117: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-26T18:31:49.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:48 smithi084 ceph-mon[29141]: pgmap v118: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-26T18:31:49.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:48 smithi103 ceph-mon[32727]: pgmap v118: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-26T18:31:49.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:48 smithi174 ceph-mon[32705]: pgmap v118: 97 pgs: 97 active+clean; 3.1 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 13 op/s 2024-04-26T18:31:51.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:50 smithi084 ceph-mon[29141]: pgmap v119: 97 pgs: 97 active+clean; 3.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-04-26T18:31:51.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:50 smithi103 ceph-mon[32727]: pgmap v119: 97 pgs: 97 active+clean; 3.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-04-26T18:31:51.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:50 smithi174 ceph-mon[32705]: pgmap v119: 97 pgs: 97 active+clean; 3.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 22 op/s 2024-04-26T18:31:53.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:52 smithi084 ceph-mon[29141]: pgmap v120: 97 pgs: 97 active+clean; 3.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:53.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:52 smithi103 ceph-mon[32727]: pgmap v120: 97 pgs: 97 active+clean; 3.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:53.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:52 smithi174 ceph-mon[32705]: pgmap v120: 97 pgs: 97 active+clean; 3.2 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:31:55.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:54 smithi084 ceph-mon[29141]: pgmap v121: 97 pgs: 97 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:31:55.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:54 smithi103 ceph-mon[32727]: pgmap v121: 97 pgs: 97 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:31:55.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:54 smithi174 ceph-mon[32705]: pgmap v121: 97 pgs: 97 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:31:56.790 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.788+0000 7f4dd2447640 1 -- 172.21.15.84:0/1925272342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc107820 msgr2=0x7f4dcc107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.788+0000 7f4dd2447640 1 --2- 172.21.15.84:0/1925272342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc107820 0x7f4dcc107c00 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009930 tx=0x7f4dbc02a420 comp rx=0 tx=0).stop 2024-04-26T18:31:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.788+0000 7f4dd2447640 1 -- 172.21.15.84:0/1925272342 shutdown_connections 2024-04-26T18:31:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.788+0000 7f4dd2447640 1 --2- 172.21.15.84:0/1925272342 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dcc102220 0x7f4dcc10e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.788+0000 7f4dd2447640 1 --2- 172.21.15.84:0/1925272342 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dcc101860 0x7f4dcc101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.788+0000 7f4dd2447640 1 --2- 172.21.15.84:0/1925272342 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc107820 0x7f4dcc107c00 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.788+0000 7f4dd2447640 1 -- 172.21.15.84:0/1925272342 >> 172.21.15.84:0/1925272342 conn(0x7f4dcc0789b0 msgr2=0x7f4dcc078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.788+0000 7f4dd2447640 1 -- 172.21.15.84:0/1925272342 shutdown_connections 2024-04-26T18:31:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.788+0000 7f4dd2447640 1 -- 172.21.15.84:0/1925272342 wait complete. 2024-04-26T18:31:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.788+0000 7f4dd2447640 1 Processor -- start 2024-04-26T18:31:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.788+0000 7f4dd2447640 1 -- start start 2024-04-26T18:31:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dd2447640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc101860 0x7f4dcc199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dd2447640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dcc102220 0x7f4dcc1995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dd2447640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dcc107820 0x7f4dcc19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dd2447640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc112bb0 con 0x7f4dcc101860 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dd2447640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc112a30 con 0x7f4dcc102220 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dd2447640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc112d30 con 0x7f4dcc107820 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dcbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc101860 0x7f4dcc199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dcbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc101860 0x7f4dcc199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50212/0 (socket says 172.21.15.84:50212) 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dcbfff640 1 -- 172.21.15.84:0/3723828951 learned_addr learned my addr 172.21.15.84:0/3723828951 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dd09e4640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dcc107820 0x7f4dcc19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dcbfff640 1 -- 172.21.15.84:0/3723828951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dcc107820 msgr2=0x7f4dcc19d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dcc107820 0x7f4dcc19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dcbfff640 1 -- 172.21.15.84:0/3723828951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dcc102220 msgr2=0x7f4dcc1995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dcb7fe640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dcc102220 0x7f4dcc1995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dcc102220 0x7f4dcc1995d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dcbfff640 1 -- 172.21.15.84:0/3723828951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dcc19e030 con 0x7f4dcc101860 2024-04-26T18:31:56.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dcb7fe640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dcc102220 0x7f4dcc1995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:31:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc101860 0x7f4dcc199090 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc005940 tx=0x7f4dbc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dc97fa640 1 -- 172.21.15.84:0/3723828951 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc03f030 con 0x7f4dcc101860 2024-04-26T18:31:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dc97fa640 1 -- 172.21.15.84:0/3723828951 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dbc003aa0 con 0x7f4dcc101860 2024-04-26T18:31:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dd2447640 1 -- 172.21.15.84:0/3723828951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dcc19e2c0 con 0x7f4dcc101860 2024-04-26T18:31:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.789+0000 7f4dc97fa640 1 -- 172.21.15.84:0/3723828951 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc035070 con 0x7f4dcc101860 2024-04-26T18:31:56.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.790+0000 7f4dd2447640 1 -- 172.21.15.84:0/3723828951 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dcc1a5bb0 con 0x7f4dcc101860 2024-04-26T18:31:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.792+0000 7f4dc97fa640 1 -- 172.21.15.84:0/3723828951 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4dbc042030 con 0x7f4dcc101860 2024-04-26T18:31:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.792+0000 7f4dd4dfd740 1 Processor -- start 2024-04-26T18:31:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.792+0000 7f4dc97fa640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4db40a3dc0 0x7f4db40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.792+0000 7f4dd4dfd740 1 -- start start 2024-04-26T18:31:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.792+0000 7f4dd09e4640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4db40a3dc0 0x7f4db40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.792+0000 7f4dc97fa640 1 -- 172.21.15.84:0/3723828951 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f4dbc0f1060 con 0x7f4dcc101860 2024-04-26T18:31:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.792+0000 7f4dd4dfd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967ec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.792+0000 7f4dd4dfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967ec740 0x5573967fafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dd4dfd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967fb4f0 0x5573967fd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dd4dfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557396752460 con 0x5573967ec740 2024-04-26T18:31:56.800 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dd4dfd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55739670c590 con 0x5573967ebe20 2024-04-26T18:31:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dd4dfd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55739674e420 con 0x5573967fb4f0 2024-04-26T18:31:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dcb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967ec740 0x5573967fafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dcb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967ec740 0x5573967fafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50214/0 (socket says 172.21.15.84:50214) 2024-04-26T18:31:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dcb7fe640 1 -- 172.21.15.84:0/506591106 learned_addr learned my addr 172.21.15.84:0/506591106 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dd09e4640 1 --2- 172.21.15.84:0/506591106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967fb4f0 0x5573967fd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/506591106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967ec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dd09e4640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4db40a3dc0 0x7f4db40a6280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4005cc0 tx=0x7f4dc400d710 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dcb7fe640 1 -- 172.21.15.84:0/506591106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967fb4f0 msgr2=0x5573967fd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dcb7fe640 1 --2- 172.21.15.84:0/506591106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967fb4f0 0x5573967fd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dcb7fe640 1 -- 172.21.15.84:0/506591106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 msgr2=0x5573967ec200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dcb7fe640 1 --2- 172.21.15.84:0/506591106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967ec200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dcb7fe640 1 -- 172.21.15.84:0/506591106 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573967190d0 con 0x5573967ec740 2024-04-26T18:31:56.801 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/506591106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967ec200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4dcb7fe640 1 --2- 172.21.15.84:0/506591106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967ec740 0x5573967fafb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000bbe0 tx=0x7f4dc000d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa7b974386240013 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4db27fc640 1 -- 172.21.15.84:0/506591106 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0017620 con 0x5573967ec740 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4db27fc640 1 -- 172.21.15.84:0/506591106 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc0017c70 con 0x5573967ec740 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.793+0000 7f4db27fc640 1 -- 172.21.15.84:0/506591106 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc000fc40 con 0x5573967ec740 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.794+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/506591106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967ec740 msgr2=0x5573967fafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.794+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/506591106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967ec740 0x5573967fafb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000bbe0 tx=0x7f4dc000d710 comp rx=0 tx=0).stop 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.794+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/506591106 shutdown_connections 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.794+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/506591106 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967fb4f0 0x5573967fd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.794+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/506591106 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967ec740 0x5573967fafb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.794+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/506591106 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967ec200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.794+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/506591106 >> 172.21.15.84:0/506591106 conn(0x5573967e8410 msgr2=0x5573967e87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.794+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/506591106 shutdown_connections 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.794+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/506591106 wait complete. 2024-04-26T18:31:56.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.794+0000 7f4dd4dfd740 1 -- start start 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd4dfd740 1 Processor -- start 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd4dfd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967f8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd4dfd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 0x5573967f8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd4dfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573967f8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd4dfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5573967fe120 con 0x5573967fb4f0 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd4dfd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5573966d9b60 con 0x5573967ebe20 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd4dfd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5573967fe2a0 con 0x5573967ec740 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd4dfd740 10 client.0 _ll_register_callbacks cb 0x5573966fe2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd09e4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573967f8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd09e4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573967f8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50230/0 (socket says 172.21.15.84:50230) 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd09e4640 1 -- 172.21.15.84:0/2249272325 learned_addr learned my addr 172.21.15.84:0/2249272325 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/2249272325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967f8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dcb7fe640 1 --2- 172.21.15.84:0/2249272325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 0x5573967f8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd09e4640 1 -- 172.21.15.84:0/2249272325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 msgr2=0x5573967f8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd09e4640 1 --2- 172.21.15.84:0/2249272325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 0x5573967f8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd09e4640 1 -- 172.21.15.84:0/2249272325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 msgr2=0x5573967f8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd09e4640 1 --2- 172.21.15.84:0/2249272325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967f8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.795+0000 7f4dd09e4640 1 -- 172.21.15.84:0/2249272325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557396464210 con 0x5573967fb4f0 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dd09e4640 1 --2- 172.21.15.84:0/2249272325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573967f8eb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4011df0 tx=0x7f4dc4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b09c7573201ce5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dab7fe640 1 -- 172.21.15.84:0/2249272325 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc401d430 con 0x5573967fb4f0 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dab7fe640 1 -- 172.21.15.84:0/2249272325 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f4dc401d5d0 con 0x5573967fb4f0 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/2249272325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc0017c70 con 0x5573967fb4f0 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dab7fe640 1 -- 172.21.15.84:0/2249272325 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4dc401d7d0 con 0x5573967fb4f0 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dd4dfd740 10 client.15891 fetch_fsmap learned FSMap version 29 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/2249272325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557396530620 con 0x5573967fb4f0 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dab7fe640 1 -- 172.21.15.84:0/2249272325 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc0017c70 con 0x5573967fb4f0 2024-04-26T18:31:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dab7fe640 1 -- 172.21.15.84:0/2249272325 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4dc4025bd0 con 0x5573967fb4f0 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dd4dfd740 10 client.15891 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dd4dfd740 10 client.15891 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dd4dfd740 20 client.15891 populate_metadata read hostname 'smithi084' 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/2249272325 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55739680b780 0x55739680db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dd4dfd740 4 client.15891 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.796+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/2249272325 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557396760070 con 0x55739680b780 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.797+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/2249272325 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55739680b780 0x55739680db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.797+0000 7f4dab7fe640 10 client.15891 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.797+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/2249272325 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55739680b780 0x55739680db90 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=267b4590b43bb80a server_cookie=27f69654bc355e37 in_seq=0 out_seq=0 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dab7fe640 1 -- 172.21.15.84:0/2249272325 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557396760070 con 0x55739680b780 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dab7fe640 10 client.15891 handle_command_reply: tid=0 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dd4dfd740 1 client.15891 shutdown 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/2249272325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 msgr2=0x5573967f8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/2249272325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573967f8eb0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4011df0 tx=0x7f4dc4004ab0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/2249272325 shutdown_connections 2024-04-26T18:31:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/2249272325 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55739680b780 0x55739680db90 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/2249272325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573967f8eb0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dab7fe640 0 client.15891 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/2249272325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 0x5573967f8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/2249272325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967f8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/2249272325 >> 172.21.15.84:0/2249272325 conn(0x5573967e8410 msgr2=0x5573967ee280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/2249272325 shutdown_connections 2024-04-26T18:31:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/2249272325 wait complete. 2024-04-26T18:31:56.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.798+0000 7f4dd4dfd740 20 client.15891 trim_cache size 0 max 16384 2024-04-26T18:31:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd4dfd740 1 Processor -- start 2024-04-26T18:31:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd4dfd740 1 -- start start 2024-04-26T18:31:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd4dfd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x557396806fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd4dfd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 0x5573967f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd4dfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573968134a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd4dfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1ac260 con 0x5573967fb4f0 2024-04-26T18:31:56.818 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd4dfd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1adda0 con 0x5573967ebe20 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd4dfd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1ac930 con 0x5573967ec740 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd09e4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573968134a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd09e4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573968134a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50232/0 (socket says 172.21.15.84:50232) 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd09e4640 1 -- 172.21.15.84:0/1126007176 learned_addr learned my addr 172.21.15.84:0/1126007176 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dcb7fe640 1 --2- 172.21.15.84:0/1126007176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 0x5573967f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd09e4640 1 -- 172.21.15.84:0/1126007176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 msgr2=0x5573967f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd09e4640 1 --2- 172.21.15.84:0/1126007176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 0x5573967f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd09e4640 1 -- 172.21.15.84:0/1126007176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 msgr2=0x557396806fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/1126007176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x557396806fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.813+0000 7f4dd09e4640 1 --2- 172.21.15.84:0/1126007176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x557396806fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4dd09e4640 1 -- 172.21.15.84:0/1126007176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573967f69c0 con 0x5573967fb4f0 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4dcb7fe640 1 --2- 172.21.15.84:0/1126007176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 0x5573967f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/1126007176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x557396806fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:31:56.819 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4dd09e4640 1 --2- 172.21.15.84:0/1126007176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573968134a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4011b50 tx=0x7f4dc40049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac2ce6ccbf28425 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4db1ffb640 1 -- 172.21.15.84:0/1126007176 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc401d430 con 0x5573967fb4f0 2024-04-26T18:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4db1ffb640 1 -- 172.21.15.84:0/1126007176 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc4021070 con 0x5573967fb4f0 2024-04-26T18:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4db1ffb640 1 -- 172.21.15.84:0/1126007176 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc401ce60 con 0x5573967fb4f0 2024-04-26T18:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/1126007176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 msgr2=0x5573968134a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/1126007176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573968134a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4011b50 tx=0x7f4dc40049c0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/1126007176 shutdown_connections 2024-04-26T18:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/1126007176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573968134a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/1126007176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 0x5573967f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/1126007176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x557396806fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/1126007176 >> 172.21.15.84:0/1126007176 conn(0x5573967e8410 msgr2=0x5573967f83f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/1126007176 shutdown_connections 2024-04-26T18:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.814+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/1126007176 wait complete. 2024-04-26T18:31:56.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.815+0000 7f4dd4dfd740 1 -- start start 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.815+0000 7f4dd4dfd740 1 Processor -- start 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.815+0000 7f4dd4dfd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967f5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.815+0000 7f4dd4dfd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 0x5573967f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.815+0000 7f4dd4dfd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573967f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.815+0000 7f4dd4dfd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1ad6d0 con 0x5573967fb4f0 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.815+0000 7f4dd4dfd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1ad000 con 0x5573967ebe20 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.815+0000 7f4dd4dfd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4dcc1aa240 con 0x5573967ec740 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.815+0000 7f4dd4dfd740 10 client.0 _ll_register_callbacks cb 0x7f4dc4008c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.815+0000 7f4dd09e4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573967f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.815+0000 7f4dd09e4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573967f5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50238/0 (socket says 172.21.15.84:50238) 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.815+0000 7f4dd09e4640 1 -- 172.21.15.84:0/1511272174 learned_addr learned my addr 172.21.15.84:0/1511272174 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dcb7fe640 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 0x5573967f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967f5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dd09e4640 1 -- 172.21.15.84:0/1511272174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 msgr2=0x5573967f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dd09e4640 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 0x5573967f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dd09e4640 1 -- 172.21.15.84:0/1511272174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 msgr2=0x5573967f5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dd09e4640 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967f5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dd09e4640 1 -- 172.21.15.84:0/1511272174 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557396870e70 con 0x5573967fb4f0 2024-04-26T18:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dd09e4640 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573967f5cf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4025f70 tx=0x7f4dc401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123dd30ae8e0457 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dab7fe640 1 -- 172.21.15.84:0/1511272174 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc403f070 con 0x5573967fb4f0 2024-04-26T18:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dab7fe640 1 -- 172.21.15.84:0/1511272174 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f4dc401cd60 con 0x5573967fb4f0 2024-04-26T18:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/1511272174 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc4021070 con 0x5573967fb4f0 2024-04-26T18:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dab7fe640 1 -- 172.21.15.84:0/1511272174 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4dc4004a60 con 0x5573967fb4f0 2024-04-26T18:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dd4dfd740 10 client.15897 fetch_fsmap learned FSMap version 29 2024-04-26T18:31:56.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/1511272174 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573968720d0 con 0x5573967fb4f0 2024-04-26T18:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.816+0000 7f4dab7fe640 1 -- 172.21.15.84:0/1511272174 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc4021070 con 0x5573967fb4f0 2024-04-26T18:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.817+0000 7f4dab7fe640 1 -- 172.21.15.84:0/1511272174 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4dc4037070 con 0x5573967fb4f0 2024-04-26T18:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.817+0000 7f4dd4dfd740 10 client.15897 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.817+0000 7f4dd4dfd740 10 client.15897 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.817+0000 7f4dd4dfd740 20 client.15897 populate_metadata read hostname 'smithi084' 2024-04-26T18:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.817+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5573968726d0 0x557396874af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.817+0000 7f4dd4dfd740 4 client.15897 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.817+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/1511272174 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557396759ee0 con 0x5573968726d0 2024-04-26T18:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.817+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5573968726d0 0x557396874af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.817+0000 7f4dcbfff640 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5573968726d0 0x557396874af0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d951e6b03abc6fed server_cookie=3257a9ba69323659 in_seq=0 out_seq=0 2024-04-26T18:31:56.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.817+0000 7f4dab7fe640 10 client.15897 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dab7fe640 1 -- 172.21.15.84:0/1511272174 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557396759ee0 con 0x5573968726d0 2024-04-26T18:31:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dab7fe640 10 client.15897 handle_command_reply: tid=0 2024-04-26T18:31:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dd4dfd740 1 client.15897 shutdown 2024-04-26T18:31:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/1511272174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 msgr2=0x5573967f5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573967f5cf0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4025f70 tx=0x7f4dc401c9a0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/1511272174 shutdown_connections 2024-04-26T18:31:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5573968726d0 0x557396874af0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573967fb4f0 0x5573967f5cf0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573967ec740 0x5573967f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dab7fe640 0 client.15897 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:31:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dd4dfd740 1 --2- 172.21.15.84:0/1511272174 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573967ebe20 0x5573967f5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/1511272174 >> 172.21.15.84:0/1511272174 conn(0x5573967e8410 msgr2=0x5573967ee170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/1511272174 shutdown_connections 2024-04-26T18:31:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dd4dfd740 1 -- 172.21.15.84:0/1511272174 wait complete. 2024-04-26T18:31:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.833+0000 7f4dd4dfd740 20 client.15897 trim_cache size 0 max 16384 2024-04-26T18:31:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:31:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:31:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:31:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:31:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.834+0000 7f4dd2447640 1 -- 172.21.15.84:0/3723828951 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4db40a3dc0 msgr2=0x7f4db40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.834+0000 7f4dd2447640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4db40a3dc0 0x7f4db40a6280 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4dc4005cc0 tx=0x7f4dc400d710 comp rx=0 tx=0).stop 2024-04-26T18:31:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.834+0000 7f4dd2447640 1 -- 172.21.15.84:0/3723828951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc101860 msgr2=0x7f4dcc199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:31:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.834+0000 7f4dd2447640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc101860 0x7f4dcc199090 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc005940 tx=0x7f4dbc004270 comp rx=0 tx=0).stop 2024-04-26T18:31:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.834+0000 7f4dd2447640 1 -- 172.21.15.84:0/3723828951 shutdown_connections 2024-04-26T18:31:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.834+0000 7f4dd2447640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4db40a3dc0 0x7f4db40a6280 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.834+0000 7f4dd2447640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4dcc107820 0x7f4dcc19d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.834+0000 7f4dd2447640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4dcc102220 0x7f4dcc1995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.834+0000 7f4dd2447640 1 --2- 172.21.15.84:0/3723828951 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4dcc101860 0x7f4dcc199090 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:31:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.834+0000 7f4dd2447640 1 -- 172.21.15.84:0/3723828951 >> 172.21.15.84:0/3723828951 conn(0x7f4dcc0789b0 msgr2=0x7f4dcc104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:31:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.834+0000 7f4dd2447640 1 -- 172.21.15.84:0/3723828951 shutdown_connections 2024-04-26T18:31:56.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:31:56.834+0000 7f4dd2447640 1 -- 172.21.15.84:0/3723828951 wait complete. 2024-04-26T18:31:57.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:56 smithi084 ceph-mon[29141]: pgmap v122: 97 pgs: 97 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-26T18:31:57.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:56 smithi103 ceph-mon[32727]: pgmap v122: 97 pgs: 97 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-26T18:31:57.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:56 smithi174 ceph-mon[32705]: pgmap v122: 97 pgs: 97 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-26T18:31:59.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:31:58 smithi084 ceph-mon[29141]: pgmap v123: 97 pgs: 97 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-26T18:31:59.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:31:58 smithi103 ceph-mon[32727]: pgmap v123: 97 pgs: 97 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-26T18:31:59.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:31:58 smithi174 ceph-mon[32705]: pgmap v123: 97 pgs: 97 active+clean; 3.3 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-04-26T18:32:01.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:00 smithi084 ceph-mon[29141]: pgmap v124: 97 pgs: 97 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-04-26T18:32:01.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:00 smithi103 ceph-mon[32727]: pgmap v124: 97 pgs: 97 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-04-26T18:32:01.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:00 smithi174 ceph-mon[32705]: pgmap v124: 97 pgs: 97 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 37 KiB/s wr, 24 op/s 2024-04-26T18:32:03.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:02 smithi084 ceph-mon[29141]: pgmap v125: 97 pgs: 97 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:32:03.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:02 smithi103 ceph-mon[32727]: pgmap v125: 97 pgs: 97 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:32:03.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:02 smithi174 ceph-mon[32705]: pgmap v125: 97 pgs: 97 active+clean; 3.5 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-04-26T18:32:05.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:04 smithi084 ceph-mon[29141]: pgmap v126: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 57 KiB/s wr, 37 op/s 2024-04-26T18:32:05.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:04 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:32:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:04 smithi103 ceph-mon[32727]: pgmap v126: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 57 KiB/s wr, 37 op/s 2024-04-26T18:32:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:04 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:32:05.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:04 smithi174 ceph-mon[32705]: pgmap v126: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 57 KiB/s wr, 37 op/s 2024-04-26T18:32:05.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:04 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:32:06.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:05 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:32:06.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:05 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:32:06.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:05 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:32:06.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:05 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:32:06.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:05 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:32:06.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:05 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:32:06.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:05 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:32:06.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:05 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:32:06.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:05 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:32:07.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:07 smithi174 ceph-mon[32705]: pgmap v127: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-26T18:32:07.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:07 smithi084 ceph-mon[29141]: pgmap v127: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-26T18:32:07.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:07 smithi103 ceph-mon[32727]: pgmap v127: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-26T18:32:09.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:09 smithi174 ceph-mon[32705]: pgmap v128: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-26T18:32:09.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:09 smithi084 ceph-mon[29141]: pgmap v128: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-26T18:32:09.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:09 smithi103 ceph-mon[32727]: pgmap v128: 97 pgs: 97 active+clean; 3.6 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-26T18:32:09.928 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.928+0000 7f23864d1640 1 -- 172.21.15.84:0/538573229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2380106800 msgr2=0x7f2380106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.928+0000 7f23864d1640 1 --2- 172.21.15.84:0/538573229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2380106800 0x7f2380106be0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2370009900 tx=0x7f237002a410 comp rx=0 tx=0).stop 2024-04-26T18:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.928+0000 7f23864d1640 1 -- 172.21.15.84:0/538573229 shutdown_connections 2024-04-26T18:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.928+0000 7f23864d1640 1 --2- 172.21.15.84:0/538573229 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f23801024d0 0x7f238010e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.928+0000 7f23864d1640 1 --2- 172.21.15.84:0/538573229 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2380101b10 0x7f2380101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.928+0000 7f23864d1640 1 --2- 172.21.15.84:0/538573229 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2380106800 0x7f2380106be0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.928+0000 7f23864d1640 1 -- 172.21.15.84:0/538573229 >> 172.21.15.84:0/538573229 conn(0x7f2380078610 msgr2=0x7f2380078a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.928+0000 7f23864d1640 1 -- 172.21.15.84:0/538573229 shutdown_connections 2024-04-26T18:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.928+0000 7f23864d1640 1 -- 172.21.15.84:0/538573229 wait complete. 2024-04-26T18:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.929+0000 7f23864d1640 1 Processor -- start 2024-04-26T18:32:09.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.929+0000 7f23864d1640 1 -- start start 2024-04-26T18:32:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.929+0000 7f23864d1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2380101b10 0x7f2380199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.929+0000 7f23864d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23801024d0 0x7f238019a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.931 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.929+0000 7f23864d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2380106800 0x7f23801a4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.929+0000 7f23864d1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2380111aa0 con 0x7f23801024d0 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.929+0000 7f23864d1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2380111920 con 0x7f2380101b10 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.929+0000 7f23864d1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2380111c20 con 0x7f2380106800 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.929+0000 7f237f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23801024d0 0x7f238019a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.929+0000 7f237f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23801024d0 0x7f238019a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52166/0 (socket says 172.21.15.84:52166) 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.929+0000 7f237f7fe640 1 -- 172.21.15.84:0/2213645921 learned_addr learned my addr 172.21.15.84:0/2213645921 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.930+0000 7f2384a6e640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2380106800 0x7f23801a4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.930+0000 7f237ffff640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2380101b10 0x7f2380199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.930+0000 7f237f7fe640 1 -- 172.21.15.84:0/2213645921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2380106800 msgr2=0x7f23801a4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.930+0000 7f237f7fe640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2380106800 0x7f23801a4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.930+0000 7f237f7fe640 1 -- 172.21.15.84:0/2213645921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2380101b10 msgr2=0x7f2380199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.930+0000 7f237f7fe640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2380101b10 0x7f2380199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.930+0000 7f237f7fe640 1 -- 172.21.15.84:0/2213645921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23801a53c0 con 0x7f23801024d0 2024-04-26T18:32:09.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.930+0000 7f237f7fe640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23801024d0 0x7f238019a340 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f237400d950 tx=0x7f237400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.930+0000 7f237d7fa640 1 -- 172.21.15.84:0/2213645921 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2374017030 con 0x7f23801024d0 2024-04-26T18:32:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.930+0000 7f237d7fa640 1 -- 172.21.15.84:0/2213645921 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2374004020 con 0x7f23801024d0 2024-04-26T18:32:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.930+0000 7f23864d1640 1 -- 172.21.15.84:0/2213645921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23801a5590 con 0x7f23801024d0 2024-04-26T18:32:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.930+0000 7f237d7fa640 1 -- 172.21.15.84:0/2213645921 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23740047a0 con 0x7f23801024d0 2024-04-26T18:32:09.933 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.930+0000 7f23864d1640 1 -- 172.21.15.84:0/2213645921 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23801a5ae0 con 0x7f23801024d0 2024-04-26T18:32:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f237d7fa640 1 -- 172.21.15.84:0/2213645921 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f2374009a30 con 0x7f23801024d0 2024-04-26T18:32:09.939 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f23891ea740 1 Processor -- start 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f23891ea740 1 -- start start 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f23891ea740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 0x55b58436e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f23891ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436e820 0x55b584378ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f23891ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 0x55b58437b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f23891ea740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b5842d4460 con 0x55b58436df00 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f23891ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b58428e590 con 0x55b5843790e0 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f23891ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b5842d0420 con 0x55b58436e820 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f237d7fa640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f23680a3dc0 0x7f23680a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f2384a6e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 0x55b58436e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f2384a6e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 0x55b58436e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52168/0 (socket says 172.21.15.84:52168) 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f2384a6e640 1 -- 172.21.15.84:0/1214324190 learned_addr learned my addr 172.21.15.84:0/1214324190 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f237f7fe640 1 --2- 172.21.15.84:0/1214324190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 0x55b58437b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f2384a6e640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f23680a3dc0 0x7f23680a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.940 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f237ffff640 1 --2- 172.21.15.84:0/1214324190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436e820 0x55b584378ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.933+0000 7f237d7fa640 1 -- 172.21.15.84:0/2213645921 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f23740d01e0 con 0x7f23801024d0 2024-04-26T18:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.934+0000 7f2384a6e640 1 -- 172.21.15.84:0/1214324190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436e820 msgr2=0x55b584378ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.934+0000 7f2384a6e640 1 --2- 172.21.15.84:0/1214324190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436e820 0x55b584378ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.934+0000 7f2384a6e640 1 -- 172.21.15.84:0/1214324190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 msgr2=0x55b58437b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.934+0000 7f2384a6e640 1 --2- 172.21.15.84:0/1214324190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 0x55b58437b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.934+0000 7f2384a6e640 1 -- 172.21.15.84:0/1214324190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b58429b0d0 con 0x55b58436df00 2024-04-26T18:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.934+0000 7f237ffff640 1 --2- 172.21.15.84:0/1214324190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436e820 0x55b584378ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.934+0000 7f2384a6e640 1 --2- 172.21.15.84:0/1214324190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 0x55b58436e2e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2378005cc0 tx=0x7f23780117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8fb2d3160cfd12 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.934+0000 7f2384a6e640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f23680a3dc0 0x7f23680a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2378011cb0 tx=0x7f2378016000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.934+0000 7f23667fc640 1 -- 172.21.15.84:0/1214324190 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378024070 con 0x55b58436df00 2024-04-26T18:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.934+0000 7f23667fc640 1 -- 172.21.15.84:0/1214324190 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f237800ac90 con 0x55b58436df00 2024-04-26T18:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.934+0000 7f23667fc640 1 -- 172.21.15.84:0/1214324190 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378020420 con 0x55b58436df00 2024-04-26T18:32:09.941 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.935+0000 7f23891ea740 1 -- 172.21.15.84:0/1214324190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 msgr2=0x55b58436e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.935+0000 7f23891ea740 1 --2- 172.21.15.84:0/1214324190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 0x55b58436e2e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2378005cc0 tx=0x7f23780117b0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.935+0000 7f23891ea740 1 -- 172.21.15.84:0/1214324190 shutdown_connections 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.935+0000 7f23891ea740 1 --2- 172.21.15.84:0/1214324190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 0x55b58437b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.935+0000 7f23891ea740 1 --2- 172.21.15.84:0/1214324190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436e820 0x55b584378ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.935+0000 7f23891ea740 1 --2- 172.21.15.84:0/1214324190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 0x55b58436e2e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.935+0000 7f23891ea740 1 -- 172.21.15.84:0/1214324190 >> 172.21.15.84:0/1214324190 conn(0x55b58436a410 msgr2=0x55b58436a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.935+0000 7f23891ea740 1 -- 172.21.15.84:0/1214324190 shutdown_connections 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.935+0000 7f23891ea740 1 -- 172.21.15.84:0/1214324190 wait complete. 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.935+0000 7f23891ea740 1 -- start start 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.935+0000 7f23891ea740 1 Processor -- start 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f23891ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436df00 0x55b584376110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f23891ea740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 0x55b584376650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f23891ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 0x55b584376b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.942 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f23891ea740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55b5843800c0 con 0x55b58436e820 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f23891ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55b58425bb60 con 0x55b5843790e0 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f23891ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55b584380240 con 0x55b58436df00 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f23891ea740 10 client.0 _ll_register_callbacks cb 0x55b5842802b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f237f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 0x55b584376650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f237f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 0x55b584376650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52182/0 (socket says 172.21.15.84:52182) 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f237f7fe640 1 -- 172.21.15.84:0/3365873056 learned_addr learned my addr 172.21.15.84:0/3365873056 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f237ffff640 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436df00 0x55b584376110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f2384a6e640 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 0x55b584376b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f237f7fe640 1 -- 172.21.15.84:0/3365873056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436df00 msgr2=0x55b584376110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f237f7fe640 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436df00 0x55b584376110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f237f7fe640 1 -- 172.21.15.84:0/3365873056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 msgr2=0x55b584376b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f237f7fe640 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 0x55b584376b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f237f7fe640 1 -- 172.21.15.84:0/3365873056 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b583fe6210 con 0x55b58436e820 2024-04-26T18:32:09.943 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f237ffff640 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436df00 0x55b584376110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f237f7fe640 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 0x55b584376650 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2374098b40 tx=0x7f237409bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45cd2b702a8a2c92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f235f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f235f7fe640 1 -- 172.21.15.84:0/3365873056 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23740b5060 con 0x55b58436e820 2024-04-26T18:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f235f7fe640 1 -- 172.21.15.84:0/3365873056 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f23740cdba0 con 0x55b58436e820 2024-04-26T18:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.936+0000 7f23891ea740 1 -- 172.21.15.84:0/3365873056 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f237800ac90 con 0x55b58436e820 2024-04-26T18:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.937+0000 7f235f7fe640 1 -- 172.21.15.84:0/3365873056 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f237409c170 con 0x55b58436e820 2024-04-26T18:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.937+0000 7f23891ea740 10 client.15909 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.937+0000 7f23891ea740 1 -- 172.21.15.84:0/3365873056 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5840b2620 con 0x55b58436e820 2024-04-26T18:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.937+0000 7f235f7fe640 1 -- 172.21.15.84:0/3365873056 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f237800ac90 con 0x55b58436e820 2024-04-26T18:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.937+0000 7f235f7fe640 1 -- 172.21.15.84:0/3365873056 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2374096ce0 con 0x55b58436e820 2024-04-26T18:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.937+0000 7f23891ea740 10 client.15909 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.937+0000 7f23891ea740 10 client.15909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:32:09.944 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.937+0000 7f23891ea740 20 client.15909 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.937+0000 7f23891ea740 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b58438d6b0 0x55b58438faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.937+0000 7f23891ea740 4 client.15909 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.937+0000 7f23891ea740 1 -- 172.21.15.84:0/3365873056 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5842e2070 con 0x55b58438d6b0 2024-04-26T18:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.937+0000 7f237ffff640 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b58438d6b0 0x55b58438faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.938+0000 7f235f7fe640 10 client.15909 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.938+0000 7f237ffff640 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b58438d6b0 0x55b58438faa0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a75f086a04e1df server_cookie=9d6b56876e50c6af in_seq=0 out_seq=0 2024-04-26T18:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.938+0000 7f235f7fe640 1 -- 172.21.15.84:0/3365873056 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b5842e2070 con 0x55b58438d6b0 2024-04-26T18:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.938+0000 7f235f7fe640 10 client.15909 handle_command_reply: tid=0 2024-04-26T18:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.938+0000 7f23891ea740 1 client.15909 shutdown 2024-04-26T18:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.939+0000 7f23891ea740 1 -- 172.21.15.84:0/3365873056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 msgr2=0x55b584376650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.939+0000 7f23891ea740 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 0x55b584376650 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2374098b40 tx=0x7f237409bde0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.939+0000 7f23891ea740 1 -- 172.21.15.84:0/3365873056 shutdown_connections 2024-04-26T18:32:09.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.939+0000 7f23891ea740 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b58438d6b0 0x55b58438faa0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.939+0000 7f23891ea740 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 0x55b584376b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.939+0000 7f23891ea740 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 0x55b584376650 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.939+0000 7f235f7fe640 0 client.15909 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.939+0000 7f23891ea740 1 --2- 172.21.15.84:0/3365873056 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436df00 0x55b584376110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.939+0000 7f23891ea740 1 -- 172.21.15.84:0/3365873056 >> 172.21.15.84:0/3365873056 conn(0x55b58436a410 msgr2=0x55b584370140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.939+0000 7f23891ea740 1 -- 172.21.15.84:0/3365873056 shutdown_connections 2024-04-26T18:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.939+0000 7f23891ea740 1 -- 172.21.15.84:0/3365873056 wait complete. 2024-04-26T18:32:09.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.939+0000 7f23891ea740 20 client.15909 trim_cache size 0 max 16384 2024-04-26T18:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f23891ea740 1 Processor -- start 2024-04-26T18:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f23891ea740 1 -- start start 2024-04-26T18:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f23891ea740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 0x55b584388f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f23891ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b58436e820 0x55b584369d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f23891ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5843790e0 0x55b584395370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f23891ea740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f23801ac330 con 0x55b58436df00 2024-04-26T18:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f23891ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f23801ade70 con 0x55b58436e820 2024-04-26T18:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f23891ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23801aca00 con 0x55b5843790e0 2024-04-26T18:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f237ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 0x55b584388f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f237ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 0x55b584388f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52190/0 (socket says 172.21.15.84:52190) 2024-04-26T18:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f237ffff640 1 -- 172.21.15.84:0/1984360434 learned_addr learned my addr 172.21.15.84:0/1984360434 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f2384a6e640 1 --2- 172.21.15.84:0/1984360434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5843790e0 0x55b584395370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.957 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f237f7fe640 1 --2- 172.21.15.84:0/1984360434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b58436e820 0x55b584369d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f237ffff640 1 -- 172.21.15.84:0/1984360434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5843790e0 msgr2=0x55b584395370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f237ffff640 1 --2- 172.21.15.84:0/1984360434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5843790e0 0x55b584395370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f237ffff640 1 -- 172.21.15.84:0/1984360434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b58436e820 msgr2=0x55b584369d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f237ffff640 1 --2- 172.21.15.84:0/1984360434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b58436e820 0x55b584369d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f237ffff640 1 -- 172.21.15.84:0/1984360434 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b58438fe10 con 0x55b58436df00 2024-04-26T18:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f237f7fe640 1 --2- 172.21.15.84:0/1984360434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b58436e820 0x55b584369d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.955+0000 7f237ffff640 1 --2- 172.21.15.84:0/1984360434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 0x55b584388f80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2370031440 tx=0x7f237002ab10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3292a7ef2a548f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.956+0000 7f2365ffb640 1 -- 172.21.15.84:0/1984360434 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370049070 con 0x55b58436df00 2024-04-26T18:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.956+0000 7f2365ffb640 1 -- 172.21.15.84:0/1984360434 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23700044b0 con 0x55b58436df00 2024-04-26T18:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.956+0000 7f2365ffb640 1 -- 172.21.15.84:0/1984360434 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2370035040 con 0x55b58436df00 2024-04-26T18:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.956+0000 7f23891ea740 1 -- 172.21.15.84:0/1984360434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 msgr2=0x55b584388f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.956+0000 7f23891ea740 1 --2- 172.21.15.84:0/1984360434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 0x55b584388f80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f2370031440 tx=0x7f237002ab10 comp rx=0 tx=0).stop 2024-04-26T18:32:09.958 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.956+0000 7f23891ea740 1 -- 172.21.15.84:0/1984360434 shutdown_connections 2024-04-26T18:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.956+0000 7f23891ea740 1 --2- 172.21.15.84:0/1984360434 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b5843790e0 0x55b584395370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.956+0000 7f23891ea740 1 --2- 172.21.15.84:0/1984360434 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b58436e820 0x55b584369d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.956+0000 7f23891ea740 1 --2- 172.21.15.84:0/1984360434 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436df00 0x55b584388f80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.956+0000 7f23891ea740 1 -- 172.21.15.84:0/1984360434 >> 172.21.15.84:0/1984360434 conn(0x55b58436a410 msgr2=0x55b5843781d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.956+0000 7f23891ea740 1 -- 172.21.15.84:0/1984360434 shutdown_connections 2024-04-26T18:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.956+0000 7f23891ea740 1 -- 172.21.15.84:0/1984360434 wait complete. 2024-04-26T18:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.957+0000 7f23891ea740 1 -- start start 2024-04-26T18:32:09.959 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.957+0000 7f23891ea740 1 Processor -- start 2024-04-26T18:32:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.957+0000 7f23891ea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436df00 0x55b584385680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.957+0000 7f23891ea740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 0x55b584382460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.957+0000 7f23891ea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 0x55b5843829a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.957+0000 7f23891ea740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f23801ad7a0 con 0x55b58436e820 2024-04-26T18:32:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.957+0000 7f23891ea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f23801ad0d0 con 0x55b5843790e0 2024-04-26T18:32:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.957+0000 7f23891ea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f23801aa310 con 0x55b58436df00 2024-04-26T18:32:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.957+0000 7f23891ea740 10 client.0 _ll_register_callbacks cb 0x7f23740044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:09.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.957+0000 7f237f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 0x55b584382460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.957+0000 7f237f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 0x55b584382460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52194/0 (socket says 172.21.15.84:52194) 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.957+0000 7f237f7fe640 1 -- 172.21.15.84:0/3757164666 learned_addr learned my addr 172.21.15.84:0/3757164666 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f2384a6e640 1 --2- 172.21.15.84:0/3757164666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 0x55b5843829a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f237f7fe640 1 -- 172.21.15.84:0/3757164666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436df00 msgr2=0x55b584385680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f237f7fe640 1 --2- 172.21.15.84:0/3757164666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436df00 0x55b584385680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f237f7fe640 1 -- 172.21.15.84:0/3757164666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 msgr2=0x55b5843829a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f237f7fe640 1 --2- 172.21.15.84:0/3757164666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 0x55b5843829a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f237f7fe640 1 -- 172.21.15.84:0/3757164666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b584382ee0 con 0x55b58436e820 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f237f7fe640 1 --2- 172.21.15.84:0/3757164666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 0x55b584382460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2374005fd0 tx=0x7f2374004400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b3da3794a7a8658 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f235f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f235f7fe640 1 -- 172.21.15.84:0/3757164666 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23740a4e30 con 0x55b58436e820 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f235f7fe640 1 -- 172.21.15.84:0/3757164666 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f23740a4b50 con 0x55b58436e820 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f23891ea740 1 -- 172.21.15.84:0/3757164666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23700044b0 con 0x55b58436e820 2024-04-26T18:32:09.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f235f7fe640 1 -- 172.21.15.84:0/3757164666 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f237409bf40 con 0x55b58436e820 2024-04-26T18:32:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f23891ea740 10 client.15915 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f23891ea740 1 -- 172.21.15.84:0/3757164666 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5843f4820 con 0x55b58436e820 2024-04-26T18:32:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f235f7fe640 1 -- 172.21.15.84:0/3757164666 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23700044b0 con 0x55b58436e820 2024-04-26T18:32:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f235f7fe640 1 -- 172.21.15.84:0/3757164666 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2374096ce0 con 0x55b58436e820 2024-04-26T18:32:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f23891ea740 10 client.15915 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f23891ea740 10 client.15915 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:32:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f23891ea740 20 client.15915 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f23891ea740 1 --2- 172.21.15.84:0/3757164666 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b5843f4de0 0x55b5843f7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f23891ea740 4 client.15915 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:32:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.958+0000 7f23891ea740 1 -- 172.21.15.84:0/3757164666 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b5842dbee0 con 0x55b5843f4de0 2024-04-26T18:32:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.959+0000 7f237ffff640 1 --2- 172.21.15.84:0/3757164666 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b5843f4de0 0x55b5843f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.959+0000 7f237ffff640 1 --2- 172.21.15.84:0/3757164666 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b5843f4de0 0x55b5843f7220 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=846765b8aef32430 server_cookie=827124c09a85727d in_seq=0 out_seq=0 2024-04-26T18:32:09.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.959+0000 7f235f7fe640 10 client.15915 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f235f7fe640 1 -- 172.21.15.84:0/3757164666 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b5842dbee0 con 0x55b5843f4de0 2024-04-26T18:32:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f235f7fe640 10 client.15915 handle_command_reply: tid=0 2024-04-26T18:32:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23891ea740 1 client.15915 shutdown 2024-04-26T18:32:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23891ea740 1 -- 172.21.15.84:0/3757164666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 msgr2=0x55b584382460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:09.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23891ea740 1 --2- 172.21.15.84:0/3757164666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 0x55b584382460 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f2374005fd0 tx=0x7f2374004400 comp rx=0 tx=0).stop 2024-04-26T18:32:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23891ea740 1 -- 172.21.15.84:0/3757164666 shutdown_connections 2024-04-26T18:32:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23891ea740 1 --2- 172.21.15.84:0/3757164666 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55b5843f4de0 0x55b5843f7220 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23891ea740 1 --2- 172.21.15.84:0/3757164666 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55b5843790e0 0x55b5843829a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23891ea740 1 --2- 172.21.15.84:0/3757164666 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55b58436e820 0x55b584382460 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23891ea740 1 --2- 172.21.15.84:0/3757164666 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55b58436df00 0x55b584385680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f235f7fe640 0 client.15915 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23891ea740 1 -- 172.21.15.84:0/3757164666 >> 172.21.15.84:0/3757164666 conn(0x55b58436a410 msgr2=0x55b58436f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23891ea740 1 -- 172.21.15.84:0/3757164666 shutdown_connections 2024-04-26T18:32:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23891ea740 1 -- 172.21.15.84:0/3757164666 wait complete. 2024-04-26T18:32:09.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23891ea740 20 client.15915 trim_cache size 0 max 16384 2024-04-26T18:32:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:32:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:32:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:32:09.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:32:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23864d1640 1 -- 172.21.15.84:0/2213645921 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f23680a3dc0 msgr2=0x7f23680a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23864d1640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f23680a3dc0 0x7f23680a6280 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2378011cb0 tx=0x7f2378016000 comp rx=0 tx=0).stop 2024-04-26T18:32:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.975+0000 7f23864d1640 1 -- 172.21.15.84:0/2213645921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23801024d0 msgr2=0x7f238019a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.976+0000 7f23864d1640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23801024d0 0x7f238019a340 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f237400d950 tx=0x7f237400de10 comp rx=0 tx=0).stop 2024-04-26T18:32:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.976+0000 7f23864d1640 1 -- 172.21.15.84:0/2213645921 shutdown_connections 2024-04-26T18:32:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.976+0000 7f23864d1640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f23680a3dc0 0x7f23680a6280 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.976+0000 7f23864d1640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2380106800 0x7f23801a4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.976+0000 7f23864d1640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f23801024d0 0x7f238019a340 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.976+0000 7f23864d1640 1 --2- 172.21.15.84:0/2213645921 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2380101b10 0x7f2380199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.976+0000 7f23864d1640 1 -- 172.21.15.84:0/2213645921 >> 172.21.15.84:0/2213645921 conn(0x7f2380078610 msgr2=0x7f23800ff810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.976+0000 7f23864d1640 1 -- 172.21.15.84:0/2213645921 shutdown_connections 2024-04-26T18:32:09.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:09.976+0000 7f23864d1640 1 -- 172.21.15.84:0/2213645921 wait complete. 2024-04-26T18:32:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:11 smithi174 ceph-mon[32705]: pgmap v129: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 60 KiB/s wr, 37 op/s 2024-04-26T18:32:11.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:11 smithi084 ceph-mon[29141]: pgmap v129: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 60 KiB/s wr, 37 op/s 2024-04-26T18:32:11.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:11 smithi103 ceph-mon[32727]: pgmap v129: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 60 KiB/s wr, 37 op/s 2024-04-26T18:32:12.526 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.0) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-26T18:32:12.527 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.5 2024-04-26T18:32:12.527 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:32:12.776 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:32:12.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:12 smithi084 ceph-mon[29141]: pgmap v130: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 KiB/s wr, 27 op/s 2024-04-26T18:32:12.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:12 smithi103 ceph-mon[32727]: pgmap v130: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 KiB/s wr, 27 op/s 2024-04-26T18:32:12.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:12 smithi174 ceph-mon[32705]: pgmap v130: 97 pgs: 97 active+clean; 3.7 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 46 KiB/s wr, 27 op/s 2024-04-26T18:32:14.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.542+0000 7f6286037700 1 -- 172.21.15.84:0/3448491286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6280108390 msgr2=0x7f6280108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:14.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.542+0000 7f6286037700 1 --2- 172.21.15.84:0/3448491286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6280108390 0x7f6280108770 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f6270009990 tx=0x7f6270009ca0 comp rx=0 tx=0).stop 2024-04-26T18:32:14.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.542+0000 7f6286037700 1 -- 172.21.15.84:0/3448491286 shutdown_connections 2024-04-26T18:32:14.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.542+0000 7f6286037700 1 --2- 172.21.15.84:0/3448491286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6280108390 0x7f6280108770 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:14.568 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.542+0000 7f6286037700 1 --2- 172.21.15.84:0/3448491286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6280102ce0 0x7f628010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:14.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.542+0000 7f6286037700 1 --2- 172.21.15.84:0/3448491286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6280102320 0x7f62801027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:14.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.542+0000 7f6286037700 1 -- 172.21.15.84:0/3448491286 >> 172.21.15.84:0/3448491286 conn(0x7f62800792d0 msgr2=0x7f62800796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:14.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.542+0000 7f6286037700 1 -- 172.21.15.84:0/3448491286 shutdown_connections 2024-04-26T18:32:14.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.542+0000 7f6286037700 1 -- 172.21.15.84:0/3448491286 wait complete. 2024-04-26T18:32:14.569 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.542+0000 7f6286037700 1 Processor -- start 2024-04-26T18:32:14.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.542+0000 7f6286037700 1 -- start start 2024-04-26T18:32:14.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f6286037700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6280102320 0x7f628019f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:14.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f6286037700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6280102ce0 0x7f628019fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:14.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f6286037700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6280108390 0x7f6280199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:14.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f6286037700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f62801125b0 con 0x7f6280108390 2024-04-26T18:32:14.570 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f6286037700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6280112430 con 0x7f6280102320 2024-04-26T18:32:14.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f6286037700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6280112730 con 0x7f6280102ce0 2024-04-26T18:32:14.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6280108390 0x7f6280199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:14.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6280108390 0x7f6280199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52220/0 (socket says 172.21.15.84:52220) 2024-04-26T18:32:14.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627ffff700 1 -- 172.21.15.84:0/1955744811 learned_addr learned my addr 172.21.15.84:0/1955744811 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:14.571 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627effd700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6280102ce0 0x7f628019fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:14.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627f7fe700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6280102320 0x7f628019f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:14.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627ffff700 1 -- 172.21.15.84:0/1955744811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6280102ce0 msgr2=0x7f628019fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:14.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627ffff700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6280102ce0 0x7f628019fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:14.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627ffff700 1 -- 172.21.15.84:0/1955744811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6280102320 msgr2=0x7f628019f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:14.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627ffff700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6280102320 0x7f628019f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:14.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627ffff700 1 -- 172.21.15.84:0/1955744811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f628019a1c0 con 0x7f6280108390 2024-04-26T18:32:14.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627f7fe700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6280102320 0x7f628019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:14.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627ffff700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6280108390 0x7f6280199a60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f627800b970 tx=0x7f627800bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:14.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627cff9700 1 -- 172.21.15.84:0/1955744811 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6278019070 con 0x7f6280108390 2024-04-26T18:32:14.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f6286037700 1 -- 172.21.15.84:0/1955744811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f628019a4b0 con 0x7f6280108390 2024-04-26T18:32:14.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627cff9700 1 -- 172.21.15.84:0/1955744811 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6278014440 con 0x7f6280108390 2024-04-26T18:32:14.573 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f627cff9700 1 -- 172.21.15.84:0/1955744811 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f627800ce60 con 0x7f6280108390 2024-04-26T18:32:14.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.543+0000 7f6286037700 1 -- 172.21.15.84:0/1955744811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62801a6770 con 0x7f6280108390 2024-04-26T18:32:14.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.544+0000 7f6286037700 1 -- 172.21.15.84:0/1955744811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f628006a110 con 0x7f6280108390 2024-04-26T18:32:14.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.545+0000 7f627cff9700 1 -- 172.21.15.84:0/1955744811 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f627801c070 con 0x7f6280108390 2024-04-26T18:32:14.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.546+0000 7f627cff9700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f62680a3db0 0x7f62680a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:14.574 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.546+0000 7f627f7fe700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f62680a3db0 0x7f62680a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:14.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.546+0000 7f627cff9700 1 -- 172.21.15.84:0/1955744811 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f62780d0020 con 0x7f6280108390 2024-04-26T18:32:14.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.546+0000 7f627f7fe700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f62680a3db0 0x7f62680a6270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6270000c00 tx=0x7f6270005fb0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:14.575 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.550+0000 7f627cff9700 1 -- 172.21.15.84:0/1955744811 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f627809c7c0 con 0x7f6280108390 2024-04-26T18:32:14.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.801+0000 7f6286037700 1 -- 172.21.15.84:0/1955744811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62801a6a00 con 0x7f6280108390 2024-04-26T18:32:14.801 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:14 smithi084 ceph-mon[29141]: pgmap v131: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 64 KiB/s wr, 37 op/s 2024-04-26T18:32:14.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.802+0000 7f627cff9700 1 -- 172.21.15.84:0/1955744811 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f627809c0c0 con 0x7f6280108390 2024-04-26T18:32:14.802 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:32:14.802 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:32:14.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.804+0000 7f6286037700 1 -- 172.21.15.84:0/1955744811 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f62680a3db0 msgr2=0x7f62680a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:14.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.804+0000 7f6286037700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f62680a3db0 0x7f62680a6270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6270000c00 tx=0x7f6270005fb0 comp rx=0 tx=0).stop 2024-04-26T18:32:14.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.804+0000 7f6286037700 1 -- 172.21.15.84:0/1955744811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6280108390 msgr2=0x7f6280199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:14.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.804+0000 7f6286037700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6280108390 0x7f6280199a60 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f627800b970 tx=0x7f627800bd30 comp rx=0 tx=0).stop 2024-04-26T18:32:14.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.804+0000 7f6286037700 1 -- 172.21.15.84:0/1955744811 shutdown_connections 2024-04-26T18:32:14.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.804+0000 7f6286037700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f62680a3db0 0x7f62680a6270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:14.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.804+0000 7f6286037700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6280108390 0x7f6280199a60 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:14.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.804+0000 7f6286037700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6280102320 0x7f628019f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:14.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.804+0000 7f6286037700 1 --2- 172.21.15.84:0/1955744811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6280102ce0 0x7f628019fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:14.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.804+0000 7f6286037700 1 -- 172.21.15.84:0/1955744811 >> 172.21.15.84:0/1955744811 conn(0x7f62800792d0 msgr2=0x7f628006b2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:14.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.805+0000 7f6286037700 1 -- 172.21.15.84:0/1955744811 shutdown_connections 2024-04-26T18:32:14.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:14.805+0000 7f6286037700 1 -- 172.21.15.84:0/1955744811 wait complete. 2024-04-26T18:32:14.806 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:32:14.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:14 smithi103 ceph-mon[32727]: pgmap v131: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 64 KiB/s wr, 37 op/s 2024-04-26T18:32:14.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:14 smithi174 ceph-mon[32705]: pgmap v131: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 64 KiB/s wr, 37 op/s 2024-04-26T18:32:15.584 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 60 --expiration 79.5 --await-for 42.0' 2024-04-26T18:32:15.584 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db / --timeout 60 --expiration 79.5 --await-for 42.0 2024-04-26T18:32:15.835 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:32:15.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:15 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1955744811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:32:15.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:15 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1955744811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:32:15.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:15 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1955744811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:32:16.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:16 smithi084 ceph-mon[29141]: pgmap v132: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-26T18:32:16.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:16 smithi103 ceph-mon[32727]: pgmap v132: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-26T18:32:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:16 smithi174 ceph-mon[32705]: pgmap v132: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-26T18:32:17.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.665+0000 7ff434917700 1 -- 172.21.15.84:0/2118138003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff430101a10 msgr2=0x7ff430101e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:17.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.665+0000 7ff434917700 1 --2- 172.21.15.84:0/2118138003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff430101a10 0x7ff430101e90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff424009990 tx=0x7ff424009ca0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.665+0000 7ff434917700 1 -- 172.21.15.84:0/2118138003 shutdown_connections 2024-04-26T18:32:17.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.665+0000 7ff434917700 1 --2- 172.21.15.84:0/2118138003 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff4301023d0 0x7ff430110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.665+0000 7ff434917700 1 --2- 172.21.15.84:0/2118138003 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff430109520 0x7ff430109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.665+0000 7ff434917700 1 --2- 172.21.15.84:0/2118138003 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff430101a10 0x7ff430101e90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.665+0000 7ff434917700 1 -- 172.21.15.84:0/2118138003 >> 172.21.15.84:0/2118138003 conn(0x7ff4300789c0 msgr2=0x7ff4300ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.665+0000 7ff434917700 1 -- 172.21.15.84:0/2118138003 shutdown_connections 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.665+0000 7ff434917700 1 -- 172.21.15.84:0/2118138003 wait complete. 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.665+0000 7ff434917700 1 Processor -- start 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.665+0000 7ff434917700 1 -- start start 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff434917700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff430101a10 0x7ff4301a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff434917700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4301023d0 0x7ff4301a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff434917700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff430109520 0x7ff43019ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff434917700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff430113630 con 0x7ff4301023d0 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff434917700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301134b0 con 0x7ff430109520 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff434917700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301137b0 con 0x7ff430101a10 2024-04-26T18:32:17.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff42dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4301023d0 0x7ff4301a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff42dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4301023d0 0x7ff4301a1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55754/0 (socket says 172.21.15.84:55754) 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff42dd9b700 1 -- 172.21.15.84:0/4029279391 learned_addr learned my addr 172.21.15.84:0/4029279391 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff42e59c700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff430101a10 0x7ff4301a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff42dd9b700 1 -- 172.21.15.84:0/4029279391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff430101a10 msgr2=0x7ff4301a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff42dd9b700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff430101a10 0x7ff4301a18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff42dd9b700 1 -- 172.21.15.84:0/4029279391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff430109520 msgr2=0x7ff43019ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff42ed9d700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff430109520 0x7ff43019ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff42dd9b700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff430109520 0x7ff43019ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff42dd9b700 1 -- 172.21.15.84:0/4029279391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43019c1d0 con 0x7ff4301023d0 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff42ed9d700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff430109520 0x7ff43019ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff42dd9b700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4301023d0 0x7ff4301a1df0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff4240057c0 tx=0x7ff4240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff41f7fe700 1 -- 172.21.15.84:0/4029279391 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424027070 con 0x7ff4301023d0 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff434917700 1 -- 172.21.15.84:0/4029279391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43019c460 con 0x7ff4301023d0 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff41f7fe700 1 -- 172.21.15.84:0/4029279391 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff424003f20 con 0x7ff4301023d0 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff41f7fe700 1 -- 172.21.15.84:0/4029279391 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff424022bc0 con 0x7ff4301023d0 2024-04-26T18:32:17.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.666+0000 7ff434917700 1 -- 172.21.15.84:0/4029279391 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4301a8780 con 0x7ff4301023d0 2024-04-26T18:32:17.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.669+0000 7ff41f7fe700 1 -- 172.21.15.84:0/4029279391 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff42402a070 con 0x7ff4301023d0 2024-04-26T18:32:17.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.669+0000 7ff41f7fe700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff4180a3e50 0x7ff4180a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.670+0000 7ff42e59c700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff4180a3e50 0x7ff4180a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.670+0000 7ff41f7fe700 1 -- 172.21.15.84:0/4029279391 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7ff4240d1790 con 0x7ff4301023d0 2024-04-26T18:32:17.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.670+0000 7ff42e59c700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff4180a3e50 0x7ff4180a6310 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff43019cbb0 tx=0x7ff42000a380 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:17.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.670+0000 7ff43f1a9b80 1 Processor -- start 2024-04-26T18:32:17.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.670+0000 7ff43f1a9b80 1 -- start start 2024-04-26T18:32:17.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff43f1a9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4df30 0x55bad5d4e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff43f1a9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4e850 0x55bad5d482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff43f1a9b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d48d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff43f1a9b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bad5c9a9c0 con 0x55bad5d48900 2024-04-26T18:32:17.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff43f1a9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bad5c00ac0 con 0x55bad5d4df30 2024-04-26T18:32:17.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff43f1a9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bad5c96980 con 0x55bad5d4e850 2024-04-26T18:32:17.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff42ed9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d48d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff42ed9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d48d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55758/0 (socket says 172.21.15.84:55758) 2024-04-26T18:32:17.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff42ed9d700 1 -- 172.21.15.84:0/264883919 learned_addr learned my addr 172.21.15.84:0/264883919 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:17.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff42dd9b700 1 --2- 172.21.15.84:0/264883919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4e850 0x55bad5d482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.678 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff42e59c700 1 --2- 172.21.15.84:0/264883919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4df30 0x55bad5d4e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff42ed9d700 1 -- 172.21.15.84:0/264883919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4e850 msgr2=0x55bad5d482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff42ed9d700 1 --2- 172.21.15.84:0/264883919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4e850 0x55bad5d482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff42ed9d700 1 -- 172.21.15.84:0/264883919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4df30 msgr2=0x55bad5d4e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff42ed9d700 1 --2- 172.21.15.84:0/264883919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4df30 0x55bad5d4e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff42ed9d700 1 -- 172.21.15.84:0/264883919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad5be7a60 con 0x55bad5d48900 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff42ed9d700 1 --2- 172.21.15.84:0/264883919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d48d60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff428009ce0 tx=0x7ff428007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4e3cc7c1b8abff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.671+0000 7ff41cff9700 1 -- 172.21.15.84:0/264883919 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428004bb0 con 0x55bad5d48900 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.672+0000 7ff41cff9700 1 -- 172.21.15.84:0/264883919 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff428004d50 con 0x55bad5d48900 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.672+0000 7ff41cff9700 1 -- 172.21.15.84:0/264883919 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4280147c0 con 0x55bad5d48900 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.672+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/264883919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 msgr2=0x55bad5d48d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.672+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/264883919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d48d60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff428009ce0 tx=0x7ff428007500 comp rx=0 tx=0).stop 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.672+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/264883919 shutdown_connections 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.672+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/264883919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d48d60 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.679 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.672+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/264883919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4df30 0x55bad5d4e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.672+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/264883919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4e850 0x55bad5d482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.672+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/264883919 >> 172.21.15.84:0/264883919 conn(0x55bad5c76750 msgr2=0x55bad5d43090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.672+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/264883919 shutdown_connections 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.672+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/264883919 wait complete. 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff43f1a9b80 1 -- start start 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff43f1a9b80 1 Processor -- start 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff43f1a9b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d544c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff43f1a9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 0x55bad5d51860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff43f1a9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4e850 0x55bad5d51da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff43f1a9b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55bad5d552c0 con 0x55bad5d48900 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff43f1a9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55bad5c54c10 con 0x55bad5d4e850 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff43f1a9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55bad5d55440 con 0x55bad5d4df30 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff43f1a9b80 10 client.0 _ll_register_callbacks cb 0x55bad5c32ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff42e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff42e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d544c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55760/0 (socket says 172.21.15.84:55760) 2024-04-26T18:32:17.680 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff42e59c700 1 -- 172.21.15.84:0/1792214093 learned_addr learned my addr 172.21.15.84:0/1792214093 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff42ed9d700 1 --2- 172.21.15.84:0/1792214093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4e850 0x55bad5d51da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff42dd9b700 1 --2- 172.21.15.84:0/1792214093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 0x55bad5d51860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff42e59c700 1 -- 172.21.15.84:0/1792214093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 msgr2=0x55bad5d51860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.673+0000 7ff42e59c700 1 --2- 172.21.15.84:0/1792214093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 0x55bad5d51860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff42e59c700 1 -- 172.21.15.84:0/1792214093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4e850 msgr2=0x55bad5d51da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff42e59c700 1 --2- 172.21.15.84:0/1792214093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4e850 0x55bad5d51da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff42e59c700 1 -- 172.21.15.84:0/1792214093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bad5c58610 con 0x55bad5d48900 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff42e59c700 1 --2- 172.21.15.84:0/1792214093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d544c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff4200036d0 tx=0x7ff4200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb758803d4962aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff415ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff415ffb700 1 -- 172.21.15.84:0/1792214093 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff420016070 con 0x55bad5d48900 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff415ffb700 1 -- 172.21.15.84:0/1792214093 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7ff420010840 con 0x55bad5d48900 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/1792214093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff428004d50 con 0x55bad5d48900 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff415ffb700 1 -- 172.21.15.84:0/1792214093 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff420010a80 con 0x55bad5d48900 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff43f1a9b80 10 client.15933 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:17.681 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/1792214093 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bad5d5fa60 con 0x55bad5d48900 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff415ffb700 1 -- 172.21.15.84:0/1792214093 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff428004d50 con 0x55bad5d48900 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff415ffb700 1 -- 172.21.15.84:0/1792214093 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff420030cb0 con 0x55bad5d48900 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff43f1a9b80 10 client.15933 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.674+0000 7ff43f1a9b80 10 client.15933 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.675+0000 7ff43f1a9b80 20 client.15933 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.675+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/1792214093 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55bad5d61ff0 0x55bad5d64420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.675+0000 7ff43f1a9b80 4 client.15933 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.675+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/1792214093 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bad5ca83a0 con 0x55bad5d61ff0 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.675+0000 7ff42dd9b700 1 --2- 172.21.15.84:0/1792214093 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55bad5d61ff0 0x55bad5d64420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.675+0000 7ff415ffb700 10 client.15933 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.675+0000 7ff42dd9b700 1 --2- 172.21.15.84:0/1792214093 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55bad5d61ff0 0x55bad5d64420 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a12252572419bcb server_cookie=7dbf8d5f5eb88556 in_seq=0 out_seq=0 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.676+0000 7ff415ffb700 1 -- 172.21.15.84:0/1792214093 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bad5ca83a0 con 0x55bad5d61ff0 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.676+0000 7ff415ffb700 10 client.15933 handle_command_reply: tid=0 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.676+0000 7ff43f1a9b80 1 client.15933 shutdown 2024-04-26T18:32:17.682 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.677+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/1792214093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 msgr2=0x55bad5d544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:17.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.677+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/1792214093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d544c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7ff4200036d0 tx=0x7ff4200047c0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.677+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/1792214093 shutdown_connections 2024-04-26T18:32:17.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.677+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/1792214093 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55bad5d61ff0 0x55bad5d64420 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.677+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/1792214093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d544c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.677+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/1792214093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4e850 0x55bad5d51da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.677+0000 7ff415ffb700 0 client.15933 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:17.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.677+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/1792214093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 0x55bad5d51860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.677+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/1792214093 >> 172.21.15.84:0/1792214093 conn(0x55bad5c76750 msgr2=0x55bad5d472d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:17.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.677+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/1792214093 shutdown_connections 2024-04-26T18:32:17.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.677+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/1792214093 wait complete. 2024-04-26T18:32:17.683 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.677+0000 7ff43f1a9b80 20 client.15933 trim_cache size 0 max 16384 2024-04-26T18:32:17.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.696+0000 7ff43f1a9b80 1 Processor -- start 2024-04-26T18:32:17.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.696+0000 7ff43f1a9b80 1 -- start start 2024-04-26T18:32:17.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.696+0000 7ff43f1a9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d48900 0x55bad5d482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.696+0000 7ff43f1a9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 0x55bad5d5e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.696+0000 7ff43f1a9b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d4e850 0x55bad5d3ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.696+0000 7ff43f1a9b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301afdf0 con 0x55bad5d4e850 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.696+0000 7ff43f1a9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301b1930 con 0x55bad5d48900 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.696+0000 7ff43f1a9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301b04c0 con 0x55bad5d4df30 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.696+0000 7ff42ed9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d4e850 0x55bad5d3ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.696+0000 7ff42ed9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d4e850 0x55bad5d3ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55776/0 (socket says 172.21.15.84:55776) 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.696+0000 7ff42ed9d700 1 -- 172.21.15.84:0/402133433 learned_addr learned my addr 172.21.15.84:0/402133433 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.696+0000 7ff42dd9b700 1 --2- 172.21.15.84:0/402133433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 0x55bad5d5e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.697+0000 7ff42e59c700 1 --2- 172.21.15.84:0/402133433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d48900 0x55bad5d482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.697+0000 7ff42ed9d700 1 -- 172.21.15.84:0/402133433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 msgr2=0x55bad5d5e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.697+0000 7ff42ed9d700 1 --2- 172.21.15.84:0/402133433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 0x55bad5d5e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.697+0000 7ff42ed9d700 1 -- 172.21.15.84:0/402133433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d48900 msgr2=0x55bad5d482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:17.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.697+0000 7ff42ed9d700 1 --2- 172.21.15.84:0/402133433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d48900 0x55bad5d482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.697+0000 7ff42ed9d700 1 -- 172.21.15.84:0/402133433 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bad5d64aa0 con 0x55bad5d4e850 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.697+0000 7ff42e59c700 1 --2- 172.21.15.84:0/402133433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d48900 0x55bad5d482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.697+0000 7ff42ed9d700 1 --2- 172.21.15.84:0/402133433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d4e850 0x55bad5d3ce10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff428007850 tx=0x7ff428009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24543524178970f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.697+0000 7ff417fff700 1 -- 172.21.15.84:0/402133433 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff428023070 con 0x55bad5d4e850 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.697+0000 7ff417fff700 1 -- 172.21.15.84:0/402133433 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4280056f0 con 0x55bad5d4e850 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.697+0000 7ff417fff700 1 -- 172.21.15.84:0/402133433 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42801fb40 con 0x55bad5d4e850 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/402133433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d4e850 msgr2=0x55bad5d3ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/402133433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d4e850 0x55bad5d3ce10 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7ff428007850 tx=0x7ff428009310 comp rx=0 tx=0).stop 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/402133433 shutdown_connections 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/402133433 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d4e850 0x55bad5d3ce10 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/402133433 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d48900 0x55bad5d482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/402133433 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 0x55bad5d5e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/402133433 >> 172.21.15.84:0/402133433 conn(0x55bad5c76750 msgr2=0x55bad5d3c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/402133433 shutdown_connections 2024-04-26T18:32:17.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/402133433 wait complete. 2024-04-26T18:32:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 -- start start 2024-04-26T18:32:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 Processor -- start 2024-04-26T18:32:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d576a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 0x55bad5d57be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4e850 0x55bad5d58120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301b1260 con 0x55bad5d48900 2024-04-26T18:32:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301b0b90 con 0x55bad5d4e850 2024-04-26T18:32:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff4301add90 con 0x55bad5d4df30 2024-04-26T18:32:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff43f1a9b80 10 client.0 _ll_register_callbacks cb 0x55bad5c32ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff42e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d576a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff42e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d576a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55790/0 (socket says 172.21.15.84:55790) 2024-04-26T18:32:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.698+0000 7ff42e59c700 1 -- 172.21.15.84:0/3498004553 learned_addr learned my addr 172.21.15.84:0/3498004553 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:17.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff42dd9b700 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 0x55bad5d57be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff42ed9d700 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4e850 0x55bad5d58120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff42e59c700 1 -- 172.21.15.84:0/3498004553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 msgr2=0x55bad5d57be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff42e59c700 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 0x55bad5d57be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff42e59c700 1 -- 172.21.15.84:0/3498004553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4e850 msgr2=0x55bad5d58120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff42e59c700 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4e850 0x55bad5d58120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff42e59c700 1 -- 172.21.15.84:0/3498004553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bad5dc85f0 con 0x55bad5d48900 2024-04-26T18:32:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff42ed9d700 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4e850 0x55bad5d58120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff42e59c700 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d576a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff420000c00 tx=0x7ff420012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e4d5ed6f9e6220 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff415ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff415ffb700 1 -- 172.21.15.84:0/3498004553 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff42001ec80 con 0x55bad5d48900 2024-04-26T18:32:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff415ffb700 1 -- 172.21.15.84:0/3498004553 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7ff420027470 con 0x55bad5d48900 2024-04-26T18:32:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/3498004553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff4280056f0 con 0x55bad5d48900 2024-04-26T18:32:17.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff415ffb700 1 -- 172.21.15.84:0/3498004553 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff42001ee20 con 0x55bad5d48900 2024-04-26T18:32:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff43f1a9b80 10 client.15939 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/3498004553 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bad5dc9880 con 0x55bad5d48900 2024-04-26T18:32:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff415ffb700 1 -- 172.21.15.84:0/3498004553 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff4280056f0 con 0x55bad5d48900 2024-04-26T18:32:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff415ffb700 1 -- 172.21.15.84:0/3498004553 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff4200306a0 con 0x55bad5d48900 2024-04-26T18:32:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff43f1a9b80 10 client.15939 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.699+0000 7ff43f1a9b80 10 client.15939 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:32:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.700+0000 7ff43f1a9b80 20 client.15939 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.700+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55bad5dca030 0x55bad5dcc450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.700+0000 7ff43f1a9b80 4 client.15939 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.5, "await_for": 42.0}] 2024-04-26T18:32:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.700+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/3498004553 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.5, "await_for": 42.0}) -- 0x55bad5ca2440 con 0x55bad5dca030 2024-04-26T18:32:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.700+0000 7ff42dd9b700 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55bad5dca030 0x55bad5dcc450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.700+0000 7ff42dd9b700 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55bad5dca030 0x55bad5dcc450 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e84f2f612c261577 server_cookie=fa6ecbbf164db6e0 in_seq=0 out_seq=0 2024-04-26T18:32:17.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.700+0000 7ff415ffb700 10 client.15939 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:17.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff415ffb700 1 -- 172.21.15.84:0/3498004553 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55bad5ca2440 con 0x55bad5dca030 2024-04-26T18:32:17.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff415ffb700 10 client.15939 handle_command_reply: tid=0 2024-04-26T18:32:17.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff43f1a9b80 1 client.15939 shutdown 2024-04-26T18:32:17.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/3498004553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 msgr2=0x55bad5d576a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:17.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d576a0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7ff420000c00 tx=0x7ff420012000 comp rx=0 tx=0).stop 2024-04-26T18:32:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/3498004553 shutdown_connections 2024-04-26T18:32:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55bad5dca030 0x55bad5dcc450 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55bad5d48900 0x55bad5d576a0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55bad5d4e850 0x55bad5d58120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff43f1a9b80 1 --2- 172.21.15.84:0/3498004553 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55bad5d4df30 0x55bad5d57be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/3498004553 >> 172.21.15.84:0/3498004553 conn(0x55bad5c76750 msgr2=0x55bad5d3cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff415ffb700 0 client.15939 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/3498004553 shutdown_connections 2024-04-26T18:32:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff43f1a9b80 1 -- 172.21.15.84:0/3498004553 wait complete. 2024-04-26T18:32:17.718 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.717+0000 7ff43f1a9b80 20 client.15939 trim_cache size 0 max 16384 2024-04-26T18:32:17.718 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 10, 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "ce477bbf": { 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 10, 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 79.5, 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:32:17.719 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:32:17.720 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:32:17.720 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:32:17.720 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:32:17.720 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:17.720 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:17.720 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:17.720 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:17.720 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:17.720 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:32:17.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.718+0000 7ff434917700 1 -- 172.21.15.84:0/4029279391 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff4180a3e50 msgr2=0x7ff4180a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:17.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.718+0000 7ff434917700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff4180a3e50 0x7ff4180a6310 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7ff43019cbb0 tx=0x7ff42000a380 comp rx=0 tx=0).stop 2024-04-26T18:32:17.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.718+0000 7ff434917700 1 -- 172.21.15.84:0/4029279391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4301023d0 msgr2=0x7ff4301a1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:17.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.718+0000 7ff434917700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4301023d0 0x7ff4301a1df0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff4240057c0 tx=0x7ff4240057f0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.718+0000 7ff434917700 1 -- 172.21.15.84:0/4029279391 shutdown_connections 2024-04-26T18:32:17.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.718+0000 7ff434917700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff4180a3e50 0x7ff4180a6310 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.718+0000 7ff434917700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff430109520 0x7ff43019ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.718+0000 7ff434917700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff430101a10 0x7ff4301a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.718+0000 7ff434917700 1 --2- 172.21.15.84:0/4029279391 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff4301023d0 0x7ff4301a1df0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:17.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.718+0000 7ff434917700 1 -- 172.21.15.84:0/4029279391 >> 172.21.15.84:0/4029279391 conn(0x7ff4300789c0 msgr2=0x7ff430106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:17.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.718+0000 7ff434917700 1 -- 172.21.15.84:0/4029279391 shutdown_connections 2024-04-26T18:32:17.721 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:17.718+0000 7ff434917700 1 -- 172.21.15.84:0/4029279391 wait complete. 2024-04-26T18:32:18.618 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'ce477bbf', quiesce took 6.1 seconds. Will release after: 19.5 2024-04-26T18:32:18.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:18 smithi084 ceph-mon[29141]: pgmap v133: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-26T18:32:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:18 smithi103 ceph-mon[32727]: pgmap v133: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-26T18:32:18.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:18 smithi174 ceph-mon[32705]: pgmap v133: 97 pgs: 97 active+clean; 3.9 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-26T18:32:20.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:20 smithi084 ceph-mon[29141]: pgmap v134: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-26T18:32:20.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:20 smithi103 ceph-mon[32727]: pgmap v134: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-26T18:32:20.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:20 smithi174 ceph-mon[32705]: pgmap v134: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-26T18:32:22.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:22 smithi103 ceph-mon[32727]: pgmap v135: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-26T18:32:22.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:22 smithi174 ceph-mon[32705]: pgmap v135: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-26T18:32:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.072+0000 7fef9bf14640 1 -- 172.21.15.84:0/4035089450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef94103bc0 msgr2=0x7fef94103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.072+0000 7fef9bf14640 1 --2- 172.21.15.84:0/4035089450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef94103bc0 0x7fef94103fa0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fef90009900 tx=0x7fef9002a510 comp rx=0 tx=0).stop 2024-04-26T18:32:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.072+0000 7fef9bf14640 1 -- 172.21.15.84:0/4035089450 shutdown_connections 2024-04-26T18:32:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.072+0000 7fef9bf14640 1 --2- 172.21.15.84:0/4035089450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef9410cf00 0x7fef9410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.072+0000 7fef9bf14640 1 --2- 172.21.15.84:0/4035089450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef941044e0 0x7fef9410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.072+0000 7fef9bf14640 1 --2- 172.21.15.84:0/4035089450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef94103bc0 0x7fef94103fa0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.072+0000 7fef9bf14640 1 -- 172.21.15.84:0/4035089450 >> 172.21.15.84:0/4035089450 conn(0x7fef940789b0 msgr2=0x7fef94078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.072+0000 7fef9bf14640 1 -- 172.21.15.84:0/4035089450 shutdown_connections 2024-04-26T18:32:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.072+0000 7fef9bf14640 1 -- 172.21.15.84:0/4035089450 wait complete. 2024-04-26T18:32:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.073+0000 7fef9bf14640 1 Processor -- start 2024-04-26T18:32:23.075 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.073+0000 7fef9bf14640 1 -- start start 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.073+0000 7fef9bf14640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef94103bc0 0x7fef94199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.073+0000 7fef9bf14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef941044e0 0x7fef94199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.073+0000 7fef9bf14640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef9410cf00 0x7fef9419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.073+0000 7fef9bf14640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fef94112bb0 con 0x7fef941044e0 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.073+0000 7fef9bf14640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef94112a30 con 0x7fef9410cf00 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.073+0000 7fef9bf14640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef94112d30 con 0x7fef94103bc0 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.073+0000 7fef994af640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef941044e0 0x7fef94199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.073+0000 7fef994af640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef941044e0 0x7fef94199590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55818/0 (socket says 172.21.15.84:55818) 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.073+0000 7fef994af640 1 -- 172.21.15.84:0/4049650699 learned_addr learned my addr 172.21.15.84:0/4049650699 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef9a4b1640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef9410cf00 0x7fef9419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef99cb0640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef94103bc0 0x7fef94199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef994af640 1 -- 172.21.15.84:0/4049650699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef94103bc0 msgr2=0x7fef94199050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef994af640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef94103bc0 0x7fef94199050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef994af640 1 -- 172.21.15.84:0/4049650699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef9410cf00 msgr2=0x7fef9419d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef994af640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef9410cf00 0x7fef9419d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef994af640 1 -- 172.21.15.84:0/4049650699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9419e040 con 0x7fef941044e0 2024-04-26T18:32:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef99cb0640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef94103bc0 0x7fef94199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef994af640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef941044e0 0x7fef94199590 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fef8400c920 tx=0x7fef8400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef8affd640 1 -- 172.21.15.84:0/4049650699 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84007c70 con 0x7fef941044e0 2024-04-26T18:32:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef9bf14640 1 -- 172.21.15.84:0/4049650699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9419e330 con 0x7fef941044e0 2024-04-26T18:32:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef8affd640 1 -- 172.21.15.84:0/4049650699 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef84007e10 con 0x7fef941044e0 2024-04-26T18:32:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef8affd640 1 -- 172.21.15.84:0/4049650699 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef84018820 con 0x7fef941044e0 2024-04-26T18:32:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.074+0000 7fef9bf14640 1 -- 172.21.15.84:0/4049650699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef941a5bc0 con 0x7fef941044e0 2024-04-26T18:32:23.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.076+0000 7fef8affd640 1 -- 172.21.15.84:0/4049650699 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fef8400f450 con 0x7fef941044e0 2024-04-26T18:32:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.077+0000 7fef9ec2f740 1 Processor -- start 2024-04-26T18:32:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.077+0000 7fef9ec2f740 1 -- start start 2024-04-26T18:32:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.077+0000 7fef9ec2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.077+0000 7fef9ec2f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7b0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.077+0000 7fef9ec2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 0x56285e7b34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.077+0000 7fef9ec2f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56285e70c460 con 0x56285e7a6820 2024-04-26T18:32:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.077+0000 7fef9ec2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56285e6c6590 con 0x56285e7a5f00 2024-04-26T18:32:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.077+0000 7fef9ec2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56285e708420 con 0x56285e7b10e0 2024-04-26T18:32:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.077+0000 7fef99cb0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7b0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.077+0000 7fef8affd640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fef800a3dc0 0x7fef800a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.077+0000 7fef99cb0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7b0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55830/0 (socket says 172.21.15.84:55830) 2024-04-26T18:32:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.077+0000 7fef99cb0640 1 -- 172.21.15.84:0/3194846179 learned_addr learned my addr 172.21.15.84:0/3194846179 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:23.083 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef9a4b1640 1 --2- 172.21.15.84:0/3194846179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef994af640 1 --2- 172.21.15.84:0/3194846179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 0x56285e7b34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef9a4b1640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fef800a3dc0 0x7fef800a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef8affd640 1 -- 172.21.15.84:0/4049650699 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fef840d0a40 con 0x7fef941044e0 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef99cb0640 1 -- 172.21.15.84:0/3194846179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 msgr2=0x56285e7b34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef99cb0640 1 --2- 172.21.15.84:0/3194846179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 0x56285e7b34d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef99cb0640 1 -- 172.21.15.84:0/3194846179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 msgr2=0x56285e7a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef99cb0640 1 --2- 172.21.15.84:0/3194846179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef99cb0640 1 -- 172.21.15.84:0/3194846179 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56285e6d30d0 con 0x56285e7a6820 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef994af640 1 --2- 172.21.15.84:0/3194846179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 0x56285e7b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef9a4b1640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fef800a3dc0 0x7fef800a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fef8c005c40 tx=0x7fef8c005bd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef99cb0640 1 --2- 172.21.15.84:0/3194846179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7b0ba0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fef90000c00 tx=0x7fef90032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f68f30028af8f75 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef7ffff640 1 -- 172.21.15.84:0/3194846179 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90035070 con 0x56285e7a6820 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef7ffff640 1 -- 172.21.15.84:0/3194846179 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef900316a0 con 0x56285e7a6820 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.078+0000 7fef7ffff640 1 -- 172.21.15.84:0/3194846179 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9003c6f0 con 0x56285e7a6820 2024-04-26T18:32:23.084 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.079+0000 7fef9ec2f740 1 -- 172.21.15.84:0/3194846179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 msgr2=0x56285e7b0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.079+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/3194846179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7b0ba0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fef90000c00 tx=0x7fef90032ce0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.079+0000 7fef9ec2f740 1 -- 172.21.15.84:0/3194846179 shutdown_connections 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.079+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/3194846179 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 0x56285e7b34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.079+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/3194846179 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7b0ba0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.079+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/3194846179 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.079+0000 7fef9ec2f740 1 -- 172.21.15.84:0/3194846179 >> 172.21.15.84:0/3194846179 conn(0x56285e7a2410 msgr2=0x56285e7a27f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.079+0000 7fef9ec2f740 1 -- 172.21.15.84:0/3194846179 shutdown_connections 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.079+0000 7fef9ec2f740 1 -- 172.21.15.84:0/3194846179 wait complete. 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.079+0000 7fef9ec2f740 1 -- start start 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef9ec2f740 1 Processor -- start 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef9ec2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef9ec2f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7ae580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef9ec2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 0x56285e7aeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef9ec2f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56285e7b3cb0 con 0x56285e7a6820 2024-04-26T18:32:23.085 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef9ec2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56285e693b60 con 0x56285e7a5f00 2024-04-26T18:32:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef9ec2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56285e7b3e30 con 0x56285e7b10e0 2024-04-26T18:32:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef9ec2f740 10 client.0 _ll_register_callbacks cb 0x56285e6b82b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef994af640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef994af640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7ae580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55834/0 (socket says 172.21.15.84:55834) 2024-04-26T18:32:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef994af640 1 -- 172.21.15.84:0/1613074026 learned_addr learned my addr 172.21.15.84:0/1613074026 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef9a4b1640 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 0x56285e7aeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef99cb0640 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef994af640 1 -- 172.21.15.84:0/1613074026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 msgr2=0x56285e7aeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef994af640 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 0x56285e7aeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef994af640 1 -- 172.21.15.84:0/1613074026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 msgr2=0x56285e7ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef994af640 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef994af640 1 -- 172.21.15.84:0/1613074026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56285e41e210 con 0x56285e7a6820 2024-04-26T18:32:23.086 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.080+0000 7fef99cb0640 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef994af640 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7ae580 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fef84098dd0 tx=0x7fef8406c250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4703a55d64e4942 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef7d7fa640 1 -- 172.21.15.84:0/1613074026 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8409d070 con 0x56285e7a6820 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef7d7fa640 1 -- 172.21.15.84:0/1613074026 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fef840ce3a0 con 0x56285e7a6820 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef9ec2f740 1 -- 172.21.15.84:0/1613074026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef900316a0 con 0x56285e7a6820 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef7d7fa640 1 -- 172.21.15.84:0/1613074026 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fef840ce5a0 con 0x56285e7a6820 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef9ec2f740 10 client.15951 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef9ec2f740 1 -- 172.21.15.84:0/1613074026 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56285e4ea620 con 0x56285e7a6820 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef7d7fa640 1 -- 172.21.15.84:0/1613074026 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef900316a0 con 0x56285e7a6820 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef7d7fa640 1 -- 172.21.15.84:0/1613074026 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fef840a5150 con 0x56285e7a6820 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef9ec2f740 10 client.15951 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef9ec2f740 10 client.15951 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef9ec2f740 20 client.15951 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:23.087 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56285e7c1100 0x56285e7c34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef9ec2f740 4 client.15951 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:32:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.081+0000 7fef9ec2f740 1 -- 172.21.15.84:0/1613074026 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56285e71a070 con 0x56285e7c1100 2024-04-26T18:32:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.082+0000 7fef99cb0640 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56285e7c1100 0x56285e7c34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.082+0000 7fef99cb0640 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56285e7c1100 0x56285e7c34f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=864eff19bf621f2e server_cookie=bfe08b615e3e00b1 in_seq=0 out_seq=0 2024-04-26T18:32:23.088 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.082+0000 7fef7d7fa640 10 client.15951 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef7d7fa640 1 -- 172.21.15.84:0/1613074026 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56285e71a070 con 0x56285e7c1100 2024-04-26T18:32:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef7d7fa640 10 client.15951 handle_command_reply: tid=0 2024-04-26T18:32:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef9ec2f740 1 client.15951 shutdown 2024-04-26T18:32:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef9ec2f740 1 -- 172.21.15.84:0/1613074026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 msgr2=0x56285e7ae580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7ae580 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fef84098dd0 tx=0x7fef8406c250 comp rx=0 tx=0).stop 2024-04-26T18:32:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef9ec2f740 1 -- 172.21.15.84:0/1613074026 shutdown_connections 2024-04-26T18:32:23.089 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56285e7c1100 0x56285e7c34f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 0x56285e7aeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7ae580 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef7d7fa640 0 client.15951 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/1613074026 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef9ec2f740 1 -- 172.21.15.84:0/1613074026 >> 172.21.15.84:0/1613074026 conn(0x56285e7a2410 msgr2=0x56285e7a8190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef9ec2f740 1 -- 172.21.15.84:0/1613074026 shutdown_connections 2024-04-26T18:32:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef9ec2f740 1 -- 172.21.15.84:0/1613074026 wait complete. 2024-04-26T18:32:23.090 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.083+0000 7fef9ec2f740 20 client.15951 trim_cache size 0 max 16384 2024-04-26T18:32:23.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:22 smithi084 ceph-mon[29141]: pgmap v135: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-26T18:32:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.098+0000 7fef9ec2f740 1 Processor -- start 2024-04-26T18:32:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.098+0000 7fef9ec2f740 1 -- start start 2024-04-26T18:32:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.098+0000 7fef9ec2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7bcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.098+0000 7fef9ec2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7a6820 0x56285e7a88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.098+0000 7fef9ec2f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7b10e0 0x56285e7c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.104 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.098+0000 7fef9ec2f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fef941ac340 con 0x56285e7b10e0 2024-04-26T18:32:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.098+0000 7fef9ec2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef941ade80 con 0x56285e7a5f00 2024-04-26T18:32:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.098+0000 7fef9ec2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef941aca10 con 0x56285e7a6820 2024-04-26T18:32:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef9a4b1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7b10e0 0x56285e7c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef9a4b1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7b10e0 0x56285e7c8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55836/0 (socket says 172.21.15.84:55836) 2024-04-26T18:32:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef9a4b1640 1 -- 172.21.15.84:0/3353714263 learned_addr learned my addr 172.21.15.84:0/3353714263 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef99cb0640 1 --2- 172.21.15.84:0/3353714263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7bcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef994af640 1 --2- 172.21.15.84:0/3353714263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7a6820 0x56285e7a88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef9a4b1640 1 -- 172.21.15.84:0/3353714263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7a6820 msgr2=0x56285e7a88c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef9a4b1640 1 --2- 172.21.15.84:0/3353714263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7a6820 0x56285e7a88c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef9a4b1640 1 -- 172.21.15.84:0/3353714263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 msgr2=0x56285e7bcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:23.105 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef9a4b1640 1 --2- 172.21.15.84:0/3353714263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7bcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef9a4b1640 1 -- 172.21.15.84:0/3353714263 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56285e7c3b70 con 0x56285e7b10e0 2024-04-26T18:32:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef99cb0640 1 --2- 172.21.15.84:0/3353714263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:32:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef9a4b1640 1 --2- 172.21.15.84:0/3353714263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7b10e0 0x56285e7c8e00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fef8c012910 tx=0x7fef8c0096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1571e9d37d952949 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef7f7fe640 1 -- 172.21.15.84:0/3353714263 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c02d070 con 0x56285e7b10e0 2024-04-26T18:32:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef7f7fe640 1 -- 172.21.15.84:0/3353714263 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef8c010b80 con 0x56285e7b10e0 2024-04-26T18:32:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.099+0000 7fef7f7fe640 1 -- 172.21.15.84:0/3353714263 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c01f9b0 con 0x56285e7b10e0 2024-04-26T18:32:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.100+0000 7fef9ec2f740 1 -- 172.21.15.84:0/3353714263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7b10e0 msgr2=0x56285e7c8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.100+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/3353714263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7b10e0 0x56285e7c8e00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fef8c012910 tx=0x7fef8c0096c0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.100+0000 7fef9ec2f740 1 -- 172.21.15.84:0/3353714263 shutdown_connections 2024-04-26T18:32:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.100+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/3353714263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7b10e0 0x56285e7c8e00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.106 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.100+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/3353714263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7a6820 0x56285e7a88c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.100+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/3353714263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7bcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.100+0000 7fef9ec2f740 1 -- 172.21.15.84:0/3353714263 >> 172.21.15.84:0/3353714263 conn(0x56285e7a2410 msgr2=0x56285e7ae890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.100+0000 7fef9ec2f740 1 -- 172.21.15.84:0/3353714263 shutdown_connections 2024-04-26T18:32:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.100+0000 7fef9ec2f740 1 -- 172.21.15.84:0/3353714263 wait complete. 2024-04-26T18:32:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.100+0000 7fef9ec2f740 1 -- start start 2024-04-26T18:32:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.100+0000 7fef9ec2f740 1 Processor -- start 2024-04-26T18:32:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef9ec2f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7b4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef9ec2f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef9ec2f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 0x56285e7b5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef9ec2f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fef941ad7b0 con 0x56285e7a6820 2024-04-26T18:32:23.107 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef9ec2f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fef941ad0e0 con 0x56285e7a5f00 2024-04-26T18:32:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef9ec2f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fef941aa320 con 0x56285e7b10e0 2024-04-26T18:32:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef9ec2f740 10 client.0 _ll_register_callbacks cb 0x7fef84098c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef994af640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef994af640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7b50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55850/0 (socket says 172.21.15.84:55850) 2024-04-26T18:32:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef994af640 1 -- 172.21.15.84:0/953532838 learned_addr learned my addr 172.21.15.84:0/953532838 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef9a4b1640 1 --2- 172.21.15.84:0/953532838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 0x56285e7b5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef99cb0640 1 --2- 172.21.15.84:0/953532838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7b4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef994af640 1 -- 172.21.15.84:0/953532838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 msgr2=0x56285e7b5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef994af640 1 --2- 172.21.15.84:0/953532838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 0x56285e7b5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.108 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef994af640 1 -- 172.21.15.84:0/953532838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 msgr2=0x56285e7b4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef994af640 1 --2- 172.21.15.84:0/953532838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7b4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef994af640 1 -- 172.21.15.84:0/953532838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56285e826770 con 0x56285e7a6820 2024-04-26T18:32:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef994af640 1 --2- 172.21.15.84:0/953532838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7b50d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fef8409c670 tx=0x7fef8406f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f05a1dbb975739a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef7d7fa640 1 -- 172.21.15.84:0/953532838 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8409c4b0 con 0x56285e7a6820 2024-04-26T18:32:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef7d7fa640 1 -- 172.21.15.84:0/953532838 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fef840bbb30 con 0x56285e7a6820 2024-04-26T18:32:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.101+0000 7fef9ec2f740 1 -- 172.21.15.84:0/953532838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef8c010b80 con 0x56285e7a6820 2024-04-26T18:32:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.102+0000 7fef7d7fa640 1 -- 172.21.15.84:0/953532838 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fef840ce5a0 con 0x56285e7a6820 2024-04-26T18:32:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.102+0000 7fef9ec2f740 10 client.15957 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.102+0000 7fef9ec2f740 1 -- 172.21.15.84:0/953532838 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56285e827a60 con 0x56285e7a6820 2024-04-26T18:32:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.102+0000 7fef7d7fa640 1 -- 172.21.15.84:0/953532838 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef8c010b80 con 0x56285e7a6820 2024-04-26T18:32:23.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.102+0000 7fef7d7fa640 1 -- 172.21.15.84:0/953532838 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fef840986c0 con 0x56285e7a6820 2024-04-26T18:32:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.102+0000 7fef9ec2f740 10 client.15957 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.102+0000 7fef9ec2f740 10 client.15957 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:32:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.102+0000 7fef9ec2f740 20 client.15957 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.102+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/953532838 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56285e828080 0x56285e82a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.102+0000 7fef9ec2f740 4 client.15957 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:32:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.102+0000 7fef9ec2f740 1 -- 172.21.15.84:0/953532838 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56285e713ee0 con 0x56285e828080 2024-04-26T18:32:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.102+0000 7fef99cb0640 1 --2- 172.21.15.84:0/953532838 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56285e828080 0x56285e82a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.103+0000 7fef99cb0640 1 --2- 172.21.15.84:0/953532838 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56285e828080 0x56285e82a4a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc966a057bbf4b51 server_cookie=fe53a91d53bcd20a in_seq=0 out_seq=0 2024-04-26T18:32:23.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.103+0000 7fef7d7fa640 10 client.15957 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef7d7fa640 1 -- 172.21.15.84:0/953532838 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56285e713ee0 con 0x56285e828080 2024-04-26T18:32:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef7d7fa640 10 client.15957 handle_command_reply: tid=0 2024-04-26T18:32:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef9ec2f740 1 client.15957 shutdown 2024-04-26T18:32:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef9ec2f740 1 -- 172.21.15.84:0/953532838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 msgr2=0x56285e7b50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/953532838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7b50d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fef8409c670 tx=0x7fef8406f460 comp rx=0 tx=0).stop 2024-04-26T18:32:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef9ec2f740 1 -- 172.21.15.84:0/953532838 shutdown_connections 2024-04-26T18:32:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/953532838 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56285e828080 0x56285e82a4a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/953532838 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56285e7b10e0 0x56285e7b5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/953532838 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56285e7a6820 0x56285e7b50d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.118 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef7d7fa640 0 client.15957 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef9ec2f740 1 --2- 172.21.15.84:0/953532838 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56285e7a5f00 0x56285e7b4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef9ec2f740 1 -- 172.21.15.84:0/953532838 >> 172.21.15.84:0/953532838 conn(0x56285e7a2410 msgr2=0x56285e7a0d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef9ec2f740 1 -- 172.21.15.84:0/953532838 shutdown_connections 2024-04-26T18:32:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef9ec2f740 1 -- 172.21.15.84:0/953532838 wait complete. 2024-04-26T18:32:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.117+0000 7fef9ec2f740 20 client.15957 trim_cache size 0 max 16384 2024-04-26T18:32:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:32:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:32:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:32:23.119 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:32:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.118+0000 7fef9bf14640 1 -- 172.21.15.84:0/4049650699 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fef800a3dc0 msgr2=0x7fef800a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.118+0000 7fef9bf14640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fef800a3dc0 0x7fef800a6280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fef8c005c40 tx=0x7fef8c005bd0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.118+0000 7fef9bf14640 1 -- 172.21.15.84:0/4049650699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef941044e0 msgr2=0x7fef94199590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:23.120 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.118+0000 7fef9bf14640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef941044e0 0x7fef94199590 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fef8400c920 tx=0x7fef8400cde0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.118+0000 7fef9bf14640 1 -- 172.21.15.84:0/4049650699 shutdown_connections 2024-04-26T18:32:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.118+0000 7fef9bf14640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fef800a3dc0 0x7fef800a6280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.118+0000 7fef9bf14640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fef9410cf00 0x7fef9419d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.118+0000 7fef9bf14640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fef941044e0 0x7fef94199590 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.118+0000 7fef9bf14640 1 --2- 172.21.15.84:0/4049650699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fef94103bc0 0x7fef94199050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.118+0000 7fef9bf14640 1 -- 172.21.15.84:0/4049650699 >> 172.21.15.84:0/4049650699 conn(0x7fef940789b0 msgr2=0x7fef940fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.118+0000 7fef9bf14640 1 -- 172.21.15.84:0/4049650699 shutdown_connections 2024-04-26T18:32:23.121 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:23.118+0000 7fef9bf14640 1 -- 172.21.15.84:0/4049650699 wait complete. 2024-04-26T18:32:24.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:24 smithi103 ceph-mon[32727]: pgmap v136: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:32:24.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:24 smithi174 ceph-mon[32705]: pgmap v136: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:32:25.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:24 smithi084 ceph-mon[29141]: pgmap v136: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 22 op/s 2024-04-26T18:32:26.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:26 smithi103 ceph-mon[32727]: pgmap v137: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T18:32:26.889 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:26 smithi084 ceph-mon[29141]: pgmap v137: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T18:32:26.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:26 smithi174 ceph-mon[32705]: pgmap v137: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T18:32:28.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:28 smithi103 ceph-mon[32727]: pgmap v138: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T18:32:28.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:28 smithi174 ceph-mon[32705]: pgmap v138: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T18:32:29.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:28 smithi084 ceph-mon[29141]: pgmap v138: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T18:32:30.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:30 smithi103 ceph-mon[32727]: pgmap v139: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-26T18:32:30.897 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:30 smithi084 ceph-mon[29141]: pgmap v139: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-26T18:32:30.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:30 smithi174 ceph-mon[32705]: pgmap v139: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-26T18:32:32.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:32 smithi103 ceph-mon[32727]: pgmap v140: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-04-26T18:32:32.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:32 smithi174 ceph-mon[32705]: pgmap v140: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-04-26T18:32:33.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:32 smithi084 ceph-mon[29141]: pgmap v140: 97 pgs: 97 active+clean; 4.0 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.7 KiB/s wr, 11 op/s 2024-04-26T18:32:34.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:34 smithi103 ceph-mon[32727]: pgmap v141: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-04-26T18:32:34.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:34 smithi174 ceph-mon[32705]: pgmap v141: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-04-26T18:32:35.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:34 smithi084 ceph-mon[29141]: pgmap v141: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-04-26T18:32:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.214+0000 7f2eb0485640 1 -- 172.21.15.84:0/3825958276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ea810cfb0 msgr2=0x7f2ea810f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.214+0000 7f2eb0485640 1 --2- 172.21.15.84:0/3825958276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ea810cfb0 0x7f2ea810f3e0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009ca0 tx=0x7f2ea002a950 comp rx=0 tx=0).stop 2024-04-26T18:32:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.214+0000 7f2eb0485640 1 -- 172.21.15.84:0/3825958276 shutdown_connections 2024-04-26T18:32:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.214+0000 7f2eb0485640 1 --2- 172.21.15.84:0/3825958276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ea810cfb0 0x7f2ea810f3e0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.214+0000 7f2eb0485640 1 --2- 172.21.15.84:0/3825958276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ea8104500 0x7f2ea810c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.214+0000 7f2eb0485640 1 --2- 172.21.15.84:0/3825958276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ea8103be0 0x7f2ea8103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.214+0000 7f2eb0485640 1 -- 172.21.15.84:0/3825958276 >> 172.21.15.84:0/3825958276 conn(0x7f2ea8077bb0 msgr2=0x7f2ea8077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.214+0000 7f2eb0485640 1 -- 172.21.15.84:0/3825958276 shutdown_connections 2024-04-26T18:32:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.214+0000 7f2eb0485640 1 -- 172.21.15.84:0/3825958276 wait complete. 2024-04-26T18:32:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eb0485640 1 Processor -- start 2024-04-26T18:32:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eb0485640 1 -- start start 2024-04-26T18:32:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eb0485640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ea8103be0 0x7f2ea8198f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eb0485640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ea8104500 0x7f2ea81994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eb0485640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ea810cfb0 0x7f2ea819d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eb0485640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea8112bf0 con 0x7f2ea8103be0 2024-04-26T18:32:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eb0485640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea8112a70 con 0x7f2ea8104500 2024-04-26T18:32:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eb0485640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea8112d70 con 0x7f2ea810cfb0 2024-04-26T18:32:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eae221640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ea8103be0 0x7f2ea8198f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eae221640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ea8103be0 0x7f2ea8198f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50748/0 (socket says 172.21.15.84:50748) 2024-04-26T18:32:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eae221640 1 -- 172.21.15.84:0/3762495949 learned_addr learned my addr 172.21.15.84:0/3762495949 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eaea22640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ea810cfb0 0x7f2ea819d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eada20640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ea8104500 0x7f2ea81994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eae221640 1 -- 172.21.15.84:0/3762495949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ea810cfb0 msgr2=0x7f2ea819d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eae221640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ea810cfb0 0x7f2ea819d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eae221640 1 -- 172.21.15.84:0/3762495949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ea8104500 msgr2=0x7f2ea81994d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eae221640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ea8104500 0x7f2ea81994d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eae221640 1 -- 172.21.15.84:0/3762495949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ea819df80 con 0x7f2ea8103be0 2024-04-26T18:32:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.215+0000 7f2eaea22640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ea810cfb0 0x7f2ea819d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:32:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.216+0000 7f2eae221640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ea8103be0 0x7f2ea8198f90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400d770 tx=0x7f2ea400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.216+0000 7f2e9f7fe640 1 -- 172.21.15.84:0/3762495949 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4017070 con 0x7f2ea8103be0 2024-04-26T18:32:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.216+0000 7f2eb0485640 1 -- 172.21.15.84:0/3762495949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ea819e270 con 0x7f2ea8103be0 2024-04-26T18:32:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.216+0000 7f2e9f7fe640 1 -- 172.21.15.84:0/3762495949 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ea40054d0 con 0x7f2ea8103be0 2024-04-26T18:32:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.216+0000 7f2e9f7fe640 1 -- 172.21.15.84:0/3762495949 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4004c30 con 0x7f2ea8103be0 2024-04-26T18:32:36.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.216+0000 7f2eb0485640 1 -- 172.21.15.84:0/3762495949 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ea81a5b00 con 0x7f2ea8103be0 2024-04-26T18:32:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.218+0000 7f2e9f7fe640 1 -- 172.21.15.84:0/3762495949 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f2ea4004e10 con 0x7f2ea8103be0 2024-04-26T18:32:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.218+0000 7f2eb319f740 1 Processor -- start 2024-04-26T18:32:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eb319f740 1 -- start start 2024-04-26T18:32:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eb319f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac52f00 0x55df6ac532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eb319f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac53820 0x55df6ac5dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eb319f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 0x55df6ac604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eb319f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55df6abb9460 con 0x55df6ac5e0e0 2024-04-26T18:32:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eb319f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55df6ab73590 con 0x55df6ac53820 2024-04-26T18:32:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eb319f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55df6abb5420 con 0x55df6ac52f00 2024-04-26T18:32:36.226 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2e9f7fe640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f2e940a3dc0 0x7f2e940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eada20640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 0x55df6ac604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eada20640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 0x55df6ac604d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50764/0 (socket says 172.21.15.84:50764) 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eada20640 1 -- 172.21.15.84:0/649335118 learned_addr learned my addr 172.21.15.84:0/649335118 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eaea22640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f2e940a3dc0 0x7f2e940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eaea22640 1 --2- 172.21.15.84:0/649335118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac52f00 0x55df6ac532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2e9f7fe640 1 -- 172.21.15.84:0/3762495949 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f2ea40d0390 con 0x7f2ea8103be0 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eae221640 1 --2- 172.21.15.84:0/649335118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac53820 0x55df6ac5dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eada20640 1 -- 172.21.15.84:0/649335118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac52f00 msgr2=0x55df6ac532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eada20640 1 --2- 172.21.15.84:0/649335118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac52f00 0x55df6ac532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eada20640 1 -- 172.21.15.84:0/649335118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac53820 msgr2=0x55df6ac5dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eada20640 1 --2- 172.21.15.84:0/649335118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac53820 0x55df6ac5dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eada20640 1 -- 172.21.15.84:0/649335118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6ab800d0 con 0x55df6ac5e0e0 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.219+0000 7f2eae221640 1 --2- 172.21.15.84:0/649335118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac53820 0x55df6ac5dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:32:36.227 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.220+0000 7f2eada20640 1 --2- 172.21.15.84:0/649335118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 0x55df6ac604d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800cbb0 tx=0x7f2e9800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857ff99193b5ed0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.220+0000 7f2eaea22640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f2e940a3dc0 0x7f2e940a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009a60 tx=0x7f2ea0005d80 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.220+0000 7f2e9cbfe640 1 -- 172.21.15.84:0/649335118 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98005220 con 0x55df6ac5e0e0 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.220+0000 7f2e9cbfe640 1 -- 172.21.15.84:0/649335118 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e98004c50 con 0x55df6ac5e0e0 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.220+0000 7f2e9cbfe640 1 -- 172.21.15.84:0/649335118 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98010690 con 0x55df6ac5e0e0 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 -- 172.21.15.84:0/649335118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 msgr2=0x55df6ac604d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 --2- 172.21.15.84:0/649335118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 0x55df6ac604d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f2e9800cbb0 tx=0x7f2e9800e720 comp rx=0 tx=0).stop 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 -- 172.21.15.84:0/649335118 shutdown_connections 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 --2- 172.21.15.84:0/649335118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 0x55df6ac604d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 --2- 172.21.15.84:0/649335118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac53820 0x55df6ac5dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 --2- 172.21.15.84:0/649335118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac52f00 0x55df6ac532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 -- 172.21.15.84:0/649335118 >> 172.21.15.84:0/649335118 conn(0x55df6ac4f410 msgr2=0x55df6ac4f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 -- 172.21.15.84:0/649335118 shutdown_connections 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 -- 172.21.15.84:0/649335118 wait complete. 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 -- start start 2024-04-26T18:32:36.228 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 Processor -- start 2024-04-26T18:32:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac52f00 0x55df6ac5b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac53820 0x55df6ac5b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 0x55df6ac5baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55df6ac60d10 con 0x55df6ac5e0e0 2024-04-26T18:32:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.221+0000 7f2eb319f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55df6ab40b60 con 0x55df6ac52f00 2024-04-26T18:32:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eb319f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55df6ac60e90 con 0x55df6ac53820 2024-04-26T18:32:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eb319f740 10 client.0 _ll_register_callbacks cb 0x55df6ab652b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eaea22640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 0x55df6ac5baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eaea22640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 0x55df6ac5baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50780/0 (socket says 172.21.15.84:50780) 2024-04-26T18:32:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eaea22640 1 -- 172.21.15.84:0/2245389217 learned_addr learned my addr 172.21.15.84:0/2245389217 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eada20640 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac53820 0x55df6ac5b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eae221640 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac52f00 0x55df6ac5b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.229 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eaea22640 1 -- 172.21.15.84:0/2245389217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac53820 msgr2=0x55df6ac5b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eaea22640 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac53820 0x55df6ac5b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eaea22640 1 -- 172.21.15.84:0/2245389217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac52f00 msgr2=0x55df6ac5b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eaea22640 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac52f00 0x55df6ac5b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eaea22640 1 -- 172.21.15.84:0/2245389217 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6a8cb210 con 0x55df6ac5e0e0 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eada20640 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac53820 0x55df6ac5b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eaea22640 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 0x55df6ac5baf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0003bf0 tx=0x7f2ea002a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb7a558acb26af45 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2e91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2e91ffb640 1 -- 172.21.15.84:0/2245389217 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea004e030 con 0x55df6ac5e0e0 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2e91ffb640 1 -- 172.21.15.84:0/2245389217 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f2ea0044070 con 0x55df6ac5e0e0 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eb319f740 1 -- 172.21.15.84:0/2245389217 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e98004c50 con 0x55df6ac5e0e0 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2e91ffb640 1 -- 172.21.15.84:0/2245389217 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2ea0034040 con 0x55df6ac5e0e0 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eb319f740 10 client.15969 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.222+0000 7f2eb319f740 1 -- 172.21.15.84:0/2245389217 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55df6a997620 con 0x55df6ac5e0e0 2024-04-26T18:32:36.230 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.223+0000 7f2e91ffb640 1 -- 172.21.15.84:0/2245389217 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e98004c50 con 0x55df6ac5e0e0 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.223+0000 7f2e91ffb640 1 -- 172.21.15.84:0/2245389217 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2ea0047050 con 0x55df6ac5e0e0 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.223+0000 7f2eb319f740 10 client.15969 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.223+0000 7f2eb319f740 10 client.15969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.223+0000 7f2eb319f740 20 client.15969 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.223+0000 7f2eb319f740 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55df6ac6e200 0x55df6ac70630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.223+0000 7f2eb319f740 4 client.15969 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.223+0000 7f2eb319f740 1 -- 172.21.15.84:0/2245389217 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55df6abc7070 con 0x55df6ac6e200 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.223+0000 7f2eae221640 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55df6ac6e200 0x55df6ac70630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.223+0000 7f2e91ffb640 10 client.15969 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.223+0000 7f2eae221640 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55df6ac6e200 0x55df6ac70630 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e00d11f373fa4132 server_cookie=52be948f711dbb74 in_seq=0 out_seq=0 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2e91ffb640 1 -- 172.21.15.84:0/2245389217 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55df6abc7070 con 0x55df6ac6e200 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2e91ffb640 10 client.15969 handle_command_reply: tid=0 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2eb319f740 1 client.15969 shutdown 2024-04-26T18:32:36.231 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2eb319f740 1 -- 172.21.15.84:0/2245389217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 msgr2=0x55df6ac5baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2eb319f740 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 0x55df6ac5baf0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0003bf0 tx=0x7f2ea002a950 comp rx=0 tx=0).stop 2024-04-26T18:32:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2eb319f740 1 -- 172.21.15.84:0/2245389217 shutdown_connections 2024-04-26T18:32:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2eb319f740 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55df6ac6e200 0x55df6ac70630 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2eb319f740 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac5e0e0 0x55df6ac5baf0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2eb319f740 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac53820 0x55df6ac5b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2eb319f740 1 --2- 172.21.15.84:0/2245389217 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac52f00 0x55df6ac5b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2e91ffb640 0 client.15969 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2eb319f740 1 -- 172.21.15.84:0/2245389217 >> 172.21.15.84:0/2245389217 conn(0x55df6ac4f410 msgr2=0x55df6ac55140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2eb319f740 1 -- 172.21.15.84:0/2245389217 shutdown_connections 2024-04-26T18:32:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2eb319f740 1 -- 172.21.15.84:0/2245389217 wait complete. 2024-04-26T18:32:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.224+0000 7f2eb319f740 20 client.15969 trim_cache size 0 max 16384 2024-04-26T18:32:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.239+0000 7f2eb319f740 1 Processor -- start 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.239+0000 7f2eb319f740 1 -- start start 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.239+0000 7f2eb319f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac52f00 0x55df6ac69bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.239+0000 7f2eb319f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac53820 0x55df6ac558c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.239+0000 7f2eb319f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac5e0e0 0x55df6ac5cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.239+0000 7f2eb319f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81ae570 con 0x55df6ac53820 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.239+0000 7f2eb319f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81b00b0 con 0x55df6ac52f00 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.239+0000 7f2eb319f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81aec40 con 0x55df6ac5e0e0 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.240+0000 7f2eada20640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac53820 0x55df6ac558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.240+0000 7f2eada20640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac53820 0x55df6ac558c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50784/0 (socket says 172.21.15.84:50784) 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.240+0000 7f2eada20640 1 -- 172.21.15.84:0/447718467 learned_addr learned my addr 172.21.15.84:0/447718467 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.240+0000 7f2eaea22640 1 --2- 172.21.15.84:0/447718467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac5e0e0 0x55df6ac5cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.240+0000 7f2eada20640 1 -- 172.21.15.84:0/447718467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac5e0e0 msgr2=0x55df6ac5cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.240+0000 7f2eada20640 1 --2- 172.21.15.84:0/447718467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac5e0e0 0x55df6ac5cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.240+0000 7f2eada20640 1 -- 172.21.15.84:0/447718467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac52f00 msgr2=0x55df6ac69bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.240+0000 7f2eada20640 1 --2- 172.21.15.84:0/447718467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac52f00 0x55df6ac69bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.240+0000 7f2eada20640 1 -- 172.21.15.84:0/447718467 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df6ac595c0 con 0x55df6ac53820 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.240+0000 7f2eada20640 1 --2- 172.21.15.84:0/447718467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac53820 0x55df6ac558c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000c00 tx=0x7f2e98005870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93d03286444021e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.240+0000 7f2e93fff640 1 -- 172.21.15.84:0/447718467 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98028070 con 0x55df6ac53820 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.240+0000 7f2e93fff640 1 -- 172.21.15.84:0/447718467 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e98005560 con 0x55df6ac53820 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.240+0000 7f2e93fff640 1 -- 172.21.15.84:0/447718467 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e98010430 con 0x55df6ac53820 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 -- 172.21.15.84:0/447718467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac53820 msgr2=0x55df6ac558c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 --2- 172.21.15.84:0/447718467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac53820 0x55df6ac558c0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f2e98000c00 tx=0x7f2e98005870 comp rx=0 tx=0).stop 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 -- 172.21.15.84:0/447718467 shutdown_connections 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 --2- 172.21.15.84:0/447718467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac5e0e0 0x55df6ac5cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 --2- 172.21.15.84:0/447718467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac53820 0x55df6ac558c0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 --2- 172.21.15.84:0/447718467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac52f00 0x55df6ac69bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 -- 172.21.15.84:0/447718467 >> 172.21.15.84:0/447718467 conn(0x55df6ac4f410 msgr2=0x55df6ac5c8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 -- 172.21.15.84:0/447718467 shutdown_connections 2024-04-26T18:32:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 -- 172.21.15.84:0/447718467 wait complete. 2024-04-26T18:32:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 -- start start 2024-04-26T18:32:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 Processor -- start 2024-04-26T18:32:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac52f00 0x55df6ac62130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac53820 0x55df6ac62670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac5e0e0 0x55df6acd37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81af9e0 con 0x55df6ac52f00 2024-04-26T18:32:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81af310 con 0x55df6ac5e0e0 2024-04-26T18:32:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f2ea81ac550 con 0x55df6ac53820 2024-04-26T18:32:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eb319f740 10 client.0 _ll_register_callbacks cb 0x7f2ea0009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eae221640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac52f00 0x55df6ac62130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eae221640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac52f00 0x55df6ac62130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50798/0 (socket says 172.21.15.84:50798) 2024-04-26T18:32:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.241+0000 7f2eae221640 1 -- 172.21.15.84:0/1620482644 learned_addr learned my addr 172.21.15.84:0/1620482644 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2eaea22640 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac5e0e0 0x55df6acd37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2eada20640 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac53820 0x55df6ac62670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2eae221640 1 -- 172.21.15.84:0/1620482644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac53820 msgr2=0x55df6ac62670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2eae221640 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac53820 0x55df6ac62670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2eae221640 1 -- 172.21.15.84:0/1620482644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac5e0e0 msgr2=0x55df6acd37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2eae221640 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac5e0e0 0x55df6acd37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2eae221640 1 -- 172.21.15.84:0/1620482644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df6acd3cf0 con 0x55df6ac52f00 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2eaea22640 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac5e0e0 0x55df6acd37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2eada20640 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac53820 0x55df6ac62670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2eae221640 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac52f00 0x55df6ac62130 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400d740 tx=0x7f2ea409c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366582a22b31a592 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2e91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2e91ffb640 1 -- 172.21.15.84:0/1620482644 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ea4020040 con 0x55df6ac52f00 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2e91ffb640 1 -- 172.21.15.84:0/1620482644 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f2ea409a620 con 0x55df6ac52f00 2024-04-26T18:32:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2eb319f740 1 -- 172.21.15.84:0/1620482644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2e98005560 con 0x55df6ac52f00 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2e91ffb640 1 -- 172.21.15.84:0/1620482644 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2ea40a4f90 con 0x55df6ac52f00 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2eb319f740 10 client.15975 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2eb319f740 1 -- 172.21.15.84:0/1620482644 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55df6acd4f60 con 0x55df6ac52f00 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2e91ffb640 1 -- 172.21.15.84:0/1620482644 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e98005560 con 0x55df6ac52f00 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.242+0000 7f2e91ffb640 1 -- 172.21.15.84:0/1620482644 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2ea40a0150 con 0x55df6ac52f00 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.243+0000 7f2eb319f740 10 client.15975 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.243+0000 7f2eb319f740 10 client.15975 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.243+0000 7f2eb319f740 20 client.15975 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.243+0000 7f2eb319f740 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55df6acd5630 0x55df6acd7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.243+0000 7f2eb319f740 4 client.15975 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.243+0000 7f2eb319f740 1 -- 172.21.15.84:0/1620482644 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55df6abc0ee0 con 0x55df6acd5630 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.243+0000 7f2eada20640 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55df6acd5630 0x55df6acd7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.243+0000 7f2e91ffb640 10 client.15975 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:36.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.243+0000 7f2eada20640 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55df6acd5630 0x55df6acd7a50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=806f4aea1383b4ed server_cookie=f1e5dcf08c84b948 in_seq=0 out_seq=0 2024-04-26T18:32:36.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2e91ffb640 1 -- 172.21.15.84:0/1620482644 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55df6abc0ee0 con 0x55df6acd5630 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2e91ffb640 10 client.15975 handle_command_reply: tid=0 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2eb319f740 1 client.15975 shutdown 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2eb319f740 1 -- 172.21.15.84:0/1620482644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac52f00 msgr2=0x55df6ac62130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2eb319f740 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac52f00 0x55df6ac62130 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400d740 tx=0x7f2ea409c3e0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2eb319f740 1 -- 172.21.15.84:0/1620482644 shutdown_connections 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2eb319f740 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55df6acd5630 0x55df6acd7a50 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2eb319f740 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55df6ac5e0e0 0x55df6acd37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2eb319f740 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55df6ac53820 0x55df6ac62670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2e91ffb640 0 client.15975 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2eb319f740 1 --2- 172.21.15.84:0/1620482644 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55df6ac52f00 0x55df6ac62130 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2eb319f740 1 -- 172.21.15.84:0/1620482644 >> 172.21.15.84:0/1620482644 conn(0x55df6ac4f410 msgr2=0x55df6ac5d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2eb319f740 1 -- 172.21.15.84:0/1620482644 shutdown_connections 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2eb319f740 1 -- 172.21.15.84:0/1620482644 wait complete. 2024-04-26T18:32:36.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2eb319f740 20 client.15975 trim_cache size 0 max 16384 2024-04-26T18:32:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:32:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:32:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:32:36.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:32:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.255+0000 7f2eb0485640 1 -- 172.21.15.84:0/3762495949 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f2e940a3dc0 msgr2=0x7f2e940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.256+0000 7f2eb0485640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f2e940a3dc0 0x7f2e940a6280 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009a60 tx=0x7f2ea0005d80 comp rx=0 tx=0).stop 2024-04-26T18:32:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.256+0000 7f2eb0485640 1 -- 172.21.15.84:0/3762495949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ea8103be0 msgr2=0x7f2ea8198f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.256+0000 7f2eb0485640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ea8103be0 0x7f2ea8198f90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400d770 tx=0x7f2ea400dc30 comp rx=0 tx=0).stop 2024-04-26T18:32:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.256+0000 7f2eb0485640 1 -- 172.21.15.84:0/3762495949 shutdown_connections 2024-04-26T18:32:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.256+0000 7f2eb0485640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f2e940a3dc0 0x7f2e940a6280 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.256+0000 7f2eb0485640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f2ea810cfb0 0x7f2ea819d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.256+0000 7f2eb0485640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f2ea8104500 0x7f2ea81994d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.256+0000 7f2eb0485640 1 --2- 172.21.15.84:0/3762495949 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f2ea8103be0 0x7f2ea8198f90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:36.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.256+0000 7f2eb0485640 1 -- 172.21.15.84:0/3762495949 >> 172.21.15.84:0/3762495949 conn(0x7f2ea8077bb0 msgr2=0x7f2ea80fe660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:36.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.256+0000 7f2eb0485640 1 -- 172.21.15.84:0/3762495949 shutdown_connections 2024-04-26T18:32:36.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:36.256+0000 7f2eb0485640 1 -- 172.21.15.84:0/3762495949 wait complete. 2024-04-26T18:32:36.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:36 smithi103 ceph-mon[32727]: pgmap v142: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:32:36.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:36 smithi174 ceph-mon[32705]: pgmap v142: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:32:37.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:36 smithi084 ceph-mon[29141]: pgmap v142: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:32:38.119 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:32:38.371 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:32:38.769 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:38 smithi084 ceph-mon[29141]: pgmap v143: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:32:38.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:38 smithi103 ceph-mon[32727]: pgmap v143: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:32:38.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:38 smithi174 ceph-mon[32705]: pgmap v143: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:32:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.121+0000 7f07e1017700 1 -- 172.21.15.84:0/708862210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07dc102310 msgr2=0x7f07dc102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.121+0000 7f07e1017700 1 --2- 172.21.15.84:0/708862210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07dc102310 0x7f07dc102790 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f07d0009990 tx=0x7f07d0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:32:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.121+0000 7f07e1017700 1 -- 172.21.15.84:0/708862210 shutdown_connections 2024-04-26T18:32:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.121+0000 7f07e1017700 1 --2- 172.21.15.84:0/708862210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07dc102cd0 0x7f07dc10f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.121+0000 7f07e1017700 1 --2- 172.21.15.84:0/708862210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07dc108380 0x7f07dc108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.121+0000 7f07e1017700 1 --2- 172.21.15.84:0/708862210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07dc102310 0x7f07dc102790 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.121+0000 7f07e1017700 1 -- 172.21.15.84:0/708862210 >> 172.21.15.84:0/708862210 conn(0x7f07dc079290 msgr2=0x7f07dc0796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.121+0000 7f07e1017700 1 -- 172.21.15.84:0/708862210 shutdown_connections 2024-04-26T18:32:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.121+0000 7f07e1017700 1 -- 172.21.15.84:0/708862210 wait complete. 2024-04-26T18:32:40.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.122+0000 7f07e1017700 1 Processor -- start 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.122+0000 7f07e1017700 1 -- start start 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.122+0000 7f07e1017700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07dc102310 0x7f07dc19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07e1017700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07dc102cd0 0x7f07dc19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07e1017700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07dc108380 0x7f07dc198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07e1017700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc1125a0 con 0x7f07dc102cd0 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07e1017700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc112420 con 0x7f07dc102310 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07e1017700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f07dc112720 con 0x7f07dc108380 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07da59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07dc102cd0 0x7f07dc19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07da59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07dc102cd0 0x7f07dc19ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36542/0 (socket says 172.21.15.84:36542) 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07da59c700 1 -- 172.21.15.84:0/3572873286 learned_addr learned my addr 172.21.15.84:0/3572873286 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07db59e700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07dc108380 0x7f07dc198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07dad9d700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07dc102310 0x7f07dc19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07da59c700 1 -- 172.21.15.84:0/3572873286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07dc108380 msgr2=0x7f07dc198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:40.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07da59c700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07dc108380 0x7f07dc198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07da59c700 1 -- 172.21.15.84:0/3572873286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07dc102310 msgr2=0x7f07dc19e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07da59c700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07dc102310 0x7f07dc19e7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07da59c700 1 -- 172.21.15.84:0/3572873286 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07dc1990c0 con 0x7f07dc102cd0 2024-04-26T18:32:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07db59e700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07dc108380 0x7f07dc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07dad9d700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07dc102310 0x7f07dc19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07da59c700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07dc102cd0 0x7f07dc19ece0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f07d000b5c0 tx=0x7f07d0004630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07c3fff700 1 -- 172.21.15.84:0/3572873286 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0026070 con 0x7f07dc102cd0 2024-04-26T18:32:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07e1017700 1 -- 172.21.15.84:0/3572873286 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07dc199350 con 0x7f07dc102cd0 2024-04-26T18:32:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07c3fff700 1 -- 172.21.15.84:0/3572873286 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07d0004170 con 0x7f07dc102cd0 2024-04-26T18:32:40.126 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07c3fff700 1 -- 172.21.15.84:0/3572873286 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07d0005250 con 0x7f07dc102cd0 2024-04-26T18:32:40.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.123+0000 7f07e1017700 1 -- 172.21.15.84:0/3572873286 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07dc1a5670 con 0x7f07dc102cd0 2024-04-26T18:32:40.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.124+0000 7f07e1017700 1 -- 172.21.15.84:0/3572873286 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07dc10a8f0 con 0x7f07dc102cd0 2024-04-26T18:32:40.127 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.126+0000 7f07c3fff700 1 -- 172.21.15.84:0/3572873286 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f07d0029070 con 0x7f07dc102cd0 2024-04-26T18:32:40.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.127+0000 7f07c3fff700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f07c40a3db0 0x7f07c40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:40.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.127+0000 7f07dad9d700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f07c40a3db0 0x7f07c40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:40.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.127+0000 7f07c3fff700 1 -- 172.21.15.84:0/3572873286 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f07d00d14d0 con 0x7f07dc102cd0 2024-04-26T18:32:40.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.127+0000 7f07dad9d700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f07c40a3db0 0x7f07c40a6270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f07dc199aa0 tx=0x7f07cc008040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:40.130 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.130+0000 7f07c3fff700 1 -- 172.21.15.84:0/3572873286 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07d0013070 con 0x7f07dc102cd0 2024-04-26T18:32:40.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.383+0000 7f07e1017700 1 -- 172.21.15.84:0/3572873286 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f07dc199e90 con 0x7f07dc102cd0 2024-04-26T18:32:40.384 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.384+0000 7f07c3fff700 1 -- 172.21.15.84:0/3572873286 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f07d009dd10 con 0x7f07dc102cd0 2024-04-26T18:32:40.384 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:32:40.385 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:32:40.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.386+0000 7f07e1017700 1 -- 172.21.15.84:0/3572873286 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f07c40a3db0 msgr2=0x7f07c40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:40.386 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.386+0000 7f07e1017700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f07c40a3db0 0x7f07c40a6270 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f07dc199aa0 tx=0x7f07cc008040 comp rx=0 tx=0).stop 2024-04-26T18:32:40.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.386+0000 7f07e1017700 1 -- 172.21.15.84:0/3572873286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07dc102cd0 msgr2=0x7f07dc19ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:40.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.386+0000 7f07e1017700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07dc102cd0 0x7f07dc19ece0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f07d000b5c0 tx=0x7f07d0004630 comp rx=0 tx=0).stop 2024-04-26T18:32:40.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.386+0000 7f07e1017700 1 -- 172.21.15.84:0/3572873286 shutdown_connections 2024-04-26T18:32:40.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.386+0000 7f07e1017700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f07c40a3db0 0x7f07c40a6270 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:40.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.386+0000 7f07e1017700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f07dc108380 0x7f07dc198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:40.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.386+0000 7f07e1017700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f07dc102310 0x7f07dc19e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:40.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.386+0000 7f07e1017700 1 --2- 172.21.15.84:0/3572873286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f07dc102cd0 0x7f07dc19ece0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:40.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.386+0000 7f07e1017700 1 -- 172.21.15.84:0/3572873286 >> 172.21.15.84:0/3572873286 conn(0x7f07dc079290 msgr2=0x7f07dc101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:40.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.386+0000 7f07e1017700 1 -- 172.21.15.84:0/3572873286 shutdown_connections 2024-04-26T18:32:40.387 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:40.386+0000 7f07e1017700 1 -- 172.21.15.84:0/3572873286 wait complete. 2024-04-26T18:32:40.388 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:32:40.716 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:40 smithi084 ceph-mon[29141]: pgmap v144: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.5 KiB/s wr, 14 op/s 2024-04-26T18:32:40.716 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:40 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3572873286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:32:40.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:40 smithi103 ceph-mon[32727]: pgmap v144: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.5 KiB/s wr, 14 op/s 2024-04-26T18:32:40.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:40 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3572873286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:32:40.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:40 smithi174 ceph-mon[32705]: pgmap v144: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.5 KiB/s wr, 14 op/s 2024-04-26T18:32:40.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:40 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3572873286' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:32:41.208 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id ce477bbf --release --await' 2024-04-26T18:32:41.208 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db --set-id ce477bbf --release --await 2024-04-26T18:32:41.449 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:32:42.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:42 smithi103 ceph-mon[32727]: pgmap v145: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-04-26T18:32:42.920 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:42 smithi084 ceph-mon[29141]: pgmap v145: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-04-26T18:32:42.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:42 smithi174 ceph-mon[32705]: pgmap v145: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.5 KiB/s wr, 10 op/s 2024-04-26T18:32:43.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.187+0000 7f693a43c700 1 -- 172.21.15.84:0/2199774774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6934102610 msgr2=0x7f69341029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:43.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.187+0000 7f693a43c700 1 --2- 172.21.15.84:0/2199774774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6934102610 0x7f69341029f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f69240099c0 tx=0x7f6924009cd0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.187+0000 7f693a43c700 1 -- 172.21.15.84:0/2199774774 shutdown_connections 2024-04-26T18:32:43.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.187+0000 7f693a43c700 1 --2- 172.21.15.84:0/2199774774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6934102610 0x7f69341029f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.187+0000 7f693a43c700 1 --2- 172.21.15.84:0/2199774774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6934107660 0x7f693410f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.187+0000 7f693a43c700 1 --2- 172.21.15.84:0/2199774774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6934102fc0 0x7f6934106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.190 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.187+0000 7f693a43c700 1 -- 172.21.15.84:0/2199774774 >> 172.21.15.84:0/2199774774 conn(0x7f6934078370 msgr2=0x7f6934078780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:43.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.187+0000 7f693a43c700 1 -- 172.21.15.84:0/2199774774 shutdown_connections 2024-04-26T18:32:43.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.187+0000 7f693a43c700 1 -- 172.21.15.84:0/2199774774 wait complete. 2024-04-26T18:32:43.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.188+0000 7f693a43c700 1 Processor -- start 2024-04-26T18:32:43.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.188+0000 7f693a43c700 1 -- start start 2024-04-26T18:32:43.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.188+0000 7f693a43c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6934102610 0x7f693419a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.188+0000 7f693a43c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6934102fc0 0x7f693419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.188+0000 7f693a43c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6934107660 0x7f69341a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.188+0000 7f693a43c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69341124f0 con 0x7f6934102610 2024-04-26T18:32:43.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.188+0000 7f693a43c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6934112370 con 0x7f6934107660 2024-04-26T18:32:43.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.188+0000 7f693a43c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6934112670 con 0x7f6934102fc0 2024-04-26T18:32:43.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f6933fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6934102610 0x7f693419a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.191 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f6933fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6934102610 0x7f693419a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36556/0 (socket says 172.21.15.84:36556) 2024-04-26T18:32:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f6933fff700 1 -- 172.21.15.84:0/2810461640 learned_addr learned my addr 172.21.15.84:0/2810461640 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f69337fe700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6934102fc0 0x7f693419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f69389d9700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6934107660 0x7f69341a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f6933fff700 1 -- 172.21.15.84:0/2810461640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6934102fc0 msgr2=0x7f693419aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f6933fff700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6934102fc0 0x7f693419aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f6933fff700 1 -- 172.21.15.84:0/2810461640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6934107660 msgr2=0x7f69341a5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f6933fff700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6934107660 0x7f69341a5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f6933fff700 1 -- 172.21.15.84:0/2810461640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69341a5f50 con 0x7f6934102610 2024-04-26T18:32:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f69389d9700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6934107660 0x7f69341a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f69337fe700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6934102fc0 0x7f693419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f6933fff700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6934102610 0x7f693419a990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f69240098e0 tx=0x7f6924005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f69317fa700 1 -- 172.21.15.84:0/2810461640 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924027070 con 0x7f6934102610 2024-04-26T18:32:43.192 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f69317fa700 1 -- 172.21.15.84:0/2810461640 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6924003fa0 con 0x7f6934102610 2024-04-26T18:32:43.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f693a43c700 1 -- 172.21.15.84:0/2810461640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69341a6120 con 0x7f6934102610 2024-04-26T18:32:43.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f69317fa700 1 -- 172.21.15.84:0/2810461640 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6924022d40 con 0x7f6934102610 2024-04-26T18:32:43.193 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.189+0000 7f693a43c700 1 -- 172.21.15.84:0/2810461640 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69341a65d0 con 0x7f6934102610 2024-04-26T18:32:43.201 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.191+0000 7f69317fa700 1 -- 172.21.15.84:0/2810461640 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f692402a070 con 0x7f6934102610 2024-04-26T18:32:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.192+0000 7f69317fa700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f691c0a3e30 0x7f691c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.192+0000 7f69337fe700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f691c0a3e30 0x7f691c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.192+0000 7f69317fa700 1 -- 172.21.15.84:0/2810461640 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f69240d1550 con 0x7f6934102610 2024-04-26T18:32:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.193+0000 7f69337fe700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f691c0a3e30 0x7f691c0a62f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6928005fd0 tx=0x7f6928005ee0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.193+0000 7f6944cceb80 1 Processor -- start 2024-04-26T18:32:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.193+0000 7f6944cceb80 1 -- start start 2024-04-26T18:32:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.193+0000 7f6944cceb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b331580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.193+0000 7f6944cceb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b331ac0 0x56534b340840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.193+0000 7f6944cceb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b340d80 0x56534b343170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.193+0000 7f6944cceb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56534b284840 con 0x56534b3311a0 2024-04-26T18:32:43.202 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.193+0000 7f6944cceb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56534b2236e0 con 0x56534b340d80 2024-04-26T18:32:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.193+0000 7f6944cceb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56534b280800 con 0x56534b331ac0 2024-04-26T18:32:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.193+0000 7f6933fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b331580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.193+0000 7f6933fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b331580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36564/0 (socket says 172.21.15.84:36564) 2024-04-26T18:32:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.193+0000 7f6933fff700 1 -- 172.21.15.84:0/1203461276 learned_addr learned my addr 172.21.15.84:0/1203461276 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.194+0000 7f69389d9700 1 --2- 172.21.15.84:0/1203461276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b340d80 0x56534b343170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.194+0000 7f6933fff700 1 -- 172.21.15.84:0/1203461276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b331ac0 msgr2=0x56534b340840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.194+0000 7f69337fe700 1 --2- 172.21.15.84:0/1203461276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b331ac0 0x56534b340840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.194+0000 7f6933fff700 1 --2- 172.21.15.84:0/1203461276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b331ac0 0x56534b340840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.194+0000 7f6933fff700 1 -- 172.21.15.84:0/1203461276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b340d80 msgr2=0x56534b343170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.194+0000 7f6933fff700 1 --2- 172.21.15.84:0/1203461276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b340d80 0x56534b343170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.194+0000 7f6933fff700 1 -- 172.21.15.84:0/1203461276 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534b1d19d0 con 0x56534b3311a0 2024-04-26T18:32:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.194+0000 7f69389d9700 1 --2- 172.21.15.84:0/1203461276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b340d80 0x56534b343170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:43.203 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.194+0000 7f6933fff700 1 --2- 172.21.15.84:0/1203461276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b331580 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f692400c010 tx=0x7f692409d0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e8c606baa04642 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.194+0000 7f691a7fc700 1 -- 172.21.15.84:0/1203461276 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692409d570 con 0x56534b3311a0 2024-04-26T18:32:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.194+0000 7f691a7fc700 1 -- 172.21.15.84:0/1203461276 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69240a1920 con 0x56534b3311a0 2024-04-26T18:32:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.194+0000 7f691a7fc700 1 -- 172.21.15.84:0/1203461276 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692409da30 con 0x56534b3311a0 2024-04-26T18:32:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.195+0000 7f6944cceb80 1 -- 172.21.15.84:0/1203461276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 msgr2=0x56534b331580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.195+0000 7f6944cceb80 1 --2- 172.21.15.84:0/1203461276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b331580 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f692400c010 tx=0x7f692409d0b0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.195+0000 7f6944cceb80 1 -- 172.21.15.84:0/1203461276 shutdown_connections 2024-04-26T18:32:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.195+0000 7f6944cceb80 1 --2- 172.21.15.84:0/1203461276 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b331580 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.195+0000 7f6944cceb80 1 --2- 172.21.15.84:0/1203461276 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b340d80 0x56534b343170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.195+0000 7f6944cceb80 1 --2- 172.21.15.84:0/1203461276 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b331ac0 0x56534b340840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.195+0000 7f6944cceb80 1 -- 172.21.15.84:0/1203461276 >> 172.21.15.84:0/1203461276 conn(0x56534b25ee40 msgr2=0x56534b32d690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.195+0000 7f6944cceb80 1 -- 172.21.15.84:0/1203461276 shutdown_connections 2024-04-26T18:32:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.195+0000 7f6944cceb80 1 -- 172.21.15.84:0/1203461276 wait complete. 2024-04-26T18:32:43.204 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.195+0000 7f6944cceb80 1 -- start start 2024-04-26T18:32:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.195+0000 7f6944cceb80 1 Processor -- start 2024-04-26T18:32:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6944cceb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b3405a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6944cceb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b331ac0 0x56534b33d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6944cceb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 0x56534b33dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6944cceb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56534b343710 con 0x56534b3311a0 2024-04-26T18:32:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6944cceb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56534b1eaa50 con 0x56534b331ac0 2024-04-26T18:32:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6944cceb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56534b343890 con 0x56534b340d80 2024-04-26T18:32:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6944cceb80 10 client.0 _ll_register_callbacks cb 0x56534b218680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6933fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b3405a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6933fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b3405a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36574/0 (socket says 172.21.15.84:36574) 2024-04-26T18:32:43.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6933fff700 1 -- 172.21.15.84:0/3581130589 learned_addr learned my addr 172.21.15.84:0/3581130589 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:43.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f69337fe700 1 --2- 172.21.15.84:0/3581130589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b331ac0 0x56534b33d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f69389d9700 1 --2- 172.21.15.84:0/3581130589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 0x56534b33dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6933fff700 1 -- 172.21.15.84:0/3581130589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 msgr2=0x56534b33dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:43.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6933fff700 1 --2- 172.21.15.84:0/3581130589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 0x56534b33dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6933fff700 1 -- 172.21.15.84:0/3581130589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b331ac0 msgr2=0x56534b33d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:43.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6933fff700 1 --2- 172.21.15.84:0/3581130589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b331ac0 0x56534b33d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6933fff700 1 -- 172.21.15.84:0/3581130589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534b33e530 con 0x56534b3311a0 2024-04-26T18:32:43.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6933fff700 1 --2- 172.21.15.84:0/3581130589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b3405a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6924099f40 tx=0x7f692409cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede4f841ef3d4f3c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:43.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f69137fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:43.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f69137fe700 1 -- 172.21.15.84:0/3581130589 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69240bb060 con 0x56534b3311a0 2024-04-26T18:32:43.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f69137fe700 1 -- 172.21.15.84:0/3581130589 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f692409d9e0 con 0x56534b3311a0 2024-04-26T18:32:43.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.196+0000 7f6944cceb80 1 -- 172.21.15.84:0/3581130589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69240a1920 con 0x56534b3311a0 2024-04-26T18:32:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.197+0000 7f69137fe700 1 -- 172.21.15.84:0/3581130589 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69240bc250 con 0x56534b3311a0 2024-04-26T18:32:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.197+0000 7f6944cceb80 10 client.15993 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.197+0000 7f6944cceb80 1 -- 172.21.15.84:0/3581130589 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56534b343710 con 0x56534b3311a0 2024-04-26T18:32:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.197+0000 7f69137fe700 1 -- 172.21.15.84:0/3581130589 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69240a1920 con 0x56534b3311a0 2024-04-26T18:32:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.197+0000 7f69137fe700 1 -- 172.21.15.84:0/3581130589 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f692409ca50 con 0x56534b3311a0 2024-04-26T18:32:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.197+0000 7f6944cceb80 10 client.15993 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.197+0000 7f6944cceb80 10 client.15993 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:32:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.197+0000 7f6944cceb80 20 client.15993 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.197+0000 7f6944cceb80 1 --2- 172.21.15.84:0/3581130589 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56534b3503e0 0x56534b3527d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.197+0000 7f6944cceb80 4 client.15993 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:32:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.197+0000 7f6944cceb80 1 -- 172.21.15.84:0/3581130589 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56534b292220 con 0x56534b3503e0 2024-04-26T18:32:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.197+0000 7f69337fe700 1 --2- 172.21.15.84:0/3581130589 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56534b3503e0 0x56534b3527d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.198+0000 7f69137fe700 10 client.15993 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.198+0000 7f69337fe700 1 --2- 172.21.15.84:0/3581130589 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56534b3503e0 0x56534b3527d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b5e3fe1242389c server_cookie=26ef17879a27cee7 in_seq=0 out_seq=0 2024-04-26T18:32:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.198+0000 7f69137fe700 1 -- 172.21.15.84:0/3581130589 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56534b292220 con 0x56534b3503e0 2024-04-26T18:32:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.198+0000 7f69137fe700 10 client.15993 handle_command_reply: tid=0 2024-04-26T18:32:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.198+0000 7f6944cceb80 1 client.15993 shutdown 2024-04-26T18:32:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.198+0000 7f6944cceb80 1 -- 172.21.15.84:0/3581130589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 msgr2=0x56534b3405a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.198+0000 7f6944cceb80 1 --2- 172.21.15.84:0/3581130589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b3405a0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6924099f40 tx=0x7f692409cfd0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.199+0000 7f6944cceb80 1 -- 172.21.15.84:0/3581130589 shutdown_connections 2024-04-26T18:32:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.199+0000 7f6944cceb80 1 --2- 172.21.15.84:0/3581130589 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56534b3503e0 0x56534b3527d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.199+0000 7f69137fe700 0 client.15993 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:43.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.199+0000 7f6944cceb80 1 --2- 172.21.15.84:0/3581130589 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b3405a0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.199+0000 7f6944cceb80 1 --2- 172.21.15.84:0/3581130589 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 0x56534b33dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.199+0000 7f6944cceb80 1 --2- 172.21.15.84:0/3581130589 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b331ac0 0x56534b33d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.199+0000 7f6944cceb80 1 -- 172.21.15.84:0/3581130589 >> 172.21.15.84:0/3581130589 conn(0x56534b25ee40 msgr2=0x56534b32d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.199+0000 7f6944cceb80 1 -- 172.21.15.84:0/3581130589 shutdown_connections 2024-04-26T18:32:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.199+0000 7f6944cceb80 1 -- 172.21.15.84:0/3581130589 wait complete. 2024-04-26T18:32:43.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.199+0000 7f6944cceb80 20 client.15993 trim_cache size 0 max 16384 2024-04-26T18:32:43.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.218+0000 7f6944cceb80 1 Processor -- start 2024-04-26T18:32:43.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.218+0000 7f6944cceb80 1 -- start start 2024-04-26T18:32:43.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.218+0000 7f6944cceb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b3311a0 0x56534b338e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.218+0000 7f6944cceb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b331ac0 0x56534b34c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.218+0000 7f6944cceb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 0x56534b3580a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.218+0000 7f6944cceb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69341abb90 con 0x56534b331ac0 2024-04-26T18:32:43.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.218+0000 7f6944cceb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69341ad6d0 con 0x56534b3311a0 2024-04-26T18:32:43.223 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.218+0000 7f6944cceb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69341ac260 con 0x56534b340d80 2024-04-26T18:32:43.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.219+0000 7f69337fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b331ac0 0x56534b34c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.219+0000 7f69337fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b331ac0 0x56534b34c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36586/0 (socket says 172.21.15.84:36586) 2024-04-26T18:32:43.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.219+0000 7f69337fe700 1 -- 172.21.15.84:0/980477149 learned_addr learned my addr 172.21.15.84:0/980477149 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:43.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.219+0000 7f69337fe700 1 -- 172.21.15.84:0/980477149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 msgr2=0x56534b3580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:43.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.219+0000 7f69337fe700 1 --2- 172.21.15.84:0/980477149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 0x56534b3580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.219+0000 7f6933fff700 1 --2- 172.21.15.84:0/980477149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b3311a0 0x56534b338e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.219+0000 7f69337fe700 1 -- 172.21.15.84:0/980477149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b3311a0 msgr2=0x56534b338e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:43.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.219+0000 7f69337fe700 1 --2- 172.21.15.84:0/980477149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b3311a0 0x56534b338e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.219+0000 7f69337fe700 1 -- 172.21.15.84:0/980477149 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534b324f40 con 0x56534b331ac0 2024-04-26T18:32:43.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.219+0000 7f69337fe700 1 --2- 172.21.15.84:0/980477149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b331ac0 0x56534b34c520 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6934100f50 tx=0x7f6928009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ac29b1491c44d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:43.224 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.219+0000 7f6919ffb700 1 -- 172.21.15.84:0/980477149 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928011c60 con 0x56534b331ac0 2024-04-26T18:32:43.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.219+0000 7f6919ffb700 1 -- 172.21.15.84:0/980477149 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6928017040 con 0x56534b331ac0 2024-04-26T18:32:43.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.219+0000 7f6919ffb700 1 -- 172.21.15.84:0/980477149 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6928004a90 con 0x56534b331ac0 2024-04-26T18:32:43.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.220+0000 7f6944cceb80 1 -- 172.21.15.84:0/980477149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b331ac0 msgr2=0x56534b34c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:43.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.220+0000 7f6944cceb80 1 --2- 172.21.15.84:0/980477149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b331ac0 0x56534b34c520 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f6934100f50 tx=0x7f6928009f90 comp rx=0 tx=0).stop 2024-04-26T18:32:43.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.220+0000 7f6944cceb80 1 -- 172.21.15.84:0/980477149 shutdown_connections 2024-04-26T18:32:43.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.220+0000 7f6944cceb80 1 --2- 172.21.15.84:0/980477149 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 0x56534b3580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.220+0000 7f6944cceb80 1 --2- 172.21.15.84:0/980477149 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b3311a0 0x56534b338e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.220+0000 7f6944cceb80 1 --2- 172.21.15.84:0/980477149 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b331ac0 0x56534b34c520 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.220+0000 7f6944cceb80 1 -- 172.21.15.84:0/980477149 >> 172.21.15.84:0/980477149 conn(0x56534b25ee40 msgr2=0x56534b33f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:43.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.220+0000 7f6944cceb80 1 -- 172.21.15.84:0/980477149 shutdown_connections 2024-04-26T18:32:43.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.220+0000 7f6944cceb80 1 -- 172.21.15.84:0/980477149 wait complete. 2024-04-26T18:32:43.225 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.220+0000 7f6944cceb80 1 -- start start 2024-04-26T18:32:43.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.220+0000 7f6944cceb80 1 Processor -- start 2024-04-26T18:32:43.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6944cceb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b3459b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6944cceb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b331ac0 0x56534b345ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6944cceb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 0x56534b3b6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6944cceb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69341ad000 con 0x56534b3311a0 2024-04-26T18:32:43.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6944cceb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69341ac930 con 0x56534b331ac0 2024-04-26T18:32:43.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6944cceb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69341a9b30 con 0x56534b340d80 2024-04-26T18:32:43.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6944cceb80 10 client.0 _ll_register_callbacks cb 0x56534b218680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:43.226 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6933fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b3459b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6933fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b3459b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36596/0 (socket says 172.21.15.84:36596) 2024-04-26T18:32:43.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6933fff700 1 -- 172.21.15.84:0/2737106860 learned_addr learned my addr 172.21.15.84:0/2737106860 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:43.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f69389d9700 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 0x56534b3b6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f69337fe700 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b331ac0 0x56534b345ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6933fff700 1 -- 172.21.15.84:0/2737106860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 msgr2=0x56534b3b6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:43.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6933fff700 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 0x56534b3b6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6933fff700 1 -- 172.21.15.84:0/2737106860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b331ac0 msgr2=0x56534b345ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:43.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6933fff700 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b331ac0 0x56534b345ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6933fff700 1 -- 172.21.15.84:0/2737106860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534b3b6e60 con 0x56534b3311a0 2024-04-26T18:32:43.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f69389d9700 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 0x56534b3b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:32:43.227 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6933fff700 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b3459b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6924005a80 tx=0x7f6924022810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145ddf77ec7d573b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f69137fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f69137fe700 1 -- 172.21.15.84:0/2737106860 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69240bb050 con 0x56534b3311a0 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f69137fe700 1 -- 172.21.15.84:0/2737106860 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f69240a5cc0 con 0x56534b3311a0 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f6944cceb80 1 -- 172.21.15.84:0/2737106860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6928017040 con 0x56534b3311a0 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.221+0000 7f69137fe700 1 -- 172.21.15.84:0/2737106860 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f69240a1920 con 0x56534b3311a0 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.222+0000 7f6944cceb80 10 client.15999 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.222+0000 7f6944cceb80 1 -- 172.21.15.84:0/2737106860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56534b3b6e60 con 0x56534b3311a0 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.222+0000 7f69137fe700 1 -- 172.21.15.84:0/2737106860 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6928017040 con 0x56534b3311a0 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.222+0000 7f69137fe700 1 -- 172.21.15.84:0/2737106860 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f69240992c0 con 0x56534b3311a0 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.222+0000 7f6944cceb80 10 client.15999 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.222+0000 7f6944cceb80 10 client.15999 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.222+0000 7f6944cceb80 20 client.15999 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.222+0000 7f6944cceb80 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56534b3503e0 0x56534b3ba350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:43.228 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.222+0000 7f6944cceb80 4 client.15999 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ce477bbf", "release": true, "await": true}] 2024-04-26T18:32:43.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.222+0000 7f6944cceb80 1 -- 172.21.15.84:0/2737106860 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ce477bbf", "release": true, "await": true}) -- 0x56534b28c2c0 con 0x56534b3503e0 2024-04-26T18:32:43.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.222+0000 7f69337fe700 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56534b3503e0 0x56534b3ba350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:43.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.222+0000 7f69337fe700 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56534b3503e0 0x56534b3ba350 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38ecfd413a1a7720 server_cookie=47958c1e37ef2ac in_seq=0 out_seq=0 2024-04-26T18:32:43.229 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.222+0000 7f69137fe700 10 client.15999 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:43.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f69137fe700 1 -- 172.21.15.84:0/2737106860 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56534b28c2c0 con 0x56534b3503e0 2024-04-26T18:32:43.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f69137fe700 10 client.15999 handle_command_reply: tid=0 2024-04-26T18:32:43.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f6944cceb80 1 client.15999 shutdown 2024-04-26T18:32:43.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f6944cceb80 1 -- 172.21.15.84:0/2737106860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 msgr2=0x56534b3459b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:43.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f6944cceb80 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b3459b0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6924005a80 tx=0x7f6924022810 comp rx=0 tx=0).stop 2024-04-26T18:32:43.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f6944cceb80 1 -- 172.21.15.84:0/2737106860 shutdown_connections 2024-04-26T18:32:43.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f6944cceb80 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56534b3503e0 0x56534b3ba350 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f6944cceb80 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56534b3311a0 0x56534b3459b0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f6944cceb80 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56534b340d80 0x56534b3b6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f69137fe700 0 client.15999 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:43.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f6944cceb80 1 --2- 172.21.15.84:0/2737106860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56534b331ac0 0x56534b345ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f6944cceb80 1 -- 172.21.15.84:0/2737106860 >> 172.21.15.84:0/2737106860 conn(0x56534b25ee40 msgr2=0x56534b333930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:43.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f6944cceb80 1 -- 172.21.15.84:0/2737106860 shutdown_connections 2024-04-26T18:32:43.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f6944cceb80 1 -- 172.21.15.84:0/2737106860 wait complete. 2024-04-26T18:32:43.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f6944cceb80 20 client.15999 trim_cache size 0 max 16384 2024-04-26T18:32:43.231 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:32:43.232 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:32:43.232 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:32:43.232 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 12, 2024-04-26T18:32:43.232 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:32:43.232 INFO:teuthology.orchestra.run.smithi084.stdout: "ce477bbf": { 2024-04-26T18:32:43.232 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 12, 2024-04-26T18:32:43.232 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:32:43.232 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:32:43.232 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:32:43.232 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:32:43.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:32:43.232 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:32:43.232 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 79.5, 2024-04-26T18:32:43.233 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:32:43.233 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:32:43.233 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:32:43.233 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:32:43.233 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:32:43.233 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:32:43.233 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:43.233 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:43.233 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:43.233 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:43.233 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:43.233 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:32:43.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.227+0000 7f693a43c700 1 -- 172.21.15.84:0/2810461640 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f691c0a3e30 msgr2=0x7f691c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:43.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.228+0000 7f693a43c700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f691c0a3e30 0x7f691c0a62f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6928005fd0 tx=0x7f6928005ee0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.228+0000 7f693a43c700 1 -- 172.21.15.84:0/2810461640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6934102610 msgr2=0x7f693419a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:43.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.228+0000 7f693a43c700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6934102610 0x7f693419a990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f69240098e0 tx=0x7f6924005e80 comp rx=0 tx=0).stop 2024-04-26T18:32:43.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.228+0000 7f693a43c700 1 -- 172.21.15.84:0/2810461640 shutdown_connections 2024-04-26T18:32:43.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.228+0000 7f693a43c700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f691c0a3e30 0x7f691c0a62f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.228+0000 7f693a43c700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6934102610 0x7f693419a990 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.228+0000 7f693a43c700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6934107660 0x7f69341a5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.228+0000 7f693a43c700 1 --2- 172.21.15.84:0/2810461640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6934102fc0 0x7f693419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:43.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.228+0000 7f693a43c700 1 -- 172.21.15.84:0/2810461640 >> 172.21.15.84:0/2810461640 conn(0x7f6934078370 msgr2=0x7f6934108830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:43.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.228+0000 7f693a43c700 1 -- 172.21.15.84:0/2810461640 shutdown_connections 2024-04-26T18:32:43.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:43.228+0000 7f693a43c700 1 -- 172.21.15.84:0/2810461640 wait complete. 2024-04-26T18:32:44.022 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'ce477bbf', seconds elapsed: 31.5 2024-04-26T18:32:44.022 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.5 2024-04-26T18:32:44.023 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:32:44.260 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:32:44.652 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:44 smithi084 ceph-mon[29141]: pgmap v146: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-04-26T18:32:44.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:44 smithi174 ceph-mon[32705]: pgmap v146: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-04-26T18:32:45.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:44 smithi103 ceph-mon[32727]: pgmap v146: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 6.8 KiB/s wr, 14 op/s 2024-04-26T18:32:46.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.080+0000 7f83a9d6d700 1 -- 172.21.15.84:0/4122776955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83a4107640 msgr2=0x7f83a410f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:46.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.080+0000 7f83a9d6d700 1 --2- 172.21.15.84:0/4122776955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83a4107640 0x7f83a410f210 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f839c00b0d0 tx=0x7f839c00b3e0 comp rx=0 tx=0).stop 2024-04-26T18:32:46.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.080+0000 7f83a9d6d700 1 -- 172.21.15.84:0/4122776955 shutdown_connections 2024-04-26T18:32:46.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.080+0000 7f83a9d6d700 1 --2- 172.21.15.84:0/4122776955 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83a4107640 0x7f83a410f210 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:46.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.080+0000 7f83a9d6d700 1 --2- 172.21.15.84:0/4122776955 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83a41025b0 0x7f83a4102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:46.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.080+0000 7f83a9d6d700 1 --2- 172.21.15.84:0/4122776955 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83a4102f60 0x7f83a4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:46.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.080+0000 7f83a9d6d700 1 -- 172.21.15.84:0/4122776955 >> 172.21.15.84:0/4122776955 conn(0x7f83a40792d0 msgr2=0x7f83a40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:46.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.080+0000 7f83a9d6d700 1 -- 172.21.15.84:0/4122776955 shutdown_connections 2024-04-26T18:32:46.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.080+0000 7f83a9d6d700 1 -- 172.21.15.84:0/4122776955 wait complete. 2024-04-26T18:32:46.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.080+0000 7f83a9d6d700 1 Processor -- start 2024-04-26T18:32:46.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a9d6d700 1 -- start start 2024-04-26T18:32:46.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a9d6d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83a41025b0 0x7f83a419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:46.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a9d6d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83a4102f60 0x7f83a419af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:46.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a9d6d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83a4107640 0x7f83a41a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:46.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a9d6d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f83a4112550 con 0x7f83a4107640 2024-04-26T18:32:46.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a9d6d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41123d0 con 0x7f83a4102f60 2024-04-26T18:32:46.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a9d6d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f83a41126d0 con 0x7f83a41025b0 2024-04-26T18:32:46.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83a4107640 0x7f83a41a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:46.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83a4107640 0x7f83a41a5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36628/0 (socket says 172.21.15.84:36628) 2024-04-26T18:32:46.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a3fff700 1 -- 172.21.15.84:0/3003495803 learned_addr learned my addr 172.21.15.84:0/3003495803 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:46.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a37fe700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83a41025b0 0x7f83a419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:46.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a2ffd700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83a4102f60 0x7f83a419af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:46.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a3fff700 1 -- 172.21.15.84:0/3003495803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83a41025b0 msgr2=0x7f83a419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:46.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a3fff700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83a41025b0 0x7f83a419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:46.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a3fff700 1 -- 172.21.15.84:0/3003495803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83a4102f60 msgr2=0x7f83a419af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:46.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a3fff700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83a4102f60 0x7f83a419af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:46.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a3fff700 1 -- 172.21.15.84:0/3003495803 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83a41a5fb0 con 0x7f83a4107640 2024-04-26T18:32:46.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.081+0000 7f83a3fff700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83a4107640 0x7f83a41a5a70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f839c0095c0 tx=0x7f839c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:46.084 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.082+0000 7f83a0ff9700 1 -- 172.21.15.84:0/3003495803 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c01f070 con 0x7f83a4107640 2024-04-26T18:32:46.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.082+0000 7f83a0ff9700 1 -- 172.21.15.84:0/3003495803 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f839c007e20 con 0x7f83a4107640 2024-04-26T18:32:46.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.082+0000 7f83a9d6d700 1 -- 172.21.15.84:0/3003495803 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83a41a6180 con 0x7f83a4107640 2024-04-26T18:32:46.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.082+0000 7f83a0ff9700 1 -- 172.21.15.84:0/3003495803 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f839c01b4a0 con 0x7f83a4107640 2024-04-26T18:32:46.085 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.082+0000 7f83a9d6d700 1 -- 172.21.15.84:0/3003495803 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83a41a6570 con 0x7f83a4107640 2024-04-26T18:32:46.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.083+0000 7f83a9d6d700 1 -- 172.21.15.84:0/3003495803 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83a406a110 con 0x7f83a4107640 2024-04-26T18:32:46.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.084+0000 7f83a0ff9700 1 -- 172.21.15.84:0/3003495803 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f839c022070 con 0x7f83a4107640 2024-04-26T18:32:46.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.084+0000 7f83a0ff9700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f838c0a3db0 0x7f838c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:46.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.085+0000 7f83a37fe700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f838c0a3db0 0x7f838c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:46.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.085+0000 7f83a0ff9700 1 -- 172.21.15.84:0/3003495803 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f839c00f020 con 0x7f83a4107640 2024-04-26T18:32:46.086 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.085+0000 7f83a37fe700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f838c0a3db0 0x7f838c0a6270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8394005950 tx=0x7f839400b410 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:46.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.089+0000 7f83a0ff9700 1 -- 172.21.15.84:0/3003495803 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f839c029070 con 0x7f83a4107640 2024-04-26T18:32:46.338 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.338+0000 7f83a9d6d700 1 -- 172.21.15.84:0/3003495803 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83a402e650 con 0x7f83a4107640 2024-04-26T18:32:46.339 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.339+0000 7f83a0ff9700 1 -- 172.21.15.84:0/3003495803 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f839c022e40 con 0x7f83a4107640 2024-04-26T18:32:46.339 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:32:46.340 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:32:46.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.342+0000 7f83a9d6d700 1 -- 172.21.15.84:0/3003495803 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f838c0a3db0 msgr2=0x7f838c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:46.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.342+0000 7f83a9d6d700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f838c0a3db0 0x7f838c0a6270 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8394005950 tx=0x7f839400b410 comp rx=0 tx=0).stop 2024-04-26T18:32:46.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.342+0000 7f83a9d6d700 1 -- 172.21.15.84:0/3003495803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83a4107640 msgr2=0x7f83a41a5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:46.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.342+0000 7f83a9d6d700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83a4107640 0x7f83a41a5a70 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f839c0095c0 tx=0x7f839c007a20 comp rx=0 tx=0).stop 2024-04-26T18:32:46.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.342+0000 7f83a9d6d700 1 -- 172.21.15.84:0/3003495803 shutdown_connections 2024-04-26T18:32:46.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.342+0000 7f83a9d6d700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f838c0a3db0 0x7f838c0a6270 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:46.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.342+0000 7f83a9d6d700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f83a4107640 0x7f83a41a5a70 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:46.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.342+0000 7f83a9d6d700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f83a41025b0 0x7f83a419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:46.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.342+0000 7f83a9d6d700 1 --2- 172.21.15.84:0/3003495803 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f83a4102f60 0x7f83a419af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:46.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.342+0000 7f83a9d6d700 1 -- 172.21.15.84:0/3003495803 >> 172.21.15.84:0/3003495803 conn(0x7f83a40792d0 msgr2=0x7f83a41003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:46.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.342+0000 7f83a9d6d700 1 -- 172.21.15.84:0/3003495803 shutdown_connections 2024-04-26T18:32:46.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:46.342+0000 7f83a9d6d700 1 -- 172.21.15.84:0/3003495803 wait complete. 2024-04-26T18:32:46.344 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:32:46.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:46 smithi174 ceph-mon[32705]: pgmap v147: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-04-26T18:32:46.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:46 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3003495803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:32:47.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:46 smithi084 ceph-mon[29141]: pgmap v147: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-04-26T18:32:47.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:46 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3003495803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:32:47.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:46 smithi103 ceph-mon[32727]: pgmap v147: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-04-26T18:32:47.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:46 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3003495803' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:32:47.170 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 60 --expiration 79.5 --await-for 42.0' 2024-04-26T18:32:47.170 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db / --timeout 60 --expiration 79.5 --await-for 42.0 2024-04-26T18:32:47.424 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:32:48.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:48 smithi084 ceph-mon[29141]: pgmap v148: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-04-26T18:32:48.928 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:48 smithi103 ceph-mon[32727]: pgmap v148: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-04-26T18:32:48.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:48 smithi174 ceph-mon[32705]: pgmap v148: 97 pgs: 97 active+clean; 4.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.6 KiB/s wr, 10 op/s 2024-04-26T18:32:49.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.193+0000 7fa324818700 1 -- 172.21.15.84:0/3962435899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa320103500 msgr2=0x7fa320103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.193+0000 7fa324818700 1 --2- 172.21.15.84:0/3962435899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa320103500 0x7fa320103980 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa3140099c0 tx=0x7fa314009cd0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.193+0000 7fa324818700 1 -- 172.21.15.84:0/3962435899 shutdown_connections 2024-04-26T18:32:49.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.193+0000 7fa324818700 1 --2- 172.21.15.84:0/3962435899 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa320103ec0 0x7fa320110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.195 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.193+0000 7fa324818700 1 --2- 172.21.15.84:0/3962435899 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa320109540 0x7fa320109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.193+0000 7fa324818700 1 --2- 172.21.15.84:0/3962435899 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa320103500 0x7fa320103980 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.193+0000 7fa324818700 1 -- 172.21.15.84:0/3962435899 >> 172.21.15.84:0/3962435899 conn(0x7fa32007a610 msgr2=0x7fa32007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.193+0000 7fa324818700 1 -- 172.21.15.84:0/3962435899 shutdown_connections 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.193+0000 7fa324818700 1 -- 172.21.15.84:0/3962435899 wait complete. 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.194+0000 7fa324818700 1 Processor -- start 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.194+0000 7fa324818700 1 -- start start 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.194+0000 7fa324818700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa320103500 0x7fa320075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.194+0000 7fa324818700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa320103ec0 0x7fa320075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.194+0000 7fa324818700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa320109540 0x7fa3200761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.194+0000 7fa324818700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201136c0 con 0x7fa320103500 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.194+0000 7fa324818700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa320113540 con 0x7fa320103ec0 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.194+0000 7fa324818700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa320113840 con 0x7fa320109540 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.194+0000 7fa31e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa320103500 0x7fa320075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.196 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.194+0000 7fa31e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa320103500 0x7fa320075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50806/0 (socket says 172.21.15.84:50806) 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.194+0000 7fa31e59c700 1 -- 172.21.15.84:0/433719181 learned_addr learned my addr 172.21.15.84:0/433719181 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.195+0000 7fa31ed9d700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa320109540 0x7fa3200761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.195+0000 7fa31dd9b700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa320103ec0 0x7fa320075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.195+0000 7fa31e59c700 1 -- 172.21.15.84:0/433719181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa320109540 msgr2=0x7fa3200761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.195+0000 7fa31e59c700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa320109540 0x7fa3200761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.195+0000 7fa31e59c700 1 -- 172.21.15.84:0/433719181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa320103ec0 msgr2=0x7fa320075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.195+0000 7fa31e59c700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa320103ec0 0x7fa320075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.195+0000 7fa31e59c700 1 -- 172.21.15.84:0/433719181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3201a7f30 con 0x7fa320103500 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.195+0000 7fa31e59c700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa320103500 0x7fa320075720 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa31000d980 tx=0x7fa31000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.195+0000 7fa30f7fe700 1 -- 172.21.15.84:0/433719181 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa310017070 con 0x7fa320103500 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.195+0000 7fa30f7fe700 1 -- 172.21.15.84:0/433719181 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa310005020 con 0x7fa320103500 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.195+0000 7fa324818700 1 -- 172.21.15.84:0/433719181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3201a8220 con 0x7fa320103500 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.195+0000 7fa30f7fe700 1 -- 172.21.15.84:0/433719181 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31000fa10 con 0x7fa320103500 2024-04-26T18:32:49.197 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.195+0000 7fa324818700 1 -- 172.21.15.84:0/433719181 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3201a86e0 con 0x7fa320103500 2024-04-26T18:32:49.198 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.197+0000 7fa30f7fe700 1 -- 172.21.15.84:0/433719181 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa310004a30 con 0x7fa320103500 2024-04-26T18:32:49.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.198+0000 7fa30f7fe700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa3080a3e50 0x7fa3080a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.198+0000 7fa31dd9b700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa3080a3e50 0x7fa3080a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.198+0000 7fa30f7fe700 1 -- 172.21.15.84:0/433719181 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fa3100d06c0 con 0x7fa320103500 2024-04-26T18:32:49.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.199+0000 7fa31dd9b700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa3080a3e50 0x7fa3080a6310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa314009990 tx=0x7fa31400b790 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:49.199 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.199+0000 7fa32f0aab80 1 Processor -- start 2024-04-26T18:32:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.199+0000 7fa32f0aab80 1 -- start start 2024-04-26T18:32:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.199+0000 7fa32f0aab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f00cf30 0x56180f00d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.199+0000 7fa32f0aab80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 0x56180f0072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.199+0000 7fa32f0aab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f007900 0x56180f007d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.199+0000 7fa32f0aab80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56180ef599c0 con 0x56180f00d850 2024-04-26T18:32:49.205 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.199+0000 7fa32f0aab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56180eebfac0 con 0x56180f00cf30 2024-04-26T18:32:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.199+0000 7fa32f0aab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56180ef55980 con 0x56180f007900 2024-04-26T18:32:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.199+0000 7fa31dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 0x56180f0072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.199+0000 7fa31dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 0x56180f0072e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50822/0 (socket says 172.21.15.84:50822) 2024-04-26T18:32:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.199+0000 7fa31dd9b700 1 -- 172.21.15.84:0/4242407360 learned_addr learned my addr 172.21.15.84:0/4242407360 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.199+0000 7fa31e59c700 1 --2- 172.21.15.84:0/4242407360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f00cf30 0x56180f00d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.200+0000 7fa31ed9d700 1 --2- 172.21.15.84:0/4242407360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f007900 0x56180f007d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.200+0000 7fa31dd9b700 1 -- 172.21.15.84:0/4242407360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f007900 msgr2=0x56180f007d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.200+0000 7fa31dd9b700 1 --2- 172.21.15.84:0/4242407360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f007900 0x56180f007d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.200+0000 7fa31dd9b700 1 -- 172.21.15.84:0/4242407360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f00cf30 msgr2=0x56180f00d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.200+0000 7fa31dd9b700 1 --2- 172.21.15.84:0/4242407360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f00cf30 0x56180f00d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.200+0000 7fa31dd9b700 1 -- 172.21.15.84:0/4242407360 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56180eea6a60 con 0x56180f00d850 2024-04-26T18:32:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.200+0000 7fa31ed9d700 1 --2- 172.21.15.84:0/4242407360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f007900 0x56180f007d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:49.206 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.200+0000 7fa31e59c700 1 --2- 172.21.15.84:0/4242407360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f00cf30 0x56180f00d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.200+0000 7fa31dd9b700 1 --2- 172.21.15.84:0/4242407360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 0x56180f0072e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa3140099c0 tx=0x7fa31401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aab60553fd5a4294 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.200+0000 7fa30cff9700 1 -- 172.21.15.84:0/4242407360 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314034070 con 0x56180f00d850 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.200+0000 7fa30cff9700 1 -- 172.21.15.84:0/4242407360 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa314024930 con 0x56180f00d850 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.200+0000 7fa30cff9700 1 -- 172.21.15.84:0/4242407360 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31402fa60 con 0x56180f00d850 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.201+0000 7fa32f0aab80 1 -- 172.21.15.84:0/4242407360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 msgr2=0x56180f0072e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.201+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/4242407360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 0x56180f0072e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa3140099c0 tx=0x7fa31401f000 comp rx=0 tx=0).stop 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.201+0000 7fa32f0aab80 1 -- 172.21.15.84:0/4242407360 shutdown_connections 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.201+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/4242407360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f007900 0x56180f007d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.201+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/4242407360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f00cf30 0x56180f00d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.201+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/4242407360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 0x56180f0072e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.201+0000 7fa32f0aab80 1 -- 172.21.15.84:0/4242407360 >> 172.21.15.84:0/4242407360 conn(0x56180ef35750 msgr2=0x56180f002090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.201+0000 7fa32f0aab80 1 -- 172.21.15.84:0/4242407360 shutdown_connections 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.201+0000 7fa32f0aab80 1 -- 172.21.15.84:0/4242407360 wait complete. 2024-04-26T18:32:49.207 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.201+0000 7fa32f0aab80 1 -- start start 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.201+0000 7fa32f0aab80 1 Processor -- start 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa32f0aab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f007900 0x56180f013520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa32f0aab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f00cf30 0x56180f0108c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa32f0aab80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 0x56180f010e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa32f0aab80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56180f014260 con 0x56180f00d850 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa32f0aab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56180ef13c10 con 0x56180f007900 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa32f0aab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56180f0143e0 con 0x56180f00cf30 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa32f0aab80 10 client.0 _ll_register_callbacks cb 0x56180eef1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa31ed9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 0x56180f010e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa31ed9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 0x56180f010e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50828/0 (socket says 172.21.15.84:50828) 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa31ed9d700 1 -- 172.21.15.84:0/3081388633 learned_addr learned my addr 172.21.15.84:0/3081388633 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa31dd9b700 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f00cf30 0x56180f0108c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa31e59c700 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f007900 0x56180f013520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa31ed9d700 1 -- 172.21.15.84:0/3081388633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f00cf30 msgr2=0x56180f0108c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.208 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa31ed9d700 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f00cf30 0x56180f0108c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa31ed9d700 1 -- 172.21.15.84:0/3081388633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f007900 msgr2=0x56180f013520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa31ed9d700 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f007900 0x56180f013520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa31ed9d700 1 -- 172.21.15.84:0/3081388633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56180ef17610 con 0x56180f00d850 2024-04-26T18:32:49.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa31e59c700 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f007900 0x56180f013520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:49.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa31ed9d700 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 0x56180f010e00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa31800aee0 tx=0x7fa318009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b4daff72580e1b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:49.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa305ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:49.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa305ffb700 1 -- 172.21.15.84:0/3081388633 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa318007b30 con 0x56180f00d850 2024-04-26T18:32:49.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa32f0aab80 1 -- 172.21.15.84:0/3081388633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa314024930 con 0x56180f00d850 2024-04-26T18:32:49.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.202+0000 7fa305ffb700 1 -- 172.21.15.84:0/3081388633 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fa318007cd0 con 0x56180f00d850 2024-04-26T18:32:49.209 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.203+0000 7fa305ffb700 1 -- 172.21.15.84:0/3081388633 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa3180153f0 con 0x56180f00d850 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.203+0000 7fa32f0aab80 10 client.16017 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.203+0000 7fa32f0aab80 1 -- 172.21.15.84:0/3081388633 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56180f01eab0 con 0x56180f00d850 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.203+0000 7fa305ffb700 1 -- 172.21.15.84:0/3081388633 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa314024930 con 0x56180f00d850 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.203+0000 7fa305ffb700 1 -- 172.21.15.84:0/3081388633 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa31801e530 con 0x56180f00d850 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.203+0000 7fa32f0aab80 10 client.16017 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.203+0000 7fa32f0aab80 10 client.16017 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.203+0000 7fa32f0aab80 20 client.16017 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.203+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56180f0210f0 0x56180f023550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.203+0000 7fa32f0aab80 4 client.16017 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.203+0000 7fa32f0aab80 1 -- 172.21.15.84:0/3081388633 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56180ef673a0 con 0x56180f0210f0 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.203+0000 7fa31e59c700 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56180f0210f0 0x56180f023550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.204+0000 7fa31e59c700 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56180f0210f0 0x56180f023550 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61f897858766c9d4 server_cookie=88e028aa459e5f6 in_seq=0 out_seq=0 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.204+0000 7fa305ffb700 10 client.16017 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:49.210 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.204+0000 7fa305ffb700 1 -- 172.21.15.84:0/3081388633 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56180ef673a0 con 0x56180f0210f0 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.204+0000 7fa305ffb700 10 client.16017 handle_command_reply: tid=0 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.204+0000 7fa32f0aab80 1 client.16017 shutdown 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.204+0000 7fa32f0aab80 1 -- 172.21.15.84:0/3081388633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 msgr2=0x56180f010e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.204+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 0x56180f010e00 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa31800aee0 tx=0x7fa318009d20 comp rx=0 tx=0).stop 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.205+0000 7fa32f0aab80 1 -- 172.21.15.84:0/3081388633 shutdown_connections 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.205+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56180f0210f0 0x56180f023550 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.205+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f00d850 0x56180f010e00 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.205+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f007900 0x56180f013520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.205+0000 7fa305ffb700 0 client.16017 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.205+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/3081388633 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f00cf30 0x56180f0108c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.205+0000 7fa32f0aab80 1 -- 172.21.15.84:0/3081388633 >> 172.21.15.84:0/3081388633 conn(0x56180ef35750 msgr2=0x56180f0062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.205+0000 7fa32f0aab80 1 -- 172.21.15.84:0/3081388633 shutdown_connections 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.205+0000 7fa32f0aab80 1 -- 172.21.15.84:0/3081388633 wait complete. 2024-04-26T18:32:49.211 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.205+0000 7fa32f0aab80 20 client.16017 trim_cache size 0 max 16384 2024-04-26T18:32:49.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.224+0000 7fa32f0aab80 1 Processor -- start 2024-04-26T18:32:49.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.224+0000 7fa32f0aab80 1 -- start start 2024-04-26T18:32:49.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa32f0aab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180ef35750 0x56180f0072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa32f0aab80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 0x56180f0101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa32f0aab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f00cf30 0x56180f01d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa32f0aab80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201afd10 con 0x56180f007900 2024-04-26T18:32:49.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa32f0aab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201b1850 con 0x56180ef35750 2024-04-26T18:32:49.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa32f0aab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201b03e0 con 0x56180f00cf30 2024-04-26T18:32:49.230 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa31dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 0x56180f0101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa31dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 0x56180f0101c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50844/0 (socket says 172.21.15.84:50844) 2024-04-26T18:32:49.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa31dd9b700 1 -- 172.21.15.84:0/1147657035 learned_addr learned my addr 172.21.15.84:0/1147657035 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:49.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa31ed9d700 1 --2- 172.21.15.84:0/1147657035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f00cf30 0x56180f01d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa31e59c700 1 --2- 172.21.15.84:0/1147657035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180ef35750 0x56180f0072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa31dd9b700 1 -- 172.21.15.84:0/1147657035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f00cf30 msgr2=0x56180f01d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa31dd9b700 1 --2- 172.21.15.84:0/1147657035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f00cf30 0x56180f01d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa31dd9b700 1 -- 172.21.15.84:0/1147657035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180ef35750 msgr2=0x56180f0072e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa31dd9b700 1 --2- 172.21.15.84:0/1147657035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180ef35750 0x56180f0072e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa31dd9b700 1 -- 172.21.15.84:0/1147657035 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56180effe880 con 0x56180f007900 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa31e59c700 1 --2- 172.21.15.84:0/1147657035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180ef35750 0x56180f0072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa31dd9b700 1 --2- 172.21.15.84:0/1147657035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 0x56180f0101c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa31400c010 tx=0x7fa314024630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4432123a55ce3f50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa307fff700 1 -- 172.21.15.84:0/1147657035 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314034040 con 0x56180f007900 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa307fff700 1 -- 172.21.15.84:0/1147657035 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa314024d90 con 0x56180f007900 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.225+0000 7fa307fff700 1 -- 172.21.15.84:0/1147657035 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa31402fdf0 con 0x56180f007900 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.226+0000 7fa32f0aab80 1 -- 172.21.15.84:0/1147657035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 msgr2=0x56180f0101c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.226+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/1147657035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 0x56180f0101c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa31400c010 tx=0x7fa314024630 comp rx=0 tx=0).stop 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.226+0000 7fa32f0aab80 1 -- 172.21.15.84:0/1147657035 shutdown_connections 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.226+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/1147657035 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180f00cf30 0x56180f01d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.226+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/1147657035 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180ef35750 0x56180f0072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.226+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/1147657035 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 0x56180f0101c0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.226+0000 7fa32f0aab80 1 -- 172.21.15.84:0/1147657035 >> 172.21.15.84:0/1147657035 conn(0x56180f00b200 msgr2=0x56180effbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:49.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.226+0000 7fa32f0aab80 1 -- 172.21.15.84:0/1147657035 shutdown_connections 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.226+0000 7fa32f0aab80 1 -- 172.21.15.84:0/1147657035 wait complete. 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.226+0000 7fa32f0aab80 1 -- start start 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.226+0000 7fa32f0aab80 1 Processor -- start 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa32f0aab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180ef35750 0x56180f016670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa32f0aab80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 0x56180f016bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa32f0aab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f00cf30 0x56180f0170f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa32f0aab80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201b1180 con 0x56180f007900 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa32f0aab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201b0ab0 con 0x56180f00cf30 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa32f0aab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa3201adcb0 con 0x56180ef35750 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa32f0aab80 10 client.0 _ll_register_callbacks cb 0x56180eef1ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa31dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 0x56180f016bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa31dd9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 0x56180f016bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50850/0 (socket says 172.21.15.84:50850) 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa31dd9b700 1 -- 172.21.15.84:0/931482982 learned_addr learned my addr 172.21.15.84:0/931482982 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:49.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa31e59c700 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180ef35750 0x56180f016670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa31ed9d700 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f00cf30 0x56180f0170f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa31dd9b700 1 -- 172.21.15.84:0/931482982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180ef35750 msgr2=0x56180f016670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa31dd9b700 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180ef35750 0x56180f016670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa31dd9b700 1 -- 172.21.15.84:0/931482982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f00cf30 msgr2=0x56180f0170f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa31dd9b700 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f00cf30 0x56180f0170f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.234 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa31dd9b700 1 -- 172.21.15.84:0/931482982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56180f087720 con 0x56180f007900 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa31ed9d700 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f00cf30 0x56180f0170f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa31dd9b700 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 0x56180f016bb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa314032040 tx=0x7fa3140095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5249dead964e81a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa305ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa305ffb700 1 -- 172.21.15.84:0/931482982 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa314034040 con 0x56180f007900 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa305ffb700 1 -- 172.21.15.84:0/931482982 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fa31402fc80 con 0x56180f007900 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.227+0000 7fa32f0aab80 1 -- 172.21.15.84:0/931482982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa314024d90 con 0x56180f007900 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.228+0000 7fa305ffb700 1 -- 172.21.15.84:0/931482982 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa314053050 con 0x56180f007900 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.228+0000 7fa32f0aab80 10 client.16023 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.228+0000 7fa32f0aab80 1 -- 172.21.15.84:0/931482982 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56180f088950 con 0x56180f007900 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.228+0000 7fa305ffb700 1 -- 172.21.15.84:0/931482982 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa314024d90 con 0x56180f007900 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.228+0000 7fa305ffb700 1 -- 172.21.15.84:0/931482982 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa314037030 con 0x56180f007900 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.228+0000 7fa32f0aab80 10 client.16023 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.228+0000 7fa32f0aab80 10 client.16023 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.228+0000 7fa32f0aab80 20 client.16023 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:49.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.228+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56180f00d850 0x56180f08abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.228+0000 7fa32f0aab80 4 client.16023 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.5, "await_for": 42.0}] 2024-04-26T18:32:49.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.228+0000 7fa32f0aab80 1 -- 172.21.15.84:0/931482982 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.5, "await_for": 42.0}) -- 0x56180ef61440 con 0x56180f00d850 2024-04-26T18:32:49.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.228+0000 7fa31e59c700 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56180f00d850 0x56180f08abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.229+0000 7fa31e59c700 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56180f00d850 0x56180f08abf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dc6d9a1fcd98115 server_cookie=21e9544f339771eb in_seq=0 out_seq=0 2024-04-26T18:32:49.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.229+0000 7fa305ffb700 10 client.16023 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:49.242 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.241+0000 7fa305ffb700 1 -- 172.21.15.84:0/931482982 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56180ef61440 con 0x56180f00d850 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.241+0000 7fa305ffb700 10 client.16023 handle_command_reply: tid=0 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.241+0000 7fa32f0aab80 1 client.16023 shutdown 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.241+0000 7fa32f0aab80 1 -- 172.21.15.84:0/931482982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 msgr2=0x56180f016bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.241+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 0x56180f016bb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fa314032040 tx=0x7fa3140095d0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa32f0aab80 1 -- 172.21.15.84:0/931482982 shutdown_connections 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56180f00d850 0x56180f08abf0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa305ffb700 0 client.16023 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56180f00cf30 0x56180f0170f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56180ef35750 0x56180f016670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa32f0aab80 1 --2- 172.21.15.84:0/931482982 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56180f007900 0x56180f016bb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa32f0aab80 1 -- 172.21.15.84:0/931482982 >> 172.21.15.84:0/931482982 conn(0x56180f00b200 msgr2=0x56180f00ba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa32f0aab80 1 -- 172.21.15.84:0/931482982 shutdown_connections 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa32f0aab80 1 -- 172.21.15.84:0/931482982 wait complete. 2024-04-26T18:32:49.243 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa32f0aab80 20 client.16023 trim_cache size 0 max 16384 2024-04-26T18:32:49.244 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:32:49.244 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 14, 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: "f0da206a": { 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 14, 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 79.5, 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:32:49.245 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:32:49.246 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:32:49.246 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:32:49.246 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:32:49.246 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:32:49.246 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:49.246 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:49.246 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:49.246 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:49.246 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:32:49.246 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:32:49.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa324818700 1 -- 172.21.15.84:0/433719181 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa3080a3e50 msgr2=0x7fa3080a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa324818700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa3080a3e50 0x7fa3080a6310 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa314009990 tx=0x7fa31400b790 comp rx=0 tx=0).stop 2024-04-26T18:32:49.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa324818700 1 -- 172.21.15.84:0/433719181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa320103500 msgr2=0x7fa320075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa324818700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa320103500 0x7fa320075720 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa31000d980 tx=0x7fa31000dc90 comp rx=0 tx=0).stop 2024-04-26T18:32:49.247 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa324818700 1 -- 172.21.15.84:0/433719181 shutdown_connections 2024-04-26T18:32:49.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa324818700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa3080a3e50 0x7fa3080a6310 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa324818700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa320103500 0x7fa320075720 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa324818700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa320109540 0x7fa3200761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa324818700 1 --2- 172.21.15.84:0/433719181 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa320103ec0 0x7fa320075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa324818700 1 -- 172.21.15.84:0/433719181 >> 172.21.15.84:0/433719181 conn(0x7fa32007a610 msgr2=0x7fa320101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:49.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa324818700 1 -- 172.21.15.84:0/433719181 shutdown_connections 2024-04-26T18:32:49.248 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:32:49.242+0000 7fa324818700 1 -- 172.21.15.84:0/433719181 wait complete. 2024-04-26T18:32:49.352 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.342+0000 7fdc8d9bd640 1 -- 172.21.15.84:0/690007859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc88102540 msgr2=0x7fdc8810e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.342+0000 7fdc8d9bd640 1 --2- 172.21.15.84:0/690007859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc88102540 0x7fdc8810e650 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fdc80009ca0 tx=0x7fdc8002a8c0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.342+0000 7fdc8d9bd640 1 -- 172.21.15.84:0/690007859 shutdown_connections 2024-04-26T18:32:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.342+0000 7fdc8d9bd640 1 --2- 172.21.15.84:0/690007859 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc88102540 0x7fdc8810e650 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.342+0000 7fdc8d9bd640 1 --2- 172.21.15.84:0/690007859 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc88101b80 0x7fdc88102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.342+0000 7fdc8d9bd640 1 --2- 172.21.15.84:0/690007859 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc881067d0 0x7fdc88106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.342+0000 7fdc8d9bd640 1 -- 172.21.15.84:0/690007859 >> 172.21.15.84:0/690007859 conn(0x7fdc88077630 msgr2=0x7fdc88077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.342+0000 7fdc8d9bd640 1 -- 172.21.15.84:0/690007859 shutdown_connections 2024-04-26T18:32:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.342+0000 7fdc8d9bd640 1 -- 172.21.15.84:0/690007859 wait complete. 2024-04-26T18:32:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.343+0000 7fdc8d9bd640 1 Processor -- start 2024-04-26T18:32:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.343+0000 7fdc8d9bd640 1 -- start start 2024-04-26T18:32:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.343+0000 7fdc8d9bd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc88101b80 0x7fdc88199db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.353 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.343+0000 7fdc8d9bd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc88102540 0x7fdc8819a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.343+0000 7fdc8d9bd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc881067d0 0x7fdc881a4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.343+0000 7fdc8d9bd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88111ab0 con 0x7fdc88102540 2024-04-26T18:32:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.343+0000 7fdc8d9bd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88111930 con 0x7fdc88101b80 2024-04-26T18:32:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.343+0000 7fdc8d9bd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc88111c30 con 0x7fdc881067d0 2024-04-26T18:32:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.343+0000 7fdc7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc88102540 0x7fdc8819a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.343+0000 7fdc7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc88102540 0x7fdc8819a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50864/0 (socket says 172.21.15.84:50864) 2024-04-26T18:32:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.343+0000 7fdc7ffff640 1 -- 172.21.15.84:0/2850919415 learned_addr learned my addr 172.21.15.84:0/2850919415 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.343+0000 7fdc86ffd640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc88101b80 0x7fdc88199db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.343+0000 7fdc877fe640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc881067d0 0x7fdc881a4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.344+0000 7fdc7ffff640 1 -- 172.21.15.84:0/2850919415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc881067d0 msgr2=0x7fdc881a4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.344+0000 7fdc7ffff640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc881067d0 0x7fdc881a4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.344+0000 7fdc7ffff640 1 -- 172.21.15.84:0/2850919415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc88101b80 msgr2=0x7fdc88199db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.344+0000 7fdc7ffff640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc88101b80 0x7fdc88199db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.344+0000 7fdc7ffff640 1 -- 172.21.15.84:0/2850919415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc881a5370 con 0x7fdc88102540 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.344+0000 7fdc877fe640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc881067d0 0x7fdc881a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.344+0000 7fdc7ffff640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc88102540 0x7fdc8819a2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000d950 tx=0x7fdc7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.344+0000 7fdc8c9bb640 1 -- 172.21.15.84:0/2850919415 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc70017030 con 0x7fdc88102540 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.344+0000 7fdc8c9bb640 1 -- 172.21.15.84:0/2850919415 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc70004020 con 0x7fdc88102540 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.344+0000 7fdc8d9bd640 1 -- 172.21.15.84:0/2850919415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc881a5540 con 0x7fdc88102540 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.344+0000 7fdc8c9bb640 1 -- 172.21.15.84:0/2850919415 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc700047a0 con 0x7fdc88102540 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.344+0000 7fdc8d9bd640 1 -- 172.21.15.84:0/2850919415 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc881a5ab0 con 0x7fdc88102540 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.346+0000 7fdc8c9bb640 1 -- 172.21.15.84:0/2850919415 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fdc70009a30 con 0x7fdc88102540 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.346+0000 7fdc907b0740 1 Processor -- start 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.346+0000 7fdc907b0740 1 -- start start 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.346+0000 7fdc907b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbcef00 0x56501bbcf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.346+0000 7fdc907b0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 0x56501bbd9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.346+0000 7fdc907b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbda0e0 0x56501bbdc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.346+0000 7fdc907b0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56501bb35460 con 0x56501bbcf820 2024-04-26T18:32:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.346+0000 7fdc907b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56501baef590 con 0x56501bbcef00 2024-04-26T18:32:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.346+0000 7fdc907b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56501bb31420 con 0x56501bbda0e0 2024-04-26T18:32:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc86ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 0x56501bbd9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc86ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 0x56501bbd9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50872/0 (socket says 172.21.15.84:50872) 2024-04-26T18:32:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc86ffd640 1 -- 172.21.15.84:0/963980547 learned_addr learned my addr 172.21.15.84:0/963980547 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc8c9bb640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fdc740a3e40 0x7fdc740a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc7ffff640 1 --2- 172.21.15.84:0/963980547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbda0e0 0x56501bbdc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc86ffd640 1 -- 172.21.15.84:0/963980547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbda0e0 msgr2=0x56501bbdc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc877fe640 1 --2- 172.21.15.84:0/963980547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbcef00 0x56501bbcf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc86ffd640 1 --2- 172.21.15.84:0/963980547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbda0e0 0x56501bbdc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc86ffd640 1 -- 172.21.15.84:0/963980547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbcef00 msgr2=0x56501bbcf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc86ffd640 1 --2- 172.21.15.84:0/963980547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbcef00 0x56501bbcf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc86ffd640 1 -- 172.21.15.84:0/963980547 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56501bafc0d0 con 0x56501bbcf820 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc7ffff640 1 --2- 172.21.15.84:0/963980547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbda0e0 0x56501bbdc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc877fe640 1 --2- 172.21.15.84:0/963980547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbcef00 0x56501bbcf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc877fe640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fdc740a3e40 0x7fdc740a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc8c9bb640 1 -- 172.21.15.84:0/2850919415 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fdc700d01e0 con 0x7fdc88102540 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc86ffd640 1 --2- 172.21.15.84:0/963980547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 0x56501bbd9ba0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800bc90 tx=0x7fdc780107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc6ad7b1c35ce60 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc7dffb640 1 -- 172.21.15.84:0/963980547 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7801c480 con 0x56501bbcf820 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc7dffb640 1 -- 172.21.15.84:0/963980547 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc7801cad0 con 0x56501bbcf820 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.347+0000 7fdc7dffb640 1 -- 172.21.15.84:0/963980547 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc78011af0 con 0x56501bbcf820 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.348+0000 7fdc907b0740 1 -- 172.21.15.84:0/963980547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 msgr2=0x56501bbd9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.348+0000 7fdc877fe640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fdc740a3e40 0x7fdc740a6300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdc8002f500 tx=0x7fdc80004a40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.348+0000 7fdc907b0740 1 --2- 172.21.15.84:0/963980547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 0x56501bbd9ba0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdc7800bc90 tx=0x7fdc780107a0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.348+0000 7fdc907b0740 1 -- 172.21.15.84:0/963980547 shutdown_connections 2024-04-26T18:32:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.348+0000 7fdc907b0740 1 --2- 172.21.15.84:0/963980547 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbda0e0 0x56501bbdc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.348+0000 7fdc907b0740 1 --2- 172.21.15.84:0/963980547 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 0x56501bbd9ba0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.348+0000 7fdc907b0740 1 --2- 172.21.15.84:0/963980547 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbcef00 0x56501bbcf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.348+0000 7fdc907b0740 1 -- 172.21.15.84:0/963980547 >> 172.21.15.84:0/963980547 conn(0x56501bbcb410 msgr2=0x56501bbcb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.348+0000 7fdc907b0740 1 -- 172.21.15.84:0/963980547 shutdown_connections 2024-04-26T18:32:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.348+0000 7fdc907b0740 1 -- 172.21.15.84:0/963980547 wait complete. 2024-04-26T18:32:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.348+0000 7fdc907b0740 1 -- start start 2024-04-26T18:32:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc907b0740 1 Processor -- start 2024-04-26T18:32:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc907b0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcef00 0x56501bbd7000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc907b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcf820 0x56501bbd7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc907b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbda0e0 0x56501bbd7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc907b0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56501bbdcd10 con 0x56501bbcef00 2024-04-26T18:32:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc907b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56501babcb60 con 0x56501bbda0e0 2024-04-26T18:32:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc907b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56501bbdce90 con 0x56501bbcf820 2024-04-26T18:32:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc907b0740 10 client.0 _ll_register_callbacks cb 0x56501bae12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc86ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcef00 0x56501bbd7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc86ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcef00 0x56501bbd7000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50884/0 (socket says 172.21.15.84:50884) 2024-04-26T18:32:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc86ffd640 1 -- 172.21.15.84:0/2635402384 learned_addr learned my addr 172.21.15.84:0/2635402384 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc7ffff640 1 --2- 172.21.15.84:0/2635402384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcf820 0x56501bbd7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc877fe640 1 --2- 172.21.15.84:0/2635402384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbda0e0 0x56501bbd7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc86ffd640 1 -- 172.21.15.84:0/2635402384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcf820 msgr2=0x56501bbd7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc86ffd640 1 --2- 172.21.15.84:0/2635402384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcf820 0x56501bbd7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc86ffd640 1 -- 172.21.15.84:0/2635402384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbda0e0 msgr2=0x56501bbd7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc86ffd640 1 --2- 172.21.15.84:0/2635402384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbda0e0 0x56501bbd7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc86ffd640 1 -- 172.21.15.84:0/2635402384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56501b847210 con 0x56501bbcef00 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc86ffd640 1 --2- 172.21.15.84:0/2635402384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcef00 0x56501bbd7000 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdc78011730 tx=0x7fdc7801ceb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db6b23b7404476b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.349+0000 7fdc66ffd640 1 -- 172.21.15.84:0/2635402384 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc7801c890 con 0x56501bbcef00 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc66ffd640 1 -- 172.21.15.84:0/2635402384 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fdc78011890 con 0x56501bbcef00 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc907b0740 1 -- 172.21.15.84:0/2635402384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc7801cad0 con 0x56501bbcef00 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc66ffd640 1 -- 172.21.15.84:0/2635402384 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc78011a90 con 0x56501bbcef00 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc907b0740 10 client.16035 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc907b0740 1 -- 172.21.15.84:0/2635402384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56501b913620 con 0x56501bbcef00 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc66ffd640 1 -- 172.21.15.84:0/2635402384 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc7801cad0 con 0x56501bbcef00 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc66ffd640 1 -- 172.21.15.84:0/2635402384 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdc780047e0 con 0x56501bbcef00 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc907b0740 10 client.16035 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc907b0740 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:32:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc907b0740 20 client.16035 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc907b0740 1 --2- 172.21.15.84:0/2635402384 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56501bbe9f70 0x56501bbec360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc907b0740 4 client.16035 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:32:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc907b0740 1 -- 172.21.15.84:0/2635402384 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56501bb43070 con 0x56501bbe9f70 2024-04-26T18:32:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.350+0000 7fdc7ffff640 1 --2- 172.21.15.84:0/2635402384 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56501bbe9f70 0x56501bbec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.351+0000 7fdc66ffd640 10 client.16035 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.351+0000 7fdc7ffff640 1 --2- 172.21.15.84:0/2635402384 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56501bbe9f70 0x56501bbec360 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24b33253ce8def94 server_cookie=223de06c8fbb0b49 in_seq=0 out_seq=0 2024-04-26T18:32:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.351+0000 7fdc66ffd640 1 -- 172.21.15.84:0/2635402384 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56501bb43070 con 0x56501bbe9f70 2024-04-26T18:32:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.351+0000 7fdc66ffd640 10 client.16035 handle_command_reply: tid=0 2024-04-26T18:32:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.351+0000 7fdc907b0740 1 client.16035 shutdown 2024-04-26T18:32:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.352+0000 7fdc907b0740 1 -- 172.21.15.84:0/2635402384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcef00 msgr2=0x56501bbd7000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.352+0000 7fdc907b0740 1 --2- 172.21.15.84:0/2635402384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcef00 0x56501bbd7000 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdc78011730 tx=0x7fdc7801ceb0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.352+0000 7fdc907b0740 1 -- 172.21.15.84:0/2635402384 shutdown_connections 2024-04-26T18:32:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.352+0000 7fdc907b0740 1 --2- 172.21.15.84:0/2635402384 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56501bbe9f70 0x56501bbec360 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.352+0000 7fdc907b0740 1 --2- 172.21.15.84:0/2635402384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbda0e0 0x56501bbd7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.352+0000 7fdc907b0740 1 --2- 172.21.15.84:0/2635402384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcf820 0x56501bbd7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.352+0000 7fdc66ffd640 0 client.16035 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.352+0000 7fdc907b0740 1 --2- 172.21.15.84:0/2635402384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcef00 0x56501bbd7000 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.352+0000 7fdc907b0740 1 -- 172.21.15.84:0/2635402384 >> 172.21.15.84:0/2635402384 conn(0x56501bbcb410 msgr2=0x56501bbd1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.352+0000 7fdc907b0740 1 -- 172.21.15.84:0/2635402384 shutdown_connections 2024-04-26T18:32:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.352+0000 7fdc907b0740 1 -- 172.21.15.84:0/2635402384 wait complete. 2024-04-26T18:32:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.352+0000 7fdc907b0740 20 client.16035 trim_cache size 0 max 16384 2024-04-26T18:32:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.366+0000 7fdc907b0740 1 Processor -- start 2024-04-26T18:32:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.366+0000 7fdc907b0740 1 -- start start 2024-04-26T18:32:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc907b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcef00 0x56501bbd1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.368 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc907b0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 0x56501bbd67b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc907b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbda0e0 0x56501bbd4260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc907b0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881ae5d0 con 0x56501bbcf820 2024-04-26T18:32:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc907b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881b0110 con 0x56501bbda0e0 2024-04-26T18:32:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc907b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881aeca0 con 0x56501bbcef00 2024-04-26T18:32:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 0x56501bbd67b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc7ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 0x56501bbd67b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50890/0 (socket says 172.21.15.84:50890) 2024-04-26T18:32:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc7ffff640 1 -- 172.21.15.84:0/304510719 learned_addr learned my addr 172.21.15.84:0/304510719 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:49.369 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc7ffff640 1 -- 172.21.15.84:0/304510719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcef00 msgr2=0x56501bbd1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc7ffff640 1 --2- 172.21.15.84:0/304510719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcef00 0x56501bbd1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc7ffff640 1 -- 172.21.15.84:0/304510719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbda0e0 msgr2=0x56501bbd4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc7ffff640 1 --2- 172.21.15.84:0/304510719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbda0e0 0x56501bbd4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc7ffff640 1 -- 172.21.15.84:0/304510719 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56501bbcb240 con 0x56501bbcf820 2024-04-26T18:32:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc7ffff640 1 --2- 172.21.15.84:0/304510719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 0x56501bbd67b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdc70005fd0 tx=0x7fdc700d26f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8df7121c77de4a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc7d7fa640 1 -- 172.21.15.84:0/304510719 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc700b4070 con 0x56501bbcf820 2024-04-26T18:32:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc7d7fa640 1 -- 172.21.15.84:0/304510719 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc7009c170 con 0x56501bbcf820 2024-04-26T18:32:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.367+0000 7fdc7d7fa640 1 -- 172.21.15.84:0/304510719 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc700041d0 con 0x56501bbcf820 2024-04-26T18:32:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.368+0000 7fdc907b0740 1 -- 172.21.15.84:0/304510719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 msgr2=0x56501bbd67b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.368+0000 7fdc907b0740 1 --2- 172.21.15.84:0/304510719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 0x56501bbd67b0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdc70005fd0 tx=0x7fdc700d26f0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.370 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.368+0000 7fdc907b0740 1 -- 172.21.15.84:0/304510719 shutdown_connections 2024-04-26T18:32:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.368+0000 7fdc907b0740 1 --2- 172.21.15.84:0/304510719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbda0e0 0x56501bbd4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.368+0000 7fdc907b0740 1 --2- 172.21.15.84:0/304510719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbcf820 0x56501bbd67b0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.368+0000 7fdc907b0740 1 --2- 172.21.15.84:0/304510719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcef00 0x56501bbd1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.368+0000 7fdc907b0740 1 -- 172.21.15.84:0/304510719 >> 172.21.15.84:0/304510719 conn(0x56501bbcb410 msgr2=0x56501bbe5970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.368+0000 7fdc907b0740 1 -- 172.21.15.84:0/304510719 shutdown_connections 2024-04-26T18:32:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.368+0000 7fdc907b0740 1 -- 172.21.15.84:0/304510719 wait complete. 2024-04-26T18:32:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.368+0000 7fdc907b0740 1 -- start start 2024-04-26T18:32:49.371 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.368+0000 7fdc907b0740 1 Processor -- start 2024-04-26T18:32:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc907b0740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbcef00 0x56501bbef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc907b0740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcf820 0x56501bbec840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc907b0740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbda0e0 0x56501bbecd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.374 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc907b0740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881afa40 con 0x56501bbda0e0 2024-04-26T18:32:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc907b0740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881af370 con 0x56501bbcef00 2024-04-26T18:32:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc907b0740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdc881ac5b0 con 0x56501bbcf820 2024-04-26T18:32:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc907b0740 10 client.0 _ll_register_callbacks cb 0x7fdc7800bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:32:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbda0e0 0x56501bbecd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc877fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbda0e0 0x56501bbecd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:50904/0 (socket says 172.21.15.84:50904) 2024-04-26T18:32:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc877fe640 1 -- 172.21.15.84:0/780313352 learned_addr learned my addr 172.21.15.84:0/780313352 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:32:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc86ffd640 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbcef00 0x56501bbef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc7ffff640 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcf820 0x56501bbec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc877fe640 1 -- 172.21.15.84:0/780313352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcf820 msgr2=0x56501bbec840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.375 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc877fe640 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcf820 0x56501bbec840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc877fe640 1 -- 172.21.15.84:0/780313352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbcef00 msgr2=0x56501bbef4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:32:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc877fe640 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbcef00 0x56501bbef4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc877fe640 1 -- 172.21.15.84:0/780313352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56501bbed2c0 con 0x56501bbda0e0 2024-04-26T18:32:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc7ffff640 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcf820 0x56501bbec840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:32:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc877fe640 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbda0e0 0x56501bbecd80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdc80035000 tx=0x7fdc8002a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c26b0070f442e34 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:32:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:32:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc66ffd640 1 -- 172.21.15.84:0/780313352 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80004250 con 0x56501bbda0e0 2024-04-26T18:32:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc66ffd640 1 -- 172.21.15.84:0/780313352 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fdc800043f0 con 0x56501bbda0e0 2024-04-26T18:32:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc907b0740 1 -- 172.21.15.84:0/780313352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc7009c170 con 0x56501bbda0e0 2024-04-26T18:32:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc66ffd640 1 -- 172.21.15.84:0/780313352 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc800045f0 con 0x56501bbda0e0 2024-04-26T18:32:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc907b0740 10 client.16041 fetch_fsmap learned FSMap version 29 2024-04-26T18:32:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.369+0000 7fdc907b0740 1 -- 172.21.15.84:0/780313352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56501bc4feb0 con 0x56501bbda0e0 2024-04-26T18:32:49.376 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.370+0000 7fdc66ffd640 1 -- 172.21.15.84:0/780313352 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc7009c170 con 0x56501bbda0e0 2024-04-26T18:32:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.370+0000 7fdc66ffd640 1 -- 172.21.15.84:0/780313352 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdc80043040 con 0x56501bbda0e0 2024-04-26T18:32:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.370+0000 7fdc907b0740 10 client.16041 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:32:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.370+0000 7fdc907b0740 10 client.16041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:32:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.370+0000 7fdc907b0740 20 client.16041 populate_metadata read hostname 'smithi084' 2024-04-26T18:32:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.370+0000 7fdc907b0740 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56501bc50660 0x56501bc52a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:32:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.370+0000 7fdc907b0740 4 client.16041 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:32:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.370+0000 7fdc907b0740 1 -- 172.21.15.84:0/780313352 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56501bb3cee0 con 0x56501bc50660 2024-04-26T18:32:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.370+0000 7fdc86ffd640 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56501bc50660 0x56501bc52a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:32:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.370+0000 7fdc86ffd640 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56501bc50660 0x56501bc52a80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8aeb09d96dfd7f69 server_cookie=7c0dea18fad23db8 in_seq=0 out_seq=0 2024-04-26T18:32:49.377 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.370+0000 7fdc66ffd640 10 client.16041 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc66ffd640 1 -- 172.21.15.84:0/780313352 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56501bb3cee0 con 0x56501bc50660 2024-04-26T18:32:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc66ffd640 10 client.16041 handle_command_reply: tid=0 2024-04-26T18:32:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc907b0740 1 client.16041 shutdown 2024-04-26T18:32:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc907b0740 1 -- 172.21.15.84:0/780313352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbda0e0 msgr2=0x56501bbecd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc907b0740 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbda0e0 0x56501bbecd80 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdc80035000 tx=0x7fdc8002a8c0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc907b0740 1 -- 172.21.15.84:0/780313352 shutdown_connections 2024-04-26T18:32:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc907b0740 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56501bc50660 0x56501bc52a80 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc907b0740 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56501bbda0e0 0x56501bbecd80 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc66ffd640 0 client.16041 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:32:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc907b0740 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56501bbcf820 0x56501bbec840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc907b0740 1 --2- 172.21.15.84:0/780313352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56501bbcef00 0x56501bbef4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc907b0740 1 -- 172.21.15.84:0/780313352 >> 172.21.15.84:0/780313352 conn(0x56501bbcb410 msgr2=0x56501bbc9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc907b0740 1 -- 172.21.15.84:0/780313352 shutdown_connections 2024-04-26T18:32:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc907b0740 1 -- 172.21.15.84:0/780313352 wait complete. 2024-04-26T18:32:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.383+0000 7fdc907b0740 20 client.16041 trim_cache size 0 max 16384 2024-04-26T18:32:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:32:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:32:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:32:49.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:32:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.384+0000 7fdc8d9bd640 1 -- 172.21.15.84:0/2850919415 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fdc740a3e40 msgr2=0x7fdc740a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.384+0000 7fdc8d9bd640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fdc740a3e40 0x7fdc740a6300 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fdc8002f500 tx=0x7fdc80004a40 comp rx=0 tx=0).stop 2024-04-26T18:32:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.384+0000 7fdc8d9bd640 1 -- 172.21.15.84:0/2850919415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc88102540 msgr2=0x7fdc8819a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:32:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.384+0000 7fdc8d9bd640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc88102540 0x7fdc8819a2f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fdc7000d950 tx=0x7fdc7000de10 comp rx=0 tx=0).stop 2024-04-26T18:32:49.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.384+0000 7fdc8d9bd640 1 -- 172.21.15.84:0/2850919415 shutdown_connections 2024-04-26T18:32:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.384+0000 7fdc8d9bd640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fdc740a3e40 0x7fdc740a6300 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.384+0000 7fdc8d9bd640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdc881067d0 0x7fdc881a4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.384+0000 7fdc8d9bd640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdc88102540 0x7fdc8819a2f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.384+0000 7fdc8d9bd640 1 --2- 172.21.15.84:0/2850919415 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdc88101b80 0x7fdc88199db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:32:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.384+0000 7fdc8d9bd640 1 -- 172.21.15.84:0/2850919415 >> 172.21.15.84:0/2850919415 conn(0x7fdc88077630 msgr2=0x7fdc880ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:32:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.384+0000 7fdc8d9bd640 1 -- 172.21.15.84:0/2850919415 shutdown_connections 2024-04-26T18:32:49.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:32:49.384+0000 7fdc8d9bd640 1 -- 172.21.15.84:0/2850919415 wait complete. 2024-04-26T18:32:49.904 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f0da206a', quiesce took 5.9 seconds. Will release after: 19.5 2024-04-26T18:32:50.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:50 smithi174 ceph-mon[32705]: pgmap v149: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-26T18:32:51.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:50 smithi084 ceph-mon[29141]: pgmap v149: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-26T18:32:51.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:50 smithi103 ceph-mon[32727]: pgmap v149: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 16 op/s 2024-04-26T18:32:52.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:52 smithi174 ceph-mon[32705]: pgmap v150: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-04-26T18:32:53.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:52 smithi084 ceph-mon[29141]: pgmap v150: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-04-26T18:32:53.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:52 smithi103 ceph-mon[32727]: pgmap v150: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 10 op/s 2024-04-26T18:32:54.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:54 smithi174 ceph-mon[32705]: pgmap v151: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-26T18:32:55.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:54 smithi084 ceph-mon[29141]: pgmap v151: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-26T18:32:55.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:54 smithi103 ceph-mon[32727]: pgmap v151: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 18 KiB/s wr, 17 op/s 2024-04-26T18:32:56.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:56 smithi174 ceph-mon[32705]: pgmap v152: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T18:32:57.041 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:56 smithi084 ceph-mon[29141]: pgmap v152: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T18:32:57.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:56 smithi103 ceph-mon[32727]: pgmap v152: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T18:32:58.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:32:58 smithi174 ceph-mon[32705]: pgmap v153: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T18:32:59.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:32:58 smithi084 ceph-mon[29141]: pgmap v153: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T18:32:59.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:32:58 smithi103 ceph-mon[32727]: pgmap v153: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 16 KiB/s wr, 13 op/s 2024-04-26T18:33:00.911 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:00 smithi084 ceph-mon[29141]: pgmap v154: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-26T18:33:00.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:00 smithi174 ceph-mon[32705]: pgmap v154: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-26T18:33:01.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:00 smithi103 ceph-mon[32727]: pgmap v154: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 16 KiB/s wr, 17 op/s 2024-04-26T18:33:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.471+0000 7f9f59469640 1 -- 172.21.15.84:0/2125808655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f54108950 msgr2=0x7f9f54108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.471+0000 7f9f59469640 1 --2- 172.21.15.84:0/2125808655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f54108950 0x7f9f54108d30 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c009cd0 tx=0x7f9f4c02a8e0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.471+0000 7f9f59469640 1 -- 172.21.15.84:0/2125808655 shutdown_connections 2024-04-26T18:33:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.471+0000 7f9f59469640 1 --2- 172.21.15.84:0/2125808655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54103300 0x7f9f5410f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.471+0000 7f9f59469640 1 --2- 172.21.15.84:0/2125808655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f54102940 0x7f9f54102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.471+0000 7f9f59469640 1 --2- 172.21.15.84:0/2125808655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f54108950 0x7f9f54108d30 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.471+0000 7f9f59469640 1 -- 172.21.15.84:0/2125808655 >> 172.21.15.84:0/2125808655 conn(0x7f9f54079dc0 msgr2=0x7f9f5407a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.472+0000 7f9f59469640 1 -- 172.21.15.84:0/2125808655 shutdown_connections 2024-04-26T18:33:02.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.472+0000 7f9f59469640 1 -- 172.21.15.84:0/2125808655 wait complete. 2024-04-26T18:33:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.472+0000 7f9f59469640 1 Processor -- start 2024-04-26T18:33:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.472+0000 7f9f59469640 1 -- start start 2024-04-26T18:33:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.472+0000 7f9f59469640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f54102940 0x7f9f541990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.472+0000 7f9f59469640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f54103300 0x7f9f54199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.472+0000 7f9f59469640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54108950 0x7f9f5419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f59469640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54112c10 con 0x7f9f54102940 2024-04-26T18:33:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f59469640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54112a90 con 0x7f9f54108950 2024-04-26T18:33:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f59469640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f54112d90 con 0x7f9f54103300 2024-04-26T18:33:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f54102940 0x7f9f541990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f54102940 0x7f9f541990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35976/0 (socket says 172.21.15.84:35976) 2024-04-26T18:33:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f52ffd640 1 -- 172.21.15.84:0/279628165 learned_addr learned my addr 172.21.15.84:0/279628165 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f527fc640 1 --2- 172.21.15.84:0/279628165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f54103300 0x7f9f54199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f52ffd640 1 -- 172.21.15.84:0/279628165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f54103300 msgr2=0x7f9f54199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f52ffd640 1 --2- 172.21.15.84:0/279628165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f54103300 0x7f9f54199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f52ffd640 1 -- 172.21.15.84:0/279628165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54108950 msgr2=0x7f9f5419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f52ffd640 1 --2- 172.21.15.84:0/279628165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54108950 0x7f9f5419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f52ffd640 1 -- 172.21.15.84:0/279628165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f5419e0c0 con 0x7f9f54102940 2024-04-26T18:33:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f52ffd640 1 --2- 172.21.15.84:0/279628165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f54102940 0x7f9f541990d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c02ae00 tx=0x7f9f4c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:02.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f3bfff640 1 -- 172.21.15.84:0/279628165 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c004240 con 0x7f9f54102940 2024-04-26T18:33:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f3bfff640 1 -- 172.21.15.84:0/279628165 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4c0305a0 con 0x7f9f54102940 2024-04-26T18:33:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f59469640 1 -- 172.21.15.84:0/279628165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f5419e350 con 0x7f9f54102940 2024-04-26T18:33:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f3bfff640 1 -- 172.21.15.84:0/279628165 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c03b640 con 0x7f9f54102940 2024-04-26T18:33:02.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.473+0000 7f9f59469640 1 -- 172.21.15.84:0/279628165 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f541a5c40 con 0x7f9f54102940 2024-04-26T18:33:02.484 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.475+0000 7f9f3bfff640 1 -- 172.21.15.84:0/279628165 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9f4c030a50 con 0x7f9f54102940 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f5c178740 1 Processor -- start 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f5c178740 1 -- start start 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f3bfff640 1 --2- 172.21.15.84:0/279628165 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9f3c0a3dc0 0x7f9f3c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f5c178740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923eef00 0x55aa923ef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f5c178740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923ef820 0x55aa923f9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f537fe640 1 --2- 172.21.15.84:0/279628165 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9f3c0a3dc0 0x7f9f3c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f3bfff640 1 -- 172.21.15.84:0/279628165 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f9f4c02e030 con 0x7f9f54102940 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f5c178740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 0x55aa923fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f5c178740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55aa92355460 con 0x55aa923fa0e0 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f5c178740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aa9230f590 con 0x55aa923ef820 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f5c178740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aa92351420 con 0x55aa923eef00 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 0x55aa923fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 0x55aa923fc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35986/0 (socket says 172.21.15.84:35986) 2024-04-26T18:33:02.485 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f537fe640 1 -- 172.21.15.84:0/1385381000 learned_addr learned my addr 172.21.15.84:0/1385381000 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f527fc640 1 --2- 172.21.15.84:0/1385381000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923ef820 0x55aa923f9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f52ffd640 1 --2- 172.21.15.84:0/1385381000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923eef00 0x55aa923ef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f537fe640 1 -- 172.21.15.84:0/1385381000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923eef00 msgr2=0x55aa923ef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f537fe640 1 --2- 172.21.15.84:0/1385381000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923eef00 0x55aa923ef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f537fe640 1 -- 172.21.15.84:0/1385381000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923ef820 msgr2=0x55aa923f9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f537fe640 1 --2- 172.21.15.84:0/1385381000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923ef820 0x55aa923f9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f537fe640 1 -- 172.21.15.84:0/1385381000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa9231c0d0 con 0x55aa923fa0e0 2024-04-26T18:33:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f527fc640 1 --2- 172.21.15.84:0/1385381000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923ef820 0x55aa923f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:33:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.476+0000 7f9f537fe640 1 --2- 172.21.15.84:0/279628165 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9f3c0a3dc0 0x7f9f3c0a6280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9f44013740 tx=0x7f9f4400c910 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.477+0000 7f9f537fe640 1 --2- 172.21.15.84:0/1385381000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 0x55aa923fc4d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9f44013fd0 tx=0x7f9f44018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ba842d618fe6e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.477+0000 7f9f397fa640 1 -- 172.21.15.84:0/1385381000 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f44011db0 con 0x55aa923fa0e0 2024-04-26T18:33:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.477+0000 7f9f397fa640 1 -- 172.21.15.84:0/1385381000 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4400e6f0 con 0x55aa923fa0e0 2024-04-26T18:33:02.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.477+0000 7f9f397fa640 1 -- 172.21.15.84:0/1385381000 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f440035f0 con 0x55aa923fa0e0 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.477+0000 7f9f5c178740 1 -- 172.21.15.84:0/1385381000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 msgr2=0x55aa923fc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.477+0000 7f9f5c178740 1 --2- 172.21.15.84:0/1385381000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 0x55aa923fc4d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f9f44013fd0 tx=0x7f9f44018000 comp rx=0 tx=0).stop 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.477+0000 7f9f5c178740 1 -- 172.21.15.84:0/1385381000 shutdown_connections 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.477+0000 7f9f5c178740 1 --2- 172.21.15.84:0/1385381000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 0x55aa923fc4d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.477+0000 7f9f5c178740 1 --2- 172.21.15.84:0/1385381000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923ef820 0x55aa923f9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.477+0000 7f9f5c178740 1 --2- 172.21.15.84:0/1385381000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923eef00 0x55aa923ef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.477+0000 7f9f5c178740 1 -- 172.21.15.84:0/1385381000 >> 172.21.15.84:0/1385381000 conn(0x55aa923eb410 msgr2=0x55aa923eb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.477+0000 7f9f5c178740 1 -- 172.21.15.84:0/1385381000 shutdown_connections 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.477+0000 7f9f5c178740 1 -- 172.21.15.84:0/1385381000 wait complete. 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f5c178740 1 -- start start 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f5c178740 1 Processor -- start 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f5c178740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 0x55aa923f7110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f5c178740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923ef820 0x55aa923f7650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f5c178740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923fa0e0 0x55aa923f7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f5c178740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55aa923fccb0 con 0x55aa923eef00 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f5c178740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55aa922dcb60 con 0x55aa923ef820 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f5c178740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55aa923fce30 con 0x55aa923fa0e0 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f5c178740 10 client.0 _ll_register_callbacks cb 0x55aa923012b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 0x55aa923f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 0x55aa923f7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36000/0 (socket says 172.21.15.84:36000) 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f52ffd640 1 -- 172.21.15.84:0/2293027061 learned_addr learned my addr 172.21.15.84:0/2293027061 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f527fc640 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923ef820 0x55aa923f7650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f537fe640 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923fa0e0 0x55aa923f7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f52ffd640 1 -- 172.21.15.84:0/2293027061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923fa0e0 msgr2=0x55aa923f7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f52ffd640 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923fa0e0 0x55aa923f7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f52ffd640 1 -- 172.21.15.84:0/2293027061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923ef820 msgr2=0x55aa923f7650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f52ffd640 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923ef820 0x55aa923f7650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f52ffd640 1 -- 172.21.15.84:0/2293027061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa92067210 con 0x55aa923eef00 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f537fe640 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923fa0e0 0x55aa923f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f52ffd640 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 0x55aa923f7110 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0ef680 tx=0x7f9f4c0bd620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7078aefd44700964 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.478+0000 7f9f36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f36ff5640 1 -- 172.21.15.84:0/2293027061 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0c1d90 con 0x55aa923eef00 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f36ff5640 1 -- 172.21.15.84:0/2293027061 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f9f4c0ba050 con 0x55aa923eef00 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f5c178740 1 -- 172.21.15.84:0/2293027061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4400e6f0 con 0x55aa923eef00 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f36ff5640 1 -- 172.21.15.84:0/2293027061 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c0c6650 con 0x55aa923eef00 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f5c178740 10 client.16053 fetch_fsmap learned FSMap version 29 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f5c178740 1 -- 172.21.15.84:0/2293027061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa92133620 con 0x55aa923eef00 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f36ff5640 1 -- 172.21.15.84:0/2293027061 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4400e6f0 con 0x55aa923eef00 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f36ff5640 1 -- 172.21.15.84:0/2293027061 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9f4c04d020 con 0x55aa923eef00 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f5c178740 10 client.16053 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f5c178740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f5c178740 20 client.16053 populate_metadata read hostname 'smithi084' 2024-04-26T18:33:02.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f5c178740 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55aa9240a290 0x55aa9240c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f5c178740 4 client.16053 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:33:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f5c178740 1 -- 172.21.15.84:0/2293027061 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aa92363070 con 0x55aa9240a290 2024-04-26T18:33:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.479+0000 7f9f527fc640 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55aa9240a290 0x55aa9240c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.480+0000 7f9f36ff5640 10 client.16053 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.480+0000 7f9f527fc640 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55aa9240a290 0x55aa9240c6d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329095c11a6dc3f4 server_cookie=c87ade48534676f1 in_seq=0 out_seq=0 2024-04-26T18:33:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.480+0000 7f9f36ff5640 1 -- 172.21.15.84:0/2293027061 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55aa92363070 con 0x55aa9240a290 2024-04-26T18:33:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.480+0000 7f9f36ff5640 10 client.16053 handle_command_reply: tid=0 2024-04-26T18:33:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.481+0000 7f9f5c178740 1 client.16053 shutdown 2024-04-26T18:33:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.481+0000 7f9f5c178740 1 -- 172.21.15.84:0/2293027061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 msgr2=0x55aa923f7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.481+0000 7f9f5c178740 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 0x55aa923f7110 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0ef680 tx=0x7f9f4c0bd620 comp rx=0 tx=0).stop 2024-04-26T18:33:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.481+0000 7f9f5c178740 1 -- 172.21.15.84:0/2293027061 shutdown_connections 2024-04-26T18:33:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.481+0000 7f9f5c178740 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55aa9240a290 0x55aa9240c6d0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.481+0000 7f9f5c178740 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923fa0e0 0x55aa923f7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.481+0000 7f9f5c178740 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923ef820 0x55aa923f7650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.481+0000 7f9f36ff5640 0 client.16053 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.481+0000 7f9f5c178740 1 --2- 172.21.15.84:0/2293027061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 0x55aa923f7110 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.481+0000 7f9f5c178740 1 -- 172.21.15.84:0/2293027061 >> 172.21.15.84:0/2293027061 conn(0x55aa923eb410 msgr2=0x55aa923f1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.481+0000 7f9f5c178740 1 -- 172.21.15.84:0/2293027061 shutdown_connections 2024-04-26T18:33:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.481+0000 7f9f5c178740 1 -- 172.21.15.84:0/2293027061 wait complete. 2024-04-26T18:33:02.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.481+0000 7f9f5c178740 20 client.16053 trim_cache size 0 max 16384 2024-04-26T18:33:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.495+0000 7f9f5c178740 1 Processor -- start 2024-04-26T18:33:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.495+0000 7f9f5c178740 1 -- start start 2024-04-26T18:33:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.495+0000 7f9f5c178740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923eef00 0x55aa923f79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f5c178740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923ef820 0x55aa923ea430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f5c178740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 0x55aa92411fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f5c178740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541ae4f0 con 0x55aa923fa0e0 2024-04-26T18:33:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f5c178740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541b0030 con 0x55aa923eef00 2024-04-26T18:33:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f5c178740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541aebc0 con 0x55aa923ef820 2024-04-26T18:33:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 0x55aa92411fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.501 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 0x55aa92411fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36012/0 (socket says 172.21.15.84:36012) 2024-04-26T18:33:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f537fe640 1 -- 172.21.15.84:0/2065920388 learned_addr learned my addr 172.21.15.84:0/2065920388 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f537fe640 1 -- 172.21.15.84:0/2065920388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923ef820 msgr2=0x55aa923ea430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f537fe640 1 --2- 172.21.15.84:0/2065920388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923ef820 0x55aa923ea430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f52ffd640 1 --2- 172.21.15.84:0/2065920388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923eef00 0x55aa923f79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f537fe640 1 -- 172.21.15.84:0/2065920388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923eef00 msgr2=0x55aa923f79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f537fe640 1 --2- 172.21.15.84:0/2065920388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923eef00 0x55aa923f79b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f537fe640 1 -- 172.21.15.84:0/2065920388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa923e6b20 con 0x55aa923fa0e0 2024-04-26T18:33:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f52ffd640 1 --2- 172.21.15.84:0/2065920388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923eef00 0x55aa923f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f537fe640 1 --2- 172.21.15.84:0/2065920388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 0x55aa92411fe0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9f44005fd0 tx=0x7f9f44000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51808b2c56621f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f38ff9640 1 -- 172.21.15.84:0/2065920388 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4402d070 con 0x55aa923fa0e0 2024-04-26T18:33:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f38ff9640 1 -- 172.21.15.84:0/2065920388 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4400cbe0 con 0x55aa923fa0e0 2024-04-26T18:33:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.496+0000 7f9f38ff9640 1 -- 172.21.15.84:0/2065920388 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4400ee40 con 0x55aa923fa0e0 2024-04-26T18:33:02.502 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 -- 172.21.15.84:0/2065920388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 msgr2=0x55aa92411fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 --2- 172.21.15.84:0/2065920388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 0x55aa92411fe0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9f44005fd0 tx=0x7f9f44000bc0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 -- 172.21.15.84:0/2065920388 shutdown_connections 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 --2- 172.21.15.84:0/2065920388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923fa0e0 0x55aa92411fe0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 --2- 172.21.15.84:0/2065920388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923ef820 0x55aa923ea430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 --2- 172.21.15.84:0/2065920388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923eef00 0x55aa923f79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 -- 172.21.15.84:0/2065920388 >> 172.21.15.84:0/2065920388 conn(0x55aa923eb410 msgr2=0x55aa923e9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 -- 172.21.15.84:0/2065920388 shutdown_connections 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 -- 172.21.15.84:0/2065920388 wait complete. 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 -- start start 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 Processor -- start 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 0x55aa923fe220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923ef820 0x55aa923fe760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923fa0e0 0x55aa9246f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541af960 con 0x55aa923eef00 2024-04-26T18:33:02.503 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541af290 con 0x55aa923fa0e0 2024-04-26T18:33:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9f541ac4d0 con 0x55aa923ef820 2024-04-26T18:33:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.497+0000 7f9f5c178740 10 client.0 _ll_register_callbacks cb 0x7f9f4c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 0x55aa923fe220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f52ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 0x55aa923fe220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36020/0 (socket says 172.21.15.84:36020) 2024-04-26T18:33:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f52ffd640 1 -- 172.21.15.84:0/1821012101 learned_addr learned my addr 172.21.15.84:0/1821012101 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f52ffd640 1 -- 172.21.15.84:0/1821012101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923ef820 msgr2=0x55aa923fe760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f52ffd640 1 --2- 172.21.15.84:0/1821012101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923ef820 0x55aa923fe760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f52ffd640 1 -- 172.21.15.84:0/1821012101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923fa0e0 msgr2=0x55aa9246f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f537fe640 1 --2- 172.21.15.84:0/1821012101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923fa0e0 0x55aa9246f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f52ffd640 1 --2- 172.21.15.84:0/1821012101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923fa0e0 0x55aa9246f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f52ffd640 1 -- 172.21.15.84:0/1821012101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa9246fde0 con 0x55aa923eef00 2024-04-26T18:33:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f537fe640 1 --2- 172.21.15.84:0/1821012101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923fa0e0 0x55aa9246f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f52ffd640 1 --2- 172.21.15.84:0/1821012101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 0x55aa923fe220 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0de010 tx=0x7f9f4c0bbef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ef2428b6a3d431 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f36ff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f36ff5640 1 -- 172.21.15.84:0/1821012101 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f4c0dcda0 con 0x55aa923eef00 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f36ff5640 1 -- 172.21.15.84:0/1821012101 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f9f4c0882b0 con 0x55aa923eef00 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f5c178740 1 -- 172.21.15.84:0/1821012101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f4400cbe0 con 0x55aa923eef00 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f36ff5640 1 -- 172.21.15.84:0/1821012101 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9f4c0bd7b0 con 0x55aa923eef00 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f5c178740 10 client.16059 fetch_fsmap learned FSMap version 29 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f5c178740 1 -- 172.21.15.84:0/1821012101 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55aa92471060 con 0x55aa923eef00 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f36ff5640 1 -- 172.21.15.84:0/1821012101 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f4400cbe0 con 0x55aa923eef00 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.498+0000 7f9f36ff5640 1 -- 172.21.15.84:0/1821012101 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9f4c04d050 con 0x55aa923eef00 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.499+0000 7f9f5c178740 10 client.16059 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.499+0000 7f9f5c178740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.499+0000 7f9f5c178740 20 client.16059 populate_metadata read hostname 'smithi084' 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.499+0000 7f9f5c178740 1 --2- 172.21.15.84:0/1821012101 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55aa92471570 0x55aa924739d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.499+0000 7f9f5c178740 4 client.16059 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:33:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.499+0000 7f9f5c178740 1 -- 172.21.15.84:0/1821012101 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aa9235cee0 con 0x55aa92471570 2024-04-26T18:33:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.499+0000 7f9f527fc640 1 --2- 172.21.15.84:0/1821012101 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55aa92471570 0x55aa924739d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.499+0000 7f9f527fc640 1 --2- 172.21.15.84:0/1821012101 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55aa92471570 0x55aa924739d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f51048ba2bf042ec server_cookie=5e09005fc9cca2ce in_seq=0 out_seq=0 2024-04-26T18:33:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.499+0000 7f9f36ff5640 10 client.16059 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f36ff5640 1 -- 172.21.15.84:0/1821012101 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aa9235cee0 con 0x55aa92471570 2024-04-26T18:33:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f36ff5640 10 client.16059 handle_command_reply: tid=0 2024-04-26T18:33:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f5c178740 1 client.16059 shutdown 2024-04-26T18:33:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f5c178740 1 -- 172.21.15.84:0/1821012101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 msgr2=0x55aa923fe220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f5c178740 1 --2- 172.21.15.84:0/1821012101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 0x55aa923fe220 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c0de010 tx=0x7f9f4c0bbef0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f5c178740 1 -- 172.21.15.84:0/1821012101 shutdown_connections 2024-04-26T18:33:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f5c178740 1 --2- 172.21.15.84:0/1821012101 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55aa92471570 0x55aa924739d0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f5c178740 1 --2- 172.21.15.84:0/1821012101 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55aa923fa0e0 0x55aa9246f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f36ff5640 0 client.16059 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f5c178740 1 --2- 172.21.15.84:0/1821012101 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55aa923ef820 0x55aa923fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f5c178740 1 --2- 172.21.15.84:0/1821012101 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55aa923eef00 0x55aa923fe220 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f5c178740 1 -- 172.21.15.84:0/1821012101 >> 172.21.15.84:0/1821012101 conn(0x55aa923eb410 msgr2=0x55aa923ea5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f5c178740 1 -- 172.21.15.84:0/1821012101 shutdown_connections 2024-04-26T18:33:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f5c178740 1 -- 172.21.15.84:0/1821012101 wait complete. 2024-04-26T18:33:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.508+0000 7f9f5c178740 20 client.16059 trim_cache size 0 max 16384 2024-04-26T18:33:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:33:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:33:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:33:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:33:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.509+0000 7f9f59469640 1 -- 172.21.15.84:0/279628165 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9f3c0a3dc0 msgr2=0x7f9f3c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.509+0000 7f9f59469640 1 --2- 172.21.15.84:0/279628165 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9f3c0a3dc0 0x7f9f3c0a6280 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9f44013740 tx=0x7f9f4400c910 comp rx=0 tx=0).stop 2024-04-26T18:33:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.509+0000 7f9f59469640 1 -- 172.21.15.84:0/279628165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f54102940 msgr2=0x7f9f541990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.509+0000 7f9f59469640 1 --2- 172.21.15.84:0/279628165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f54102940 0x7f9f541990d0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c02ae00 tx=0x7f9f4c003ba0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.509+0000 7f9f59469640 1 -- 172.21.15.84:0/279628165 shutdown_connections 2024-04-26T18:33:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.509+0000 7f9f59469640 1 --2- 172.21.15.84:0/279628165 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9f3c0a3dc0 0x7f9f3c0a6280 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.509+0000 7f9f59469640 1 --2- 172.21.15.84:0/279628165 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9f54108950 0x7f9f5419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.509+0000 7f9f59469640 1 --2- 172.21.15.84:0/279628165 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9f54103300 0x7f9f54199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.509+0000 7f9f59469640 1 --2- 172.21.15.84:0/279628165 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9f54102940 0x7f9f541990d0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.509+0000 7f9f59469640 1 -- 172.21.15.84:0/279628165 >> 172.21.15.84:0/279628165 conn(0x7f9f54079dc0 msgr2=0x7f9f541002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.509+0000 7f9f59469640 1 -- 172.21.15.84:0/279628165 shutdown_connections 2024-04-26T18:33:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:02.509+0000 7f9f59469640 1 -- 172.21.15.84:0/279628165 wait complete. 2024-04-26T18:33:02.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:02 smithi174 ceph-mon[32705]: pgmap v155: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-04-26T18:33:03.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:02 smithi084 ceph-mon[29141]: pgmap v155: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-04-26T18:33:03.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:02 smithi103 ceph-mon[32727]: pgmap v155: 97 pgs: 97 active+clean; 4.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 3.9 KiB/s wr, 11 op/s 2024-04-26T18:33:04.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:04 smithi174 ceph-mon[32705]: pgmap v156: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 KiB/s wr, 31 op/s 2024-04-26T18:33:05.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:04 smithi084 ceph-mon[29141]: pgmap v156: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 KiB/s wr, 31 op/s 2024-04-26T18:33:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:04 smithi103 ceph-mon[32727]: pgmap v156: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 KiB/s wr, 31 op/s 2024-04-26T18:33:05.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:05 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:33:06.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:05 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:33:06.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:05 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:33:06.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:06 smithi174 ceph-mon[32705]: pgmap v157: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-26T18:33:06.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:06 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:33:06.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:06 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:33:06.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:06 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:33:07.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:06 smithi084 ceph-mon[29141]: pgmap v157: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-26T18:33:07.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:06 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:33:07.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:06 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:33:07.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:06 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:33:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:06 smithi103 ceph-mon[32727]: pgmap v157: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-26T18:33:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:06 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:33:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:06 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:33:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:06 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:33:08.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:08 smithi174 ceph-mon[32705]: pgmap v158: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-26T18:33:09.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:08 smithi084 ceph-mon[29141]: pgmap v158: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-26T18:33:09.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:08 smithi103 ceph-mon[32727]: pgmap v158: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-26T18:33:09.405 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:33:09.656 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:33:10.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:10 smithi174 ceph-mon[32705]: pgmap v159: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-04-26T18:33:10.978 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:10 smithi084 ceph-mon[29141]: pgmap v159: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-04-26T18:33:11.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:10 smithi103 ceph-mon[32727]: pgmap v159: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-04-26T18:33:11.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.507+0000 7f79d2a0c700 1 -- 172.21.15.84:0/2642243346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79cc1094c0 msgr2=0x7f79cc1098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:11.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.507+0000 7f79d2a0c700 1 --2- 172.21.15.84:0/2642243346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79cc1094c0 0x7f79cc1098a0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009990 tx=0x7f79bc009ca0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.507+0000 7f79d2a0c700 1 -- 172.21.15.84:0/2642243346 shutdown_connections 2024-04-26T18:33:11.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.507+0000 7f79d2a0c700 1 --2- 172.21.15.84:0/2642243346 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79cc1094c0 0x7f79cc1098a0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.507+0000 7f79d2a0c700 1 --2- 172.21.15.84:0/2642243346 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79cc103e30 0x7f79cc110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.507+0000 7f79d2a0c700 1 --2- 172.21.15.84:0/2642243346 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79cc103470 0x7f79cc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.507+0000 7f79d2a0c700 1 -- 172.21.15.84:0/2642243346 >> 172.21.15.84:0/2642243346 conn(0x7f79cc07a580 msgr2=0x7f79cc07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:11.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.507+0000 7f79d2a0c700 1 -- 172.21.15.84:0/2642243346 shutdown_connections 2024-04-26T18:33:11.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.507+0000 7f79d2a0c700 1 -- 172.21.15.84:0/2642243346 wait complete. 2024-04-26T18:33:11.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.507+0000 7f79d2a0c700 1 Processor -- start 2024-04-26T18:33:11.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.507+0000 7f79d2a0c700 1 -- start start 2024-04-26T18:33:11.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79d2a0c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79cc103470 0x7f79cc19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:11.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79d2a0c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79cc103e30 0x7f79cc19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:11.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79d2a0c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79cc1094c0 0x7f79cc1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:11.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79d2a0c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc113640 con 0x7f79cc103e30 2024-04-26T18:33:11.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79d2a0c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1134c0 con 0x7f79cc103470 2024-04-26T18:33:11.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79d2a0c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f79cc1137c0 con 0x7f79cc1094c0 2024-04-26T18:33:11.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79cb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79cc103e30 0x7f79cc19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:11.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79cb7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79cc103e30 0x7f79cc19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38702/0 (socket says 172.21.15.84:38702) 2024-04-26T18:33:11.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79cb7fe700 1 -- 172.21.15.84:0/3232385297 learned_addr learned my addr 172.21.15.84:0/3232385297 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79d0fa9700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79cc1094c0 0x7f79cc1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79cbfff700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79cc103470 0x7f79cc19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79cb7fe700 1 -- 172.21.15.84:0/3232385297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79cc1094c0 msgr2=0x7f79cc1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79cb7fe700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79cc1094c0 0x7f79cc1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79cb7fe700 1 -- 172.21.15.84:0/3232385297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79cc103470 msgr2=0x7f79cc19bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79cb7fe700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79cc103470 0x7f79cc19bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79cb7fe700 1 -- 172.21.15.84:0/3232385297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79cc1a0d80 con 0x7f79cc103e30 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79cb7fe700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79cc103e30 0x7f79cc19c2b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f79c000d950 tx=0x7f79c000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79c97fa700 1 -- 172.21.15.84:0/3232385297 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c0016070 con 0x7f79cc103e30 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79c97fa700 1 -- 172.21.15.84:0/3232385297 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79c0009df0 con 0x7f79cc103e30 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79d2a0c700 1 -- 172.21.15.84:0/3232385297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79cc1a1070 con 0x7f79cc103e30 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79c97fa700 1 -- 172.21.15.84:0/3232385297 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c000f3c0 con 0x7f79cc103e30 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.508+0000 7f79d2a0c700 1 -- 172.21.15.84:0/3232385297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79cc1a88f0 con 0x7f79cc103e30 2024-04-26T18:33:11.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.509+0000 7f79d2a0c700 1 -- 172.21.15.84:0/3232385297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79cc06a110 con 0x7f79cc103e30 2024-04-26T18:33:11.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.510+0000 7f79c97fa700 1 -- 172.21.15.84:0/3232385297 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f79c0019070 con 0x7f79cc103e30 2024-04-26T18:33:11.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.511+0000 7f79c97fa700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f79b40a3db0 0x7f79b40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:11.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.511+0000 7f79cbfff700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f79b40a3db0 0x7f79b40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:11.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.511+0000 7f79c97fa700 1 -- 172.21.15.84:0/3232385297 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f79c00d0410 con 0x7f79cc103e30 2024-04-26T18:33:11.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.512+0000 7f79cbfff700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f79b40a3db0 0x7f79b40a6270 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79bc005cd0 tx=0x7f79bc005c40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:11.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.515+0000 7f79c97fa700 1 -- 172.21.15.84:0/3232385297 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f79c009caa0 con 0x7f79cc103e30 2024-04-26T18:33:11.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.768+0000 7f79d2a0c700 1 -- 172.21.15.84:0/3232385297 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79cc02e490 con 0x7f79cc103e30 2024-04-26T18:33:11.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.768+0000 7f79c97fa700 1 -- 172.21.15.84:0/3232385297 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f79c009c3a0 con 0x7f79cc103e30 2024-04-26T18:33:11.768 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:33:11.769 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:33:11.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.770+0000 7f79d2a0c700 1 -- 172.21.15.84:0/3232385297 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f79b40a3db0 msgr2=0x7f79b40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:11.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.770+0000 7f79d2a0c700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f79b40a3db0 0x7f79b40a6270 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f79bc005cd0 tx=0x7f79bc005c40 comp rx=0 tx=0).stop 2024-04-26T18:33:11.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.770+0000 7f79d2a0c700 1 -- 172.21.15.84:0/3232385297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79cc103e30 msgr2=0x7f79cc19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:11.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.770+0000 7f79d2a0c700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79cc103e30 0x7f79cc19c2b0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f79c000d950 tx=0x7f79c000dc60 comp rx=0 tx=0).stop 2024-04-26T18:33:11.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.771+0000 7f79d2a0c700 1 -- 172.21.15.84:0/3232385297 shutdown_connections 2024-04-26T18:33:11.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.771+0000 7f79d2a0c700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f79b40a3db0 0x7f79b40a6270 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.771+0000 7f79d2a0c700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f79cc1094c0 0x7f79cc1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.771+0000 7f79d2a0c700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f79cc103470 0x7f79cc19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.771+0000 7f79d2a0c700 1 --2- 172.21.15.84:0/3232385297 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f79cc103e30 0x7f79cc19c2b0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:11.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.771+0000 7f79d2a0c700 1 -- 172.21.15.84:0/3232385297 >> 172.21.15.84:0/3232385297 conn(0x7f79cc07a580 msgr2=0x7f79cc10e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:11.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.771+0000 7f79d2a0c700 1 -- 172.21.15.84:0/3232385297 shutdown_connections 2024-04-26T18:33:11.771 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:11.771+0000 7f79d2a0c700 1 -- 172.21.15.84:0/3232385297 wait complete. 2024-04-26T18:33:11.772 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:33:12.571 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id f0da206a --release --await' 2024-04-26T18:33:12.571 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db --set-id f0da206a --release --await 2024-04-26T18:33:12.811 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:33:12.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:12 smithi084 ceph-mon[29141]: pgmap v160: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-26T18:33:12.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:12 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3232385297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:33:12.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:12 smithi174 ceph-mon[32705]: pgmap v160: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-26T18:33:12.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:12 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3232385297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:33:13.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:12 smithi103 ceph-mon[32727]: pgmap v160: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 24 op/s 2024-04-26T18:33:13.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:12 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3232385297' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:33:14.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.465+0000 7fd0d3b1e700 1 -- 172.21.15.84:0/994039908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0cc103e40 msgr2=0x7fd0cc110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:14.467 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.465+0000 7fd0d3b1e700 1 --2- 172.21.15.84:0/994039908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0cc103e40 0x7fd0cc110320 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009990 tx=0x7fd0bc009ca0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.465+0000 7fd0d3b1e700 1 -- 172.21.15.84:0/994039908 shutdown_connections 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.465+0000 7fd0d3b1e700 1 --2- 172.21.15.84:0/994039908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0cc103e40 0x7fd0cc110320 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.465+0000 7fd0d3b1e700 1 --2- 172.21.15.84:0/994039908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0cc1094e0 0x7fd0cc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.465+0000 7fd0d3b1e700 1 --2- 172.21.15.84:0/994039908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0cc1034a0 0x7fd0cc103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.465+0000 7fd0d3b1e700 1 -- 172.21.15.84:0/994039908 >> 172.21.15.84:0/994039908 conn(0x7fd0cc07a4f0 msgr2=0x7fd0cc07a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.465+0000 7fd0d3b1e700 1 -- 172.21.15.84:0/994039908 shutdown_connections 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.465+0000 7fd0d3b1e700 1 -- 172.21.15.84:0/994039908 wait complete. 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d3b1e700 1 Processor -- start 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d3b1e700 1 -- start start 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d3b1e700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0cc1034a0 0x7fd0cc19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d3b1e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0cc103e40 0x7fd0cc19fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d3b1e700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0cc1094e0 0x7fd0cc1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d3b1e700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc113690 con 0x7fd0cc103e40 2024-04-26T18:33:14.468 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d3b1e700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc113510 con 0x7fd0cc1094e0 2024-04-26T18:33:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d3b1e700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc113810 con 0x7fd0cc1034a0 2024-04-26T18:33:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d10b9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0cc103e40 0x7fd0cc19fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d10b9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0cc103e40 0x7fd0cc19fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38718/0 (socket says 172.21.15.84:38718) 2024-04-26T18:33:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d10b9700 1 -- 172.21.15.84:0/3585919591 learned_addr learned my addr 172.21.15.84:0/3585919591 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d18ba700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0cc1034a0 0x7fd0cc19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d20bb700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0cc1094e0 0x7fd0cc1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d10b9700 1 -- 172.21.15.84:0/3585919591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0cc1034a0 msgr2=0x7fd0cc19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d10b9700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0cc1034a0 0x7fd0cc19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d10b9700 1 -- 172.21.15.84:0/3585919591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0cc1094e0 msgr2=0x7fd0cc1998b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d10b9700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0cc1094e0 0x7fd0cc1998b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.466+0000 7fd0d10b9700 1 -- 172.21.15.84:0/3585919591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0cc19a010 con 0x7fd0cc103e40 2024-04-26T18:33:14.469 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.467+0000 7fd0d10b9700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0cc103e40 0x7fd0cc19fc30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000b530 tx=0x7fd0c000b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:14.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.467+0000 7fd0caffd700 1 -- 172.21.15.84:0/3585919591 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0015020 con 0x7fd0cc103e40 2024-04-26T18:33:14.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.467+0000 7fd0d3b1e700 1 -- 172.21.15.84:0/3585919591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0cc19a300 con 0x7fd0cc103e40 2024-04-26T18:33:14.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.467+0000 7fd0caffd700 1 -- 172.21.15.84:0/3585919591 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c0011ce0 con 0x7fd0cc103e40 2024-04-26T18:33:14.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.467+0000 7fd0caffd700 1 -- 172.21.15.84:0/3585919591 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c0010d20 con 0x7fd0cc103e40 2024-04-26T18:33:14.470 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.467+0000 7fd0d3b1e700 1 -- 172.21.15.84:0/3585919591 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0cc1a65c0 con 0x7fd0cc103e40 2024-04-26T18:33:14.472 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.469+0000 7fd0caffd700 1 -- 172.21.15.84:0/3585919591 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fd0c0018020 con 0x7fd0cc103e40 2024-04-26T18:33:14.472 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.470+0000 7fd0caffd700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd0b40a3db0 0x7fd0b40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.472 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.470+0000 7fd0d18ba700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd0b40a3db0 0x7fd0b40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.472 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.470+0000 7fd0caffd700 1 -- 172.21.15.84:0/3585919591 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fd0c00cff70 con 0x7fd0cc103e40 2024-04-26T18:33:14.472 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.470+0000 7fd0d18ba700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd0b40a3db0 0x7fd0b40a6270 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc19a9f0 tx=0x7fd0c4009040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:14.472 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.471+0000 7fd0de3b0b80 1 Processor -- start 2024-04-26T18:33:14.472 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.471+0000 7fd0de3b0b80 1 -- start start 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.471+0000 7fd0de3b0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916ee1a0 0x5651916ee580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.471+0000 7fd0de3b0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916eeac0 0x5651916f9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.471+0000 7fd0de3b0b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x5651916fbd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.471+0000 7fd0de3b0b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x565191641840 con 0x5651916f9970 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.471+0000 7fd0de3b0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5651915e06e0 con 0x5651916ee1a0 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.471+0000 7fd0de3b0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56519163d800 con 0x5651916eeac0 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.471+0000 7fd0d20bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x5651916fbd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.471+0000 7fd0d20bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x5651916fbd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38730/0 (socket says 172.21.15.84:38730) 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.471+0000 7fd0d20bb700 1 -- 172.21.15.84:0/1705973835 learned_addr learned my addr 172.21.15.84:0/1705973835 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.471+0000 7fd0d18ba700 1 --2- 172.21.15.84:0/1705973835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916ee1a0 0x5651916ee580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.472+0000 7fd0d10b9700 1 --2- 172.21.15.84:0/1705973835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916eeac0 0x5651916f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.472+0000 7fd0d20bb700 1 -- 172.21.15.84:0/1705973835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916eeac0 msgr2=0x5651916f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.472+0000 7fd0d20bb700 1 --2- 172.21.15.84:0/1705973835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916eeac0 0x5651916f9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.472+0000 7fd0d20bb700 1 -- 172.21.15.84:0/1705973835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916ee1a0 msgr2=0x5651916ee580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:14.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.472+0000 7fd0d20bb700 1 --2- 172.21.15.84:0/1705973835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916ee1a0 0x5651916ee580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.472+0000 7fd0d20bb700 1 -- 172.21.15.84:0/1705973835 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56519158e9d0 con 0x5651916f9970 2024-04-26T18:33:14.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.472+0000 7fd0d20bb700 1 --2- 172.21.15.84:0/1705973835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x5651916fbd60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009fc0 tx=0x7fd0bc005bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc76bdd7fd7bdb11 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:14.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.472+0000 7fd0b3fff700 1 -- 172.21.15.84:0/1705973835 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc027070 con 0x5651916f9970 2024-04-26T18:33:14.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.472+0000 7fd0b3fff700 1 -- 172.21.15.84:0/1705973835 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0bc005070 con 0x5651916f9970 2024-04-26T18:33:14.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.472+0000 7fd0b3fff700 1 -- 172.21.15.84:0/1705973835 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc00bbd0 con 0x5651916f9970 2024-04-26T18:33:14.478 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.473+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/1705973835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 msgr2=0x5651916fbd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.473+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/1705973835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x5651916fbd60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009fc0 tx=0x7fd0bc005bc0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.473+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/1705973835 shutdown_connections 2024-04-26T18:33:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.473+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/1705973835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x5651916fbd60 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.473+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/1705973835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916ee1a0 0x5651916ee580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.473+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/1705973835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916eeac0 0x5651916f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.473+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/1705973835 >> 172.21.15.84:0/1705973835 conn(0x56519161be40 msgr2=0x5651916ea690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.473+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/1705973835 shutdown_connections 2024-04-26T18:33:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.473+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/1705973835 wait complete. 2024-04-26T18:33:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.473+0000 7fd0de3b0b80 1 -- start start 2024-04-26T18:33:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.473+0000 7fd0de3b0b80 1 Processor -- start 2024-04-26T18:33:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0de3b0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916ee1a0 0x5651916f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0de3b0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916eeac0 0x5651916f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.479 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0de3b0b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x5651916f6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0de3b0b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5651916fc4e0 con 0x5651916f9970 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0de3b0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5651915a7a50 con 0x5651916ee1a0 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0de3b0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5651916fc660 con 0x5651916eeac0 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0de3b0b80 10 client.0 _ll_register_callbacks cb 0x5651915d5680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0d20bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x5651916f6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0d20bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x5651916f6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38738/0 (socket says 172.21.15.84:38738) 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0d20bb700 1 -- 172.21.15.84:0/920416360 learned_addr learned my addr 172.21.15.84:0/920416360 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0d18ba700 1 --2- 172.21.15.84:0/920416360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916ee1a0 0x5651916f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0d10b9700 1 --2- 172.21.15.84:0/920416360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916eeac0 0x5651916f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0d20bb700 1 -- 172.21.15.84:0/920416360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916eeac0 msgr2=0x5651916f63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0d20bb700 1 --2- 172.21.15.84:0/920416360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916eeac0 0x5651916f63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0d20bb700 1 -- 172.21.15.84:0/920416360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916ee1a0 msgr2=0x5651916f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0d20bb700 1 --2- 172.21.15.84:0/920416360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916ee1a0 0x5651916f90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.480 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0d20bb700 1 -- 172.21.15.84:0/920416360 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651916f6fe0 con 0x5651916f9970 2024-04-26T18:33:14.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0d20bb700 1 --2- 172.21.15.84:0/920416360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x5651916f6930 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009ce0 tx=0x7fd0bc0054b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69e64e90f6c89dc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:14.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.474+0000 7fd0b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:33:14.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0b17fa700 1 -- 172.21.15.84:0/920416360 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc027070 con 0x5651916f9970 2024-04-26T18:33:14.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0b17fa700 1 -- 172.21.15.84:0/920416360 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fd0bc003c70 con 0x5651916f9970 2024-04-26T18:33:14.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/920416360 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0bc005070 con 0x5651916f9970 2024-04-26T18:33:14.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0b17fa700 1 -- 172.21.15.84:0/920416360 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0bc039050 con 0x5651916f9970 2024-04-26T18:33:14.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0de3b0b80 10 client.16077 fetch_fsmap learned FSMap version 29 2024-04-26T18:33:14.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/920416360 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565191706f60 con 0x5651916f9970 2024-04-26T18:33:14.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0b17fa700 1 -- 172.21.15.84:0/920416360 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0bc005070 con 0x5651916f9970 2024-04-26T18:33:14.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0b17fa700 1 -- 172.21.15.84:0/920416360 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd0bc034030 con 0x5651916f9970 2024-04-26T18:33:14.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0de3b0b80 10 client.16077 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:33:14.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0de3b0b80 10 client.16077 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:33:14.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0de3b0b80 20 client.16077 populate_metadata read hostname 'smithi084' 2024-04-26T18:33:14.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/920416360 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x565191709430 0x56519170b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0de3b0b80 4 client.16077 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:33:14.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.475+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/920416360 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56519164f220 con 0x565191709430 2024-04-26T18:33:14.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.476+0000 7fd0d18ba700 1 --2- 172.21.15.84:0/920416360 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x565191709430 0x56519170b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.476+0000 7fd0d18ba700 1 --2- 172.21.15.84:0/920416360 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x565191709430 0x56519170b820 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=591ac94676bea3e0 server_cookie=4efedb33bfd80f03 in_seq=0 out_seq=0 2024-04-26T18:33:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.476+0000 7fd0b17fa700 10 client.16077 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0b17fa700 1 -- 172.21.15.84:0/920416360 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56519164f220 con 0x565191709430 2024-04-26T18:33:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0b17fa700 10 client.16077 handle_command_reply: tid=0 2024-04-26T18:33:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0de3b0b80 1 client.16077 shutdown 2024-04-26T18:33:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/920416360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 msgr2=0x5651916f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/920416360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x5651916f6930 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc009ce0 tx=0x7fd0bc0054b0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/920416360 shutdown_connections 2024-04-26T18:33:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/920416360 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x565191709430 0x56519170b820 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/920416360 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x5651916f6930 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/920416360 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916ee1a0 0x5651916f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0b17fa700 0 client.16077 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:14.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/920416360 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916eeac0 0x5651916f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/920416360 >> 172.21.15.84:0/920416360 conn(0x56519161be40 msgr2=0x5651916ea2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/920416360 shutdown_connections 2024-04-26T18:33:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/920416360 wait complete. 2024-04-26T18:33:14.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.477+0000 7fd0de3b0b80 20 client.16077 trim_cache size 0 max 16384 2024-04-26T18:33:14.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.496+0000 7fd0de3b0b80 1 Processor -- start 2024-04-26T18:33:14.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.496+0000 7fd0de3b0b80 1 -- start start 2024-04-26T18:33:14.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.496+0000 7fd0de3b0b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916ee1a0 0x5651916f6a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.496+0000 7fd0de3b0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916eeac0 0x5651916f1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.496+0000 7fd0de3b0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916f9970 0x5651916f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.496+0000 7fd0de3b0b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1adce0 con 0x5651916ee1a0 2024-04-26T18:33:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.496+0000 7fd0de3b0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1af820 con 0x5651916eeac0 2024-04-26T18:33:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.496+0000 7fd0de3b0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1ae3b0 con 0x5651916f9970 2024-04-26T18:33:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.496+0000 7fd0d18ba700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916ee1a0 0x5651916f6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.496+0000 7fd0d18ba700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916ee1a0 0x5651916f6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38744/0 (socket says 172.21.15.84:38744) 2024-04-26T18:33:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.496+0000 7fd0d18ba700 1 -- 172.21.15.84:0/4213668802 learned_addr learned my addr 172.21.15.84:0/4213668802 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.496+0000 7fd0d20bb700 1 --2- 172.21.15.84:0/4213668802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916f9970 0x5651916f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.497+0000 7fd0d10b9700 1 --2- 172.21.15.84:0/4213668802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916eeac0 0x5651916f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.497+0000 7fd0d18ba700 1 -- 172.21.15.84:0/4213668802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916f9970 msgr2=0x5651916f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:14.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.497+0000 7fd0d18ba700 1 --2- 172.21.15.84:0/4213668802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916f9970 0x5651916f8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.497+0000 7fd0d18ba700 1 -- 172.21.15.84:0/4213668802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916eeac0 msgr2=0x5651916f1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.497+0000 7fd0d18ba700 1 --2- 172.21.15.84:0/4213668802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916eeac0 0x5651916f1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.497+0000 7fd0d18ba700 1 -- 172.21.15.84:0/4213668802 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651916f63b0 con 0x5651916ee1a0 2024-04-26T18:33:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.497+0000 7fd0d10b9700 1 --2- 172.21.15.84:0/4213668802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916eeac0 0x5651916f1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.497+0000 7fd0d18ba700 1 --2- 172.21.15.84:0/4213668802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916ee1a0 0x5651916f6a80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4007740 tx=0x7fd0c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58a9f35498cdb50 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.497+0000 7fd0b37fe700 1 -- 172.21.15.84:0/4213668802 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c402ab50 con 0x5651916ee1a0 2024-04-26T18:33:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.497+0000 7fd0b37fe700 1 -- 172.21.15.84:0/4213668802 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c402acf0 con 0x5651916ee1a0 2024-04-26T18:33:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.497+0000 7fd0b37fe700 1 -- 172.21.15.84:0/4213668802 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c402a8b0 con 0x5651916ee1a0 2024-04-26T18:33:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/4213668802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916ee1a0 msgr2=0x5651916f6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/4213668802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916ee1a0 0x5651916f6a80 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fd0c4007740 tx=0x7fd0c4014000 comp rx=0 tx=0).stop 2024-04-26T18:33:14.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/4213668802 shutdown_connections 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/4213668802 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916ee1a0 0x5651916f6a80 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/4213668802 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916f9970 0x5651916f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/4213668802 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916eeac0 0x5651916f1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/4213668802 >> 172.21.15.84:0/4213668802 conn(0x56519161be40 msgr2=0x5651916f09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/4213668802 shutdown_connections 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/4213668802 wait complete. 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 -- start start 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 Processor -- start 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916ee1a0 0x56519170ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916eeac0 0x56519170d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.498+0000 7fd0de3b0b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x56519170d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0de3b0b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1af150 con 0x5651916f9970 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0de3b0b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1aea80 con 0x5651916eeac0 2024-04-26T18:33:14.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0de3b0b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd0cc1abc80 con 0x5651916ee1a0 2024-04-26T18:33:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0de3b0b80 10 client.0 _ll_register_callbacks cb 0x5651915d5680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0d20bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x56519170d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0d20bb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x56519170d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38754/0 (socket says 172.21.15.84:38754) 2024-04-26T18:33:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0d20bb700 1 -- 172.21.15.84:0/1890463594 learned_addr learned my addr 172.21.15.84:0/1890463594 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0d10b9700 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916eeac0 0x56519170d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0d18ba700 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916ee1a0 0x56519170ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0d10b9700 1 -- 172.21.15.84:0/1890463594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916ee1a0 msgr2=0x56519170ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0d10b9700 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916ee1a0 0x56519170ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0d10b9700 1 -- 172.21.15.84:0/1890463594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 msgr2=0x56519170d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0d10b9700 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x56519170d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0d10b9700 1 -- 172.21.15.84:0/1890463594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56519170de80 con 0x5651916eeac0 2024-04-26T18:33:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0d20bb700 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x56519170d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:33:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0d10b9700 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916eeac0 0x56519170d080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00cd7c0 tx=0x7fd0c00983e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e957ac6c5096dcb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.103:3300/0 2024-04-26T18:33:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.499+0000 7fd0b17fa700 1 -- 172.21.15.84:0/1890463594 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0c00b9020 con 0x5651916eeac0 2024-04-26T18:33:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/1890463594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd0c402acf0 con 0x5651916eeac0 2024-04-26T18:33:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0b17fa700 1 -- 172.21.15.84:0/1890463594 <== mon.2 v2:172.21.15.103:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fd0c00bc040 con 0x5651916eeac0 2024-04-26T18:33:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0b17fa700 1 -- 172.21.15.84:0/1890463594 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd0c009c010 con 0x5651916eeac0 2024-04-26T18:33:14.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0de3b0b80 10 client.25382 fetch_fsmap learned FSMap version 29 2024-04-26T18:33:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/1890463594 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565191770780 con 0x5651916eeac0 2024-04-26T18:33:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0b17fa700 1 -- 172.21.15.84:0/1890463594 <== mon.2 v2:172.21.15.103:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0c402acf0 con 0x5651916eeac0 2024-04-26T18:33:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0b17fa700 1 -- 172.21.15.84:0/1890463594 <== mon.2 v2:172.21.15.103:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd0c00a47f0 con 0x5651916eeac0 2024-04-26T18:33:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0de3b0b80 10 client.25382 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:33:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0de3b0b80 10 client.25382 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:33:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0de3b0b80 20 client.25382 populate_metadata read hostname 'smithi084' 2024-04-26T18:33:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x565191770de0 0x5651917731d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0de3b0b80 4 client.25382 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f0da206a", "release": true, "await": true}] 2024-04-26T18:33:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/1890463594 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f0da206a", "release": true, "await": true}) -- 0x5651916492c0 con 0x565191770de0 2024-04-26T18:33:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.500+0000 7fd0d18ba700 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x565191770de0 0x5651917731d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.501+0000 7fd0d18ba700 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x565191770de0 0x5651917731d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23815574f21e3a8a server_cookie=e17222e912df0bbd in_seq=0 out_seq=0 2024-04-26T18:33:14.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.501+0000 7fd0b17fa700 10 client.25382 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:14.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0b17fa700 1 -- 172.21.15.84:0/1890463594 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5651916492c0 con 0x565191770de0 2024-04-26T18:33:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0b17fa700 10 client.25382 handle_command_reply: tid=0 2024-04-26T18:33:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0de3b0b80 1 client.25382 shutdown 2024-04-26T18:33:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/1890463594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916eeac0 msgr2=0x56519170d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916eeac0 0x56519170d080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00cd7c0 tx=0x7fd0c00983e0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/1890463594 shutdown_connections 2024-04-26T18:33:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x565191770de0 0x5651917731d0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5651916f9970 0x56519170d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5651916ee1a0 0x56519170ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0b17fa700 0 client.25382 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0de3b0b80 1 --2- 172.21.15.84:0/1890463594 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5651916eeac0 0x56519170d080 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/1890463594 >> 172.21.15.84:0/1890463594 conn(0x56519161be40 msgr2=0x5651917048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/1890463594 shutdown_connections 2024-04-26T18:33:14.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0de3b0b80 1 -- 172.21.15.84:0/1890463594 wait complete. 2024-04-26T18:33:14.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.505+0000 7fd0de3b0b80 20 client.25382 trim_cache size 0 max 16384 2024-04-26T18:33:14.510 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:33:14.510 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:33:14.510 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:33:14.511 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 16, 2024-04-26T18:33:14.511 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:33:14.511 INFO:teuthology.orchestra.run.smithi084.stdout: "f0da206a": { 2024-04-26T18:33:14.511 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 16, 2024-04-26T18:33:14.511 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:33:14.511 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:33:14.511 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:33:14.511 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:33:14.511 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:33:14.511 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:33:14.511 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 79.5, 2024-04-26T18:33:14.511 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:33:14.511 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:33:14.512 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:33:14.512 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:33:14.512 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:33:14.512 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:33:14.512 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:33:14.512 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:33:14.512 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:33:14.512 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:33:14.512 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:33:14.512 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:33:14.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.506+0000 7fd0d3b1e700 1 -- 172.21.15.84:0/3585919591 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd0b40a3db0 msgr2=0x7fd0b40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:14.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.506+0000 7fd0d3b1e700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd0b40a3db0 0x7fd0b40a6270 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd0cc19a9f0 tx=0x7fd0c4009040 comp rx=0 tx=0).stop 2024-04-26T18:33:14.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.506+0000 7fd0d3b1e700 1 -- 172.21.15.84:0/3585919591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0cc103e40 msgr2=0x7fd0cc19fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:14.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.506+0000 7fd0d3b1e700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0cc103e40 0x7fd0cc19fc30 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd0c000b530 tx=0x7fd0c000b8f0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.506+0000 7fd0d3b1e700 1 -- 172.21.15.84:0/3585919591 shutdown_connections 2024-04-26T18:33:14.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.506+0000 7fd0d3b1e700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd0b40a3db0 0x7fd0b40a6270 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.506+0000 7fd0d3b1e700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd0cc1094e0 0x7fd0cc1998b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.506+0000 7fd0d3b1e700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd0cc1034a0 0x7fd0cc19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.506+0000 7fd0d3b1e700 1 --2- 172.21.15.84:0/3585919591 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd0cc103e40 0x7fd0cc19fc30 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:14.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.506+0000 7fd0d3b1e700 1 -- 172.21.15.84:0/3585919591 >> 172.21.15.84:0/3585919591 conn(0x7fd0cc07a4f0 msgr2=0x7fd0cc101550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:14.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.506+0000 7fd0d3b1e700 1 -- 172.21.15.84:0/3585919591 shutdown_connections 2024-04-26T18:33:14.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:33:14.506+0000 7fd0d3b1e700 1 -- 172.21.15.84:0/3585919591 wait complete. 2024-04-26T18:33:14.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:14 smithi084 ceph-mon[29141]: pgmap v161: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-04-26T18:33:14.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:14 smithi174 ceph-mon[32705]: pgmap v161: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-04-26T18:33:15.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:14 smithi103 ceph-mon[32727]: pgmap v161: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-04-26T18:33:15.318 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f0da206a', seconds elapsed: 31.3 2024-04-26T18:33:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.602+0000 7f36e449c640 1 -- 172.21.15.84:0/1211423688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36dc102940 msgr2=0x7f36dc102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.602+0000 7f36e449c640 1 --2- 172.21.15.84:0/1211423688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36dc102940 0x7f36dc102dc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f36d0009930 tx=0x7f36d002a420 comp rx=0 tx=0).stop 2024-04-26T18:33:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.602+0000 7f36e449c640 1 -- 172.21.15.84:0/1211423688 shutdown_connections 2024-04-26T18:33:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.602+0000 7f36e449c640 1 --2- 172.21.15.84:0/1211423688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc103300 0x7f36dc10f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.602+0000 7f36e449c640 1 --2- 172.21.15.84:0/1211423688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36dc102940 0x7f36dc102dc0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.602+0000 7f36e449c640 1 --2- 172.21.15.84:0/1211423688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36dc108950 0x7f36dc108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.602+0000 7f36e449c640 1 -- 172.21.15.84:0/1211423688 >> 172.21.15.84:0/1211423688 conn(0x7f36dc079dc0 msgr2=0x7f36dc07a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.602+0000 7f36e449c640 1 -- 172.21.15.84:0/1211423688 shutdown_connections 2024-04-26T18:33:15.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.602+0000 7f36e449c640 1 -- 172.21.15.84:0/1211423688 wait complete. 2024-04-26T18:33:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.603+0000 7f36e449c640 1 Processor -- start 2024-04-26T18:33:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.603+0000 7f36e449c640 1 -- start start 2024-04-26T18:33:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.603+0000 7f36e449c640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc102940 0x7f36dc1990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.603+0000 7f36e449c640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36dc103300 0x7f36dc199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.603+0000 7f36e449c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36dc108950 0x7f36dc19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.603+0000 7f36e449c640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc112c30 con 0x7f36dc108950 2024-04-26T18:33:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.603+0000 7f36e449c640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc112ab0 con 0x7f36dc102940 2024-04-26T18:33:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.603+0000 7f36e449c640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc112db0 con 0x7f36dc103300 2024-04-26T18:33:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.603+0000 7f36e2a39640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36dc108950 0x7f36dc19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.603+0000 7f36e2a39640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36dc108950 0x7f36dc19d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38772/0 (socket says 172.21.15.84:38772) 2024-04-26T18:33:15.606 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.603+0000 7f36e2a39640 1 -- 172.21.15.84:0/2432412401 learned_addr learned my addr 172.21.15.84:0/2432412401 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.603+0000 7f36e2238640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc102940 0x7f36dc1990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.603+0000 7f36e1a37640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36dc103300 0x7f36dc199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.604+0000 7f36e2a39640 1 -- 172.21.15.84:0/2432412401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36dc103300 msgr2=0x7f36dc199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.604+0000 7f36e2a39640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36dc103300 0x7f36dc199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.604+0000 7f36e2a39640 1 -- 172.21.15.84:0/2432412401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc102940 msgr2=0x7f36dc1990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.604+0000 7f36e2a39640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc102940 0x7f36dc1990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.604+0000 7f36e2a39640 1 -- 172.21.15.84:0/2432412401 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36dc19e0e0 con 0x7f36dc108950 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.604+0000 7f36e1a37640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36dc103300 0x7f36dc199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.604+0000 7f36e2238640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc102940 0x7f36dc1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.604+0000 7f36e2a39640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36dc108950 0x7f36dc19d9b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00b9e0 tx=0x7f36cc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.604+0000 7f36db7fe640 1 -- 172.21.15.84:0/2432412401 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc019070 con 0x7f36dc108950 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.604+0000 7f36db7fe640 1 -- 172.21.15.84:0/2432412401 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36cc014440 con 0x7f36dc108950 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.604+0000 7f36e449c640 1 -- 172.21.15.84:0/2432412401 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36dc19e3d0 con 0x7f36dc108950 2024-04-26T18:33:15.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.604+0000 7f36db7fe640 1 -- 172.21.15.84:0/2432412401 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0075d0 con 0x7f36dc108950 2024-04-26T18:33:15.608 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.604+0000 7f36e449c640 1 -- 172.21.15.84:0/2432412401 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36dc1a5c60 con 0x7f36dc108950 2024-04-26T18:33:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.607+0000 7f36db7fe640 1 -- 172.21.15.84:0/2432412401 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f36cc01c070 con 0x7f36dc108950 2024-04-26T18:33:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.607+0000 7f36e71b6740 1 Processor -- start 2024-04-26T18:33:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.607+0000 7f36e71b6740 1 -- start start 2024-04-26T18:33:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.607+0000 7f36e71b6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 0x55edefdb42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.607+0000 7f36e71b6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdbeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.607+0000 7f36e71b6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdbf0e0 0x55edefdc14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.607+0000 7f36e71b6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55edefd1a460 con 0x55edefdb3f00 2024-04-26T18:33:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.607+0000 7f36e71b6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55edefcd4590 con 0x55edefdb4820 2024-04-26T18:33:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.607+0000 7f36e71b6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55edefd16420 con 0x55edefdbf0e0 2024-04-26T18:33:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e1a37640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 0x55edefdb42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e1a37640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 0x55edefdb42e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38786/0 (socket says 172.21.15.84:38786) 2024-04-26T18:33:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e1a37640 1 -- 172.21.15.84:0/558943758 learned_addr learned my addr 172.21.15.84:0/558943758 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36db7fe640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36c40a3dc0 0x7f36c40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e2238640 1 --2- 172.21.15.84:0/558943758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdbeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e1a37640 1 --2- 172.21.15.84:0/558943758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdbf0e0 0x55edefdc14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e2a39640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36c40a3dc0 0x7f36c40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e1a37640 1 -- 172.21.15.84:0/558943758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdbf0e0 msgr2=0x55edefdc14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e1a37640 1 --2- 172.21.15.84:0/558943758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdbf0e0 0x55edefdc14d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e1a37640 1 -- 172.21.15.84:0/558943758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 msgr2=0x55edefdbeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e1a37640 1 --2- 172.21.15.84:0/558943758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdbeba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e1a37640 1 -- 172.21.15.84:0/558943758 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edefce10d0 con 0x55edefdb3f00 2024-04-26T18:33:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e2238640 1 --2- 172.21.15.84:0/558943758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdbeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36db7fe640 1 -- 172.21.15.84:0/2432412401 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f36cc0cfd60 con 0x7f36dc108950 2024-04-26T18:33:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e1a37640 1 --2- 172.21.15.84:0/558943758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 0x55edefdb42e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f36d0005ab0 tx=0x7f36d0009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f59edf9d4dabed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36e2a39640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36c40a3dc0 0x7f36c40a6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f36cc09bcf0 tx=0x7f36cc014b60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36d8bfe640 1 -- 172.21.15.84:0/558943758 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0035050 con 0x55edefdb3f00 2024-04-26T18:33:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36d8bfe640 1 -- 172.21.15.84:0/558943758 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d0041070 con 0x55edefdb3f00 2024-04-26T18:33:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.608+0000 7f36d8bfe640 1 -- 172.21.15.84:0/558943758 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d00047a0 con 0x55edefdb3f00 2024-04-26T18:33:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.609+0000 7f36e71b6740 1 -- 172.21.15.84:0/558943758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 msgr2=0x55edefdb42e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.609+0000 7f36e71b6740 1 --2- 172.21.15.84:0/558943758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 0x55edefdb42e0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f36d0005ab0 tx=0x7f36d0009560 comp rx=0 tx=0).stop 2024-04-26T18:33:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.609+0000 7f36e71b6740 1 -- 172.21.15.84:0/558943758 shutdown_connections 2024-04-26T18:33:15.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.609+0000 7f36e71b6740 1 --2- 172.21.15.84:0/558943758 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdbf0e0 0x55edefdc14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.609+0000 7f36e71b6740 1 --2- 172.21.15.84:0/558943758 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdbeba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.609+0000 7f36e71b6740 1 --2- 172.21.15.84:0/558943758 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 0x55edefdb42e0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.609+0000 7f36e71b6740 1 -- 172.21.15.84:0/558943758 >> 172.21.15.84:0/558943758 conn(0x55edefdb0410 msgr2=0x55edefdb07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.609+0000 7f36e71b6740 1 -- 172.21.15.84:0/558943758 shutdown_connections 2024-04-26T18:33:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.609+0000 7f36e71b6740 1 -- 172.21.15.84:0/558943758 wait complete. 2024-04-26T18:33:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e71b6740 1 -- start start 2024-04-26T18:33:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e71b6740 1 Processor -- start 2024-04-26T18:33:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e71b6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb3f00 0x55edefdbc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e71b6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdbc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e71b6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 0x55edefdbcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e71b6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55edefdc1cb0 con 0x55edefdbf0e0 2024-04-26T18:33:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e71b6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55edefca1b60 con 0x55edefdb4820 2024-04-26T18:33:15.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e71b6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55edefdc1e30 con 0x55edefdb3f00 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e71b6740 10 client.0 _ll_register_callbacks cb 0x55edefcc62b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e1a37640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 0x55edefdbcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e1a37640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 0x55edefdbcbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38790/0 (socket says 172.21.15.84:38790) 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e1a37640 1 -- 172.21.15.84:0/2331473005 learned_addr learned my addr 172.21.15.84:0/2331473005 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e2238640 1 --2- 172.21.15.84:0/2331473005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdbc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e1a37640 1 --2- 172.21.15.84:0/2331473005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb3f00 0x55edefdbc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e1a37640 1 -- 172.21.15.84:0/2331473005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb3f00 msgr2=0x55edefdbc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e1a37640 1 --2- 172.21.15.84:0/2331473005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb3f00 0x55edefdbc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e1a37640 1 -- 172.21.15.84:0/2331473005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 msgr2=0x55edefdbc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e1a37640 1 --2- 172.21.15.84:0/2331473005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdbc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.610+0000 7f36e1a37640 1 -- 172.21.15.84:0/2331473005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edefa2c210 con 0x55edefdbf0e0 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36e1a37640 1 --2- 172.21.15.84:0/2331473005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 0x55edefdbcbb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f36d0005760 tx=0x7f36d0005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b77c44c1a34cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:33:15.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36c1ffb640 1 -- 172.21.15.84:0/2331473005 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0035030 con 0x55edefdbf0e0 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36c1ffb640 1 -- 172.21.15.84:0/2331473005 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f36d00319f0 con 0x55edefdbf0e0 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36e71b6740 1 -- 172.21.15.84:0/2331473005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d0041070 con 0x55edefdbf0e0 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36c1ffb640 1 -- 172.21.15.84:0/2331473005 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f36d0059050 con 0x55edefdbf0e0 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36e71b6740 10 client.16095 fetch_fsmap learned FSMap version 29 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36e71b6740 1 -- 172.21.15.84:0/2331473005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55edefaf8620 con 0x55edefdbf0e0 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36c1ffb640 1 -- 172.21.15.84:0/2331473005 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d0041070 con 0x55edefdbf0e0 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36c1ffb640 1 -- 172.21.15.84:0/2331473005 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f36d0003a60 con 0x55edefdbf0e0 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36e71b6740 10 client.16095 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36e71b6740 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36e71b6740 20 client.16095 populate_metadata read hostname 'smithi084' 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36e71b6740 1 --2- 172.21.15.84:0/2331473005 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55edefdceff0 0x55edefdd13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36e71b6740 4 client.16095 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.611+0000 7f36e71b6740 1 -- 172.21.15.84:0/2331473005 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55edefd28070 con 0x55edefdceff0 2024-04-26T18:33:15.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.612+0000 7f36e2238640 1 --2- 172.21.15.84:0/2331473005 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55edefdceff0 0x55edefdd13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.612+0000 7f36e2238640 1 --2- 172.21.15.84:0/2331473005 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55edefdceff0 0x55edefdd13e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d842f198c4fe8f8c server_cookie=5c0e75f9828e2ba5 in_seq=0 out_seq=0 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.612+0000 7f36c1ffb640 10 client.16095 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36c1ffb640 1 -- 172.21.15.84:0/2331473005 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55edefd28070 con 0x55edefdceff0 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36c1ffb640 10 client.16095 handle_command_reply: tid=0 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36e71b6740 1 client.16095 shutdown 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36e71b6740 1 -- 172.21.15.84:0/2331473005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 msgr2=0x55edefdbcbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36e71b6740 1 --2- 172.21.15.84:0/2331473005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 0x55edefdbcbb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f36d0005760 tx=0x7f36d0005400 comp rx=0 tx=0).stop 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36e71b6740 1 -- 172.21.15.84:0/2331473005 shutdown_connections 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36e71b6740 1 --2- 172.21.15.84:0/2331473005 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55edefdceff0 0x55edefdd13e0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36e71b6740 1 --2- 172.21.15.84:0/2331473005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 0x55edefdbcbb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36c1ffb640 0 client.16095 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36e71b6740 1 --2- 172.21.15.84:0/2331473005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdbc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36e71b6740 1 --2- 172.21.15.84:0/2331473005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb3f00 0x55edefdbc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36e71b6740 1 -- 172.21.15.84:0/2331473005 >> 172.21.15.84:0/2331473005 conn(0x55edefdb0410 msgr2=0x55edefdb6190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36e71b6740 1 -- 172.21.15.84:0/2331473005 shutdown_connections 2024-04-26T18:33:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36e71b6740 1 -- 172.21.15.84:0/2331473005 wait complete. 2024-04-26T18:33:15.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.613+0000 7f36e71b6740 20 client.16095 trim_cache size 0 max 16384 2024-04-26T18:33:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.627+0000 7f36e71b6740 1 Processor -- start 2024-04-26T18:33:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.627+0000 7f36e71b6740 1 -- start start 2024-04-26T18:33:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e71b6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 0x55edefdbc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e71b6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb4820 0x55edefdb9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.633 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e71b6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdbf0e0 0x55edefdb9950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e71b6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1ae4a0 con 0x55edefdb3f00 2024-04-26T18:33:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e71b6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1affe0 con 0x55edefdbf0e0 2024-04-26T18:33:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e71b6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1aeb70 con 0x55edefdb4820 2024-04-26T18:33:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e1a37640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 0x55edefdbc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e1a37640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 0x55edefdbc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38798/0 (socket says 172.21.15.84:38798) 2024-04-26T18:33:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e1a37640 1 -- 172.21.15.84:0/3861105775 learned_addr learned my addr 172.21.15.84:0/3861105775 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e1a37640 1 -- 172.21.15.84:0/3861105775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb4820 msgr2=0x55edefdb9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e1a37640 1 --2- 172.21.15.84:0/3861105775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb4820 0x55edefdb9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e1a37640 1 -- 172.21.15.84:0/3861105775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdbf0e0 msgr2=0x55edefdb9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e1a37640 1 --2- 172.21.15.84:0/3861105775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdbf0e0 0x55edefdb9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e1a37640 1 -- 172.21.15.84:0/3861105775 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55edefdb3cc0 con 0x55edefdb3f00 2024-04-26T18:33:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36e1a37640 1 --2- 172.21.15.84:0/3861105775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 0x55edefdbc800 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f36d0000c00 tx=0x7f36d003d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ec998cfcf9f123 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:15.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36c3fff640 1 -- 172.21.15.84:0/3861105775 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0035070 con 0x55edefdb3f00 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36c3fff640 1 -- 172.21.15.84:0/3861105775 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d0041070 con 0x55edefdb3f00 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.628+0000 7f36c3fff640 1 -- 172.21.15.84:0/3861105775 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d004e040 con 0x55edefdb3f00 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 -- 172.21.15.84:0/3861105775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 msgr2=0x55edefdbc800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 --2- 172.21.15.84:0/3861105775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 0x55edefdbc800 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f36d0000c00 tx=0x7f36d003d520 comp rx=0 tx=0).stop 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 -- 172.21.15.84:0/3861105775 shutdown_connections 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 --2- 172.21.15.84:0/3861105775 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdbf0e0 0x55edefdb9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 --2- 172.21.15.84:0/3861105775 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb4820 0x55edefdb9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 --2- 172.21.15.84:0/3861105775 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdb3f00 0x55edefdbc800 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 -- 172.21.15.84:0/3861105775 >> 172.21.15.84:0/3861105775 conn(0x55edefdb0410 msgr2=0x55edefdbc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 -- 172.21.15.84:0/3861105775 shutdown_connections 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 -- 172.21.15.84:0/3861105775 wait complete. 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 -- start start 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 Processor -- start 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb3f00 0x55edefdd46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdd2a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 0x55edefdd2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1af910 con 0x55edefdbf0e0 2024-04-26T18:33:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1af240 con 0x55edefdb4820 2024-04-26T18:33:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36dc1ac480 con 0x55edefdb3f00 2024-04-26T18:33:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.629+0000 7f36e71b6740 10 client.0 _ll_register_callbacks cb 0x7f36d0031d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e1a37640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 0x55edefdd2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e1a37640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 0x55edefdd2fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38804/0 (socket says 172.21.15.84:38804) 2024-04-26T18:33:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e1a37640 1 -- 172.21.15.84:0/2668211972 learned_addr learned my addr 172.21.15.84:0/2668211972 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e1a37640 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb3f00 0x55edefdd46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e2238640 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdd2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e1a37640 1 -- 172.21.15.84:0/2668211972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb3f00 msgr2=0x55edefdd46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:15.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e1a37640 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb3f00 0x55edefdd46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e1a37640 1 -- 172.21.15.84:0/2668211972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 msgr2=0x55edefdd2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e1a37640 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdd2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e1a37640 1 -- 172.21.15.84:0/2668211972 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55edefdd36e0 con 0x55edefdbf0e0 2024-04-26T18:33:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e2238640 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdd2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e1a37640 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 0x55edefdd2fa0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f36d00095e0 tx=0x7f36d0005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b443ea39dc8028dd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:33:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36c1ffb640 1 -- 172.21.15.84:0/2668211972 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d0035070 con 0x55edefdbf0e0 2024-04-26T18:33:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36c1ffb640 1 -- 172.21.15.84:0/2668211972 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f36d00040b0 con 0x55edefdbf0e0 2024-04-26T18:33:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e71b6740 1 -- 172.21.15.84:0/2668211972 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d0041070 con 0x55edefdbf0e0 2024-04-26T18:33:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36c1ffb640 1 -- 172.21.15.84:0/2668211972 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f36d0005400 con 0x55edefdbf0e0 2024-04-26T18:33:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e71b6740 10 client.16101 fetch_fsmap learned FSMap version 29 2024-04-26T18:33:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36e71b6740 1 -- 172.21.15.84:0/2668211972 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55edefe36160 con 0x55edefdbf0e0 2024-04-26T18:33:15.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.630+0000 7f36c1ffb640 1 -- 172.21.15.84:0/2668211972 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d0041070 con 0x55edefdbf0e0 2024-04-26T18:33:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.631+0000 7f36c1ffb640 1 -- 172.21.15.84:0/2668211972 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f36d003db40 con 0x55edefdbf0e0 2024-04-26T18:33:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.631+0000 7f36e71b6740 10 client.16101 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:33:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.631+0000 7f36e71b6740 10 client.16101 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.631+0000 7f36e71b6740 20 client.16101 populate_metadata read hostname 'smithi084' 2024-04-26T18:33:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.631+0000 7f36e71b6740 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55edefdceff0 0x55edefe38800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.631+0000 7f36e71b6740 4 client.16101 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:33:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.631+0000 7f36e71b6740 1 -- 172.21.15.84:0/2668211972 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55edefd21ee0 con 0x55edefdceff0 2024-04-26T18:33:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.631+0000 7f36e2238640 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55edefdceff0 0x55edefe38800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.631+0000 7f36c1ffb640 10 client.16101 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:15.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.631+0000 7f36e2238640 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55edefdceff0 0x55edefe38800 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=972c343fd186411c server_cookie=6bde6da58069193b in_seq=0 out_seq=0 2024-04-26T18:33:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.640+0000 7f36c1ffb640 1 -- 172.21.15.84:0/2668211972 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55edefd21ee0 con 0x55edefdceff0 2024-04-26T18:33:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.640+0000 7f36c1ffb640 10 client.16101 handle_command_reply: tid=0 2024-04-26T18:33:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.640+0000 7f36e71b6740 1 client.16101 shutdown 2024-04-26T18:33:15.641 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.640+0000 7f36e71b6740 1 -- 172.21.15.84:0/2668211972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 msgr2=0x55edefdd2fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.640+0000 7f36e71b6740 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 0x55edefdd2fa0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f36d00095e0 tx=0x7f36d0005760 comp rx=0 tx=0).stop 2024-04-26T18:33:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.640+0000 7f36e71b6740 1 -- 172.21.15.84:0/2668211972 shutdown_connections 2024-04-26T18:33:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.640+0000 7f36e71b6740 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55edefdceff0 0x55edefe38800 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.640+0000 7f36e71b6740 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55edefdbf0e0 0x55edefdd2fa0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.640+0000 7f36e71b6740 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55edefdb4820 0x55edefdd2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.640+0000 7f36e71b6740 1 --2- 172.21.15.84:0/2668211972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55edefdb3f00 0x55edefdd46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.640+0000 7f36c1ffb640 0 client.16101 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.640+0000 7f36e71b6740 1 -- 172.21.15.84:0/2668211972 >> 172.21.15.84:0/2668211972 conn(0x55edefdb0410 msgr2=0x55edefdaf5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e71b6740 1 -- 172.21.15.84:0/2668211972 shutdown_connections 2024-04-26T18:33:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e71b6740 1 -- 172.21.15.84:0/2668211972 wait complete. 2024-04-26T18:33:15.642 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e71b6740 20 client.16101 trim_cache size 0 max 16384 2024-04-26T18:33:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:33:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:33:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:33:15.643 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:33:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e449c640 1 -- 172.21.15.84:0/2432412401 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36c40a3dc0 msgr2=0x7f36c40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e449c640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36c40a3dc0 0x7f36c40a6280 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f36cc09bcf0 tx=0x7f36cc014b60 comp rx=0 tx=0).stop 2024-04-26T18:33:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e449c640 1 -- 172.21.15.84:0/2432412401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36dc108950 msgr2=0x7f36dc19d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e449c640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36dc108950 0x7f36dc19d9b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f36cc00b9e0 tx=0x7f36cc00bea0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e449c640 1 -- 172.21.15.84:0/2432412401 shutdown_connections 2024-04-26T18:33:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e449c640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36c40a3dc0 0x7f36c40a6280 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e449c640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36dc108950 0x7f36dc19d9b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e449c640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f36dc103300 0x7f36dc199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e449c640 1 --2- 172.21.15.84:0/2432412401 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36dc102940 0x7f36dc1990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:15.644 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e449c640 1 -- 172.21.15.84:0/2432412401 >> 172.21.15.84:0/2432412401 conn(0x7f36dc079dc0 msgr2=0x7f36dc100260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e449c640 1 -- 172.21.15.84:0/2432412401 shutdown_connections 2024-04-26T18:33:15.645 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:15.641+0000 7f36e449c640 1 -- 172.21.15.84:0/2432412401 wait complete. 2024-04-26T18:33:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:16 smithi174 ceph-mon[32705]: pgmap v162: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:33:17.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:16 smithi084 ceph-mon[29141]: pgmap v162: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:33:17.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:16 smithi103 ceph-mon[32727]: pgmap v162: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:33:17.528 INFO:tasks.workunit.client.0.smithi084.stdout:64/600: open fd = 3 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:65/600: open fd = 3 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:66/600: open fd = 3 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:67/600: open fd = 3 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.529 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.530 INFO:tasks.workunit.client.0.smithi084.stdout:68/600: open fd = 3 2024-04-26T18:33:17.530 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.530 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.530 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.530 INFO:tasks.workunit.client.0.smithi084.stdout:69/600: open fd = 3 2024-04-26T18:33:17.530 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.530 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.530 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.530 INFO:tasks.workunit.client.0.smithi084.stdout:70/600: open fd = 3 2024-04-26T18:33:17.530 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.530 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.530 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.530 INFO:tasks.workunit.client.0.smithi084.stdout:71/600: open fd = 3 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:72/600: open fd = 3 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:73/600: open fd = 3 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:74/600: open fd = 3 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.531 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:75/600: open fd = 3 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:76/600: open fd = 3 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:77/600: open fd = 3 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:78/600: open fd = 3 2024-04-26T18:33:17.532 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:79/600: open fd = 3 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:80/600: open fd = 3 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:81/600: open fd = 3 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.533 INFO:tasks.workunit.client.0.smithi084.stdout:82/600: open fd = 3 2024-04-26T18:33:17.534 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.534 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.534 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.534 INFO:tasks.workunit.client.0.smithi084.stdout:83/600: open fd = 3 2024-04-26T18:33:17.534 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.534 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.534 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.534 INFO:tasks.workunit.client.0.smithi084.stdout:84/600: open fd = 3 2024-04-26T18:33:17.534 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.534 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.534 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.534 INFO:tasks.workunit.client.0.smithi084.stdout:85/600: open fd = 3 2024-04-26T18:33:17.534 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:86/600: open fd = 3 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:87/600: open fd = 3 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:88/600: open fd = 3 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.535 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:89/600: open fd = 3 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:90/600: open fd = 3 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:91/600: open fd = 3 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:92/600: open fd = 3 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.536 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:93/600: open fd = 3 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:94/600: open fd = 3 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:95/600: open fd = 3 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:96/600: open fd = 3 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.537 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:97/600: open fd = 3 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:98/600: open fd = 3 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:99/600: open fd = 3 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:100/600: open fd = 3 2024-04-26T18:33:17.538 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.539 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.539 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.539 INFO:tasks.workunit.client.0.smithi084.stdout:101/600: open fd = 3 2024-04-26T18:33:17.539 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.539 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.539 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.539 INFO:tasks.workunit.client.0.smithi084.stdout:102/600: open fd = 3 2024-04-26T18:33:17.539 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.539 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.539 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.539 INFO:tasks.workunit.client.0.smithi084.stdout:103/600: open fd = 3 2024-04-26T18:33:17.539 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:104/600: open fd = 3 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:105/600: open fd = 3 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:106/600: open fd = 3 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:107/600: open fd = 3 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.540 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:108/600: open fd = 3 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:109/600: open fd = 3 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:110/600: open fd = 3 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:111/600: open fd = 3 2024-04-26T18:33:17.541 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:112/600: open fd = 3 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:113/600: open fd = 3 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:114/600: open fd = 3 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.542 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:115/600: open fd = 3 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:116/600: open fd = 3 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:117/600: open fd = 3 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:118/600: open fd = 3 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.543 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:119/600: open fd = 3 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:120/600: open fd = 3 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:121/600: open fd = 3 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.544 INFO:tasks.workunit.client.0.smithi084.stdout:122/600: open fd = 3 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:123/600: open fd = 3 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:124/600: open fd = 3 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:125/600: open fd = 3 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:17.545 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:33:17.546 INFO:tasks.workunit.client.0.smithi084.stdout:126/600: open fd = 3 2024-04-26T18:33:17.546 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:33:18.929 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:18 smithi103 ceph-mon[32727]: pgmap v163: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:33:18.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:18 smithi174 ceph-mon[32705]: pgmap v163: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:33:19.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:18 smithi084 ceph-mon[29141]: pgmap v163: 97 pgs: 97 active+clean; 4.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:33:20.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:20 smithi174 ceph-mon[32705]: pgmap v164: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-26T18:33:21.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:20 smithi084 ceph-mon[29141]: pgmap v164: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-26T18:33:21.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:20 smithi103 ceph-mon[32727]: pgmap v164: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 16 op/s 2024-04-26T18:33:22.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:22 smithi174 ceph-mon[32705]: pgmap v165: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-26T18:33:23.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:22 smithi084 ceph-mon[29141]: pgmap v165: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-26T18:33:23.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:22 smithi103 ceph-mon[32727]: pgmap v165: 97 pgs: 97 active+clean; 4.4 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 12 op/s 2024-04-26T18:33:24.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:24 smithi174 ceph-mon[32705]: pgmap v166: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-04-26T18:33:25.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:24 smithi084 ceph-mon[29141]: pgmap v166: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-04-26T18:33:25.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:24 smithi103 ceph-mon[32727]: pgmap v166: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 18 op/s 2024-04-26T18:33:27.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:26 smithi084 ceph-mon[29141]: pgmap v167: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-26T18:33:27.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:26 smithi103 ceph-mon[32727]: pgmap v167: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-26T18:33:27.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:26 smithi174 ceph-mon[32705]: pgmap v167: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-26T18:33:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.737+0000 7fafdb551640 1 -- 172.21.15.84:0/3377555458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd410cf30 msgr2=0x7fafd410f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.737+0000 7fafdb551640 1 --2- 172.21.15.84:0/3377555458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd410cf30 0x7fafd410f3b0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009ca0 tx=0x7fafcc02a950 comp rx=0 tx=0).stop 2024-04-26T18:33:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.737+0000 7fafdb551640 1 -- 172.21.15.84:0/3377555458 shutdown_connections 2024-04-26T18:33:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.737+0000 7fafdb551640 1 --2- 172.21.15.84:0/3377555458 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd410cf30 0x7fafd410f3b0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.737+0000 7fafdb551640 1 --2- 172.21.15.84:0/3377555458 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafd4104480 0x7fafd410c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.737+0000 7fafdb551640 1 --2- 172.21.15.84:0/3377555458 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafd4103b60 0x7fafd4103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.737+0000 7fafdb551640 1 -- 172.21.15.84:0/3377555458 >> 172.21.15.84:0/3377555458 conn(0x7fafd40789b0 msgr2=0x7fafd4078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.738+0000 7fafdb551640 1 -- 172.21.15.84:0/3377555458 shutdown_connections 2024-04-26T18:33:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.738+0000 7fafdb551640 1 -- 172.21.15.84:0/3377555458 wait complete. 2024-04-26T18:33:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.738+0000 7fafdb551640 1 Processor -- start 2024-04-26T18:33:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.738+0000 7fafdb551640 1 -- start start 2024-04-26T18:33:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.738+0000 7fafdb551640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd4103b60 0x7fafd4199000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.738+0000 7fafdb551640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafd4104480 0x7fafd4199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.738+0000 7fafdb551640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafd410cf30 0x7fafd419d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.738+0000 7fafdb551640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fafd4112bc0 con 0x7fafd4103b60 2024-04-26T18:33:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.738+0000 7fafdb551640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fafd4112a40 con 0x7fafd410cf30 2024-04-26T18:33:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.738+0000 7fafdb551640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fafd4112d40 con 0x7fafd4104480 2024-04-26T18:33:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.738+0000 7fafd92ed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd4103b60 0x7fafd4199000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafd92ed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd4103b60 0x7fafd4199000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47194/0 (socket says 172.21.15.84:47194) 2024-04-26T18:33:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafd92ed640 1 -- 172.21.15.84:0/1588285746 learned_addr learned my addr 172.21.15.84:0/1588285746 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafd8aec640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafd4104480 0x7fafd4199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafd9aee640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafd410cf30 0x7fafd419d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafd92ed640 1 -- 172.21.15.84:0/1588285746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafd4104480 msgr2=0x7fafd4199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafd92ed640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafd4104480 0x7fafd4199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafd92ed640 1 -- 172.21.15.84:0/1588285746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafd410cf30 msgr2=0x7fafd419d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafd92ed640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafd410cf30 0x7fafd419d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafd92ed640 1 -- 172.21.15.84:0/1588285746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafd419dff0 con 0x7fafd4103b60 2024-04-26T18:33:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafd92ed640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd4103b60 0x7fafd4199000 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fafd000d770 tx=0x7fafd000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafca7fc640 1 -- 172.21.15.84:0/1588285746 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0017070 con 0x7fafd4103b60 2024-04-26T18:33:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafca7fc640 1 -- 172.21.15.84:0/1588285746 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafd00054d0 con 0x7fafd4103b60 2024-04-26T18:33:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafdb551640 1 -- 172.21.15.84:0/1588285746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafd419e2e0 con 0x7fafd4103b60 2024-04-26T18:33:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafca7fc640 1 -- 172.21.15.84:0/1588285746 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0004c30 con 0x7fafd4103b60 2024-04-26T18:33:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.739+0000 7fafdb551640 1 -- 172.21.15.84:0/1588285746 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafd41a5b70 con 0x7fafd4103b60 2024-04-26T18:33:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.742+0000 7fafca7fc640 1 -- 172.21.15.84:0/1588285746 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fafd0004e10 con 0x7fafd4103b60 2024-04-26T18:33:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.742+0000 7fafde26c740 1 Processor -- start 2024-04-26T18:33:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.742+0000 7fafde26c740 1 -- start start 2024-04-26T18:33:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafde26c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 0x56148aaef2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafde26c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aaef820 0x56148aaf9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafde26c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aafa0e0 0x56148aafc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafde26c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56148aa55460 con 0x56148aafa0e0 2024-04-26T18:33:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafde26c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56148aa0f590 con 0x56148aaeef00 2024-04-26T18:33:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafde26c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56148aa51420 con 0x56148aaef820 2024-04-26T18:33:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafd8aec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aafa0e0 0x56148aafc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafd8aec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aafa0e0 0x56148aafc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47208/0 (socket says 172.21.15.84:47208) 2024-04-26T18:33:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafd8aec640 1 -- 172.21.15.84:0/1447409811 learned_addr learned my addr 172.21.15.84:0/1447409811 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafd92ed640 1 --2- 172.21.15.84:0/1447409811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aaef820 0x56148aaf9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafca7fc640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fafc00a3dc0 0x7fafc00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafd9aee640 1 --2- 172.21.15.84:0/1447409811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 0x56148aaef2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafd8aec640 1 -- 172.21.15.84:0/1447409811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aaef820 msgr2=0x56148aaf9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafd8aec640 1 --2- 172.21.15.84:0/1447409811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aaef820 0x56148aaf9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafd8aec640 1 -- 172.21.15.84:0/1447409811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 msgr2=0x56148aaef2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafd8aec640 1 --2- 172.21.15.84:0/1447409811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 0x56148aaef2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafd8aec640 1 -- 172.21.15.84:0/1447409811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56148aa1c0d0 con 0x56148aafa0e0 2024-04-26T18:33:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafd9aee640 1 --2- 172.21.15.84:0/1447409811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 0x56148aaef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafd9aee640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fafc00a3dc0 0x7fafc00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafca7fc640 1 -- 172.21.15.84:0/1588285746 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fafd00d0390 con 0x7fafd4103b60 2024-04-26T18:33:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.743+0000 7fafd8aec640 1 --2- 172.21.15.84:0/1447409811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aafa0e0 0x56148aafc4d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fafc400dc00 tx=0x7fafc400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25277887f04cf2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.744+0000 7fafbf7fe640 1 -- 172.21.15.84:0/1447409811 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4011440 con 0x56148aafa0e0 2024-04-26T18:33:28.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.744+0000 7fafbf7fe640 1 -- 172.21.15.84:0/1447409811 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc4011a90 con 0x56148aafa0e0 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.744+0000 7fafbf7fe640 1 -- 172.21.15.84:0/1447409811 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc40109c0 con 0x56148aafa0e0 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.744+0000 7fafd9aee640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fafc00a3dc0 0x7fafc00a6280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fafcc005cc0 tx=0x7fafcc005fd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.744+0000 7fafde26c740 1 -- 172.21.15.84:0/1447409811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aafa0e0 msgr2=0x56148aafc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.744+0000 7fafde26c740 1 --2- 172.21.15.84:0/1447409811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aafa0e0 0x56148aafc4d0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fafc400dc00 tx=0x7fafc400f730 comp rx=0 tx=0).stop 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.745+0000 7fafde26c740 1 -- 172.21.15.84:0/1447409811 shutdown_connections 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.745+0000 7fafde26c740 1 --2- 172.21.15.84:0/1447409811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aafa0e0 0x56148aafc4d0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.745+0000 7fafde26c740 1 --2- 172.21.15.84:0/1447409811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aaef820 0x56148aaf9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.745+0000 7fafde26c740 1 --2- 172.21.15.84:0/1447409811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 0x56148aaef2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.745+0000 7fafde26c740 1 -- 172.21.15.84:0/1447409811 >> 172.21.15.84:0/1447409811 conn(0x56148aaeb410 msgr2=0x56148aaeb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.745+0000 7fafde26c740 1 -- 172.21.15.84:0/1447409811 shutdown_connections 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.745+0000 7fafde26c740 1 -- 172.21.15.84:0/1447409811 wait complete. 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.745+0000 7fafde26c740 1 -- start start 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.745+0000 7fafde26c740 1 Processor -- start 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafde26c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 0x56148aaf6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafde26c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 0x56148aaf7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafde26c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aafa0e0 0x56148aaf7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafde26c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56148aafcd10 con 0x56148aaef820 2024-04-26T18:33:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafde26c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56148a9dcb60 con 0x56148aaeef00 2024-04-26T18:33:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafde26c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56148aafce90 con 0x56148aafa0e0 2024-04-26T18:33:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafde26c740 10 client.0 _ll_register_callbacks cb 0x56148aa012b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafd8aec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 0x56148aaf7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafd8aec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 0x56148aaf7530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47222/0 (socket says 172.21.15.84:47222) 2024-04-26T18:33:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafd92ed640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 0x56148aaf6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafd8aec640 1 -- 172.21.15.84:0/4229442572 learned_addr learned my addr 172.21.15.84:0/4229442572 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafd9aee640 1 --2- 172.21.15.84:0/4229442572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aafa0e0 0x56148aaf7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafd8aec640 1 -- 172.21.15.84:0/4229442572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aafa0e0 msgr2=0x56148aaf7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafd8aec640 1 --2- 172.21.15.84:0/4229442572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aafa0e0 0x56148aaf7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafd8aec640 1 -- 172.21.15.84:0/4229442572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 msgr2=0x56148aaf6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafd8aec640 1 --2- 172.21.15.84:0/4229442572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 0x56148aaf6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafd8aec640 1 -- 172.21.15.84:0/4229442572 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56148a767210 con 0x56148aaef820 2024-04-26T18:33:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafd8aec640 1 --2- 172.21.15.84:0/4229442572 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 0x56148aaf7530 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fafc4010470 tx=0x7fafc400d740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff9a33064ab2ebb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:33:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafbcff9640 1 -- 172.21.15.84:0/4229442572 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4011650 con 0x56148aaef820 2024-04-26T18:33:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafbcff9640 1 -- 172.21.15.84:0/4229442572 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fafc40117f0 con 0x56148aaef820 2024-04-26T18:33:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.746+0000 7fafde26c740 1 -- 172.21.15.84:0/4229442572 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafc4011a90 con 0x56148aaef820 2024-04-26T18:33:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.747+0000 7fafbcff9640 1 -- 172.21.15.84:0/4229442572 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fafc4005040 con 0x56148aaef820 2024-04-26T18:33:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.747+0000 7fafde26c740 10 client.16113 fetch_fsmap learned FSMap version 29 2024-04-26T18:33:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.747+0000 7fafde26c740 1 -- 172.21.15.84:0/4229442572 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56148a833620 con 0x56148aaef820 2024-04-26T18:33:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.747+0000 7fafbcff9640 1 -- 172.21.15.84:0/4229442572 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafc4011a90 con 0x56148aaef820 2024-04-26T18:33:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.747+0000 7fafbcff9640 1 -- 172.21.15.84:0/4229442572 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fafc40046c0 con 0x56148aaef820 2024-04-26T18:33:28.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.747+0000 7fafde26c740 10 client.16113 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:33:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.747+0000 7fafde26c740 10 client.16113 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.747+0000 7fafde26c740 20 client.16113 populate_metadata read hostname 'smithi084' 2024-04-26T18:33:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.747+0000 7fafde26c740 1 --2- 172.21.15.84:0/4229442572 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56148ab0a050 0x56148ab0c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.747+0000 7fafde26c740 4 client.16113 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:33:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.747+0000 7fafde26c740 1 -- 172.21.15.84:0/4229442572 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56148aa63070 con 0x56148ab0a050 2024-04-26T18:33:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.747+0000 7fafd92ed640 1 --2- 172.21.15.84:0/4229442572 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56148ab0a050 0x56148ab0c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.748+0000 7fafd92ed640 1 --2- 172.21.15.84:0/4229442572 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56148ab0a050 0x56148ab0c440 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc99de214129712 server_cookie=d5d16f4a7a80b6d9 in_seq=0 out_seq=0 2024-04-26T18:33:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.748+0000 7fafbcff9640 10 client.16113 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.748+0000 7fafbcff9640 1 -- 172.21.15.84:0/4229442572 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56148aa63070 con 0x56148ab0a050 2024-04-26T18:33:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.748+0000 7fafbcff9640 10 client.16113 handle_command_reply: tid=0 2024-04-26T18:33:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.748+0000 7fafde26c740 1 client.16113 shutdown 2024-04-26T18:33:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.749+0000 7fafde26c740 1 -- 172.21.15.84:0/4229442572 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 msgr2=0x56148aaf7530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:28.756 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.749+0000 7fafde26c740 1 --2- 172.21.15.84:0/4229442572 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 0x56148aaf7530 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fafc4010470 tx=0x7fafc400d740 comp rx=0 tx=0).stop 2024-04-26T18:33:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.749+0000 7fafde26c740 1 -- 172.21.15.84:0/4229442572 shutdown_connections 2024-04-26T18:33:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.749+0000 7fafde26c740 1 --2- 172.21.15.84:0/4229442572 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56148ab0a050 0x56148ab0c440 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.749+0000 7fafde26c740 1 --2- 172.21.15.84:0/4229442572 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aafa0e0 0x56148aaf7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.749+0000 7fafde26c740 1 --2- 172.21.15.84:0/4229442572 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 0x56148aaf7530 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.749+0000 7fafde26c740 1 --2- 172.21.15.84:0/4229442572 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 0x56148aaf6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.749+0000 7fafbcff9640 0 client.16113 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.749+0000 7fafde26c740 1 -- 172.21.15.84:0/4229442572 >> 172.21.15.84:0/4229442572 conn(0x56148aaeb410 msgr2=0x56148aaf1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.749+0000 7fafde26c740 1 -- 172.21.15.84:0/4229442572 shutdown_connections 2024-04-26T18:33:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.749+0000 7fafde26c740 1 -- 172.21.15.84:0/4229442572 wait complete. 2024-04-26T18:33:28.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.749+0000 7fafde26c740 20 client.16113 trim_cache size 0 max 16384 2024-04-26T18:33:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafde26c740 1 Processor -- start 2024-04-26T18:33:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafde26c740 1 -- start start 2024-04-26T18:33:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafde26c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaeef00 0x56148aaead30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafde26c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aaef820 0x56148aaf18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafde26c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aafa0e0 0x56148ab11bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafde26c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fafd41ae540 con 0x56148aaeef00 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafde26c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fafd41b0080 con 0x56148aafa0e0 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafde26c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fafd41aec10 con 0x56148aaef820 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafd92ed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaeef00 0x56148aaead30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafd92ed640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaeef00 0x56148aaead30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47230/0 (socket says 172.21.15.84:47230) 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafd92ed640 1 -- 172.21.15.84:0/2120361328 learned_addr learned my addr 172.21.15.84:0/2120361328 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafd8aec640 1 --2- 172.21.15.84:0/2120361328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aaef820 0x56148aaf18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafd9aee640 1 --2- 172.21.15.84:0/2120361328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aafa0e0 0x56148ab11bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafd92ed640 1 -- 172.21.15.84:0/2120361328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aaef820 msgr2=0x56148aaf18c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafd92ed640 1 --2- 172.21.15.84:0/2120361328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aaef820 0x56148aaf18c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafd92ed640 1 -- 172.21.15.84:0/2120361328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aafa0e0 msgr2=0x56148ab11bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafd92ed640 1 --2- 172.21.15.84:0/2120361328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aafa0e0 0x56148ab11bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafd92ed640 1 -- 172.21.15.84:0/2120361328 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56148aae6d60 con 0x56148aaeef00 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafd8aec640 1 --2- 172.21.15.84:0/2120361328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aaef820 0x56148aaf18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafd9aee640 1 --2- 172.21.15.84:0/2120361328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aafa0e0 0x56148ab11bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafd92ed640 1 --2- 172.21.15.84:0/2120361328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaeef00 0x56148aaead30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009610 tx=0x7fafd00d28a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21d9f615d1eec8bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.764+0000 7fafbeffd640 1 -- 172.21.15.84:0/2120361328 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd0020040 con 0x56148aaeef00 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.765+0000 7fafbeffd640 1 -- 172.21.15.84:0/2120361328 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafd009c320 con 0x56148aaeef00 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.765+0000 7fafbeffd640 1 -- 172.21.15.84:0/2120361328 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafd00a4b00 con 0x56148aaeef00 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.765+0000 7fafde26c740 1 -- 172.21.15.84:0/2120361328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaeef00 msgr2=0x56148aaead30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.765+0000 7fafde26c740 1 --2- 172.21.15.84:0/2120361328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaeef00 0x56148aaead30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fafd0009610 tx=0x7fafd00d28a0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.765+0000 7fafde26c740 1 -- 172.21.15.84:0/2120361328 shutdown_connections 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.765+0000 7fafde26c740 1 --2- 172.21.15.84:0/2120361328 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aafa0e0 0x56148ab11bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.765+0000 7fafde26c740 1 --2- 172.21.15.84:0/2120361328 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aaef820 0x56148aaf18c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.765+0000 7fafde26c740 1 --2- 172.21.15.84:0/2120361328 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaeef00 0x56148aaead30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.765+0000 7fafde26c740 1 -- 172.21.15.84:0/2120361328 >> 172.21.15.84:0/2120361328 conn(0x56148aaeb410 msgr2=0x56148aaf90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:28.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.765+0000 7fafde26c740 1 -- 172.21.15.84:0/2120361328 shutdown_connections 2024-04-26T18:33:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.765+0000 7fafde26c740 1 -- 172.21.15.84:0/2120361328 wait complete. 2024-04-26T18:33:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.765+0000 7fafde26c740 1 -- start start 2024-04-26T18:33:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafde26c740 1 Processor -- start 2024-04-26T18:33:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafde26c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 0x56148aaf3e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafde26c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 0x56148aaf43d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafde26c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aafa0e0 0x56148aaf4910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafde26c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fafd41af9b0 con 0x56148aaef820 2024-04-26T18:33:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafde26c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fafd41af2e0 con 0x56148aaeef00 2024-04-26T18:33:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafde26c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fafd41ac520 con 0x56148aafa0e0 2024-04-26T18:33:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafde26c740 10 client.0 _ll_register_callbacks cb 0x7fafc4015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafd8aec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 0x56148aaf43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafd8aec640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 0x56148aaf43d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47246/0 (socket says 172.21.15.84:47246) 2024-04-26T18:33:28.771 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafd8aec640 1 -- 172.21.15.84:0/4084767054 learned_addr learned my addr 172.21.15.84:0/4084767054 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafd92ed640 1 --2- 172.21.15.84:0/4084767054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 0x56148aaf3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafd9aee640 1 --2- 172.21.15.84:0/4084767054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aafa0e0 0x56148aaf4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafd8aec640 1 -- 172.21.15.84:0/4084767054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aafa0e0 msgr2=0x56148aaf4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafd8aec640 1 --2- 172.21.15.84:0/4084767054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aafa0e0 0x56148aaf4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafd8aec640 1 -- 172.21.15.84:0/4084767054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 msgr2=0x56148aaf3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafd8aec640 1 --2- 172.21.15.84:0/4084767054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 0x56148aaf3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.766+0000 7fafd8aec640 1 -- 172.21.15.84:0/4084767054 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56148ab6f640 con 0x56148aaef820 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafd8aec640 1 --2- 172.21.15.84:0/4084767054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 0x56148aaf43d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fafc4014040 tx=0x7fafc4004fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2f84a5a29e08f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafbcff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafbcff9640 1 -- 172.21.15.84:0/4084767054 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fafc4011860 con 0x56148aaef820 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafbcff9640 1 -- 172.21.15.84:0/4084767054 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fafc4011a00 con 0x56148aaef820 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafde26c740 1 -- 172.21.15.84:0/4084767054 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fafd009c320 con 0x56148aaef820 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafbcff9640 1 -- 172.21.15.84:0/4084767054 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fafc403b050 con 0x56148aaef820 2024-04-26T18:33:28.772 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafde26c740 10 client.16119 fetch_fsmap learned FSMap version 29 2024-04-26T18:33:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafde26c740 1 -- 172.21.15.84:0/4084767054 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56148ab708a0 con 0x56148aaef820 2024-04-26T18:33:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafbcff9640 1 -- 172.21.15.84:0/4084767054 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fafd009c320 con 0x56148aaef820 2024-04-26T18:33:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafbcff9640 1 -- 172.21.15.84:0/4084767054 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fafc40107d0 con 0x56148aaef820 2024-04-26T18:33:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafde26c740 10 client.16119 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:33:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafde26c740 10 client.16119 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafde26c740 20 client.16119 populate_metadata read hostname 'smithi084' 2024-04-26T18:33:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafde26c740 1 --2- 172.21.15.84:0/4084767054 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56148ab71070 0x56148ab73490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafde26c740 4 client.16119 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:33:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.767+0000 7fafde26c740 1 -- 172.21.15.84:0/4084767054 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56148aa5cee0 con 0x56148ab71070 2024-04-26T18:33:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.768+0000 7fafd92ed640 1 --2- 172.21.15.84:0/4084767054 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56148ab71070 0x56148ab73490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.768+0000 7fafbcff9640 10 client.16119 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:28.773 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.768+0000 7fafd92ed640 1 --2- 172.21.15.84:0/4084767054 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56148ab71070 0x56148ab73490 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fc7e877dbc0f9b7 server_cookie=b8b52a0418e0fbcf in_seq=0 out_seq=0 2024-04-26T18:33:28.783 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafbcff9640 1 -- 172.21.15.84:0/4084767054 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56148aa5cee0 con 0x56148ab71070 2024-04-26T18:33:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafbcff9640 10 client.16119 handle_command_reply: tid=0 2024-04-26T18:33:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafde26c740 1 client.16119 shutdown 2024-04-26T18:33:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafde26c740 1 -- 172.21.15.84:0/4084767054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 msgr2=0x56148aaf43d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafde26c740 1 --2- 172.21.15.84:0/4084767054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 0x56148aaf43d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fafc4014040 tx=0x7fafc4004fd0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafde26c740 1 -- 172.21.15.84:0/4084767054 shutdown_connections 2024-04-26T18:33:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafde26c740 1 --2- 172.21.15.84:0/4084767054 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56148ab71070 0x56148ab73490 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafde26c740 1 --2- 172.21.15.84:0/4084767054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56148aafa0e0 0x56148aaf4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafde26c740 1 --2- 172.21.15.84:0/4084767054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56148aaef820 0x56148aaf43d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafde26c740 1 --2- 172.21.15.84:0/4084767054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56148aaeef00 0x56148aaf3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafbcff9640 0 client.16119 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafde26c740 1 -- 172.21.15.84:0/4084767054 >> 172.21.15.84:0/4084767054 conn(0x56148aaeb410 msgr2=0x56148aaf90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafde26c740 1 -- 172.21.15.84:0/4084767054 shutdown_connections 2024-04-26T18:33:28.784 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafde26c740 1 -- 172.21.15.84:0/4084767054 wait complete. 2024-04-26T18:33:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.783+0000 7fafde26c740 20 client.16119 trim_cache size 0 max 16384 2024-04-26T18:33:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:33:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:33:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:33:28.785 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:33:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.784+0000 7fafdb551640 1 -- 172.21.15.84:0/1588285746 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fafc00a3dc0 msgr2=0x7fafc00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.784+0000 7fafdb551640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fafc00a3dc0 0x7fafc00a6280 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fafcc005cc0 tx=0x7fafcc005fd0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.784+0000 7fafdb551640 1 -- 172.21.15.84:0/1588285746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd4103b60 msgr2=0x7fafd4199000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.784+0000 7fafdb551640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd4103b60 0x7fafd4199000 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fafd000d770 tx=0x7fafd000dc30 comp rx=0 tx=0).stop 2024-04-26T18:33:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.784+0000 7fafdb551640 1 -- 172.21.15.84:0/1588285746 shutdown_connections 2024-04-26T18:33:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.784+0000 7fafdb551640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fafc00a3dc0 0x7fafc00a6280 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.784+0000 7fafdb551640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fafd410cf30 0x7fafd419d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.786 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.784+0000 7fafdb551640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fafd4104480 0x7fafd4199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.784+0000 7fafdb551640 1 --2- 172.21.15.84:0/1588285746 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fafd4103b60 0x7fafd4199000 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.784+0000 7fafdb551640 1 -- 172.21.15.84:0/1588285746 >> 172.21.15.84:0/1588285746 conn(0x7fafd40789b0 msgr2=0x7fafd40fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.784+0000 7fafdb551640 1 -- 172.21.15.84:0/1588285746 shutdown_connections 2024-04-26T18:33:28.787 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:28.784+0000 7fafdb551640 1 -- 172.21.15.84:0/1588285746 wait complete. 2024-04-26T18:33:29.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:28 smithi084 ceph-mon[29141]: pgmap v168: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-26T18:33:29.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:28 smithi103 ceph-mon[32727]: pgmap v168: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-26T18:33:29.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:28 smithi174 ceph-mon[32705]: pgmap v168: 97 pgs: 97 active+clean; 4.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 26 KiB/s wr, 14 op/s 2024-04-26T18:33:31.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:30 smithi084 ceph-mon[29141]: pgmap v169: 97 pgs: 97 active+clean; 4.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-26T18:33:31.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:30 smithi103 ceph-mon[32727]: pgmap v169: 97 pgs: 97 active+clean; 4.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-26T18:33:31.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:30 smithi174 ceph-mon[32705]: pgmap v169: 97 pgs: 97 active+clean; 4.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-26T18:33:33.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:32 smithi084 ceph-mon[29141]: pgmap v170: 97 pgs: 97 active+clean; 4.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-26T18:33:33.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:32 smithi103 ceph-mon[32727]: pgmap v170: 97 pgs: 97 active+clean; 4.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-26T18:33:33.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:32 smithi174 ceph-mon[32705]: pgmap v170: 97 pgs: 97 active+clean; 4.7 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-26T18:33:35.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:34 smithi084 ceph-mon[29141]: pgmap v171: 97 pgs: 97 active+clean; 4.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-26T18:33:35.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:34 smithi103 ceph-mon[32727]: pgmap v171: 97 pgs: 97 active+clean; 4.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-26T18:33:35.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:34 smithi174 ceph-mon[32705]: pgmap v171: 97 pgs: 97 active+clean; 4.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-26T18:33:37.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:36 smithi084 ceph-mon[29141]: pgmap v172: 97 pgs: 97 active+clean; 4.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-26T18:33:37.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:36 smithi103 ceph-mon[32727]: pgmap v172: 97 pgs: 97 active+clean; 4.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-26T18:33:37.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:36 smithi174 ceph-mon[32705]: pgmap v172: 97 pgs: 97 active+clean; 4.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-26T18:33:39.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:38 smithi084 ceph-mon[29141]: pgmap v173: 97 pgs: 97 active+clean; 4.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-26T18:33:39.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:38 smithi103 ceph-mon[32727]: pgmap v173: 97 pgs: 97 active+clean; 4.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-26T18:33:39.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:38 smithi174 ceph-mon[32705]: pgmap v173: 97 pgs: 97 active+clean; 4.9 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-26T18:33:41.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:40 smithi084 ceph-mon[29141]: pgmap v174: 97 pgs: 97 active+clean; 5.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-26T18:33:41.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:40 smithi103 ceph-mon[32727]: pgmap v174: 97 pgs: 97 active+clean; 5.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-26T18:33:41.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:40 smithi174 ceph-mon[32705]: pgmap v174: 97 pgs: 97 active+clean; 5.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-26T18:33:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.880+0000 7fd9f74b0640 1 -- 172.21.15.84:0/2300286886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9f0101b80 msgr2=0x7fd9f0102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.880+0000 7fd9f74b0640 1 --2- 172.21.15.84:0/2300286886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9f0101b80 0x7fd9f0102000 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009900 tx=0x7fd9e002a410 comp rx=0 tx=0).stop 2024-04-26T18:33:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 -- 172.21.15.84:0/2300286886 shutdown_connections 2024-04-26T18:33:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 --2- 172.21.15.84:0/2300286886 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9f0102540 0x7fd9f010e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 --2- 172.21.15.84:0/2300286886 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9f0101b80 0x7fd9f0102000 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 --2- 172.21.15.84:0/2300286886 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9f01067e0 0x7fd9f0106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 -- 172.21.15.84:0/2300286886 >> 172.21.15.84:0/2300286886 conn(0x7fd9f0077630 msgr2=0x7fd9f0077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 -- 172.21.15.84:0/2300286886 shutdown_connections 2024-04-26T18:33:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 -- 172.21.15.84:0/2300286886 wait complete. 2024-04-26T18:33:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 Processor -- start 2024-04-26T18:33:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 -- start start 2024-04-26T18:33:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9f0101b80 0x7fd9f00ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9f0102540 0x7fd9f00ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9f01067e0 0x7fd9f01000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f0111b20 con 0x7fd9f0101b80 2024-04-26T18:33:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f01119a0 con 0x7fd9f01067e0 2024-04-26T18:33:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.881+0000 7fd9f74b0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f0111ca0 con 0x7fd9f0102540 2024-04-26T18:33:41.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f524c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9f0101b80 0x7fd9f00ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f524c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9f0101b80 0x7fd9f00ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35728/0 (socket says 172.21.15.84:35728) 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f524c640 1 -- 172.21.15.84:0/2273251372 learned_addr learned my addr 172.21.15.84:0/2273251372 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f4a4b640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9f0102540 0x7fd9f00ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f5a4d640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9f01067e0 0x7fd9f01000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f524c640 1 -- 172.21.15.84:0/2273251372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9f0102540 msgr2=0x7fd9f00ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f524c640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9f0102540 0x7fd9f00ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f524c640 1 -- 172.21.15.84:0/2273251372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9f01067e0 msgr2=0x7fd9f01000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f524c640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9f01067e0 0x7fd9f01000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f524c640 1 -- 172.21.15.84:0/2273251372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9f01a7570 con 0x7fd9f0101b80 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f5a4d640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9f01067e0 0x7fd9f01000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f524c640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9f0101b80 0x7fd9f00ff670 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00d950 tx=0x7fd9ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9e67fc640 1 -- 172.21.15.84:0/2273251372 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec017070 con 0x7fd9f0101b80 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9e67fc640 1 -- 172.21.15.84:0/2273251372 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ec009c30 con 0x7fd9f0101b80 2024-04-26T18:33:41.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f74b0640 1 -- 172.21.15.84:0/2273251372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9f01a7860 con 0x7fd9f0101b80 2024-04-26T18:33:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9e67fc640 1 -- 172.21.15.84:0/2273251372 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec00f620 con 0x7fd9f0101b80 2024-04-26T18:33:41.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.882+0000 7fd9f74b0640 1 -- 172.21.15.84:0/2273251372 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9f01a7d80 con 0x7fd9f0101b80 2024-04-26T18:33:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.884+0000 7fd9e67fc640 1 -- 172.21.15.84:0/2273251372 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fd9ec010460 con 0x7fd9f0101b80 2024-04-26T18:33:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.885+0000 7fd9fa1ca740 1 Processor -- start 2024-04-26T18:33:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.885+0000 7fd9fa1ca740 1 -- start start 2024-04-26T18:33:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.885+0000 7fd9fa1ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a14f00 0x561666a152e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.885+0000 7fd9fa1ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a15820 0x561666a1fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.885+0000 7fd9fa1ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 0x561666a224d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.885+0000 7fd9fa1ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56166697b460 con 0x561666a200e0 2024-04-26T18:33:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.885+0000 7fd9fa1ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561666935590 con 0x561666a14f00 2024-04-26T18:33:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.885+0000 7fd9fa1ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561666977420 con 0x561666a15820 2024-04-26T18:33:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.885+0000 7fd9e67fc640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd9dc0a3dc0 0x7fd9dc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.885+0000 7fd9f4a4b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 0x561666a224d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.885+0000 7fd9f4a4b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 0x561666a224d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35736/0 (socket says 172.21.15.84:35736) 2024-04-26T18:33:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.885+0000 7fd9f4a4b640 1 -- 172.21.15.84:0/3449705039 learned_addr learned my addr 172.21.15.84:0/3449705039 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.885+0000 7fd9f5a4d640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd9dc0a3dc0 0x7fd9dc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.886+0000 7fd9e67fc640 1 -- 172.21.15.84:0/2273251372 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fd9ec0d04c0 con 0x7fd9f0101b80 2024-04-26T18:33:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.886+0000 7fd9f5a4d640 1 --2- 172.21.15.84:0/3449705039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a14f00 0x561666a152e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.886+0000 7fd9f524c640 1 --2- 172.21.15.84:0/3449705039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a15820 0x561666a1fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.886+0000 7fd9f4a4b640 1 -- 172.21.15.84:0/3449705039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a15820 msgr2=0x561666a1fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.886+0000 7fd9f4a4b640 1 --2- 172.21.15.84:0/3449705039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a15820 0x561666a1fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.886+0000 7fd9f4a4b640 1 -- 172.21.15.84:0/3449705039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a14f00 msgr2=0x561666a152e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.886+0000 7fd9f4a4b640 1 --2- 172.21.15.84:0/3449705039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a14f00 0x561666a152e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.886+0000 7fd9f4a4b640 1 -- 172.21.15.84:0/3449705039 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5616669420d0 con 0x561666a200e0 2024-04-26T18:33:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.886+0000 7fd9f5a4d640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd9dc0a3dc0 0x7fd9dc0a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800afd0 tx=0x7fd9e8014ab0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:41.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.886+0000 7fd9f4a4b640 1 --2- 172.21.15.84:0/3449705039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 0x561666a224d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009900 tx=0x7fd9e0004390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeab36d19a8ed402 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.886+0000 7fd9db7fe640 1 -- 172.21.15.84:0/3449705039 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0035030 con 0x561666a200e0 2024-04-26T18:33:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.886+0000 7fd9db7fe640 1 -- 172.21.15.84:0/3449705039 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9e00046c0 con 0x561666a200e0 2024-04-26T18:33:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.886+0000 7fd9db7fe640 1 -- 172.21.15.84:0/3449705039 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0005d50 con 0x561666a200e0 2024-04-26T18:33:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.887+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/3449705039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 msgr2=0x561666a224d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.887+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/3449705039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 0x561666a224d0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd9e0009900 tx=0x7fd9e0004390 comp rx=0 tx=0).stop 2024-04-26T18:33:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.887+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/3449705039 shutdown_connections 2024-04-26T18:33:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.887+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/3449705039 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 0x561666a224d0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.887+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/3449705039 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a15820 0x561666a1fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.887+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/3449705039 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a14f00 0x561666a152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.887+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/3449705039 >> 172.21.15.84:0/3449705039 conn(0x561666a11410 msgr2=0x561666a117f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.887+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/3449705039 shutdown_connections 2024-04-26T18:33:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.887+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/3449705039 wait complete. 2024-04-26T18:33:41.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.887+0000 7fd9fa1ca740 1 -- start start 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.887+0000 7fd9fa1ca740 1 Processor -- start 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9fa1ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a14f00 0x561666a1d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9fa1ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a15820 0x561666a1d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9fa1ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 0x561666a1db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9fa1ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561666a22cb0 con 0x561666a200e0 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9fa1ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561666902b60 con 0x561666a15820 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9fa1ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561666a22e30 con 0x561666a14f00 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9fa1ca740 10 client.0 _ll_register_callbacks cb 0x5616669272b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9f5a4d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 0x561666a1db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9f5a4d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 0x561666a1db00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35744/0 (socket says 172.21.15.84:35744) 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9f5a4d640 1 -- 172.21.15.84:0/2830954808 learned_addr learned my addr 172.21.15.84:0/2830954808 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9f524c640 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a14f00 0x561666a1d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9f4a4b640 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a15820 0x561666a1d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9f5a4d640 1 -- 172.21.15.84:0/2830954808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a14f00 msgr2=0x561666a1d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9f5a4d640 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a14f00 0x561666a1d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9f5a4d640 1 -- 172.21.15.84:0/2830954808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a15820 msgr2=0x561666a1d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9f5a4d640 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a15820 0x561666a1d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9f5a4d640 1 -- 172.21.15.84:0/2830954808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56166668d210 con 0x561666a200e0 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9f4a4b640 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a15820 0x561666a1d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9f5a4d640 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 0x561666a1db00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80119e0 tx=0x7fd9e8005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e96d98fc8f05720 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.888+0000 7fd9d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9d8ff9640 1 -- 172.21.15.84:0/2830954808 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8023070 con 0x561666a200e0 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9d8ff9640 1 -- 172.21.15.84:0/2830954808 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fd9e8012de0 con 0x561666a200e0 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/2830954808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9e00046c0 con 0x561666a200e0 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9d8ff9640 1 -- 172.21.15.84:0/2830954808 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9e802f050 con 0x561666a200e0 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9fa1ca740 10 client.16131 fetch_fsmap learned FSMap version 29 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/2830954808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561666759620 con 0x561666a200e0 2024-04-26T18:33:41.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9d8ff9640 1 -- 172.21.15.84:0/2830954808 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9e00046c0 con 0x561666a200e0 2024-04-26T18:33:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9d8ff9640 1 -- 172.21.15.84:0/2830954808 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd9e801e070 con 0x561666a200e0 2024-04-26T18:33:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9fa1ca740 10 client.16131 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:33:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9fa1ca740 10 client.16131 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9fa1ca740 20 client.16131 populate_metadata read hostname 'smithi084' 2024-04-26T18:33:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561666a302a0 0x561666a32710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9fa1ca740 4 client.16131 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:33:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.889+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/2830954808 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561666989070 con 0x561666a302a0 2024-04-26T18:33:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.890+0000 7fd9f524c640 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561666a302a0 0x561666a32710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.890+0000 7fd9f524c640 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561666a302a0 0x561666a32710 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b7a7895fd1cf161 server_cookie=e116ecb6cbc58946 in_seq=0 out_seq=0 2024-04-26T18:33:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.890+0000 7fd9d8ff9640 10 client.16131 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9d8ff9640 1 -- 172.21.15.84:0/2830954808 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561666989070 con 0x561666a302a0 2024-04-26T18:33:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9d8ff9640 10 client.16131 handle_command_reply: tid=0 2024-04-26T18:33:41.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9fa1ca740 1 client.16131 shutdown 2024-04-26T18:33:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/2830954808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 msgr2=0x561666a1db00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 0x561666a1db00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fd9e80119e0 tx=0x7fd9e8005f50 comp rx=0 tx=0).stop 2024-04-26T18:33:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/2830954808 shutdown_connections 2024-04-26T18:33:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561666a302a0 0x561666a32710 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a200e0 0x561666a1db00 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a15820 0x561666a1d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/2830954808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a14f00 0x561666a1d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9d8ff9640 0 client.16131 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/2830954808 >> 172.21.15.84:0/2830954808 conn(0x561666a11410 msgr2=0x561666a17140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/2830954808 shutdown_connections 2024-04-26T18:33:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/2830954808 wait complete. 2024-04-26T18:33:41.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.891+0000 7fd9fa1ca740 20 client.16131 trim_cache size 0 max 16384 2024-04-26T18:33:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.905+0000 7fd9fa1ca740 1 Processor -- start 2024-04-26T18:33:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9fa1ca740 1 -- start start 2024-04-26T18:33:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9fa1ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a14f00 0x561666a2bb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9fa1ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a15820 0x561666a178c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9fa1ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a200e0 0x561666a38020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9fa1ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f01b06b0 con 0x561666a15820 2024-04-26T18:33:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9fa1ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f01b21f0 con 0x561666a200e0 2024-04-26T18:33:41.911 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9fa1ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f01b0d80 con 0x561666a14f00 2024-04-26T18:33:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9f4a4b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a15820 0x561666a178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9f4a4b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a15820 0x561666a178c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35758/0 (socket says 172.21.15.84:35758) 2024-04-26T18:33:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9f4a4b640 1 -- 172.21.15.84:0/1088258571 learned_addr learned my addr 172.21.15.84:0/1088258571 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9f524c640 1 --2- 172.21.15.84:0/1088258571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a14f00 0x561666a2bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9f5a4d640 1 --2- 172.21.15.84:0/1088258571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a200e0 0x561666a38020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9f4a4b640 1 -- 172.21.15.84:0/1088258571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a14f00 msgr2=0x561666a2bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9f4a4b640 1 --2- 172.21.15.84:0/1088258571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a14f00 0x561666a2bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9f4a4b640 1 -- 172.21.15.84:0/1088258571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a200e0 msgr2=0x561666a38020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9f4a4b640 1 --2- 172.21.15.84:0/1088258571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a200e0 0x561666a38020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9f4a4b640 1 -- 172.21.15.84:0/1088258571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561666a11d90 con 0x561666a15820 2024-04-26T18:33:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9f524c640 1 --2- 172.21.15.84:0/1088258571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a14f00 0x561666a2bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9f5a4d640 1 --2- 172.21.15.84:0/1088258571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a200e0 0x561666a38020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:41.912 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9f4a4b640 1 --2- 172.21.15.84:0/1088258571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a15820 0x561666a178c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd9e002aa00 tx=0x7fd9e0004580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c229e7f1952e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9daffd640 1 -- 172.21.15.84:0/1088258571 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e0035030 con 0x561666a15820 2024-04-26T18:33:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9daffd640 1 -- 172.21.15.84:0/1088258571 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9e004b070 con 0x561666a15820 2024-04-26T18:33:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.906+0000 7fd9daffd640 1 -- 172.21.15.84:0/1088258571 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9e003b690 con 0x561666a15820 2024-04-26T18:33:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.907+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/1088258571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a15820 msgr2=0x561666a178c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.907+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/1088258571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a15820 0x561666a178c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7fd9e002aa00 tx=0x7fd9e0004580 comp rx=0 tx=0).stop 2024-04-26T18:33:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.907+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/1088258571 shutdown_connections 2024-04-26T18:33:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.907+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/1088258571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a200e0 0x561666a38020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.907+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/1088258571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a15820 0x561666a178c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.907+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/1088258571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a14f00 0x561666a2bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.907+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/1088258571 >> 172.21.15.84:0/1088258571 conn(0x561666a11410 msgr2=0x561666a1f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.907+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/1088258571 shutdown_connections 2024-04-26T18:33:41.913 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.907+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/1088258571 wait complete. 2024-04-26T18:33:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.907+0000 7fd9fa1ca740 1 -- start start 2024-04-26T18:33:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9fa1ca740 1 Processor -- start 2024-04-26T18:33:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9fa1ca740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a14f00 0x561666a28080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9fa1ca740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a15820 0x561666a285c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9fa1ca740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a200e0 0x561666a25050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9fa1ca740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f01b1b20 con 0x561666a14f00 2024-04-26T18:33:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9fa1ca740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f01b1450 con 0x561666a15820 2024-04-26T18:33:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9fa1ca740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd9f01ae690 con 0x561666a200e0 2024-04-26T18:33:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9fa1ca740 10 client.0 _ll_register_callbacks cb 0x7fd9f007ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9f524c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a14f00 0x561666a28080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9f524c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a14f00 0x561666a28080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35770/0 (socket says 172.21.15.84:35770) 2024-04-26T18:33:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9f524c640 1 -- 172.21.15.84:0/4043119412 learned_addr learned my addr 172.21.15.84:0/4043119412 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:41.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9f5a4d640 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a200e0 0x561666a25050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9f4a4b640 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a15820 0x561666a285c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9f524c640 1 -- 172.21.15.84:0/4043119412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a200e0 msgr2=0x561666a25050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9f524c640 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a200e0 0x561666a25050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9f524c640 1 -- 172.21.15.84:0/4043119412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a15820 msgr2=0x561666a285c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9f524c640 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a15820 0x561666a285c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9f524c640 1 -- 172.21.15.84:0/4043119412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561666a25880 con 0x561666a14f00 2024-04-26T18:33:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.908+0000 7fd9f5a4d640 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a200e0 0x561666a25050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:33:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9f524c640 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a14f00 0x561666a28080 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0ce200 tx=0x7fd9ec00d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f656d61da4f303 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:33:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9d8ff9640 1 -- 172.21.15.84:0/4043119412 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ec020040 con 0x561666a14f00 2024-04-26T18:33:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9d8ff9640 1 -- 172.21.15.84:0/4043119412 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7fd9ec098ac0 con 0x561666a14f00 2024-04-26T18:33:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/4043119412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9e004b070 con 0x561666a14f00 2024-04-26T18:33:41.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9d8ff9640 1 -- 172.21.15.84:0/4043119412 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9ec0a0800 con 0x561666a14f00 2024-04-26T18:33:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9fa1ca740 10 client.16137 fetch_fsmap learned FSMap version 29 2024-04-26T18:33:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/4043119412 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561666a97360 con 0x561666a14f00 2024-04-26T18:33:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9d8ff9640 1 -- 172.21.15.84:0/4043119412 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9e004b070 con 0x561666a14f00 2024-04-26T18:33:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9d8ff9640 1 -- 172.21.15.84:0/4043119412 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd9ec0b5bd0 con 0x561666a14f00 2024-04-26T18:33:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9fa1ca740 10 client.16137 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:33:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9fa1ca740 10 client.16137 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9fa1ca740 20 client.16137 populate_metadata read hostname 'smithi084' 2024-04-26T18:33:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561666a97a30 0x561666a99e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9fa1ca740 4 client.16137 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:33:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/4043119412 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561666982ee0 con 0x561666a97a30 2024-04-26T18:33:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.909+0000 7fd9f4a4b640 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561666a97a30 0x561666a99e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.910+0000 7fd9f4a4b640 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561666a97a30 0x561666a99e60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d739e49456b77daf server_cookie=97f52ba06234254f in_seq=0 out_seq=0 2024-04-26T18:33:41.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.910+0000 7fd9d8ff9640 10 client.16137 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9d8ff9640 1 -- 172.21.15.84:0/4043119412 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561666982ee0 con 0x561666a97a30 2024-04-26T18:33:41.924 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9d8ff9640 10 client.16137 handle_command_reply: tid=0 2024-04-26T18:33:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9fa1ca740 1 client.16137 shutdown 2024-04-26T18:33:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/4043119412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a14f00 msgr2=0x561666a28080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a14f00 0x561666a28080 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0ce200 tx=0x7fd9ec00d8d0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/4043119412 shutdown_connections 2024-04-26T18:33:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561666a97a30 0x561666a99e60 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561666a200e0 0x561666a25050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561666a15820 0x561666a285c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9fa1ca740 1 --2- 172.21.15.84:0/4043119412 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561666a14f00 0x561666a28080 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9d8ff9640 0 client.16137 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/4043119412 >> 172.21.15.84:0/4043119412 conn(0x561666a11410 msgr2=0x561666a0fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/4043119412 shutdown_connections 2024-04-26T18:33:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9fa1ca740 1 -- 172.21.15.84:0/4043119412 wait complete. 2024-04-26T18:33:41.925 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.924+0000 7fd9fa1ca740 20 client.16137 trim_cache size 0 max 16384 2024-04-26T18:33:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:33:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:33:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:33:41.926 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:33:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.925+0000 7fd9f74b0640 1 -- 172.21.15.84:0/2273251372 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd9dc0a3dc0 msgr2=0x7fd9dc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.925+0000 7fd9f74b0640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd9dc0a3dc0 0x7fd9dc0a6280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd9e800afd0 tx=0x7fd9e8014ab0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.925+0000 7fd9f74b0640 1 -- 172.21.15.84:0/2273251372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9f0101b80 msgr2=0x7fd9f00ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.925+0000 7fd9f74b0640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9f0101b80 0x7fd9f00ff670 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec00d950 tx=0x7fd9ec00de10 comp rx=0 tx=0).stop 2024-04-26T18:33:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.925+0000 7fd9f74b0640 1 -- 172.21.15.84:0/2273251372 shutdown_connections 2024-04-26T18:33:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.925+0000 7fd9f74b0640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd9dc0a3dc0 0x7fd9dc0a6280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.927 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.925+0000 7fd9f74b0640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd9f01067e0 0x7fd9f01000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.925+0000 7fd9f74b0640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd9f0102540 0x7fd9f00ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.925+0000 7fd9f74b0640 1 --2- 172.21.15.84:0/2273251372 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd9f0101b80 0x7fd9f00ff670 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.925+0000 7fd9f74b0640 1 -- 172.21.15.84:0/2273251372 >> 172.21.15.84:0/2273251372 conn(0x7fd9f0077630 msgr2=0x7fd9f006b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.925+0000 7fd9f74b0640 1 -- 172.21.15.84:0/2273251372 shutdown_connections 2024-04-26T18:33:41.928 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:41.925+0000 7fd9f74b0640 1 -- 172.21.15.84:0/2273251372 wait complete. 2024-04-26T18:33:43.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:42 smithi084 ceph-mon[29141]: pgmap v175: 97 pgs: 97 active+clean; 5.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-26T18:33:43.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:42 smithi103 ceph-mon[32727]: pgmap v175: 97 pgs: 97 active+clean; 5.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-26T18:33:43.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:42 smithi174 ceph-mon[32705]: pgmap v175: 97 pgs: 97 active+clean; 5.0 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-04-26T18:33:45.098 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:44 smithi084 ceph-mon[29141]: pgmap v176: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 24 op/s 2024-04-26T18:33:45.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:44 smithi103 ceph-mon[32727]: pgmap v176: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 24 op/s 2024-04-26T18:33:45.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:44 smithi174 ceph-mon[32705]: pgmap v176: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 24 op/s 2024-04-26T18:33:47.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:46 smithi084 ceph-mon[29141]: pgmap v177: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-26T18:33:47.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:46 smithi103 ceph-mon[32727]: pgmap v177: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-26T18:33:47.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:46 smithi174 ceph-mon[32705]: pgmap v177: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-26T18:33:48.991 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:48 smithi174 ceph-mon[32705]: pgmap v178: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-26T18:33:49.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:48 smithi084 ceph-mon[29141]: pgmap v178: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-26T18:33:49.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:48 smithi103 ceph-mon[32727]: pgmap v178: 97 pgs: 97 active+clean; 5.2 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 15 op/s 2024-04-26T18:33:51.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:50 smithi084 ceph-mon[29141]: pgmap v179: 97 pgs: 97 active+clean; 5.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-26T18:33:51.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:50 smithi103 ceph-mon[32727]: pgmap v179: 97 pgs: 97 active+clean; 5.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-26T18:33:51.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:50 smithi174 ceph-mon[32705]: pgmap v179: 97 pgs: 97 active+clean; 5.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 46 KiB/s wr, 22 op/s 2024-04-26T18:33:53.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:52 smithi084 ceph-mon[29141]: pgmap v180: 97 pgs: 97 active+clean; 5.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-26T18:33:53.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:52 smithi103 ceph-mon[32727]: pgmap v180: 97 pgs: 97 active+clean; 5.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-26T18:33:53.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:52 smithi174 ceph-mon[32705]: pgmap v180: 97 pgs: 97 active+clean; 5.4 MiB data, 398 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 16 op/s 2024-04-26T18:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.020+0000 7f8abab9d640 1 -- 172.21.15.84:0/2207726421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4107820 msgr2=0x7f8ab4107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:55.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.020+0000 7f8abab9d640 1 --2- 172.21.15.84:0/2207726421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4107820 0x7f8ab4107c00 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009930 tx=0x7f8aa802a420 comp rx=0 tx=0).stop 2024-04-26T18:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.020+0000 7f8abab9d640 1 -- 172.21.15.84:0/2207726421 shutdown_connections 2024-04-26T18:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.020+0000 7f8abab9d640 1 --2- 172.21.15.84:0/2207726421 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab4102220 0x7f8ab410e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.020+0000 7f8abab9d640 1 --2- 172.21.15.84:0/2207726421 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ab4101860 0x7f8ab4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.020+0000 7f8abab9d640 1 --2- 172.21.15.84:0/2207726421 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4107820 0x7f8ab4107c00 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.020+0000 7f8abab9d640 1 -- 172.21.15.84:0/2207726421 >> 172.21.15.84:0/2207726421 conn(0x7f8ab40789b0 msgr2=0x7f8ab4078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.020+0000 7f8abab9d640 1 -- 172.21.15.84:0/2207726421 shutdown_connections 2024-04-26T18:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.020+0000 7f8abab9d640 1 -- 172.21.15.84:0/2207726421 wait complete. 2024-04-26T18:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.020+0000 7f8abab9d640 1 Processor -- start 2024-04-26T18:33:55.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8abab9d640 1 -- start start 2024-04-26T18:33:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8abab9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab4101860 0x7f8ab419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8abab9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4102220 0x7f8ab419b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8abab9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ab4107820 0x7f8ab419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8abab9d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111b60 con 0x7f8ab4102220 2024-04-26T18:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8abab9d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41119e0 con 0x7f8ab4101860 2024-04-26T18:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8abab9d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab4111ce0 con 0x7f8ab4107820 2024-04-26T18:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4102220 0x7f8ab419b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4102220 0x7f8ab419b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59646/0 (socket says 172.21.15.84:59646) 2024-04-26T18:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8ab8939640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab4101860 0x7f8ab419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8ab3fff640 1 -- 172.21.15.84:0/840474867 learned_addr learned my addr 172.21.15.84:0/840474867 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8ab913a640 1 --2- 172.21.15.84:0/840474867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ab4107820 0x7f8ab419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8ab3fff640 1 -- 172.21.15.84:0/840474867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ab4107820 msgr2=0x7f8ab419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8ab3fff640 1 --2- 172.21.15.84:0/840474867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ab4107820 0x7f8ab419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8ab3fff640 1 -- 172.21.15.84:0/840474867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab4101860 msgr2=0x7f8ab419b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:55.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8ab3fff640 1 --2- 172.21.15.84:0/840474867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab4101860 0x7f8ab419b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8ab3fff640 1 -- 172.21.15.84:0/840474867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ab41a03d0 con 0x7f8ab4102220 2024-04-26T18:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.021+0000 7f8ab3fff640 1 --2- 172.21.15.84:0/840474867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4102220 0x7f8ab419b970 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000b4a0 tx=0x7f8aa000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.022+0000 7f8ab1ffb640 1 -- 172.21.15.84:0/840474867 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0014070 con 0x7f8ab4102220 2024-04-26T18:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.022+0000 7f8ab1ffb640 1 -- 172.21.15.84:0/840474867 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aa0010ac0 con 0x7f8ab4102220 2024-04-26T18:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.022+0000 7f8abab9d640 1 -- 172.21.15.84:0/840474867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ab41a06c0 con 0x7f8ab4102220 2024-04-26T18:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.022+0000 7f8ab1ffb640 1 -- 172.21.15.84:0/840474867 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa0017c30 con 0x7f8ab4102220 2024-04-26T18:33:55.025 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.022+0000 7f8abab9d640 1 -- 172.21.15.84:0/840474867 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ab41a7f50 con 0x7f8ab4102220 2024-04-26T18:33:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.024+0000 7f8ab1ffb640 1 -- 172.21.15.84:0/840474867 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8aa0017070 con 0x7f8ab4102220 2024-04-26T18:33:55.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.024+0000 7f8abd994740 1 Processor -- start 2024-04-26T18:33:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.024+0000 7f8abd994740 1 -- start start 2024-04-26T18:33:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab1ffb640 1 --2- 172.21.15.84:0/840474867 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8aa40a3dc0 0x7f8aa40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8abd994740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed3a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8abd994740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 0x56142ed44ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8abd994740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 0x56142ed474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8abd994740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56142eca0460 con 0x56142ed39f00 2024-04-26T18:33:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8abd994740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56142ec5a590 con 0x56142ed450e0 2024-04-26T18:33:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8abd994740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56142ec9c420 con 0x56142ed3a820 2024-04-26T18:33:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab8939640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed3a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab8939640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed3a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59660/0 (socket says 172.21.15.84:59660) 2024-04-26T18:33:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab913a640 1 --2- 172.21.15.84:0/840474867 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8aa40a3dc0 0x7f8aa40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab8939640 1 -- 172.21.15.84:0/1484287454 learned_addr learned my addr 172.21.15.84:0/1484287454 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:55.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab1ffb640 1 -- 172.21.15.84:0/840474867 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8aa00cf9e0 con 0x7f8ab4102220 2024-04-26T18:33:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab3fff640 1 --2- 172.21.15.84:0/1484287454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 0x56142ed44ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab8939640 1 -- 172.21.15.84:0/1484287454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 msgr2=0x56142ed44ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab8939640 1 --2- 172.21.15.84:0/1484287454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 0x56142ed44ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab8939640 1 -- 172.21.15.84:0/1484287454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 msgr2=0x56142ed474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab8939640 1 --2- 172.21.15.84:0/1484287454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 0x56142ed474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab8939640 1 -- 172.21.15.84:0/1484287454 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142ec670d0 con 0x56142ed39f00 2024-04-26T18:33:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab913a640 1 --2- 172.21.15.84:0/840474867 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8aa40a3dc0 0x7f8aa40a6280 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009900 tx=0x7f8aac009650 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8ab8939640 1 --2- 172.21.15.84:0/1484287454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed3a2e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009930 tx=0x7f8aa8032d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b2332cb3b2be359 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.025+0000 7f8a9effd640 1 -- 172.21.15.84:0/1484287454 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8035040 con 0x56142ed39f00 2024-04-26T18:33:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.026+0000 7f8a9effd640 1 -- 172.21.15.84:0/1484287454 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aa80316a0 con 0x56142ed39f00 2024-04-26T18:33:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.026+0000 7f8a9effd640 1 -- 172.21.15.84:0/1484287454 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8005640 con 0x56142ed39f00 2024-04-26T18:33:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.026+0000 7f8abd994740 1 -- 172.21.15.84:0/1484287454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 msgr2=0x56142ed3a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:55.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.026+0000 7f8abd994740 1 --2- 172.21.15.84:0/1484287454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed3a2e0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8009930 tx=0x7f8aa8032d80 comp rx=0 tx=0).stop 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.026+0000 7f8abd994740 1 -- 172.21.15.84:0/1484287454 shutdown_connections 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.026+0000 7f8abd994740 1 --2- 172.21.15.84:0/1484287454 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 0x56142ed474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.026+0000 7f8abd994740 1 --2- 172.21.15.84:0/1484287454 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 0x56142ed44ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.026+0000 7f8abd994740 1 --2- 172.21.15.84:0/1484287454 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed3a2e0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.026+0000 7f8abd994740 1 -- 172.21.15.84:0/1484287454 >> 172.21.15.84:0/1484287454 conn(0x56142ed36410 msgr2=0x56142ed367f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.026+0000 7f8abd994740 1 -- 172.21.15.84:0/1484287454 shutdown_connections 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.026+0000 7f8abd994740 1 -- 172.21.15.84:0/1484287454 wait complete. 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.027+0000 7f8abd994740 1 -- start start 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.027+0000 7f8abd994740 1 Processor -- start 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.027+0000 7f8abd994740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed39f00 0x56142ed41f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.027+0000 7f8abd994740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed3a820 0x56142ed424b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.027+0000 7f8abd994740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed450e0 0x56142ed429f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.027+0000 7f8abd994740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56142ed47cb0 con 0x56142ed3a820 2024-04-26T18:33:55.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.027+0000 7f8abd994740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56142ec27b60 con 0x56142ed39f00 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.027+0000 7f8abd994740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56142ed47e30 con 0x56142ed450e0 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.027+0000 7f8abd994740 10 client.0 _ll_register_callbacks cb 0x56142ec4c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.027+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed3a820 0x56142ed424b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.027+0000 7f8ab3fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed3a820 0x56142ed424b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59666/0 (socket says 172.21.15.84:59666) 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.027+0000 7f8ab3fff640 1 -- 172.21.15.84:0/422020869 learned_addr learned my addr 172.21.15.84:0/422020869 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.027+0000 7f8ab8939640 1 --2- 172.21.15.84:0/422020869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed39f00 0x56142ed41f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8ab913a640 1 --2- 172.21.15.84:0/422020869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed450e0 0x56142ed429f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8ab3fff640 1 -- 172.21.15.84:0/422020869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed450e0 msgr2=0x56142ed429f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8ab3fff640 1 --2- 172.21.15.84:0/422020869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed450e0 0x56142ed429f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8ab3fff640 1 -- 172.21.15.84:0/422020869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed39f00 msgr2=0x56142ed41f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8ab3fff640 1 --2- 172.21.15.84:0/422020869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed39f00 0x56142ed41f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8ab3fff640 1 -- 172.21.15.84:0/422020869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56142e9b2210 con 0x56142ed3a820 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8ab3fff640 1 --2- 172.21.15.84:0/422020869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed3a820 0x56142ed424b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0097f90 tx=0x7f8aa00047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79bcdb7554e65595 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:55.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8a93fff640 1 -- 172.21.15.84:0/422020869 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa001e070 con 0x56142ed3a820 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8a93fff640 1 -- 172.21.15.84:0/422020869 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8aa0099e30 con 0x56142ed3a820 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8abd994740 1 -- 172.21.15.84:0/422020869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aa80316a0 con 0x56142ed3a820 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8a93fff640 1 -- 172.21.15.84:0/422020869 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8aa009bb30 con 0x56142ed3a820 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8abd994740 10 client.16149 fetch_fsmap learned FSMap version 29 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8abd994740 1 -- 172.21.15.84:0/422020869 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56142ea7e620 con 0x56142ed3a820 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.028+0000 7f8a93fff640 1 -- 172.21.15.84:0/422020869 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aa80316a0 con 0x56142ed3a820 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.029+0000 7f8a93fff640 1 -- 172.21.15.84:0/422020869 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8aa00981a0 con 0x56142ed3a820 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.029+0000 7f8abd994740 10 client.16149 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.029+0000 7f8abd994740 10 client.16149 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.029+0000 7f8abd994740 20 client.16149 populate_metadata read hostname 'smithi084' 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.029+0000 7f8abd994740 1 --2- 172.21.15.84:0/422020869 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56142ed550a0 0x56142ed574d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.029+0000 7f8abd994740 4 client.16149 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:33:55.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.029+0000 7f8abd994740 1 -- 172.21.15.84:0/422020869 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56142ecae070 con 0x56142ed550a0 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.029+0000 7f8ab8939640 1 --2- 172.21.15.84:0/422020869 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56142ed550a0 0x56142ed574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.029+0000 7f8a93fff640 10 client.16149 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.029+0000 7f8ab8939640 1 --2- 172.21.15.84:0/422020869 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56142ed550a0 0x56142ed574d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da03fb520ec50bd2 server_cookie=ed321c70350d4dd0 in_seq=0 out_seq=0 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.030+0000 7f8a93fff640 1 -- 172.21.15.84:0/422020869 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56142ecae070 con 0x56142ed550a0 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.030+0000 7f8a93fff640 10 client.16149 handle_command_reply: tid=0 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.030+0000 7f8abd994740 1 client.16149 shutdown 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.030+0000 7f8abd994740 1 -- 172.21.15.84:0/422020869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed3a820 msgr2=0x56142ed424b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.030+0000 7f8abd994740 1 --2- 172.21.15.84:0/422020869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed3a820 0x56142ed424b0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f8aa0097f90 tx=0x7f8aa00047d0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.030+0000 7f8abd994740 1 -- 172.21.15.84:0/422020869 shutdown_connections 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.030+0000 7f8abd994740 1 --2- 172.21.15.84:0/422020869 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56142ed550a0 0x56142ed574d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.030+0000 7f8abd994740 1 --2- 172.21.15.84:0/422020869 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed450e0 0x56142ed429f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.030+0000 7f8abd994740 1 --2- 172.21.15.84:0/422020869 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed3a820 0x56142ed424b0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.030+0000 7f8a93fff640 0 client.16149 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:55.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.030+0000 7f8abd994740 1 --2- 172.21.15.84:0/422020869 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed39f00 0x56142ed41f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.030+0000 7f8abd994740 1 -- 172.21.15.84:0/422020869 >> 172.21.15.84:0/422020869 conn(0x56142ed36410 msgr2=0x56142ed3c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.031+0000 7f8abd994740 1 -- 172.21.15.84:0/422020869 shutdown_connections 2024-04-26T18:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.031+0000 7f8abd994740 1 -- 172.21.15.84:0/422020869 wait complete. 2024-04-26T18:33:55.038 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.031+0000 7f8abd994740 20 client.16149 trim_cache size 0 max 16384 2024-04-26T18:33:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.045+0000 7f8abd994740 1 Processor -- start 2024-04-26T18:33:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.045+0000 7f8abd994740 1 -- start start 2024-04-26T18:33:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.045+0000 7f8abd994740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed5cde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.045+0000 7f8abd994740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 0x56142ed5d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.045+0000 7f8abd994740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 0x56142ed3f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.045+0000 7f8abd994740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41b0850 con 0x56142ed39f00 2024-04-26T18:33:55.051 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.045+0000 7f8abd994740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41b2390 con 0x56142ed450e0 2024-04-26T18:33:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.045+0000 7f8abd994740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41b0f20 con 0x56142ed3a820 2024-04-26T18:33:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8ab8939640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed5cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8ab8939640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed5cde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59672/0 (socket says 172.21.15.84:59672) 2024-04-26T18:33:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8ab8939640 1 -- 172.21.15.84:0/385989986 learned_addr learned my addr 172.21.15.84:0/385989986 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8ab3fff640 1 --2- 172.21.15.84:0/385989986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 0x56142ed5d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8ab8939640 1 -- 172.21.15.84:0/385989986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 msgr2=0x56142ed5d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8ab8939640 1 --2- 172.21.15.84:0/385989986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 0x56142ed5d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8ab8939640 1 -- 172.21.15.84:0/385989986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 msgr2=0x56142ed3f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8ab913a640 1 --2- 172.21.15.84:0/385989986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 0x56142ed3f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8ab8939640 1 --2- 172.21.15.84:0/385989986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 0x56142ed3f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8ab8939640 1 -- 172.21.15.84:0/385989986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56142ed309c0 con 0x56142ed39f00 2024-04-26T18:33:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8ab3fff640 1 --2- 172.21.15.84:0/385989986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 0x56142ed5d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:55.052 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8ab913a640 1 --2- 172.21.15.84:0/385989986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 0x56142ed3f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8ab8939640 1 --2- 172.21.15.84:0/385989986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed5cde0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8000c00 tx=0x7f8aa8005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969d38517d466218 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8a9e7fc640 1 -- 172.21.15.84:0/385989986 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8035040 con 0x56142ed39f00 2024-04-26T18:33:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8a9e7fc640 1 -- 172.21.15.84:0/385989986 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aa8031b50 con 0x56142ed39f00 2024-04-26T18:33:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.046+0000 7f8a9e7fc640 1 -- 172.21.15.84:0/385989986 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8004d00 con 0x56142ed39f00 2024-04-26T18:33:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.047+0000 7f8abd994740 1 -- 172.21.15.84:0/385989986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 msgr2=0x56142ed5cde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.047+0000 7f8abd994740 1 --2- 172.21.15.84:0/385989986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed5cde0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f8aa8000c00 tx=0x7f8aa8005240 comp rx=0 tx=0).stop 2024-04-26T18:33:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.047+0000 7f8abd994740 1 -- 172.21.15.84:0/385989986 shutdown_connections 2024-04-26T18:33:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.047+0000 7f8abd994740 1 --2- 172.21.15.84:0/385989986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 0x56142ed3f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.047+0000 7f8abd994740 1 --2- 172.21.15.84:0/385989986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 0x56142ed5d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.047+0000 7f8abd994740 1 --2- 172.21.15.84:0/385989986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed5cde0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.047+0000 7f8abd994740 1 -- 172.21.15.84:0/385989986 >> 172.21.15.84:0/385989986 conn(0x56142ed36410 msgr2=0x56142ed427c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:55.053 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.047+0000 7f8abd994740 1 -- 172.21.15.84:0/385989986 shutdown_connections 2024-04-26T18:33:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.047+0000 7f8abd994740 1 -- 172.21.15.84:0/385989986 wait complete. 2024-04-26T18:33:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.047+0000 7f8abd994740 1 -- start start 2024-04-26T18:33:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.047+0000 7f8abd994740 1 Processor -- start 2024-04-26T18:33:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8abd994740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed48b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8abd994740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 0x56142ed49050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8abd994740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 0x56142ed49590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8abd994740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41b1cc0 con 0x56142ed39f00 2024-04-26T18:33:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8abd994740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41b15f0 con 0x56142ed450e0 2024-04-26T18:33:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8abd994740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8ab41ae830 con 0x56142ed3a820 2024-04-26T18:33:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8abd994740 10 client.0 _ll_register_callbacks cb 0x7f8aa000b3f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:33:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8ab8939640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed48b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8ab8939640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed48b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59684/0 (socket says 172.21.15.84:59684) 2024-04-26T18:33:55.054 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8ab8939640 1 -- 172.21.15.84:0/2056343919 learned_addr learned my addr 172.21.15.84:0/2056343919 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8ab3fff640 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 0x56142ed49050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8ab913a640 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 0x56142ed49590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8ab8939640 1 -- 172.21.15.84:0/2056343919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 msgr2=0x56142ed49050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8ab8939640 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 0x56142ed49050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8ab8939640 1 -- 172.21.15.84:0/2056343919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 msgr2=0x56142ed49590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8ab8939640 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 0x56142ed49590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8ab8939640 1 -- 172.21.15.84:0/2056343919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56142edba6a0 con 0x56142ed39f00 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8ab913a640 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 0x56142ed49590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8ab8939640 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed48b10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80095f0 tx=0x7f8aa8005350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc4d9882237c709 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8a93fff640 1 -- 172.21.15.84:0/2056343919 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8aa80316a0 con 0x56142ed39f00 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8a93fff640 1 -- 172.21.15.84:0/2056343919 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 8535+0+0 (secure 0 0 0) 0x7f8aa8031840 con 0x56142ed39f00 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.048+0000 7f8abd994740 1 -- 172.21.15.84:0/2056343919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8aa8031b50 con 0x56142ed39f00 2024-04-26T18:33:55.055 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.049+0000 7f8a93fff640 1 -- 172.21.15.84:0/2056343919 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8aa8004a80 con 0x56142ed39f00 2024-04-26T18:33:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.049+0000 7f8abd994740 10 client.16155 fetch_fsmap learned FSMap version 29 2024-04-26T18:33:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.049+0000 7f8abd994740 1 -- 172.21.15.84:0/2056343919 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56142edbb900 con 0x56142ed39f00 2024-04-26T18:33:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.049+0000 7f8a93fff640 1 -- 172.21.15.84:0/2056343919 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8aa8031b50 con 0x56142ed39f00 2024-04-26T18:33:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.049+0000 7f8a93fff640 1 -- 172.21.15.84:0/2056343919 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f8aa8052740 con 0x56142ed39f00 2024-04-26T18:33:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.049+0000 7f8abd994740 10 client.16155 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:33:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.049+0000 7f8abd994740 10 client.16155 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:33:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.049+0000 7f8abd994740 20 client.16155 populate_metadata read hostname 'smithi084' 2024-04-26T18:33:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.049+0000 7f8abd994740 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56142edbbb80 0x56142edbdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:33:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.049+0000 7f8abd994740 4 client.16155 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:33:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.049+0000 7f8abd994740 1 -- 172.21.15.84:0/2056343919 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56142eca7ee0 con 0x56142edbbb80 2024-04-26T18:33:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.049+0000 7f8ab3fff640 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56142edbbb80 0x56142edbdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:33:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.050+0000 7f8ab3fff640 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56142edbbb80 0x56142edbdf70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6215b1e3f44e831d server_cookie=d5d184de8591afa1 in_seq=0 out_seq=0 2024-04-26T18:33:55.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.050+0000 7f8a93fff640 10 client.16155 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.063+0000 7f8a93fff640 1 -- 172.21.15.84:0/2056343919 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56142eca7ee0 con 0x56142edbbb80 2024-04-26T18:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.063+0000 7f8a93fff640 10 client.16155 handle_command_reply: tid=0 2024-04-26T18:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.063+0000 7f8abd994740 1 client.16155 shutdown 2024-04-26T18:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abd994740 1 -- 172.21.15.84:0/2056343919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 msgr2=0x56142ed48b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abd994740 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed48b10 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f8aa80095f0 tx=0x7f8aa8005350 comp rx=0 tx=0).stop 2024-04-26T18:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abd994740 1 -- 172.21.15.84:0/2056343919 shutdown_connections 2024-04-26T18:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abd994740 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56142edbbb80 0x56142edbdf70 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.064 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abd994740 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56142ed450e0 0x56142ed49590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abd994740 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56142ed3a820 0x56142ed49050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abd994740 1 --2- 172.21.15.84:0/2056343919 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56142ed39f00 0x56142ed48b10 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8a93fff640 0 client.16155 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abd994740 1 -- 172.21.15.84:0/2056343919 >> 172.21.15.84:0/2056343919 conn(0x56142ed36410 msgr2=0x56142ed3b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abd994740 1 -- 172.21.15.84:0/2056343919 shutdown_connections 2024-04-26T18:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abd994740 1 -- 172.21.15.84:0/2056343919 wait complete. 2024-04-26T18:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abd994740 20 client.16155 trim_cache size 0 max 16384 2024-04-26T18:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:33:55.065 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:33:55.066 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:33:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abab9d640 1 -- 172.21.15.84:0/840474867 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8aa40a3dc0 msgr2=0x7f8aa40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abab9d640 1 --2- 172.21.15.84:0/840474867 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8aa40a3dc0 0x7f8aa40a6280 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f8aac009900 tx=0x7f8aac009650 comp rx=0 tx=0).stop 2024-04-26T18:33:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abab9d640 1 -- 172.21.15.84:0/840474867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4102220 msgr2=0x7f8ab419b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:33:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abab9d640 1 --2- 172.21.15.84:0/840474867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4102220 0x7f8ab419b970 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000b4a0 tx=0x7f8aa000b960 comp rx=0 tx=0).stop 2024-04-26T18:33:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abab9d640 1 -- 172.21.15.84:0/840474867 shutdown_connections 2024-04-26T18:33:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abab9d640 1 --2- 172.21.15.84:0/840474867 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f8aa40a3dc0 0x7f8aa40a6280 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abab9d640 1 --2- 172.21.15.84:0/840474867 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8ab4107820 0x7f8ab419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abab9d640 1 --2- 172.21.15.84:0/840474867 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8ab4102220 0x7f8ab419b970 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abab9d640 1 --2- 172.21.15.84:0/840474867 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8ab4101860 0x7f8ab419b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:33:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.064+0000 7f8abab9d640 1 -- 172.21.15.84:0/840474867 >> 172.21.15.84:0/840474867 conn(0x7f8ab40789b0 msgr2=0x7f8ab410d010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:33:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.065+0000 7f8abab9d640 1 -- 172.21.15.84:0/840474867 shutdown_connections 2024-04-26T18:33:55.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:33:55.065+0000 7f8abab9d640 1 -- 172.21.15.84:0/840474867 wait complete. 2024-04-26T18:33:55.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:54 smithi084 ceph-mon[29141]: pgmap v181: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 23 op/s 2024-04-26T18:33:55.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:54 smithi103 ceph-mon[32727]: pgmap v181: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 23 op/s 2024-04-26T18:33:55.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:54 smithi174 ceph-mon[32705]: pgmap v181: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 47 KiB/s wr, 23 op/s 2024-04-26T18:33:57.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:56 smithi084 ceph-mon[29141]: pgmap v182: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-26T18:33:57.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:56 smithi103 ceph-mon[32727]: pgmap v182: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-26T18:33:57.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:56 smithi174 ceph-mon[32705]: pgmap v182: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-26T18:33:59.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:33:58 smithi084 ceph-mon[29141]: pgmap v183: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-26T18:33:59.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:33:58 smithi103 ceph-mon[32727]: pgmap v183: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-26T18:33:59.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:33:58 smithi174 ceph-mon[32705]: pgmap v183: 97 pgs: 97 active+clean; 5.5 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 13 op/s 2024-04-26T18:34:01.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:00 smithi084 ceph-mon[29141]: pgmap v184: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 23 op/s 2024-04-26T18:34:01.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:00 smithi103 ceph-mon[32727]: pgmap v184: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 23 op/s 2024-04-26T18:34:01.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:00 smithi174 ceph-mon[32705]: pgmap v184: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 46 KiB/s wr, 23 op/s 2024-04-26T18:34:03.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:02 smithi084 ceph-mon[29141]: pgmap v185: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-04-26T18:34:03.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:02 smithi103 ceph-mon[32727]: pgmap v185: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-04-26T18:34:03.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:02 smithi174 ceph-mon[32705]: pgmap v185: 97 pgs: 97 active+clean; 1.7 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 16 op/s 2024-04-26T18:34:05.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:04 smithi084 ceph-mon[29141]: pgmap v186: 97 pgs: 97 active+clean; 1.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 94 KiB/s wr, 39 op/s 2024-04-26T18:34:05.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:04 smithi084 ceph-mon[29141]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T18:34:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:04 smithi103 ceph-mon[32727]: pgmap v186: 97 pgs: 97 active+clean; 1.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 94 KiB/s wr, 39 op/s 2024-04-26T18:34:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:04 smithi103 ceph-mon[32727]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T18:34:05.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:04 smithi174 ceph-mon[32705]: pgmap v186: 97 pgs: 97 active+clean; 1.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 94 KiB/s wr, 39 op/s 2024-04-26T18:34:05.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:04 smithi174 ceph-mon[32705]: osdmap e95: 12 total, 12 up, 12 in 2024-04-26T18:34:07.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:06 smithi084 ceph-mon[29141]: pgmap v188: 97 pgs: 97 active+clean; 1.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 95 KiB/s wr, 39 op/s 2024-04-26T18:34:07.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:06 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:34:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:06 smithi103 ceph-mon[32727]: pgmap v188: 97 pgs: 97 active+clean; 1.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 95 KiB/s wr, 39 op/s 2024-04-26T18:34:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:06 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:34:07.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:06 smithi174 ceph-mon[32705]: pgmap v188: 97 pgs: 97 active+clean; 1.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 95 KiB/s wr, 39 op/s 2024-04-26T18:34:07.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:06 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:34:08.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:07 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:34:08.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:07 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:34:08.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:07 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:34:08.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:07 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:34:08.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:07 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:34:08.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:07 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:34:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.161+0000 7f62b703f640 1 -- 172.21.15.84:0/3975671245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62b0101830 msgr2=0x7f62b0101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.161+0000 7f62b703f640 1 --2- 172.21.15.84:0/3975671245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62b0101830 0x7f62b0101cb0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f629c009930 tx=0x7f629c02a420 comp rx=0 tx=0).stop 2024-04-26T18:34:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.161+0000 7f62b703f640 1 -- 172.21.15.84:0/3975671245 shutdown_connections 2024-04-26T18:34:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.161+0000 7f62b703f640 1 --2- 172.21.15.84:0/3975671245 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b01021f0 0x7f62b010e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.161+0000 7f62b703f640 1 --2- 172.21.15.84:0/3975671245 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62b0101830 0x7f62b0101cb0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.161+0000 7f62b703f640 1 --2- 172.21.15.84:0/3975671245 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62b0107840 0x7f62b0107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.161+0000 7f62b703f640 1 -- 172.21.15.84:0/3975671245 >> 172.21.15.84:0/3975671245 conn(0x7f62b00789b0 msgr2=0x7f62b0078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.161+0000 7f62b703f640 1 -- 172.21.15.84:0/3975671245 shutdown_connections 2024-04-26T18:34:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.161+0000 7f62b703f640 1 -- 172.21.15.84:0/3975671245 wait complete. 2024-04-26T18:34:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.161+0000 7f62b703f640 1 Processor -- start 2024-04-26T18:34:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.161+0000 7f62b703f640 1 -- start start 2024-04-26T18:34:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62b703f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b0101830 0x7f62b00fe790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62b703f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62b01021f0 0x7f62b00fecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62b703f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62b0107840 0x7f62b00ff210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62b703f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0111b10 con 0x7f62b01021f0 2024-04-26T18:34:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62b703f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0111990 con 0x7f62b0107840 2024-04-26T18:34:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62b703f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62b0111c90 con 0x7f62b0101830 2024-04-26T18:34:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62b01021f0 0x7f62b00fecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62b01021f0 0x7f62b00fecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49452/0 (socket says 172.21.15.84:49452) 2024-04-26T18:34:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62affff640 1 -- 172.21.15.84:0/1520623166 learned_addr learned my addr 172.21.15.84:0/1520623166 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62b4ddb640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b0101830 0x7f62b00fe790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62b55dc640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62b0107840 0x7f62b00ff210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62affff640 1 -- 172.21.15.84:0/1520623166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b0101830 msgr2=0x7f62b00fe790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62affff640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b0101830 0x7f62b00fe790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62affff640 1 -- 172.21.15.84:0/1520623166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62b0107840 msgr2=0x7f62b00ff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62affff640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62b0107840 0x7f62b00ff210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62affff640 1 -- 172.21.15.84:0/1520623166 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b01a75e0 con 0x7f62b01021f0 2024-04-26T18:34:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62b55dc640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62b0107840 0x7f62b00ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:34:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62b4ddb640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b0101830 0x7f62b00fe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:34:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62affff640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62b01021f0 0x7f62b00fecd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f629c005ab0 tx=0x7f629c003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62adffb640 1 -- 172.21.15.84:0/1520623166 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c03f070 con 0x7f62b01021f0 2024-04-26T18:34:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62adffb640 1 -- 172.21.15.84:0/1520623166 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f629c031b50 con 0x7f62b01021f0 2024-04-26T18:34:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.162+0000 7f62b703f640 1 -- 172.21.15.84:0/1520623166 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b01a7870 con 0x7f62b01021f0 2024-04-26T18:34:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.163+0000 7f62adffb640 1 -- 172.21.15.84:0/1520623166 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c035030 con 0x7f62b01021f0 2024-04-26T18:34:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.163+0000 7f62b703f640 1 -- 172.21.15.84:0/1520623166 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b01a7d60 con 0x7f62b01021f0 2024-04-26T18:34:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.165+0000 7f62adffb640 1 -- 172.21.15.84:0/1520623166 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f629c042070 con 0x7f62b01021f0 2024-04-26T18:34:08.173 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b9e30740 1 Processor -- start 2024-04-26T18:34:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b9e30740 1 -- start start 2024-04-26T18:34:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b9e30740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321804f00 0x5563218052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b9e30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x55632180fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b9e30740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x5563218124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b9e30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55632176b460 con 0x556321805820 2024-04-26T18:34:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b9e30740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556321725590 con 0x556321804f00 2024-04-26T18:34:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b9e30740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556321767420 con 0x5563218100e0 2024-04-26T18:34:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62adffb640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f62a00a3dc0 0x7f62a00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b4ddb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x55632180fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b4ddb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x55632180fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49464/0 (socket says 172.21.15.84:49464) 2024-04-26T18:34:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b4ddb640 1 -- 172.21.15.84:0/4251015766 learned_addr learned my addr 172.21.15.84:0/4251015766 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:08.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b55dc640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f62a00a3dc0 0x7f62a00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b55dc640 1 --2- 172.21.15.84:0/4251015766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321804f00 0x5563218052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62adffb640 1 -- 172.21.15.84:0/1520623166 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8623+0+0 (secure 0 0 0) 0x7f629c0f0ee0 con 0x7f62b01021f0 2024-04-26T18:34:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62affff640 1 --2- 172.21.15.84:0/4251015766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x5563218124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b4ddb640 1 -- 172.21.15.84:0/4251015766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 msgr2=0x5563218124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b4ddb640 1 --2- 172.21.15.84:0/4251015766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x5563218124d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b4ddb640 1 -- 172.21.15.84:0/4251015766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321804f00 msgr2=0x5563218052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b4ddb640 1 --2- 172.21.15.84:0/4251015766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321804f00 0x5563218052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62b4ddb640 1 -- 172.21.15.84:0/4251015766 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563217320d0 con 0x556321805820 2024-04-26T18:34:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.166+0000 7f62affff640 1 --2- 172.21.15.84:0/4251015766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x5563218124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:34:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.167+0000 7f62b4ddb640 1 --2- 172.21.15.84:0/4251015766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x55632180fba0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f62a40114f0 tx=0x7f62a40119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17da36630ee8bbbc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.167+0000 7f62b55dc640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f62a00a3dc0 0x7f62a00a6280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62a8005fd0 tx=0x7f62a80098e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:08.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.167+0000 7f629affd640 1 -- 172.21.15.84:0/4251015766 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a401e070 con 0x556321805820 2024-04-26T18:34:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.167+0000 7f629affd640 1 -- 172.21.15.84:0/4251015766 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a400fa70 con 0x556321805820 2024-04-26T18:34:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.167+0000 7f629affd640 1 -- 172.21.15.84:0/4251015766 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4019a80 con 0x556321805820 2024-04-26T18:34:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.167+0000 7f62b9e30740 1 -- 172.21.15.84:0/4251015766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 msgr2=0x55632180fba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 --2- 172.21.15.84:0/4251015766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x55632180fba0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f62a40114f0 tx=0x7f62a40119b0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 -- 172.21.15.84:0/4251015766 shutdown_connections 2024-04-26T18:34:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 --2- 172.21.15.84:0/4251015766 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x5563218124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 --2- 172.21.15.84:0/4251015766 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x55632180fba0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 --2- 172.21.15.84:0/4251015766 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321804f00 0x5563218052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 -- 172.21.15.84:0/4251015766 >> 172.21.15.84:0/4251015766 conn(0x556321801410 msgr2=0x5563218017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 -- 172.21.15.84:0/4251015766 shutdown_connections 2024-04-26T18:34:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 -- 172.21.15.84:0/4251015766 wait complete. 2024-04-26T18:34:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 -- start start 2024-04-26T18:34:08.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 Processor -- start 2024-04-26T18:34:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321804f00 0x55632180d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x55632180d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x55632180dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556321812d10 con 0x556321805820 2024-04-26T18:34:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.168+0000 7f62b9e30740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5563216f2b60 con 0x556321804f00 2024-04-26T18:34:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62b9e30740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556321812e90 con 0x5563218100e0 2024-04-26T18:34:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62b9e30740 10 client.0 _ll_register_callbacks cb 0x5563217172b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:34:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x55632180d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x55632180d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49470/0 (socket says 172.21.15.84:49470) 2024-04-26T18:34:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62affff640 1 -- 172.21.15.84:0/505429017 learned_addr learned my addr 172.21.15.84:0/505429017 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:08.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62b4ddb640 1 --2- 172.21.15.84:0/505429017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321804f00 0x55632180d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62b55dc640 1 --2- 172.21.15.84:0/505429017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x55632180dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62affff640 1 -- 172.21.15.84:0/505429017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 msgr2=0x55632180dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62affff640 1 --2- 172.21.15.84:0/505429017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x55632180dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62affff640 1 -- 172.21.15.84:0/505429017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321804f00 msgr2=0x55632180d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62affff640 1 --2- 172.21.15.84:0/505429017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321804f00 0x55632180d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62affff640 1 -- 172.21.15.84:0/505429017 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55632147d210 con 0x556321805820 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62affff640 1 --2- 172.21.15.84:0/505429017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x55632180d5a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f629c0b9220 tx=0x7f629c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fac30348c7112175 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f628ffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f628ffff640 1 -- 172.21.15.84:0/505429017 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c049040 con 0x556321805820 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f628ffff640 1 -- 172.21.15.84:0/505429017 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8623+0+0 (secure 0 0 0) 0x7f629c0b9430 con 0x556321805820 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62b9e30740 1 -- 172.21.15.84:0/505429017 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62a400fa70 con 0x556321805820 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f628ffff640 1 -- 172.21.15.84:0/505429017 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f629c0bcdc0 con 0x556321805820 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62b9e30740 10 client.16167 fetch_fsmap learned FSMap version 29 2024-04-26T18:34:08.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f62b9e30740 1 -- 172.21.15.84:0/505429017 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556321549620 con 0x556321805820 2024-04-26T18:34:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.169+0000 7f628ffff640 1 -- 172.21.15.84:0/505429017 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62a400fa70 con 0x556321805820 2024-04-26T18:34:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.170+0000 7f628ffff640 1 -- 172.21.15.84:0/505429017 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f629c0ee7f0 con 0x556321805820 2024-04-26T18:34:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.170+0000 7f62b9e30740 10 client.16167 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:34:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.170+0000 7f62b9e30740 10 client.16167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:34:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.170+0000 7f62b9e30740 20 client.16167 populate_metadata read hostname 'smithi084' 2024-04-26T18:34:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.170+0000 7f62b9e30740 1 --2- 172.21.15.84:0/505429017 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556321820140 0x5563218225d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.170+0000 7f62b9e30740 4 client.16167 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:34:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.170+0000 7f62b9e30740 1 -- 172.21.15.84:0/505429017 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556321779070 con 0x556321820140 2024-04-26T18:34:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.170+0000 7f62b4ddb640 1 --2- 172.21.15.84:0/505429017 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556321820140 0x5563218225d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.170+0000 7f628ffff640 10 client.16167 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.170+0000 7f62b4ddb640 1 --2- 172.21.15.84:0/505429017 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556321820140 0x5563218225d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57a9574efea20a5a server_cookie=f412f4c3c0bed927 in_seq=0 out_seq=0 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f628ffff640 1 -- 172.21.15.84:0/505429017 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556321779070 con 0x556321820140 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f628ffff640 10 client.16167 handle_command_reply: tid=0 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f62b9e30740 1 client.16167 shutdown 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f62b9e30740 1 -- 172.21.15.84:0/505429017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 msgr2=0x55632180d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f62b9e30740 1 --2- 172.21.15.84:0/505429017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x55632180d5a0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f629c0b9220 tx=0x7f629c000bc0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f62b9e30740 1 -- 172.21.15.84:0/505429017 shutdown_connections 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f62b9e30740 1 --2- 172.21.15.84:0/505429017 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556321820140 0x5563218225d0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f62b9e30740 1 --2- 172.21.15.84:0/505429017 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x55632180dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f62b9e30740 1 --2- 172.21.15.84:0/505429017 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x55632180d5a0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f62b9e30740 1 --2- 172.21.15.84:0/505429017 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321804f00 0x55632180d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f628ffff640 0 client.16167 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f62b9e30740 1 -- 172.21.15.84:0/505429017 >> 172.21.15.84:0/505429017 conn(0x556321801410 msgr2=0x556321807190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f62b9e30740 1 -- 172.21.15.84:0/505429017 shutdown_connections 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.171+0000 7f62b9e30740 1 -- 172.21.15.84:0/505429017 wait complete. 2024-04-26T18:34:08.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.172+0000 7f62b9e30740 20 client.16167 trim_cache size 0 max 16384 2024-04-26T18:34:08.191 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:07 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:34:08.192 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:07 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:34:08.192 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:07 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.186+0000 7f62b9e30740 1 Processor -- start 2024-04-26T18:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.186+0000 7f62b9e30740 1 -- start start 2024-04-26T18:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.186+0000 7f62b9e30740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556321804f00 0x55632181bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.186+0000 7f62b9e30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x556321807910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.186+0000 7f62b9e30740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563218100e0 0x55632180d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.186+0000 7f62b9e30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01b0890 con 0x556321805820 2024-04-26T18:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.186+0000 7f62b9e30740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01b23d0 con 0x5563218100e0 2024-04-26T18:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.186+0000 7f62b9e30740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01b0f60 con 0x556321804f00 2024-04-26T18:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.187+0000 7f62affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x556321807910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.192 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.187+0000 7f62affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x556321807910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49478/0 (socket says 172.21.15.84:49478) 2024-04-26T18:34:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.187+0000 7f62affff640 1 -- 172.21.15.84:0/3561664813 learned_addr learned my addr 172.21.15.84:0/3561664813 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.187+0000 7f62affff640 1 -- 172.21.15.84:0/3561664813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556321804f00 msgr2=0x55632181bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.187+0000 7f62affff640 1 --2- 172.21.15.84:0/3561664813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556321804f00 0x55632181bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.187+0000 7f62b55dc640 1 --2- 172.21.15.84:0/3561664813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563218100e0 0x55632180d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.187+0000 7f62affff640 1 -- 172.21.15.84:0/3561664813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563218100e0 msgr2=0x55632180d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.187+0000 7f62affff640 1 --2- 172.21.15.84:0/3561664813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563218100e0 0x55632180d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.187+0000 7f62affff640 1 -- 172.21.15.84:0/3561664813 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556321805650 con 0x556321805820 2024-04-26T18:34:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.187+0000 7f62affff640 1 --2- 172.21.15.84:0/3561664813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x556321807910 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f629c0b8aa0 tx=0x7f629c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19ea84ccffbfa67e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.187+0000 7f629a7fc640 1 -- 172.21.15.84:0/3561664813 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c049070 con 0x556321805820 2024-04-26T18:34:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.187+0000 7f629a7fc640 1 -- 172.21.15.84:0/3561664813 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f629c0c1170 con 0x556321805820 2024-04-26T18:34:08.193 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.187+0000 7f629a7fc640 1 -- 172.21.15.84:0/3561664813 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629c0ee7f0 con 0x556321805820 2024-04-26T18:34:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 -- 172.21.15.84:0/3561664813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 msgr2=0x556321807910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 --2- 172.21.15.84:0/3561664813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x556321807910 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f629c0b8aa0 tx=0x7f629c009260 comp rx=0 tx=0).stop 2024-04-26T18:34:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 -- 172.21.15.84:0/3561664813 shutdown_connections 2024-04-26T18:34:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 --2- 172.21.15.84:0/3561664813 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5563218100e0 0x55632180d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 --2- 172.21.15.84:0/3561664813 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321805820 0x556321807910 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 --2- 172.21.15.84:0/3561664813 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556321804f00 0x55632181bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 -- 172.21.15.84:0/3561664813 >> 172.21.15.84:0/3561664813 conn(0x556321801410 msgr2=0x55632180d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 -- 172.21.15.84:0/3561664813 shutdown_connections 2024-04-26T18:34:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 -- 172.21.15.84:0/3561664813 wait complete. 2024-04-26T18:34:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 -- start start 2024-04-26T18:34:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 Processor -- start 2024-04-26T18:34:08.194 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321804f00 0x55632180a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321805820 0x55632180a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x556321885720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01b1d00 con 0x556321804f00 2024-04-26T18:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01b1630 con 0x556321805820 2024-04-26T18:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.188+0000 7f62b9e30740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f62b01ae890 con 0x5563218100e0 2024-04-26T18:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b9e30740 10 client.0 _ll_register_callbacks cb 0x7f629c0b89f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b4ddb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321804f00 0x55632180a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b4ddb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321804f00 0x55632180a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:49492/0 (socket says 172.21.15.84:49492) 2024-04-26T18:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b4ddb640 1 -- 172.21.15.84:0/2670391488 learned_addr learned my addr 172.21.15.84:0/2670391488 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b55dc640 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x556321885720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b4ddb640 1 -- 172.21.15.84:0/2670391488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 msgr2=0x556321885720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b4ddb640 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x556321885720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.195 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62affff640 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321805820 0x55632180a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b4ddb640 1 -- 172.21.15.84:0/2670391488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321805820 msgr2=0x55632180a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b4ddb640 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321805820 0x55632180a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b4ddb640 1 -- 172.21.15.84:0/2670391488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556321885c60 con 0x556321804f00 2024-04-26T18:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b55dc640 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x556321885720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b4ddb640 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321804f00 0x55632180a3c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009230 tx=0x7f62a4009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0e1dd54b59a017 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f628ffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f628ffff640 1 -- 172.21.15.84:0/2670391488 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a401e040 con 0x556321804f00 2024-04-26T18:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f628ffff640 1 -- 172.21.15.84:0/2670391488 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 8623+0+0 (secure 0 0 0) 0x7f62a400f5c0 con 0x556321804f00 2024-04-26T18:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b9e30740 1 -- 172.21.15.84:0/2670391488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f629c0c1170 con 0x556321804f00 2024-04-26T18:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f628ffff640 1 -- 172.21.15.84:0/2670391488 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f62a400f7c0 con 0x556321804f00 2024-04-26T18:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b9e30740 10 client.16173 fetch_fsmap learned FSMap version 29 2024-04-26T18:34:08.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f62b9e30740 1 -- 172.21.15.84:0/2670391488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556321886ed0 con 0x556321804f00 2024-04-26T18:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.189+0000 7f628ffff640 1 -- 172.21.15.84:0/2670391488 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f629c0c1170 con 0x556321804f00 2024-04-26T18:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.190+0000 7f628ffff640 1 -- 172.21.15.84:0/2670391488 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f62a4019870 con 0x556321804f00 2024-04-26T18:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.190+0000 7f62b9e30740 10 client.16173 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.190+0000 7f62b9e30740 10 client.16173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.190+0000 7f62b9e30740 20 client.16173 populate_metadata read hostname 'smithi084' 2024-04-26T18:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.190+0000 7f62b9e30740 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556321887510 0x556321889930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.190+0000 7f62b9e30740 4 client.16173 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.190+0000 7f62b9e30740 1 -- 172.21.15.84:0/2670391488 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556321772ee0 con 0x556321887510 2024-04-26T18:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.190+0000 7f62affff640 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556321887510 0x556321889930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.190+0000 7f628ffff640 10 client.16173 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:08.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.190+0000 7f62affff640 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556321887510 0x556321889930 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db80380e0fc2cfb0 server_cookie=e4ffd5101bfe83a8 in_seq=0 out_seq=0 2024-04-26T18:34:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.205+0000 7f628ffff640 1 -- 172.21.15.84:0/2670391488 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556321772ee0 con 0x556321887510 2024-04-26T18:34:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.205+0000 7f628ffff640 10 client.16173 handle_command_reply: tid=0 2024-04-26T18:34:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.205+0000 7f62b9e30740 1 client.16173 shutdown 2024-04-26T18:34:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.205+0000 7f62b9e30740 1 -- 172.21.15.84:0/2670391488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321804f00 msgr2=0x55632180a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.205+0000 7f62b9e30740 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321804f00 0x55632180a3c0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f62a4009230 tx=0x7f62a4009db0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.205+0000 7f62b9e30740 1 -- 172.21.15.84:0/2670391488 shutdown_connections 2024-04-26T18:34:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.205+0000 7f62b9e30740 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556321887510 0x556321889930 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.205+0000 7f62b9e30740 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5563218100e0 0x556321885720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.205+0000 7f628ffff640 0 client.16173 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:08.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.205+0000 7f62b9e30740 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556321805820 0x55632180a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.205+0000 7f62b9e30740 1 --2- 172.21.15.84:0/2670391488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556321804f00 0x55632180a3c0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.205+0000 7f62b9e30740 1 -- 172.21.15.84:0/2670391488 >> 172.21.15.84:0/2670391488 conn(0x556321801410 msgr2=0x5563218005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b9e30740 1 -- 172.21.15.84:0/2670391488 shutdown_connections 2024-04-26T18:34:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b9e30740 1 -- 172.21.15.84:0/2670391488 wait complete. 2024-04-26T18:34:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b9e30740 20 client.16173 trim_cache size 0 max 16384 2024-04-26T18:34:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:34:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:34:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:34:08.207 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:34:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b703f640 1 -- 172.21.15.84:0/1520623166 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f62a00a3dc0 msgr2=0x7f62a00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b703f640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f62a00a3dc0 0x7f62a00a6280 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f62a8005fd0 tx=0x7f62a80098e0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b703f640 1 -- 172.21.15.84:0/1520623166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62b01021f0 msgr2=0x7f62b00fecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:08.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b703f640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62b01021f0 0x7f62b00fecd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f629c005ab0 tx=0x7f629c003b90 comp rx=0 tx=0).stop 2024-04-26T18:34:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b703f640 1 -- 172.21.15.84:0/1520623166 shutdown_connections 2024-04-26T18:34:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b703f640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f62a00a3dc0 0x7f62a00a6280 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b703f640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f62b0107840 0x7f62b00ff210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b703f640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f62b01021f0 0x7f62b00fecd0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b703f640 1 --2- 172.21.15.84:0/1520623166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f62b0101830 0x7f62b00fe790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b703f640 1 -- 172.21.15.84:0/1520623166 >> 172.21.15.84:0/1520623166 conn(0x7f62b00789b0 msgr2=0x7f62b006b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b703f640 1 -- 172.21.15.84:0/1520623166 shutdown_connections 2024-04-26T18:34:08.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:08.206+0000 7f62b703f640 1 -- 172.21.15.84:0/1520623166 wait complete. 2024-04-26T18:34:09.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:08 smithi084 ceph-mon[29141]: pgmap v189: 97 pgs: 97 active+clean; 1.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 95 KiB/s wr, 39 op/s 2024-04-26T18:34:09.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:08 smithi103 ceph-mon[32727]: pgmap v189: 97 pgs: 97 active+clean; 1.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 95 KiB/s wr, 39 op/s 2024-04-26T18:34:09.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:08 smithi174 ceph-mon[32705]: pgmap v189: 97 pgs: 97 active+clean; 1.9 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 95 KiB/s wr, 39 op/s 2024-04-26T18:34:11.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:10 smithi084 ceph-mon[29141]: pgmap v190: 97 pgs: 97 active+clean; 2.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 101 KiB/s wr, 39 op/s 2024-04-26T18:34:11.377 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:10 smithi103 ceph-mon[32727]: pgmap v190: 97 pgs: 97 active+clean; 2.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 101 KiB/s wr, 39 op/s 2024-04-26T18:34:11.439 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:10 smithi174 ceph-mon[32705]: pgmap v190: 97 pgs: 97 active+clean; 2.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 101 KiB/s wr, 39 op/s 2024-04-26T18:34:13.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:13 smithi103 ceph-mon[32727]: pgmap v191: 97 pgs: 97 active+clean; 2.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 101 KiB/s wr, 39 op/s 2024-04-26T18:34:13.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:13 smithi103 ceph-mon[32727]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T18:34:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:13 smithi174 ceph-mon[32705]: pgmap v191: 97 pgs: 97 active+clean; 2.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 101 KiB/s wr, 39 op/s 2024-04-26T18:34:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:13 smithi174 ceph-mon[32705]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T18:34:13.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:13 smithi084 ceph-mon[29141]: pgmap v191: 97 pgs: 97 active+clean; 2.1 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 101 KiB/s wr, 39 op/s 2024-04-26T18:34:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:13 smithi084 ceph-mon[29141]: osdmap e96: 12 total, 12 up, 12 in 2024-04-26T18:34:15.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:15 smithi103 ceph-mon[32727]: pgmap v193: 97 pgs: 97 active+clean; 2.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-26T18:34:15.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:15 smithi174 ceph-mon[32705]: pgmap v193: 97 pgs: 97 active+clean; 2.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-26T18:34:15.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:15 smithi084 ceph-mon[29141]: pgmap v193: 97 pgs: 97 active+clean; 2.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-26T18:34:17.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:17 smithi103 ceph-mon[32727]: pgmap v194: 97 pgs: 97 active+clean; 2.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-04-26T18:34:17.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:17 smithi174 ceph-mon[32705]: pgmap v194: 97 pgs: 97 active+clean; 2.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-04-26T18:34:17.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:17 smithi084 ceph-mon[29141]: pgmap v194: 97 pgs: 97 active+clean; 2.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-04-26T18:34:19.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:19 smithi103 ceph-mon[32727]: pgmap v195: 97 pgs: 97 active+clean; 2.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-04-26T18:34:19.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:19 smithi174 ceph-mon[32705]: pgmap v195: 97 pgs: 97 active+clean; 2.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-04-26T18:34:19.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:19 smithi084 ceph-mon[29141]: pgmap v195: 97 pgs: 97 active+clean; 2.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 46 KiB/s wr, 19 op/s 2024-04-26T18:34:20.040 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.040 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.040 INFO:tasks.workunit.client.0.smithi084.stdout:127/600: open fd = 3 2024-04-26T18:34:20.040 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.041 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.041 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.041 INFO:tasks.workunit.client.0.smithi084.stdout:128/600: open fd = 3 2024-04-26T18:34:20.041 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.041 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.041 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.041 INFO:tasks.workunit.client.0.smithi084.stdout:129/600: open fd = 3 2024-04-26T18:34:20.041 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.041 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.041 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.041 INFO:tasks.workunit.client.0.smithi084.stdout:130/600: open fd = 3 2024-04-26T18:34:20.041 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.041 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:131/600: open fd = 3 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:132/600: open fd = 3 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:133/600: open fd = 3 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:134/600: open fd = 3 2024-04-26T18:34:20.042 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:135/600: open fd = 3 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:136/600: open fd = 3 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:137/600: open fd = 3 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.043 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:138/600: open fd = 3 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:139/600: open fd = 3 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:140/600: open fd = 3 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:141/600: open fd = 3 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.044 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.045 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.045 INFO:tasks.workunit.client.0.smithi084.stdout:142/600: open fd = 3 2024-04-26T18:34:20.045 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.045 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.045 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.045 INFO:tasks.workunit.client.0.smithi084.stdout:143/600: open fd = 3 2024-04-26T18:34:20.045 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.045 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.045 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.045 INFO:tasks.workunit.client.0.smithi084.stdout:144/600: open fd = 3 2024-04-26T18:34:20.045 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.045 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.045 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:145/600: open fd = 3 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:146/600: open fd = 3 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:147/600: open fd = 3 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:148/600: open fd = 3 2024-04-26T18:34:20.046 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:149/600: open fd = 3 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:150/600: open fd = 3 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:151/600: open fd = 3 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.047 INFO:tasks.workunit.client.0.smithi084.stdout:152/600: open fd = 3 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:153/600: open fd = 3 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:154/600: open fd = 3 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:155/600: open fd = 3 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.048 INFO:tasks.workunit.client.0.smithi084.stdout:156/600: open fd = 3 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:157/600: open fd = 3 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:158/600: open fd = 3 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:159/600: open fd = 3 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.049 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:160/600: open fd = 3 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:161/600: open fd = 3 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:162/600: open fd = 3 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:163/600: open fd = 3 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.050 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:164/600: open fd = 3 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:165/600: open fd = 3 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:166/600: open fd = 3 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:167/600: open fd = 3 2024-04-26T18:34:20.051 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:168/600: open fd = 3 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:169/600: open fd = 3 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:170/600: open fd = 3 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.052 INFO:tasks.workunit.client.0.smithi084.stdout:171/600: open fd = 3 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:172/600: open fd = 3 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:173/600: open fd = 3 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:174/600: open fd = 3 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.053 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:175/600: open fd = 3 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:176/600: open fd = 3 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:177/600: open fd = 3 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.054 INFO:tasks.workunit.client.0.smithi084.stdout:178/600: open fd = 3 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:179/600: open fd = 3 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:180/600: open fd = 3 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:181/600: open fd = 3 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.055 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:182/600: open fd = 3 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:183/600: open fd = 3 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:184/600: open fd = 3 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:185/600: open fd = 3 2024-04-26T18:34:20.056 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.057 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.057 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.057 INFO:tasks.workunit.client.0.smithi084.stdout:186/600: open fd = 3 2024-04-26T18:34:20.057 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.057 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.057 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.057 INFO:tasks.workunit.client.0.smithi084.stdout:187/600: open fd = 3 2024-04-26T18:34:20.057 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.057 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.057 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:34:20.057 INFO:tasks.workunit.client.0.smithi084.stdout:188/600: open fd = 3 2024-04-26T18:34:20.057 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:20.057 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:34:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.302+0000 7fc6d15c6640 1 -- 172.21.15.84:0/3520557172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6cc108940 msgr2=0x7fc6cc108d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.302+0000 7fc6d15c6640 1 --2- 172.21.15.84:0/3520557172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6cc108940 0x7fc6cc108d20 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009930 tx=0x7fc6bc02a420 comp rx=0 tx=0).stop 2024-04-26T18:34:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.302+0000 7fc6d15c6640 1 -- 172.21.15.84:0/3520557172 shutdown_connections 2024-04-26T18:34:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.302+0000 7fc6d15c6640 1 --2- 172.21.15.84:0/3520557172 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6cc1032f0 0x7fc6cc10f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.302+0000 7fc6d15c6640 1 --2- 172.21.15.84:0/3520557172 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc102930 0x7fc6cc102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.302+0000 7fc6d15c6640 1 --2- 172.21.15.84:0/3520557172 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6cc108940 0x7fc6cc108d20 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.302+0000 7fc6d15c6640 1 -- 172.21.15.84:0/3520557172 >> 172.21.15.84:0/3520557172 conn(0x7fc6cc0fe420 msgr2=0x7fc6cc100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.302+0000 7fc6d15c6640 1 -- 172.21.15.84:0/3520557172 shutdown_connections 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.302+0000 7fc6d15c6640 1 -- 172.21.15.84:0/3520557172 wait complete. 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.302+0000 7fc6d15c6640 1 Processor -- start 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6d15c6640 1 -- start start 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6d15c6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6cc102930 0x7fc6cc199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6d15c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc1032f0 0x7fc6cc199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6d15c6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6cc108940 0x7fc6cc19d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6d15c6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc112c00 con 0x7fc6cc102930 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6d15c6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc112a80 con 0x7fc6cc1032f0 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6d15c6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc112d80 con 0x7fc6cc108940 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6caffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6cc102930 0x7fc6cc199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6caffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6cc102930 0x7fc6cc199140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37584/0 (socket says 172.21.15.84:37584) 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6caffd640 1 -- 172.21.15.84:0/4221647398 learned_addr learned my addr 172.21.15.84:0/4221647398 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:21.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6ca7fc640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc1032f0 0x7fc6cc199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6cb7fe640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6cc108940 0x7fc6cc19d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6caffd640 1 -- 172.21.15.84:0/4221647398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6cc108940 msgr2=0x7fc6cc19d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6caffd640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6cc108940 0x7fc6cc19d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6caffd640 1 -- 172.21.15.84:0/4221647398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc1032f0 msgr2=0x7fc6cc199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6caffd640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc1032f0 0x7fc6cc199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6caffd640 1 -- 172.21.15.84:0/4221647398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6cc19e0e0 con 0x7fc6cc102930 2024-04-26T18:34:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.303+0000 7fc6caffd640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6cc102930 0x7fc6cc199140 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc004770 tx=0x7fc6bc0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.304+0000 7fc6b3fff640 1 -- 172.21.15.84:0/4221647398 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc03f070 con 0x7fc6cc102930 2024-04-26T18:34:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.304+0000 7fc6d15c6640 1 -- 172.21.15.84:0/4221647398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6cc19e370 con 0x7fc6cc102930 2024-04-26T18:34:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.304+0000 7fc6b3fff640 1 -- 172.21.15.84:0/4221647398 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6bc03b460 con 0x7fc6cc102930 2024-04-26T18:34:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.304+0000 7fc6b3fff640 1 -- 172.21.15.84:0/4221647398 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc035070 con 0x7fc6cc102930 2024-04-26T18:34:21.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.304+0000 7fc6d15c6640 1 -- 172.21.15.84:0/4221647398 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6cc1a5c60 con 0x7fc6cc102930 2024-04-26T18:34:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.306+0000 7fc6b3fff640 1 -- 172.21.15.84:0/4221647398 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc6bc03b8d0 con 0x7fc6cc102930 2024-04-26T18:34:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.306+0000 7fc6d43be740 1 Processor -- start 2024-04-26T18:34:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.306+0000 7fc6d43be740 1 -- start start 2024-04-26T18:34:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6d43be740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93ab72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6d43be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ab7820 0x564e93ac1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6d43be740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ac20e0 0x564e93ac44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6b3fff640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc6b40a3dc0 0x7fc6b40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6d43be740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564e93a1d460 con 0x564e93ab6f00 2024-04-26T18:34:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6d43be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564e939d7590 con 0x564e93ab7820 2024-04-26T18:34:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6d43be740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564e93a19420 con 0x564e93ac20e0 2024-04-26T18:34:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6cb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93ab72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6cb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93ab72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37600/0 (socket says 172.21.15.84:37600) 2024-04-26T18:34:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6cb7fe640 1 -- 172.21.15.84:0/3368983128 learned_addr learned my addr 172.21.15.84:0/3368983128 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6ca7fc640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc6b40a3dc0 0x7fc6b40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6b3fff640 1 -- 172.21.15.84:0/4221647398 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7fc6bc0f0ad0 con 0x7fc6cc102930 2024-04-26T18:34:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6caffd640 1 --2- 172.21.15.84:0/3368983128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ab7820 0x564e93ac1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6cb7fe640 1 --2- 172.21.15.84:0/3368983128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ac20e0 0x564e93ac44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6cb7fe640 1 -- 172.21.15.84:0/3368983128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ac20e0 msgr2=0x564e93ac44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6cb7fe640 1 --2- 172.21.15.84:0/3368983128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ac20e0 0x564e93ac44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6cb7fe640 1 -- 172.21.15.84:0/3368983128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ab7820 msgr2=0x564e93ac1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:21.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6cb7fe640 1 --2- 172.21.15.84:0/3368983128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ab7820 0x564e93ac1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6cb7fe640 1 -- 172.21.15.84:0/3368983128 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e939e40d0 con 0x564e93ab6f00 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6caffd640 1 --2- 172.21.15.84:0/3368983128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ab7820 0x564e93ac1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6ca7fc640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc6b40a3dc0 0x7fc6b40a6280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc100ab0 tx=0x7fc6c000a2d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.307+0000 7fc6cb7fe640 1 --2- 172.21.15.84:0/3368983128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93ab72e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x564e93ab45a0 tx=0x7fc6c4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264d46f99da8db35 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.308+0000 7fc6b17fa640 1 -- 172.21.15.84:0/3368983128 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c401a030 con 0x564e93ab6f00 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.308+0000 7fc6b17fa640 1 -- 172.21.15.84:0/3368983128 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6c4014e40 con 0x564e93ab6f00 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.308+0000 7fc6b17fa640 1 -- 172.21.15.84:0/3368983128 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4012480 con 0x564e93ab6f00 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.308+0000 7fc6d43be740 1 -- 172.21.15.84:0/3368983128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 msgr2=0x564e93ab72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.308+0000 7fc6d43be740 1 --2- 172.21.15.84:0/3368983128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93ab72e0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x564e93ab45a0 tx=0x7fc6c4009c30 comp rx=0 tx=0).stop 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.308+0000 7fc6d43be740 1 -- 172.21.15.84:0/3368983128 shutdown_connections 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.308+0000 7fc6d43be740 1 --2- 172.21.15.84:0/3368983128 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ac20e0 0x564e93ac44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.308+0000 7fc6d43be740 1 --2- 172.21.15.84:0/3368983128 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ab7820 0x564e93ac1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.308+0000 7fc6d43be740 1 --2- 172.21.15.84:0/3368983128 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93ab72e0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.308+0000 7fc6d43be740 1 -- 172.21.15.84:0/3368983128 >> 172.21.15.84:0/3368983128 conn(0x564e93ab3410 msgr2=0x564e93ab37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.308+0000 7fc6d43be740 1 -- 172.21.15.84:0/3368983128 shutdown_connections 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.308+0000 7fc6d43be740 1 -- 172.21.15.84:0/3368983128 wait complete. 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6d43be740 1 -- start start 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6d43be740 1 Processor -- start 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6d43be740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93abf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6d43be740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ab7820 0x564e93abf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6d43be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ac20e0 0x564e93abfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6d43be740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x564e93ac4d10 con 0x564e93ab6f00 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6d43be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564e939a4b60 con 0x564e93ac20e0 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6d43be740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564e93ac4e90 con 0x564e93ab7820 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6d43be740 10 client.0 _ll_register_callbacks cb 0x564e939c92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6caffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93abf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6caffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93abf050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37614/0 (socket says 172.21.15.84:37614) 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6caffd640 1 -- 172.21.15.84:0/2431657751 learned_addr learned my addr 172.21.15.84:0/2431657751 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:21.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6cb7fe640 1 --2- 172.21.15.84:0/2431657751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ac20e0 0x564e93abfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6ca7fc640 1 --2- 172.21.15.84:0/2431657751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ab7820 0x564e93abf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6caffd640 1 -- 172.21.15.84:0/2431657751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ab7820 msgr2=0x564e93abf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6caffd640 1 --2- 172.21.15.84:0/2431657751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ab7820 0x564e93abf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6caffd640 1 -- 172.21.15.84:0/2431657751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ac20e0 msgr2=0x564e93abfad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6caffd640 1 --2- 172.21.15.84:0/2431657751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ac20e0 0x564e93abfad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.309+0000 7fc6caffd640 1 -- 172.21.15.84:0/2431657751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e9372f210 con 0x564e93ab6f00 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6caffd640 1 --2- 172.21.15.84:0/2431657751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93abf050 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0bc7e0 tx=0x7fc6bc0045a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08a04d0085e85fd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6aeff5640 1 -- 172.21.15.84:0/2431657751 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc048040 con 0x564e93ab6f00 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6aeff5640 1 -- 172.21.15.84:0/2431657751 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7fc6bc0b9150 con 0x564e93ab6f00 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6d43be740 1 -- 172.21.15.84:0/2431657751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564e93a2b070 con 0x564e93ab6f00 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6aeff5640 1 -- 172.21.15.84:0/2431657751 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6bc0dd070 con 0x564e93ab6f00 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6d43be740 10 client.16185 fetch_fsmap learned FSMap version 29 2024-04-26T18:34:21.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6d43be740 1 -- 172.21.15.84:0/2431657751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564e937fb620 con 0x564e93ab6f00 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6aeff5640 1 -- 172.21.15.84:0/2431657751 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x564e93a2b070 con 0x564e93ab6f00 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6aeff5640 1 -- 172.21.15.84:0/2431657751 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc6bc04b060 con 0x564e93ab6f00 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6d43be740 10 client.16185 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6d43be740 10 client.16185 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6d43be740 20 client.16185 populate_metadata read hostname 'smithi084' 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6d43be740 1 --2- 172.21.15.84:0/2431657751 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x564e93ad21f0 0x564e93ad4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6d43be740 4 client.16185 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.310+0000 7fc6d43be740 1 -- 172.21.15.84:0/2431657751 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e93a24ee0 con 0x564e93ad21f0 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.311+0000 7fc6ca7fc640 1 --2- 172.21.15.84:0/2431657751 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x564e93ad21f0 0x564e93ad4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.311+0000 7fc6ca7fc640 1 --2- 172.21.15.84:0/2431657751 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x564e93ad21f0 0x564e93ad4660 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5ea6749c8a0254e server_cookie=8f6d0c04c2ae671c in_seq=0 out_seq=0 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.311+0000 7fc6aeff5640 10 client.16185 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6aeff5640 1 -- 172.21.15.84:0/2431657751 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e93a24ee0 con 0x564e93ad21f0 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6aeff5640 10 client.16185 handle_command_reply: tid=0 2024-04-26T18:34:21.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6d43be740 1 client.16185 shutdown 2024-04-26T18:34:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6d43be740 1 -- 172.21.15.84:0/2431657751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 msgr2=0x564e93abf050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6d43be740 1 --2- 172.21.15.84:0/2431657751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93abf050 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0bc7e0 tx=0x7fc6bc0045a0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6d43be740 1 -- 172.21.15.84:0/2431657751 shutdown_connections 2024-04-26T18:34:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6d43be740 1 --2- 172.21.15.84:0/2431657751 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x564e93ad21f0 0x564e93ad4660 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6d43be740 1 --2- 172.21.15.84:0/2431657751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ac20e0 0x564e93abfad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6d43be740 1 --2- 172.21.15.84:0/2431657751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ab7820 0x564e93abf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6d43be740 1 --2- 172.21.15.84:0/2431657751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93abf050 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6aeff5640 0 client.16185 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6d43be740 1 -- 172.21.15.84:0/2431657751 >> 172.21.15.84:0/2431657751 conn(0x564e93ab3410 msgr2=0x564e93ab9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6d43be740 1 -- 172.21.15.84:0/2431657751 shutdown_connections 2024-04-26T18:34:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6d43be740 1 -- 172.21.15.84:0/2431657751 wait complete. 2024-04-26T18:34:21.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.312+0000 7fc6d43be740 20 client.16185 trim_cache size 0 max 16384 2024-04-26T18:34:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6d43be740 1 Processor -- start 2024-04-26T18:34:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6d43be740 1 -- start start 2024-04-26T18:34:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6d43be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ab6f00 0x564e93acdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6d43be740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab7820 0x564e93ab98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.331 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6d43be740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ac20e0 0x564e93ad9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6d43be740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1ae6b0 con 0x564e93ab7820 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6d43be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1b01f0 con 0x564e93ab6f00 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6d43be740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1aed80 con 0x564e93ac20e0 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6ca7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab7820 0x564e93ab98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6ca7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab7820 0x564e93ab98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37628/0 (socket says 172.21.15.84:37628) 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6ca7fc640 1 -- 172.21.15.84:0/9633824 learned_addr learned my addr 172.21.15.84:0/9633824 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6cb7fe640 1 --2- 172.21.15.84:0/9633824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ac20e0 0x564e93ad9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6ca7fc640 1 -- 172.21.15.84:0/9633824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ac20e0 msgr2=0x564e93ad9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6ca7fc640 1 --2- 172.21.15.84:0/9633824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ac20e0 0x564e93ad9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6ca7fc640 1 -- 172.21.15.84:0/9633824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ab6f00 msgr2=0x564e93acdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6ca7fc640 1 --2- 172.21.15.84:0/9633824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ab6f00 0x564e93acdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6ca7fc640 1 -- 172.21.15.84:0/9633824 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e93ad4ce0 con 0x564e93ab7820 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6cb7fe640 1 --2- 172.21.15.84:0/9633824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ac20e0 0x564e93ad9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:34:21.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.327+0000 7fc6ca7fc640 1 --2- 172.21.15.84:0/9633824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab7820 0x564e93ab98c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x564e93abe8b0 tx=0x7fc6c0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9cab6ea7035ab0d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.328+0000 7fc6b0ff9640 1 -- 172.21.15.84:0/9633824 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c0016070 con 0x564e93ab7820 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.328+0000 7fc6b0ff9640 1 -- 172.21.15.84:0/9633824 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6c002a070 con 0x564e93ab7820 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.328+0000 7fc6b0ff9640 1 -- 172.21.15.84:0/9633824 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c001b410 con 0x564e93ab7820 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.328+0000 7fc6d43be740 1 -- 172.21.15.84:0/9633824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab7820 msgr2=0x564e93ab98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.328+0000 7fc6d43be740 1 --2- 172.21.15.84:0/9633824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab7820 0x564e93ab98c0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x564e93abe8b0 tx=0x7fc6c0012000 comp rx=0 tx=0).stop 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.328+0000 7fc6d43be740 1 -- 172.21.15.84:0/9633824 shutdown_connections 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.328+0000 7fc6d43be740 1 --2- 172.21.15.84:0/9633824 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ac20e0 0x564e93ad9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.328+0000 7fc6d43be740 1 --2- 172.21.15.84:0/9633824 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab7820 0x564e93ab98c0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.328+0000 7fc6d43be740 1 --2- 172.21.15.84:0/9633824 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ab6f00 0x564e93acdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.328+0000 7fc6d43be740 1 -- 172.21.15.84:0/9633824 >> 172.21.15.84:0/9633824 conn(0x564e93ab3410 msgr2=0x564e93abf8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.328+0000 7fc6d43be740 1 -- 172.21.15.84:0/9633824 shutdown_connections 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.328+0000 7fc6d43be740 1 -- 172.21.15.84:0/9633824 wait complete. 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.328+0000 7fc6d43be740 1 -- start start 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6d43be740 1 Processor -- start 2024-04-26T18:34:21.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6d43be740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93ac59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6d43be740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ab7820 0x564e93ac5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6d43be740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ac20e0 0x564e93ac6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6d43be740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1afb20 con 0x564e93ab6f00 2024-04-26T18:34:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6d43be740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1af450 con 0x564e93ac20e0 2024-04-26T18:34:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6d43be740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc6cc1ac6b0 con 0x564e93ab7820 2024-04-26T18:34:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6d43be740 10 client.0 _ll_register_callbacks cb 0x7fc6bc009290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:34:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6caffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93ac59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6caffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93ac59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37638/0 (socket says 172.21.15.84:37638) 2024-04-26T18:34:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6caffd640 1 -- 172.21.15.84:0/4020563915 learned_addr learned my addr 172.21.15.84:0/4020563915 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6ca7fc640 1 --2- 172.21.15.84:0/4020563915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ab7820 0x564e93ac5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6cb7fe640 1 --2- 172.21.15.84:0/4020563915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ac20e0 0x564e93ac6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6caffd640 1 -- 172.21.15.84:0/4020563915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ab7820 msgr2=0x564e93ac5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6caffd640 1 --2- 172.21.15.84:0/4020563915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ab7820 0x564e93ac5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6caffd640 1 -- 172.21.15.84:0/4020563915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ac20e0 msgr2=0x564e93ac6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6caffd640 1 --2- 172.21.15.84:0/4020563915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ac20e0 0x564e93ac6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6caffd640 1 -- 172.21.15.84:0/4020563915 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e93b377e0 con 0x564e93ab6f00 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6caffd640 1 --2- 172.21.15.84:0/4020563915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93ac59f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009900 tx=0x7fc6bc0bc370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240f178ba569d91c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6aeff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6aeff5640 1 -- 172.21.15.84:0/4020563915 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc048070 con 0x564e93ab6f00 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6aeff5640 1 -- 172.21.15.84:0/4020563915 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7fc6bc0ee460 con 0x564e93ab6f00 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.329+0000 7fc6d43be740 1 -- 172.21.15.84:0/4020563915 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc6c002a070 con 0x564e93ab6f00 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.330+0000 7fc6aeff5640 1 -- 172.21.15.84:0/4020563915 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc6bc0c56a0 con 0x564e93ab6f00 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.330+0000 7fc6d43be740 10 client.16191 fetch_fsmap learned FSMap version 29 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.330+0000 7fc6d43be740 1 -- 172.21.15.84:0/4020563915 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564e93b38940 con 0x564e93ab6f00 2024-04-26T18:34:21.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.330+0000 7fc6aeff5640 1 -- 172.21.15.84:0/4020563915 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6c002a070 con 0x564e93ab6f00 2024-04-26T18:34:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.330+0000 7fc6aeff5640 1 -- 172.21.15.84:0/4020563915 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc6bc0a1080 con 0x564e93ab6f00 2024-04-26T18:34:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.330+0000 7fc6d43be740 10 client.16191 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:34:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.330+0000 7fc6d43be740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:34:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.330+0000 7fc6d43be740 20 client.16191 populate_metadata read hostname 'smithi084' 2024-04-26T18:34:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.330+0000 7fc6d43be740 1 --2- 172.21.15.84:0/4020563915 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x564e93b38f00 0x564e93b3b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.330+0000 7fc6d43be740 4 client.16191 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:34:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.330+0000 7fc6d43be740 1 -- 172.21.15.84:0/4020563915 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564e939a05f0 con 0x564e93b38f00 2024-04-26T18:34:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.330+0000 7fc6ca7fc640 1 --2- 172.21.15.84:0/4020563915 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x564e93b38f00 0x564e93b3b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.331+0000 7fc6aeff5640 10 client.16191 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:21.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.331+0000 7fc6ca7fc640 1 --2- 172.21.15.84:0/4020563915 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x564e93b38f00 0x564e93b3b320 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6b4e6814243281 server_cookie=8df801cbf29e4623 in_seq=0 out_seq=0 2024-04-26T18:34:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6aeff5640 1 -- 172.21.15.84:0/4020563915 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564e939a05f0 con 0x564e93b38f00 2024-04-26T18:34:21.345 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6aeff5640 10 client.16191 handle_command_reply: tid=0 2024-04-26T18:34:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6d43be740 1 client.16191 shutdown 2024-04-26T18:34:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6d43be740 1 -- 172.21.15.84:0/4020563915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 msgr2=0x564e93ac59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6d43be740 1 --2- 172.21.15.84:0/4020563915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93ac59f0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc009900 tx=0x7fc6bc0bc370 comp rx=0 tx=0).stop 2024-04-26T18:34:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6d43be740 1 -- 172.21.15.84:0/4020563915 shutdown_connections 2024-04-26T18:34:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6d43be740 1 --2- 172.21.15.84:0/4020563915 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x564e93b38f00 0x564e93b3b320 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6d43be740 1 --2- 172.21.15.84:0/4020563915 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564e93ac20e0 0x564e93ac6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6d43be740 1 --2- 172.21.15.84:0/4020563915 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564e93ab7820 0x564e93ac5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6d43be740 1 --2- 172.21.15.84:0/4020563915 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564e93ab6f00 0x564e93ac59f0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6aeff5640 0 client.16191 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6d43be740 1 -- 172.21.15.84:0/4020563915 >> 172.21.15.84:0/4020563915 conn(0x564e93ab3410 msgr2=0x564e93ab1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6d43be740 1 -- 172.21.15.84:0/4020563915 shutdown_connections 2024-04-26T18:34:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6d43be740 1 -- 172.21.15.84:0/4020563915 wait complete. 2024-04-26T18:34:21.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.345+0000 7fc6d43be740 20 client.16191 trim_cache size 0 max 16384 2024-04-26T18:34:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:34:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:34:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:34:21.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:34:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.346+0000 7fc6d15c6640 1 -- 172.21.15.84:0/4221647398 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc6b40a3dc0 msgr2=0x7fc6b40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.346+0000 7fc6d15c6640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc6b40a3dc0 0x7fc6b40a6280 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc6cc100ab0 tx=0x7fc6c000a2d0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.346+0000 7fc6d15c6640 1 -- 172.21.15.84:0/4221647398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6cc102930 msgr2=0x7fc6cc199140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.346+0000 7fc6d15c6640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6cc102930 0x7fc6cc199140 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc004770 tx=0x7fc6bc0042f0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.346+0000 7fc6d15c6640 1 -- 172.21.15.84:0/4221647398 shutdown_connections 2024-04-26T18:34:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.346+0000 7fc6d15c6640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc6b40a3dc0 0x7fc6b40a6280 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.346+0000 7fc6d15c6640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc6cc108940 0x7fc6cc19d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.346+0000 7fc6d15c6640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc6cc1032f0 0x7fc6cc199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.346+0000 7fc6d15c6640 1 --2- 172.21.15.84:0/4221647398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc6cc102930 0x7fc6cc199140 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.346+0000 7fc6d15c6640 1 -- 172.21.15.84:0/4221647398 >> 172.21.15.84:0/4221647398 conn(0x7fc6cc0fe420 msgr2=0x7fc6cc0fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.346+0000 7fc6d15c6640 1 -- 172.21.15.84:0/4221647398 shutdown_connections 2024-04-26T18:34:21.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:21.346+0000 7fc6d15c6640 1 -- 172.21.15.84:0/4221647398 wait complete. 2024-04-26T18:34:21.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:21 smithi103 ceph-mon[32727]: pgmap v196: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-26T18:34:21.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:21 smithi174 ceph-mon[32705]: pgmap v196: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-26T18:34:21.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:21 smithi084 ceph-mon[29141]: pgmap v196: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-26T18:34:23.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:23 smithi103 ceph-mon[32727]: pgmap v197: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-26T18:34:23.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:23 smithi174 ceph-mon[32705]: pgmap v197: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-26T18:34:23.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:23 smithi084 ceph-mon[29141]: pgmap v197: 97 pgs: 97 active+clean; 2.5 MiB data, 400 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-26T18:34:25.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:25 smithi174 ceph-mon[32705]: pgmap v198: 97 pgs: 97 active+clean; 2.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-04-26T18:34:25.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:25 smithi084 ceph-mon[29141]: pgmap v198: 97 pgs: 97 active+clean; 2.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-04-26T18:34:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:25 smithi103 ceph-mon[32727]: pgmap v198: 97 pgs: 97 active+clean; 2.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 23 op/s 2024-04-26T18:34:27.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:27 smithi174 ceph-mon[32705]: pgmap v199: 97 pgs: 97 active+clean; 2.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-26T18:34:27.748 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 31.7 2024-04-26T18:34:27.748 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:34:27.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:27 smithi084 ceph-mon[29141]: pgmap v199: 97 pgs: 97 active+clean; 2.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-26T18:34:27.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:27 smithi103 ceph-mon[32727]: pgmap v199: 97 pgs: 97 active+clean; 2.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-26T18:34:28.000 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:34:29.652 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:29 smithi084 ceph-mon[29141]: pgmap v200: 97 pgs: 97 active+clean; 2.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-26T18:34:29.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:29 smithi174 ceph-mon[32705]: pgmap v200: 97 pgs: 97 active+clean; 2.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-26T18:34:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.753+0000 7f61ec20f700 1 -- 172.21.15.84:0/2641163954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e41025b0 msgr2=0x7f61e4102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.753+0000 7f61ec20f700 1 --2- 172.21.15.84:0/2641163954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e41025b0 0x7f61e4102990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f61e00099c0 tx=0x7f61e0009cd0 comp rx=0 tx=0).stop 2024-04-26T18:34:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.753+0000 7f61ec20f700 1 -- 172.21.15.84:0/2641163954 shutdown_connections 2024-04-26T18:34:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.753+0000 7f61ec20f700 1 --2- 172.21.15.84:0/2641163954 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e41025b0 0x7f61e4102990 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.753+0000 7f61ec20f700 1 --2- 172.21.15.84:0/2641163954 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61e4107640 0x7f61e410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.753+0000 7f61ec20f700 1 --2- 172.21.15.84:0/2641163954 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61e4102f60 0x7f61e4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:29.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.753+0000 7f61ec20f700 1 -- 172.21.15.84:0/2641163954 >> 172.21.15.84:0/2641163954 conn(0x7f61e40792f0 msgr2=0x7f61e4079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:29.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.753+0000 7f61ec20f700 1 -- 172.21.15.84:0/2641163954 shutdown_connections 2024-04-26T18:34:29.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.753+0000 7f61ec20f700 1 -- 172.21.15.84:0/2641163954 wait complete. 2024-04-26T18:34:29.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.753+0000 7f61ec20f700 1 Processor -- start 2024-04-26T18:34:29.755 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.753+0000 7f61ec20f700 1 -- start start 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61ec20f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61e41025b0 0x7f61e419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61ec20f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e4102f60 0x7f61e419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61ec20f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61e4107640 0x7f61e4199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61ec20f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f61e4112550 con 0x7f61e4102f60 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61ec20f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41123d0 con 0x7f61e41025b0 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61ec20f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f61e41126d0 con 0x7f61e4107640 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61e97aa700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e4102f60 0x7f61e419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61e97aa700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e4102f60 0x7f61e419fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60104/0 (socket says 172.21.15.84:60104) 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61e97aa700 1 -- 172.21.15.84:0/4125031065 learned_addr learned my addr 172.21.15.84:0/4125031065 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61e97aa700 1 -- 172.21.15.84:0/4125031065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61e4107640 msgr2=0x7f61e4199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61e97aa700 1 --2- 172.21.15.84:0/4125031065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61e4107640 0x7f61e4199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61e97aa700 1 -- 172.21.15.84:0/4125031065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61e41025b0 msgr2=0x7f61e419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61e97aa700 1 --2- 172.21.15.84:0/4125031065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61e41025b0 0x7f61e419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:29.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61e97aa700 1 -- 172.21.15.84:0/4125031065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e419a180 con 0x7f61e4102f60 2024-04-26T18:34:29.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61e97aa700 1 --2- 172.21.15.84:0/4125031065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e4102f60 0x7f61e419fd50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f61d400d980 tx=0x7f61d400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:29.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61daffd700 1 -- 172.21.15.84:0/4125031065 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d4017070 con 0x7f61e4102f60 2024-04-26T18:34:29.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61ec20f700 1 -- 172.21.15.84:0/4125031065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e419a470 con 0x7f61e4102f60 2024-04-26T18:34:29.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61daffd700 1 -- 172.21.15.84:0/4125031065 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f61d4010460 con 0x7f61e4102f60 2024-04-26T18:34:29.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61daffd700 1 -- 172.21.15.84:0/4125031065 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61d400de60 con 0x7f61e4102f60 2024-04-26T18:34:29.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.754+0000 7f61ec20f700 1 -- 172.21.15.84:0/4125031065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e41a66e0 con 0x7f61e4102f60 2024-04-26T18:34:29.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.755+0000 7f61ec20f700 1 -- 172.21.15.84:0/4125031065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e406a110 con 0x7f61e4102f60 2024-04-26T18:34:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.756+0000 7f61daffd700 1 -- 172.21.15.84:0/4125031065 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f61d4010610 con 0x7f61e4102f60 2024-04-26T18:34:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.757+0000 7f61daffd700 1 --2- 172.21.15.84:0/4125031065 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f61d00a3db0 0x7f61d00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.757+0000 7f61e9fab700 1 --2- 172.21.15.84:0/4125031065 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f61d00a3db0 0x7f61d00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.757+0000 7f61daffd700 1 -- 172.21.15.84:0/4125031065 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f61d40d0430 con 0x7f61e4102f60 2024-04-26T18:34:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.758+0000 7f61e9fab700 1 --2- 172.21.15.84:0/4125031065 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f61d00a3db0 0x7f61d00a6270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009fd0 tx=0x7f61e0000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:29.761+0000 7f61daffd700 1 -- 172.21.15.84:0/4125031065 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f61d409ca90 con 0x7f61e4102f60 2024-04-26T18:34:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:29 smithi103 ceph-mon[32727]: pgmap v200: 97 pgs: 97 active+clean; 2.7 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-26T18:34:30.007 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.007+0000 7f61ec20f700 1 -- 172.21.15.84:0/4125031065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f61e402e490 con 0x7f61e4102f60 2024-04-26T18:34:30.008 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.008+0000 7f61daffd700 1 -- 172.21.15.84:0/4125031065 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f61d409c390 con 0x7f61e4102f60 2024-04-26T18:34:30.008 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:34:30.008 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:34:30.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.010+0000 7f61ec20f700 1 -- 172.21.15.84:0/4125031065 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f61d00a3db0 msgr2=0x7f61d00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:30.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.010+0000 7f61ec20f700 1 --2- 172.21.15.84:0/4125031065 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f61d00a3db0 0x7f61d00a6270 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f61e0009fd0 tx=0x7f61e0000bc0 comp rx=0 tx=0).stop 2024-04-26T18:34:30.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.010+0000 7f61ec20f700 1 -- 172.21.15.84:0/4125031065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e4102f60 msgr2=0x7f61e419fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:30.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.010+0000 7f61ec20f700 1 --2- 172.21.15.84:0/4125031065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e4102f60 0x7f61e419fd50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f61d400d980 tx=0x7f61d400dd40 comp rx=0 tx=0).stop 2024-04-26T18:34:30.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.010+0000 7f61ec20f700 1 -- 172.21.15.84:0/4125031065 shutdown_connections 2024-04-26T18:34:30.011 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.010+0000 7f61ec20f700 1 --2- 172.21.15.84:0/4125031065 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f61d00a3db0 0x7f61d00a6270 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.010+0000 7f61ec20f700 1 --2- 172.21.15.84:0/4125031065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f61e4107640 0x7f61e4199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.010+0000 7f61ec20f700 1 --2- 172.21.15.84:0/4125031065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f61e41025b0 0x7f61e419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.010+0000 7f61ec20f700 1 --2- 172.21.15.84:0/4125031065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f61e4102f60 0x7f61e419fd50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.010+0000 7f61ec20f700 1 -- 172.21.15.84:0/4125031065 >> 172.21.15.84:0/4125031065 conn(0x7f61e40792f0 msgr2=0x7f61e406bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.010+0000 7f61ec20f700 1 -- 172.21.15.84:0/4125031065 shutdown_connections 2024-04-26T18:34:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:30.010+0000 7f61ec20f700 1 -- 172.21.15.84:0/4125031065 wait complete. 2024-04-26T18:34:30.012 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:34:30.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:30 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/4125031065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:34:30.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:30 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/4125031065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:34:30.812 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 60 --expiration 91.7 --await-for 42.0' 2024-04-26T18:34:30.812 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db / --timeout 60 --expiration 91.7 --await-for 42.0 2024-04-26T18:34:30.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:30 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/4125031065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:34:31.101 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:34:31.487 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:31 smithi084 ceph-mon[29141]: pgmap v201: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-26T18:34:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:31 smithi174 ceph-mon[32705]: pgmap v201: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-26T18:34:31.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:31 smithi103 ceph-mon[32727]: pgmap v201: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-26T18:34:32.916 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.914+0000 7f94811d8700 1 -- 172.21.15.84:0/1750271033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c1034d0 msgr2=0x7f947c103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.914+0000 7f94811d8700 1 --2- 172.21.15.84:0/1750271033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c1034d0 0x7f947c103950 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f946c0099c0 tx=0x7f946c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.914+0000 7f94811d8700 1 -- 172.21.15.84:0/1750271033 shutdown_connections 2024-04-26T18:34:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.914+0000 7f94811d8700 1 --2- 172.21.15.84:0/1750271033 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f947c103e90 0x7f947c1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.914+0000 7f94811d8700 1 --2- 172.21.15.84:0/1750271033 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f947c1094a0 0x7f947c109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.914+0000 7f94811d8700 1 --2- 172.21.15.84:0/1750271033 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c1034d0 0x7f947c103950 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.914+0000 7f94811d8700 1 -- 172.21.15.84:0/1750271033 >> 172.21.15.84:0/1750271033 conn(0x7f947c07a4f0 msgr2=0x7f947c07a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.914+0000 7f94811d8700 1 -- 172.21.15.84:0/1750271033 shutdown_connections 2024-04-26T18:34:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.914+0000 7f94811d8700 1 -- 172.21.15.84:0/1750271033 wait complete. 2024-04-26T18:34:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.914+0000 7f94811d8700 1 Processor -- start 2024-04-26T18:34:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f94811d8700 1 -- start start 2024-04-26T18:34:32.917 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f94811d8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f947c1034d0 0x7f947c075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f94811d8700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c103e90 0x7f947c075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f94811d8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f947c1094a0 0x7f947c0761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f94811d8700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1136f0 con 0x7f947c103e90 2024-04-26T18:34:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f94811d8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f947c113570 con 0x7f947c1034d0 2024-04-26T18:34:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f94811d8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f947c113870 con 0x7f947c1094a0 2024-04-26T18:34:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f947a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c103e90 0x7f947c075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f947a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c103e90 0x7f947c075c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60122/0 (socket says 172.21.15.84:60122) 2024-04-26T18:34:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f947a59c700 1 -- 172.21.15.84:0/611961332 learned_addr learned my addr 172.21.15.84:0/611961332 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f947ad9d700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f947c1034d0 0x7f947c075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f947b59e700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f947c1094a0 0x7f947c0761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.918 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f947a59c700 1 -- 172.21.15.84:0/611961332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f947c1094a0 msgr2=0x7f947c0761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f947a59c700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f947c1094a0 0x7f947c0761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f947a59c700 1 -- 172.21.15.84:0/611961332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f947c1034d0 msgr2=0x7f947c075720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f947a59c700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f947c1034d0 0x7f947c075720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f947a59c700 1 -- 172.21.15.84:0/611961332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947c1a8110 con 0x7f947c103e90 2024-04-26T18:34:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.915+0000 7f947b59e700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f947c1094a0 0x7f947c0761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:34:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.916+0000 7f947a59c700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c103e90 0x7f947c075c60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f946c000c00 tx=0x7f946c0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.916+0000 7f9463fff700 1 -- 172.21.15.84:0/611961332 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c027070 con 0x7f947c103e90 2024-04-26T18:34:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.916+0000 7f94811d8700 1 -- 172.21.15.84:0/611961332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f947c1a83a0 con 0x7f947c103e90 2024-04-26T18:34:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.916+0000 7f9463fff700 1 -- 172.21.15.84:0/611961332 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f946c004020 con 0x7f947c103e90 2024-04-26T18:34:32.919 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.916+0000 7f9463fff700 1 -- 172.21.15.84:0/611961332 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c022de0 con 0x7f947c103e90 2024-04-26T18:34:32.920 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.916+0000 7f94811d8700 1 -- 172.21.15.84:0/611961332 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f947c1a87f0 con 0x7f947c103e90 2024-04-26T18:34:32.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.919+0000 7f9463fff700 1 -- 172.21.15.84:0/611961332 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f946c02a070 con 0x7f947c103e90 2024-04-26T18:34:32.928 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.919+0000 7f9463fff700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f94640a3e50 0x7f94640a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.919+0000 7f9463fff700 1 -- 172.21.15.84:0/611961332 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f946c0d15d0 con 0x7f947c103e90 2024-04-26T18:34:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.919+0000 7f947ad9d700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f94640a3e50 0x7f94640a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.920+0000 7f947ad9d700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f94640a3e50 0x7f94640a6310 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9470009a10 tx=0x7f9470009190 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.920+0000 7f948ba6ab80 1 Processor -- start 2024-04-26T18:34:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.920+0000 7f948ba6ab80 1 -- start start 2024-04-26T18:34:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.920+0000 7f948ba6ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56460528ef30 0x56460528f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.920+0000 7f948ba6ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x5646052892e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.920+0000 7f948ba6ab80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 0x564605289d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.920+0000 7f948ba6ab80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5646051db9c0 con 0x564605289900 2024-04-26T18:34:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.920+0000 7f948ba6ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564605141ac0 con 0x56460528f850 2024-04-26T18:34:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.920+0000 7f948ba6ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646051d7980 con 0x56460528ef30 2024-04-26T18:34:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.920+0000 7f947b59e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 0x564605289d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.929 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.920+0000 7f947b59e700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 0x564605289d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60124/0 (socket says 172.21.15.84:60124) 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.920+0000 7f947b59e700 1 -- 172.21.15.84:0/944539166 learned_addr learned my addr 172.21.15.84:0/944539166 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.920+0000 7f947a59c700 1 --2- 172.21.15.84:0/944539166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x5646052892e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.921+0000 7f947b59e700 1 -- 172.21.15.84:0/944539166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56460528ef30 msgr2=0x56460528f310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.921+0000 7f947b59e700 1 --2- 172.21.15.84:0/944539166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56460528ef30 0x56460528f310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.921+0000 7f947b59e700 1 -- 172.21.15.84:0/944539166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 msgr2=0x5646052892e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.921+0000 7f947b59e700 1 --2- 172.21.15.84:0/944539166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x5646052892e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.921+0000 7f947b59e700 1 -- 172.21.15.84:0/944539166 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564605128a60 con 0x564605289900 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.921+0000 7f947a59c700 1 --2- 172.21.15.84:0/944539166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x5646052892e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.921+0000 7f947b59e700 1 --2- 172.21.15.84:0/944539166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 0x564605289d60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f947400c450 tx=0x7f947400c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8af6d31a7bb0c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.921+0000 7f94617fa700 1 -- 172.21.15.84:0/944539166 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947401a070 con 0x564605289900 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.921+0000 7f94617fa700 1 -- 172.21.15.84:0/944539166 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f947400c0d0 con 0x564605289900 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.921+0000 7f94617fa700 1 -- 172.21.15.84:0/944539166 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9474015dc0 con 0x564605289900 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 -- 172.21.15.84:0/944539166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 msgr2=0x564605289d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/944539166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 0x564605289d60 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f947400c450 tx=0x7f947400c480 comp rx=0 tx=0).stop 2024-04-26T18:34:32.930 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 -- 172.21.15.84:0/944539166 shutdown_connections 2024-04-26T18:34:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/944539166 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 0x564605289d60 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/944539166 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56460528ef30 0x56460528f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/944539166 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x5646052892e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 -- 172.21.15.84:0/944539166 >> 172.21.15.84:0/944539166 conn(0x5646051b7750 msgr2=0x564605284090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 -- 172.21.15.84:0/944539166 shutdown_connections 2024-04-26T18:34:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 -- 172.21.15.84:0/944539166 wait complete. 2024-04-26T18:34:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 -- start start 2024-04-26T18:34:32.931 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 Processor -- start 2024-04-26T18:34:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564605289900 0x564605295560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528ef30 0x564605292900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x564605292e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5646052962c0 con 0x56460528ef30 2024-04-26T18:34:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x564605195c10 con 0x56460528f850 2024-04-26T18:34:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x564605296440 con 0x564605289900 2024-04-26T18:34:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f948ba6ab80 10 client.0 _ll_register_callbacks cb 0x564605173ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:34:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.922+0000 7f947a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528ef30 0x564605292900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f947a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528ef30 0x564605292900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60136/0 (socket says 172.21.15.84:60136) 2024-04-26T18:34:32.932 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f947a59c700 1 -- 172.21.15.84:0/2352896158 learned_addr learned my addr 172.21.15.84:0/2352896158 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f947b59e700 1 --2- 172.21.15.84:0/2352896158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x564605292e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f947ad9d700 1 --2- 172.21.15.84:0/2352896158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564605289900 0x564605295560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f947a59c700 1 -- 172.21.15.84:0/2352896158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564605289900 msgr2=0x564605295560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f947a59c700 1 --2- 172.21.15.84:0/2352896158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564605289900 0x564605295560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f947a59c700 1 -- 172.21.15.84:0/2352896158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 msgr2=0x564605292e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f947a59c700 1 --2- 172.21.15.84:0/2352896158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x564605292e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f947a59c700 1 -- 172.21.15.84:0/2352896158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564605199610 con 0x56460528ef30 2024-04-26T18:34:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f947a59c700 1 --2- 172.21.15.84:0/2352896158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528ef30 0x564605292900 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f946c099f90 tx=0x7f946c09d180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a70b2ef0024319e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f945eff5700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:34:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f945eff5700 1 -- 172.21.15.84:0/2352896158 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c0b6060 con 0x56460528ef30 2024-04-26T18:34:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f945eff5700 1 -- 172.21.15.84:0/2352896158 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f946c0a5d10 con 0x56460528ef30 2024-04-26T18:34:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f948ba6ab80 1 -- 172.21.15.84:0/2352896158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f947400c0d0 con 0x56460528ef30 2024-04-26T18:34:32.933 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f945eff5700 1 -- 172.21.15.84:0/2352896158 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f946c09dcc0 con 0x56460528ef30 2024-04-26T18:34:32.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f948ba6ab80 10 client.16209 fetch_fsmap learned FSMap version 29 2024-04-26T18:34:32.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f948ba6ab80 1 -- 172.21.15.84:0/2352896158 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646052a0b00 con 0x56460528ef30 2024-04-26T18:34:32.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.923+0000 7f945eff5700 1 -- 172.21.15.84:0/2352896158 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f947400c0d0 con 0x56460528ef30 2024-04-26T18:34:32.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.924+0000 7f945eff5700 1 -- 172.21.15.84:0/2352896158 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f946c09caa0 con 0x56460528ef30 2024-04-26T18:34:32.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.924+0000 7f948ba6ab80 10 client.16209 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:34:32.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.924+0000 7f948ba6ab80 10 client.16209 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:34:32.934 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.924+0000 7f948ba6ab80 20 client.16209 populate_metadata read hostname 'smithi084' 2024-04-26T18:34:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.924+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/2352896158 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5646052a3070 0x5646052a5490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.924+0000 7f948ba6ab80 4 client.16209 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:34:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.924+0000 7f948ba6ab80 1 -- 172.21.15.84:0/2352896158 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646051e93a0 con 0x5646052a3070 2024-04-26T18:34:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.924+0000 7f947ad9d700 1 --2- 172.21.15.84:0/2352896158 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5646052a3070 0x5646052a5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.924+0000 7f945eff5700 10 client.16209 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.924+0000 7f947ad9d700 1 --2- 172.21.15.84:0/2352896158 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5646052a3070 0x5646052a5490 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b1801e6ddeb8848 server_cookie=bf0b2c82f104b805 in_seq=0 out_seq=0 2024-04-26T18:34:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f945eff5700 1 -- 172.21.15.84:0/2352896158 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646051e93a0 con 0x5646052a3070 2024-04-26T18:34:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f945eff5700 10 client.16209 handle_command_reply: tid=0 2024-04-26T18:34:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f948ba6ab80 1 client.16209 shutdown 2024-04-26T18:34:32.935 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f948ba6ab80 1 -- 172.21.15.84:0/2352896158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528ef30 msgr2=0x564605292900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/2352896158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528ef30 0x564605292900 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f946c099f90 tx=0x7f946c09d180 comp rx=0 tx=0).stop 2024-04-26T18:34:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f948ba6ab80 1 -- 172.21.15.84:0/2352896158 shutdown_connections 2024-04-26T18:34:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/2352896158 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5646052a3070 0x5646052a5490 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/2352896158 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x564605292e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/2352896158 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x564605289900 0x564605295560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f945eff5700 0 client.16209 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/2352896158 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528ef30 0x564605292900 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.936 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f948ba6ab80 1 -- 172.21.15.84:0/2352896158 >> 172.21.15.84:0/2352896158 conn(0x5646051b7750 msgr2=0x5646052882d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:32.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f948ba6ab80 1 -- 172.21.15.84:0/2352896158 shutdown_connections 2024-04-26T18:34:32.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.925+0000 7f948ba6ab80 1 -- 172.21.15.84:0/2352896158 wait complete. 2024-04-26T18:34:32.937 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.926+0000 7f948ba6ab80 20 client.16209 trim_cache size 0 max 16384 2024-04-26T18:34:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f948ba6ab80 1 Processor -- start 2024-04-26T18:34:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f948ba6ab80 1 -- start start 2024-04-26T18:34:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f948ba6ab80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 0x5646052892e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f948ba6ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56460528ef30 0x56460529f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f948ba6ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x564605290360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f948ba6ab80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1afed0 con 0x564605289900 2024-04-26T18:34:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f948ba6ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1b1a10 con 0x56460528f850 2024-04-26T18:34:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f948ba6ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1b05a0 con 0x56460528ef30 2024-04-26T18:34:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f947ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 0x5646052892e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f947ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 0x5646052892e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60140/0 (socket says 172.21.15.84:60140) 2024-04-26T18:34:32.946 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f947ad9d700 1 -- 172.21.15.84:0/50280925 learned_addr learned my addr 172.21.15.84:0/50280925 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f947b59e700 1 --2- 172.21.15.84:0/50280925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x564605290360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f947ad9d700 1 -- 172.21.15.84:0/50280925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56460528ef30 msgr2=0x56460529f1c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T18:34:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f947ad9d700 1 --2- 172.21.15.84:0/50280925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56460528ef30 0x56460529f1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f947ad9d700 1 -- 172.21.15.84:0/50280925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 msgr2=0x564605290360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f947ad9d700 1 --2- 172.21.15.84:0/50280925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x564605290360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f947ad9d700 1 -- 172.21.15.84:0/50280925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564605284590 con 0x564605289900 2024-04-26T18:34:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f947b59e700 1 --2- 172.21.15.84:0/50280925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x564605290360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:34:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.945+0000 7f947ad9d700 1 --2- 172.21.15.84:0/50280925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 0x5646052892e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x5646052a5ce0 tx=0x7f9470011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccf2d5a38197f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.946+0000 7f9460ff9700 1 -- 172.21.15.84:0/50280925 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f947001b040 con 0x564605289900 2024-04-26T18:34:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.946+0000 7f9460ff9700 1 -- 172.21.15.84:0/50280925 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f947002a030 con 0x564605289900 2024-04-26T18:34:32.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.946+0000 7f9460ff9700 1 -- 172.21.15.84:0/50280925 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94700158b0 con 0x564605289900 2024-04-26T18:34:32.950 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.946+0000 7f948ba6ab80 1 -- 172.21.15.84:0/50280925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 msgr2=0x5646052892e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:32.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.946+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/50280925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 0x5646052892e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x5646052a5ce0 tx=0x7f9470011000 comp rx=0 tx=0).stop 2024-04-26T18:34:32.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.946+0000 7f948ba6ab80 1 -- 172.21.15.84:0/50280925 shutdown_connections 2024-04-26T18:34:32.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.946+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/50280925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x564605289900 0x5646052892e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.946+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/50280925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56460528f850 0x564605290360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.946+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/50280925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56460528ef30 0x56460529f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.946+0000 7f948ba6ab80 1 -- 172.21.15.84:0/50280925 >> 172.21.15.84:0/50280925 conn(0x5646051b7750 msgr2=0x56460527df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:32.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.947+0000 7f948ba6ab80 1 -- 172.21.15.84:0/50280925 shutdown_connections 2024-04-26T18:34:32.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.947+0000 7f948ba6ab80 1 -- 172.21.15.84:0/50280925 wait complete. 2024-04-26T18:34:32.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.947+0000 7f948ba6ab80 1 -- start start 2024-04-26T18:34:32.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.947+0000 7f948ba6ab80 1 Processor -- start 2024-04-26T18:34:32.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.947+0000 7f948ba6ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56460528ef30 0x56460529b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.947+0000 7f948ba6ab80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528f850 0x56460529bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.951 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.947+0000 7f948ba6ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564605298c00 0x564605299080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.947+0000 7f948ba6ab80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1b1340 con 0x56460528f850 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.947+0000 7f948ba6ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1b0c70 con 0x564605298c00 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.947+0000 7f948ba6ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f947c1ade90 con 0x56460528ef30 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.947+0000 7f948ba6ab80 10 client.0 _ll_register_callbacks cb 0x564605173ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f947a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528f850 0x56460529bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f947a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528f850 0x56460529bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60150/0 (socket says 172.21.15.84:60150) 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f947a59c700 1 -- 172.21.15.84:0/4201368772 learned_addr learned my addr 172.21.15.84:0/4201368772 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f947a59c700 1 -- 172.21.15.84:0/4201368772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56460528ef30 msgr2=0x56460529b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f947a59c700 1 --2- 172.21.15.84:0/4201368772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56460528ef30 0x56460529b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f947a59c700 1 -- 172.21.15.84:0/4201368772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564605298c00 msgr2=0x564605299080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f947a59c700 1 --2- 172.21.15.84:0/4201368772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564605298c00 0x564605299080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f947a59c700 1 -- 172.21.15.84:0/4201368772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646052998c0 con 0x56460528f850 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f947a59c700 1 --2- 172.21.15.84:0/4201368772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528f850 0x56460529bbf0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f946c09b870 tx=0x7f946c022470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6994ae539b99e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f945eff5700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:34:32.952 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f945eff5700 1 -- 172.21.15.84:0/4201368772 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c0a5f10 con 0x56460528f850 2024-04-26T18:34:32.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f945eff5700 1 -- 172.21.15.84:0/4201368772 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f946c003b70 con 0x56460528f850 2024-04-26T18:34:32.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f948ba6ab80 1 -- 172.21.15.84:0/4201368772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f947002a030 con 0x56460528f850 2024-04-26T18:34:32.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f945eff5700 1 -- 172.21.15.84:0/4201368772 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f946c09d5c0 con 0x56460528f850 2024-04-26T18:34:32.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f948ba6ab80 10 client.16215 fetch_fsmap learned FSMap version 29 2024-04-26T18:34:32.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.948+0000 7f948ba6ab80 1 -- 172.21.15.84:0/4201368772 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56460530b600 con 0x56460528f850 2024-04-26T18:34:32.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.949+0000 7f945eff5700 1 -- 172.21.15.84:0/4201368772 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f947002a030 con 0x56460528f850 2024-04-26T18:34:32.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.949+0000 7f945eff5700 1 -- 172.21.15.84:0/4201368772 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f946c0a2ce0 con 0x56460528f850 2024-04-26T18:34:32.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.949+0000 7f948ba6ab80 10 client.16215 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:34:32.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.949+0000 7f948ba6ab80 10 client.16215 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:34:32.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.949+0000 7f948ba6ab80 20 client.16215 populate_metadata read hostname 'smithi084' 2024-04-26T18:34:32.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.949+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/4201368772 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56460530bb60 0x56460530df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:32.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.949+0000 7f948ba6ab80 4 client.16215 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 91.7, "await_for": 42.0}] 2024-04-26T18:34:32.953 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.949+0000 7f948ba6ab80 1 -- 172.21.15.84:0/4201368772 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 91.7, "await_for": 42.0}) -- 0x5646051e3440 con 0x56460530bb60 2024-04-26T18:34:32.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.949+0000 7f947ad9d700 1 --2- 172.21.15.84:0/4201368772 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56460530bb60 0x56460530df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:32.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.949+0000 7f947ad9d700 1 --2- 172.21.15.84:0/4201368772 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56460530bb60 0x56460530df80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a229b239b643a4a6 server_cookie=b73ef7c430fff575 in_seq=0 out_seq=0 2024-04-26T18:34:32.954 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.949+0000 7f945eff5700 10 client.16215 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:32.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.965+0000 7f945eff5700 1 -- 172.21.15.84:0/4201368772 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5646051e3440 con 0x56460530bb60 2024-04-26T18:34:32.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.965+0000 7f945eff5700 10 client.16215 handle_command_reply: tid=0 2024-04-26T18:34:32.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.965+0000 7f948ba6ab80 1 client.16215 shutdown 2024-04-26T18:34:32.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.965+0000 7f948ba6ab80 1 -- 172.21.15.84:0/4201368772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528f850 msgr2=0x56460529bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:32.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.965+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/4201368772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528f850 0x56460529bbf0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f946c09b870 tx=0x7f946c022470 comp rx=0 tx=0).stop 2024-04-26T18:34:32.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f948ba6ab80 1 -- 172.21.15.84:0/4201368772 shutdown_connections 2024-04-26T18:34:32.966 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/4201368772 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56460530bb60 0x56460530df80 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/4201368772 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x564605298c00 0x564605299080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f945eff5700 0 client.16215 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:32.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/4201368772 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56460528ef30 0x56460529b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f948ba6ab80 1 --2- 172.21.15.84:0/4201368772 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56460528f850 0x56460529bbf0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f948ba6ab80 1 -- 172.21.15.84:0/4201368772 >> 172.21.15.84:0/4201368772 conn(0x5646051b7750 msgr2=0x564605292a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:32.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f948ba6ab80 1 -- 172.21.15.84:0/4201368772 shutdown_connections 2024-04-26T18:34:32.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f948ba6ab80 1 -- 172.21.15.84:0/4201368772 wait complete. 2024-04-26T18:34:32.967 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f948ba6ab80 20 client.16215 trim_cache size 0 max 16384 2024-04-26T18:34:32.967 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 18, 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "db61a169": { 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 18, 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 91.7, 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:34:32.968 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:34:32.969 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:34:32.969 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:34:32.969 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:34:32.969 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:34:32.969 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:34:32.969 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:34:32.969 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:34:32.969 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:34:32.969 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:34:32.969 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:34:32.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f94811d8700 1 -- 172.21.15.84:0/611961332 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f94640a3e50 msgr2=0x7f94640a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:32.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f94811d8700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f94640a3e50 0x7f94640a6310 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9470009a10 tx=0x7f9470009190 comp rx=0 tx=0).stop 2024-04-26T18:34:32.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f94811d8700 1 -- 172.21.15.84:0/611961332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c103e90 msgr2=0x7f947c075c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:32.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f94811d8700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c103e90 0x7f947c075c60 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f946c000c00 tx=0x7f946c0038b0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f94811d8700 1 -- 172.21.15.84:0/611961332 shutdown_connections 2024-04-26T18:34:32.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f94811d8700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f94640a3e50 0x7f94640a6310 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f94811d8700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f947c1094a0 0x7f947c0761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f94811d8700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f947c1034d0 0x7f947c075720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.970 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f94811d8700 1 --2- 172.21.15.84:0/611961332 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f947c103e90 0x7f947c075c60 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:32.971 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f94811d8700 1 -- 172.21.15.84:0/611961332 >> 172.21.15.84:0/611961332 conn(0x7f947c07a4f0 msgr2=0x7f947c1022e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:32.971 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.966+0000 7f94811d8700 1 -- 172.21.15.84:0/611961332 shutdown_connections 2024-04-26T18:34:32.971 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:34:32.967+0000 7f94811d8700 1 -- 172.21.15.84:0/611961332 wait complete. 2024-04-26T18:34:33.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:33 smithi174 ceph-mon[32705]: pgmap v202: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-26T18:34:33.719 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:33 smithi084 ceph-mon[29141]: pgmap v202: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-26T18:34:33.751 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'db61a169', quiesce took 6.0 seconds. Will release after: 31.7 2024-04-26T18:34:33.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:33 smithi103 ceph-mon[32727]: pgmap v202: 97 pgs: 97 active+clean; 2.8 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-26T18:34:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.444+0000 7f18b7f4e640 1 -- 172.21.15.84:0/2821617542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b0101860 msgr2=0x7f18b0101ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.444+0000 7f18b7f4e640 1 --2- 172.21.15.84:0/2821617542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b0101860 0x7f18b0101ce0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f18a0009930 tx=0x7f18a002a420 comp rx=0 tx=0).stop 2024-04-26T18:34:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.444+0000 7f18b7f4e640 1 -- 172.21.15.84:0/2821617542 shutdown_connections 2024-04-26T18:34:34.446 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.444+0000 7f18b7f4e640 1 --2- 172.21.15.84:0/2821617542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b0102220 0x7f18b010e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.444+0000 7f18b7f4e640 1 --2- 172.21.15.84:0/2821617542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b0101860 0x7f18b0101ce0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.444+0000 7f18b7f4e640 1 --2- 172.21.15.84:0/2821617542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18b0107820 0x7f18b0107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.444+0000 7f18b7f4e640 1 -- 172.21.15.84:0/2821617542 >> 172.21.15.84:0/2821617542 conn(0x7f18b00789b0 msgr2=0x7f18b0078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.444+0000 7f18b7f4e640 1 -- 172.21.15.84:0/2821617542 shutdown_connections 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.444+0000 7f18b7f4e640 1 -- 172.21.15.84:0/2821617542 wait complete. 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b7f4e640 1 Processor -- start 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b7f4e640 1 -- start start 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b7f4e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b0101860 0x7f18b00fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b7f4e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18b0102220 0x7f18b00fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b7f4e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b0107820 0x7f18b00ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b7f4e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0111ae0 con 0x7f18b0107820 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b7f4e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0111960 con 0x7f18b0102220 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b7f4e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f18b0111c60 con 0x7f18b0101860 2024-04-26T18:34:34.447 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b64eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b0107820 0x7f18b00ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b64eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b0107820 0x7f18b00ff270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60172/0 (socket says 172.21.15.84:60172) 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b64eb640 1 -- 172.21.15.84:0/615798765 learned_addr learned my addr 172.21.15.84:0/615798765 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b54e9640 1 --2- 172.21.15.84:0/615798765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18b0102220 0x7f18b00fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b64eb640 1 -- 172.21.15.84:0/615798765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b0101860 msgr2=0x7f18b00fe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b64eb640 1 --2- 172.21.15.84:0/615798765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b0101860 0x7f18b00fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b64eb640 1 -- 172.21.15.84:0/615798765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18b0102220 msgr2=0x7f18b00fed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b64eb640 1 --2- 172.21.15.84:0/615798765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18b0102220 0x7f18b00fed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.445+0000 7f18b64eb640 1 -- 172.21.15.84:0/615798765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18b01a7590 con 0x7f18b0107820 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.446+0000 7f18b64eb640 1 --2- 172.21.15.84:0/615798765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b0107820 0x7f18b00ff270 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f18a800eaa0 tx=0x7f18a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.446+0000 7f18a6ffd640 1 -- 172.21.15.84:0/615798765 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a800cdb0 con 0x7f18b0107820 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.446+0000 7f18a6ffd640 1 -- 172.21.15.84:0/615798765 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a8005020 con 0x7f18b0107820 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.446+0000 7f18b7f4e640 1 -- 172.21.15.84:0/615798765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18b01a7880 con 0x7f18b0107820 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.446+0000 7f18a6ffd640 1 -- 172.21.15.84:0/615798765 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a80116d0 con 0x7f18b0107820 2024-04-26T18:34:34.448 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.446+0000 7f18b7f4e640 1 -- 172.21.15.84:0/615798765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18b01a7df0 con 0x7f18b0107820 2024-04-26T18:34:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.448+0000 7f18a6ffd640 1 -- 172.21.15.84:0/615798765 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f18a80051d0 con 0x7f18b0107820 2024-04-26T18:34:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.448+0000 7f18bac60740 1 Processor -- start 2024-04-26T18:34:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18bac60740 1 -- start start 2024-04-26T18:34:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18bac60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce572b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18a6ffd640 1 --2- 172.21.15.84:0/615798765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f189c0a3e70 0x7f189c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18bac60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce572b820 0x559ce5735ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18bac60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce57360e0 0x559ce57384d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18bac60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559ce5691460 con 0x559ce572af00 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18bac60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559ce564b590 con 0x559ce572b820 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18bac60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ce568d420 con 0x559ce57360e0 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18b54e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce572b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18b5cea640 1 --2- 172.21.15.84:0/615798765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f189c0a3e70 0x7f189c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18b54e9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce572b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60176/0 (socket says 172.21.15.84:60176) 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18b54e9640 1 -- 172.21.15.84:0/3333388587 learned_addr learned my addr 172.21.15.84:0/3333388587 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18a6ffd640 1 -- 172.21.15.84:0/615798765 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f18a80d09c0 con 0x7f18b0107820 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18b54e9640 1 --2- 172.21.15.84:0/3333388587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce572b820 0x559ce5735ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18b54e9640 1 -- 172.21.15.84:0/3333388587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce57360e0 msgr2=0x559ce57384d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18b54e9640 1 --2- 172.21.15.84:0/3333388587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce57360e0 0x559ce57384d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18b54e9640 1 -- 172.21.15.84:0/3333388587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce572b820 msgr2=0x559ce5735ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18b54e9640 1 --2- 172.21.15.84:0/3333388587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce572b820 0x559ce5735ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.449+0000 7f18b54e9640 1 -- 172.21.15.84:0/3333388587 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce56580d0 con 0x559ce572af00 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.450+0000 7f18b5cea640 1 --2- 172.21.15.84:0/615798765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f189c0a3e70 0x7f189c0a6330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18ac005d80 tx=0x7f18ac005c90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.450+0000 7f18b54e9640 1 --2- 172.21.15.84:0/3333388587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce572b2e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f18a0000c00 tx=0x7f18a0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=843bf97b6a0261bc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.450+0000 7f189bfff640 1 -- 172.21.15.84:0/3333388587 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0044030 con 0x559ce572af00 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.450+0000 7f189bfff640 1 -- 172.21.15.84:0/3333388587 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a0031660 con 0x559ce572af00 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.450+0000 7f189bfff640 1 -- 172.21.15.84:0/3333388587 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a0035040 con 0x559ce572af00 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.451+0000 7f18bac60740 1 -- 172.21.15.84:0/3333388587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 msgr2=0x559ce572b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.451+0000 7f18bac60740 1 --2- 172.21.15.84:0/3333388587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce572b2e0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f18a0000c00 tx=0x7f18a0005c00 comp rx=0 tx=0).stop 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.451+0000 7f18bac60740 1 -- 172.21.15.84:0/3333388587 shutdown_connections 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.451+0000 7f18bac60740 1 --2- 172.21.15.84:0/3333388587 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce57360e0 0x559ce57384d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.451+0000 7f18bac60740 1 --2- 172.21.15.84:0/3333388587 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce572b820 0x559ce5735ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.451+0000 7f18bac60740 1 --2- 172.21.15.84:0/3333388587 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce572b2e0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.451+0000 7f18bac60740 1 -- 172.21.15.84:0/3333388587 >> 172.21.15.84:0/3333388587 conn(0x559ce5727410 msgr2=0x559ce57277f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:34.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.451+0000 7f18bac60740 1 -- 172.21.15.84:0/3333388587 shutdown_connections 2024-04-26T18:34:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.451+0000 7f18bac60740 1 -- 172.21.15.84:0/3333388587 wait complete. 2024-04-26T18:34:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.451+0000 7f18bac60740 1 -- start start 2024-04-26T18:34:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.451+0000 7f18bac60740 1 Processor -- start 2024-04-26T18:34:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18bac60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce572af00 0x559ce5733140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18bac60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce572b820 0x559ce5733680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18bac60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce57360e0 0x559ce5733bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18bac60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559ce5738cb0 con 0x559ce57360e0 2024-04-26T18:34:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18bac60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559ce5618b60 con 0x559ce572b820 2024-04-26T18:34:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18bac60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559ce5738e30 con 0x559ce572af00 2024-04-26T18:34:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18bac60740 10 client.0 _ll_register_callbacks cb 0x559ce563d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:34:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18b64eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce57360e0 0x559ce5733bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18b64eb640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce57360e0 0x559ce5733bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60178/0 (socket says 172.21.15.84:60178) 2024-04-26T18:34:34.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18b64eb640 1 -- 172.21.15.84:0/977333545 learned_addr learned my addr 172.21.15.84:0/977333545 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18b5cea640 1 --2- 172.21.15.84:0/977333545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce572af00 0x559ce5733140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18b64eb640 1 -- 172.21.15.84:0/977333545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce572af00 msgr2=0x559ce5733140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18b54e9640 1 --2- 172.21.15.84:0/977333545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce572b820 0x559ce5733680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18b64eb640 1 --2- 172.21.15.84:0/977333545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce572af00 0x559ce5733140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18b64eb640 1 -- 172.21.15.84:0/977333545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce572b820 msgr2=0x559ce5733680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18b64eb640 1 --2- 172.21.15.84:0/977333545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce572b820 0x559ce5733680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18b64eb640 1 -- 172.21.15.84:0/977333545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce53a3210 con 0x559ce57360e0 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18b64eb640 1 --2- 172.21.15.84:0/977333545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce57360e0 0x559ce5733bc0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f18a80ce0d0 tx=0x7f18a809d100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32c5d8a1eb40094 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18997fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18997fa640 1 -- 172.21.15.84:0/977333545 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18a801f040 con 0x559ce57360e0 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18bac60740 1 -- 172.21.15.84:0/977333545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18a0031660 con 0x559ce57360e0 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.452+0000 7f18997fa640 1 -- 172.21.15.84:0/977333545 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f18a80be280 con 0x559ce57360e0 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.453+0000 7f18997fa640 1 -- 172.21.15.84:0/977333545 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18a80054f0 con 0x559ce57360e0 2024-04-26T18:34:34.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.453+0000 7f18bac60740 10 client.16227 fetch_fsmap learned FSMap version 29 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.453+0000 7f18bac60740 1 -- 172.21.15.84:0/977333545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ce546f620 con 0x559ce57360e0 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.453+0000 7f18997fa640 1 -- 172.21.15.84:0/977333545 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18a0031660 con 0x559ce57360e0 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.453+0000 7f18997fa640 1 -- 172.21.15.84:0/977333545 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f18a800f040 con 0x559ce57360e0 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.453+0000 7f18bac60740 10 client.16227 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.453+0000 7f18bac60740 10 client.16227 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.453+0000 7f18bac60740 20 client.16227 populate_metadata read hostname 'smithi084' 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.453+0000 7f18bac60740 1 --2- 172.21.15.84:0/977333545 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559ce57462b0 0x559ce57486a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.453+0000 7f18bac60740 4 client.16227 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.453+0000 7f18bac60740 1 -- 172.21.15.84:0/977333545 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559ce569f070 con 0x559ce57462b0 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.453+0000 7f18b5cea640 1 --2- 172.21.15.84:0/977333545 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559ce57462b0 0x559ce57486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.454+0000 7f18b5cea640 1 --2- 172.21.15.84:0/977333545 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559ce57462b0 0x559ce57486a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a32a4d834fe7256c server_cookie=3da46c43f4eeda2f in_seq=0 out_seq=0 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.454+0000 7f18997fa640 10 client.16227 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.454+0000 7f18997fa640 1 -- 172.21.15.84:0/977333545 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559ce569f070 con 0x559ce57462b0 2024-04-26T18:34:34.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.454+0000 7f18997fa640 10 client.16227 handle_command_reply: tid=0 2024-04-26T18:34:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.454+0000 7f18bac60740 1 client.16227 shutdown 2024-04-26T18:34:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.455+0000 7f18bac60740 1 -- 172.21.15.84:0/977333545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce57360e0 msgr2=0x559ce5733bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.455+0000 7f18bac60740 1 --2- 172.21.15.84:0/977333545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce57360e0 0x559ce5733bc0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f18a80ce0d0 tx=0x7f18a809d100 comp rx=0 tx=0).stop 2024-04-26T18:34:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.455+0000 7f18bac60740 1 -- 172.21.15.84:0/977333545 shutdown_connections 2024-04-26T18:34:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.455+0000 7f18bac60740 1 --2- 172.21.15.84:0/977333545 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559ce57462b0 0x559ce57486a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.455+0000 7f18bac60740 1 --2- 172.21.15.84:0/977333545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce57360e0 0x559ce5733bc0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.455+0000 7f18bac60740 1 --2- 172.21.15.84:0/977333545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce572b820 0x559ce5733680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.455+0000 7f18997fa640 0 client.16227 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.455+0000 7f18bac60740 1 --2- 172.21.15.84:0/977333545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce572af00 0x559ce5733140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.455+0000 7f18bac60740 1 -- 172.21.15.84:0/977333545 >> 172.21.15.84:0/977333545 conn(0x559ce5727410 msgr2=0x559ce572d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.455+0000 7f18bac60740 1 -- 172.21.15.84:0/977333545 shutdown_connections 2024-04-26T18:34:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.455+0000 7f18bac60740 1 -- 172.21.15.84:0/977333545 wait complete. 2024-04-26T18:34:34.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.455+0000 7f18bac60740 20 client.16227 trim_cache size 0 max 16384 2024-04-26T18:34:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18bac60740 1 Processor -- start 2024-04-26T18:34:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18bac60740 1 -- start start 2024-04-26T18:34:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18bac60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce572d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18bac60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce572b820 0x559ce574df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18bac60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce57360e0 0x559ce574e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18bac60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b0840 con 0x559ce572af00 2024-04-26T18:34:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18bac60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b2380 con 0x559ce57360e0 2024-04-26T18:34:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18bac60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b0f10 con 0x559ce572b820 2024-04-26T18:34:34.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18b5cea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce572d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18b5cea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce572d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60182/0 (socket says 172.21.15.84:60182) 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18b5cea640 1 -- 172.21.15.84:0/4069316249 learned_addr learned my addr 172.21.15.84:0/4069316249 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18b5cea640 1 -- 172.21.15.84:0/4069316249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce572b820 msgr2=0x559ce574df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18b5cea640 1 --2- 172.21.15.84:0/4069316249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce572b820 0x559ce574df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18b5cea640 1 -- 172.21.15.84:0/4069316249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce57360e0 msgr2=0x559ce574e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18b5cea640 1 --2- 172.21.15.84:0/4069316249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce57360e0 0x559ce574e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18b5cea640 1 -- 172.21.15.84:0/4069316249 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce57322c0 con 0x559ce572af00 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f18b5cea640 1 --2- 172.21.15.84:0/4069316249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce572d010 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f18ac005fa0 tx=0x7f18ac013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f58546d02ea569 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.470+0000 7f189b7fe640 1 -- 172.21.15.84:0/4069316249 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac0108b0 con 0x559ce572af00 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.471+0000 7f189b7fe640 1 -- 172.21.15.84:0/4069316249 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18ac02b070 con 0x559ce572af00 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.471+0000 7f189b7fe640 1 -- 172.21.15.84:0/4069316249 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac01e650 con 0x559ce572af00 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.471+0000 7f18bac60740 1 -- 172.21.15.84:0/4069316249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 msgr2=0x559ce572d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.471+0000 7f18bac60740 1 --2- 172.21.15.84:0/4069316249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce572d010 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f18ac005fa0 tx=0x7f18ac013000 comp rx=0 tx=0).stop 2024-04-26T18:34:34.475 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.471+0000 7f18bac60740 1 -- 172.21.15.84:0/4069316249 shutdown_connections 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.471+0000 7f18bac60740 1 --2- 172.21.15.84:0/4069316249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce57360e0 0x559ce574e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.471+0000 7f18bac60740 1 --2- 172.21.15.84:0/4069316249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce572b820 0x559ce574df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.471+0000 7f18bac60740 1 --2- 172.21.15.84:0/4069316249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce572d010 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.471+0000 7f18bac60740 1 -- 172.21.15.84:0/4069316249 >> 172.21.15.84:0/4069316249 conn(0x559ce5727410 msgr2=0x559ce572c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.471+0000 7f18bac60740 1 -- 172.21.15.84:0/4069316249 shutdown_connections 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.471+0000 7f18bac60740 1 -- 172.21.15.84:0/4069316249 wait complete. 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.471+0000 7f18bac60740 1 -- start start 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18bac60740 1 Processor -- start 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18bac60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce573e270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18bac60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce572b820 0x559ce573b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18bac60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce57360e0 0x559ce573b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18bac60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b1cb0 con 0x559ce572af00 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18bac60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01b15e0 con 0x559ce57360e0 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18bac60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f18b01ae840 con 0x559ce572b820 2024-04-26T18:34:34.476 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18bac60740 10 client.0 _ll_register_callbacks cb 0x7f18a800cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18b5cea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce573e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18b5cea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce573e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60196/0 (socket says 172.21.15.84:60196) 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18b5cea640 1 -- 172.21.15.84:0/2739633604 learned_addr learned my addr 172.21.15.84:0/2739633604 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18b5cea640 1 -- 172.21.15.84:0/2739633604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce572b820 msgr2=0x559ce573b050 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18b5cea640 1 --2- 172.21.15.84:0/2739633604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce572b820 0x559ce573b050 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18b5cea640 1 -- 172.21.15.84:0/2739633604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce57360e0 msgr2=0x559ce573b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18b5cea640 1 --2- 172.21.15.84:0/2739633604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce57360e0 0x559ce573b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.472+0000 7f18b5cea640 1 -- 172.21.15.84:0/2739633604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce573bb00 con 0x559ce572af00 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18b5cea640 1 --2- 172.21.15.84:0/2739633604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce573e270 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f18ac012040 tx=0x7f18ac025310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1243049592b82819 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18997fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18997fa640 1 -- 172.21.15.84:0/2739633604 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18ac036070 con 0x559ce572af00 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18997fa640 1 -- 172.21.15.84:0/2739633604 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f18ac0108b0 con 0x559ce572af00 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18bac60740 1 -- 172.21.15.84:0/2739633604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18ac02b070 con 0x559ce572af00 2024-04-26T18:34:34.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18997fa640 1 -- 172.21.15.84:0/2739633604 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f18ac042050 con 0x559ce572af00 2024-04-26T18:34:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18bac60740 10 client.16233 fetch_fsmap learned FSMap version 29 2024-04-26T18:34:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18bac60740 1 -- 172.21.15.84:0/2739633604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559ce57ad510 con 0x559ce572af00 2024-04-26T18:34:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18997fa640 1 -- 172.21.15.84:0/2739633604 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18ac02b070 con 0x559ce572af00 2024-04-26T18:34:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18997fa640 1 -- 172.21.15.84:0/2739633604 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f18ac02fcb0 con 0x559ce572af00 2024-04-26T18:34:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18bac60740 10 client.16233 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:34:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18bac60740 10 client.16233 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:34:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18bac60740 20 client.16233 populate_metadata read hostname 'smithi084' 2024-04-26T18:34:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18bac60740 1 --2- 172.21.15.84:0/2739633604 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559ce57adbf0 0x559ce57b0030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18bac60740 4 client.16233 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:34:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18bac60740 1 -- 172.21.15.84:0/2739633604 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559ce5727260 con 0x559ce57adbf0 2024-04-26T18:34:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.473+0000 7f18b54e9640 1 --2- 172.21.15.84:0/2739633604 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559ce57adbf0 0x559ce57b0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.474+0000 7f18b54e9640 1 --2- 172.21.15.84:0/2739633604 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559ce57adbf0 0x559ce57b0030 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fcfb5a4dc004fc1 server_cookie=ad04e7fc0623c182 in_seq=0 out_seq=0 2024-04-26T18:34:34.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.474+0000 7f18997fa640 10 client.16233 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.491+0000 7f18997fa640 1 -- 172.21.15.84:0/2739633604 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559ce5727260 con 0x559ce57adbf0 2024-04-26T18:34:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.491+0000 7f18997fa640 10 client.16233 handle_command_reply: tid=0 2024-04-26T18:34:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.491+0000 7f18bac60740 1 client.16233 shutdown 2024-04-26T18:34:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.491+0000 7f18bac60740 1 -- 172.21.15.84:0/2739633604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 msgr2=0x559ce573e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.491+0000 7f18bac60740 1 --2- 172.21.15.84:0/2739633604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce573e270 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f18ac012040 tx=0x7f18ac025310 comp rx=0 tx=0).stop 2024-04-26T18:34:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.491+0000 7f18bac60740 1 -- 172.21.15.84:0/2739633604 shutdown_connections 2024-04-26T18:34:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.491+0000 7f18bac60740 1 --2- 172.21.15.84:0/2739633604 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559ce57adbf0 0x559ce57b0030 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.491+0000 7f18bac60740 1 --2- 172.21.15.84:0/2739633604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559ce57360e0 0x559ce573b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.491+0000 7f18bac60740 1 --2- 172.21.15.84:0/2739633604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559ce572b820 0x559ce573b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.491+0000 7f18997fa640 0 client.16233 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:34.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.491+0000 7f18bac60740 1 --2- 172.21.15.84:0/2739633604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559ce572af00 0x559ce573e270 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18bac60740 1 -- 172.21.15.84:0/2739633604 >> 172.21.15.84:0/2739633604 conn(0x559ce5727410 msgr2=0x559ce572d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18bac60740 1 -- 172.21.15.84:0/2739633604 shutdown_connections 2024-04-26T18:34:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18bac60740 1 -- 172.21.15.84:0/2739633604 wait complete. 2024-04-26T18:34:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18bac60740 20 client.16233 trim_cache size 0 max 16384 2024-04-26T18:34:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:34:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:34:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:34:34.493 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:34:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18b7f4e640 1 -- 172.21.15.84:0/615798765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f189c0a3e70 msgr2=0x7f189c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18b7f4e640 1 --2- 172.21.15.84:0/615798765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f189c0a3e70 0x7f189c0a6330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f18ac005d80 tx=0x7f18ac005c90 comp rx=0 tx=0).stop 2024-04-26T18:34:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18b7f4e640 1 -- 172.21.15.84:0/615798765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b0107820 msgr2=0x7f18b00ff270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18b7f4e640 1 --2- 172.21.15.84:0/615798765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b0107820 0x7f18b00ff270 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f18a800eaa0 tx=0x7f18a800ef60 comp rx=0 tx=0).stop 2024-04-26T18:34:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18b7f4e640 1 -- 172.21.15.84:0/615798765 shutdown_connections 2024-04-26T18:34:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18b7f4e640 1 --2- 172.21.15.84:0/615798765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f189c0a3e70 0x7f189c0a6330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.494 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18b7f4e640 1 --2- 172.21.15.84:0/615798765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f18b0107820 0x7f18b00ff270 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18b7f4e640 1 --2- 172.21.15.84:0/615798765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f18b0102220 0x7f18b00fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18b7f4e640 1 --2- 172.21.15.84:0/615798765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f18b0101860 0x7f18b00fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18b7f4e640 1 -- 172.21.15.84:0/615798765 >> 172.21.15.84:0/615798765 conn(0x7f18b00789b0 msgr2=0x7f18b0100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18b7f4e640 1 -- 172.21.15.84:0/615798765 shutdown_connections 2024-04-26T18:34:34.495 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:34.492+0000 7f18b7f4e640 1 -- 172.21.15.84:0/615798765 wait complete. 2024-04-26T18:34:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:35 smithi174 ceph-mon[32705]: pgmap v203: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-26T18:34:35.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:35 smithi084 ceph-mon[29141]: pgmap v203: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-26T18:34:35.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:35 smithi103 ceph-mon[32727]: pgmap v203: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-26T18:34:37.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:37 smithi174 ceph-mon[32705]: pgmap v204: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-26T18:34:37.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:37 smithi084 ceph-mon[29141]: pgmap v204: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-26T18:34:37.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:37 smithi103 ceph-mon[32727]: pgmap v204: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-26T18:34:39.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:39 smithi174 ceph-mon[32705]: pgmap v205: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-26T18:34:39.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:39 smithi084 ceph-mon[29141]: pgmap v205: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-26T18:34:39.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:39 smithi103 ceph-mon[32727]: pgmap v205: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 12 op/s 2024-04-26T18:34:41.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:41 smithi084 ceph-mon[29141]: pgmap v206: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-04-26T18:34:41.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:41 smithi174 ceph-mon[32705]: pgmap v206: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-04-26T18:34:41.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:41 smithi103 ceph-mon[32727]: pgmap v206: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-04-26T18:34:43.691 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:43 smithi174 ceph-mon[32705]: pgmap v207: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-04-26T18:34:43.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:43 smithi084 ceph-mon[29141]: pgmap v207: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-04-26T18:34:43.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:43 smithi103 ceph-mon[32727]: pgmap v207: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 19 KiB/s wr, 13 op/s 2024-04-26T18:34:45.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:45 smithi174 ceph-mon[32705]: pgmap v208: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-26T18:34:45.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:45 smithi084 ceph-mon[29141]: pgmap v208: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-26T18:34:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:45 smithi103 ceph-mon[32727]: pgmap v208: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 17 op/s 2024-04-26T18:34:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.587+0000 7f9dc8f24640 1 -- 172.21.15.84:0/1179573216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc4107820 msgr2=0x7f9dc4107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.587+0000 7f9dc8f24640 1 --2- 172.21.15.84:0/1179573216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc4107820 0x7f9dc4107c00 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f9dac009930 tx=0x7f9dac02a440 comp rx=0 tx=0).stop 2024-04-26T18:34:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.587+0000 7f9dc8f24640 1 -- 172.21.15.84:0/1179573216 shutdown_connections 2024-04-26T18:34:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.587+0000 7f9dc8f24640 1 --2- 172.21.15.84:0/1179573216 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dc4102220 0x7f9dc410e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.587+0000 7f9dc8f24640 1 --2- 172.21.15.84:0/1179573216 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc4101860 0x7f9dc4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.587+0000 7f9dc8f24640 1 --2- 172.21.15.84:0/1179573216 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc4107820 0x7f9dc4107c00 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.587+0000 7f9dc8f24640 1 -- 172.21.15.84:0/1179573216 >> 172.21.15.84:0/1179573216 conn(0x7f9dc40789b0 msgr2=0x7f9dc4078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.587+0000 7f9dc8f24640 1 -- 172.21.15.84:0/1179573216 shutdown_connections 2024-04-26T18:34:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.587+0000 7f9dc8f24640 1 -- 172.21.15.84:0/1179573216 wait complete. 2024-04-26T18:34:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc8f24640 1 Processor -- start 2024-04-26T18:34:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc8f24640 1 -- start start 2024-04-26T18:34:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc8f24640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc4101860 0x7f9dc4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc8f24640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc4102220 0x7f9dc4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc8f24640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dc4107820 0x7f9dc419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc8f24640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4111b20 con 0x7f9dc4102220 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc8f24640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41119a0 con 0x7f9dc4101860 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc8f24640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc4111ca0 con 0x7f9dc4107820 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc4102220 0x7f9dc4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc4102220 0x7f9dc4198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35288/0 (socket says 172.21.15.84:35288) 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc1d9b640 1 -- 172.21.15.84:0/751269266 learned_addr learned my addr 172.21.15.84:0/751269266 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc259c640 1 --2- 172.21.15.84:0/751269266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc4101860 0x7f9dc4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc1d9b640 1 -- 172.21.15.84:0/751269266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dc4107820 msgr2=0x7f9dc419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc1d9b640 1 --2- 172.21.15.84:0/751269266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dc4107820 0x7f9dc419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc1d9b640 1 -- 172.21.15.84:0/751269266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc4101860 msgr2=0x7f9dc4198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc1d9b640 1 --2- 172.21.15.84:0/751269266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc4101860 0x7f9dc4198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.588+0000 7f9dc1d9b640 1 -- 172.21.15.84:0/751269266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9dc419d010 con 0x7f9dc4102220 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.589+0000 7f9dc1d9b640 1 --2- 172.21.15.84:0/751269266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc4102220 0x7f9dc4198560 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9db400d770 tx=0x7f9db400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.589+0000 7f9dbb7fe640 1 -- 172.21.15.84:0/751269266 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4017070 con 0x7f9dc4102220 2024-04-26T18:34:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.589+0000 7f9dc8f24640 1 -- 172.21.15.84:0/751269266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9dc419d300 con 0x7f9dc4102220 2024-04-26T18:34:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.589+0000 7f9dbb7fe640 1 -- 172.21.15.84:0/751269266 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9db40054d0 con 0x7f9dc4102220 2024-04-26T18:34:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.589+0000 7f9dbb7fe640 1 -- 172.21.15.84:0/751269266 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db4020060 con 0x7f9dc4102220 2024-04-26T18:34:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.589+0000 7f9dc8f24640 1 -- 172.21.15.84:0/751269266 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9dc41a4b90 con 0x7f9dc4102220 2024-04-26T18:34:47.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.591+0000 7f9dbb7fe640 1 -- 172.21.15.84:0/751269266 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9db4004c10 con 0x7f9dc4102220 2024-04-26T18:34:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.591+0000 7f9dcbc44740 1 Processor -- start 2024-04-26T18:34:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dcbc44740 1 -- start start 2024-04-26T18:34:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dcbc44740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98210f00 0x563d982112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dcbc44740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98211820 0x563d9821bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dcbc44740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9821c0e0 0x563d9821e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dcbc44740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563d98177460 con 0x563d98210f00 2024-04-26T18:34:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dcbc44740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d98131590 con 0x563d9821c0e0 2024-04-26T18:34:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dcbc44740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563d98173420 con 0x563d98211820 2024-04-26T18:34:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dc2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98210f00 0x563d982112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dc2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98210f00 0x563d982112e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35296/0 (socket says 172.21.15.84:35296) 2024-04-26T18:34:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dbb7fe640 1 --2- 172.21.15.84:0/751269266 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9db00a3dc0 0x7f9db00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dc2d9d640 1 -- 172.21.15.84:0/2784031986 learned_addr learned my addr 172.21.15.84:0/2784031986 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:47.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dc259c640 1 --2- 172.21.15.84:0/2784031986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98211820 0x563d9821bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dc2d9d640 1 --2- 172.21.15.84:0/751269266 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9db00a3dc0 0x7f9db00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dc1d9b640 1 --2- 172.21.15.84:0/2784031986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9821c0e0 0x563d9821e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dc2d9d640 1 -- 172.21.15.84:0/2784031986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98211820 msgr2=0x563d9821bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dc2d9d640 1 --2- 172.21.15.84:0/2784031986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98211820 0x563d9821bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dc2d9d640 1 -- 172.21.15.84:0/2784031986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9821c0e0 msgr2=0x563d9821e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dc2d9d640 1 --2- 172.21.15.84:0/2784031986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9821c0e0 0x563d9821e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dbb7fe640 1 -- 172.21.15.84:0/751269266 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f9db40d0390 con 0x7f9dc4102220 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dc2d9d640 1 -- 172.21.15.84:0/2784031986 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d9813e0d0 con 0x563d98210f00 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.592+0000 7f9dc1d9b640 1 --2- 172.21.15.84:0/2784031986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9821c0e0 0x563d9821e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.593+0000 7f9dc2d9d640 1 --2- 172.21.15.84:0/2784031986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98210f00 0x563d982112e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc012710 tx=0x7f9dbc012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f145893653576ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.593+0000 7f9db8ff9640 1 -- 172.21.15.84:0/2784031986 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc01b070 con 0x563d98210f00 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.593+0000 7f9db8ff9640 1 -- 172.21.15.84:0/2784031986 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dbc013040 con 0x563d98210f00 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.593+0000 7f9dc2d9d640 1 --2- 172.21.15.84:0/751269266 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9db00a3dc0 0x7f9db00a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc01d000 tx=0x7f9dbc01b000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.593+0000 7f9db8ff9640 1 -- 172.21.15.84:0/2784031986 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc017400 con 0x563d98210f00 2024-04-26T18:34:47.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 -- 172.21.15.84:0/2784031986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98210f00 msgr2=0x563d982112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/2784031986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98210f00 0x563d982112e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc012710 tx=0x7f9dbc012bd0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 -- 172.21.15.84:0/2784031986 shutdown_connections 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/2784031986 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9821c0e0 0x563d9821e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/2784031986 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98211820 0x563d9821bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/2784031986 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98210f00 0x563d982112e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 -- 172.21.15.84:0/2784031986 >> 172.21.15.84:0/2784031986 conn(0x563d9820d410 msgr2=0x563d9820d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 -- 172.21.15.84:0/2784031986 shutdown_connections 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 -- 172.21.15.84:0/2784031986 wait complete. 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 -- start start 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 Processor -- start 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98210f00 0x563d98219070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 0x563d982195b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9821c0e0 0x563d98219af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563d9821ed10 con 0x563d98211820 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.594+0000 7f9dcbc44740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563d980feb60 con 0x563d9821c0e0 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dcbc44740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563d9821ee90 con 0x563d98210f00 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dcbc44740 10 client.0 _ll_register_callbacks cb 0x563d981232b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dc1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 0x563d982195b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dc1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 0x563d982195b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35312/0 (socket says 172.21.15.84:35312) 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dc1d9b640 1 -- 172.21.15.84:0/2682585694 learned_addr learned my addr 172.21.15.84:0/2682585694 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dc259c640 1 --2- 172.21.15.84:0/2682585694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98210f00 0x563d98219070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dc1d9b640 1 -- 172.21.15.84:0/2682585694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98210f00 msgr2=0x563d98219070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dc1d9b640 1 --2- 172.21.15.84:0/2682585694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98210f00 0x563d98219070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dc1d9b640 1 -- 172.21.15.84:0/2682585694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9821c0e0 msgr2=0x563d98219af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dc1d9b640 1 --2- 172.21.15.84:0/2682585694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9821c0e0 0x563d98219af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dc1d9b640 1 -- 172.21.15.84:0/2682585694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d97e89210 con 0x563d98211820 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dc259c640 1 --2- 172.21.15.84:0/2682585694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98210f00 0x563d98219070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:34:47.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dc1d9b640 1 --2- 172.21.15.84:0/2682585694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 0x563d982195b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9db40cdb50 tx=0x7f9db4017ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a785dc19ec77803 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9da9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9da9ffb640 1 -- 172.21.15.84:0/2682585694 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db40b4020 con 0x563d98211820 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9da9ffb640 1 -- 172.21.15.84:0/2682585694 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f9db4098960 con 0x563d98211820 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dcbc44740 1 -- 172.21.15.84:0/2682585694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dbc013040 con 0x563d98211820 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9da9ffb640 1 -- 172.21.15.84:0/2682585694 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9db4096e60 con 0x563d98211820 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dcbc44740 10 client.16245 fetch_fsmap learned FSMap version 29 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.595+0000 7f9dcbc44740 1 -- 172.21.15.84:0/2682585694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d97f55620 con 0x563d98211820 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.596+0000 7f9da9ffb640 1 -- 172.21.15.84:0/2682585694 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dbc013040 con 0x563d98211820 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.596+0000 7f9da9ffb640 1 -- 172.21.15.84:0/2682585694 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9db401eb60 con 0x563d98211820 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.596+0000 7f9dcbc44740 10 client.16245 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.596+0000 7f9dcbc44740 10 client.16245 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.596+0000 7f9dcbc44740 20 client.16245 populate_metadata read hostname 'smithi084' 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.596+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/2682585694 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563d9822c290 0x563d9822e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.596+0000 7f9dcbc44740 4 client.16245 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.596+0000 7f9dcbc44740 1 -- 172.21.15.84:0/2682585694 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d98185070 con 0x563d9822c290 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.596+0000 7f9dc259c640 1 --2- 172.21.15.84:0/2682585694 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563d9822c290 0x563d9822e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9dc259c640 1 --2- 172.21.15.84:0/2682585694 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563d9822c290 0x563d9822e720 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d98f8011d207aadb server_cookie=fc49ca82aff9263 in_seq=0 out_seq=0 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9da9ffb640 10 client.16245 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9da9ffb640 1 -- 172.21.15.84:0/2682585694 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d98185070 con 0x563d9822c290 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9da9ffb640 10 client.16245 handle_command_reply: tid=0 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9dcbc44740 1 client.16245 shutdown 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9dcbc44740 1 -- 172.21.15.84:0/2682585694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 msgr2=0x563d982195b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/2682585694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 0x563d982195b0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f9db40cdb50 tx=0x7f9db4017ea0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9dcbc44740 1 -- 172.21.15.84:0/2682585694 shutdown_connections 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/2682585694 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563d9822c290 0x563d9822e720 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/2682585694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d9821c0e0 0x563d98219af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9da9ffb640 0 client.16245 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:47.604 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/2682585694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 0x563d982195b0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/2682585694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98210f00 0x563d98219070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.597+0000 7f9dcbc44740 1 -- 172.21.15.84:0/2682585694 >> 172.21.15.84:0/2682585694 conn(0x563d9820d410 msgr2=0x563d98213140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.598+0000 7f9dcbc44740 1 -- 172.21.15.84:0/2682585694 shutdown_connections 2024-04-26T18:34:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.598+0000 7f9dcbc44740 1 -- 172.21.15.84:0/2682585694 wait complete. 2024-04-26T18:34:47.605 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.598+0000 7f9dcbc44740 20 client.16245 trim_cache size 0 max 16384 2024-04-26T18:34:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.612+0000 7f9dcbc44740 1 Processor -- start 2024-04-26T18:34:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.612+0000 7f9dcbc44740 1 -- start start 2024-04-26T18:34:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.612+0000 7f9dcbc44740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98210f00 0x563d98227bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.612+0000 7f9dcbc44740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d98211820 0x563d982242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.612+0000 7f9dcbc44740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d9821c0e0 0x563d982128b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.612+0000 7f9dcbc44740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41ad440 con 0x563d9821c0e0 2024-04-26T18:34:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.612+0000 7f9dcbc44740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41aef80 con 0x563d98211820 2024-04-26T18:34:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.612+0000 7f9dcbc44740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41adb10 con 0x563d98210f00 2024-04-26T18:34:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.612+0000 7f9dc2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d9821c0e0 0x563d982128b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dc2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d9821c0e0 0x563d982128b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35328/0 (socket says 172.21.15.84:35328) 2024-04-26T18:34:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dc2d9d640 1 -- 172.21.15.84:0/112060646 learned_addr learned my addr 172.21.15.84:0/112060646 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:47.617 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dc1d9b640 1 --2- 172.21.15.84:0/112060646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d98211820 0x563d982242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dc2d9d640 1 -- 172.21.15.84:0/112060646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98210f00 msgr2=0x563d98227bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dc259c640 1 --2- 172.21.15.84:0/112060646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98210f00 0x563d98227bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dc2d9d640 1 --2- 172.21.15.84:0/112060646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98210f00 0x563d98227bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dc2d9d640 1 -- 172.21.15.84:0/112060646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d98211820 msgr2=0x563d982242d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dc2d9d640 1 --2- 172.21.15.84:0/112060646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d98211820 0x563d982242d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dc2d9d640 1 -- 172.21.15.84:0/112060646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d982210b0 con 0x563d9821c0e0 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dc1d9b640 1 --2- 172.21.15.84:0/112060646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d98211820 0x563d982242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dc259c640 1 --2- 172.21.15.84:0/112060646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98210f00 0x563d98227bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dc2d9d640 1 --2- 172.21.15.84:0/112060646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d9821c0e0 0x563d982128b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00f610 tx=0x7f9dbc00aec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e599590cd6e8e0c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dabfff640 1 -- 172.21.15.84:0/112060646 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc012040 con 0x563d9821c0e0 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dabfff640 1 -- 172.21.15.84:0/112060646 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dbc017400 con 0x563d9821c0e0 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.613+0000 7f9dabfff640 1 -- 172.21.15.84:0/112060646 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9dbc017b50 con 0x563d9821c0e0 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 -- 172.21.15.84:0/112060646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d9821c0e0 msgr2=0x563d982128b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:47.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/112060646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d9821c0e0 0x563d982128b0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc00f610 tx=0x7f9dbc00aec0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 -- 172.21.15.84:0/112060646 shutdown_connections 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/112060646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d9821c0e0 0x563d982128b0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/112060646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d98211820 0x563d982242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/112060646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d98210f00 0x563d98227bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 -- 172.21.15.84:0/112060646 >> 172.21.15.84:0/112060646 conn(0x563d9820d410 msgr2=0x563d9820d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 -- 172.21.15.84:0/112060646 shutdown_connections 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 -- 172.21.15.84:0/112060646 wait complete. 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 -- start start 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 Processor -- start 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d98210f00 0x563d98215cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 0x563d98216200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9821c0e0 0x563d98216740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41ae8b0 con 0x563d98211820 2024-04-26T18:34:47.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41ae1e0 con 0x563d98210f00 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9dc41ab440 con 0x563d9821c0e0 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dcbc44740 10 client.0 _ll_register_callbacks cb 0x7f9db40046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.614+0000 7f9dc1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 0x563d98216200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9dc1d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 0x563d98216200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35340/0 (socket says 172.21.15.84:35340) 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9dc1d9b640 1 -- 172.21.15.84:0/1606719247 learned_addr learned my addr 172.21.15.84:0/1606719247 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9dc259c640 1 --2- 172.21.15.84:0/1606719247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d98210f00 0x563d98215cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9dc1d9b640 1 -- 172.21.15.84:0/1606719247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9821c0e0 msgr2=0x563d98216740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9dc1d9b640 1 --2- 172.21.15.84:0/1606719247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9821c0e0 0x563d98216740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9dc1d9b640 1 -- 172.21.15.84:0/1606719247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d98210f00 msgr2=0x563d98215cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9dc1d9b640 1 --2- 172.21.15.84:0/1606719247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d98210f00 0x563d98215cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9dc1d9b640 1 -- 172.21.15.84:0/1606719247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d98295e60 con 0x563d98211820 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9dc1d9b640 1 --2- 172.21.15.84:0/1606719247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 0x563d98216200 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9db4004770 tx=0x7f9db40cdf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cad2c2b67a930cca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9da9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:34:47.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9da9ffb640 1 -- 172.21.15.84:0/1606719247 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9db40b4040 con 0x563d98211820 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9da9ffb640 1 -- 172.21.15.84:0/1606719247 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f9db409b880 con 0x563d98211820 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9dcbc44740 1 -- 172.21.15.84:0/1606719247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9dbc017400 con 0x563d98211820 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9da9ffb640 1 -- 172.21.15.84:0/1606719247 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9db40a4f60 con 0x563d98211820 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9dcbc44740 10 client.16251 fetch_fsmap learned FSMap version 29 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9dcbc44740 1 -- 172.21.15.84:0/1606719247 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d982970a0 con 0x563d98211820 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9da9ffb640 1 -- 172.21.15.84:0/1606719247 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9dbc017400 con 0x563d98211820 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.615+0000 7f9da9ffb640 1 -- 172.21.15.84:0/1606719247 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9db409b1b0 con 0x563d98211820 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.616+0000 7f9dcbc44740 10 client.16251 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.616+0000 7f9dcbc44740 10 client.16251 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.616+0000 7f9dcbc44740 20 client.16251 populate_metadata read hostname 'smithi084' 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.616+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/1606719247 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563d98297810 0x563d98299c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.616+0000 7f9dcbc44740 4 client.16251 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.616+0000 7f9dcbc44740 1 -- 172.21.15.84:0/1606719247 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563d9817eee0 con 0x563d98297810 2024-04-26T18:34:47.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.616+0000 7f9dc259c640 1 --2- 172.21.15.84:0/1606719247 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563d98297810 0x563d98299c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:34:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.616+0000 7f9dc259c640 1 --2- 172.21.15.84:0/1606719247 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563d98297810 0x563d98299c30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e987d87d05104cd0 server_cookie=af4796b04cd0a181 in_seq=0 out_seq=0 2024-04-26T18:34:47.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.616+0000 7f9da9ffb640 10 client.16251 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9da9ffb640 1 -- 172.21.15.84:0/1606719247 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563d9817eee0 con 0x563d98297810 2024-04-26T18:34:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9da9ffb640 10 client.16251 handle_command_reply: tid=0 2024-04-26T18:34:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9dcbc44740 1 client.16251 shutdown 2024-04-26T18:34:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9dcbc44740 1 -- 172.21.15.84:0/1606719247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 msgr2=0x563d98216200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/1606719247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 0x563d98216200 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f9db4004770 tx=0x7f9db40cdf50 comp rx=0 tx=0).stop 2024-04-26T18:34:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9dcbc44740 1 -- 172.21.15.84:0/1606719247 shutdown_connections 2024-04-26T18:34:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/1606719247 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563d98297810 0x563d98299c30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/1606719247 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563d9821c0e0 0x563d98216740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9da9ffb640 0 client.16251 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:34:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/1606719247 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563d98211820 0x563d98216200 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9dcbc44740 1 --2- 172.21.15.84:0/1606719247 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563d98210f00 0x563d98215cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9dcbc44740 1 -- 172.21.15.84:0/1606719247 >> 172.21.15.84:0/1606719247 conn(0x563d9820d410 msgr2=0x563d98219030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:47.630 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9dcbc44740 1 -- 172.21.15.84:0/1606719247 shutdown_connections 2024-04-26T18:34:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9dcbc44740 1 -- 172.21.15.84:0/1606719247 wait complete. 2024-04-26T18:34:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.629+0000 7f9dcbc44740 20 client.16251 trim_cache size 0 max 16384 2024-04-26T18:34:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:34:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:34:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:34:47.631 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:34:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.630+0000 7f9dc8f24640 1 -- 172.21.15.84:0/751269266 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9db00a3dc0 msgr2=0x7f9db00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.630+0000 7f9dc8f24640 1 --2- 172.21.15.84:0/751269266 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9db00a3dc0 0x7f9db00a6280 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9dbc01d000 tx=0x7f9dbc01b000 comp rx=0 tx=0).stop 2024-04-26T18:34:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.630+0000 7f9dc8f24640 1 -- 172.21.15.84:0/751269266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc4102220 msgr2=0x7f9dc4198560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:34:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.630+0000 7f9dc8f24640 1 --2- 172.21.15.84:0/751269266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc4102220 0x7f9dc4198560 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f9db400d770 tx=0x7f9db400dc30 comp rx=0 tx=0).stop 2024-04-26T18:34:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.630+0000 7f9dc8f24640 1 -- 172.21.15.84:0/751269266 shutdown_connections 2024-04-26T18:34:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.630+0000 7f9dc8f24640 1 --2- 172.21.15.84:0/751269266 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9db00a3dc0 0x7f9db00a6280 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.630+0000 7f9dc8f24640 1 --2- 172.21.15.84:0/751269266 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9dc4107820 0x7f9dc419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.630+0000 7f9dc8f24640 1 --2- 172.21.15.84:0/751269266 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9dc4102220 0x7f9dc4198560 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.632 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.630+0000 7f9dc8f24640 1 --2- 172.21.15.84:0/751269266 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9dc4101860 0x7f9dc4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:34:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.630+0000 7f9dc8f24640 1 -- 172.21.15.84:0/751269266 >> 172.21.15.84:0/751269266 conn(0x7f9dc40789b0 msgr2=0x7f9dc41043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:34:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.630+0000 7f9dc8f24640 1 -- 172.21.15.84:0/751269266 shutdown_connections 2024-04-26T18:34:47.633 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:34:47.630+0000 7f9dc8f24640 1 -- 172.21.15.84:0/751269266 wait complete. 2024-04-26T18:34:47.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:47 smithi174 ceph-mon[32705]: pgmap v209: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-26T18:34:47.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:47 smithi084 ceph-mon[29141]: pgmap v209: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-26T18:34:47.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:47 smithi103 ceph-mon[32727]: pgmap v209: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-26T18:34:49.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:49 smithi174 ceph-mon[32705]: pgmap v210: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-26T18:34:49.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:49 smithi084 ceph-mon[29141]: pgmap v210: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-26T18:34:49.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:49 smithi103 ceph-mon[32727]: pgmap v210: 97 pgs: 97 active+clean; 3.0 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 8.0 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-26T18:34:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:51 smithi174 ceph-mon[32705]: pgmap v211: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-26T18:34:51.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:51 smithi084 ceph-mon[29141]: pgmap v211: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-26T18:34:51.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:51 smithi103 ceph-mon[32727]: pgmap v211: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 17 op/s 2024-04-26T18:34:53.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:53 smithi174 ceph-mon[32705]: pgmap v212: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-26T18:34:53.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:53 smithi084 ceph-mon[29141]: pgmap v212: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-26T18:34:53.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:53 smithi103 ceph-mon[32727]: pgmap v212: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-26T18:34:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:55 smithi084 ceph-mon[29141]: pgmap v213: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-04-26T18:34:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:55 smithi103 ceph-mon[32727]: pgmap v213: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-04-26T18:34:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:55 smithi174 ceph-mon[32705]: pgmap v213: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.2 KiB/s wr, 14 op/s 2024-04-26T18:34:57.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:57 smithi084 ceph-mon[29141]: pgmap v214: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-26T18:34:57.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:57 smithi103 ceph-mon[32727]: pgmap v214: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-26T18:34:57.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:57 smithi174 ceph-mon[32705]: pgmap v214: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-26T18:34:59.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:34:59 smithi084 ceph-mon[29141]: pgmap v215: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-26T18:34:59.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:34:59 smithi103 ceph-mon[32727]: pgmap v215: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-26T18:34:59.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:34:59 smithi174 ceph-mon[32705]: pgmap v215: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 6.2 KiB/s wr, 10 op/s 2024-04-26T18:35:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.726+0000 7f139ac7e640 1 -- 172.21.15.84:0/355965093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1394102930 msgr2=0x7f1394102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.726+0000 7f139ac7e640 1 --2- 172.21.15.84:0/355965093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1394102930 0x7f1394102db0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1380009930 tx=0x7f138002a420 comp rx=0 tx=0).stop 2024-04-26T18:35:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.726+0000 7f139ac7e640 1 -- 172.21.15.84:0/355965093 shutdown_connections 2024-04-26T18:35:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.726+0000 7f139ac7e640 1 --2- 172.21.15.84:0/355965093 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f13941032f0 0x7f139410f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.726+0000 7f139ac7e640 1 --2- 172.21.15.84:0/355965093 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f1394102930 0x7f1394102db0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.726+0000 7f139ac7e640 1 --2- 172.21.15.84:0/355965093 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1394108940 0x7f1394108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.726+0000 7f139ac7e640 1 -- 172.21.15.84:0/355965093 >> 172.21.15.84:0/355965093 conn(0x7f13940fe400 msgr2=0x7f1394100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.726+0000 7f139ac7e640 1 -- 172.21.15.84:0/355965093 shutdown_connections 2024-04-26T18:35:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.726+0000 7f139ac7e640 1 -- 172.21.15.84:0/355965093 wait complete. 2024-04-26T18:35:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f139ac7e640 1 Processor -- start 2024-04-26T18:35:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f139ac7e640 1 -- start start 2024-04-26T18:35:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f139ac7e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1394102930 0x7f13941990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f139ac7e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13941032f0 0x7f1394199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f139ac7e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1394108940 0x7f139419d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f139ac7e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f1394112c50 con 0x7f13941032f0 2024-04-26T18:35:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f139ac7e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f1394112ad0 con 0x7f1394102930 2024-04-26T18:35:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f139ac7e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f1394112dd0 con 0x7f1394108940 2024-04-26T18:35:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f1393fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13941032f0 0x7f1394199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f1393fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13941032f0 0x7f1394199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46356/0 (socket says 172.21.15.84:46356) 2024-04-26T18:35:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f1393fff640 1 -- 172.21.15.84:0/2165572668 learned_addr learned my addr 172.21.15.84:0/2165572668 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f139921b640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1394108940 0x7f139419d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f1398a1a640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1394102930 0x7f13941990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f1393fff640 1 -- 172.21.15.84:0/2165572668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1394108940 msgr2=0x7f139419d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f1393fff640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1394108940 0x7f139419d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f1393fff640 1 -- 172.21.15.84:0/2165572668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1394102930 msgr2=0x7f13941990d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f1393fff640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1394102930 0x7f13941990d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.730 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.727+0000 7f1393fff640 1 -- 172.21.15.84:0/2165572668 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f139419e0c0 con 0x7f13941032f0 2024-04-26T18:35:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.728+0000 7f1393fff640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13941032f0 0x7f1394199610 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1380005ab0 tx=0x7f1380003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.728+0000 7f1391ffb640 1 -- 172.21.15.84:0/2165572668 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138003f070 con 0x7f13941032f0 2024-04-26T18:35:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.728+0000 7f139ac7e640 1 -- 172.21.15.84:0/2165572668 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f139419e350 con 0x7f13941032f0 2024-04-26T18:35:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.728+0000 7f1391ffb640 1 -- 172.21.15.84:0/2165572668 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1380031d60 con 0x7f13941032f0 2024-04-26T18:35:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.728+0000 7f1391ffb640 1 -- 172.21.15.84:0/2165572668 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380035030 con 0x7f13941032f0 2024-04-26T18:35:00.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.728+0000 7f139ac7e640 1 -- 172.21.15.84:0/2165572668 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13941a5c40 con 0x7f13941032f0 2024-04-26T18:35:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.730+0000 7f1391ffb640 1 -- 172.21.15.84:0/2165572668 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f1380042070 con 0x7f13941032f0 2024-04-26T18:35:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f139d996740 1 Processor -- start 2024-04-26T18:35:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f139d996740 1 -- start start 2024-04-26T18:35:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f139d996740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 0x559d53d7c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f139d996740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d8afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f139d996740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d8b4f0 0x559d53d8d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f139d996740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559d53ce2460 con 0x559d53d7c720 2024-04-26T18:35:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f139d996740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559d53c9c590 con 0x559d53d7be00 2024-04-26T18:35:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f139d996740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559d53cde420 con 0x559d53d8b4f0 2024-04-26T18:35:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f1391ffb640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f13840a3dc0 0x7f13840a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f1398a1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d8afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f1398a1a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d8afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46370/0 (socket says 172.21.15.84:46370) 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f1398a1a640 1 -- 172.21.15.84:0/1135193384 learned_addr learned my addr 172.21.15.84:0/1135193384 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f139921b640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f13840a3dc0 0x7f13840a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f1391ffb640 1 -- 172.21.15.84:0/2165572668 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f13800f0fd0 con 0x7f13941032f0 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f1393fff640 1 --2- 172.21.15.84:0/1135193384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d8b4f0 0x559d53d8d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.731+0000 7f139921b640 1 --2- 172.21.15.84:0/1135193384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 0x559d53d7c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.732+0000 7f1398a1a640 1 -- 172.21.15.84:0/1135193384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d8b4f0 msgr2=0x559d53d8d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.732+0000 7f1398a1a640 1 --2- 172.21.15.84:0/1135193384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d8b4f0 0x559d53d8d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.732+0000 7f1398a1a640 1 -- 172.21.15.84:0/1135193384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 msgr2=0x559d53d7c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.732+0000 7f1398a1a640 1 --2- 172.21.15.84:0/1135193384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 0x559d53d7c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.732+0000 7f1398a1a640 1 -- 172.21.15.84:0/1135193384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d53ca90d0 con 0x559d53d7c720 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.732+0000 7f1393fff640 1 --2- 172.21.15.84:0/1135193384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d8b4f0 0x559d53d8d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.732+0000 7f139921b640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f13840a3dc0 0x7f13840a6280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f138c0077f0 tx=0x7f138c007de0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:00.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.732+0000 7f1398a1a640 1 --2- 172.21.15.84:0/1135193384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d8afb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f138800b710 tx=0x7f1388004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375be804884c1ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.732+0000 7f137effd640 1 -- 172.21.15.84:0/1135193384 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388003bd0 con 0x559d53d7c720 2024-04-26T18:35:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.732+0000 7f137effd640 1 -- 172.21.15.84:0/1135193384 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1388015070 con 0x559d53d7c720 2024-04-26T18:35:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.732+0000 7f137effd640 1 -- 172.21.15.84:0/1135193384 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388004d10 con 0x559d53d7c720 2024-04-26T18:35:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.733+0000 7f139d996740 1 -- 172.21.15.84:0/1135193384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 msgr2=0x559d53d8afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.733+0000 7f139d996740 1 --2- 172.21.15.84:0/1135193384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d8afb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f138800b710 tx=0x7f1388004610 comp rx=0 tx=0).stop 2024-04-26T18:35:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.733+0000 7f139d996740 1 -- 172.21.15.84:0/1135193384 shutdown_connections 2024-04-26T18:35:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.733+0000 7f139d996740 1 --2- 172.21.15.84:0/1135193384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d8b4f0 0x559d53d8d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.733+0000 7f139d996740 1 --2- 172.21.15.84:0/1135193384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d8afb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.733+0000 7f139d996740 1 --2- 172.21.15.84:0/1135193384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 0x559d53d7c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.733+0000 7f139d996740 1 -- 172.21.15.84:0/1135193384 >> 172.21.15.84:0/1135193384 conn(0x559d53d78410 msgr2=0x559d53d787f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.733+0000 7f139d996740 1 -- 172.21.15.84:0/1135193384 shutdown_connections 2024-04-26T18:35:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.733+0000 7f139d996740 1 -- 172.21.15.84:0/1135193384 wait complete. 2024-04-26T18:35:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.733+0000 7f139d996740 1 -- start start 2024-04-26T18:35:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.733+0000 7f139d996740 1 Processor -- start 2024-04-26T18:35:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139d996740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 0x559d53d88440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139d996740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d7c720 0x559d53d88980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139d996740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d8b4f0 0x559d53d88ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139d996740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559d53d8e120 con 0x559d53d8b4f0 2024-04-26T18:35:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139d996740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559d53c69b60 con 0x559d53d7be00 2024-04-26T18:35:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139d996740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559d53d8e2a0 con 0x559d53d7c720 2024-04-26T18:35:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139d996740 10 client.0 _ll_register_callbacks cb 0x559d53c8e2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139921b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d8b4f0 0x559d53d88ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139921b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d8b4f0 0x559d53d88ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46378/0 (socket says 172.21.15.84:46378) 2024-04-26T18:35:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139921b640 1 -- 172.21.15.84:0/2346159323 learned_addr learned my addr 172.21.15.84:0/2346159323 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f1398a1a640 1 --2- 172.21.15.84:0/2346159323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 0x559d53d88440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f1393fff640 1 --2- 172.21.15.84:0/2346159323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d7c720 0x559d53d88980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139921b640 1 -- 172.21.15.84:0/2346159323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d7c720 msgr2=0x559d53d88980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139921b640 1 --2- 172.21.15.84:0/2346159323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d7c720 0x559d53d88980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139921b640 1 -- 172.21.15.84:0/2346159323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 msgr2=0x559d53d88440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139921b640 1 --2- 172.21.15.84:0/2346159323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 0x559d53d88440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139921b640 1 -- 172.21.15.84:0/2346159323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d539f4210 con 0x559d53d8b4f0 2024-04-26T18:35:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139921b640 1 --2- 172.21.15.84:0/2346159323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d8b4f0 0x559d53d88ec0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f138c0126b0 tx=0x7f138c007520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e3c5a581fb7c81 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f1373fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:35:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f1373fff640 1 -- 172.21.15.84:0/2346159323 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138c026070 con 0x559d53d8b4f0 2024-04-26T18:35:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f1373fff640 1 -- 172.21.15.84:0/2346159323 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f138c004d20 con 0x559d53d8b4f0 2024-04-26T18:35:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.734+0000 7f139d996740 1 -- 172.21.15.84:0/2346159323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1388015070 con 0x559d53d8b4f0 2024-04-26T18:35:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.735+0000 7f1373fff640 1 -- 172.21.15.84:0/2346159323 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f138c031050 con 0x559d53d8b4f0 2024-04-26T18:35:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.735+0000 7f139d996740 10 client.16263 fetch_fsmap learned FSMap version 29 2024-04-26T18:35:00.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.735+0000 7f139d996740 1 -- 172.21.15.84:0/2346159323 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d53ac0620 con 0x559d53d8b4f0 2024-04-26T18:35:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.735+0000 7f1373fff640 1 -- 172.21.15.84:0/2346159323 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1388015070 con 0x559d53d8b4f0 2024-04-26T18:35:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.735+0000 7f1373fff640 1 -- 172.21.15.84:0/2346159323 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f138c005020 con 0x559d53d8b4f0 2024-04-26T18:35:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.735+0000 7f139d996740 10 client.16263 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:35:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.735+0000 7f139d996740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.735+0000 7f139d996740 20 client.16263 populate_metadata read hostname 'smithi084' 2024-04-26T18:35:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.735+0000 7f139d996740 1 --2- 172.21.15.84:0/2346159323 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559d53d9b6a0 0x559d53d9dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.735+0000 7f139d996740 4 client.16263 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:35:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.735+0000 7f139d996740 1 -- 172.21.15.84:0/2346159323 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d53cf0070 con 0x559d53d9b6a0 2024-04-26T18:35:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.735+0000 7f1398a1a640 1 --2- 172.21.15.84:0/2346159323 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559d53d9b6a0 0x559d53d9dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.736+0000 7f1398a1a640 1 --2- 172.21.15.84:0/2346159323 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559d53d9b6a0 0x559d53d9dab0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19a204fa5161fbdb server_cookie=5c68bb10ce012b4d in_seq=0 out_seq=0 2024-04-26T18:35:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.736+0000 7f1373fff640 10 client.16263 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f1373fff640 1 -- 172.21.15.84:0/2346159323 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559d53cf0070 con 0x559d53d9b6a0 2024-04-26T18:35:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f1373fff640 10 client.16263 handle_command_reply: tid=0 2024-04-26T18:35:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f139d996740 1 client.16263 shutdown 2024-04-26T18:35:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f139d996740 1 -- 172.21.15.84:0/2346159323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d8b4f0 msgr2=0x559d53d88ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f139d996740 1 --2- 172.21.15.84:0/2346159323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d8b4f0 0x559d53d88ec0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f138c0126b0 tx=0x7f138c007520 comp rx=0 tx=0).stop 2024-04-26T18:35:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f139d996740 1 -- 172.21.15.84:0/2346159323 shutdown_connections 2024-04-26T18:35:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f139d996740 1 --2- 172.21.15.84:0/2346159323 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559d53d9b6a0 0x559d53d9dab0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f139d996740 1 --2- 172.21.15.84:0/2346159323 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d8b4f0 0x559d53d88ec0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f139d996740 1 --2- 172.21.15.84:0/2346159323 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d7c720 0x559d53d88980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f1373fff640 0 client.16263 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f139d996740 1 --2- 172.21.15.84:0/2346159323 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 0x559d53d88440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f139d996740 1 -- 172.21.15.84:0/2346159323 >> 172.21.15.84:0/2346159323 conn(0x559d53d78410 msgr2=0x559d53d7e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f139d996740 1 -- 172.21.15.84:0/2346159323 shutdown_connections 2024-04-26T18:35:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f139d996740 1 -- 172.21.15.84:0/2346159323 wait complete. 2024-04-26T18:35:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.737+0000 7f139d996740 20 client.16263 trim_cache size 0 max 16384 2024-04-26T18:35:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f139d996740 1 Processor -- start 2024-04-26T18:35:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f139d996740 1 -- start start 2024-04-26T18:35:00.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f139d996740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d7be00 0x559d53d96fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f139d996740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d77430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f139d996740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d8b4f0 0x559d53da33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f139d996740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13941ac310 con 0x559d53d7c720 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f139d996740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13941ade50 con 0x559d53d8b4f0 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f139d996740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f13941ac9e0 con 0x559d53d7be00 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f1393fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d77430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f1393fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d77430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46380/0 (socket says 172.21.15.84:46380) 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f1393fff640 1 -- 172.21.15.84:0/1331961294 learned_addr learned my addr 172.21.15.84:0/1331961294 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f139921b640 1 --2- 172.21.15.84:0/1331961294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d8b4f0 0x559d53da33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f1393fff640 1 -- 172.21.15.84:0/1331961294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d7be00 msgr2=0x559d53d96fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f1398a1a640 1 --2- 172.21.15.84:0/1331961294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d7be00 0x559d53d96fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f1393fff640 1 --2- 172.21.15.84:0/1331961294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d7be00 0x559d53d96fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f1393fff640 1 -- 172.21.15.84:0/1331961294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d8b4f0 msgr2=0x559d53da33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:00.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f1393fff640 1 --2- 172.21.15.84:0/1331961294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d8b4f0 0x559d53da33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f1393fff640 1 -- 172.21.15.84:0/1331961294 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d53d78240 con 0x559d53d7c720 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.752+0000 7f1398a1a640 1 --2- 172.21.15.84:0/1331961294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d7be00 0x559d53d96fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.753+0000 7f1393fff640 1 --2- 172.21.15.84:0/1331961294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d77430 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f13800b99b0 tx=0x7f13800045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f9a707daa35607e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.753+0000 7f137e7fc640 1 -- 172.21.15.84:0/1331961294 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1380049040 con 0x559d53d7c720 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.753+0000 7f137e7fc640 1 -- 172.21.15.84:0/1331961294 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13800c5ba0 con 0x559d53d7c720 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.753+0000 7f137e7fc640 1 -- 172.21.15.84:0/1331961294 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13800c5710 con 0x559d53d7c720 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.753+0000 7f139d996740 1 -- 172.21.15.84:0/1331961294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 msgr2=0x559d53d77430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.753+0000 7f139d996740 1 --2- 172.21.15.84:0/1331961294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d77430 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f13800b99b0 tx=0x7f13800045e0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.753+0000 7f139d996740 1 -- 172.21.15.84:0/1331961294 shutdown_connections 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.753+0000 7f139d996740 1 --2- 172.21.15.84:0/1331961294 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d8b4f0 0x559d53da33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.753+0000 7f139d996740 1 --2- 172.21.15.84:0/1331961294 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d77430 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.753+0000 7f139d996740 1 --2- 172.21.15.84:0/1331961294 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d7be00 0x559d53d96fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.753+0000 7f139d996740 1 -- 172.21.15.84:0/1331961294 >> 172.21.15.84:0/1331961294 conn(0x559d53d78410 msgr2=0x559d53d76d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:00.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.753+0000 7f139d996740 1 -- 172.21.15.84:0/1331961294 shutdown_connections 2024-04-26T18:35:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.753+0000 7f139d996740 1 -- 172.21.15.84:0/1331961294 wait complete. 2024-04-26T18:35:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f139d996740 1 -- start start 2024-04-26T18:35:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f139d996740 1 Processor -- start 2024-04-26T18:35:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f139d996740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 0x559d53d85840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f139d996740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d85d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f139d996740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d8b4f0 0x559d53e00c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f139d996740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f13941ad780 con 0x559d53d7c720 2024-04-26T18:35:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f139d996740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f13941ad0b0 con 0x559d53d7be00 2024-04-26T18:35:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f139d996740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f13941aa330 con 0x559d53d8b4f0 2024-04-26T18:35:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f139d996740 10 client.0 _ll_register_callbacks cb 0x7f138c014280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f1393fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d85d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f1393fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d85d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46390/0 (socket says 172.21.15.84:46390) 2024-04-26T18:35:00.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f1393fff640 1 -- 172.21.15.84:0/2513740087 learned_addr learned my addr 172.21.15.84:0/2513740087 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f1398a1a640 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 0x559d53d85840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.754+0000 7f139921b640 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d8b4f0 0x559d53e00c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f1393fff640 1 -- 172.21.15.84:0/2513740087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d8b4f0 msgr2=0x559d53e00c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f1393fff640 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d8b4f0 0x559d53e00c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f1393fff640 1 -- 172.21.15.84:0/2513740087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 msgr2=0x559d53d85840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f1393fff640 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 0x559d53d85840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f1393fff640 1 -- 172.21.15.84:0/2513740087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d53d79640 con 0x559d53d7c720 2024-04-26T18:35:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f139921b640 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d8b4f0 0x559d53e00c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:35:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f1393fff640 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d85d80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f13800eece0 tx=0x7f1380009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b554e7f8dd65a3b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f1373fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:35:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f1373fff640 1 -- 172.21.15.84:0/2513740087 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138009f030 con 0x559d53d7c720 2024-04-26T18:35:00.762 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f1373fff640 1 -- 172.21.15.84:0/2513740087 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7f138009be00 con 0x559d53d7c720 2024-04-26T18:35:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f139d996740 1 -- 172.21.15.84:0/2513740087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13800c5ba0 con 0x559d53d7c720 2024-04-26T18:35:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f1373fff640 1 -- 172.21.15.84:0/2513740087 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1380049070 con 0x559d53d7c720 2024-04-26T18:35:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f139d996740 10 client.16269 fetch_fsmap learned FSMap version 29 2024-04-26T18:35:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f139d996740 1 -- 172.21.15.84:0/2513740087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d53d7cc60 con 0x559d53d7c720 2024-04-26T18:35:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f1373fff640 1 -- 172.21.15.84:0/2513740087 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13800c5ba0 con 0x559d53d7c720 2024-04-26T18:35:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f1373fff640 1 -- 172.21.15.84:0/2513740087 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f13800c5710 con 0x559d53d7c720 2024-04-26T18:35:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f139d996740 10 client.16269 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:35:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f139d996740 10 client.16269 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f139d996740 20 client.16269 populate_metadata read hostname 'smithi084' 2024-04-26T18:35:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f139d996740 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559d53e02710 0x559d53e04b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f139d996740 4 client.16269 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:35:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.755+0000 7f139d996740 1 -- 172.21.15.84:0/2513740087 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559d53ce9ee0 con 0x559d53e02710 2024-04-26T18:35:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.756+0000 7f1398a1a640 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559d53e02710 0x559d53e04b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.756+0000 7f1373fff640 10 client.16269 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.756+0000 7f1398a1a640 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559d53e02710 0x559d53e04b30 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7390e4900cd98069 server_cookie=52df7debd20b85d3 in_seq=0 out_seq=0 2024-04-26T18:35:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f1373fff640 1 -- 172.21.15.84:0/2513740087 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559d53ce9ee0 con 0x559d53e02710 2024-04-26T18:35:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f1373fff640 10 client.16269 handle_command_reply: tid=0 2024-04-26T18:35:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f139d996740 1 client.16269 shutdown 2024-04-26T18:35:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f139d996740 1 -- 172.21.15.84:0/2513740087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 msgr2=0x559d53d85d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f139d996740 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d85d80 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f13800eece0 tx=0x7f1380009260 comp rx=0 tx=0).stop 2024-04-26T18:35:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f139d996740 1 -- 172.21.15.84:0/2513740087 shutdown_connections 2024-04-26T18:35:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f139d996740 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559d53e02710 0x559d53e04b30 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f139d996740 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559d53d8b4f0 0x559d53e00c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f139d996740 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559d53d7c720 0x559d53d85d80 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f139d996740 1 --2- 172.21.15.84:0/2513740087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559d53d7be00 0x559d53d85840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f1373fff640 0 client.16269 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:00.767 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f139d996740 1 -- 172.21.15.84:0/2513740087 >> 172.21.15.84:0/2513740087 conn(0x559d53d78410 msgr2=0x559d53d88c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f139d996740 1 -- 172.21.15.84:0/2513740087 shutdown_connections 2024-04-26T18:35:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.766+0000 7f139d996740 1 -- 172.21.15.84:0/2513740087 wait complete. 2024-04-26T18:35:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.767+0000 7f139d996740 20 client.16269 trim_cache size 0 max 16384 2024-04-26T18:35:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:35:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:35:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:35:00.768 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:35:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.767+0000 7f139ac7e640 1 -- 172.21.15.84:0/2165572668 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f13840a3dc0 msgr2=0x7f13840a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.767+0000 7f139ac7e640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f13840a3dc0 0x7f13840a6280 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f138c0077f0 tx=0x7f138c007de0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.767+0000 7f139ac7e640 1 -- 172.21.15.84:0/2165572668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13941032f0 msgr2=0x7f1394199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.767+0000 7f139ac7e640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13941032f0 0x7f1394199610 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1380005ab0 tx=0x7f1380003cd0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.767+0000 7f139ac7e640 1 -- 172.21.15.84:0/2165572668 shutdown_connections 2024-04-26T18:35:00.769 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.767+0000 7f139ac7e640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f13840a3dc0 0x7f13840a6280 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.767+0000 7f139ac7e640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f1394108940 0x7f139419d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.767+0000 7f139ac7e640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f13941032f0 0x7f1394199610 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.767+0000 7f139ac7e640 1 --2- 172.21.15.84:0/2165572668 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f1394102930 0x7f13941990d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.767+0000 7f139ac7e640 1 -- 172.21.15.84:0/2165572668 >> 172.21.15.84:0/2165572668 conn(0x7f13940fe400 msgr2=0x7f13940fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.767+0000 7f139ac7e640 1 -- 172.21.15.84:0/2165572668 shutdown_connections 2024-04-26T18:35:00.770 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:00.767+0000 7f139ac7e640 1 -- 172.21.15.84:0/2165572668 wait complete. 2024-04-26T18:35:01.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:01 smithi084 ceph-mon[29141]: pgmap v216: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-04-26T18:35:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:01 smithi103 ceph-mon[32727]: pgmap v216: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-04-26T18:35:01.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:01 smithi174 ceph-mon[32705]: pgmap v216: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.8 KiB/s wr, 15 op/s 2024-04-26T18:35:03.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:03 smithi084 ceph-mon[29141]: pgmap v217: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T18:35:03.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:03 smithi103 ceph-mon[32727]: pgmap v217: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T18:35:03.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:03 smithi174 ceph-mon[32705]: pgmap v217: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 8 op/s 2024-04-26T18:35:05.453 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:35:05.707 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:35:05.828 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:05 smithi084 ceph-mon[29141]: pgmap v218: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 48 KiB/s wr, 31 op/s 2024-04-26T18:35:05.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:05 smithi103 ceph-mon[32727]: pgmap v218: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 48 KiB/s wr, 31 op/s 2024-04-26T18:35:05.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:05 smithi174 ceph-mon[32705]: pgmap v218: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 48 KiB/s wr, 31 op/s 2024-04-26T18:35:06.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:06 smithi084 ceph-mon[29141]: pgmap v219: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-04-26T18:35:06.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:06 smithi103 ceph-mon[32727]: pgmap v219: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-04-26T18:35:06.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:06 smithi174 ceph-mon[32705]: pgmap v219: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-04-26T18:35:07.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.431+0000 7fedb7c37700 1 -- 172.21.15.84:0/2488629521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb0075650 msgr2=0x7fedb0075a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:07.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.431+0000 7fedb7c37700 1 --2- 172.21.15.84:0/2488629521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb0075650 0x7fedb0075a30 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fedac009990 tx=0x7fedac009ca0 comp rx=0 tx=0).stop 2024-04-26T18:35:07.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.432+0000 7fedb7c37700 1 -- 172.21.15.84:0/2488629521 shutdown_connections 2024-04-26T18:35:07.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.432+0000 7fedb7c37700 1 --2- 172.21.15.84:0/2488629521 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb0075650 0x7fedb0075a30 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:07.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.432+0000 7fedb7c37700 1 --2- 172.21.15.84:0/2488629521 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedb010fe20 0x7fedb0112210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:07.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.432+0000 7fedb7c37700 1 --2- 172.21.15.84:0/2488629521 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedb0075f70 0x7fedb010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:07.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.432+0000 7fedb7c37700 1 -- 172.21.15.84:0/2488629521 >> 172.21.15.84:0/2488629521 conn(0x7fedb007a580 msgr2=0x7fedb007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:07.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.432+0000 7fedb7c37700 1 -- 172.21.15.84:0/2488629521 shutdown_connections 2024-04-26T18:35:07.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.432+0000 7fedb7c37700 1 -- 172.21.15.84:0/2488629521 wait complete. 2024-04-26T18:35:07.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.432+0000 7fedb7c37700 1 Processor -- start 2024-04-26T18:35:07.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.432+0000 7fedb7c37700 1 -- start start 2024-04-26T18:35:07.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.432+0000 7fedb7c37700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedb0075650 0x7fedb01a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:07.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.432+0000 7fedb7c37700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedb0075f70 0x7fedb01a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:07.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.432+0000 7fedb7c37700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb010fe20 0x7fedb019bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:07.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb7c37700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fedb0115800 con 0x7fedb010fe20 2024-04-26T18:35:07.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb7c37700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fedb0115680 con 0x7fedb0075650 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb7c37700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fedb0115980 con 0x7fedb0075f70 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb61d4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb010fe20 0x7fedb019bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb61d4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb010fe20 0x7fedb019bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33960/0 (socket says 172.21.15.84:33960) 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb59d3700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedb0075650 0x7fedb01a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb61d4700 1 -- 172.21.15.84:0/3758180036 learned_addr learned my addr 172.21.15.84:0/3758180036 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb51d2700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedb0075f70 0x7fedb01a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb61d4700 1 -- 172.21.15.84:0/3758180036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedb0075f70 msgr2=0x7fedb01a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb61d4700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedb0075f70 0x7fedb01a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb61d4700 1 -- 172.21.15.84:0/3758180036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedb0075650 msgr2=0x7fedb01a18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb61d4700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedb0075650 0x7fedb01a18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb61d4700 1 -- 172.21.15.84:0/3758180036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb019c260 con 0x7fedb010fe20 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb59d3700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedb0075650 0x7fedb01a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb61d4700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb010fe20 0x7fedb019bb00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7feda8007ee0 tx=0x7feda800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:07.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7feda6ffd700 1 -- 172.21.15.84:0/3758180036 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8004bb0 con 0x7fedb010fe20 2024-04-26T18:35:07.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7feda6ffd700 1 -- 172.21.15.84:0/3758180036 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feda8004d50 con 0x7fedb010fe20 2024-04-26T18:35:07.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7feda6ffd700 1 -- 172.21.15.84:0/3758180036 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feda8005920 con 0x7fedb010fe20 2024-04-26T18:35:07.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb7c37700 1 -- 172.21.15.84:0/3758180036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb019c550 con 0x7fedb010fe20 2024-04-26T18:35:07.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.433+0000 7fedb7c37700 1 -- 172.21.15.84:0/3758180036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb01a87c0 con 0x7fedb010fe20 2024-04-26T18:35:07.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.434+0000 7fedb7c37700 1 -- 172.21.15.84:0/3758180036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb006a110 con 0x7fedb010fe20 2024-04-26T18:35:07.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.436+0000 7feda6ffd700 1 -- 172.21.15.84:0/3758180036 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7feda801a020 con 0x7fedb010fe20 2024-04-26T18:35:07.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.436+0000 7feda6ffd700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fed9c0a3db0 0x7fed9c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:07.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.436+0000 7fedb59d3700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fed9c0a3db0 0x7fed9c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:07.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.436+0000 7feda6ffd700 1 -- 172.21.15.84:0/3758180036 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8579+0+0 (secure 0 0 0) 0x7feda80d0510 con 0x7fedb010fe20 2024-04-26T18:35:07.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.437+0000 7fedb59d3700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fed9c0a3db0 0x7fed9c0a6270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fedb007ad90 tx=0x7fedac005960 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:07.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.440+0000 7feda6ffd700 1 -- 172.21.15.84:0/3758180036 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feda8010070 con 0x7fedb010fe20 2024-04-26T18:35:07.703 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.703+0000 7fedb7c37700 1 -- 172.21.15.84:0/3758180036 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fedb019cfe0 con 0x7fedb010fe20 2024-04-26T18:35:07.703 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:07 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:35:07.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.703+0000 7feda6ffd700 1 -- 172.21.15.84:0/3758180036 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7feda809cb70 con 0x7fedb010fe20 2024-04-26T18:35:07.704 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:35:07.704 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:35:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.706+0000 7fedb7c37700 1 -- 172.21.15.84:0/3758180036 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fed9c0a3db0 msgr2=0x7fed9c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.706+0000 7fedb7c37700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fed9c0a3db0 0x7fed9c0a6270 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fedb007ad90 tx=0x7fedac005960 comp rx=0 tx=0).stop 2024-04-26T18:35:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.706+0000 7fedb7c37700 1 -- 172.21.15.84:0/3758180036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb010fe20 msgr2=0x7fedb019bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.706+0000 7fedb7c37700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb010fe20 0x7fedb019bb00 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7feda8007ee0 tx=0x7feda800d3b0 comp rx=0 tx=0).stop 2024-04-26T18:35:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.706+0000 7fedb7c37700 1 -- 172.21.15.84:0/3758180036 shutdown_connections 2024-04-26T18:35:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.706+0000 7fedb7c37700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fed9c0a3db0 0x7fed9c0a6270 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:07.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.706+0000 7fedb7c37700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fedb010fe20 0x7fedb019bb00 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.706+0000 7fedb7c37700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fedb0075650 0x7fedb01a18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.706+0000 7fedb7c37700 1 --2- 172.21.15.84:0/3758180036 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fedb0075f70 0x7fedb01a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.706+0000 7fedb7c37700 1 -- 172.21.15.84:0/3758180036 >> 172.21.15.84:0/3758180036 conn(0x7fedb007a580 msgr2=0x7fedb01006d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.706+0000 7fedb7c37700 1 -- 172.21.15.84:0/3758180036 shutdown_connections 2024-04-26T18:35:07.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:07.706+0000 7fedb7c37700 1 -- 172.21.15.84:0/3758180036 wait complete. 2024-04-26T18:35:07.709 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:35:07.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:07 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:35:07.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:07 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:35:08.403 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id db61a169 --release --await' 2024-04-26T18:35:08.404 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db --set-id db61a169 --release --await 2024-04-26T18:35:08.647 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:35:08.769 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:08 smithi084 ceph-mon[29141]: pgmap v220: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-04-26T18:35:08.769 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:08 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3758180036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:35:08.769 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:08 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:35:08.769 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:08 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:35:08.769 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:08 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:35:08.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:08 smithi103 ceph-mon[32727]: pgmap v220: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-04-26T18:35:08.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:08 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3758180036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:35:08.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:08 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:35:08.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:08 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:35:08.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:08 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:35:08.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:08 smithi174 ceph-mon[32705]: pgmap v220: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 48 KiB/s wr, 27 op/s 2024-04-26T18:35:08.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:08 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3758180036' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:35:08.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:08 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:35:08.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:08 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:35:08.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:08 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:35:10.331 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.329+0000 7f3be5613700 1 -- 172.21.15.84:0/3003890148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be0102330 msgr2=0x7f3be01102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.329+0000 7f3be5613700 1 --2- 172.21.15.84:0/3003890148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be0102330 0x7f3be01102d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800b0d0 tx=0x7f3bd800b3e0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.329+0000 7f3be5613700 1 -- 172.21.15.84:0/3003890148 shutdown_connections 2024-04-26T18:35:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.329+0000 7f3be5613700 1 --2- 172.21.15.84:0/3003890148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be0102330 0x7f3be01102d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.329+0000 7f3be5613700 1 --2- 172.21.15.84:0/3003890148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3be01094d0 0x7f3be01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.329+0000 7f3be5613700 1 --2- 172.21.15.84:0/3003890148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3be0101970 0x7f3be0101df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.329+0000 7f3be5613700 1 -- 172.21.15.84:0/3003890148 >> 172.21.15.84:0/3003890148 conn(0x7f3be0078920 msgr2=0x7f3be00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.329+0000 7f3be5613700 1 -- 172.21.15.84:0/3003890148 shutdown_connections 2024-04-26T18:35:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.329+0000 7f3be5613700 1 -- 172.21.15.84:0/3003890148 wait complete. 2024-04-26T18:35:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.330+0000 7f3be5613700 1 Processor -- start 2024-04-26T18:35:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.330+0000 7f3be5613700 1 -- start start 2024-04-26T18:35:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.330+0000 7f3be5613700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3be0101970 0x7f3be019f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.332 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.330+0000 7f3be5613700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3be0102330 0x7f3be019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.330+0000 7f3be5613700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be01094d0 0x7f3be01998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.330+0000 7f3be5613700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0113610 con 0x7f3be01094d0 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.330+0000 7f3be5613700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0113490 con 0x7f3be0101970 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.330+0000 7f3be5613700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3be0113790 con 0x7f3be0102330 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.330+0000 7f3bdf7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be01094d0 0x7f3be01998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.330+0000 7f3bdf7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be01094d0 0x7f3be01998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33978/0 (socket says 172.21.15.84:33978) 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.330+0000 7f3bdf7fe700 1 -- 172.21.15.84:0/3975443380 learned_addr learned my addr 172.21.15.84:0/3975443380 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.330+0000 7f3bdeffd700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3be0101970 0x7f3be019f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.330+0000 7f3bd67fc700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3be0102330 0x7f3be019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.331+0000 7f3bdf7fe700 1 -- 172.21.15.84:0/3975443380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3be0102330 msgr2=0x7f3be019fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.331+0000 7f3bdf7fe700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3be0102330 0x7f3be019fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.331+0000 7f3bdf7fe700 1 -- 172.21.15.84:0/3975443380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3be0101970 msgr2=0x7f3be019f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.331+0000 7f3bdf7fe700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3be0101970 0x7f3be019f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.333 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.331+0000 7f3bdf7fe700 1 -- 172.21.15.84:0/3975443380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3be019a030 con 0x7f3be01094d0 2024-04-26T18:35:10.334 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.331+0000 7f3bd67fc700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3be0102330 0x7f3be019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:35:10.334 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.331+0000 7f3bdf7fe700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be01094d0 0x7f3be01998d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8007a20 tx=0x7f3bd8007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:10.334 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.331+0000 7f3bdcff9700 1 -- 172.21.15.84:0/3975443380 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd801f070 con 0x7f3be01094d0 2024-04-26T18:35:10.334 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.331+0000 7f3bdcff9700 1 -- 172.21.15.84:0/3975443380 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bd8003c00 con 0x7f3be01094d0 2024-04-26T18:35:10.334 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.331+0000 7f3be5613700 1 -- 172.21.15.84:0/3975443380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3be019a2c0 con 0x7f3be01094d0 2024-04-26T18:35:10.334 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.331+0000 7f3bdcff9700 1 -- 172.21.15.84:0/3975443380 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd801b4a0 con 0x7f3be01094d0 2024-04-26T18:35:10.334 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.331+0000 7f3be5613700 1 -- 172.21.15.84:0/3975443380 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3be01a65e0 con 0x7f3be01094d0 2024-04-26T18:35:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.333+0000 7f3bdcff9700 1 -- 172.21.15.84:0/3975443380 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3bd8022070 con 0x7f3be01094d0 2024-04-26T18:35:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.334+0000 7f3bdcff9700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3bcc0a3db0 0x7f3bcc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.334+0000 7f3bdeffd700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3bcc0a3db0 0x7f3bcc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.334+0000 7f3bdcff9700 1 -- 172.21.15.84:0/3975443380 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8623+0+0 (secure 0 0 0) 0x7f3bd800f020 con 0x7f3be01094d0 2024-04-26T18:35:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.334+0000 7f3bdeffd700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3bcc0a3db0 0x7f3bcc0a6270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0005950 tx=0x7f3bd000a300 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3befea5b80 1 Processor -- start 2024-04-26T18:35:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3befea5b80 1 -- start start 2024-04-26T18:35:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3befea5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0cea0 0x560e5dc0d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3befea5b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 0x560e5dc07250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3befea5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc07cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3befea5b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560e5db59930 con 0x560e5dc0d7c0 2024-04-26T18:35:10.341 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3befea5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560e5dabfa30 con 0x560e5dc07870 2024-04-26T18:35:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3befea5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e5db558f0 con 0x560e5dc0cea0 2024-04-26T18:35:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3bd67fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 0x560e5dc07250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3bd67fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 0x560e5dc07250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33988/0 (socket says 172.21.15.84:33988) 2024-04-26T18:35:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3bd67fc700 1 -- 172.21.15.84:0/1207656686 learned_addr learned my addr 172.21.15.84:0/1207656686 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3bdf7fe700 1 --2- 172.21.15.84:0/1207656686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc07cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3bdeffd700 1 --2- 172.21.15.84:0/1207656686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0cea0 0x560e5dc0d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3bd67fc700 1 -- 172.21.15.84:0/1207656686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0cea0 msgr2=0x560e5dc0d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3bd67fc700 1 --2- 172.21.15.84:0/1207656686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0cea0 0x560e5dc0d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3bd67fc700 1 -- 172.21.15.84:0/1207656686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 msgr2=0x560e5dc07cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3bd67fc700 1 --2- 172.21.15.84:0/1207656686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc07cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3bd67fc700 1 -- 172.21.15.84:0/1207656686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e5daa69d0 con 0x560e5dc0d7c0 2024-04-26T18:35:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.335+0000 7f3bdeffd700 1 --2- 172.21.15.84:0/1207656686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0cea0 0x560e5dc0d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:35:10.342 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.336+0000 7f3bd67fc700 1 --2- 172.21.15.84:0/1207656686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 0x560e5dc07250 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800bda0 tx=0x7f3bc800f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d389a3709631ca server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.336+0000 7f3bd57fa700 1 -- 172.21.15.84:0/1207656686 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8015070 con 0x560e5dc0d7c0 2024-04-26T18:35:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.336+0000 7f3bd57fa700 1 -- 172.21.15.84:0/1207656686 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bc800fe50 con 0x560e5dc0d7c0 2024-04-26T18:35:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.336+0000 7f3bd57fa700 1 -- 172.21.15.84:0/1207656686 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8011750 con 0x560e5dc0d7c0 2024-04-26T18:35:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 -- 172.21.15.84:0/1207656686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 msgr2=0x560e5dc07250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 --2- 172.21.15.84:0/1207656686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 0x560e5dc07250 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800bda0 tx=0x7f3bc800f640 comp rx=0 tx=0).stop 2024-04-26T18:35:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 -- 172.21.15.84:0/1207656686 shutdown_connections 2024-04-26T18:35:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 --2- 172.21.15.84:0/1207656686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc07cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 --2- 172.21.15.84:0/1207656686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0cea0 0x560e5dc0d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 --2- 172.21.15.84:0/1207656686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 0x560e5dc07250 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 -- 172.21.15.84:0/1207656686 >> 172.21.15.84:0/1207656686 conn(0x560e5db356c0 msgr2=0x560e5dc02000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 -- 172.21.15.84:0/1207656686 shutdown_connections 2024-04-26T18:35:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 -- 172.21.15.84:0/1207656686 wait complete. 2024-04-26T18:35:10.343 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 -- start start 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 Processor -- start 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc13440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0cea0 0x560e5dc107e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 0x560e5dc10d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560e5dc14230 con 0x560e5dc0d7c0 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560e5db13b80 con 0x560e5dc07870 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.337+0000 7f3befea5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560e5dc143b0 con 0x560e5dc0cea0 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bdf7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 0x560e5dc10d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3befea5b80 10 client.0 _ll_register_callbacks cb 0x560e5daf1a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bdf7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 0x560e5dc10d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33992/0 (socket says 172.21.15.84:33992) 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bdf7fe700 1 -- 172.21.15.84:0/2226465559 learned_addr learned my addr 172.21.15.84:0/2226465559 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bdeffd700 1 --2- 172.21.15.84:0/2226465559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc13440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bd67fc700 1 --2- 172.21.15.84:0/2226465559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0cea0 0x560e5dc107e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.344 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bdf7fe700 1 -- 172.21.15.84:0/2226465559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0cea0 msgr2=0x560e5dc107e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bdf7fe700 1 --2- 172.21.15.84:0/2226465559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0cea0 0x560e5dc107e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bdf7fe700 1 -- 172.21.15.84:0/2226465559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 msgr2=0x560e5dc13440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bdf7fe700 1 --2- 172.21.15.84:0/2226465559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc13440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bdf7fe700 1 -- 172.21.15.84:0/2226465559 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e5db17580 con 0x560e5dc0d7c0 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bdf7fe700 1 --2- 172.21.15.84:0/2226465559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 0x560e5dc10d20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3bd809d910 tx=0x7f3bd809d5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d8b084bcf7d101 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bc67fc700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bc67fc700 1 -- 172.21.15.84:0/2226465559 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bd809e080 con 0x560e5dc0d7c0 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bc67fc700 1 -- 172.21.15.84:0/2226465559 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8623+0+0 (secure 0 0 0) 0x7f3bd809dbe0 con 0x560e5dc0d7c0 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3befea5b80 1 -- 172.21.15.84:0/2226465559 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bc800fe50 con 0x560e5dc0d7c0 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3bc67fc700 1 -- 172.21.15.84:0/2226465559 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3bd809d980 con 0x560e5dc0d7c0 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3befea5b80 10 client.16287 fetch_fsmap learned FSMap version 29 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.338+0000 7f3befea5b80 1 -- 172.21.15.84:0/2226465559 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e5dc1e970 con 0x560e5dc0d7c0 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.339+0000 7f3bc67fc700 1 -- 172.21.15.84:0/2226465559 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bc800fe50 con 0x560e5dc0d7c0 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.339+0000 7f3bc67fc700 1 -- 172.21.15.84:0/2226465559 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3bd8011070 con 0x560e5dc0d7c0 2024-04-26T18:35:10.345 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.339+0000 7f3befea5b80 10 client.16287 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:35:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.339+0000 7f3befea5b80 10 client.16287 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:35:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.339+0000 7f3befea5b80 20 client.16287 populate_metadata read hostname 'smithi084' 2024-04-26T18:35:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.339+0000 7f3befea5b80 1 --2- 172.21.15.84:0/2226465559 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e5dc20ee0 0x560e5dc23310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.339+0000 7f3befea5b80 4 client.16287 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:35:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.339+0000 7f3befea5b80 1 -- 172.21.15.84:0/2226465559 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e5db67310 con 0x560e5dc20ee0 2024-04-26T18:35:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.339+0000 7f3bdeffd700 1 --2- 172.21.15.84:0/2226465559 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e5dc20ee0 0x560e5dc23310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.339+0000 7f3bc67fc700 10 client.16287 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.339+0000 7f3bdeffd700 1 --2- 172.21.15.84:0/2226465559 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e5dc20ee0 0x560e5dc23310 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=590e35e403b0a1da server_cookie=6bce309cac7bd990 in_seq=0 out_seq=0 2024-04-26T18:35:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.340+0000 7f3bc67fc700 1 -- 172.21.15.84:0/2226465559 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e5db67310 con 0x560e5dc20ee0 2024-04-26T18:35:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.340+0000 7f3bc67fc700 10 client.16287 handle_command_reply: tid=0 2024-04-26T18:35:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.340+0000 7f3befea5b80 1 client.16287 shutdown 2024-04-26T18:35:10.346 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.340+0000 7f3befea5b80 1 -- 172.21.15.84:0/2226465559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 msgr2=0x560e5dc10d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.340+0000 7f3befea5b80 1 --2- 172.21.15.84:0/2226465559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 0x560e5dc10d20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f3bd809d910 tx=0x7f3bd809d5b0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.341+0000 7f3befea5b80 1 -- 172.21.15.84:0/2226465559 shutdown_connections 2024-04-26T18:35:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.341+0000 7f3befea5b80 1 --2- 172.21.15.84:0/2226465559 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e5dc20ee0 0x560e5dc23310 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.341+0000 7f3befea5b80 1 --2- 172.21.15.84:0/2226465559 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0d7c0 0x560e5dc10d20 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.341+0000 7f3befea5b80 1 --2- 172.21.15.84:0/2226465559 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc13440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.341+0000 7f3befea5b80 1 --2- 172.21.15.84:0/2226465559 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0cea0 0x560e5dc107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.341+0000 7f3befea5b80 1 -- 172.21.15.84:0/2226465559 >> 172.21.15.84:0/2226465559 conn(0x560e5db356c0 msgr2=0x560e5dc06240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.341+0000 7f3bc67fc700 0 client.16287 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.341+0000 7f3befea5b80 1 -- 172.21.15.84:0/2226465559 shutdown_connections 2024-04-26T18:35:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.341+0000 7f3befea5b80 1 -- 172.21.15.84:0/2226465559 wait complete. 2024-04-26T18:35:10.347 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.341+0000 7f3befea5b80 20 client.16287 trim_cache size 0 max 16384 2024-04-26T18:35:10.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3befea5b80 1 Processor -- start 2024-04-26T18:35:10.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3befea5b80 1 -- start start 2024-04-26T18:35:10.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3befea5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc07250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3befea5b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 0x560e5dc1d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3befea5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0d7c0 0x560e5dc0e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3befea5b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01adcf0 con 0x560e5dc0cea0 2024-04-26T18:35:10.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3befea5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01af830 con 0x560e5dc07870 2024-04-26T18:35:10.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3befea5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01ae3c0 con 0x560e5dc0d7c0 2024-04-26T18:35:10.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3bd67fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 0x560e5dc1d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3bd67fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 0x560e5dc1d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33996/0 (socket says 172.21.15.84:33996) 2024-04-26T18:35:10.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3bd67fc700 1 -- 172.21.15.84:0/1712669312 learned_addr learned my addr 172.21.15.84:0/1712669312 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:10.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3bd67fc700 1 -- 172.21.15.84:0/1712669312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0d7c0 msgr2=0x560e5dc0e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:10.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3bdeffd700 1 --2- 172.21.15.84:0/1712669312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc07250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3bd67fc700 1 --2- 172.21.15.84:0/1712669312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0d7c0 0x560e5dc0e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3bd67fc700 1 -- 172.21.15.84:0/1712669312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 msgr2=0x560e5dc07250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:10.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3bd67fc700 1 --2- 172.21.15.84:0/1712669312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc07250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.360+0000 7f3bd67fc700 1 -- 172.21.15.84:0/1712669312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e5dbfe7f0 con 0x560e5dc0cea0 2024-04-26T18:35:10.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.361+0000 7f3bd67fc700 1 --2- 172.21.15.84:0/1712669312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 0x560e5dc1d160 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3bc801af60 tx=0x7f3bc800fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8da18a7c9a921a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:10.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.361+0000 7f3bd4ff9700 1 -- 172.21.15.84:0/1712669312 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8029070 con 0x560e5dc0cea0 2024-04-26T18:35:10.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.361+0000 7f3bd4ff9700 1 -- 172.21.15.84:0/1712669312 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bc8004710 con 0x560e5dc0cea0 2024-04-26T18:35:10.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.361+0000 7f3bd4ff9700 1 -- 172.21.15.84:0/1712669312 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8015040 con 0x560e5dc0cea0 2024-04-26T18:35:10.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.361+0000 7f3befea5b80 1 -- 172.21.15.84:0/1712669312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 msgr2=0x560e5dc1d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:10.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.361+0000 7f3befea5b80 1 --2- 172.21.15.84:0/1712669312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 0x560e5dc1d160 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3bc801af60 tx=0x7f3bc800fe00 comp rx=0 tx=0).stop 2024-04-26T18:35:10.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.361+0000 7f3befea5b80 1 -- 172.21.15.84:0/1712669312 shutdown_connections 2024-04-26T18:35:10.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.361+0000 7f3befea5b80 1 --2- 172.21.15.84:0/1712669312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0d7c0 0x560e5dc0e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.365 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.361+0000 7f3befea5b80 1 --2- 172.21.15.84:0/1712669312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc07250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.361+0000 7f3befea5b80 1 --2- 172.21.15.84:0/1712669312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 0x560e5dc1d160 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.361+0000 7f3befea5b80 1 -- 172.21.15.84:0/1712669312 >> 172.21.15.84:0/1712669312 conn(0x560e5db356c0 msgr2=0x560e5dc0dba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3befea5b80 1 -- 172.21.15.84:0/1712669312 shutdown_connections 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3befea5b80 1 -- 172.21.15.84:0/1712669312 wait complete. 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3befea5b80 1 -- start start 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3befea5b80 1 Processor -- start 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3befea5b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc16640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3befea5b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 0x560e5dc16b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3befea5b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0d7c0 0x560e5dc170c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3befea5b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01af160 con 0x560e5dc0cea0 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3befea5b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01aea90 con 0x560e5dc07870 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3befea5b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3be01abcb0 con 0x560e5dc0d7c0 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3befea5b80 10 client.0 _ll_register_callbacks cb 0x560e5daf1a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:10.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3bd67fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 0x560e5dc16b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3bd67fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 0x560e5dc16b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34010/0 (socket says 172.21.15.84:34010) 2024-04-26T18:35:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3bd67fc700 1 -- 172.21.15.84:0/2772410698 learned_addr learned my addr 172.21.15.84:0/2772410698 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3bdeffd700 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc16640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3bdf7fe700 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0d7c0 0x560e5dc170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3bd67fc700 1 -- 172.21.15.84:0/2772410698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0d7c0 msgr2=0x560e5dc170c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.362+0000 7f3bd67fc700 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0d7c0 0x560e5dc170c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3bd67fc700 1 -- 172.21.15.84:0/2772410698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 msgr2=0x560e5dc16640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3bd67fc700 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc16640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3bd67fc700 1 -- 172.21.15.84:0/2772410698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e5dc8ba80 con 0x560e5dc0cea0 2024-04-26T18:35:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3bdf7fe700 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0d7c0 0x560e5dc170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:35:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3bd67fc700 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 0x560e5dc16b80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800fa40 tx=0x7f3bc801a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cbe21b3fa5329a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3bc67fc700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:35:10.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3bc67fc700 1 -- 172.21.15.84:0/2772410698 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bc8031020 con 0x560e5dc0cea0 2024-04-26T18:35:10.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3bc67fc700 1 -- 172.21.15.84:0/2772410698 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8623+0+0 (secure 0 0 0) 0x7f3bc801abb0 con 0x560e5dc0cea0 2024-04-26T18:35:10.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3befea5b80 1 -- 172.21.15.84:0/2772410698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bc8004710 con 0x560e5dc0cea0 2024-04-26T18:35:10.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3bc67fc700 1 -- 172.21.15.84:0/2772410698 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3bc8029040 con 0x560e5dc0cea0 2024-04-26T18:35:10.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3befea5b80 10 client.16293 fetch_fsmap learned FSMap version 29 2024-04-26T18:35:10.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3befea5b80 1 -- 172.21.15.84:0/2772410698 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e5dc8cda0 con 0x560e5dc0cea0 2024-04-26T18:35:10.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3bc67fc700 1 -- 172.21.15.84:0/2772410698 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bc8004710 con 0x560e5dc0cea0 2024-04-26T18:35:10.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3bc67fc700 1 -- 172.21.15.84:0/2772410698 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3bc802c070 con 0x560e5dc0cea0 2024-04-26T18:35:10.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3befea5b80 10 client.16293 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:35:10.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3befea5b80 10 client.16293 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:35:10.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3befea5b80 20 client.16293 populate_metadata read hostname 'smithi084' 2024-04-26T18:35:10.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3befea5b80 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e5dc8d250 0x560e5dc8f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:10.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3befea5b80 4 client.16293 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "db61a169", "release": true, "await": true}] 2024-04-26T18:35:10.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.363+0000 7f3befea5b80 1 -- 172.21.15.84:0/2772410698 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "set_id": "db61a169", "release": true, "await": true}) -- 0x560e5db613b0 con 0x560e5dc8d250 2024-04-26T18:35:10.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.364+0000 7f3bdeffd700 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e5dc8d250 0x560e5dc8f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:10.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.364+0000 7f3bdeffd700 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e5dc8d250 0x560e5dc8f670 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46b3217b19eaedac server_cookie=5d9061996daf2971 in_seq=0 out_seq=0 2024-04-26T18:35:10.369 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.364+0000 7f3bc67fc700 10 client.16293 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:10.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3bc67fc700 1 -- 172.21.15.84:0/2772410698 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x560e5db613b0 con 0x560e5dc8d250 2024-04-26T18:35:10.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3bc67fc700 10 client.16293 handle_command_reply: tid=0 2024-04-26T18:35:10.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3befea5b80 1 client.16293 shutdown 2024-04-26T18:35:10.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3befea5b80 1 -- 172.21.15.84:0/2772410698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 msgr2=0x560e5dc16b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:10.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3befea5b80 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 0x560e5dc16b80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f3bc800fa40 tx=0x7f3bc801a690 comp rx=0 tx=0).stop 2024-04-26T18:35:10.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3befea5b80 1 -- 172.21.15.84:0/2772410698 shutdown_connections 2024-04-26T18:35:10.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3befea5b80 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560e5dc8d250 0x560e5dc8f670 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3befea5b80 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560e5dc0d7c0 0x560e5dc170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3befea5b80 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560e5dc07870 0x560e5dc16640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3bc67fc700 0 client.16293 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:10.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3befea5b80 1 --2- 172.21.15.84:0/2772410698 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560e5dc0cea0 0x560e5dc16b80 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3befea5b80 1 -- 172.21.15.84:0/2772410698 >> 172.21.15.84:0/2772410698 conn(0x560e5db356c0 msgr2=0x560e5dbfb670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:10.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3befea5b80 1 -- 172.21.15.84:0/2772410698 shutdown_connections 2024-04-26T18:35:10.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3befea5b80 1 -- 172.21.15.84:0/2772410698 wait complete. 2024-04-26T18:35:10.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3befea5b80 20 client.16293 trim_cache size 0 max 16384 2024-04-26T18:35:10.374 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:35:10.374 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:35:10.374 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:35:10.374 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 20, 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "db61a169": { 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 20, 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 91.7, 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:35:10.375 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:35:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:35:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:35:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:35:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:35:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:35:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:35:10.376 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:35:10.376 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:35:10.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3be5613700 1 -- 172.21.15.84:0/3975443380 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3bcc0a3db0 msgr2=0x7f3bcc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:10.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3be5613700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3bcc0a3db0 0x7f3bcc0a6270 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3bd0005950 tx=0x7f3bd000a300 comp rx=0 tx=0).stop 2024-04-26T18:35:10.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3be5613700 1 -- 172.21.15.84:0/3975443380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be01094d0 msgr2=0x7f3be01998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:10.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.369+0000 7f3be5613700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be01094d0 0x7f3be01998d0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3bd8007a20 tx=0x7f3bd8007a50 comp rx=0 tx=0).stop 2024-04-26T18:35:10.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.370+0000 7f3be5613700 1 -- 172.21.15.84:0/3975443380 shutdown_connections 2024-04-26T18:35:10.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.370+0000 7f3be5613700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3bcc0a3db0 0x7f3bcc0a6270 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.370+0000 7f3be5613700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3be01094d0 0x7f3be01998d0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.370+0000 7f3be5613700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3be0101970 0x7f3be019f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.370+0000 7f3be5613700 1 --2- 172.21.15.84:0/3975443380 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3be0102330 0x7f3be019fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:10.377 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.370+0000 7f3be5613700 1 -- 172.21.15.84:0/3975443380 >> 172.21.15.84:0/3975443380 conn(0x7f3be0078920 msgr2=0x7f3be00fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:10.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.370+0000 7f3be5613700 1 -- 172.21.15.84:0/3975443380 shutdown_connections 2024-04-26T18:35:10.378 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:35:10.370+0000 7f3be5613700 1 -- 172.21.15.84:0/3975443380 wait complete. 2024-04-26T18:35:10.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:10 smithi084 ceph-mon[29141]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T18:35:10.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:10 smithi084 ceph-mon[29141]: pgmap v222: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-26T18:35:10.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:10 smithi103 ceph-mon[32727]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T18:35:10.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:10 smithi103 ceph-mon[32727]: pgmap v222: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-26T18:35:10.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:10 smithi174 ceph-mon[32705]: osdmap e97: 12 total, 12 up, 12 in 2024-04-26T18:35:10.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:10 smithi174 ceph-mon[32705]: pgmap v222: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-26T18:35:11.114 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'db61a169', seconds elapsed: 43.4 2024-04-26T18:35:12.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:12 smithi174 ceph-mon[32705]: pgmap v223: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-26T18:35:13.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:12 smithi084 ceph-mon[29141]: pgmap v223: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-26T18:35:13.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:12 smithi103 ceph-mon[32727]: pgmap v223: 97 pgs: 97 active+clean; 3.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 57 KiB/s wr, 32 op/s 2024-04-26T18:35:13.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.864+0000 7ff01a3a4640 1 -- 172.21.15.84:0/1658273180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff01410cf20 msgr2=0x7ff01410f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.864+0000 7ff01a3a4640 1 --2- 172.21.15.84:0/1658273180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff01410cf20 0x7ff01410f390 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff00c009ca0 tx=0x7ff00c02a950 comp rx=0 tx=0).stop 2024-04-26T18:35:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.864+0000 7ff01a3a4640 1 -- 172.21.15.84:0/1658273180 shutdown_connections 2024-04-26T18:35:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.864+0000 7ff01a3a4640 1 --2- 172.21.15.84:0/1658273180 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff01410cf20 0x7ff01410f390 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.864+0000 7ff01a3a4640 1 --2- 172.21.15.84:0/1658273180 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff014104500 0x7ff01410c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.864+0000 7ff01a3a4640 1 --2- 172.21.15.84:0/1658273180 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff014103be0 0x7ff014103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.864+0000 7ff01a3a4640 1 -- 172.21.15.84:0/1658273180 >> 172.21.15.84:0/1658273180 conn(0x7ff0140789b0 msgr2=0x7ff014078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.864+0000 7ff01a3a4640 1 -- 172.21.15.84:0/1658273180 shutdown_connections 2024-04-26T18:35:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.864+0000 7ff01a3a4640 1 -- 172.21.15.84:0/1658273180 wait complete. 2024-04-26T18:35:13.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.865+0000 7ff01a3a4640 1 Processor -- start 2024-04-26T18:35:13.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.865+0000 7ff01a3a4640 1 -- start start 2024-04-26T18:35:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.865+0000 7ff01a3a4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff014103be0 0x7ff014101830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.865+0000 7ff01a3a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff014104500 0x7ff014101d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.865+0000 7ff01a3a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01410cf20 0x7ff0141022b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.865+0000 7ff01a3a4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff014112be0 con 0x7ff014104500 2024-04-26T18:35:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.865+0000 7ff01a3a4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff014112a60 con 0x7ff01410cf20 2024-04-26T18:35:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.865+0000 7ff01a3a4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff014112d60 con 0x7ff014103be0 2024-04-26T18:35:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.865+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff014104500 0x7ff014101d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.865+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff014104500 0x7ff014101d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34036/0 (socket says 172.21.15.84:34036) 2024-04-26T18:35:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.865+0000 7ff0137fe640 1 -- 172.21.15.84:0/1344767646 learned_addr learned my addr 172.21.15.84:0/1344767646 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.865+0000 7ff018941640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01410cf20 0x7ff0141022b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.865+0000 7ff013fff640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff014103be0 0x7ff014101830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.866+0000 7ff0137fe640 1 -- 172.21.15.84:0/1344767646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff014103be0 msgr2=0x7ff014101830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.866+0000 7ff0137fe640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff014103be0 0x7ff014101830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.866+0000 7ff0137fe640 1 -- 172.21.15.84:0/1344767646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01410cf20 msgr2=0x7ff0141022b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.866+0000 7ff0137fe640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01410cf20 0x7ff0141022b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.866+0000 7ff0137fe640 1 -- 172.21.15.84:0/1344767646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0141a7640 con 0x7ff014104500 2024-04-26T18:35:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.866+0000 7ff013fff640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff014103be0 0x7ff014101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:35:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.866+0000 7ff0137fe640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff014104500 0x7ff014101d70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff00800b4a0 tx=0x7ff00800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.866+0000 7ff0117fa640 1 -- 172.21.15.84:0/1344767646 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008014070 con 0x7ff014104500 2024-04-26T18:35:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.866+0000 7ff0117fa640 1 -- 172.21.15.84:0/1344767646 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff008010ac0 con 0x7ff014104500 2024-04-26T18:35:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.866+0000 7ff01a3a4640 1 -- 172.21.15.84:0/1344767646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0141a7930 con 0x7ff014104500 2024-04-26T18:35:13.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.866+0000 7ff0117fa640 1 -- 172.21.15.84:0/1344767646 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff008017c30 con 0x7ff014104500 2024-04-26T18:35:13.870 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.866+0000 7ff01a3a4640 1 -- 172.21.15.84:0/1344767646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0141a7e80 con 0x7ff014104500 2024-04-26T18:35:13.878 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.868+0000 7ff0117fa640 1 -- 172.21.15.84:0/1344767646 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff008017070 con 0x7ff014104500 2024-04-26T18:35:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff01d199740 1 Processor -- start 2024-04-26T18:35:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff01d199740 1 -- start start 2024-04-26T18:35:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff01d199740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f2f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff01d199740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f3043b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff01d199740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 0x56140f306ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff01d199740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56140f257460 con 0x56140f2fa030 2024-04-26T18:35:13.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff01d199740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56140f211590 con 0x56140f2f9710 2024-04-26T18:35:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff01d199740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56140f253420 con 0x56140f3048f0 2024-04-26T18:35:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff0117fa640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7feffc0a3dc0 0x7feffc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff013fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f3043b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff013fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f3043b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34052/0 (socket says 172.21.15.84:34052) 2024-04-26T18:35:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff013fff640 1 -- 172.21.15.84:0/4056930324 learned_addr learned my addr 172.21.15.84:0/4056930324 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff018941640 1 --2- 172.21.15.84:0/4056930324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f2f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff0137fe640 1 --2- 172.21.15.84:0/4056930324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 0x56140f306ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff018941640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7feffc0a3dc0 0x7feffc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.869+0000 7ff0117fa640 1 -- 172.21.15.84:0/1344767646 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8623+0+0 (secure 0 0 0) 0x7ff0080cfe30 con 0x7ff014104500 2024-04-26T18:35:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.870+0000 7ff013fff640 1 -- 172.21.15.84:0/4056930324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 msgr2=0x56140f306ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.870+0000 7ff013fff640 1 --2- 172.21.15.84:0/4056930324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 0x56140f306ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.870+0000 7ff013fff640 1 -- 172.21.15.84:0/4056930324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 msgr2=0x56140f2f9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.870+0000 7ff013fff640 1 --2- 172.21.15.84:0/4056930324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f2f9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.870+0000 7ff013fff640 1 -- 172.21.15.84:0/4056930324 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56140f21e0d0 con 0x56140f2fa030 2024-04-26T18:35:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.870+0000 7ff0137fe640 1 --2- 172.21.15.84:0/4056930324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 0x56140f306ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:35:13.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.870+0000 7ff018941640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7feffc0a3dc0 0x7feffc0a6280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff00c005fe0 tx=0x7ff00c005c50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.870+0000 7ff013fff640 1 --2- 172.21.15.84:0/4056930324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f3043b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff00400bb70 tx=0x7ff00400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e952073deb6e9363 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.870+0000 7feffa7fc640 1 -- 172.21.15.84:0/4056930324 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004017620 con 0x56140f2fa030 2024-04-26T18:35:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.870+0000 7feffa7fc640 1 -- 172.21.15.84:0/4056930324 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff004017c70 con 0x56140f2fa030 2024-04-26T18:35:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.870+0000 7feffa7fc640 1 -- 172.21.15.84:0/4056930324 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00400fba0 con 0x56140f2fa030 2024-04-26T18:35:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.871+0000 7ff01d199740 1 -- 172.21.15.84:0/4056930324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 msgr2=0x56140f3043b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:13.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.871+0000 7ff01d199740 1 --2- 172.21.15.84:0/4056930324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f3043b0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff00400bb70 tx=0x7ff00400d710 comp rx=0 tx=0).stop 2024-04-26T18:35:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.871+0000 7ff01d199740 1 -- 172.21.15.84:0/4056930324 shutdown_connections 2024-04-26T18:35:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.871+0000 7ff01d199740 1 --2- 172.21.15.84:0/4056930324 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 0x56140f306ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.871+0000 7ff01d199740 1 --2- 172.21.15.84:0/4056930324 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f3043b0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.871+0000 7ff01d199740 1 --2- 172.21.15.84:0/4056930324 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f2f9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.871+0000 7ff01d199740 1 -- 172.21.15.84:0/4056930324 >> 172.21.15.84:0/4056930324 conn(0x56140f2f5c20 msgr2=0x56140f2f6000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:13.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.871+0000 7ff01d199740 1 -- 172.21.15.84:0/4056930324 shutdown_connections 2024-04-26T18:35:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.871+0000 7ff01d199740 1 -- 172.21.15.84:0/4056930324 wait complete. 2024-04-26T18:35:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.871+0000 7ff01d199740 1 -- start start 2024-04-26T18:35:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.871+0000 7ff01d199740 1 Processor -- start 2024-04-26T18:35:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff01d199740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f3017e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff01d199740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f2fa030 0x56140f301d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff01d199740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f3048f0 0x56140f302260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff01d199740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56140f307520 con 0x56140f3048f0 2024-04-26T18:35:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff01d199740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56140f1deb60 con 0x56140f2f9710 2024-04-26T18:35:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff01d199740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56140f3076a0 con 0x56140f2fa030 2024-04-26T18:35:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff01d199740 10 client.0 _ll_register_callbacks cb 0x56140f2032b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff018941640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f3048f0 0x56140f302260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff018941640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f3048f0 0x56140f302260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34068/0 (socket says 172.21.15.84:34068) 2024-04-26T18:35:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff018941640 1 -- 172.21.15.84:0/3401614267 learned_addr learned my addr 172.21.15.84:0/3401614267 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff0137fe640 1 --2- 172.21.15.84:0/3401614267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f2fa030 0x56140f301d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff013fff640 1 --2- 172.21.15.84:0/3401614267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f3017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff018941640 1 -- 172.21.15.84:0/3401614267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f2fa030 msgr2=0x56140f301d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff018941640 1 --2- 172.21.15.84:0/3401614267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f2fa030 0x56140f301d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff018941640 1 -- 172.21.15.84:0/3401614267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 msgr2=0x56140f3017e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff018941640 1 --2- 172.21.15.84:0/3401614267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f3017e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff018941640 1 -- 172.21.15.84:0/3401614267 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56140ef69210 con 0x56140f3048f0 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff018941640 1 --2- 172.21.15.84:0/3401614267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f3048f0 0x56140f302260 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff00c009ca0 tx=0x7ff00c02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92206108cf39112 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7feff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7feff37fe640 1 -- 172.21.15.84:0/3401614267 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00c04f030 con 0x56140f3048f0 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7feff37fe640 1 -- 172.21.15.84:0/3401614267 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8623+0+0 (secure 0 0 0) 0x7ff00c004590 con 0x56140f3048f0 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.872+0000 7ff01d199740 1 -- 172.21.15.84:0/3401614267 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff004017c70 con 0x56140f3048f0 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.873+0000 7feff37fe640 1 -- 172.21.15.84:0/3401614267 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff00c034040 con 0x56140f3048f0 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.873+0000 7ff01d199740 10 client.16305 fetch_fsmap learned FSMap version 29 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.873+0000 7ff01d199740 1 -- 172.21.15.84:0/3401614267 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56140f035620 con 0x56140f3048f0 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.873+0000 7feff37fe640 1 -- 172.21.15.84:0/3401614267 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff004017c70 con 0x56140f3048f0 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.873+0000 7feff37fe640 1 -- 172.21.15.84:0/3401614267 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff00c04bbd0 con 0x56140f3048f0 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.873+0000 7ff01d199740 10 client.16305 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.873+0000 7ff01d199740 10 client.16305 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.873+0000 7ff01d199740 20 client.16305 populate_metadata read hostname 'smithi084' 2024-04-26T18:35:13.886 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.873+0000 7ff01d199740 1 --2- 172.21.15.84:0/3401614267 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56140f314a10 0x56140f316e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.873+0000 7ff01d199740 4 client.16305 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.873+0000 7ff01d199740 1 -- 172.21.15.84:0/3401614267 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56140f265070 con 0x56140f314a10 2024-04-26T18:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.873+0000 7ff013fff640 1 --2- 172.21.15.84:0/3401614267 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56140f314a10 0x56140f316e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.874+0000 7feff37fe640 10 client.16305 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.874+0000 7ff013fff640 1 --2- 172.21.15.84:0/3401614267 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56140f314a10 0x56140f316e20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94f655ff1d38bd0d server_cookie=edd242c156f41684 in_seq=0 out_seq=0 2024-04-26T18:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.874+0000 7feff37fe640 1 -- 172.21.15.84:0/3401614267 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56140f265070 con 0x56140f314a10 2024-04-26T18:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.874+0000 7feff37fe640 10 client.16305 handle_command_reply: tid=0 2024-04-26T18:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.875+0000 7ff01d199740 1 client.16305 shutdown 2024-04-26T18:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.875+0000 7ff01d199740 1 -- 172.21.15.84:0/3401614267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f3048f0 msgr2=0x56140f302260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.875+0000 7ff01d199740 1 --2- 172.21.15.84:0/3401614267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f3048f0 0x56140f302260 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff00c009ca0 tx=0x7ff00c02a950 comp rx=0 tx=0).stop 2024-04-26T18:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.875+0000 7ff01d199740 1 -- 172.21.15.84:0/3401614267 shutdown_connections 2024-04-26T18:35:13.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.875+0000 7ff01d199740 1 --2- 172.21.15.84:0/3401614267 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56140f314a10 0x56140f316e20 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.875+0000 7ff01d199740 1 --2- 172.21.15.84:0/3401614267 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f3048f0 0x56140f302260 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.875+0000 7ff01d199740 1 --2- 172.21.15.84:0/3401614267 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f2fa030 0x56140f301d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.875+0000 7ff01d199740 1 --2- 172.21.15.84:0/3401614267 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f3017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.875+0000 7feff37fe640 0 client.16305 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.875+0000 7ff01d199740 1 -- 172.21.15.84:0/3401614267 >> 172.21.15.84:0/3401614267 conn(0x56140f2f5c20 msgr2=0x56140f2ec5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.875+0000 7ff01d199740 1 -- 172.21.15.84:0/3401614267 shutdown_connections 2024-04-26T18:35:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.875+0000 7ff01d199740 1 -- 172.21.15.84:0/3401614267 wait complete. 2024-04-26T18:35:13.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.875+0000 7ff01d199740 20 client.16305 trim_cache size 0 max 16384 2024-04-26T18:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff01d199740 1 Processor -- start 2024-04-26T18:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff01d199740 1 -- start start 2024-04-26T18:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff01d199740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f3103e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff01d199740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f2fb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff01d199740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 0x56140f31c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff01d199740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141ac500 con 0x56140f2fa030 2024-04-26T18:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff01d199740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141ae040 con 0x56140f2f9710 2024-04-26T18:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff01d199740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141acbd0 con 0x56140f3048f0 2024-04-26T18:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f2fb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f2fb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34070/0 (socket says 172.21.15.84:34070) 2024-04-26T18:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff0137fe640 1 -- 172.21.15.84:0/1907176450 learned_addr learned my addr 172.21.15.84:0/1907176450 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:13.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff013fff640 1 --2- 172.21.15.84:0/1907176450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f3103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff018941640 1 --2- 172.21.15.84:0/1907176450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 0x56140f31c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff0137fe640 1 -- 172.21.15.84:0/1907176450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 msgr2=0x56140f31c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff0137fe640 1 --2- 172.21.15.84:0/1907176450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 0x56140f31c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff0137fe640 1 -- 172.21.15.84:0/1907176450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 msgr2=0x56140f3103e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff0137fe640 1 --2- 172.21.15.84:0/1907176450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f3103e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff0137fe640 1 -- 172.21.15.84:0/1907176450 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56140f2f5a50 con 0x56140f2fa030 2024-04-26T18:35:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.890+0000 7ff0137fe640 1 --2- 172.21.15.84:0/1907176450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f2fb7d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff0080cdb20 tx=0x7ff008012040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c6f4dcd2d2f47e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.891+0000 7feff9ffb640 1 -- 172.21.15.84:0/1907176450 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00801e040 con 0x56140f2fa030 2024-04-26T18:35:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.891+0000 7feff9ffb640 1 -- 172.21.15.84:0/1907176450 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0080a4af0 con 0x56140f2fa030 2024-04-26T18:35:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.891+0000 7feff9ffb640 1 -- 172.21.15.84:0/1907176450 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0080984f0 con 0x56140f2fa030 2024-04-26T18:35:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.891+0000 7ff01d199740 1 -- 172.21.15.84:0/1907176450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 msgr2=0x56140f2fb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.891+0000 7ff01d199740 1 --2- 172.21.15.84:0/1907176450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f2fb7d0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff0080cdb20 tx=0x7ff008012040 comp rx=0 tx=0).stop 2024-04-26T18:35:13.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.891+0000 7ff01d199740 1 -- 172.21.15.84:0/1907176450 shutdown_connections 2024-04-26T18:35:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.891+0000 7ff01d199740 1 --2- 172.21.15.84:0/1907176450 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 0x56140f31c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.891+0000 7ff01d199740 1 --2- 172.21.15.84:0/1907176450 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f2fb7d0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.891+0000 7ff01d199740 1 --2- 172.21.15.84:0/1907176450 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f3103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.891+0000 7ff01d199740 1 -- 172.21.15.84:0/1907176450 >> 172.21.15.84:0/1907176450 conn(0x56140f2f5c20 msgr2=0x56140f2fb0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.891+0000 7ff01d199740 1 -- 172.21.15.84:0/1907176450 shutdown_connections 2024-04-26T18:35:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.891+0000 7ff01d199740 1 -- 172.21.15.84:0/1907176450 wait complete. 2024-04-26T18:35:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.891+0000 7ff01d199740 1 -- start start 2024-04-26T18:35:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff01d199740 1 Processor -- start 2024-04-26T18:35:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff01d199740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f308610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff01d199740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f308b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff01d199740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 0x56140f3090b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff01d199740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141ad970 con 0x56140f2fa030 2024-04-26T18:35:13.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff01d199740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141ad2a0 con 0x56140f2f9710 2024-04-26T18:35:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff01d199740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff0141aa520 con 0x56140f3048f0 2024-04-26T18:35:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff01d199740 10 client.0 _ll_register_callbacks cb 0x7ff00c02f900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f308b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff0137fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f308b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34084/0 (socket says 172.21.15.84:34084) 2024-04-26T18:35:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff0137fe640 1 -- 172.21.15.84:0/3641686511 learned_addr learned my addr 172.21.15.84:0/3641686511 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff013fff640 1 --2- 172.21.15.84:0/3641686511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f308610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff0137fe640 1 -- 172.21.15.84:0/3641686511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 msgr2=0x56140f3090b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff0137fe640 1 --2- 172.21.15.84:0/3641686511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 0x56140f3090b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff0137fe640 1 -- 172.21.15.84:0/3641686511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 msgr2=0x56140f308610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:13.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff0137fe640 1 --2- 172.21.15.84:0/3641686511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f308610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff0137fe640 1 -- 172.21.15.84:0/3641686511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56140f37a2c0 con 0x56140f2fa030 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff013fff640 1 --2- 172.21.15.84:0/3641686511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f308610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7ff0137fe640 1 --2- 172.21.15.84:0/3641686511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f308b50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff00800be60 tx=0x7ff0080cd7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ae3d45e5b8ec6f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.892+0000 7feff37fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7feff37fe640 1 -- 172.21.15.84:0/3641686511 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00801e040 con 0x56140f2fa030 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7feff37fe640 1 -- 172.21.15.84:0/3641686511 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 8623+0+0 (secure 0 0 0) 0x7ff008010c70 con 0x56140f2fa030 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7ff01d199740 1 -- 172.21.15.84:0/3641686511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0080a4af0 con 0x56140f2fa030 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7feff37fe640 1 -- 172.21.15.84:0/3641686511 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0080851a0 con 0x56140f2fa030 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7ff01d199740 10 client.16311 fetch_fsmap learned FSMap version 29 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7ff01d199740 1 -- 172.21.15.84:0/3641686511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56140f37b570 con 0x56140f2fa030 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7feff37fe640 1 -- 172.21.15.84:0/3641686511 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0080a4af0 con 0x56140f2fa030 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7feff37fe640 1 -- 172.21.15.84:0/3641686511 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff0080a4660 con 0x56140f2fa030 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7ff01d199740 10 client.16311 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:35:13.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7ff01d199740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7ff01d199740 20 client.16311 populate_metadata read hostname 'smithi084' 2024-04-26T18:35:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7ff01d199740 1 --2- 172.21.15.84:0/3641686511 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56140f37bb80 0x56140f37df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7ff01d199740 4 client.16311 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:35:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7ff01d199740 1 -- 172.21.15.84:0/3641686511 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56140f25eee0 con 0x56140f37bb80 2024-04-26T18:35:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.893+0000 7ff013fff640 1 --2- 172.21.15.84:0/3641686511 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56140f37bb80 0x56140f37df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.894+0000 7feff37fe640 10 client.16311 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:13.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.894+0000 7ff013fff640 1 --2- 172.21.15.84:0/3641686511 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56140f37bb80 0x56140f37df70 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2f097f0b74fb7f0 server_cookie=ec0c447f829ccf9e in_seq=0 out_seq=0 2024-04-26T18:35:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7feff37fe640 1 -- 172.21.15.84:0/3641686511 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56140f25eee0 con 0x56140f37bb80 2024-04-26T18:35:13.905 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7feff37fe640 10 client.16311 handle_command_reply: tid=0 2024-04-26T18:35:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7ff01d199740 1 client.16311 shutdown 2024-04-26T18:35:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7ff01d199740 1 -- 172.21.15.84:0/3641686511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 msgr2=0x56140f308b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7ff01d199740 1 --2- 172.21.15.84:0/3641686511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f308b50 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7ff00800be60 tx=0x7ff0080cd7a0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7ff01d199740 1 -- 172.21.15.84:0/3641686511 shutdown_connections 2024-04-26T18:35:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7ff01d199740 1 --2- 172.21.15.84:0/3641686511 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56140f37bb80 0x56140f37df70 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7ff01d199740 1 --2- 172.21.15.84:0/3641686511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56140f3048f0 0x56140f3090b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7ff01d199740 1 --2- 172.21.15.84:0/3641686511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56140f2fa030 0x56140f308b50 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7feff37fe640 0 client.16311 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7ff01d199740 1 --2- 172.21.15.84:0/3641686511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56140f2f9710 0x56140f308610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7ff01d199740 1 -- 172.21.15.84:0/3641686511 >> 172.21.15.84:0/3641686511 conn(0x56140f2f5c20 msgr2=0x56140f2ec5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:13.906 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7ff01d199740 1 -- 172.21.15.84:0/3641686511 shutdown_connections 2024-04-26T18:35:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7ff01d199740 1 -- 172.21.15.84:0/3641686511 wait complete. 2024-04-26T18:35:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.905+0000 7ff01d199740 20 client.16311 trim_cache size 0 max 16384 2024-04-26T18:35:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:35:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:35:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:35:13.907 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:35:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.906+0000 7ff01a3a4640 1 -- 172.21.15.84:0/1344767646 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7feffc0a3dc0 msgr2=0x7feffc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:13.908 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.906+0000 7ff01a3a4640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7feffc0a3dc0 0x7feffc0a6280 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff00c005fe0 tx=0x7ff00c005c50 comp rx=0 tx=0).stop 2024-04-26T18:35:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.906+0000 7ff01a3a4640 1 -- 172.21.15.84:0/1344767646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff014104500 msgr2=0x7ff014101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.906+0000 7ff01a3a4640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff014104500 0x7ff014101d70 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff00800b4a0 tx=0x7ff00800b960 comp rx=0 tx=0).stop 2024-04-26T18:35:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.906+0000 7ff01a3a4640 1 -- 172.21.15.84:0/1344767646 shutdown_connections 2024-04-26T18:35:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.906+0000 7ff01a3a4640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7feffc0a3dc0 0x7feffc0a6280 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.906+0000 7ff01a3a4640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff01410cf20 0x7ff0141022b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.906+0000 7ff01a3a4640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff014104500 0x7ff014101d70 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.906+0000 7ff01a3a4640 1 --2- 172.21.15.84:0/1344767646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff014103be0 0x7ff014101830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.906+0000 7ff01a3a4640 1 -- 172.21.15.84:0/1344767646 >> 172.21.15.84:0/1344767646 conn(0x7ff0140789b0 msgr2=0x7ff0140ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.906+0000 7ff01a3a4640 1 -- 172.21.15.84:0/1344767646 shutdown_connections 2024-04-26T18:35:13.909 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:13.906+0000 7ff01a3a4640 1 -- 172.21.15.84:0/1344767646 wait complete. 2024-04-26T18:35:14.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:14 smithi174 ceph-mon[32705]: pgmap v224: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-26T18:35:15.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:14 smithi084 ceph-mon[29141]: pgmap v224: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-26T18:35:15.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:14 smithi103 ceph-mon[32727]: pgmap v224: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-26T18:35:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:16 smithi174 ceph-mon[32705]: pgmap v225: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-26T18:35:17.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:16 smithi084 ceph-mon[29141]: pgmap v225: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-26T18:35:17.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:16 smithi103 ceph-mon[32727]: pgmap v225: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 16 KiB/s wr, 12 op/s 2024-04-26T18:35:18.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:18 smithi174 ceph-mon[32705]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T18:35:18.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:18 smithi084 ceph-mon[29141]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T18:35:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:18 smithi103 ceph-mon[32727]: osdmap e98: 12 total, 12 up, 12 in 2024-04-26T18:35:19.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:19 smithi174 ceph-mon[32705]: pgmap v227: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 8 op/s 2024-04-26T18:35:19.778 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:19 smithi103 ceph-mon[32727]: pgmap v227: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 8 op/s 2024-04-26T18:35:19.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:19 smithi084 ceph-mon[29141]: pgmap v227: 97 pgs: 97 active+clean; 3.2 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 20 KiB/s wr, 8 op/s 2024-04-26T18:35:21.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:21 smithi174 ceph-mon[32705]: pgmap v228: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-26T18:35:21.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:21 smithi084 ceph-mon[29141]: pgmap v228: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-26T18:35:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:21 smithi103 ceph-mon[32727]: pgmap v228: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-26T18:35:23.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:23 smithi174 ceph-mon[32705]: pgmap v229: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-26T18:35:23.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:23 smithi084 ceph-mon[29141]: pgmap v229: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-26T18:35:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:23 smithi103 ceph-mon[32727]: pgmap v229: 97 pgs: 97 active+clean; 3.4 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-26T18:35:25.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:25 smithi174 ceph-mon[32705]: pgmap v230: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T18:35:25.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:25 smithi084 ceph-mon[29141]: pgmap v230: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T18:35:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:25 smithi103 ceph-mon[32727]: pgmap v230: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T18:35:27.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.001+0000 7f3f28dc9640 1 -- 172.21.15.84:0/2938205381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f2410cf00 msgr2=0x7f3f2410f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.001+0000 7f3f28dc9640 1 --2- 172.21.15.84:0/2938205381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f2410cf00 0x7f3f2410f3a0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c009ca0 tx=0x7f3f1c02a9e0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.001+0000 7f3f28dc9640 1 -- 172.21.15.84:0/2938205381 shutdown_connections 2024-04-26T18:35:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.001+0000 7f3f28dc9640 1 --2- 172.21.15.84:0/2938205381 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f2410cf00 0x7f3f2410f3a0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.001+0000 7f3f28dc9640 1 --2- 172.21.15.84:0/2938205381 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f241044e0 0x7f3f2410c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.001+0000 7f3f28dc9640 1 --2- 172.21.15.84:0/2938205381 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f24103bc0 0x7f3f24103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.001+0000 7f3f28dc9640 1 -- 172.21.15.84:0/2938205381 >> 172.21.15.84:0/2938205381 conn(0x7f3f240789b0 msgr2=0x7f3f24078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.001+0000 7f3f28dc9640 1 -- 172.21.15.84:0/2938205381 shutdown_connections 2024-04-26T18:35:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.001+0000 7f3f28dc9640 1 -- 172.21.15.84:0/2938205381 wait complete. 2024-04-26T18:35:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.002+0000 7f3f28dc9640 1 Processor -- start 2024-04-26T18:35:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.002+0000 7f3f28dc9640 1 -- start start 2024-04-26T18:35:27.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.002+0000 7f3f28dc9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f24103bc0 0x7f3f24101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.002+0000 7f3f28dc9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f241044e0 0x7f3f24101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.002+0000 7f3f28dc9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f2410cf00 0x7f3f24102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.002+0000 7f3f28dc9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3f24112bb0 con 0x7f3f241044e0 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.002+0000 7f3f28dc9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f24112a30 con 0x7f3f2410cf00 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.002+0000 7f3f28dc9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f24112d30 con 0x7f3f24103bc0 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.002+0000 7f3f21d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f241044e0 0x7f3f24101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.002+0000 7f3f21d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f241044e0 0x7f3f24101d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43084/0 (socket says 172.21.15.84:43084) 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.002+0000 7f3f21d9b640 1 -- 172.21.15.84:0/63634533 learned_addr learned my addr 172.21.15.84:0/63634533 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f22d9d640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f2410cf00 0x7f3f24102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f2259c640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f24103bc0 0x7f3f24101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f21d9b640 1 -- 172.21.15.84:0/63634533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f24103bc0 msgr2=0x7f3f24101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f24103bc0 0x7f3f24101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f21d9b640 1 -- 172.21.15.84:0/63634533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f2410cf00 msgr2=0x7f3f24102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:27.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f2410cf00 0x7f3f24102290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f21d9b640 1 -- 172.21.15.84:0/63634533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f241a75b0 con 0x7f3f241044e0 2024-04-26T18:35:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f2259c640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f24103bc0 0x7f3f24101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:35:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f241044e0 0x7f3f24101d50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400d770 tx=0x7f3f1400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f1b7fe640 1 -- 172.21.15.84:0/63634533 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14017070 con 0x7f3f241044e0 2024-04-26T18:35:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f28dc9640 1 -- 172.21.15.84:0/63634533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f241a78a0 con 0x7f3f241044e0 2024-04-26T18:35:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f1b7fe640 1 -- 172.21.15.84:0/63634533 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f140054d0 con 0x7f3f241044e0 2024-04-26T18:35:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f1b7fe640 1 -- 172.21.15.84:0/63634533 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14004c30 con 0x7f3f241044e0 2024-04-26T18:35:27.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.003+0000 7f3f28dc9640 1 -- 172.21.15.84:0/63634533 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f241a7da0 con 0x7f3f241044e0 2024-04-26T18:35:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.006+0000 7f3f1b7fe640 1 -- 172.21.15.84:0/63634533 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3f14004e10 con 0x7f3f241044e0 2024-04-26T18:35:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.006+0000 7f3f2bbb7740 1 Processor -- start 2024-04-26T18:35:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.006+0000 7f3f2bbb7740 1 -- start start 2024-04-26T18:35:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.006+0000 7f3f2bbb7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efabf00 0x55738efac2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.006+0000 7f3f2bbb7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efac820 0x55738efb6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.013 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.006+0000 7f3f2bbb7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efb70e0 0x55738efb94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.006+0000 7f3f2bbb7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55738ef12460 con 0x55738efb70e0 2024-04-26T18:35:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.006+0000 7f3f2bbb7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55738eecc590 con 0x55738efabf00 2024-04-26T18:35:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.006+0000 7f3f2bbb7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55738ef0e420 con 0x55738efac820 2024-04-26T18:35:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.006+0000 7f3f1b7fe640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3f100a3dc0 0x7f3f100a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f21d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efb70e0 0x55738efb94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f21d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efb70e0 0x55738efb94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43096/0 (socket says 172.21.15.84:43096) 2024-04-26T18:35:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f21d9b640 1 -- 172.21.15.84:0/22542316 learned_addr learned my addr 172.21.15.84:0/22542316 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f22d9d640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3f100a3dc0 0x7f3f100a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f2259c640 1 --2- 172.21.15.84:0/22542316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efac820 0x55738efb6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f22d9d640 1 --2- 172.21.15.84:0/22542316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efabf00 0x55738efac2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f1b7fe640 1 -- 172.21.15.84:0/63634533 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f3f140d0360 con 0x7f3f241044e0 2024-04-26T18:35:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f21d9b640 1 -- 172.21.15.84:0/22542316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efac820 msgr2=0x55738efb6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:27.014 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/22542316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efac820 0x55738efb6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f21d9b640 1 -- 172.21.15.84:0/22542316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efabf00 msgr2=0x55738efac2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/22542316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efabf00 0x55738efac2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f21d9b640 1 -- 172.21.15.84:0/22542316 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738eed90d0 con 0x55738efb70e0 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/22542316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efb70e0 0x55738efb94d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3f14098c90 tx=0x7f3f1409bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead660e1958f28b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f22d9d640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3f100a3dc0 0x7f3f100a6280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c009880 tx=0x7f3f1c009900 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f18ff9640 1 -- 172.21.15.84:0/22542316 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14020030 con 0x55738efb70e0 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f18ff9640 1 -- 172.21.15.84:0/22542316 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f1409c2c0 con 0x55738efb70e0 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.007+0000 7f3f18ff9640 1 -- 172.21.15.84:0/22542316 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f1409c520 con 0x55738efb70e0 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.008+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/22542316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efb70e0 msgr2=0x55738efb94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.008+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/22542316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efb70e0 0x55738efb94d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f3f14098c90 tx=0x7f3f1409bc20 comp rx=0 tx=0).stop 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.008+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/22542316 shutdown_connections 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.008+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/22542316 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efb70e0 0x55738efb94d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.008+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/22542316 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efac820 0x55738efb6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.015 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.008+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/22542316 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efabf00 0x55738efac2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.008+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/22542316 >> 172.21.15.84:0/22542316 conn(0x55738efa8410 msgr2=0x55738efa87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.008+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/22542316 shutdown_connections 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.008+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/22542316 wait complete. 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2bbb7740 1 -- start start 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2bbb7740 1 Processor -- start 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2bbb7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 0x55738efb40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2bbb7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efac820 0x55738efb45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2bbb7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efb70e0 0x55738efb4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2bbb7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55738efb9cb0 con 0x55738efabf00 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2bbb7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55738ee99b60 con 0x55738efb70e0 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2bbb7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55738efb9e30 con 0x55738efac820 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2bbb7740 10 client.0 _ll_register_callbacks cb 0x55738eebe2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 0x55738efb40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 0x55738efb40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43108/0 (socket says 172.21.15.84:43108) 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2259c640 1 -- 172.21.15.84:0/2909105188 learned_addr learned my addr 172.21.15.84:0/2909105188 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f22d9d640 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efb70e0 0x55738efb4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efac820 0x55738efb45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2259c640 1 -- 172.21.15.84:0/2909105188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efac820 msgr2=0x55738efb45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2259c640 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efac820 0x55738efb45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2259c640 1 -- 172.21.15.84:0/2909105188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efb70e0 msgr2=0x55738efb4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2259c640 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efb70e0 0x55738efb4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f2259c640 1 -- 172.21.15.84:0/2909105188 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738ec24210 con 0x55738efabf00 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f22d9d640 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efb70e0 0x55738efb4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.009+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efac820 0x55738efb45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f2259c640 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 0x55738efb40b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009e90 tx=0x7f3f0c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95eedbb8917d522b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f09ffb640 1 -- 172.21.15.84:0/2909105188 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c012860 con 0x55738efabf00 2024-04-26T18:35:27.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f09ffb640 1 -- 172.21.15.84:0/2909105188 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f3f0c012a00 con 0x55738efabf00 2024-04-26T18:35:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/2909105188 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f1409c2c0 con 0x55738efabf00 2024-04-26T18:35:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f09ffb640 1 -- 172.21.15.84:0/2909105188 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3f0c012c00 con 0x55738efabf00 2024-04-26T18:35:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f2bbb7740 10 client.16323 fetch_fsmap learned FSMap version 29 2024-04-26T18:35:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/2909105188 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55738ecf0620 con 0x55738efabf00 2024-04-26T18:35:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f09ffb640 1 -- 172.21.15.84:0/2909105188 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f1409c2c0 con 0x55738efabf00 2024-04-26T18:35:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f09ffb640 1 -- 172.21.15.84:0/2909105188 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3f0c00f040 con 0x55738efabf00 2024-04-26T18:35:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f2bbb7740 10 client.16323 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:35:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f2bbb7740 10 client.16323 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f2bbb7740 20 client.16323 populate_metadata read hostname 'smithi084' 2024-04-26T18:35:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55738efc7310 0x55738efc9760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f2bbb7740 4 client.16323 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:35:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.010+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/2909105188 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55738ef20070 con 0x55738efc7310 2024-04-26T18:35:27.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.011+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55738efc7310 0x55738efc9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.011+0000 7f3f09ffb640 10 client.16323 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.011+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55738efc7310 0x55738efc9760 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=657670846fd2041f server_cookie=526d120aa3c3f1ae in_seq=0 out_seq=0 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f09ffb640 1 -- 172.21.15.84:0/2909105188 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55738ef20070 con 0x55738efc7310 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f09ffb640 10 client.16323 handle_command_reply: tid=0 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f2bbb7740 1 client.16323 shutdown 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/2909105188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 msgr2=0x55738efb40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 0x55738efb40b0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c009e90 tx=0x7f3f0c0117b0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/2909105188 shutdown_connections 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55738efc7310 0x55738efc9760 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efb70e0 0x55738efb4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efac820 0x55738efb45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/2909105188 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 0x55738efb40b0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f09ffb640 0 client.16323 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:27.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/2909105188 >> 172.21.15.84:0/2909105188 conn(0x55738efa8410 msgr2=0x55738efae140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/2909105188 shutdown_connections 2024-04-26T18:35:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/2909105188 wait complete. 2024-04-26T18:35:27.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.012+0000 7f3f2bbb7740 20 client.16323 trim_cache size 0 max 16384 2024-04-26T18:35:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.027+0000 7f3f2bbb7740 1 Processor -- start 2024-04-26T18:35:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.027+0000 7f3f2bbb7740 1 -- start start 2024-04-26T18:35:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.027+0000 7f3f2bbb7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efabf00 0x55738efc2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.027+0000 7f3f2bbb7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efac820 0x55738efa7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.027+0000 7f3f2bbb7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efb70e0 0x55738efcf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.027+0000 7f3f2bbb7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3f241b0780 con 0x55738efac820 2024-04-26T18:35:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.027+0000 7f3f2bbb7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f241b22c0 con 0x55738efb70e0 2024-04-26T18:35:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.027+0000 7f3f2bbb7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f241b0e50 con 0x55738efabf00 2024-04-26T18:35:27.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.027+0000 7f3f21d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efac820 0x55738efa7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.027+0000 7f3f21d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efac820 0x55738efa7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43114/0 (socket says 172.21.15.84:43114) 2024-04-26T18:35:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.027+0000 7f3f21d9b640 1 -- 172.21.15.84:0/3459954243 learned_addr learned my addr 172.21.15.84:0/3459954243 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.028+0000 7f3f22d9d640 1 --2- 172.21.15.84:0/3459954243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efb70e0 0x55738efcf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.028+0000 7f3f21d9b640 1 -- 172.21.15.84:0/3459954243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efabf00 msgr2=0x55738efc2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.028+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/3459954243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efabf00 0x55738efc2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.028+0000 7f3f21d9b640 1 -- 172.21.15.84:0/3459954243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efb70e0 msgr2=0x55738efcf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.028+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/3459954243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efb70e0 0x55738efcf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.028+0000 7f3f21d9b640 1 -- 172.21.15.84:0/3459954243 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55738efac650 con 0x55738efac820 2024-04-26T18:35:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.028+0000 7f3f22d9d640 1 --2- 172.21.15.84:0/3459954243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efb70e0 0x55738efcf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:35:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.028+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/3459954243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efac820 0x55738efa7430 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3f140a1760 tx=0x7f3f140d0270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dbc370e6894b5d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.028+0000 7f3f0bfff640 1 -- 172.21.15.84:0/3459954243 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14020040 con 0x55738efac820 2024-04-26T18:35:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.028+0000 7f3f0bfff640 1 -- 172.21.15.84:0/3459954243 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f1409a5c0 con 0x55738efac820 2024-04-26T18:35:27.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.028+0000 7f3f0bfff640 1 -- 172.21.15.84:0/3459954243 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f14098ad0 con 0x55738efac820 2024-04-26T18:35:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/3459954243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efac820 msgr2=0x55738efa7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/3459954243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efac820 0x55738efa7430 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f3f140a1760 tx=0x7f3f140d0270 comp rx=0 tx=0).stop 2024-04-26T18:35:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/3459954243 shutdown_connections 2024-04-26T18:35:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/3459954243 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efb70e0 0x55738efcf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/3459954243 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efac820 0x55738efa7430 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/3459954243 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efabf00 0x55738efc2b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/3459954243 >> 172.21.15.84:0/3459954243 conn(0x55738efa8410 msgr2=0x55738efa6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/3459954243 shutdown_connections 2024-04-26T18:35:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/3459954243 wait complete. 2024-04-26T18:35:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 -- start start 2024-04-26T18:35:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 Processor -- start 2024-04-26T18:35:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 0x55738efbece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efac820 0x55738efbf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efb70e0 0x55738efbb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3f241b1bf0 con 0x55738efabf00 2024-04-26T18:35:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3f241b1520 con 0x55738efac820 2024-04-26T18:35:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3f241ae780 con 0x55738efb70e0 2024-04-26T18:35:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2bbb7740 10 client.0 _ll_register_callbacks cb 0x7f3f0c009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 0x55738efbece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 0x55738efbece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43116/0 (socket says 172.21.15.84:43116) 2024-04-26T18:35:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.029+0000 7f3f2259c640 1 -- 172.21.15.84:0/1621744482 learned_addr learned my addr 172.21.15.84:0/1621744482 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/1621744482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efac820 0x55738efbf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f22d9d640 1 --2- 172.21.15.84:0/1621744482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efb70e0 0x55738efbb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2259c640 1 -- 172.21.15.84:0/1621744482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efb70e0 msgr2=0x55738efbb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2259c640 1 --2- 172.21.15.84:0/1621744482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efb70e0 0x55738efbb1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2259c640 1 -- 172.21.15.84:0/1621744482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efac820 msgr2=0x55738efbf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2259c640 1 --2- 172.21.15.84:0/1621744482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efac820 0x55738efbf220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2259c640 1 -- 172.21.15.84:0/1621744482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55738efbb8f0 con 0x55738efabf00 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2259c640 1 --2- 172.21.15.84:0/1621744482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 0x55738efbece0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c01af70 tx=0x7f3f0c011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f60a61ee729d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f09ffb640 1 -- 172.21.15.84:0/1621744482 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c030020 con 0x55738efabf00 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f09ffb640 1 -- 172.21.15.84:0/1621744482 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f3f0c012860 con 0x55738efabf00 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/1621744482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3f1409a5c0 con 0x55738efabf00 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f09ffb640 1 -- 172.21.15.84:0/1621744482 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3f0c03b050 con 0x55738efabf00 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2bbb7740 10 client.16329 fetch_fsmap learned FSMap version 29 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/1621744482 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55738f02e270 con 0x55738efabf00 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f09ffb640 1 -- 172.21.15.84:0/1621744482 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f1409a5c0 con 0x55738efabf00 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f09ffb640 1 -- 172.21.15.84:0/1621744482 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3f0c01a5f0 con 0x55738efabf00 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2bbb7740 10 client.16329 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2bbb7740 10 client.16329 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:27.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2bbb7740 20 client.16329 populate_metadata read hostname 'smithi084' 2024-04-26T18:35:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/1621744482 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55738f02e9e0 0x55738f030e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2bbb7740 4 client.16329 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:35:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.030+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/1621744482 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55738ef19ee0 con 0x55738f02e9e0 2024-04-26T18:35:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.031+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/1621744482 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55738f02e9e0 0x55738f030e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.031+0000 7f3f21d9b640 1 --2- 172.21.15.84:0/1621744482 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55738f02e9e0 0x55738f030e00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbb9943aba81d7ce server_cookie=85988f582f75cc04 in_seq=0 out_seq=0 2024-04-26T18:35:27.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.031+0000 7f3f09ffb640 10 client.16329 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:27.045 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f09ffb640 1 -- 172.21.15.84:0/1621744482 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55738ef19ee0 con 0x55738f02e9e0 2024-04-26T18:35:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f09ffb640 10 client.16329 handle_command_reply: tid=0 2024-04-26T18:35:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f2bbb7740 1 client.16329 shutdown 2024-04-26T18:35:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/1621744482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 msgr2=0x55738efbece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/1621744482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 0x55738efbece0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c01af70 tx=0x7f3f0c011cb0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/1621744482 shutdown_connections 2024-04-26T18:35:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/1621744482 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55738f02e9e0 0x55738f030e00 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/1621744482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55738efb70e0 0x55738efbb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/1621744482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55738efac820 0x55738efbf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f09ffb640 0 client.16329 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f2bbb7740 1 --2- 172.21.15.84:0/1621744482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55738efabf00 0x55738efbece0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/1621744482 >> 172.21.15.84:0/1621744482 conn(0x55738efa8410 msgr2=0x55738efa75b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/1621744482 shutdown_connections 2024-04-26T18:35:27.046 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f2bbb7740 1 -- 172.21.15.84:0/1621744482 wait complete. 2024-04-26T18:35:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.045+0000 7f3f2bbb7740 20 client.16329 trim_cache size 0 max 16384 2024-04-26T18:35:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:35:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:35:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:35:27.047 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:35:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.046+0000 7f3f28dc9640 1 -- 172.21.15.84:0/63634533 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3f100a3dc0 msgr2=0x7f3f100a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.046+0000 7f3f28dc9640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3f100a3dc0 0x7f3f100a6280 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c009880 tx=0x7f3f1c009900 comp rx=0 tx=0).stop 2024-04-26T18:35:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.046+0000 7f3f28dc9640 1 -- 172.21.15.84:0/63634533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f241044e0 msgr2=0x7f3f24101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.046+0000 7f3f28dc9640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f241044e0 0x7f3f24101d50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400d770 tx=0x7f3f1400dc30 comp rx=0 tx=0).stop 2024-04-26T18:35:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.046+0000 7f3f28dc9640 1 -- 172.21.15.84:0/63634533 shutdown_connections 2024-04-26T18:35:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.046+0000 7f3f28dc9640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3f100a3dc0 0x7f3f100a6280 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.046+0000 7f3f28dc9640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3f2410cf00 0x7f3f24102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.046+0000 7f3f28dc9640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3f241044e0 0x7f3f24101d50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.048 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.046+0000 7f3f28dc9640 1 --2- 172.21.15.84:0/63634533 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3f24103bc0 0x7f3f24101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.046+0000 7f3f28dc9640 1 -- 172.21.15.84:0/63634533 >> 172.21.15.84:0/63634533 conn(0x7f3f240789b0 msgr2=0x7f3f240fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.046+0000 7f3f28dc9640 1 -- 172.21.15.84:0/63634533 shutdown_connections 2024-04-26T18:35:27.049 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:27.046+0000 7f3f28dc9640 1 -- 172.21.15.84:0/63634533 wait complete. 2024-04-26T18:35:27.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:27 smithi174 ceph-mon[32705]: pgmap v231: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T18:35:27.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:27 smithi084 ceph-mon[29141]: pgmap v231: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T18:35:27.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:27 smithi103 ceph-mon[32727]: pgmap v231: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 44 KiB/s wr, 18 op/s 2024-04-26T18:35:29.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:29 smithi174 ceph-mon[32705]: pgmap v232: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-26T18:35:29.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:29 smithi084 ceph-mon[29141]: pgmap v232: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-26T18:35:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:29 smithi103 ceph-mon[32727]: pgmap v232: 97 pgs: 97 active+clean; 3.6 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 43 KiB/s wr, 18 op/s 2024-04-26T18:35:31.661 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:31 smithi084 ceph-mon[29141]: pgmap v233: 97 pgs: 97 active+clean; 3.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 24 op/s 2024-04-26T18:35:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:31 smithi174 ceph-mon[32705]: pgmap v233: 97 pgs: 97 active+clean; 3.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 24 op/s 2024-04-26T18:35:31.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:31 smithi103 ceph-mon[32727]: pgmap v233: 97 pgs: 97 active+clean; 3.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 57 KiB/s wr, 24 op/s 2024-04-26T18:35:33.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:33 smithi174 ceph-mon[32705]: pgmap v234: 97 pgs: 97 active+clean; 3.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-26T18:35:33.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:33 smithi084 ceph-mon[29141]: pgmap v234: 97 pgs: 97 active+clean; 3.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-26T18:35:33.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:33 smithi103 ceph-mon[32727]: pgmap v234: 97 pgs: 97 active+clean; 3.8 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 37 KiB/s wr, 15 op/s 2024-04-26T18:35:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:35 smithi174 ceph-mon[32705]: pgmap v235: 97 pgs: 97 active+clean; 4.0 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-26T18:35:35.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:35 smithi084 ceph-mon[29141]: pgmap v235: 97 pgs: 97 active+clean; 4.0 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-26T18:35:35.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:35 smithi103 ceph-mon[32727]: pgmap v235: 97 pgs: 97 active+clean; 4.0 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-26T18:35:37.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:37 smithi174 ceph-mon[32705]: pgmap v236: 97 pgs: 97 active+clean; 4.0 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-26T18:35:37.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:37 smithi084 ceph-mon[29141]: pgmap v236: 97 pgs: 97 active+clean; 4.0 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-26T18:35:37.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:37 smithi103 ceph-mon[32727]: pgmap v236: 97 pgs: 97 active+clean; 4.0 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-26T18:35:39.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:39 smithi174 ceph-mon[32705]: pgmap v237: 97 pgs: 97 active+clean; 4.0 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-26T18:35:39.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:39 smithi084 ceph-mon[29141]: pgmap v237: 97 pgs: 97 active+clean; 4.0 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-26T18:35:39.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:39 smithi103 ceph-mon[32727]: pgmap v237: 97 pgs: 97 active+clean; 4.0 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-26T18:35:40.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.143+0000 7f78b607e640 1 -- 172.21.15.84:0/441604250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b010cf50 msgr2=0x7f78b010f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.143+0000 7f78b607e640 1 --2- 172.21.15.84:0/441604250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b010cf50 0x7f78b010f3c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f78a8009ca0 tx=0x7f78a802a950 comp rx=0 tx=0).stop 2024-04-26T18:35:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.143+0000 7f78b607e640 1 -- 172.21.15.84:0/441604250 shutdown_connections 2024-04-26T18:35:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.143+0000 7f78b607e640 1 --2- 172.21.15.84:0/441604250 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b010cf50 0x7f78b010f3c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.143+0000 7f78b607e640 1 --2- 172.21.15.84:0/441604250 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b01044a0 0x7f78b010c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.143+0000 7f78b607e640 1 --2- 172.21.15.84:0/441604250 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78b0103b80 0x7f78b0103f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.143+0000 7f78b607e640 1 -- 172.21.15.84:0/441604250 >> 172.21.15.84:0/441604250 conn(0x7f78b0078a40 msgr2=0x7f78b0078e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.143+0000 7f78b607e640 1 -- 172.21.15.84:0/441604250 shutdown_connections 2024-04-26T18:35:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.143+0000 7f78b607e640 1 -- 172.21.15.84:0/441604250 wait complete. 2024-04-26T18:35:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.144+0000 7f78b607e640 1 Processor -- start 2024-04-26T18:35:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.144+0000 7f78b607e640 1 -- start start 2024-04-26T18:35:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.144+0000 7f78b607e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78b0103b80 0x7f78b01990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.144+0000 7f78b607e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b01044a0 0x7f78b01995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.144+0000 7f78b607e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b010cf50 0x7f78b019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.144+0000 7f78b607e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f78b0112c10 con 0x7f78b01044a0 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.144+0000 7f78b607e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78b0112a90 con 0x7f78b0103b80 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.144+0000 7f78b607e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78b0112d90 con 0x7f78b010cf50 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.144+0000 7f78aeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b01044a0 0x7f78b01995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.144+0000 7f78aeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b01044a0 0x7f78b01995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39092/0 (socket says 172.21.15.84:39092) 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.144+0000 7f78aeffd640 1 -- 172.21.15.84:0/2933133576 learned_addr learned my addr 172.21.15.84:0/2933133576 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.144+0000 7f78af7fe640 1 --2- 172.21.15.84:0/2933133576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78b0103b80 0x7f78b01990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.145+0000 7f78aeffd640 1 -- 172.21.15.84:0/2933133576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b010cf50 msgr2=0x7f78b019d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.145+0000 7f78aeffd640 1 --2- 172.21.15.84:0/2933133576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b010cf50 0x7f78b019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.145+0000 7f78aeffd640 1 -- 172.21.15.84:0/2933133576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78b0103b80 msgr2=0x7f78b01990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.145+0000 7f78aeffd640 1 --2- 172.21.15.84:0/2933133576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78b0103b80 0x7f78b01990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.145+0000 7f78aeffd640 1 -- 172.21.15.84:0/2933133576 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78b019e050 con 0x7f78b01044a0 2024-04-26T18:35:40.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.145+0000 7f78aeffd640 1 --2- 172.21.15.84:0/2933133576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b01044a0 0x7f78b01995f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f78a400b4a0 tx=0x7f78a400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.145+0000 7f78acff9640 1 -- 172.21.15.84:0/2933133576 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4014070 con 0x7f78b01044a0 2024-04-26T18:35:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.145+0000 7f78acff9640 1 -- 172.21.15.84:0/2933133576 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a4010ac0 con 0x7f78b01044a0 2024-04-26T18:35:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.145+0000 7f78b607e640 1 -- 172.21.15.84:0/2933133576 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78b019e340 con 0x7f78b01044a0 2024-04-26T18:35:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.145+0000 7f78acff9640 1 -- 172.21.15.84:0/2933133576 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4017c30 con 0x7f78b01044a0 2024-04-26T18:35:40.148 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.145+0000 7f78b607e640 1 -- 172.21.15.84:0/2933133576 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78b01a5bd0 con 0x7f78b01044a0 2024-04-26T18:35:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.148+0000 7f78acff9640 1 -- 172.21.15.84:0/2933133576 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f78a4017070 con 0x7f78b01044a0 2024-04-26T18:35:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.148+0000 7f78b8d97740 1 Processor -- start 2024-04-26T18:35:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.148+0000 7f78b8d97740 1 -- start start 2024-04-26T18:35:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.148+0000 7f78b8d97740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc5cf00 0x55e18fc5d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.148+0000 7f78b8d97740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 0x55e18fc67ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.148+0000 7f78b8d97740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc680e0 0x55e18fc6a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.148+0000 7f78b8d97740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e18fbc3460 con 0x55e18fc5d820 2024-04-26T18:35:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.148+0000 7f78b8d97740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e18fb7d590 con 0x55e18fc680e0 2024-04-26T18:35:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.148+0000 7f78b8d97740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e18fbbf420 con 0x55e18fc5cf00 2024-04-26T18:35:40.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.148+0000 7f78af7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 0x55e18fc67ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.148+0000 7f78af7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 0x55e18fc67ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39106/0 (socket says 172.21.15.84:39106) 2024-04-26T18:35:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.148+0000 7f78af7fe640 1 -- 172.21.15.84:0/3994594100 learned_addr learned my addr 172.21.15.84:0/3994594100 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f78acff9640 1 --2- 172.21.15.84:0/2933133576 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f78980a3dc0 0x7f78980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f78aeffd640 1 --2- 172.21.15.84:0/3994594100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc680e0 0x55e18fc6a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f78affff640 1 --2- 172.21.15.84:0/3994594100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc5cf00 0x55e18fc5d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f78affff640 1 --2- 172.21.15.84:0/2933133576 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f78980a3dc0 0x7f78980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f78af7fe640 1 -- 172.21.15.84:0/3994594100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc5cf00 msgr2=0x55e18fc5d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f78af7fe640 1 --2- 172.21.15.84:0/3994594100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc5cf00 0x55e18fc5d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f78af7fe640 1 -- 172.21.15.84:0/3994594100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc680e0 msgr2=0x55e18fc6a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f78af7fe640 1 --2- 172.21.15.84:0/3994594100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc680e0 0x55e18fc6a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f78af7fe640 1 -- 172.21.15.84:0/3994594100 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e18fb8a0d0 con 0x55e18fc5d820 2024-04-26T18:35:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f78aeffd640 1 --2- 172.21.15.84:0/3994594100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc680e0 0x55e18fc6a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:35:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f78affff640 1 --2- 172.21.15.84:0/3994594100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc5cf00 0x55e18fc5d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:35:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f78acff9640 1 -- 172.21.15.84:0/2933133576 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f78a40cfac0 con 0x7f78b01044a0 2024-04-26T18:35:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f78af7fe640 1 --2- 172.21.15.84:0/3994594100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 0x55e18fc67ba0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f78a000bd10 tx=0x7f78a00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=136d917baf1e95c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f7895ffb640 1 -- 172.21.15.84:0/3994594100 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0004030 con 0x55e18fc5d820 2024-04-26T18:35:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f7895ffb640 1 -- 172.21.15.84:0/3994594100 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a00035f0 con 0x55e18fc5d820 2024-04-26T18:35:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.149+0000 7f7895ffb640 1 -- 172.21.15.84:0/3994594100 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0011640 con 0x55e18fc5d820 2024-04-26T18:35:40.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.150+0000 7f78b8d97740 1 -- 172.21.15.84:0/3994594100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 msgr2=0x55e18fc67ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.150+0000 7f78affff640 1 --2- 172.21.15.84:0/2933133576 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f78980a3dc0 0x7f78980a6280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f78a802ae70 tx=0x7f78a80095c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.150+0000 7f78b8d97740 1 --2- 172.21.15.84:0/3994594100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 0x55e18fc67ba0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f78a000bd10 tx=0x7f78a00107a0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.150+0000 7f78b8d97740 1 -- 172.21.15.84:0/3994594100 shutdown_connections 2024-04-26T18:35:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.150+0000 7f78b8d97740 1 --2- 172.21.15.84:0/3994594100 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc680e0 0x55e18fc6a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.150+0000 7f78b8d97740 1 --2- 172.21.15.84:0/3994594100 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 0x55e18fc67ba0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.150+0000 7f78b8d97740 1 --2- 172.21.15.84:0/3994594100 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc5cf00 0x55e18fc5d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.150+0000 7f78b8d97740 1 -- 172.21.15.84:0/3994594100 >> 172.21.15.84:0/3994594100 conn(0x55e18fc59410 msgr2=0x55e18fc597f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.150+0000 7f78b8d97740 1 -- 172.21.15.84:0/3994594100 shutdown_connections 2024-04-26T18:35:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.150+0000 7f78b8d97740 1 -- 172.21.15.84:0/3994594100 wait complete. 2024-04-26T18:35:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.150+0000 7f78b8d97740 1 -- start start 2024-04-26T18:35:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78b8d97740 1 Processor -- start 2024-04-26T18:35:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78b8d97740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5cf00 0x55e18fc65040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78b8d97740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5d820 0x55e18fc65580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78b8d97740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 0x55e18fc65ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78b8d97740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e18fc6ad10 con 0x55e18fc5cf00 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78b8d97740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e18fb4ab60 con 0x55e18fc5d820 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78b8d97740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e18fc6ae90 con 0x55e18fc680e0 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78b8d97740 10 client.0 _ll_register_callbacks cb 0x55e18fb6f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78af7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5cf00 0x55e18fc65040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78af7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5cf00 0x55e18fc65040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39112/0 (socket says 172.21.15.84:39112) 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78af7fe640 1 -- 172.21.15.84:0/3510704005 learned_addr learned my addr 172.21.15.84:0/3510704005 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78aeffd640 1 --2- 172.21.15.84:0/3510704005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5d820 0x55e18fc65580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78affff640 1 --2- 172.21.15.84:0/3510704005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 0x55e18fc65ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78af7fe640 1 -- 172.21.15.84:0/3510704005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 msgr2=0x55e18fc65ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78af7fe640 1 --2- 172.21.15.84:0/3510704005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 0x55e18fc65ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78af7fe640 1 -- 172.21.15.84:0/3510704005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5d820 msgr2=0x55e18fc65580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:40.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78af7fe640 1 --2- 172.21.15.84:0/3510704005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5d820 0x55e18fc65580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78af7fe640 1 -- 172.21.15.84:0/3510704005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e18f8d5210 con 0x55e18fc5cf00 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f78af7fe640 1 --2- 172.21.15.84:0/3510704005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5cf00 0x55e18fc65040 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f78a0011010 tx=0x7f78a00037a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f0164a1ad2638c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.151+0000 7f788effd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f788effd640 1 -- 172.21.15.84:0/3510704005 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a0029060 con 0x55e18fc5cf00 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f788effd640 1 -- 172.21.15.84:0/3510704005 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f78a0004000 con 0x55e18fc5cf00 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f78b8d97740 1 -- 172.21.15.84:0/3510704005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78a00035f0 con 0x55e18fc5cf00 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f788effd640 1 -- 172.21.15.84:0/3510704005 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f78a0003b20 con 0x55e18fc5cf00 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f78b8d97740 10 client.16341 fetch_fsmap learned FSMap version 29 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f78b8d97740 1 -- 172.21.15.84:0/3510704005 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e18f9a1620 con 0x55e18fc5cf00 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f788effd640 1 -- 172.21.15.84:0/3510704005 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a00035f0 con 0x55e18fc5cf00 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f788effd640 1 -- 172.21.15.84:0/3510704005 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f78a00048c0 con 0x55e18fc5cf00 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f78b8d97740 10 client.16341 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f78b8d97740 10 client.16341 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f78b8d97740 20 client.16341 populate_metadata read hostname 'smithi084' 2024-04-26T18:35:40.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f78b8d97740 1 --2- 172.21.15.84:0/3510704005 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e18fc780d0 0x55e18fc7a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f78b8d97740 4 client.16341 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f78b8d97740 1 -- 172.21.15.84:0/3510704005 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e18fbd1070 con 0x55e18fc780d0 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.152+0000 7f78aeffd640 1 --2- 172.21.15.84:0/3510704005 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e18fc780d0 0x55e18fc7a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.153+0000 7f788effd640 10 client.16341 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.153+0000 7f78aeffd640 1 --2- 172.21.15.84:0/3510704005 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e18fc780d0 0x55e18fc7a4c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e47115c7e6daa89 server_cookie=aba3a5095ac44e7f in_seq=0 out_seq=0 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.153+0000 7f788effd640 1 -- 172.21.15.84:0/3510704005 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e18fbd1070 con 0x55e18fc780d0 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.153+0000 7f788effd640 10 client.16341 handle_command_reply: tid=0 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.153+0000 7f78b8d97740 1 client.16341 shutdown 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.154+0000 7f78b8d97740 1 -- 172.21.15.84:0/3510704005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5cf00 msgr2=0x55e18fc65040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.154+0000 7f78b8d97740 1 --2- 172.21.15.84:0/3510704005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5cf00 0x55e18fc65040 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f78a0011010 tx=0x7f78a00037a0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.154+0000 7f78b8d97740 1 -- 172.21.15.84:0/3510704005 shutdown_connections 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.154+0000 7f78b8d97740 1 --2- 172.21.15.84:0/3510704005 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e18fc780d0 0x55e18fc7a4c0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.154+0000 7f78b8d97740 1 --2- 172.21.15.84:0/3510704005 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 0x55e18fc65ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.154+0000 7f78b8d97740 1 --2- 172.21.15.84:0/3510704005 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5d820 0x55e18fc65580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.154+0000 7f78b8d97740 1 --2- 172.21.15.84:0/3510704005 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5cf00 0x55e18fc65040 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.154+0000 7f78b8d97740 1 -- 172.21.15.84:0/3510704005 >> 172.21.15.84:0/3510704005 conn(0x55e18fc59410 msgr2=0x55e18fc585b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.154+0000 7f788effd640 0 client.16341 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.154+0000 7f78b8d97740 1 -- 172.21.15.84:0/3510704005 shutdown_connections 2024-04-26T18:35:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.154+0000 7f78b8d97740 1 -- 172.21.15.84:0/3510704005 wait complete. 2024-04-26T18:35:40.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.154+0000 7f78b8d97740 20 client.16341 trim_cache size 0 max 16384 2024-04-26T18:35:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.168+0000 7f78b8d97740 1 Processor -- start 2024-04-26T18:35:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.168+0000 7f78b8d97740 1 -- start start 2024-04-26T18:35:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78b8d97740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5cf00 0x55e18fc73bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78b8d97740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 0x55e18fc7fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78b8d97740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 0x55e18fc80000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78b8d97740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01ae680 con 0x55e18fc5d820 2024-04-26T18:35:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78b8d97740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01b01c0 con 0x55e18fc5cf00 2024-04-26T18:35:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78b8d97740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01aed50 con 0x55e18fc680e0 2024-04-26T18:35:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78aeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 0x55e18fc7fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78aeffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 0x55e18fc7fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39122/0 (socket says 172.21.15.84:39122) 2024-04-26T18:35:40.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78aeffd640 1 -- 172.21.15.84:0/1208179844 learned_addr learned my addr 172.21.15.84:0/1208179844 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78af7fe640 1 --2- 172.21.15.84:0/1208179844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5cf00 0x55e18fc73bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78aeffd640 1 -- 172.21.15.84:0/1208179844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 msgr2=0x55e18fc80000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78aeffd640 1 --2- 172.21.15.84:0/1208179844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 0x55e18fc80000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78aeffd640 1 -- 172.21.15.84:0/1208179844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5cf00 msgr2=0x55e18fc73bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78aeffd640 1 --2- 172.21.15.84:0/1208179844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5cf00 0x55e18fc73bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78aeffd640 1 -- 172.21.15.84:0/1208179844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e18fc54460 con 0x55e18fc5d820 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78af7fe640 1 --2- 172.21.15.84:0/1208179844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5cf00 0x55e18fc73bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78aeffd640 1 --2- 172.21.15.84:0/1208179844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 0x55e18fc7fac0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f78a4004700 tx=0x7f78a40b0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=839d122c6bb61743 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78957fa640 1 -- 172.21.15.84:0/1208179844 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a409ce60 con 0x55e18fc5d820 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78957fa640 1 -- 172.21.15.84:0/1208179844 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a40a47a0 con 0x55e18fc5d820 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.169+0000 7f78957fa640 1 -- 172.21.15.84:0/1208179844 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a401e040 con 0x55e18fc5d820 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.170+0000 7f78b8d97740 1 -- 172.21.15.84:0/1208179844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 msgr2=0x55e18fc7fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.170+0000 7f78b8d97740 1 --2- 172.21.15.84:0/1208179844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 0x55e18fc7fac0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f78a4004700 tx=0x7f78a40b0040 comp rx=0 tx=0).stop 2024-04-26T18:35:40.175 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.170+0000 7f78b8d97740 1 -- 172.21.15.84:0/1208179844 shutdown_connections 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.170+0000 7f78b8d97740 1 --2- 172.21.15.84:0/1208179844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 0x55e18fc80000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.170+0000 7f78b8d97740 1 --2- 172.21.15.84:0/1208179844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc5d820 0x55e18fc7fac0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.170+0000 7f78b8d97740 1 --2- 172.21.15.84:0/1208179844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5cf00 0x55e18fc73bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.170+0000 7f78b8d97740 1 -- 172.21.15.84:0/1208179844 >> 172.21.15.84:0/1208179844 conn(0x55e18fc59410 msgr2=0x55e18fc60320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.170+0000 7f78b8d97740 1 -- 172.21.15.84:0/1208179844 shutdown_connections 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.170+0000 7f78b8d97740 1 -- 172.21.15.84:0/1208179844 wait complete. 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.170+0000 7f78b8d97740 1 -- start start 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78b8d97740 1 Processor -- start 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78b8d97740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5cf00 0x55e18fc70500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78b8d97740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 0x55e18fc6b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78b8d97740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc6bd10 0x55e18fc6c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78b8d97740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01afaf0 con 0x55e18fc6bd10 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78b8d97740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01af420 con 0x55e18fc5cf00 2024-04-26T18:35:40.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78b8d97740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f78b01ac680 con 0x55e18fc680e0 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc6bd10 0x55e18fc6c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78affff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc6bd10 0x55e18fc6c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39130/0 (socket says 172.21.15.84:39130) 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78affff640 1 -- 172.21.15.84:0/1539081082 learned_addr learned my addr 172.21.15.84:0/1539081082 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78b8d97740 10 client.0 _ll_register_callbacks cb 0x7f78a000bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78aeffd640 1 --2- 172.21.15.84:0/1539081082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 0x55e18fc6b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78af7fe640 1 --2- 172.21.15.84:0/1539081082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5cf00 0x55e18fc70500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78affff640 1 -- 172.21.15.84:0/1539081082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 msgr2=0x55e18fc6b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78affff640 1 --2- 172.21.15.84:0/1539081082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 0x55e18fc6b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78affff640 1 -- 172.21.15.84:0/1539081082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5cf00 msgr2=0x55e18fc70500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78affff640 1 --2- 172.21.15.84:0/1539081082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5cf00 0x55e18fc70500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78affff640 1 -- 172.21.15.84:0/1539081082 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e18fce5cf0 con 0x55e18fc6bd10 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f78affff640 1 --2- 172.21.15.84:0/1539081082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc6bd10 0x55e18fc6c190 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f78a802f650 tx=0x7f78a802f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31719098e7d2cd3e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.171+0000 7f788effd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:35:40.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f788effd640 1 -- 172.21.15.84:0/1539081082 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a8034040 con 0x55e18fc6bd10 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f788effd640 1 -- 172.21.15.84:0/1539081082 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f78a8003d80 con 0x55e18fc6bd10 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f78b8d97740 1 -- 172.21.15.84:0/1539081082 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78a40a47a0 con 0x55e18fc6bd10 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f788effd640 1 -- 172.21.15.84:0/1539081082 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f78a8030940 con 0x55e18fc6bd10 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f78b8d97740 10 client.16347 fetch_fsmap learned FSMap version 29 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f78b8d97740 1 -- 172.21.15.84:0/1539081082 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e18fce6ef0 con 0x55e18fc6bd10 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f788effd640 1 -- 172.21.15.84:0/1539081082 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a40a47a0 con 0x55e18fc6bd10 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f788effd640 1 -- 172.21.15.84:0/1539081082 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f78a805a470 con 0x55e18fc6bd10 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f78b8d97740 10 client.16347 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f78b8d97740 10 client.16347 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f78b8d97740 20 client.16347 populate_metadata read hostname 'smithi084' 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f78b8d97740 1 --2- 172.21.15.84:0/1539081082 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e18fce76a0 0x55e18fce9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f78b8d97740 4 client.16347 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f78b8d97740 1 -- 172.21.15.84:0/1539081082 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e18fbcaee0 con 0x55e18fce76a0 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.172+0000 7f78af7fe640 1 --2- 172.21.15.84:0/1539081082 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e18fce76a0 0x55e18fce9ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:40.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.173+0000 7f78af7fe640 1 --2- 172.21.15.84:0/1539081082 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e18fce76a0 0x55e18fce9ac0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec3b7d62ba5f6a6c server_cookie=7ef381f5cde63d93 in_seq=0 out_seq=0 2024-04-26T18:35:40.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.173+0000 7f788effd640 10 client.16347 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.186+0000 7f788effd640 1 -- 172.21.15.84:0/1539081082 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e18fbcaee0 con 0x55e18fce76a0 2024-04-26T18:35:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.186+0000 7f788effd640 10 client.16347 handle_command_reply: tid=0 2024-04-26T18:35:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.186+0000 7f78b8d97740 1 client.16347 shutdown 2024-04-26T18:35:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.186+0000 7f78b8d97740 1 -- 172.21.15.84:0/1539081082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc6bd10 msgr2=0x55e18fc6c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.186+0000 7f78b8d97740 1 --2- 172.21.15.84:0/1539081082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc6bd10 0x55e18fc6c190 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f78a802f650 tx=0x7f78a802f680 comp rx=0 tx=0).stop 2024-04-26T18:35:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.186+0000 7f78b8d97740 1 -- 172.21.15.84:0/1539081082 shutdown_connections 2024-04-26T18:35:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.186+0000 7f78b8d97740 1 --2- 172.21.15.84:0/1539081082 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e18fce76a0 0x55e18fce9ac0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.186+0000 7f78b8d97740 1 --2- 172.21.15.84:0/1539081082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e18fc6bd10 0x55e18fc6c190 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.187 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.186+0000 7f78b8d97740 1 --2- 172.21.15.84:0/1539081082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e18fc680e0 0x55e18fc6b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.186+0000 7f788effd640 0 client.16347 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.186+0000 7f78b8d97740 1 --2- 172.21.15.84:0/1539081082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e18fc5cf00 0x55e18fc70500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.186+0000 7f78b8d97740 1 -- 172.21.15.84:0/1539081082 >> 172.21.15.84:0/1539081082 conn(0x55e18fc59410 msgr2=0x55e18fc64fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b8d97740 1 -- 172.21.15.84:0/1539081082 shutdown_connections 2024-04-26T18:35:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b8d97740 1 -- 172.21.15.84:0/1539081082 wait complete. 2024-04-26T18:35:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b8d97740 20 client.16347 trim_cache size 0 max 16384 2024-04-26T18:35:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:35:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:35:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:35:40.188 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:35:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b607e640 1 -- 172.21.15.84:0/2933133576 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f78980a3dc0 msgr2=0x7f78980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b607e640 1 --2- 172.21.15.84:0/2933133576 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f78980a3dc0 0x7f78980a6280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f78a802ae70 tx=0x7f78a80095c0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b607e640 1 -- 172.21.15.84:0/2933133576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b01044a0 msgr2=0x7f78b01995f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b607e640 1 --2- 172.21.15.84:0/2933133576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b01044a0 0x7f78b01995f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f78a400b4a0 tx=0x7f78a400b960 comp rx=0 tx=0).stop 2024-04-26T18:35:40.189 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b607e640 1 -- 172.21.15.84:0/2933133576 shutdown_connections 2024-04-26T18:35:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b607e640 1 --2- 172.21.15.84:0/2933133576 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f78980a3dc0 0x7f78980a6280 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b607e640 1 --2- 172.21.15.84:0/2933133576 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f78b010cf50 0x7f78b019d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b607e640 1 --2- 172.21.15.84:0/2933133576 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f78b01044a0 0x7f78b01995f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b607e640 1 --2- 172.21.15.84:0/2933133576 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f78b0103b80 0x7f78b01990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b607e640 1 -- 172.21.15.84:0/2933133576 >> 172.21.15.84:0/2933133576 conn(0x7f78b0078a40 msgr2=0x7f78b00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b607e640 1 -- 172.21.15.84:0/2933133576 shutdown_connections 2024-04-26T18:35:40.190 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:40.187+0000 7f78b607e640 1 -- 172.21.15.84:0/2933133576 wait complete. 2024-04-26T18:35:41.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:41 smithi174 ceph-mon[32705]: pgmap v238: 97 pgs: 97 active+clean; 4.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-26T18:35:41.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:41 smithi084 ceph-mon[29141]: pgmap v238: 97 pgs: 97 active+clean; 4.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-26T18:35:41.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:41 smithi103 ceph-mon[32727]: pgmap v238: 97 pgs: 97 active+clean; 4.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-26T18:35:43.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:43 smithi174 ceph-mon[32705]: pgmap v239: 97 pgs: 97 active+clean; 4.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-26T18:35:43.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:43 smithi084 ceph-mon[29141]: pgmap v239: 97 pgs: 97 active+clean; 4.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-26T18:35:43.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:43 smithi103 ceph-mon[32727]: pgmap v239: 97 pgs: 97 active+clean; 4.2 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-26T18:35:45.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:45 smithi174 ceph-mon[32705]: pgmap v240: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-26T18:35:45.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:45 smithi084 ceph-mon[29141]: pgmap v240: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-26T18:35:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:45 smithi103 ceph-mon[32727]: pgmap v240: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-26T18:35:47.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:47 smithi174 ceph-mon[32705]: pgmap v241: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:35:47.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:47 smithi084 ceph-mon[29141]: pgmap v241: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:35:47.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:47 smithi103 ceph-mon[32727]: pgmap v241: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:35:49.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:49 smithi103 ceph-mon[32727]: pgmap v242: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:35:49.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:49 smithi174 ceph-mon[32705]: pgmap v242: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:35:49.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:49 smithi084 ceph-mon[29141]: pgmap v242: 97 pgs: 97 active+clean; 4.4 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:35:51.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:51 smithi174 ceph-mon[32705]: pgmap v243: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-26T18:35:51.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:51 smithi084 ceph-mon[29141]: pgmap v243: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-26T18:35:51.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:51 smithi103 ceph-mon[32727]: pgmap v243: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-26T18:35:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.281+0000 7f8439d5f640 1 -- 172.21.15.84:0/1665183897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f843410f290 msgr2=0x7f8434111680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.281+0000 7f8439d5f640 1 --2- 172.21.15.84:0/1665183897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f843410f290 0x7f8434111680 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f842c009ca0 tx=0x7f842c02a8c0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.281+0000 7f8439d5f640 1 -- 172.21.15.84:0/1665183897 shutdown_connections 2024-04-26T18:35:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.281+0000 7f8439d5f640 1 --2- 172.21.15.84:0/1665183897 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f843410f290 0x7f8434111680 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.281+0000 7f8439d5f640 1 --2- 172.21.15.84:0/1665183897 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8434102090 0x7f843410ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.281+0000 7f8439d5f640 1 --2- 172.21.15.84:0/1665183897 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f84341016e0 0x7f8434101ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.281+0000 7f8439d5f640 1 -- 172.21.15.84:0/1665183897 >> 172.21.15.84:0/1665183897 conn(0x7f84340fd300 msgr2=0x7f84340ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:53.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.281+0000 7f8439d5f640 1 -- 172.21.15.84:0/1665183897 shutdown_connections 2024-04-26T18:35:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.281+0000 7f8439d5f640 1 -- 172.21.15.84:0/1665183897 wait complete. 2024-04-26T18:35:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.282+0000 7f8439d5f640 1 Processor -- start 2024-04-26T18:35:53.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.282+0000 7f8439d5f640 1 -- start start 2024-04-26T18:35:53.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.282+0000 7f8439d5f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84341016e0 0x7f843419b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.282+0000 7f8439d5f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8434102090 0x7f843419b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.282+0000 7f8439d5f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f843410f290 0x7f843419fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.282+0000 7f8439d5f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8434114de0 con 0x7f843410f290 2024-04-26T18:35:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.282+0000 7f8439d5f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8434114c60 con 0x7f8434102090 2024-04-26T18:35:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.282+0000 7f8439d5f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8434114f60 con 0x7f84341016e0 2024-04-26T18:35:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8433fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f843410f290 0x7f843419fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8433fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f843410f290 0x7f843419fb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59938/0 (socket says 172.21.15.84:59938) 2024-04-26T18:35:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8433fff640 1 -- 172.21.15.84:0/3736299560 learned_addr learned my addr 172.21.15.84:0/3736299560 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:53.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8432ffd640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8434102090 0x7f843419b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f84337fe640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84341016e0 0x7f843419b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8433fff640 1 -- 172.21.15.84:0/3736299560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84341016e0 msgr2=0x7f843419b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8433fff640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84341016e0 0x7f843419b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8433fff640 1 -- 172.21.15.84:0/3736299560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8434102090 msgr2=0x7f843419b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8433fff640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8434102090 0x7f843419b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8433fff640 1 -- 172.21.15.84:0/3736299560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84341a0260 con 0x7f843410f290 2024-04-26T18:35:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8433fff640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f843410f290 0x7f843419fb30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f842c02ade0 tx=0x7f842c0319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8430ff9640 1 -- 172.21.15.84:0/3736299560 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c003bf0 con 0x7f843410f290 2024-04-26T18:35:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8430ff9640 1 -- 172.21.15.84:0/3736299560 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c003d90 con 0x7f843410f290 2024-04-26T18:35:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8439d5f640 1 -- 172.21.15.84:0/3736299560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84341a04f0 con 0x7f843410f290 2024-04-26T18:35:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8430ff9640 1 -- 172.21.15.84:0/3736299560 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c02fbd0 con 0x7f843410f290 2024-04-26T18:35:53.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.283+0000 7f8439d5f640 1 -- 172.21.15.84:0/3736299560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84341a7de0 con 0x7f843410f290 2024-04-26T18:35:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.285+0000 7f8430ff9640 1 -- 172.21.15.84:0/3736299560 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f842c02fd70 con 0x7f843410f290 2024-04-26T18:35:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f843ca79740 1 Processor -- start 2024-04-26T18:35:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f843ca79740 1 -- start start 2024-04-26T18:35:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f8430ff9640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f841c0a3dc0 0x7f841c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f843ca79740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26d9f00 0x55a0e26da2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f843ca79740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 0x55a0e26e4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f843ca79740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26e50e0 0x55a0e26e74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f843ca79740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a0e2640460 con 0x55a0e26da820 2024-04-26T18:35:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f843ca79740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a0e25fa590 con 0x55a0e26e50e0 2024-04-26T18:35:53.294 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f843ca79740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a0e263c420 con 0x55a0e26d9f00 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f8432ffd640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f841c0a3dc0 0x7f841c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f8430ff9640 1 -- 172.21.15.84:0/3736299560 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f842c0f1c50 con 0x7f843410f290 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f8432ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 0x55a0e26e4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f8432ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 0x55a0e26e4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59950/0 (socket says 172.21.15.84:59950) 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f8432ffd640 1 -- 172.21.15.84:0/3273826270 learned_addr learned my addr 172.21.15.84:0/3273826270 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f84337fe640 1 --2- 172.21.15.84:0/3273826270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26d9f00 0x55a0e26da2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.286+0000 7f8433fff640 1 --2- 172.21.15.84:0/3273826270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26e50e0 0x55a0e26e74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.287+0000 7f84337fe640 1 -- 172.21.15.84:0/3273826270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26e50e0 msgr2=0x55a0e26e74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.287+0000 7f84337fe640 1 --2- 172.21.15.84:0/3273826270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26e50e0 0x55a0e26e74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.287+0000 7f84337fe640 1 -- 172.21.15.84:0/3273826270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 msgr2=0x55a0e26e4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.287+0000 7f84337fe640 1 --2- 172.21.15.84:0/3273826270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 0x55a0e26e4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.287+0000 7f84337fe640 1 -- 172.21.15.84:0/3273826270 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0e26070d0 con 0x55a0e26d9f00 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.287+0000 7f8432ffd640 1 --2- 172.21.15.84:0/3273826270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 0x55a0e26e4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.287+0000 7f8432ffd640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f841c0a3dc0 0x7f841c0a6280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8428005d20 tx=0x7f842800bb10 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:53.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.287+0000 7f84337fe640 1 --2- 172.21.15.84:0/3273826270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26d9f00 0x55a0e26da2e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f842400bbf0 tx=0x7f84240107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5a326b9fdfe2ddd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.287+0000 7f8419ffb640 1 -- 172.21.15.84:0/3273826270 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424004030 con 0x55a0e26d9f00 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.287+0000 7f8419ffb640 1 -- 172.21.15.84:0/3273826270 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84240035f0 con 0x55a0e26d9f00 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.287+0000 7f8419ffb640 1 -- 172.21.15.84:0/3273826270 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8424011680 con 0x55a0e26d9f00 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.288+0000 7f843ca79740 1 -- 172.21.15.84:0/3273826270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26d9f00 msgr2=0x55a0e26da2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.288+0000 7f843ca79740 1 --2- 172.21.15.84:0/3273826270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26d9f00 0x55a0e26da2e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f842400bbf0 tx=0x7f84240107a0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.288+0000 7f843ca79740 1 -- 172.21.15.84:0/3273826270 shutdown_connections 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.288+0000 7f843ca79740 1 --2- 172.21.15.84:0/3273826270 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26e50e0 0x55a0e26e74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.288+0000 7f843ca79740 1 --2- 172.21.15.84:0/3273826270 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 0x55a0e26e4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.288+0000 7f843ca79740 1 --2- 172.21.15.84:0/3273826270 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26d9f00 0x55a0e26da2e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.288+0000 7f843ca79740 1 -- 172.21.15.84:0/3273826270 >> 172.21.15.84:0/3273826270 conn(0x55a0e26d6410 msgr2=0x55a0e26d67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.288+0000 7f843ca79740 1 -- 172.21.15.84:0/3273826270 shutdown_connections 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.288+0000 7f843ca79740 1 -- 172.21.15.84:0/3273826270 wait complete. 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.288+0000 7f843ca79740 1 -- start start 2024-04-26T18:35:53.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f843ca79740 1 Processor -- start 2024-04-26T18:35:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f843ca79740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26d9f00 0x55a0e26e1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f843ca79740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26da820 0x55a0e26e2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f843ca79740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26e50e0 0x55a0e26e29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f843ca79740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a0e26e7d10 con 0x55a0e26d9f00 2024-04-26T18:35:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f843ca79740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a0e25c7b60 con 0x55a0e26e50e0 2024-04-26T18:35:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f843ca79740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a0e26e7e90 con 0x55a0e26da820 2024-04-26T18:35:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f843ca79740 10 client.0 _ll_register_callbacks cb 0x55a0e25ec2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f84337fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26d9f00 0x55a0e26e1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f84337fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26d9f00 0x55a0e26e1f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59962/0 (socket says 172.21.15.84:59962) 2024-04-26T18:35:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f84337fe640 1 -- 172.21.15.84:0/3144407177 learned_addr learned my addr 172.21.15.84:0/3144407177 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f8433fff640 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26e50e0 0x55a0e26e29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f8432ffd640 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26da820 0x55a0e26e2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f84337fe640 1 -- 172.21.15.84:0/3144407177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26da820 msgr2=0x55a0e26e2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f84337fe640 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26da820 0x55a0e26e2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f84337fe640 1 -- 172.21.15.84:0/3144407177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26e50e0 msgr2=0x55a0e26e29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f84337fe640 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26e50e0 0x55a0e26e29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f84337fe640 1 -- 172.21.15.84:0/3144407177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0e2352210 con 0x55a0e26d9f00 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f8433fff640 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26e50e0 0x55a0e26e29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f84337fe640 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26d9f00 0x55a0e26e1f50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f842400bfd0 tx=0x7f842400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6bb308e7bf8b380 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.289+0000 7f8412ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f8412ffd640 1 -- 172.21.15.84:0/3144407177 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842402a060 con 0x55a0e26d9f00 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f843ca79740 1 -- 172.21.15.84:0/3144407177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84240035f0 con 0x55a0e26d9f00 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f8412ffd640 1 -- 172.21.15.84:0/3144407177 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f8424003f40 con 0x55a0e26d9f00 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f8412ffd640 1 -- 172.21.15.84:0/3144407177 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8424003b30 con 0x55a0e26d9f00 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f843ca79740 10 client.16359 fetch_fsmap learned FSMap version 29 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f843ca79740 1 -- 172.21.15.84:0/3144407177 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a0e241e620 con 0x55a0e26d9f00 2024-04-26T18:35:53.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f8412ffd640 1 -- 172.21.15.84:0/3144407177 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84240035f0 con 0x55a0e26d9f00 2024-04-26T18:35:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f8412ffd640 1 -- 172.21.15.84:0/3144407177 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f84240047c0 con 0x55a0e26d9f00 2024-04-26T18:35:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f843ca79740 10 client.16359 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:35:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f843ca79740 10 client.16359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f843ca79740 20 client.16359 populate_metadata read hostname 'smithi084' 2024-04-26T18:35:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f843ca79740 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a0e26f5060 0x55a0e26f7450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f843ca79740 4 client.16359 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:35:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f843ca79740 1 -- 172.21.15.84:0/3144407177 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a0e264e070 con 0x55a0e26f5060 2024-04-26T18:35:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.290+0000 7f8432ffd640 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a0e26f5060 0x55a0e26f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.291+0000 7f8412ffd640 10 client.16359 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.291+0000 7f8432ffd640 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a0e26f5060 0x55a0e26f7450 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d6ffc90909df73c server_cookie=f04c6eaa92bd0568 in_seq=0 out_seq=0 2024-04-26T18:35:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.291+0000 7f8412ffd640 1 -- 172.21.15.84:0/3144407177 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a0e264e070 con 0x55a0e26f5060 2024-04-26T18:35:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f8412ffd640 10 client.16359 handle_command_reply: tid=0 2024-04-26T18:35:53.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f843ca79740 1 client.16359 shutdown 2024-04-26T18:35:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f843ca79740 1 -- 172.21.15.84:0/3144407177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26d9f00 msgr2=0x55a0e26e1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f843ca79740 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26d9f00 0x55a0e26e1f50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f842400bfd0 tx=0x7f842400b8d0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f843ca79740 1 -- 172.21.15.84:0/3144407177 shutdown_connections 2024-04-26T18:35:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f843ca79740 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a0e26f5060 0x55a0e26f7450 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f843ca79740 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26e50e0 0x55a0e26e29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f843ca79740 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26da820 0x55a0e26e2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f843ca79740 1 --2- 172.21.15.84:0/3144407177 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26d9f00 0x55a0e26e1f50 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f8412ffd640 0 client.16359 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f843ca79740 1 -- 172.21.15.84:0/3144407177 >> 172.21.15.84:0/3144407177 conn(0x55a0e26d6410 msgr2=0x55a0e26dc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f843ca79740 1 -- 172.21.15.84:0/3144407177 shutdown_connections 2024-04-26T18:35:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f843ca79740 1 -- 172.21.15.84:0/3144407177 wait complete. 2024-04-26T18:35:53.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.292+0000 7f843ca79740 20 client.16359 trim_cache size 0 max 16384 2024-04-26T18:35:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f843ca79740 1 Processor -- start 2024-04-26T18:35:53.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f843ca79740 1 -- start start 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f843ca79740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26d9f00 0x55a0e26e1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f843ca79740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26da820 0x55a0e26fca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f843ca79740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26e50e0 0x55a0e26fcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f843ca79740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f84341b0780 con 0x55a0e26e50e0 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f843ca79740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84341b22c0 con 0x55a0e26d9f00 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f843ca79740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84341b0e50 con 0x55a0e26da820 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f8433fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26e50e0 0x55a0e26fcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f8433fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26e50e0 0x55a0e26fcfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59978/0 (socket says 172.21.15.84:59978) 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f8433fff640 1 -- 172.21.15.84:0/3769649934 learned_addr learned my addr 172.21.15.84:0/3769649934 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f84337fe640 1 --2- 172.21.15.84:0/3769649934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26d9f00 0x55a0e26e1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f8433fff640 1 -- 172.21.15.84:0/3769649934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26da820 msgr2=0x55a0e26fca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f8433fff640 1 --2- 172.21.15.84:0/3769649934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26da820 0x55a0e26fca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f8433fff640 1 -- 172.21.15.84:0/3769649934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26d9f00 msgr2=0x55a0e26e1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:53.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f8433fff640 1 --2- 172.21.15.84:0/3769649934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26d9f00 0x55a0e26e1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f8433fff640 1 -- 172.21.15.84:0/3769649934 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a0e26da650 con 0x55a0e26e50e0 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f84337fe640 1 --2- 172.21.15.84:0/3769649934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26d9f00 0x55a0e26e1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f8433fff640 1 --2- 172.21.15.84:0/3769649934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26e50e0 0x55a0e26fcfc0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f842c005cc0 tx=0x7f842c005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f17dc7732ed1cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f84197fa640 1 -- 172.21.15.84:0/3769649934 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c0bd620 con 0x55a0e26e50e0 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.307+0000 7f84197fa640 1 -- 172.21.15.84:0/3769649934 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c036040 con 0x55a0e26e50e0 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.308+0000 7f84197fa640 1 -- 172.21.15.84:0/3769649934 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c041bc0 con 0x55a0e26e50e0 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.308+0000 7f843ca79740 1 -- 172.21.15.84:0/3769649934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26e50e0 msgr2=0x55a0e26fcfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.308+0000 7f843ca79740 1 --2- 172.21.15.84:0/3769649934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26e50e0 0x55a0e26fcfc0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f842c005cc0 tx=0x7f842c005c30 comp rx=0 tx=0).stop 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.308+0000 7f843ca79740 1 -- 172.21.15.84:0/3769649934 shutdown_connections 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.308+0000 7f843ca79740 1 --2- 172.21.15.84:0/3769649934 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26e50e0 0x55a0e26fcfc0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.308+0000 7f843ca79740 1 --2- 172.21.15.84:0/3769649934 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26da820 0x55a0e26fca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.308+0000 7f843ca79740 1 --2- 172.21.15.84:0/3769649934 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26d9f00 0x55a0e26e1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.308+0000 7f843ca79740 1 -- 172.21.15.84:0/3769649934 >> 172.21.15.84:0/3769649934 conn(0x55a0e26d6410 msgr2=0x55a0e26d55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:53.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.308+0000 7f843ca79740 1 -- 172.21.15.84:0/3769649934 shutdown_connections 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.308+0000 7f843ca79740 1 -- 172.21.15.84:0/3769649934 wait complete. 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.308+0000 7f843ca79740 1 -- start start 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f843ca79740 1 Processor -- start 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f843ca79740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26d9f00 0x55a0e26ed380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f843ca79740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 0x55a0e26de860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f843ca79740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26e50e0 0x55a0e26deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f843ca79740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f84341b1bf0 con 0x55a0e26da820 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f843ca79740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f84341b1520 con 0x55a0e26d9f00 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f843ca79740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f84341ae780 con 0x55a0e26e50e0 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f843ca79740 10 client.0 _ll_register_callbacks cb 0x7f8428012470 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f8432ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 0x55a0e26de860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f8432ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 0x55a0e26de860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:59988/0 (socket says 172.21.15.84:59988) 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f8432ffd640 1 -- 172.21.15.84:0/573075579 learned_addr learned my addr 172.21.15.84:0/573075579 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:35:53.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f84337fe640 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26d9f00 0x55a0e26ed380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f8433fff640 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26e50e0 0x55a0e26deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f8432ffd640 1 -- 172.21.15.84:0/573075579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26e50e0 msgr2=0x55a0e26deda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f8432ffd640 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26e50e0 0x55a0e26deda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f8432ffd640 1 -- 172.21.15.84:0/573075579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26d9f00 msgr2=0x55a0e26ed380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f8432ffd640 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26d9f00 0x55a0e26ed380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f8432ffd640 1 -- 172.21.15.84:0/573075579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a0e26df440 con 0x55a0e26da820 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f8433fff640 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26e50e0 0x55a0e26deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.309+0000 7f84337fe640 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26d9f00 0x55a0e26ed380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f8432ffd640 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 0x55a0e26de860 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x55a0e26db4a0 tx=0x7f84280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494ce7b18acc1f98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f8412ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f8412ffd640 1 -- 172.21.15.84:0/573075579 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842802abd0 con 0x55a0e26da820 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f8412ffd640 1 -- 172.21.15.84:0/573075579 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 8579+0+0 (secure 0 0 0) 0x7f842802ad70 con 0x55a0e26da820 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f843ca79740 1 -- 172.21.15.84:0/573075579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f842c036040 con 0x55a0e26da820 2024-04-26T18:35:53.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f8412ffd640 1 -- 172.21.15.84:0/573075579 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f84280207d0 con 0x55a0e26da820 2024-04-26T18:35:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f843ca79740 10 client.16365 fetch_fsmap learned FSMap version 29 2024-04-26T18:35:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f843ca79740 1 -- 172.21.15.84:0/573075579 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a0e275b150 con 0x55a0e26da820 2024-04-26T18:35:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f8412ffd640 1 -- 172.21.15.84:0/573075579 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c036040 con 0x55a0e26da820 2024-04-26T18:35:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f8412ffd640 1 -- 172.21.15.84:0/573075579 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f842801f070 con 0x55a0e26da820 2024-04-26T18:35:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f843ca79740 10 client.16365 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:35:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f843ca79740 10 client.16365 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:35:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f843ca79740 20 client.16365 populate_metadata read hostname 'smithi084' 2024-04-26T18:35:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f843ca79740 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a0e275b900 0x55a0e275dd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:35:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f843ca79740 4 client.16365 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:35:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.310+0000 7f843ca79740 1 -- 172.21.15.84:0/573075579 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a0e2647ee0 con 0x55a0e275b900 2024-04-26T18:35:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.311+0000 7f84337fe640 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a0e275b900 0x55a0e275dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:35:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.311+0000 7f84337fe640 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a0e275b900 0x55a0e275dd20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41731914511bc583 server_cookie=e08a37f2a298fc0e in_seq=0 out_seq=0 2024-04-26T18:35:53.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.311+0000 7f8412ffd640 10 client.16365 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f8412ffd640 1 -- 172.21.15.84:0/573075579 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a0e2647ee0 con 0x55a0e275b900 2024-04-26T18:35:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f8412ffd640 10 client.16365 handle_command_reply: tid=0 2024-04-26T18:35:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f843ca79740 1 client.16365 shutdown 2024-04-26T18:35:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f843ca79740 1 -- 172.21.15.84:0/573075579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 msgr2=0x55a0e26de860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f843ca79740 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 0x55a0e26de860 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x55a0e26db4a0 tx=0x7f84280046c0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f843ca79740 1 -- 172.21.15.84:0/573075579 shutdown_connections 2024-04-26T18:35:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f843ca79740 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a0e275b900 0x55a0e275dd20 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f843ca79740 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a0e26e50e0 0x55a0e26deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f843ca79740 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a0e26da820 0x55a0e26de860 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f843ca79740 1 --2- 172.21.15.84:0/573075579 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a0e26d9f00 0x55a0e26ed380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f8412ffd640 0 client.16365 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:35:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f843ca79740 1 -- 172.21.15.84:0/573075579 >> 172.21.15.84:0/573075579 conn(0x55a0e26d6410 msgr2=0x55a0e26db8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:53.328 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f843ca79740 1 -- 172.21.15.84:0/573075579 shutdown_connections 2024-04-26T18:35:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f843ca79740 1 -- 172.21.15.84:0/573075579 wait complete. 2024-04-26T18:35:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.326+0000 7f843ca79740 20 client.16365 trim_cache size 0 max 16384 2024-04-26T18:35:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:35:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:35:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:35:53.329 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:35:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.327+0000 7f8439d5f640 1 -- 172.21.15.84:0/3736299560 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f841c0a3dc0 msgr2=0x7f841c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.327+0000 7f8439d5f640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f841c0a3dc0 0x7f841c0a6280 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8428005d20 tx=0x7f842800bb10 comp rx=0 tx=0).stop 2024-04-26T18:35:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.327+0000 7f8439d5f640 1 -- 172.21.15.84:0/3736299560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f843410f290 msgr2=0x7f843419fb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:35:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.327+0000 7f8439d5f640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f843410f290 0x7f843419fb30 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f842c02ade0 tx=0x7f842c0319f0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.327+0000 7f8439d5f640 1 -- 172.21.15.84:0/3736299560 shutdown_connections 2024-04-26T18:35:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.327+0000 7f8439d5f640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f841c0a3dc0 0x7f841c0a6280 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.330 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.327+0000 7f8439d5f640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f843410f290 0x7f843419fb30 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.327+0000 7f8439d5f640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8434102090 0x7f843419b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.327+0000 7f8439d5f640 1 --2- 172.21.15.84:0/3736299560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f84341016e0 0x7f843419b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:35:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.327+0000 7f8439d5f640 1 -- 172.21.15.84:0/3736299560 >> 172.21.15.84:0/3736299560 conn(0x7f84340fd300 msgr2=0x7f84340ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:35:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.327+0000 7f8439d5f640 1 -- 172.21.15.84:0/3736299560 shutdown_connections 2024-04-26T18:35:53.331 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:35:53.327+0000 7f8439d5f640 1 -- 172.21.15.84:0/3736299560 wait complete. 2024-04-26T18:35:53.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:53 smithi174 ceph-mon[32705]: pgmap v244: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:35:53.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:53 smithi084 ceph-mon[29141]: pgmap v244: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:35:53.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:53 smithi103 ceph-mon[32727]: pgmap v244: 97 pgs: 97 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:35:55.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:55 smithi174 ceph-mon[32705]: pgmap v245: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-26T18:35:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:55 smithi084 ceph-mon[29141]: pgmap v245: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-26T18:35:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:55 smithi103 ceph-mon[32727]: pgmap v245: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-26T18:35:57.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:57 smithi084 ceph-mon[29141]: pgmap v246: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:35:57.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:57 smithi103 ceph-mon[32727]: pgmap v246: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:35:57.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:57 smithi174 ceph-mon[32705]: pgmap v246: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:35:59.772 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.772 INFO:tasks.workunit.client.0.smithi084.stdout:189/600: open fd = 3 2024-04-26T18:35:59.772 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.772 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:190/600: open fd = 3 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:191/600: open fd = 3 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:192/600: open fd = 3 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:193/600: open fd = 3 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.773 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:194/600: open fd = 3 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:195/600: open fd = 3 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:196/600: open fd = 3 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:197/600: open fd = 3 2024-04-26T18:35:59.774 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:198/600: open fd = 3 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:199/600: open fd = 3 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:200/600: open fd = 3 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.775 INFO:tasks.workunit.client.0.smithi084.stdout:201/600: open fd = 3 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:202/600: open fd = 3 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:203/600: open fd = 3 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:204/600: open fd = 3 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.776 INFO:tasks.workunit.client.0.smithi084.stdout:205/600: open fd = 3 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:206/600: open fd = 3 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:207/600: open fd = 3 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:208/600: open fd = 3 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.777 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:209/600: open fd = 3 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:210/600: open fd = 3 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:211/600: open fd = 3 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:212/600: open fd = 3 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.778 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:213/600: open fd = 3 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:214/600: open fd = 3 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:215/600: open fd = 3 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:216/600: open fd = 3 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.779 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:217/600: open fd = 3 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:218/600: open fd = 3 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:219/600: open fd = 3 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:220/600: open fd = 3 2024-04-26T18:35:59.780 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:221/600: open fd = 3 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:222/600: open fd = 3 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:223/600: open fd = 3 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:224/600: open fd = 3 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.781 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:225/600: open fd = 3 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:226/600: open fd = 3 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:227/600: open fd = 3 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:228/600: open fd = 3 2024-04-26T18:35:59.782 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:229/600: open fd = 3 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:230/600: open fd = 3 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:231/600: open fd = 3 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:232/600: open fd = 3 2024-04-26T18:35:59.783 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:233/600: open fd = 3 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:234/600: open fd = 3 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:235/600: open fd = 3 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.784 INFO:tasks.workunit.client.0.smithi084.stdout:236/600: open fd = 3 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:237/600: open fd = 3 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:238/600: open fd = 3 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:239/600: open fd = 3 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.785 INFO:tasks.workunit.client.0.smithi084.stdout:240/600: open fd = 3 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:241/600: open fd = 3 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:242/600: open fd = 3 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:243/600: open fd = 3 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.786 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:244/600: open fd = 3 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:245/600: open fd = 3 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:246/600: open fd = 3 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:247/600: open fd = 3 2024-04-26T18:35:59.787 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.788 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.788 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.788 INFO:tasks.workunit.client.0.smithi084.stdout:248/600: open fd = 3 2024-04-26T18:35:59.788 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.788 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.788 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.788 INFO:tasks.workunit.client.0.smithi084.stdout:249/600: open fd = 3 2024-04-26T18:35:59.788 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.788 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.788 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:35:59.788 INFO:tasks.workunit.client.0.smithi084.stdout:250/600: open fd = 3 2024-04-26T18:35:59.788 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.788 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:35:59.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:35:59 smithi084 ceph-mon[29141]: pgmap v247: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:35:59.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:35:59 smithi103 ceph-mon[32727]: pgmap v247: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:35:59.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:35:59 smithi174 ceph-mon[32705]: pgmap v247: 97 pgs: 97 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:36:01.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:01 smithi084 ceph-mon[29141]: pgmap v248: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-26T18:36:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:01 smithi103 ceph-mon[32727]: pgmap v248: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-26T18:36:01.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:01 smithi174 ceph-mon[32705]: pgmap v248: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-26T18:36:03.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:03 smithi084 ceph-mon[29141]: pgmap v249: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:36:03.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:03 smithi103 ceph-mon[32727]: pgmap v249: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:36:03.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:03 smithi174 ceph-mon[32705]: pgmap v249: 97 pgs: 97 active+clean; 5.0 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:36:05.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:05 smithi084 ceph-mon[29141]: pgmap v250: 97 pgs: 97 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-26T18:36:05.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:05 smithi084 ceph-mon[29141]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T18:36:05.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:05 smithi103 ceph-mon[32727]: pgmap v250: 97 pgs: 97 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-26T18:36:05.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:05 smithi103 ceph-mon[32727]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T18:36:05.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:05 smithi174 ceph-mon[32705]: pgmap v250: 97 pgs: 97 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-26T18:36:05.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:05 smithi174 ceph-mon[32705]: osdmap e99: 12 total, 12 up, 12 in 2024-04-26T18:36:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.424+0000 7fce01f0a640 1 -- 172.21.15.84:0/2213220835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdfc101880 msgr2=0x7fcdfc101d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.425+0000 7fce01f0a640 1 --2- 172.21.15.84:0/2213220835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdfc101880 0x7fcdfc101d00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009930 tx=0x7fcdf002a420 comp rx=0 tx=0).stop 2024-04-26T18:36:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.425+0000 7fce01f0a640 1 -- 172.21.15.84:0/2213220835 shutdown_connections 2024-04-26T18:36:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.425+0000 7fce01f0a640 1 --2- 172.21.15.84:0/2213220835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdfc102240 0x7fcdfc10e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.427 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.425+0000 7fce01f0a640 1 --2- 172.21.15.84:0/2213220835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdfc101880 0x7fcdfc101d00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.425+0000 7fce01f0a640 1 --2- 172.21.15.84:0/2213220835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdfc107840 0x7fcdfc107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.425+0000 7fce01f0a640 1 -- 172.21.15.84:0/2213220835 >> 172.21.15.84:0/2213220835 conn(0x7fcdfc0789b0 msgr2=0x7fcdfc078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.425+0000 7fce01f0a640 1 -- 172.21.15.84:0/2213220835 shutdown_connections 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.425+0000 7fce01f0a640 1 -- 172.21.15.84:0/2213220835 wait complete. 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.425+0000 7fce01f0a640 1 Processor -- start 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.425+0000 7fce01f0a640 1 -- start start 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fce01f0a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdfc101880 0x7fcdfc199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fce01f0a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdfc102240 0x7fcdfc1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fce01f0a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdfc107840 0x7fcdfc19da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fce01f0a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc111b00 con 0x7fcdfc102240 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fce01f0a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc111980 con 0x7fcdfc101880 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fce01f0a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc111c80 con 0x7fcdfc107840 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fcdfaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdfc102240 0x7fcdfc1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fcdfaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdfc102240 0x7fcdfc1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39826/0 (socket says 172.21.15.84:39826) 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fcdfaffd640 1 -- 172.21.15.84:0/3929247007 learned_addr learned my addr 172.21.15.84:0/3929247007 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fcdfbfff640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdfc107840 0x7fcdfc19da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fcdfb7fe640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdfc101880 0x7fcdfc199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fcdfaffd640 1 -- 172.21.15.84:0/3929247007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdfc107840 msgr2=0x7fcdfc19da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fcdfaffd640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdfc107840 0x7fcdfc19da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fcdfaffd640 1 -- 172.21.15.84:0/3929247007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdfc101880 msgr2=0x7fcdfc199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fcdfaffd640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdfc101880 0x7fcdfc199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fcdfaffd640 1 -- 172.21.15.84:0/3929247007 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdfc19e160 con 0x7fcdfc102240 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fcdfb7fe640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdfc101880 0x7fcdfc199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fcdfaffd640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdfc102240 0x7fcdfc1996b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005940 tx=0x7fcdf0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.426+0000 7fcdf8ff9640 1 -- 172.21.15.84:0/3929247007 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf003f030 con 0x7fcdfc102240 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.427+0000 7fcdf8ff9640 1 -- 172.21.15.84:0/3929247007 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf0003aa0 con 0x7fcdfc102240 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.427+0000 7fcdf8ff9640 1 -- 172.21.15.84:0/3929247007 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0035070 con 0x7fcdfc102240 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.427+0000 7fce01f0a640 1 -- 172.21.15.84:0/3929247007 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdfc19e3f0 con 0x7fcdfc102240 2024-04-26T18:36:06.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.427+0000 7fce01f0a640 1 -- 172.21.15.84:0/3929247007 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdfc1a5ce0 con 0x7fcdfc102240 2024-04-26T18:36:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.429+0000 7fcdf8ff9640 1 -- 172.21.15.84:0/3929247007 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fcdf0042030 con 0x7fcdfc102240 2024-04-26T18:36:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.429+0000 7fce04c23740 1 Processor -- start 2024-04-26T18:36:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.429+0000 7fce04c23740 1 -- start start 2024-04-26T18:36:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.429+0000 7fcdf8ff9640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcde40a3e40 0x7fcde40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.429+0000 7fce04c23740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbef00 0x56221abbf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.429+0000 7fce04c23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbf820 0x56221abc9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.429+0000 7fce04c23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abca0e0 0x56221abcc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.429+0000 7fce04c23740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56221ab25460 con 0x56221abbef00 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.429+0000 7fce04c23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56221aadf590 con 0x56221abbf820 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.429+0000 7fce04c23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56221ab21420 con 0x56221abca0e0 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdfbfff640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcde40a3e40 0x7fcde40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdfb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbef00 0x56221abbf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdf8ff9640 1 -- 172.21.15.84:0/3929247007 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8623+0+0 (secure 0 0 0) 0x7fcdf00f0f90 con 0x7fcdfc102240 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdfb7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbef00 0x56221abbf2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39830/0 (socket says 172.21.15.84:39830) 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdfb7fe640 1 -- 172.21.15.84:0/2282137193 learned_addr learned my addr 172.21.15.84:0/2282137193 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdfbfff640 1 --2- 172.21.15.84:0/2282137193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abca0e0 0x56221abcc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdfaffd640 1 --2- 172.21.15.84:0/2282137193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbf820 0x56221abc9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdfb7fe640 1 -- 172.21.15.84:0/2282137193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abca0e0 msgr2=0x56221abcc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdfb7fe640 1 --2- 172.21.15.84:0/2282137193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abca0e0 0x56221abcc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdfb7fe640 1 -- 172.21.15.84:0/2282137193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbf820 msgr2=0x56221abc9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdfb7fe640 1 --2- 172.21.15.84:0/2282137193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbf820 0x56221abc9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdfb7fe640 1 -- 172.21.15.84:0/2282137193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56221aaec0d0 con 0x56221abbef00 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdfbfff640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcde40a3e40 0x7fcde40a6300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4013750 tx=0x7fcdf4013b50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcdfb7fe640 1 --2- 172.21.15.84:0/2282137193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbef00 0x56221abbf2e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00a990 tx=0x7fcdec00ae50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3e7301d2eac98 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcde1ffb640 1 -- 172.21.15.84:0/2282137193 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec015070 con 0x56221abbef00 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcde1ffb640 1 -- 172.21.15.84:0/2282137193 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdec00cbe0 con 0x56221abbef00 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.430+0000 7fcde1ffb640 1 -- 172.21.15.84:0/2282137193 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdec0105b0 con 0x56221abbef00 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.431+0000 7fce04c23740 1 -- 172.21.15.84:0/2282137193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbef00 msgr2=0x56221abbf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.431+0000 7fce04c23740 1 --2- 172.21.15.84:0/2282137193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbef00 0x56221abbf2e0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fcdec00a990 tx=0x7fcdec00ae50 comp rx=0 tx=0).stop 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.431+0000 7fce04c23740 1 -- 172.21.15.84:0/2282137193 shutdown_connections 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.431+0000 7fce04c23740 1 --2- 172.21.15.84:0/2282137193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abca0e0 0x56221abcc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.431+0000 7fce04c23740 1 --2- 172.21.15.84:0/2282137193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbf820 0x56221abc9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.431+0000 7fce04c23740 1 --2- 172.21.15.84:0/2282137193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbef00 0x56221abbf2e0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.431+0000 7fce04c23740 1 -- 172.21.15.84:0/2282137193 >> 172.21.15.84:0/2282137193 conn(0x56221abbb410 msgr2=0x56221abbb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.431+0000 7fce04c23740 1 -- 172.21.15.84:0/2282137193 shutdown_connections 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.431+0000 7fce04c23740 1 -- 172.21.15.84:0/2282137193 wait complete. 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.431+0000 7fce04c23740 1 -- start start 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fce04c23740 1 Processor -- start 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fce04c23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abbef00 0x56221abc7020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fce04c23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbf820 0x56221abc7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fce04c23740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 0x56221abc7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fce04c23740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56221abccd10 con 0x56221abca0e0 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fce04c23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56221aaacb60 con 0x56221abbf820 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fce04c23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56221abcce90 con 0x56221abbef00 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fce04c23740 10 client.0 _ll_register_callbacks cb 0x56221aad12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fcdfbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 0x56221abc7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fcdfbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 0x56221abc7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39838/0 (socket says 172.21.15.84:39838) 2024-04-26T18:36:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fcdfbfff640 1 -- 172.21.15.84:0/1102881085 learned_addr learned my addr 172.21.15.84:0/1102881085 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fcdfaffd640 1 --2- 172.21.15.84:0/1102881085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbf820 0x56221abc7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fcdfb7fe640 1 --2- 172.21.15.84:0/1102881085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abbef00 0x56221abc7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fcdfbfff640 1 -- 172.21.15.84:0/1102881085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abbef00 msgr2=0x56221abc7020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fcdfbfff640 1 --2- 172.21.15.84:0/1102881085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abbef00 0x56221abc7020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.432+0000 7fcdfbfff640 1 -- 172.21.15.84:0/1102881085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbf820 msgr2=0x56221abc7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fcdfbfff640 1 --2- 172.21.15.84:0/1102881085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbf820 0x56221abc7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fcdfbfff640 1 -- 172.21.15.84:0/1102881085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56221a837210 con 0x56221abca0e0 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fcdfbfff640 1 --2- 172.21.15.84:0/1102881085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 0x56221abc7aa0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4011dc0 tx=0x7fcdf4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415eb95a5e3bebaa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fcddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fcddaffd640 1 -- 172.21.15.84:0/1102881085 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4024070 con 0x56221abca0e0 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fcddaffd640 1 -- 172.21.15.84:0/1102881085 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8623+0+0 (secure 0 0 0) 0x7fcdf4020620 con 0x56221abca0e0 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fce04c23740 1 -- 172.21.15.84:0/1102881085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdec00cbe0 con 0x56221abca0e0 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fcddaffd640 1 -- 172.21.15.84:0/1102881085 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcdf4030050 con 0x56221abca0e0 2024-04-26T18:36:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fce04c23740 10 client.16377 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fce04c23740 1 -- 172.21.15.84:0/1102881085 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56221a903620 con 0x56221abca0e0 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fcddaffd640 1 -- 172.21.15.84:0/1102881085 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdec00cbe0 con 0x56221abca0e0 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fcddaffd640 1 -- 172.21.15.84:0/1102881085 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcdf400e790 con 0x56221abca0e0 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fce04c23740 10 client.16377 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fce04c23740 10 client.16377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fce04c23740 20 client.16377 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fce04c23740 1 --2- 172.21.15.84:0/1102881085 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56221abda1d0 0x56221abdc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fce04c23740 4 client.16377 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.433+0000 7fce04c23740 1 -- 172.21.15.84:0/1102881085 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56221ab33070 con 0x56221abda1d0 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.434+0000 7fcdfb7fe640 1 --2- 172.21.15.84:0/1102881085 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56221abda1d0 0x56221abdc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.434+0000 7fcdfb7fe640 1 --2- 172.21.15.84:0/1102881085 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56221abda1d0 0x56221abdc5c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f9123e905565709 server_cookie=9eb5f000fa6a51a1 in_seq=0 out_seq=0 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.434+0000 7fcddaffd640 10 client.16377 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fcddaffd640 1 -- 172.21.15.84:0/1102881085 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56221ab33070 con 0x56221abda1d0 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fcddaffd640 10 client.16377 handle_command_reply: tid=0 2024-04-26T18:36:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fce04c23740 1 client.16377 shutdown 2024-04-26T18:36:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fce04c23740 1 -- 172.21.15.84:0/1102881085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 msgr2=0x56221abc7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fce04c23740 1 --2- 172.21.15.84:0/1102881085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 0x56221abc7aa0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4011dc0 tx=0x7fcdf4004ab0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fce04c23740 1 -- 172.21.15.84:0/1102881085 shutdown_connections 2024-04-26T18:36:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fce04c23740 1 --2- 172.21.15.84:0/1102881085 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56221abda1d0 0x56221abdc5c0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fce04c23740 1 --2- 172.21.15.84:0/1102881085 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 0x56221abc7aa0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fce04c23740 1 --2- 172.21.15.84:0/1102881085 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbf820 0x56221abc7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fcddaffd640 0 client.16377 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fce04c23740 1 --2- 172.21.15.84:0/1102881085 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abbef00 0x56221abc7020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fce04c23740 1 -- 172.21.15.84:0/1102881085 >> 172.21.15.84:0/1102881085 conn(0x56221abbb410 msgr2=0x56221abc1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fce04c23740 1 -- 172.21.15.84:0/1102881085 shutdown_connections 2024-04-26T18:36:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fce04c23740 1 -- 172.21.15.84:0/1102881085 wait complete. 2024-04-26T18:36:06.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.435+0000 7fce04c23740 20 client.16377 trim_cache size 0 max 16384 2024-04-26T18:36:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.450+0000 7fce04c23740 1 Processor -- start 2024-04-26T18:36:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.450+0000 7fce04c23740 1 -- start start 2024-04-26T18:36:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.450+0000 7fce04c23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbef00 0x56221abd5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.450+0000 7fce04c23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abbf820 0x56221abd24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.450+0000 7fce04c23740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 0x56221abc18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.450+0000 7fce04c23740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1ae5b0 con 0x56221abca0e0 2024-04-26T18:36:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.450+0000 7fce04c23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1b00f0 con 0x56221abbef00 2024-04-26T18:36:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.450+0000 7fce04c23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1aec80 con 0x56221abbf820 2024-04-26T18:36:06.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.450+0000 7fcdfbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 0x56221abc18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.450+0000 7fcdfbfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 0x56221abc18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39848/0 (socket says 172.21.15.84:39848) 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.450+0000 7fcdfbfff640 1 -- 172.21.15.84:0/660172690 learned_addr learned my addr 172.21.15.84:0/660172690 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.451+0000 7fcdfaffd640 1 --2- 172.21.15.84:0/660172690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abbf820 0x56221abd24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.451+0000 7fcdfb7fe640 1 --2- 172.21.15.84:0/660172690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbef00 0x56221abd5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.451+0000 7fcdfbfff640 1 -- 172.21.15.84:0/660172690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abbf820 msgr2=0x56221abd24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.451+0000 7fcdfbfff640 1 --2- 172.21.15.84:0/660172690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abbf820 0x56221abd24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.451+0000 7fcdfbfff640 1 -- 172.21.15.84:0/660172690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbef00 msgr2=0x56221abd5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.451+0000 7fcdfbfff640 1 --2- 172.21.15.84:0/660172690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbef00 0x56221abd5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.451+0000 7fcdfbfff640 1 -- 172.21.15.84:0/660172690 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56221abdc930 con 0x56221abca0e0 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.451+0000 7fcdfbfff640 1 --2- 172.21.15.84:0/660172690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 0x56221abc18c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4017010 tx=0x7fcdf4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=246d048ed0d8e55a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.451+0000 7fcde17fa640 1 -- 172.21.15.84:0/660172690 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4024040 con 0x56221abca0e0 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.451+0000 7fcde17fa640 1 -- 172.21.15.84:0/660172690 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf4039070 con 0x56221abca0e0 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.451+0000 7fcde17fa640 1 -- 172.21.15.84:0/660172690 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf4029820 con 0x56221abca0e0 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.452+0000 7fce04c23740 1 -- 172.21.15.84:0/660172690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 msgr2=0x56221abc18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:06.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.452+0000 7fce04c23740 1 --2- 172.21.15.84:0/660172690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 0x56221abc18c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4017010 tx=0x7fcdf4004a40 comp rx=0 tx=0).stop 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.452+0000 7fce04c23740 1 -- 172.21.15.84:0/660172690 shutdown_connections 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.452+0000 7fce04c23740 1 --2- 172.21.15.84:0/660172690 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abca0e0 0x56221abc18c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.452+0000 7fce04c23740 1 --2- 172.21.15.84:0/660172690 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abbf820 0x56221abd24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.452+0000 7fce04c23740 1 --2- 172.21.15.84:0/660172690 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbef00 0x56221abd5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.452+0000 7fce04c23740 1 -- 172.21.15.84:0/660172690 >> 172.21.15.84:0/660172690 conn(0x56221abbb410 msgr2=0x56221abc7870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.452+0000 7fce04c23740 1 -- 172.21.15.84:0/660172690 shutdown_connections 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.452+0000 7fce04c23740 1 -- 172.21.15.84:0/660172690 wait complete. 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.452+0000 7fce04c23740 1 -- start start 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fce04c23740 1 Processor -- start 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fce04c23740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbef00 0x56221abc32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fce04c23740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbf820 0x56221abc3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fce04c23740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abca0e0 0x56221abc3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fce04c23740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1afa20 con 0x56221abbf820 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fce04c23740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1af350 con 0x56221abbef00 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fce04c23740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc1ac5b0 con 0x56221abca0e0 2024-04-26T18:36:06.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fce04c23740 10 client.0 _ll_register_callbacks cb 0x7fcdf4013030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcdfaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbf820 0x56221abc3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcdfaffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbf820 0x56221abc3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39862/0 (socket says 172.21.15.84:39862) 2024-04-26T18:36:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcdfaffd640 1 -- 172.21.15.84:0/2859082193 learned_addr learned my addr 172.21.15.84:0/2859082193 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcdfbfff640 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abca0e0 0x56221abc3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcdfb7fe640 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbef00 0x56221abc32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcdfaffd640 1 -- 172.21.15.84:0/2859082193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abca0e0 msgr2=0x56221abc3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcdfaffd640 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abca0e0 0x56221abc3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcdfaffd640 1 -- 172.21.15.84:0/2859082193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbef00 msgr2=0x56221abc32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:06.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcdfaffd640 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbef00 0x56221abc32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcdfaffd640 1 -- 172.21.15.84:0/2859082193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56221abc4750 con 0x56221abbf820 2024-04-26T18:36:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcdfb7fe640 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbef00 0x56221abc32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:36:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcdfaffd640 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbf820 0x56221abc3810 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005ec0 tx=0x7fcdf00bca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f1e1270624d632 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcddaffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.453+0000 7fcddaffd640 1 -- 172.21.15.84:0/2859082193 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0049040 con 0x56221abbf820 2024-04-26T18:36:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fce04c23740 1 -- 172.21.15.84:0/2859082193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcdf4039070 con 0x56221abbf820 2024-04-26T18:36:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fcddaffd640 1 -- 172.21.15.84:0/2859082193 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 8623+0+0 (secure 0 0 0) 0x7fcdf0003c50 con 0x56221abbf820 2024-04-26T18:36:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fcddaffd640 1 -- 172.21.15.84:0/2859082193 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcdf009a310 con 0x56221abbf820 2024-04-26T18:36:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fce04c23740 10 client.16383 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fce04c23740 1 -- 172.21.15.84:0/2859082193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56221ac44e70 con 0x56221abbf820 2024-04-26T18:36:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fcddaffd640 1 -- 172.21.15.84:0/2859082193 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcdf4039070 con 0x56221abbf820 2024-04-26T18:36:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fcddaffd640 1 -- 172.21.15.84:0/2859082193 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fcdf00ee900 con 0x56221abbf820 2024-04-26T18:36:06.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fce04c23740 10 client.16383 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fce04c23740 10 client.16383 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:36:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fce04c23740 20 client.16383 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fce04c23740 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56221ac455f0 0x56221ac47a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fce04c23740 4 client.16383 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:36:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fce04c23740 1 -- 172.21.15.84:0/2859082193 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56221ab2cee0 con 0x56221ac455f0 2024-04-26T18:36:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.454+0000 7fcdfb7fe640 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56221ac455f0 0x56221ac47a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.455+0000 7fcddaffd640 10 client.16383 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:06.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.455+0000 7fcdfb7fe640 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56221ac455f0 0x56221ac47a10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51e586aca0448df7 server_cookie=809f2c3f21a956b0 in_seq=0 out_seq=0 2024-04-26T18:36:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fcddaffd640 1 -- 172.21.15.84:0/2859082193 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56221ab2cee0 con 0x56221ac455f0 2024-04-26T18:36:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fcddaffd640 10 client.16383 handle_command_reply: tid=0 2024-04-26T18:36:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fce04c23740 1 client.16383 shutdown 2024-04-26T18:36:06.470 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fce04c23740 1 -- 172.21.15.84:0/2859082193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbf820 msgr2=0x56221abc3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fce04c23740 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbf820 0x56221abc3810 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005ec0 tx=0x7fcdf00bca10 comp rx=0 tx=0).stop 2024-04-26T18:36:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fce04c23740 1 -- 172.21.15.84:0/2859082193 shutdown_connections 2024-04-26T18:36:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fce04c23740 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56221ac455f0 0x56221ac47a10 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fce04c23740 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56221abca0e0 0x56221abc3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fce04c23740 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56221abbf820 0x56221abc3810 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fcddaffd640 0 client.16383 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fce04c23740 1 --2- 172.21.15.84:0/2859082193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56221abbef00 0x56221abc32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fce04c23740 1 -- 172.21.15.84:0/2859082193 >> 172.21.15.84:0/2859082193 conn(0x56221abbb410 msgr2=0x56221abc8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fce04c23740 1 -- 172.21.15.84:0/2859082193 shutdown_connections 2024-04-26T18:36:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fce04c23740 1 -- 172.21.15.84:0/2859082193 wait complete. 2024-04-26T18:36:06.471 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.470+0000 7fce04c23740 20 client.16383 trim_cache size 0 max 16384 2024-04-26T18:36:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:36:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:36:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:36:06.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:36:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.471+0000 7fce01f0a640 1 -- 172.21.15.84:0/3929247007 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcde40a3e40 msgr2=0x7fcde40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.471+0000 7fce01f0a640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcde40a3e40 0x7fcde40a6300 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4013750 tx=0x7fcdf4013b50 comp rx=0 tx=0).stop 2024-04-26T18:36:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.471+0000 7fce01f0a640 1 -- 172.21.15.84:0/3929247007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdfc102240 msgr2=0x7fcdfc1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.471+0000 7fce01f0a640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdfc102240 0x7fcdfc1996b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0005940 tx=0x7fcdf0004270 comp rx=0 tx=0).stop 2024-04-26T18:36:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.471+0000 7fce01f0a640 1 -- 172.21.15.84:0/3929247007 shutdown_connections 2024-04-26T18:36:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.471+0000 7fce01f0a640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcde40a3e40 0x7fcde40a6300 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.471+0000 7fce01f0a640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcdfc107840 0x7fcdfc19da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.471+0000 7fce01f0a640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcdfc102240 0x7fcdfc1996b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.471+0000 7fce01f0a640 1 --2- 172.21.15.84:0/3929247007 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcdfc101880 0x7fcdfc199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.471+0000 7fce01f0a640 1 -- 172.21.15.84:0/3929247007 >> 172.21.15.84:0/3929247007 conn(0x7fcdfc0789b0 msgr2=0x7fcdfc06b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.471+0000 7fce01f0a640 1 -- 172.21.15.84:0/3929247007 shutdown_connections 2024-04-26T18:36:06.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:06.471+0000 7fce01f0a640 1 -- 172.21.15.84:0/3929247007 wait complete. 2024-04-26T18:36:07.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:07 smithi084 ceph-mon[29141]: pgmap v252: 97 pgs: 97 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 36 op/s 2024-04-26T18:36:07.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:07 smithi103 ceph-mon[32727]: pgmap v252: 97 pgs: 97 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 36 op/s 2024-04-26T18:36:07.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:07 smithi174 ceph-mon[32705]: pgmap v252: 97 pgs: 97 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 36 op/s 2024-04-26T18:36:08.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:08 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:36:08.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:08 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:36:08.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:08 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:36:09.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:09 smithi084 ceph-mon[29141]: pgmap v253: 97 pgs: 97 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 36 op/s 2024-04-26T18:36:09.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:09 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:36:09.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:09 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:36:09.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:09 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:36:09.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:09 smithi103 ceph-mon[32727]: pgmap v253: 97 pgs: 97 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 36 op/s 2024-04-26T18:36:09.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:09 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:36:09.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:09 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:36:09.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:09 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:36:09.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:09 smithi174 ceph-mon[32705]: pgmap v253: 97 pgs: 97 active+clean; 5.2 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 36 op/s 2024-04-26T18:36:09.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:09 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:36:09.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:09 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:36:09.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:09 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:36:09.986 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 29.7 2024-04-26T18:36:09.987 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:36:10.237 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:36:10.658 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:10 smithi084 ceph-mon[29141]: pgmap v254: 97 pgs: 97 active+clean; 1.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-26T18:36:10.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:10 smithi103 ceph-mon[32727]: pgmap v254: 97 pgs: 97 active+clean; 1.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-26T18:36:10.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:10 smithi174 ceph-mon[32705]: pgmap v254: 97 pgs: 97 active+clean; 1.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-26T18:36:12.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.111+0000 7f975ccda700 1 -- 172.21.15.84:0/702973640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9758102310 msgr2=0x7f9758102790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.111+0000 7f975ccda700 1 --2- 172.21.15.84:0/702973640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9758102310 0x7f9758102790 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f974c009990 tx=0x7f974c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:36:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.111+0000 7f975ccda700 1 -- 172.21.15.84:0/702973640 shutdown_connections 2024-04-26T18:36:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.111+0000 7f975ccda700 1 --2- 172.21.15.84:0/702973640 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9758102cd0 0x7f975810f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.111+0000 7f975ccda700 1 --2- 172.21.15.84:0/702973640 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9758108380 0x7f9758108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.111+0000 7f975ccda700 1 --2- 172.21.15.84:0/702973640 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9758102310 0x7f9758102790 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.111+0000 7f975ccda700 1 -- 172.21.15.84:0/702973640 >> 172.21.15.84:0/702973640 conn(0x7f9758079290 msgr2=0x7f97580796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.111+0000 7f975ccda700 1 -- 172.21.15.84:0/702973640 shutdown_connections 2024-04-26T18:36:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.111+0000 7f975ccda700 1 -- 172.21.15.84:0/702973640 wait complete. 2024-04-26T18:36:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.112+0000 7f975ccda700 1 Processor -- start 2024-04-26T18:36:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.112+0000 7f975ccda700 1 -- start start 2024-04-26T18:36:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.112+0000 7f975ccda700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9758102310 0x7f975819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.112+0000 7f975ccda700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9758102cd0 0x7f975819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:12.114 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.112+0000 7f975ccda700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9758108380 0x7f9758199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.112+0000 7f975ccda700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9758112550 con 0x7f9758102310 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.112+0000 7f975ccda700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97581123d0 con 0x7f9758102cd0 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.112+0000 7f975ccda700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97581126d0 con 0x7f9758108380 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.112+0000 7f975659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9758102310 0x7f975819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.112+0000 7f975659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9758102310 0x7f975819f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47254/0 (socket says 172.21.15.84:47254) 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.112+0000 7f975659c700 1 -- 172.21.15.84:0/2605291446 learned_addr learned my addr 172.21.15.84:0/2605291446 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f9756d9d700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9758108380 0x7f9758199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f9755d9b700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9758102cd0 0x7f975819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f975659c700 1 -- 172.21.15.84:0/2605291446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9758108380 msgr2=0x7f9758199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f975659c700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9758108380 0x7f9758199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f975659c700 1 -- 172.21.15.84:0/2605291446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9758102cd0 msgr2=0x7f975819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f975659c700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9758102cd0 0x7f975819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f975659c700 1 -- 172.21.15.84:0/2605291446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f975819a0f0 con 0x7f9758102310 2024-04-26T18:36:12.115 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f9755d9b700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9758102cd0 0x7f975819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:36:12.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f975659c700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9758102310 0x7f975819f7d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f974800b530 tx=0x7f974800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:12.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f97477fe700 1 -- 172.21.15.84:0/2605291446 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748015020 con 0x7f9758102310 2024-04-26T18:36:12.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f97477fe700 1 -- 172.21.15.84:0/2605291446 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9748011ce0 con 0x7f9758102310 2024-04-26T18:36:12.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f975ccda700 1 -- 172.21.15.84:0/2605291446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f975819a3e0 con 0x7f9758102310 2024-04-26T18:36:12.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f97477fe700 1 -- 172.21.15.84:0/2605291446 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9748010d20 con 0x7f9758102310 2024-04-26T18:36:12.116 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.113+0000 7f975ccda700 1 -- 172.21.15.84:0/2605291446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97581a66a0 con 0x7f9758102310 2024-04-26T18:36:12.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.114+0000 7f975ccda700 1 -- 172.21.15.84:0/2605291446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f975806a110 con 0x7f9758102310 2024-04-26T18:36:12.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.115+0000 7f97477fe700 1 -- 172.21.15.84:0/2605291446 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9748018020 con 0x7f9758102310 2024-04-26T18:36:12.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.116+0000 7f97477fe700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97400a3e30 0x7f97400a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:12.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.116+0000 7f9755d9b700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97400a3e30 0x7f97400a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:12.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.116+0000 7f97477fe700 1 -- 172.21.15.84:0/2605291446 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 8623+0+0 (secure 0 0 0) 0x7f97480cfef0 con 0x7f9758102310 2024-04-26T18:36:12.117 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.116+0000 7f9755d9b700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97400a3e30 0x7f97400a62f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f974c00b5c0 tx=0x7f974c005fb0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:12.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.120+0000 7f97477fe700 1 -- 172.21.15.84:0/2605291446 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f974801f020 con 0x7f9758102310 2024-04-26T18:36:12.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.369+0000 7f975ccda700 1 -- 172.21.15.84:0/2605291446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97581a6930 con 0x7f9758102310 2024-04-26T18:36:12.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.370+0000 7f97477fe700 1 -- 172.21.15.84:0/2605291446 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f974809c640 con 0x7f9758102310 2024-04-26T18:36:12.370 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:36:12.371 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:36:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.372+0000 7f975ccda700 1 -- 172.21.15.84:0/2605291446 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97400a3e30 msgr2=0x7f97400a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.372+0000 7f975ccda700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97400a3e30 0x7f97400a62f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f974c00b5c0 tx=0x7f974c005fb0 comp rx=0 tx=0).stop 2024-04-26T18:36:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.372+0000 7f975ccda700 1 -- 172.21.15.84:0/2605291446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9758102310 msgr2=0x7f975819f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.372+0000 7f975ccda700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9758102310 0x7f975819f7d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f974800b530 tx=0x7f974800b8f0 comp rx=0 tx=0).stop 2024-04-26T18:36:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.372+0000 7f975ccda700 1 -- 172.21.15.84:0/2605291446 shutdown_connections 2024-04-26T18:36:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.372+0000 7f975ccda700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97400a3e30 0x7f97400a62f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.372+0000 7f975ccda700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9758102310 0x7f975819f7d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:12.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.372+0000 7f975ccda700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9758108380 0x7f9758199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:12.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.372+0000 7f975ccda700 1 --2- 172.21.15.84:0/2605291446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9758102cd0 0x7f975819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:12.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.372+0000 7f975ccda700 1 -- 172.21.15.84:0/2605291446 >> 172.21.15.84:0/2605291446 conn(0x7f9758079290 msgr2=0x7f9758101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:12.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.372+0000 7f975ccda700 1 -- 172.21.15.84:0/2605291446 shutdown_connections 2024-04-26T18:36:12.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:12.372+0000 7f975ccda700 1 -- 172.21.15.84:0/2605291446 wait complete. 2024-04-26T18:36:12.373 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:36:12.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:12 smithi174 ceph-mon[32705]: pgmap v255: 97 pgs: 97 active+clean; 1.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-26T18:36:12.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:12 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2605291446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:36:12.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:12 smithi174 ceph-mon[32705]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T18:36:13.073 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:12 smithi084 ceph-mon[29141]: pgmap v255: 97 pgs: 97 active+clean; 1.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-26T18:36:13.074 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:12 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2605291446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:36:13.074 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:12 smithi084 ceph-mon[29141]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T18:36:13.101 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 60 --expiration 89.7 --await-for 42.0' 2024-04-26T18:36:13.101 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db / --timeout 60 --expiration 89.7 --await-for 42.0 2024-04-26T18:36:13.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:12 smithi103 ceph-mon[32727]: pgmap v255: 97 pgs: 97 active+clean; 1.4 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-26T18:36:13.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:12 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2605291446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:36:13.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:12 smithi103 ceph-mon[32727]: osdmap e100: 12 total, 12 up, 12 in 2024-04-26T18:36:13.349 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:36:14.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:14 smithi084 ceph-mon[29141]: pgmap v257: 97 pgs: 97 active+clean; 1.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-04-26T18:36:14.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:14 smithi174 ceph-mon[32705]: pgmap v257: 97 pgs: 97 active+clean; 1.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-04-26T18:36:15.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.072+0000 7fc8f2bae700 1 -- 172.21.15.84:0/2616477900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8ec1034d0 msgr2=0x7fc8ec103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:15.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.072+0000 7fc8f2bae700 1 --2- 172.21.15.84:0/2616477900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8ec1034d0 0x7fc8ec103950 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80099c0 tx=0x7fc8d8009cd0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.072+0000 7fc8f2bae700 1 -- 172.21.15.84:0/2616477900 shutdown_connections 2024-04-26T18:36:15.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.072+0000 7fc8f2bae700 1 --2- 172.21.15.84:0/2616477900 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec103e90 0x7fc8ec1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.072+0000 7fc8f2bae700 1 --2- 172.21.15.84:0/2616477900 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec1094a0 0x7fc8ec109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.072+0000 7fc8f2bae700 1 --2- 172.21.15.84:0/2616477900 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8ec1034d0 0x7fc8ec103950 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.072+0000 7fc8f2bae700 1 -- 172.21.15.84:0/2616477900 >> 172.21.15.84:0/2616477900 conn(0x7fc8ec07a4f0 msgr2=0x7fc8ec07a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:15.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.072+0000 7fc8f2bae700 1 -- 172.21.15.84:0/2616477900 shutdown_connections 2024-04-26T18:36:15.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.072+0000 7fc8f2bae700 1 -- 172.21.15.84:0/2616477900 wait complete. 2024-04-26T18:36:15.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.073+0000 7fc8f2bae700 1 Processor -- start 2024-04-26T18:36:15.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.073+0000 7fc8f2bae700 1 -- start start 2024-04-26T18:36:15.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.073+0000 7fc8f2bae700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec1034d0 0x7fc8ec19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.073+0000 7fc8f2bae700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec103e90 0x7fc8ec19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.073+0000 7fc8f2bae700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8ec1094a0 0x7fc8ec1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.073+0000 7fc8f2bae700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec113650 con 0x7fc8ec1094a0 2024-04-26T18:36:15.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.073+0000 7fc8f2bae700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1134d0 con 0x7fc8ec1034d0 2024-04-26T18:36:15.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.073+0000 7fc8f2bae700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1137d0 con 0x7fc8ec103e90 2024-04-26T18:36:15.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8f114b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8ec1094a0 0x7fc8ec1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8f114b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8ec1094a0 0x7fc8ec1998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47266/0 (socket says 172.21.15.84:47266) 2024-04-26T18:36:15.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8f114b700 1 -- 172.21.15.84:0/717314065 learned_addr learned my addr 172.21.15.84:0/717314065 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:15.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8f094a700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec1034d0 0x7fc8ec19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8ebfff700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec103e90 0x7fc8ec19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8f114b700 1 -- 172.21.15.84:0/717314065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec103e90 msgr2=0x7fc8ec19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:15.080 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8f114b700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec103e90 0x7fc8ec19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8f114b700 1 -- 172.21.15.84:0/717314065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec1034d0 msgr2=0x7fc8ec19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8f114b700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec1034d0 0x7fc8ec19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8f114b700 1 -- 172.21.15.84:0/717314065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec19a030 con 0x7fc8ec1094a0 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8f114b700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8ec1094a0 0x7fc8ec1998d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400b970 tx=0x7fc8e400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8e9ffb700 1 -- 172.21.15.84:0/717314065 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4019070 con 0x7fc8ec1094a0 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8e9ffb700 1 -- 172.21.15.84:0/717314065 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e400cde0 con 0x7fc8ec1094a0 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8f2bae700 1 -- 172.21.15.84:0/717314065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec19a320 con 0x7fc8ec1094a0 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.074+0000 7fc8e9ffb700 1 -- 172.21.15.84:0/717314065 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4007370 con 0x7fc8ec1094a0 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.075+0000 7fc8f2bae700 1 -- 172.21.15.84:0/717314065 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec1a65e0 con 0x7fc8ec1094a0 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.077+0000 7fc8e9ffb700 1 -- 172.21.15.84:0/717314065 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc8e401c070 con 0x7fc8ec1094a0 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.078+0000 7fc8e9ffb700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc8dc0a3e50 0x7fc8dc0a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.078+0000 7fc8f094a700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc8dc0a3e50 0x7fc8dc0a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.078+0000 7fc8e9ffb700 1 -- 172.21.15.84:0/717314065 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fc8e40cfaf0 con 0x7fc8ec1094a0 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.078+0000 7fc8f094a700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc8dc0a3e50 0x7fc8dc0a6310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0005950 tx=0x7fc8e000a300 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:15.081 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.078+0000 7fc8fd440b80 1 Processor -- start 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.078+0000 7fc8fd440b80 1 -- start start 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8fd440b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe010d2e0 0x556fe010d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8fd440b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010dc00 0x556fe01076f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8fd440b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe0107d10 0x556fe01081a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8fd440b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556fe00559c0 con 0x556fe0107d10 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8fd440b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556fdffbbac0 con 0x556fe010dc00 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8fd440b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556fe0051980 con 0x556fe010d2e0 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8f114b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe0107d10 0x556fe01081a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8f114b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe0107d10 0x556fe01081a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47272/0 (socket says 172.21.15.84:47272) 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8f114b700 1 -- 172.21.15.84:0/1364977132 learned_addr learned my addr 172.21.15.84:0/1364977132 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8ebfff700 1 --2- 172.21.15.84:0/1364977132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010dc00 0x556fe01076f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8f094a700 1 --2- 172.21.15.84:0/1364977132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe010d2e0 0x556fe010d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8f114b700 1 -- 172.21.15.84:0/1364977132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe010d2e0 msgr2=0x556fe010d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:15.082 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8f114b700 1 --2- 172.21.15.84:0/1364977132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe010d2e0 0x556fe010d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8f114b700 1 -- 172.21.15.84:0/1364977132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010dc00 msgr2=0x556fe01076f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:15.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8f114b700 1 --2- 172.21.15.84:0/1364977132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010dc00 0x556fe01076f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8f114b700 1 -- 172.21.15.84:0/1364977132 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fdffa2a60 con 0x556fe0107d10 2024-04-26T18:36:15.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8f114b700 1 --2- 172.21.15.84:0/1364977132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe0107d10 0x556fe01081a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4097fc0 tx=0x7fc8e4004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf69e91ea6eb638f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:15.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8d6ffd700 1 -- 172.21.15.84:0/1364977132 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40a47d0 con 0x556fe0107d10 2024-04-26T18:36:15.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8d6ffd700 1 -- 172.21.15.84:0/1364977132 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e4099e60 con 0x556fe0107d10 2024-04-26T18:36:15.083 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.079+0000 7fc8d6ffd700 1 -- 172.21.15.84:0/1364977132 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40a4340 con 0x556fe0107d10 2024-04-26T18:36:15.087 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.080+0000 7fc8fd440b80 1 -- 172.21.15.84:0/1364977132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe0107d10 msgr2=0x556fe01081a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.080+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/1364977132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe0107d10 0x556fe01081a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fc8e4097fc0 tx=0x7fc8e4004be0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.080+0000 7fc8fd440b80 1 -- 172.21.15.84:0/1364977132 shutdown_connections 2024-04-26T18:36:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.080+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/1364977132 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe0107d10 0x556fe01081a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.080+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/1364977132 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe010d2e0 0x556fe010d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.080+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/1364977132 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010dc00 0x556fe01076f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.080+0000 7fc8fd440b80 1 -- 172.21.15.84:0/1364977132 >> 172.21.15.84:0/1364977132 conn(0x556fe0031750 msgr2=0x556fe00fe090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.080+0000 7fc8fd440b80 1 -- 172.21.15.84:0/1364977132 shutdown_connections 2024-04-26T18:36:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.080+0000 7fc8fd440b80 1 -- 172.21.15.84:0/1364977132 wait complete. 2024-04-26T18:36:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.080+0000 7fc8fd440b80 1 -- start start 2024-04-26T18:36:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8fd440b80 1 Processor -- start 2024-04-26T18:36:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8fd440b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe0107d10 0x556fe0113a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8fd440b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010d2e0 0x556fe0110da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.088 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8fd440b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010dc00 0x556fe01112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8fd440b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556fe0114720 con 0x556fe010d2e0 2024-04-26T18:36:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8fd440b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556fe000fc10 con 0x556fe010dc00 2024-04-26T18:36:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8fd440b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556fe01148a0 con 0x556fe0107d10 2024-04-26T18:36:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8fd440b80 10 client.0 _ll_register_callbacks cb 0x556fdffedae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010d2e0 0x556fe0110da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8ebfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010d2e0 0x556fe0110da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47288/0 (socket says 172.21.15.84:47288) 2024-04-26T18:36:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8ebfff700 1 -- 172.21.15.84:0/123822154 learned_addr learned my addr 172.21.15.84:0/123822154 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8f114b700 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010dc00 0x556fe01112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8f094a700 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe0107d10 0x556fe0113a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8ebfff700 1 -- 172.21.15.84:0/123822154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe0107d10 msgr2=0x556fe0113a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8ebfff700 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe0107d10 0x556fe0113a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8ebfff700 1 -- 172.21.15.84:0/123822154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010dc00 msgr2=0x556fe01112e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:15.089 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8ebfff700 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010dc00 0x556fe01112e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8ebfff700 1 -- 172.21.15.84:0/123822154 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe0013610 con 0x556fe010d2e0 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.081+0000 7fc8f094a700 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe0107d10 0x556fe0113a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.082+0000 7fc8ebfff700 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010d2e0 0x556fe0110da0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800a010 tx=0x7fc8d800bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a9815d1832b21f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.082+0000 7fc8cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.082+0000 7fc8cbfff700 1 -- 172.21.15.84:0/123822154 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8d8003c70 con 0x556fe010d2e0 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.082+0000 7fc8cbfff700 1 -- 172.21.15.84:0/123822154 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fc8d8005070 con 0x556fe010d2e0 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.082+0000 7fc8fd440b80 1 -- 172.21.15.84:0/123822154 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8e4099e60 con 0x556fe010d2e0 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.082+0000 7fc8cbfff700 1 -- 172.21.15.84:0/123822154 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc8d8031050 con 0x556fe010d2e0 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.082+0000 7fc8fd440b80 10 client.16401 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.082+0000 7fc8fd440b80 1 -- 172.21.15.84:0/123822154 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556fe011f070 con 0x556fe010d2e0 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.082+0000 7fc8cbfff700 1 -- 172.21.15.84:0/123822154 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e4099e60 con 0x556fe010d2e0 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.082+0000 7fc8cbfff700 1 -- 172.21.15.84:0/123822154 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc8d8023810 con 0x556fe010d2e0 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.082+0000 7fc8fd440b80 10 client.16401 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:15.090 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.083+0000 7fc8fd440b80 10 client.16401 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.083+0000 7fc8fd440b80 20 client.16401 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.083+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556fe0121650 0x556fe0123a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.083+0000 7fc8fd440b80 4 client.16401 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.083+0000 7fc8fd440b80 1 -- 172.21.15.84:0/123822154 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556fe00633a0 con 0x556fe0121650 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.083+0000 7fc8f094a700 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556fe0121650 0x556fe0123a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.083+0000 7fc8f094a700 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556fe0121650 0x556fe0123a40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cccd938865eee186 server_cookie=ad62a4cd18db494c in_seq=0 out_seq=0 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.083+0000 7fc8cbfff700 10 client.16401 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8cbfff700 1 -- 172.21.15.84:0/123822154 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556fe00633a0 con 0x556fe0121650 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8cbfff700 10 client.16401 handle_command_reply: tid=0 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8fd440b80 1 client.16401 shutdown 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8fd440b80 1 -- 172.21.15.84:0/123822154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010d2e0 msgr2=0x556fe0110da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010d2e0 0x556fe0110da0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fc8d800a010 tx=0x7fc8d800bf50 comp rx=0 tx=0).stop 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8fd440b80 1 -- 172.21.15.84:0/123822154 shutdown_connections 2024-04-26T18:36:15.091 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556fe0121650 0x556fe0123a40 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010dc00 0x556fe01112e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe0107d10 0x556fe0113a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8cbfff700 0 client.16401 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:15.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/123822154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010d2e0 0x556fe0110da0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8fd440b80 1 -- 172.21.15.84:0/123822154 >> 172.21.15.84:0/123822154 conn(0x556fe0031750 msgr2=0x556fe01022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:15.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8fd440b80 1 -- 172.21.15.84:0/123822154 shutdown_connections 2024-04-26T18:36:15.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.084+0000 7fc8fd440b80 1 -- 172.21.15.84:0/123822154 wait complete. 2024-04-26T18:36:15.092 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.085+0000 7fc8fd440b80 20 client.16401 trim_cache size 0 max 16384 2024-04-26T18:36:15.106 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8fd440b80 1 Processor -- start 2024-04-26T18:36:15.107 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8fd440b80 1 -- start start 2024-04-26T18:36:15.107 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8fd440b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe0107d10 0x556fe010b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.107 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8fd440b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010d2e0 0x556fe00fe0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.107 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8fd440b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 0x556fe010c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.107 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8fd440b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1aba90 con 0x556fe010dc00 2024-04-26T18:36:15.107 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8fd440b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1ad5d0 con 0x556fe010d2e0 2024-04-26T18:36:15.107 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8fd440b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1ac160 con 0x556fe0107d10 2024-04-26T18:36:15.107 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8f114b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 0x556fe010c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.107 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8f114b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 0x556fe010c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47302/0 (socket says 172.21.15.84:47302) 2024-04-26T18:36:15.107 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8f114b700 1 -- 172.21.15.84:0/4272587325 learned_addr learned my addr 172.21.15.84:0/4272587325 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:15.107 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8ebfff700 1 --2- 172.21.15.84:0/4272587325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010d2e0 0x556fe00fe0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8f094a700 1 --2- 172.21.15.84:0/4272587325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe0107d10 0x556fe010b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8f114b700 1 -- 172.21.15.84:0/4272587325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe0107d10 msgr2=0x556fe010b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8f114b700 1 --2- 172.21.15.84:0/4272587325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe0107d10 0x556fe010b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8f114b700 1 -- 172.21.15.84:0/4272587325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010d2e0 msgr2=0x556fe00fe0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8f114b700 1 --2- 172.21.15.84:0/4272587325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010d2e0 0x556fe00fe0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8f114b700 1 -- 172.21.15.84:0/4272587325 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556fe00ffdc0 con 0x556fe010dc00 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8ebfff700 1 --2- 172.21.15.84:0/4272587325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010d2e0 0x556fe00fe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8f094a700 1 --2- 172.21.15.84:0/4272587325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe0107d10 0x556fe010b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.104+0000 7fc8f114b700 1 --2- 172.21.15.84:0/4272587325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 0x556fe010c800 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc8e409baf0 tx=0x7fc8e4014a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f190047685edb7ac server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.105+0000 7fc8d67fc700 1 -- 172.21.15.84:0/4272587325 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e4097e00 con 0x556fe010dc00 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.105+0000 7fc8d67fc700 1 -- 172.21.15.84:0/4272587325 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e40bc0c0 con 0x556fe010dc00 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.105+0000 7fc8d67fc700 1 -- 172.21.15.84:0/4272587325 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40a45a0 con 0x556fe010dc00 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.105+0000 7fc8fd440b80 1 -- 172.21.15.84:0/4272587325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 msgr2=0x556fe010c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:15.108 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.105+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/4272587325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 0x556fe010c800 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fc8e409baf0 tx=0x7fc8e4014a20 comp rx=0 tx=0).stop 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.105+0000 7fc8fd440b80 1 -- 172.21.15.84:0/4272587325 shutdown_connections 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.105+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/4272587325 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 0x556fe010c800 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.105+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/4272587325 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe0107d10 0x556fe010b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.105+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/4272587325 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe010d2e0 0x556fe00fe0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.105+0000 7fc8fd440b80 1 -- 172.21.15.84:0/4272587325 >> 172.21.15.84:0/4272587325 conn(0x556fe0031750 msgr2=0x556fe00f7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.105+0000 7fc8fd440b80 1 -- 172.21.15.84:0/4272587325 shutdown_connections 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.105+0000 7fc8fd440b80 1 -- 172.21.15.84:0/4272587325 wait complete. 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8fd440b80 1 -- start start 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8fd440b80 1 Processor -- start 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8fd440b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe0107d10 0x556fe0116d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8fd440b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe010d2e0 0x556fe0117260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8fd440b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 0x556fe0187b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8fd440b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1acf00 con 0x556fe010dc00 2024-04-26T18:36:15.109 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8fd440b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1ac830 con 0x556fe0107d10 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8fd440b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec1a9a50 con 0x556fe010d2e0 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8fd440b80 10 client.0 _ll_register_callbacks cb 0x556fdffedae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8f114b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 0x556fe0187b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8f094a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe0107d10 0x556fe0116d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8f114b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 0x556fe0187b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47316/0 (socket says 172.21.15.84:47316) 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8f094a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe0107d10 0x556fe0116d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:40348/0 (socket says 172.21.15.84:40348) 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8f114b700 1 -- 172.21.15.84:0/2198599972 learned_addr learned my addr 172.21.15.84:0/2198599972 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.106+0000 7fc8ebfff700 1 --2- 172.21.15.84:0/2198599972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe010d2e0 0x556fe0117260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8f114b700 1 -- 172.21.15.84:0/2198599972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe010d2e0 msgr2=0x556fe0117260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8f114b700 1 --2- 172.21.15.84:0/2198599972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe010d2e0 0x556fe0117260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8f114b700 1 -- 172.21.15.84:0/2198599972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe0107d10 msgr2=0x556fe0116d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8f114b700 1 --2- 172.21.15.84:0/2198599972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe0107d10 0x556fe0116d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8f114b700 1 -- 172.21.15.84:0/2198599972 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556fe0188150 con 0x556fe010dc00 2024-04-26T18:36:15.110 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8f094a700 1 --2- 172.21.15.84:0/2198599972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe0107d10 0x556fe0116d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:36:15.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8f114b700 1 --2- 172.21.15.84:0/2198599972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 0x556fe0187b90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc8e40a42a0 tx=0x7fc8e4004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2189770b0d7628 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:15.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:15.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8cbfff700 1 -- 172.21.15.84:0/2198599972 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e40a47d0 con 0x556fe010dc00 2024-04-26T18:36:15.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8cbfff700 1 -- 172.21.15.84:0/2198599972 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fc8e409bfd0 con 0x556fe010dc00 2024-04-26T18:36:15.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8fd440b80 1 -- 172.21.15.84:0/2198599972 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8e40bc0c0 con 0x556fe010dc00 2024-04-26T18:36:15.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8cbfff700 1 -- 172.21.15.84:0/2198599972 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc8e40a4340 con 0x556fe010dc00 2024-04-26T18:36:15.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8fd440b80 10 client.16407 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:15.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8fd440b80 1 -- 172.21.15.84:0/2198599972 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556fe0189380 con 0x556fe010dc00 2024-04-26T18:36:15.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8cbfff700 1 -- 172.21.15.84:0/2198599972 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e40bc0c0 con 0x556fe010dc00 2024-04-26T18:36:15.112 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8cbfff700 1 -- 172.21.15.84:0/2198599972 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc8e4097940 con 0x556fe010dc00 2024-04-26T18:36:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8fd440b80 10 client.16407 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8fd440b80 10 client.16407 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:36:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8fd440b80 20 client.16407 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.107+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/2198599972 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556fe0189600 0x556fe018b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.108+0000 7fc8fd440b80 4 client.16407 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.7, "await_for": 42.0}] 2024-04-26T18:36:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.108+0000 7fc8fd440b80 1 -- 172.21.15.84:0/2198599972 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 89.7, "await_for": 42.0}) -- 0x556fe005d440 con 0x556fe0189600 2024-04-26T18:36:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.108+0000 7fc8f094a700 1 --2- 172.21.15.84:0/2198599972 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556fe0189600 0x556fe018b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.108+0000 7fc8f094a700 1 --2- 172.21.15.84:0/2198599972 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556fe0189600 0x556fe018b9f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fc476cc7796c7ea server_cookie=33c3e9ef3757e621 in_seq=0 out_seq=0 2024-04-26T18:36:15.113 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.108+0000 7fc8cbfff700 10 client.16407 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:15.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.123+0000 7fc8cbfff700 1 -- 172.21.15.84:0/2198599972 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x556fe005d440 con 0x556fe0189600 2024-04-26T18:36:15.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.123+0000 7fc8cbfff700 10 client.16407 handle_command_reply: tid=0 2024-04-26T18:36:15.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.123+0000 7fc8fd440b80 1 client.16407 shutdown 2024-04-26T18:36:15.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.123+0000 7fc8fd440b80 1 -- 172.21.15.84:0/2198599972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 msgr2=0x556fe0187b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:15.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.123+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/2198599972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 0x556fe0187b90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc8e40a42a0 tx=0x7fc8e4004be0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.123+0000 7fc8fd440b80 1 -- 172.21.15.84:0/2198599972 shutdown_connections 2024-04-26T18:36:15.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.123+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/2198599972 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556fe0189600 0x556fe018b9f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.123+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/2198599972 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556fe010dc00 0x556fe0187b90 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.123+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/2198599972 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556fe0107d10 0x556fe0116d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.124+0000 7fc8cbfff700 0 client.16407 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:15.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.124+0000 7fc8fd440b80 1 --2- 172.21.15.84:0/2198599972 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556fe010d2e0 0x556fe0117260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.124+0000 7fc8fd440b80 1 -- 172.21.15.84:0/2198599972 >> 172.21.15.84:0/2198599972 conn(0x556fe0031750 msgr2=0x556fe0102b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:15.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.124+0000 7fc8fd440b80 1 -- 172.21.15.84:0/2198599972 shutdown_connections 2024-04-26T18:36:15.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.124+0000 7fc8fd440b80 1 -- 172.21.15.84:0/2198599972 wait complete. 2024-04-26T18:36:15.125 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.124+0000 7fc8fd440b80 20 client.16407 trim_cache size 0 max 16384 2024-04-26T18:36:15.125 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:36:15.125 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 22, 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "db6a0f03": { 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 22, 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 89.7, 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:36:15.126 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:36:15.127 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:36:15.127 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:36:15.127 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:36:15.127 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:36:15.127 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:36:15.127 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:36:15.127 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:36:15.127 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:36:15.127 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:36:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.124+0000 7fc8f2bae700 1 -- 172.21.15.84:0/717314065 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc8dc0a3e50 msgr2=0x7fc8dc0a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.124+0000 7fc8f2bae700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc8dc0a3e50 0x7fc8dc0a6310 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0005950 tx=0x7fc8e000a300 comp rx=0 tx=0).stop 2024-04-26T18:36:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.124+0000 7fc8f2bae700 1 -- 172.21.15.84:0/717314065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8ec1094a0 msgr2=0x7fc8ec1998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.124+0000 7fc8f2bae700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8ec1094a0 0x7fc8ec1998d0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc8e400b970 tx=0x7fc8e400bc80 comp rx=0 tx=0).stop 2024-04-26T18:36:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.125+0000 7fc8f2bae700 1 -- 172.21.15.84:0/717314065 shutdown_connections 2024-04-26T18:36:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.125+0000 7fc8f2bae700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc8dc0a3e50 0x7fc8dc0a6310 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.125+0000 7fc8f2bae700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc8ec1094a0 0x7fc8ec1998d0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.125+0000 7fc8f2bae700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc8ec1034d0 0x7fc8ec19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.125+0000 7fc8f2bae700 1 --2- 172.21.15.84:0/717314065 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc8ec103e90 0x7fc8ec19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:15.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.125+0000 7fc8f2bae700 1 -- 172.21.15.84:0/717314065 >> 172.21.15.84:0/717314065 conn(0x7fc8ec07a4f0 msgr2=0x7fc8ec1020f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:15.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.125+0000 7fc8f2bae700 1 -- 172.21.15.84:0/717314065 shutdown_connections 2024-04-26T18:36:15.129 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:15.125+0000 7fc8f2bae700 1 -- 172.21.15.84:0/717314065 wait complete. 2024-04-26T18:36:15.129 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:14 smithi103 ceph-mon[32727]: pgmap v257: 97 pgs: 97 active+clean; 1.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-04-26T18:36:15.911 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'db6a0f03', quiesce took 5.9 seconds. Will release after: 29.7 2024-04-26T18:36:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:16 smithi174 ceph-mon[32705]: pgmap v258: 97 pgs: 97 active+clean; 1.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-04-26T18:36:17.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:16 smithi084 ceph-mon[29141]: pgmap v258: 97 pgs: 97 active+clean; 1.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-04-26T18:36:17.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:16 smithi103 ceph-mon[32727]: pgmap v258: 97 pgs: 97 active+clean; 1.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-04-26T18:36:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.567+0000 7f35723f0640 1 -- 172.21.15.84:0/2892325142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f356c10cf00 msgr2=0x7f356c10f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.567+0000 7f35723f0640 1 --2- 172.21.15.84:0/2892325142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f356c10cf00 0x7f356c10f3a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f3564009ca0 tx=0x7f356402a9e0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.567+0000 7f35723f0640 1 -- 172.21.15.84:0/2892325142 shutdown_connections 2024-04-26T18:36:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.567+0000 7f35723f0640 1 --2- 172.21.15.84:0/2892325142 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f356c10cf00 0x7f356c10f3a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.567+0000 7f35723f0640 1 --2- 172.21.15.84:0/2892325142 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f356c1044e0 0x7f356c10c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.567+0000 7f35723f0640 1 --2- 172.21.15.84:0/2892325142 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f356c103bc0 0x7f356c103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.567+0000 7f35723f0640 1 -- 172.21.15.84:0/2892325142 >> 172.21.15.84:0/2892325142 conn(0x7f356c0789b0 msgr2=0x7f356c078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.567+0000 7f35723f0640 1 -- 172.21.15.84:0/2892325142 shutdown_connections 2024-04-26T18:36:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.567+0000 7f35723f0640 1 -- 172.21.15.84:0/2892325142 wait complete. 2024-04-26T18:36:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.567+0000 7f35723f0640 1 Processor -- start 2024-04-26T18:36:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f35723f0640 1 -- start start 2024-04-26T18:36:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f35723f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f356c103bc0 0x7f356c101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f35723f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f356c1044e0 0x7f356c101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f35723f0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f356c10cf00 0x7f356c102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f35723f0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f356c112bf0 con 0x7f356c1044e0 2024-04-26T18:36:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f35723f0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f356c112a70 con 0x7f356c103bc0 2024-04-26T18:36:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f35723f0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f356c112d70 con 0x7f356c10cf00 2024-04-26T18:36:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f356b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f356c1044e0 0x7f356c101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f356b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f356c1044e0 0x7f356c101d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52396/0 (socket says 172.21.15.84:52396) 2024-04-26T18:36:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f356b7fe640 1 -- 172.21.15.84:0/1078137059 learned_addr learned my addr 172.21.15.84:0/1078137059 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f357098d640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f356c10cf00 0x7f356c102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f356bfff640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f356c103bc0 0x7f356c101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f356b7fe640 1 -- 172.21.15.84:0/1078137059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f356c10cf00 msgr2=0x7f356c102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f356b7fe640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f356c10cf00 0x7f356c102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f356b7fe640 1 -- 172.21.15.84:0/1078137059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f356c103bc0 msgr2=0x7f356c101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f356b7fe640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f356c103bc0 0x7f356c101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.568+0000 7f356b7fe640 1 -- 172.21.15.84:0/1078137059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f356c1a7570 con 0x7f356c1044e0 2024-04-26T18:36:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.569+0000 7f356b7fe640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f356c1044e0 0x7f356c101d50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f356000b4a0 tx=0x7f356000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.569+0000 7f35697fa640 1 -- 172.21.15.84:0/1078137059 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560014070 con 0x7f356c1044e0 2024-04-26T18:36:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.569+0000 7f35723f0640 1 -- 172.21.15.84:0/1078137059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f356c1a7860 con 0x7f356c1044e0 2024-04-26T18:36:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.569+0000 7f35697fa640 1 -- 172.21.15.84:0/1078137059 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3560010ac0 con 0x7f356c1044e0 2024-04-26T18:36:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.569+0000 7f35697fa640 1 -- 172.21.15.84:0/1078137059 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560017c30 con 0x7f356c1044e0 2024-04-26T18:36:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.569+0000 7f35723f0640 1 -- 172.21.15.84:0/1078137059 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f356c1a7db0 con 0x7f356c1044e0 2024-04-26T18:36:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.571+0000 7f35697fa640 1 -- 172.21.15.84:0/1078137059 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3560017070 con 0x7f356c1044e0 2024-04-26T18:36:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.571+0000 7f35751e7740 1 Processor -- start 2024-04-26T18:36:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.571+0000 7f35751e7740 1 -- start start 2024-04-26T18:36:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.571+0000 7f35751e7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27ef00 0x55cc5c27f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f35751e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27f820 0x55cc5c289ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f35751e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c28a0e0 0x55cc5c28c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f35751e7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cc5c1e5460 con 0x55cc5c27ef00 2024-04-26T18:36:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f35751e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cc5c19f590 con 0x55cc5c28a0e0 2024-04-26T18:36:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f35751e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cc5c1e1420 con 0x55cc5c27f820 2024-04-26T18:36:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f357098d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27ef00 0x55cc5c27f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f357098d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27ef00 0x55cc5c27f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52398/0 (socket says 172.21.15.84:52398) 2024-04-26T18:36:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f357098d640 1 -- 172.21.15.84:0/1326392894 learned_addr learned my addr 172.21.15.84:0/1326392894 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f356bfff640 1 --2- 172.21.15.84:0/1326392894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27f820 0x55cc5c289ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f356b7fe640 1 --2- 172.21.15.84:0/1326392894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c28a0e0 0x55cc5c28c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f35697fa640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f35540a3df0 0x7f35540a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f357098d640 1 -- 172.21.15.84:0/1326392894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27f820 msgr2=0x55cc5c289ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f357098d640 1 --2- 172.21.15.84:0/1326392894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27f820 0x55cc5c289ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f357098d640 1 -- 172.21.15.84:0/1326392894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c28a0e0 msgr2=0x55cc5c28c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f357098d640 1 --2- 172.21.15.84:0/1326392894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c28a0e0 0x55cc5c28c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f357098d640 1 -- 172.21.15.84:0/1326392894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc5c1ac0d0 con 0x55cc5c27ef00 2024-04-26T18:36:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f356bfff640 1 --2- 172.21.15.84:0/1326392894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27f820 0x55cc5c289ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:36:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f35697fa640 1 -- 172.21.15.84:0/1078137059 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7f35600cfaf0 con 0x7f356c1044e0 2024-04-26T18:36:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f357098d640 1 --2- 172.21.15.84:0/1326392894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27ef00 0x55cc5c27f2e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f356402eb70 tx=0x7f35640093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8021c8a4539e979 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.572+0000 7f357098d640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f35540a3df0 0x7f35540a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.573+0000 7f35527fc640 1 -- 172.21.15.84:0/1326392894 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564033040 con 0x55cc5c27ef00 2024-04-26T18:36:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.573+0000 7f35527fc640 1 -- 172.21.15.84:0/1326392894 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f356402e8a0 con 0x55cc5c27ef00 2024-04-26T18:36:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.573+0000 7f35527fc640 1 -- 172.21.15.84:0/1326392894 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35640045e0 con 0x55cc5c27ef00 2024-04-26T18:36:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.573+0000 7f35751e7740 1 -- 172.21.15.84:0/1326392894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27ef00 msgr2=0x55cc5c27f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.573+0000 7f357098d640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f35540a3df0 0x7f35540a62b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3564049000 tx=0x7f356404c000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.573+0000 7f35751e7740 1 --2- 172.21.15.84:0/1326392894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27ef00 0x55cc5c27f2e0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f356402eb70 tx=0x7f35640093f0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.573+0000 7f35751e7740 1 -- 172.21.15.84:0/1326392894 shutdown_connections 2024-04-26T18:36:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.573+0000 7f35751e7740 1 --2- 172.21.15.84:0/1326392894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c28a0e0 0x55cc5c28c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.573+0000 7f35751e7740 1 --2- 172.21.15.84:0/1326392894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27f820 0x55cc5c289ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.573+0000 7f35751e7740 1 --2- 172.21.15.84:0/1326392894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27ef00 0x55cc5c27f2e0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.573+0000 7f35751e7740 1 -- 172.21.15.84:0/1326392894 >> 172.21.15.84:0/1326392894 conn(0x55cc5c27b410 msgr2=0x55cc5c27b7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.573+0000 7f35751e7740 1 -- 172.21.15.84:0/1326392894 shutdown_connections 2024-04-26T18:36:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.573+0000 7f35751e7740 1 -- 172.21.15.84:0/1326392894 wait complete. 2024-04-26T18:36:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.574+0000 7f35751e7740 1 -- start start 2024-04-26T18:36:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.574+0000 7f35751e7740 1 Processor -- start 2024-04-26T18:36:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.574+0000 7f35751e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27ef00 0x55cc5c287050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.574+0000 7f35751e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27f820 0x55cc5c287590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.574+0000 7f35751e7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 0x55cc5c287ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.574+0000 7f35751e7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cc5c28ccb0 con 0x55cc5c28a0e0 2024-04-26T18:36:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.574+0000 7f35751e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cc5c16cb60 con 0x55cc5c27f820 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.574+0000 7f35751e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cc5c28ce30 con 0x55cc5c27ef00 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.574+0000 7f35751e7740 10 client.0 _ll_register_callbacks cb 0x55cc5c1912b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.574+0000 7f357098d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 0x55cc5c287ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.574+0000 7f357098d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 0x55cc5c287ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52404/0 (socket says 172.21.15.84:52404) 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.574+0000 7f357098d640 1 -- 172.21.15.84:0/2684453224 learned_addr learned my addr 172.21.15.84:0/2684453224 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f356b7fe640 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27f820 0x55cc5c287590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f356bfff640 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27ef00 0x55cc5c287050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f357098d640 1 -- 172.21.15.84:0/2684453224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27ef00 msgr2=0x55cc5c287050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f357098d640 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27ef00 0x55cc5c287050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f357098d640 1 -- 172.21.15.84:0/2684453224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27f820 msgr2=0x55cc5c287590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f357098d640 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27f820 0x55cc5c287590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f357098d640 1 -- 172.21.15.84:0/2684453224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc5bef7210 con 0x55cc5c28a0e0 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f356bfff640 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27ef00 0x55cc5c287050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:36:19.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f357098d640 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 0x55cc5c287ad0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f35640042a0 tx=0x7f3564004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f798f92859bb8f51 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f354b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f354b7fe640 1 -- 172.21.15.84:0/2684453224 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564033040 con 0x55cc5c28a0e0 2024-04-26T18:36:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f354b7fe640 1 -- 172.21.15.84:0/2684453224 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7f356402fbb0 con 0x55cc5c28a0e0 2024-04-26T18:36:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f35751e7740 1 -- 172.21.15.84:0/2684453224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f356402e8a0 con 0x55cc5c28a0e0 2024-04-26T18:36:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f354b7fe640 1 -- 172.21.15.84:0/2684453224 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f35640098b0 con 0x55cc5c28a0e0 2024-04-26T18:36:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f35751e7740 10 client.16419 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f35751e7740 1 -- 172.21.15.84:0/2684453224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc5bfc3620 con 0x55cc5c28a0e0 2024-04-26T18:36:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.575+0000 7f354b7fe640 1 -- 172.21.15.84:0/2684453224 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f356402e8a0 con 0x55cc5c28a0e0 2024-04-26T18:36:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.576+0000 7f354b7fe640 1 -- 172.21.15.84:0/2684453224 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f356405f020 con 0x55cc5c28a0e0 2024-04-26T18:36:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.576+0000 7f35751e7740 10 client.16419 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.576+0000 7f35751e7740 10 client.16419 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:36:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.576+0000 7f35751e7740 20 client.16419 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.576+0000 7f35751e7740 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cc5c29a030 0x55cc5c29c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.576+0000 7f35751e7740 4 client.16419 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:36:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.576+0000 7f35751e7740 1 -- 172.21.15.84:0/2684453224 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc5c1f3070 con 0x55cc5c29a030 2024-04-26T18:36:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.576+0000 7f356bfff640 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cc5c29a030 0x55cc5c29c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.576+0000 7f354b7fe640 10 client.16419 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.576+0000 7f356bfff640 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cc5c29a030 0x55cc5c29c420 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=571874c5b0d4732c server_cookie=da5467184d2ac706 in_seq=0 out_seq=0 2024-04-26T18:36:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f354b7fe640 1 -- 172.21.15.84:0/2684453224 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc5c1f3070 con 0x55cc5c29a030 2024-04-26T18:36:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f354b7fe640 10 client.16419 handle_command_reply: tid=0 2024-04-26T18:36:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f35751e7740 1 client.16419 shutdown 2024-04-26T18:36:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f35751e7740 1 -- 172.21.15.84:0/2684453224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 msgr2=0x55cc5c287ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f35751e7740 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 0x55cc5c287ad0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f35640042a0 tx=0x7f3564004710 comp rx=0 tx=0).stop 2024-04-26T18:36:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f35751e7740 1 -- 172.21.15.84:0/2684453224 shutdown_connections 2024-04-26T18:36:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f35751e7740 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cc5c29a030 0x55cc5c29c420 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f35751e7740 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 0x55cc5c287ad0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f35751e7740 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27f820 0x55cc5c287590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f354b7fe640 0 client.16419 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f35751e7740 1 --2- 172.21.15.84:0/2684453224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27ef00 0x55cc5c287050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f35751e7740 1 -- 172.21.15.84:0/2684453224 >> 172.21.15.84:0/2684453224 conn(0x55cc5c27b410 msgr2=0x55cc5c27a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f35751e7740 1 -- 172.21.15.84:0/2684453224 shutdown_connections 2024-04-26T18:36:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f35751e7740 1 -- 172.21.15.84:0/2684453224 wait complete. 2024-04-26T18:36:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.577+0000 7f35751e7740 20 client.16419 trim_cache size 0 max 16384 2024-04-26T18:36:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.592+0000 7f35751e7740 1 Processor -- start 2024-04-26T18:36:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.592+0000 7f35751e7740 1 -- start start 2024-04-26T18:36:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.592+0000 7f35751e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27ef00 0x55cc5c295b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.597 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.592+0000 7f35751e7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27f820 0x55cc5c2818c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.592+0000 7f35751e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c28a0e0 0x55cc5c288f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.592+0000 7f35751e7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1b08d0 con 0x55cc5c27f820 2024-04-26T18:36:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.592+0000 7f35751e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1b2410 con 0x55cc5c27ef00 2024-04-26T18:36:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.592+0000 7f35751e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1b0fa0 con 0x55cc5c28a0e0 2024-04-26T18:36:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.592+0000 7f356b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27f820 0x55cc5c2818c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.592+0000 7f356b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27f820 0x55cc5c2818c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52406/0 (socket says 172.21.15.84:52406) 2024-04-26T18:36:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.592+0000 7f356b7fe640 1 -- 172.21.15.84:0/3708285629 learned_addr learned my addr 172.21.15.84:0/3708285629 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.593+0000 7f357098d640 1 --2- 172.21.15.84:0/3708285629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c28a0e0 0x55cc5c288f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.593+0000 7f356bfff640 1 --2- 172.21.15.84:0/3708285629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27ef00 0x55cc5c295b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.593+0000 7f356b7fe640 1 -- 172.21.15.84:0/3708285629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c28a0e0 msgr2=0x55cc5c288f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.593+0000 7f356b7fe640 1 --2- 172.21.15.84:0/3708285629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c28a0e0 0x55cc5c288f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.593+0000 7f356b7fe640 1 -- 172.21.15.84:0/3708285629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27ef00 msgr2=0x55cc5c295b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.593+0000 7f356b7fe640 1 --2- 172.21.15.84:0/3708285629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27ef00 0x55cc5c295b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.593+0000 7f356b7fe640 1 -- 172.21.15.84:0/3708285629 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc5c2855a0 con 0x55cc5c27f820 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.593+0000 7f356bfff640 1 --2- 172.21.15.84:0/3708285629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27ef00 0x55cc5c295b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.593+0000 7f357098d640 1 --2- 172.21.15.84:0/3708285629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c28a0e0 0x55cc5c288f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.593+0000 7f356b7fe640 1 --2- 172.21.15.84:0/3708285629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27f820 0x55cc5c2818c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f356000b470 tx=0x7f3560010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=417853637f754f04 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.593+0000 7f3551ffb640 1 -- 172.21.15.84:0/3708285629 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f356001e070 con 0x55cc5c27f820 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.593+0000 7f3551ffb640 1 -- 172.21.15.84:0/3708285629 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f356009bb60 con 0x55cc5c27f820 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.593+0000 7f3551ffb640 1 -- 172.21.15.84:0/3708285629 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3560099ef0 con 0x55cc5c27f820 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.594+0000 7f35751e7740 1 -- 172.21.15.84:0/3708285629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27f820 msgr2=0x55cc5c2818c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.594+0000 7f35751e7740 1 --2- 172.21.15.84:0/3708285629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27f820 0x55cc5c2818c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f356000b470 tx=0x7f3560010f90 comp rx=0 tx=0).stop 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.594+0000 7f35751e7740 1 -- 172.21.15.84:0/3708285629 shutdown_connections 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.594+0000 7f35751e7740 1 --2- 172.21.15.84:0/3708285629 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c28a0e0 0x55cc5c288f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.594+0000 7f35751e7740 1 --2- 172.21.15.84:0/3708285629 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c27f820 0x55cc5c2818c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.594+0000 7f35751e7740 1 --2- 172.21.15.84:0/3708285629 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27ef00 0x55cc5c295b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.594+0000 7f35751e7740 1 -- 172.21.15.84:0/3708285629 >> 172.21.15.84:0/3708285629 conn(0x55cc5c27b410 msgr2=0x55cc5c288880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.594+0000 7f35751e7740 1 -- 172.21.15.84:0/3708285629 shutdown_connections 2024-04-26T18:36:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.594+0000 7f35751e7740 1 -- 172.21.15.84:0/3708285629 wait complete. 2024-04-26T18:36:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.594+0000 7f35751e7740 1 -- start start 2024-04-26T18:36:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.594+0000 7f35751e7740 1 Processor -- start 2024-04-26T18:36:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.594+0000 7f35751e7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27ef00 0x55cc5c291d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f35751e7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27f820 0x55cc5c292280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f35751e7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 0x55cc5c28e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f35751e7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1b1d40 con 0x55cc5c28a0e0 2024-04-26T18:36:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f35751e7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1b1670 con 0x55cc5c27f820 2024-04-26T18:36:19.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f35751e7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f356c1ae8d0 con 0x55cc5c27ef00 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f35751e7740 10 client.0 _ll_register_callbacks cb 0x7f356402eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f357098d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 0x55cc5c28e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f357098d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 0x55cc5c28e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52408/0 (socket says 172.21.15.84:52408) 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f357098d640 1 -- 172.21.15.84:0/3219671907 learned_addr learned my addr 172.21.15.84:0/3219671907 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f356bfff640 1 --2- 172.21.15.84:0/3219671907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27ef00 0x55cc5c291d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f357098d640 1 -- 172.21.15.84:0/3219671907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27ef00 msgr2=0x55cc5c291d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f356b7fe640 1 --2- 172.21.15.84:0/3219671907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27f820 0x55cc5c292280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f357098d640 1 --2- 172.21.15.84:0/3219671907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27ef00 0x55cc5c291d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f357098d640 1 -- 172.21.15.84:0/3219671907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27f820 msgr2=0x55cc5c292280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f357098d640 1 --2- 172.21.15.84:0/3219671907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27f820 0x55cc5c292280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f357098d640 1 -- 172.21.15.84:0/3219671907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc5c28ea30 con 0x55cc5c28a0e0 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f357098d640 1 --2- 172.21.15.84:0/3219671907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 0x55cc5c28e230 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3564000f80 tx=0x7f35640040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fe90d0ddb11a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f354b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:19.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f354b7fe640 1 -- 172.21.15.84:0/3219671907 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3564033040 con 0x55cc5c28a0e0 2024-04-26T18:36:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f354b7fe640 1 -- 172.21.15.84:0/3219671907 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7f3564050b00 con 0x55cc5c28a0e0 2024-04-26T18:36:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f35751e7740 1 -- 172.21.15.84:0/3219671907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f356009bb60 con 0x55cc5c28a0e0 2024-04-26T18:36:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f354b7fe640 1 -- 172.21.15.84:0/3219671907 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f356406f050 con 0x55cc5c28a0e0 2024-04-26T18:36:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f35751e7740 10 client.16425 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.595+0000 7f35751e7740 1 -- 172.21.15.84:0/3219671907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc5c300c60 con 0x55cc5c28a0e0 2024-04-26T18:36:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.596+0000 7f354b7fe640 1 -- 172.21.15.84:0/3219671907 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f356009bb60 con 0x55cc5c28a0e0 2024-04-26T18:36:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.596+0000 7f354b7fe640 1 -- 172.21.15.84:0/3219671907 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f356405f040 con 0x55cc5c28a0e0 2024-04-26T18:36:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.596+0000 7f35751e7740 10 client.16425 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.596+0000 7f35751e7740 10 client.16425 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:36:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.596+0000 7f35751e7740 20 client.16425 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.596+0000 7f35751e7740 1 --2- 172.21.15.84:0/3219671907 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cc5c301310 0x55cc5c303730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.596+0000 7f35751e7740 4 client.16425 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:36:19.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.596+0000 7f35751e7740 1 -- 172.21.15.84:0/3219671907 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc5c1ecee0 con 0x55cc5c301310 2024-04-26T18:36:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.596+0000 7f356bfff640 1 --2- 172.21.15.84:0/3219671907 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cc5c301310 0x55cc5c303730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.597+0000 7f356bfff640 1 --2- 172.21.15.84:0/3219671907 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cc5c301310 0x55cc5c303730 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fef8afaa75cade75 server_cookie=71e252d43bd06bfd in_seq=0 out_seq=0 2024-04-26T18:36:19.603 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.597+0000 7f354b7fe640 10 client.16425 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.611+0000 7f354b7fe640 1 -- 172.21.15.84:0/3219671907 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc5c1ecee0 con 0x55cc5c301310 2024-04-26T18:36:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.611+0000 7f354b7fe640 10 client.16425 handle_command_reply: tid=0 2024-04-26T18:36:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.611+0000 7f35751e7740 1 client.16425 shutdown 2024-04-26T18:36:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35751e7740 1 -- 172.21.15.84:0/3219671907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 msgr2=0x55cc5c28e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35751e7740 1 --2- 172.21.15.84:0/3219671907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 0x55cc5c28e230 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3564000f80 tx=0x7f35640040c0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35751e7740 1 -- 172.21.15.84:0/3219671907 shutdown_connections 2024-04-26T18:36:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35751e7740 1 --2- 172.21.15.84:0/3219671907 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cc5c301310 0x55cc5c303730 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35751e7740 1 --2- 172.21.15.84:0/3219671907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cc5c28a0e0 0x55cc5c28e230 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35751e7740 1 --2- 172.21.15.84:0/3219671907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cc5c27f820 0x55cc5c292280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35751e7740 1 --2- 172.21.15.84:0/3219671907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cc5c27ef00 0x55cc5c291d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35751e7740 1 -- 172.21.15.84:0/3219671907 >> 172.21.15.84:0/3219671907 conn(0x55cc5c27b410 msgr2=0x55cc5c27a5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f354b7fe640 0 client.16425 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35751e7740 1 -- 172.21.15.84:0/3219671907 shutdown_connections 2024-04-26T18:36:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35751e7740 1 -- 172.21.15.84:0/3219671907 wait complete. 2024-04-26T18:36:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35751e7740 20 client.16425 trim_cache size 0 max 16384 2024-04-26T18:36:19.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:36:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:36:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:36:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:36:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35723f0640 1 -- 172.21.15.84:0/1078137059 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f35540a3df0 msgr2=0x7f35540a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35723f0640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f35540a3df0 0x7f35540a62b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f3564049000 tx=0x7f356404c000 comp rx=0 tx=0).stop 2024-04-26T18:36:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35723f0640 1 -- 172.21.15.84:0/1078137059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f356c1044e0 msgr2=0x7f356c101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.612+0000 7f35723f0640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f356c1044e0 0x7f356c101d50 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f356000b4a0 tx=0x7f356000b960 comp rx=0 tx=0).stop 2024-04-26T18:36:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.613+0000 7f35723f0640 1 -- 172.21.15.84:0/1078137059 shutdown_connections 2024-04-26T18:36:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.613+0000 7f35723f0640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f35540a3df0 0x7f35540a62b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.613+0000 7f35723f0640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f356c10cf00 0x7f356c102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.613+0000 7f35723f0640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f356c1044e0 0x7f356c101d50 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.613+0000 7f35723f0640 1 --2- 172.21.15.84:0/1078137059 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f356c103bc0 0x7f356c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.613+0000 7f35723f0640 1 -- 172.21.15.84:0/1078137059 >> 172.21.15.84:0/1078137059 conn(0x7f356c0789b0 msgr2=0x7f356c0fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.613+0000 7f35723f0640 1 -- 172.21.15.84:0/1078137059 shutdown_connections 2024-04-26T18:36:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:19.613+0000 7f35723f0640 1 -- 172.21.15.84:0/1078137059 wait complete. 2024-04-26T18:36:19.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:19 smithi174 ceph-mon[32705]: pgmap v259: 97 pgs: 97 active+clean; 1.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-04-26T18:36:19.830 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:19 smithi103 ceph-mon[32727]: pgmap v259: 97 pgs: 97 active+clean; 1.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-04-26T18:36:19.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:19 smithi084 ceph-mon[29141]: pgmap v259: 97 pgs: 97 active+clean; 1.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 49 KiB/s wr, 19 op/s 2024-04-26T18:36:21.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:21 smithi174 ceph-mon[32705]: pgmap v260: 97 pgs: 97 active+clean; 1.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-26T18:36:21.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:21 smithi084 ceph-mon[29141]: pgmap v260: 97 pgs: 97 active+clean; 1.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-26T18:36:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:21 smithi103 ceph-mon[32727]: pgmap v260: 97 pgs: 97 active+clean; 1.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-26T18:36:23.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:23 smithi174 ceph-mon[32705]: pgmap v261: 97 pgs: 97 active+clean; 1.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-26T18:36:23.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:23 smithi084 ceph-mon[29141]: pgmap v261: 97 pgs: 97 active+clean; 1.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-26T18:36:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:23 smithi103 ceph-mon[32727]: pgmap v261: 97 pgs: 97 active+clean; 1.7 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-26T18:36:25.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:25 smithi174 ceph-mon[32705]: pgmap v262: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 20 op/s 2024-04-26T18:36:25.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:25 smithi084 ceph-mon[29141]: pgmap v262: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 20 op/s 2024-04-26T18:36:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:25 smithi103 ceph-mon[32727]: pgmap v262: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 20 op/s 2024-04-26T18:36:27.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:27 smithi174 ceph-mon[32705]: pgmap v263: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-26T18:36:27.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:27 smithi084 ceph-mon[29141]: pgmap v263: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-26T18:36:27.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:27 smithi103 ceph-mon[32727]: pgmap v263: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-26T18:36:29.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:29 smithi174 ceph-mon[32705]: pgmap v264: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-26T18:36:29.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:29 smithi084 ceph-mon[29141]: pgmap v264: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-26T18:36:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:29 smithi103 ceph-mon[32727]: pgmap v264: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-26T18:36:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:31 smithi174 ceph-mon[32705]: pgmap v265: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-26T18:36:31.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:31 smithi084 ceph-mon[29141]: pgmap v265: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-26T18:36:31.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:31 smithi103 ceph-mon[32727]: pgmap v265: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-04-26T18:36:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.701+0000 7f471d566640 1 -- 172.21.15.84:0/622181896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4718101a50 msgr2=0x7f4718107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.701+0000 7f471d566640 1 --2- 172.21.15.84:0/622181896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4718101a50 0x7f4718107580 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f4700009900 tx=0x7f470002a410 comp rx=0 tx=0).stop 2024-04-26T18:36:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.701+0000 7f471d566640 1 -- 172.21.15.84:0/622181896 shutdown_connections 2024-04-26T18:36:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.701+0000 7f471d566640 1 --2- 172.21.15.84:0/622181896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4718107c60 0x7f471810f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.701+0000 7f471d566640 1 --2- 172.21.15.84:0/622181896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4718101a50 0x7f4718107580 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.701+0000 7f471d566640 1 --2- 172.21.15.84:0/622181896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47181010a0 0x7f4718101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.701+0000 7f471d566640 1 -- 172.21.15.84:0/622181896 >> 172.21.15.84:0/622181896 conn(0x7f4718077dc0 msgr2=0x7f47180ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.701+0000 7f471d566640 1 -- 172.21.15.84:0/622181896 shutdown_connections 2024-04-26T18:36:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.701+0000 7f471d566640 1 -- 172.21.15.84:0/622181896 wait complete. 2024-04-26T18:36:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f471d566640 1 Processor -- start 2024-04-26T18:36:32.704 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f471d566640 1 -- start start 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f471d566640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47181010a0 0x7f4718199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f471d566640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4718107c60 0x7f4718199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f471d566640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471819da00 0x7f471819deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f471d566640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4718112c00 con 0x7f471819da00 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f471d566640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4718112a80 con 0x7f4718107c60 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f471d566640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4718112d80 con 0x7f47181010a0 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f47177fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471819da00 0x7f471819deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f47177fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471819da00 0x7f471819deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43684/0 (socket says 172.21.15.84:43684) 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f47177fe640 1 -- 172.21.15.84:0/1733560650 learned_addr learned my addr 172.21.15.84:0/1733560650 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f470ffff640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4718107c60 0x7f4718199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f4716ffd640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47181010a0 0x7f4718199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f47177fe640 1 -- 172.21.15.84:0/1733560650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47181010a0 msgr2=0x7f4718199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f47177fe640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47181010a0 0x7f4718199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.705 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f47177fe640 1 -- 172.21.15.84:0/1733560650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4718107c60 msgr2=0x7f4718199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f47177fe640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4718107c60 0x7f4718199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.702+0000 7f47177fe640 1 -- 172.21.15.84:0/1733560650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f471819e4f0 con 0x7f471819da00 2024-04-26T18:36:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.703+0000 7f47177fe640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471819da00 0x7f471819deb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f471000eaa0 tx=0x7f471000eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.703+0000 7f4714ff9640 1 -- 172.21.15.84:0/1733560650 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4710004d20 con 0x7f471819da00 2024-04-26T18:36:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.703+0000 7f4714ff9640 1 -- 172.21.15.84:0/1733560650 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4710005020 con 0x7f471819da00 2024-04-26T18:36:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.703+0000 7f471d566640 1 -- 172.21.15.84:0/1733560650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f471819e7e0 con 0x7f471819da00 2024-04-26T18:36:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.703+0000 7f4714ff9640 1 -- 172.21.15.84:0/1733560650 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47100116b0 con 0x7f471819da00 2024-04-26T18:36:32.706 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.703+0000 7f471d566640 1 -- 172.21.15.84:0/1733560650 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47181a5f50 con 0x7f471819da00 2024-04-26T18:36:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.705+0000 7f4714ff9640 1 -- 172.21.15.84:0/1733560650 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f47100051d0 con 0x7f471819da00 2024-04-26T18:36:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.705+0000 7f472027e740 1 Processor -- start 2024-04-26T18:36:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.705+0000 7f472027e740 1 -- start start 2024-04-26T18:36:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f472027e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbff9f00 0x558bfbffa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f472027e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfbffa820 0x558bfc004ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f472027e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfc0050e0 0x558bfc0074d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.713 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f472027e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558bfbf60460 con 0x558bfbffa820 2024-04-26T18:36:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f472027e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558bfbf1a590 con 0x558bfc0050e0 2024-04-26T18:36:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f472027e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558bfbf5c420 con 0x558bfbff9f00 2024-04-26T18:36:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f4714ff9640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f47040a3dc0 0x7f47040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f470ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbff9f00 0x558bfbffa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f470ffff640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbff9f00 0x558bfbffa2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.84:40864/0 (socket says 172.21.15.84:40864) 2024-04-26T18:36:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f470ffff640 1 -- 172.21.15.84:0/1123159475 learned_addr learned my addr 172.21.15.84:0/1123159475 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f4716ffd640 1 --2- 172.21.15.84:0/1123159475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfbffa820 0x558bfc004ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f4714ff9640 1 -- 172.21.15.84:0/1733560650 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7f47100d09f0 con 0x7f471819da00 2024-04-26T18:36:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f470ffff640 1 --2- 172.21.15.84:0/1123159475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfc0050e0 0x558bfc0074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.706+0000 7f47177fe640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f47040a3dc0 0x7f47040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.707+0000 7f470ffff640 1 -- 172.21.15.84:0/1123159475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfc0050e0 msgr2=0x558bfc0074d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.707+0000 7f470ffff640 1 --2- 172.21.15.84:0/1123159475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfc0050e0 0x558bfc0074d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.714 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.707+0000 7f470ffff640 1 -- 172.21.15.84:0/1123159475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfbffa820 msgr2=0x558bfc004ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.707+0000 7f470ffff640 1 --2- 172.21.15.84:0/1123159475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfbffa820 0x558bfc004ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.707+0000 7f470ffff640 1 -- 172.21.15.84:0/1123159475 --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bfbf270d0 con 0x558bfbff9f00 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.707+0000 7f47177fe640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f47040a3dc0 0x7f47040a6280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f47100a5090 tx=0x7f4710087040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.707+0000 7f470ffff640 1 --2- 172.21.15.84:0/1123159475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbff9f00 0x558bfbffa2e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47180750b0 tx=0x7f4700005780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23a073fae7bb5dc0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.707+0000 7f470d7fa640 1 -- 172.21.15.84:0/1123159475 <== mon.1 v2:172.21.15.174:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700035070 con 0x558bfbff9f00 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.707+0000 7f470d7fa640 1 -- 172.21.15.84:0/1123159475 <== mon.1 v2:172.21.15.174:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47000035f0 con 0x558bfbff9f00 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.707+0000 7f470d7fa640 1 -- 172.21.15.84:0/1123159475 <== mon.1 v2:172.21.15.174:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47000044e0 con 0x558bfbff9f00 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 -- 172.21.15.84:0/1123159475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbff9f00 msgr2=0x558bfbffa2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 --2- 172.21.15.84:0/1123159475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbff9f00 0x558bfbffa2e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f47180750b0 tx=0x7f4700005780 comp rx=0 tx=0).stop 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 -- 172.21.15.84:0/1123159475 shutdown_connections 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 --2- 172.21.15.84:0/1123159475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfc0050e0 0x558bfc0074d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 --2- 172.21.15.84:0/1123159475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfbffa820 0x558bfc004ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 --2- 172.21.15.84:0/1123159475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbff9f00 0x558bfbffa2e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 -- 172.21.15.84:0/1123159475 >> 172.21.15.84:0/1123159475 conn(0x558bfbff6410 msgr2=0x558bfbff67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:32.715 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 -- 172.21.15.84:0/1123159475 shutdown_connections 2024-04-26T18:36:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 -- 172.21.15.84:0/1123159475 wait complete. 2024-04-26T18:36:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 -- start start 2024-04-26T18:36:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 Processor -- start 2024-04-26T18:36:32.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfbff9f00 0x558bfc002090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbffa820 0x558bfc0025d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 0x558bfc002b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558bfc007cb0 con 0x558bfc0050e0 2024-04-26T18:36:32.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558bfbee7b60 con 0x558bfbff9f00 2024-04-26T18:36:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.708+0000 7f472027e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558bfc007e30 con 0x558bfbffa820 2024-04-26T18:36:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f472027e740 10 client.0 _ll_register_callbacks cb 0x558bfbf0c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f470ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 0x558bfc002b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f470ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 0x558bfc002b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43700/0 (socket says 172.21.15.84:43700) 2024-04-26T18:36:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f470ffff640 1 -- 172.21.15.84:0/870385560 learned_addr learned my addr 172.21.15.84:0/870385560 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f4716ffd640 1 --2- 172.21.15.84:0/870385560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbffa820 0x558bfc0025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f470ffff640 1 -- 172.21.15.84:0/870385560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbffa820 msgr2=0x558bfc0025d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f470ffff640 1 --2- 172.21.15.84:0/870385560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbffa820 0x558bfc0025d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f470ffff640 1 -- 172.21.15.84:0/870385560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfbff9f00 msgr2=0x558bfc002090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f470ffff640 1 --2- 172.21.15.84:0/870385560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfbff9f00 0x558bfc002090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f470ffff640 1 -- 172.21.15.84:0/870385560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bfbc72210 con 0x558bfc0050e0 2024-04-26T18:36:32.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f4716ffd640 1 --2- 172.21.15.84:0/870385560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbffa820 0x558bfc0025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f470ffff640 1 --2- 172.21.15.84:0/870385560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 0x558bfc002b10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4700005870 tx=0x7f47000058a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9d0c9400d58b84a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f46fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f46fe7fc640 1 -- 172.21.15.84:0/870385560 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700035040 con 0x558bfc0050e0 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f46fe7fc640 1 -- 172.21.15.84:0/870385560 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7f4700004190 con 0x558bfc0050e0 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f472027e740 1 -- 172.21.15.84:0/870385560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47000035f0 con 0x558bfc0050e0 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f46fe7fc640 1 -- 172.21.15.84:0/870385560 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4700004390 con 0x558bfc0050e0 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f472027e740 10 client.16437 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.709+0000 7f472027e740 1 -- 172.21.15.84:0/870385560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558bfbd3e620 con 0x558bfc0050e0 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.710+0000 7f46fe7fc640 1 -- 172.21.15.84:0/870385560 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47000035f0 con 0x558bfc0050e0 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.710+0000 7f46fe7fc640 1 -- 172.21.15.84:0/870385560 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4700053ac0 con 0x558bfc0050e0 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.710+0000 7f472027e740 10 client.16437 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.710+0000 7f472027e740 10 client.16437 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.710+0000 7f472027e740 20 client.16437 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:32.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.710+0000 7f472027e740 1 --2- 172.21.15.84:0/870385560 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558bfc015070 0x558bfc017460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.710+0000 7f472027e740 4 client.16437 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.710+0000 7f472027e740 1 -- 172.21.15.84:0/870385560 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558bfbf6e070 con 0x558bfc015070 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.710+0000 7f4716ffd640 1 --2- 172.21.15.84:0/870385560 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558bfc015070 0x558bfc017460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.710+0000 7f4716ffd640 1 --2- 172.21.15.84:0/870385560 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558bfc015070 0x558bfc017460 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff158963c18bbab2 server_cookie=11f93a18bac5ea51 in_seq=0 out_seq=0 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.710+0000 7f46fe7fc640 10 client.16437 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f46fe7fc640 1 -- 172.21.15.84:0/870385560 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558bfbf6e070 con 0x558bfc015070 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f46fe7fc640 10 client.16437 handle_command_reply: tid=0 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f472027e740 1 client.16437 shutdown 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f472027e740 1 -- 172.21.15.84:0/870385560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 msgr2=0x558bfc002b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f472027e740 1 --2- 172.21.15.84:0/870385560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 0x558bfc002b10 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f4700005870 tx=0x7f47000058a0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f472027e740 1 -- 172.21.15.84:0/870385560 shutdown_connections 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f472027e740 1 --2- 172.21.15.84:0/870385560 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558bfc015070 0x558bfc017460 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f472027e740 1 --2- 172.21.15.84:0/870385560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 0x558bfc002b10 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f472027e740 1 --2- 172.21.15.84:0/870385560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbffa820 0x558bfc0025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f46fe7fc640 0 client.16437 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f472027e740 1 --2- 172.21.15.84:0/870385560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfbff9f00 0x558bfc002090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f472027e740 1 -- 172.21.15.84:0/870385560 >> 172.21.15.84:0/870385560 conn(0x558bfbff6410 msgr2=0x558bfbffc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f472027e740 1 -- 172.21.15.84:0/870385560 shutdown_connections 2024-04-26T18:36:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f472027e740 1 -- 172.21.15.84:0/870385560 wait complete. 2024-04-26T18:36:32.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.711+0000 7f472027e740 20 client.16437 trim_cache size 0 max 16384 2024-04-26T18:36:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f472027e740 1 Processor -- start 2024-04-26T18:36:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f472027e740 1 -- start start 2024-04-26T18:36:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f472027e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfbff9f00 0x558bfc003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f472027e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbffa820 0x558bfc01a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f472027e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 0x558bfc01a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f472027e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f47181ae640 con 0x558bfc0050e0 2024-04-26T18:36:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f472027e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47181b0180 con 0x558bfbff9f00 2024-04-26T18:36:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f472027e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47181aed10 con 0x558bfbffa820 2024-04-26T18:36:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f470ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 0x558bfc01a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f470ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 0x558bfc01a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43706/0 (socket says 172.21.15.84:43706) 2024-04-26T18:36:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f470ffff640 1 -- 172.21.15.84:0/1497346995 learned_addr learned my addr 172.21.15.84:0/1497346995 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f4716ffd640 1 --2- 172.21.15.84:0/1497346995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbffa820 0x558bfc01a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f470ffff640 1 -- 172.21.15.84:0/1497346995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbffa820 msgr2=0x558bfc01a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f470ffff640 1 --2- 172.21.15.84:0/1497346995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbffa820 0x558bfc01a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f470ffff640 1 -- 172.21.15.84:0/1497346995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfbff9f00 msgr2=0x558bfc003fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f470ffff640 1 --2- 172.21.15.84:0/1497346995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfbff9f00 0x558bfc003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.726+0000 7f470ffff640 1 -- 172.21.15.84:0/1497346995 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558bfbff67f0 con 0x558bfc0050e0 2024-04-26T18:36:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.727+0000 7f470ffff640 1 --2- 172.21.15.84:0/1497346995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 0x558bfc01a9c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4700001eb0 tx=0x7f4700005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c8c488329bb684 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:32.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.727+0000 7f470cff9640 1 -- 172.21.15.84:0/1497346995 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470005f020 con 0x558bfc0050e0 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.727+0000 7f470cff9640 1 -- 172.21.15.84:0/1497346995 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f470004e070 con 0x558bfc0050e0 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.727+0000 7f470cff9640 1 -- 172.21.15.84:0/1497346995 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4700035040 con 0x558bfc0050e0 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.727+0000 7f472027e740 1 -- 172.21.15.84:0/1497346995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 msgr2=0x558bfc01a9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.727+0000 7f472027e740 1 --2- 172.21.15.84:0/1497346995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 0x558bfc01a9c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f4700001eb0 tx=0x7f4700005c90 comp rx=0 tx=0).stop 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 -- 172.21.15.84:0/1497346995 shutdown_connections 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 --2- 172.21.15.84:0/1497346995 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfc0050e0 0x558bfc01a9c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 --2- 172.21.15.84:0/1497346995 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbffa820 0x558bfc01a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 --2- 172.21.15.84:0/1497346995 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfbff9f00 0x558bfc003fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 -- 172.21.15.84:0/1497346995 >> 172.21.15.84:0/1497346995 conn(0x558bfbff6410 msgr2=0x558bfc0038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 -- 172.21.15.84:0/1497346995 shutdown_connections 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 -- 172.21.15.84:0/1497346995 wait complete. 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 -- start start 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 Processor -- start 2024-04-26T18:36:32.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbff9f00 0x558bfc01a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfbffa820 0x558bfc01a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfc0050e0 0x558bfc018710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f47181afab0 con 0x558bfbffa820 2024-04-26T18:36:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47181af3e0 con 0x558bfc0050e0 2024-04-26T18:36:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47181ac640 con 0x558bfbff9f00 2024-04-26T18:36:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f472027e740 10 client.0 _ll_register_callbacks cb 0x7f47080093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f4716ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfbffa820 0x558bfc01a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f4716ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfbffa820 0x558bfc01a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43720/0 (socket says 172.21.15.84:43720) 2024-04-26T18:36:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f4716ffd640 1 -- 172.21.15.84:0/2230562465 learned_addr learned my addr 172.21.15.84:0/2230562465 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f470ffff640 1 --2- 172.21.15.84:0/2230562465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfc0050e0 0x558bfc018710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.728+0000 7f470ffff640 1 --2- 172.21.15.84:0/2230562465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbff9f00 0x558bfc01a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f4716ffd640 1 -- 172.21.15.84:0/2230562465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbff9f00 msgr2=0x558bfc01a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:32.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f4716ffd640 1 --2- 172.21.15.84:0/2230562465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbff9f00 0x558bfc01a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f4716ffd640 1 -- 172.21.15.84:0/2230562465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfc0050e0 msgr2=0x558bfc018710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f4716ffd640 1 --2- 172.21.15.84:0/2230562465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfc0050e0 0x558bfc018710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f4716ffd640 1 -- 172.21.15.84:0/2230562465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558bfc018f10 con 0x558bfbffa820 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f4716ffd640 1 --2- 172.21.15.84:0/2230562465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfbffa820 0x558bfc01a790 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x558bfc0018f0 tx=0x7f4708009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2a907c583dd5fb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f46fe7fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f46fe7fc640 1 -- 172.21.15.84:0/2230562465 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4708016d30 con 0x558bfbffa820 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f46fe7fc640 1 -- 172.21.15.84:0/2230562465 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7f470801d1b0 con 0x558bfbffa820 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f472027e740 1 -- 172.21.15.84:0/2230562465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f470004e070 con 0x558bfbffa820 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f46fe7fc640 1 -- 172.21.15.84:0/2230562465 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f470801d3b0 con 0x558bfbffa820 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f472027e740 10 client.16443 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f472027e740 1 -- 172.21.15.84:0/2230562465 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558bfc07bf30 con 0x558bfbffa820 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f46fe7fc640 1 -- 172.21.15.84:0/2230562465 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f470004e070 con 0x558bfbffa820 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f46fe7fc640 1 -- 172.21.15.84:0/2230562465 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f470802ca80 con 0x558bfbffa820 2024-04-26T18:36:32.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f472027e740 10 client.16443 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f472027e740 10 client.16443 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:36:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f472027e740 20 client.16443 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.729+0000 7f472027e740 1 --2- 172.21.15.84:0/2230562465 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558bfc07c6e0 0x558bfc07eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.730+0000 7f472027e740 4 client.16443 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:36:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.730+0000 7f472027e740 1 -- 172.21.15.84:0/2230562465 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558bfbf67ee0 con 0x558bfc07c6e0 2024-04-26T18:36:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.730+0000 7f470ffff640 1 --2- 172.21.15.84:0/2230562465 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558bfc07c6e0 0x558bfc07eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.730+0000 7f470ffff640 1 --2- 172.21.15.84:0/2230562465 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558bfc07c6e0 0x558bfc07eb00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86f2c52fbd16a432 server_cookie=551a97c4355aca32 in_seq=0 out_seq=0 2024-04-26T18:36:32.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.730+0000 7f46fe7fc640 10 client.16443 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.742+0000 7f46fe7fc640 1 -- 172.21.15.84:0/2230562465 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558bfbf67ee0 con 0x558bfc07c6e0 2024-04-26T18:36:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.742+0000 7f46fe7fc640 10 client.16443 handle_command_reply: tid=0 2024-04-26T18:36:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f472027e740 1 client.16443 shutdown 2024-04-26T18:36:32.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f472027e740 1 -- 172.21.15.84:0/2230562465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfbffa820 msgr2=0x558bfc01a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f472027e740 1 --2- 172.21.15.84:0/2230562465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfbffa820 0x558bfc01a790 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x558bfc0018f0 tx=0x7f4708009f60 comp rx=0 tx=0).stop 2024-04-26T18:36:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f472027e740 1 -- 172.21.15.84:0/2230562465 shutdown_connections 2024-04-26T18:36:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f472027e740 1 --2- 172.21.15.84:0/2230562465 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558bfc07c6e0 0x558bfc07eb00 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f472027e740 1 --2- 172.21.15.84:0/2230562465 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558bfc0050e0 0x558bfc018710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f472027e740 1 --2- 172.21.15.84:0/2230562465 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558bfbffa820 0x558bfc01a790 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f472027e740 1 --2- 172.21.15.84:0/2230562465 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558bfbff9f00 0x558bfc01a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f472027e740 1 -- 172.21.15.84:0/2230562465 >> 172.21.15.84:0/2230562465 conn(0x558bfbff6410 msgr2=0x558bfc0028e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f46fe7fc640 0 client.16443 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:32.744 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f472027e740 1 -- 172.21.15.84:0/2230562465 shutdown_connections 2024-04-26T18:36:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f472027e740 1 -- 172.21.15.84:0/2230562465 wait complete. 2024-04-26T18:36:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f472027e740 20 client.16443 trim_cache size 0 max 16384 2024-04-26T18:36:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:36:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:36:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:36:32.745 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:36:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f471d566640 1 -- 172.21.15.84:0/1733560650 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f47040a3dc0 msgr2=0x7f47040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f471d566640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f47040a3dc0 0x7f47040a6280 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f47100a5090 tx=0x7f4710087040 comp rx=0 tx=0).stop 2024-04-26T18:36:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f471d566640 1 -- 172.21.15.84:0/1733560650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471819da00 msgr2=0x7f471819deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f471d566640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471819da00 0x7f471819deb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f471000eaa0 tx=0x7f471000eea0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f471d566640 1 -- 172.21.15.84:0/1733560650 shutdown_connections 2024-04-26T18:36:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f471d566640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f47040a3dc0 0x7f47040a6280 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.746 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f471d566640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f471819da00 0x7f471819deb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f471d566640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4718107c60 0x7f4718199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f471d566640 1 --2- 172.21.15.84:0/1733560650 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47181010a0 0x7f4718199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.743+0000 7f471d566640 1 -- 172.21.15.84:0/1733560650 >> 172.21.15.84:0/1733560650 conn(0x7f4718077dc0 msgr2=0x7f47180ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.744+0000 7f471d566640 1 -- 172.21.15.84:0/1733560650 shutdown_connections 2024-04-26T18:36:32.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:32.744+0000 7f471d566640 1 -- 172.21.15.84:0/1733560650 wait complete. 2024-04-26T18:36:33.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:33 smithi174 ceph-mon[32705]: pgmap v266: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-04-26T18:36:33.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:33 smithi084 ceph-mon[29141]: pgmap v266: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-04-26T18:36:33.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:33 smithi103 ceph-mon[32727]: pgmap v266: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 7.0 KiB/s wr, 11 op/s 2024-04-26T18:36:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:35 smithi174 ceph-mon[32705]: pgmap v267: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-26T18:36:35.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:35 smithi084 ceph-mon[29141]: pgmap v267: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-26T18:36:35.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:35 smithi103 ceph-mon[32727]: pgmap v267: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 17 op/s 2024-04-26T18:36:37.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:37 smithi174 ceph-mon[32705]: pgmap v268: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 10 op/s 2024-04-26T18:36:37.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:37 smithi084 ceph-mon[29141]: pgmap v268: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 10 op/s 2024-04-26T18:36:37.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:37 smithi103 ceph-mon[32727]: pgmap v268: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 10 op/s 2024-04-26T18:36:39.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:39 smithi174 ceph-mon[32705]: pgmap v269: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 10 op/s 2024-04-26T18:36:39.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:39 smithi084 ceph-mon[29141]: pgmap v269: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 10 op/s 2024-04-26T18:36:39.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:39 smithi103 ceph-mon[32727]: pgmap v269: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 10 op/s 2024-04-26T18:36:41.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:41 smithi174 ceph-mon[32705]: pgmap v270: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-04-26T18:36:41.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:41 smithi084 ceph-mon[29141]: pgmap v270: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-04-26T18:36:41.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:41 smithi103 ceph-mon[32727]: pgmap v270: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-04-26T18:36:43.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:43 smithi174 ceph-mon[32705]: pgmap v271: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 10 op/s 2024-04-26T18:36:43.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:43 smithi084 ceph-mon[29141]: pgmap v271: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 10 op/s 2024-04-26T18:36:43.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:43 smithi103 ceph-mon[32727]: pgmap v271: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.9 KiB/s wr, 10 op/s 2024-04-26T18:36:45.613 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:36:45.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:45 smithi174 ceph-mon[32705]: pgmap v272: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 15 op/s 2024-04-26T18:36:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.835+0000 7efe852d1640 1 -- 172.21.15.84:0/2188398133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe80104500 msgr2=0x7efe8010c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.835+0000 7efe852d1640 1 --2- 172.21.15.84:0/2188398133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe80104500 0x7efe8010c9e0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7efe68009900 tx=0x7efe6802a480 comp rx=0 tx=0).stop 2024-04-26T18:36:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.835+0000 7efe852d1640 1 -- 172.21.15.84:0/2188398133 shutdown_connections 2024-04-26T18:36:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.835+0000 7efe852d1640 1 --2- 172.21.15.84:0/2188398133 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe8010cfb0 0x7efe8010f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.835+0000 7efe852d1640 1 --2- 172.21.15.84:0/2188398133 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe80104500 0x7efe8010c9e0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.835+0000 7efe852d1640 1 --2- 172.21.15.84:0/2188398133 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe80103be0 0x7efe80103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.835+0000 7efe852d1640 1 -- 172.21.15.84:0/2188398133 >> 172.21.15.84:0/2188398133 conn(0x7efe80077bb0 msgr2=0x7efe80077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.835+0000 7efe852d1640 1 -- 172.21.15.84:0/2188398133 shutdown_connections 2024-04-26T18:36:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.835+0000 7efe852d1640 1 -- 172.21.15.84:0/2188398133 wait complete. 2024-04-26T18:36:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.836+0000 7efe852d1640 1 Processor -- start 2024-04-26T18:36:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.836+0000 7efe852d1640 1 -- start start 2024-04-26T18:36:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe852d1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe80103be0 0x7efe801990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe852d1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe80104500 0x7efe801995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe852d1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe8010cfb0 0x7efe8019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe852d1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe80112bf0 con 0x7efe80103be0 2024-04-26T18:36:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe852d1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe80112a70 con 0x7efe8010cfb0 2024-04-26T18:36:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe852d1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efe80112d70 con 0x7efe80104500 2024-04-26T18:36:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe7effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe80103be0 0x7efe801990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe7effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe80103be0 0x7efe801990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47954/0 (socket says 172.21.15.84:47954) 2024-04-26T18:36:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe7effd640 1 -- 172.21.15.84:0/139190515 learned_addr learned my addr 172.21.15.84:0/139190515 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe77fff640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe80104500 0x7efe801995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe7f7fe640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe8010cfb0 0x7efe8019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe7effd640 1 -- 172.21.15.84:0/139190515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe80104500 msgr2=0x7efe801995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe7effd640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe80104500 0x7efe801995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe7effd640 1 -- 172.21.15.84:0/139190515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe8010cfb0 msgr2=0x7efe8019d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:45.840 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe7effd640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe8010cfb0 0x7efe8019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe7effd640 1 -- 172.21.15.84:0/139190515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe8019e050 con 0x7efe80103be0 2024-04-26T18:36:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe7effd640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe80103be0 0x7efe801990b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efe7000d950 tx=0x7efe7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe7cff9640 1 -- 172.21.15.84:0/139190515 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70017030 con 0x7efe80103be0 2024-04-26T18:36:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe852d1640 1 -- 172.21.15.84:0/139190515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe8019e340 con 0x7efe80103be0 2024-04-26T18:36:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe7cff9640 1 -- 172.21.15.84:0/139190515 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe70004020 con 0x7efe80103be0 2024-04-26T18:36:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe7cff9640 1 -- 172.21.15.84:0/139190515 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe700047a0 con 0x7efe80103be0 2024-04-26T18:36:45.841 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.837+0000 7efe852d1640 1 -- 172.21.15.84:0/139190515 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe801a5bd0 con 0x7efe80103be0 2024-04-26T18:36:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.840+0000 7efe7cff9640 1 -- 172.21.15.84:0/139190515 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7efe70009a30 con 0x7efe80103be0 2024-04-26T18:36:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.840+0000 7efe7cff9640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7efe6c0a3dc0 0x7efe6c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.840+0000 7efe77fff640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7efe6c0a3dc0 0x7efe6c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.840+0000 7efe7cff9640 1 -- 172.21.15.84:0/139190515 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7efe700d02a0 con 0x7efe80103be0 2024-04-26T18:36:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.841+0000 7efe77fff640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7efe6c0a3dc0 0x7efe6c0a6280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efe68031440 tx=0x7efe68005b50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.841+0000 7efe87fec740 1 Processor -- start 2024-04-26T18:36:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.841+0000 7efe87fec740 1 -- start start 2024-04-26T18:36:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.841+0000 7efe87fec740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848aff00 0x55a7848b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.841+0000 7efe87fec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848b0820 0x55a7848baba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.841+0000 7efe87fec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 0x55a7848bd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.841+0000 7efe87fec740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a784816460 con 0x55a7848aff00 2024-04-26T18:36:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.841+0000 7efe87fec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a7847d0590 con 0x55a7848bb0e0 2024-04-26T18:36:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.841+0000 7efe87fec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a784812420 con 0x55a7848b0820 2024-04-26T18:36:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.842+0000 7efe7effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848aff00 0x55a7848b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.842+0000 7efe7effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848aff00 0x55a7848b02e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47964/0 (socket says 172.21.15.84:47964) 2024-04-26T18:36:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.842+0000 7efe7effd640 1 -- 172.21.15.84:0/519063131 learned_addr learned my addr 172.21.15.84:0/519063131 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.842+0000 7efe7effd640 1 -- 172.21.15.84:0/519063131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848b0820 msgr2=0x55a7848baba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.842+0000 7efe7effd640 1 --2- 172.21.15.84:0/519063131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848b0820 0x55a7848baba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.842+0000 7efe7effd640 1 -- 172.21.15.84:0/519063131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 msgr2=0x55a7848bd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.842+0000 7efe7effd640 1 --2- 172.21.15.84:0/519063131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 0x55a7848bd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.842+0000 7efe7effd640 1 -- 172.21.15.84:0/519063131 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7847dd0d0 con 0x55a7848aff00 2024-04-26T18:36:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.842+0000 7efe7effd640 1 --2- 172.21.15.84:0/519063131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848aff00 0x55a7848b02e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efe70098bd0 tx=0x7efe7009c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b0b91413cacded4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.842+0000 7efe757fa640 1 -- 172.21.15.84:0/519063131 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe700b5070 con 0x55a7848aff00 2024-04-26T18:36:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.842+0000 7efe757fa640 1 -- 172.21.15.84:0/519063131 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe700a4e70 con 0x55a7848aff00 2024-04-26T18:36:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.842+0000 7efe757fa640 1 -- 172.21.15.84:0/519063131 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70098870 con 0x55a7848aff00 2024-04-26T18:36:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.843+0000 7efe87fec740 1 -- 172.21.15.84:0/519063131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848aff00 msgr2=0x55a7848b02e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.843+0000 7efe87fec740 1 --2- 172.21.15.84:0/519063131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848aff00 0x55a7848b02e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efe70098bd0 tx=0x7efe7009c9c0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.843+0000 7efe87fec740 1 -- 172.21.15.84:0/519063131 shutdown_connections 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.843+0000 7efe87fec740 1 --2- 172.21.15.84:0/519063131 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 0x55a7848bd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.843+0000 7efe87fec740 1 --2- 172.21.15.84:0/519063131 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848b0820 0x55a7848baba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.843+0000 7efe87fec740 1 --2- 172.21.15.84:0/519063131 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848aff00 0x55a7848b02e0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.843+0000 7efe87fec740 1 -- 172.21.15.84:0/519063131 >> 172.21.15.84:0/519063131 conn(0x55a7848ac410 msgr2=0x55a7848ac7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.843+0000 7efe87fec740 1 -- 172.21.15.84:0/519063131 shutdown_connections 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.843+0000 7efe87fec740 1 -- 172.21.15.84:0/519063131 wait complete. 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe87fec740 1 -- start start 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe87fec740 1 Processor -- start 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe87fec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 0x55a7848b80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe87fec740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 0x55a7848b8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe87fec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 0x55a7848b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe87fec740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55a7848bdd10 con 0x55a7848b0820 2024-04-26T18:36:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe87fec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55a78479db60 con 0x55a7848bb0e0 2024-04-26T18:36:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe87fec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55a7848bde90 con 0x55a7848aff00 2024-04-26T18:36:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe87fec740 10 client.0 _ll_register_callbacks cb 0x55a7847c22b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe77fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 0x55a7848b8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe77fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 0x55a7848b8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47980/0 (socket says 172.21.15.84:47980) 2024-04-26T18:36:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe77fff640 1 -- 172.21.15.84:0/3399922778 learned_addr learned my addr 172.21.15.84:0/3399922778 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe7effd640 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 0x55a7848b80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.849 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:45 smithi084 ceph-mon[29141]: pgmap v272: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 15 op/s 2024-04-26T18:36:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe7f7fe640 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 0x55a7848b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe77fff640 1 -- 172.21.15.84:0/3399922778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 msgr2=0x55a7848b80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.844+0000 7efe77fff640 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 0x55a7848b80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.845+0000 7efe77fff640 1 -- 172.21.15.84:0/3399922778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 msgr2=0x55a7848b8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.845+0000 7efe77fff640 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 0x55a7848b8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.845+0000 7efe77fff640 1 -- 172.21.15.84:0/3399922778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a784528210 con 0x55a7848b0820 2024-04-26T18:36:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.845+0000 7efe7f7fe640 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 0x55a7848b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:36:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.845+0000 7efe77fff640 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 0x55a7848b8630 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efe68009900 tx=0x7efe68032f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b997f356678bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:45.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.845+0000 7efe667fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.845+0000 7efe667fc640 1 -- 172.21.15.84:0/3399922778 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6804c020 con 0x55a7848b0820 2024-04-26T18:36:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.845+0000 7efe667fc640 1 -- 172.21.15.84:0/3399922778 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7efe68005030 con 0x55a7848b0820 2024-04-26T18:36:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.845+0000 7efe87fec740 1 -- 172.21.15.84:0/3399922778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe700a4e70 con 0x55a7848b0820 2024-04-26T18:36:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.845+0000 7efe667fc640 1 -- 172.21.15.84:0/3399922778 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efe68057050 con 0x55a7848b0820 2024-04-26T18:36:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.845+0000 7efe87fec740 10 client.16455 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.845+0000 7efe87fec740 1 -- 172.21.15.84:0/3399922778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a7845f4620 con 0x55a7848b0820 2024-04-26T18:36:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.845+0000 7efe667fc640 1 -- 172.21.15.84:0/3399922778 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe700a4e70 con 0x55a7848b0820 2024-04-26T18:36:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.846+0000 7efe667fc640 1 -- 172.21.15.84:0/3399922778 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7efe680035f0 con 0x55a7848b0820 2024-04-26T18:36:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.846+0000 7efe87fec740 10 client.16455 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.846+0000 7efe87fec740 10 client.16455 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:36:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.846+0000 7efe87fec740 20 client.16455 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.846+0000 7efe87fec740 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a7848cb300 0x55a7848cd6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.846+0000 7efe87fec740 4 client.16455 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:36:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.846+0000 7efe87fec740 1 -- 172.21.15.84:0/3399922778 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a784824070 con 0x55a7848cb300 2024-04-26T18:36:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.846+0000 7efe7effd640 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a7848cb300 0x55a7848cd6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.846+0000 7efe7effd640 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a7848cb300 0x55a7848cd6f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a033aa163efb1de0 server_cookie=ae55a71a39475b87 in_seq=0 out_seq=0 2024-04-26T18:36:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.846+0000 7efe667fc640 10 client.16455 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe667fc640 1 -- 172.21.15.84:0/3399922778 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a784824070 con 0x55a7848cb300 2024-04-26T18:36:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe667fc640 10 client.16455 handle_command_reply: tid=0 2024-04-26T18:36:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe87fec740 1 client.16455 shutdown 2024-04-26T18:36:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe87fec740 1 -- 172.21.15.84:0/3399922778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 msgr2=0x55a7848b8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe87fec740 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 0x55a7848b8630 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efe68009900 tx=0x7efe68032f90 comp rx=0 tx=0).stop 2024-04-26T18:36:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe87fec740 1 -- 172.21.15.84:0/3399922778 shutdown_connections 2024-04-26T18:36:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe87fec740 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a7848cb300 0x55a7848cd6f0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe87fec740 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 0x55a7848b8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe667fc640 0 client.16455 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe87fec740 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 0x55a7848b8630 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe87fec740 1 --2- 172.21.15.84:0/3399922778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 0x55a7848b80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe87fec740 1 -- 172.21.15.84:0/3399922778 >> 172.21.15.84:0/3399922778 conn(0x55a7848ac410 msgr2=0x55a7848b2140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe87fec740 1 -- 172.21.15.84:0/3399922778 shutdown_connections 2024-04-26T18:36:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe87fec740 1 -- 172.21.15.84:0/3399922778 wait complete. 2024-04-26T18:36:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.847+0000 7efe87fec740 20 client.16455 trim_cache size 0 max 16384 2024-04-26T18:36:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.873+0000 7efe87fec740 1 Processor -- start 2024-04-26T18:36:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.873+0000 7efe87fec740 1 -- start start 2024-04-26T18:36:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.873+0000 7efe87fec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 0x55a7848c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.873+0000 7efe87fec740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 0x55a7848b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe87fec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 0x55a7848b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe87fec740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe801ae6a0 con 0x55a7848b0820 2024-04-26T18:36:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe87fec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe801b01e0 con 0x55a7848bb0e0 2024-04-26T18:36:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe87fec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efe801aed70 con 0x55a7848aff00 2024-04-26T18:36:45.875 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe77fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 0x55a7848b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe77fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 0x55a7848b28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47992/0 (socket says 172.21.15.84:47992) 2024-04-26T18:36:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe77fff640 1 -- 172.21.15.84:0/3431059939 learned_addr learned my addr 172.21.15.84:0/3431059939 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe7effd640 1 --2- 172.21.15.84:0/3431059939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 0x55a7848c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe77fff640 1 -- 172.21.15.84:0/3431059939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 msgr2=0x55a7848c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe77fff640 1 --2- 172.21.15.84:0/3431059939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 0x55a7848c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe77fff640 1 -- 172.21.15.84:0/3431059939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 msgr2=0x55a7848b8990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe77fff640 1 --2- 172.21.15.84:0/3431059939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 0x55a7848b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe77fff640 1 -- 172.21.15.84:0/3431059939 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7848ad5d0 con 0x55a7848b0820 2024-04-26T18:36:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe77fff640 1 --2- 172.21.15.84:0/3431059939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 0x55a7848b28c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7efe68003c00 tx=0x7efe68004490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b382f99369956476 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.874+0000 7efe74ff9640 1 -- 172.21.15.84:0/3431059939 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6804c040 con 0x55a7848b0820 2024-04-26T18:36:45.876 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.875+0000 7efe74ff9640 1 -- 172.21.15.84:0/3431059939 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe680054e0 con 0x55a7848b0820 2024-04-26T18:36:45.877 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.875+0000 7efe74ff9640 1 -- 172.21.15.84:0/3431059939 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe68035040 con 0x55a7848b0820 2024-04-26T18:36:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:45 smithi103 ceph-mon[32727]: pgmap v272: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 15 op/s 2024-04-26T18:36:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.875+0000 7efe87fec740 1 -- 172.21.15.84:0/3431059939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 msgr2=0x55a7848b28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.875+0000 7efe87fec740 1 --2- 172.21.15.84:0/3431059939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 0x55a7848b28c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7efe68003c00 tx=0x7efe68004490 comp rx=0 tx=0).stop 2024-04-26T18:36:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.875+0000 7efe87fec740 1 -- 172.21.15.84:0/3431059939 shutdown_connections 2024-04-26T18:36:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.875+0000 7efe87fec740 1 --2- 172.21.15.84:0/3431059939 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848bb0e0 0x55a7848b8990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.875+0000 7efe87fec740 1 --2- 172.21.15.84:0/3431059939 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848b0820 0x55a7848b28c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.875+0000 7efe87fec740 1 --2- 172.21.15.84:0/3431059939 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 0x55a7848c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.875+0000 7efe87fec740 1 -- 172.21.15.84:0/3431059939 >> 172.21.15.84:0/3431059939 conn(0x55a7848ac410 msgr2=0x55a7848ba1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.875+0000 7efe87fec740 1 -- 172.21.15.84:0/3431059939 shutdown_connections 2024-04-26T18:36:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.875+0000 7efe87fec740 1 -- 172.21.15.84:0/3431059939 wait complete. 2024-04-26T18:36:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe87fec740 1 -- start start 2024-04-26T18:36:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe87fec740 1 Processor -- start 2024-04-26T18:36:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe87fec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 0x55a7848bf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe87fec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848b0820 0x55a7848bf660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe87fec740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848bb0e0 0x55a784930870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe87fec740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7efe801afb10 con 0x55a7848bb0e0 2024-04-26T18:36:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe87fec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7efe801af440 con 0x55a7848b0820 2024-04-26T18:36:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe87fec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7efe801ac6a0 con 0x55a7848aff00 2024-04-26T18:36:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe87fec740 10 client.0 _ll_register_callbacks cb 0x7efe68040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe7f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848bb0e0 0x55a784930870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe7f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848bb0e0 0x55a784930870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47996/0 (socket says 172.21.15.84:47996) 2024-04-26T18:36:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe7f7fe640 1 -- 172.21.15.84:0/1312761445 learned_addr learned my addr 172.21.15.84:0/1312761445 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe77fff640 1 --2- 172.21.15.84:0/1312761445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848b0820 0x55a7848bf660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe7effd640 1 --2- 172.21.15.84:0/1312761445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 0x55a7848bf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe7f7fe640 1 -- 172.21.15.84:0/1312761445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 msgr2=0x55a7848bf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe7f7fe640 1 --2- 172.21.15.84:0/1312761445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 0x55a7848bf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe7f7fe640 1 -- 172.21.15.84:0/1312761445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848b0820 msgr2=0x55a7848bf660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe7f7fe640 1 --2- 172.21.15.84:0/1312761445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848b0820 0x55a7848bf660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.876+0000 7efe7f7fe640 1 -- 172.21.15.84:0/1312761445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a784930db0 con 0x55a7848bb0e0 2024-04-26T18:36:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe7f7fe640 1 --2- 172.21.15.84:0/1312761445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848bb0e0 0x55a784930870 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7efe7800e630 tx=0x7efe7800eaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5217e6b1f5f92d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe667fc640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe667fc640 1 -- 172.21.15.84:0/1312761445 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78004d20 con 0x55a7848bb0e0 2024-04-26T18:36:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe667fc640 1 -- 172.21.15.84:0/1312761445 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7efe780097c0 con 0x55a7848bb0e0 2024-04-26T18:36:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe87fec740 1 -- 172.21.15.84:0/1312761445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe680054e0 con 0x55a7848bb0e0 2024-04-26T18:36:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe667fc640 1 -- 172.21.15.84:0/1312761445 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efe780099c0 con 0x55a7848bb0e0 2024-04-26T18:36:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe87fec740 10 client.16461 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe87fec740 1 -- 172.21.15.84:0/1312761445 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a784932020 con 0x55a7848bb0e0 2024-04-26T18:36:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe667fc640 1 -- 172.21.15.84:0/1312761445 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe680054e0 con 0x55a7848bb0e0 2024-04-26T18:36:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe667fc640 1 -- 172.21.15.84:0/1312761445 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7efe7801d030 con 0x55a7848bb0e0 2024-04-26T18:36:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe87fec740 10 client.16461 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe87fec740 10 client.16461 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:36:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe87fec740 20 client.16461 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe87fec740 1 --2- 172.21.15.84:0/1312761445 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a784932820 0x55a784934c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe87fec740 4 client.16461 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:36:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.877+0000 7efe87fec740 1 -- 172.21.15.84:0/1312761445 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a78481dee0 con 0x55a784932820 2024-04-26T18:36:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.878+0000 7efe7effd640 1 --2- 172.21.15.84:0/1312761445 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a784932820 0x55a784934c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.878+0000 7efe7effd640 1 --2- 172.21.15.84:0/1312761445 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a784932820 0x55a784934c40 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ee6e9c1185efe61 server_cookie=cd0f379e8fdf9225 in_seq=0 out_seq=0 2024-04-26T18:36:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.878+0000 7efe667fc640 10 client.16461 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:45.885 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:36:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe667fc640 1 -- 172.21.15.84:0/1312761445 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a78481dee0 con 0x55a784932820 2024-04-26T18:36:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe667fc640 10 client.16461 handle_command_reply: tid=0 2024-04-26T18:36:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe87fec740 1 client.16461 shutdown 2024-04-26T18:36:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe87fec740 1 -- 172.21.15.84:0/1312761445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848bb0e0 msgr2=0x55a784930870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe87fec740 1 --2- 172.21.15.84:0/1312761445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848bb0e0 0x55a784930870 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7efe7800e630 tx=0x7efe7800eaf0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe87fec740 1 -- 172.21.15.84:0/1312761445 shutdown_connections 2024-04-26T18:36:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe87fec740 1 --2- 172.21.15.84:0/1312761445 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55a784932820 0x55a784934c40 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe87fec740 1 --2- 172.21.15.84:0/1312761445 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55a7848bb0e0 0x55a784930870 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe667fc640 0 client.16461 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe87fec740 1 --2- 172.21.15.84:0/1312761445 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55a7848b0820 0x55a7848bf660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe87fec740 1 --2- 172.21.15.84:0/1312761445 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55a7848aff00 0x55a7848bf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe87fec740 1 -- 172.21.15.84:0/1312761445 >> 172.21.15.84:0/1312761445 conn(0x55a7848ac410 msgr2=0x55a7848c5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe87fec740 1 -- 172.21.15.84:0/1312761445 shutdown_connections 2024-04-26T18:36:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe87fec740 1 -- 172.21.15.84:0/1312761445 wait complete. 2024-04-26T18:36:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.888+0000 7efe87fec740 20 client.16461 trim_cache size 0 max 16384 2024-04-26T18:36:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:36:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:36:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:36:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:36:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.889+0000 7efe852d1640 1 -- 172.21.15.84:0/139190515 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7efe6c0a3dc0 msgr2=0x7efe6c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.889+0000 7efe852d1640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7efe6c0a3dc0 0x7efe6c0a6280 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efe68031440 tx=0x7efe68005b50 comp rx=0 tx=0).stop 2024-04-26T18:36:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.889+0000 7efe852d1640 1 -- 172.21.15.84:0/139190515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe80103be0 msgr2=0x7efe801990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.889+0000 7efe852d1640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe80103be0 0x7efe801990b0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7efe7000d950 tx=0x7efe7000de10 comp rx=0 tx=0).stop 2024-04-26T18:36:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.889+0000 7efe852d1640 1 -- 172.21.15.84:0/139190515 shutdown_connections 2024-04-26T18:36:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.889+0000 7efe852d1640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7efe6c0a3dc0 0x7efe6c0a6280 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.889+0000 7efe852d1640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7efe8010cfb0 0x7efe8019d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.889+0000 7efe852d1640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7efe80104500 0x7efe801995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.889+0000 7efe852d1640 1 --2- 172.21.15.84:0/139190515 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7efe80103be0 0x7efe801990b0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.889+0000 7efe852d1640 1 -- 172.21.15.84:0/139190515 >> 172.21.15.84:0/139190515 conn(0x7efe80077bb0 msgr2=0x7efe800fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.889+0000 7efe852d1640 1 -- 172.21.15.84:0/139190515 shutdown_connections 2024-04-26T18:36:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:45.889+0000 7efe852d1640 1 -- 172.21.15.84:0/139190515 wait complete. 2024-04-26T18:36:47.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.493+0000 7fec92f1b700 1 -- 172.21.15.84:0/2662818849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c102ce0 msgr2=0x7fec8c10f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:47.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.493+0000 7fec92f1b700 1 --2- 172.21.15.84:0/2662818849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c102ce0 0x7fec8c10f1d0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7fec8400b0a0 tx=0x7fec8400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:36:47.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.493+0000 7fec92f1b700 1 -- 172.21.15.84:0/2662818849 shutdown_connections 2024-04-26T18:36:47.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.493+0000 7fec92f1b700 1 --2- 172.21.15.84:0/2662818849 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c102ce0 0x7fec8c10f1d0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:47.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.493+0000 7fec92f1b700 1 --2- 172.21.15.84:0/2662818849 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec8c108390 0x7fec8c108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:47.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.493+0000 7fec92f1b700 1 --2- 172.21.15.84:0/2662818849 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec8c102320 0x7fec8c1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:47.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.493+0000 7fec92f1b700 1 -- 172.21.15.84:0/2662818849 >> 172.21.15.84:0/2662818849 conn(0x7fec8c0792d0 msgr2=0x7fec8c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:47.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.493+0000 7fec92f1b700 1 -- 172.21.15.84:0/2662818849 shutdown_connections 2024-04-26T18:36:47.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.493+0000 7fec92f1b700 1 -- 172.21.15.84:0/2662818849 wait complete. 2024-04-26T18:36:47.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec92f1b700 1 Processor -- start 2024-04-26T18:36:47.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec92f1b700 1 -- start start 2024-04-26T18:36:47.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec92f1b700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec8c102320 0x7fec8c19e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:47.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec92f1b700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec8c102ce0 0x7fec8c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:47.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec92f1b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c108390 0x7fec8c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:47.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec92f1b700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c1125b0 con 0x7fec8c108390 2024-04-26T18:36:47.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec92f1b700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c112430 con 0x7fec8c102ce0 2024-04-26T18:36:47.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec92f1b700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fec8c112730 con 0x7fec8c102320 2024-04-26T18:36:47.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec914b8700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c108390 0x7fec8c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:47.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec914b8700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c108390 0x7fec8c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55564/0 (socket says 172.21.15.84:55564) 2024-04-26T18:36:47.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec914b8700 1 -- 172.21.15.84:0/3891284618 learned_addr learned my addr 172.21.15.84:0/3891284618 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:47.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec8bfff700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec8c102ce0 0x7fec8c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:47.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec90cb7700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec8c102320 0x7fec8c19e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:47.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec914b8700 1 -- 172.21.15.84:0/3891284618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec8c102320 msgr2=0x7fec8c19e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:47.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec914b8700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec8c102320 0x7fec8c19e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:47.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec914b8700 1 -- 172.21.15.84:0/3891284618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec8c102ce0 msgr2=0x7fec8c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:47.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec914b8700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec8c102ce0 0x7fec8c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:47.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec914b8700 1 -- 172.21.15.84:0/3891284618 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec8c1990d0 con 0x7fec8c108390 2024-04-26T18:36:47.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.494+0000 7fec90cb7700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec8c102320 0x7fec8c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:36:47.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.495+0000 7fec914b8700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c108390 0x7fec8c198970 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fec840062a0 tx=0x7fec84004ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:47.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.495+0000 7fec89ffb700 1 -- 172.21.15.84:0/3891284618 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec84021070 con 0x7fec8c108390 2024-04-26T18:36:47.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.495+0000 7fec89ffb700 1 -- 172.21.15.84:0/3891284618 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec84007d00 con 0x7fec8c108390 2024-04-26T18:36:47.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.495+0000 7fec92f1b700 1 -- 172.21.15.84:0/3891284618 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec8c199360 con 0x7fec8c108390 2024-04-26T18:36:47.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.495+0000 7fec89ffb700 1 -- 172.21.15.84:0/3891284618 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec8401d450 con 0x7fec8c108390 2024-04-26T18:36:47.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.495+0000 7fec92f1b700 1 -- 172.21.15.84:0/3891284618 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec8c1a5680 con 0x7fec8c108390 2024-04-26T18:36:47.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.496+0000 7fec92f1b700 1 -- 172.21.15.84:0/3891284618 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec8c10a900 con 0x7fec8c108390 2024-04-26T18:36:47.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.497+0000 7fec89ffb700 1 -- 172.21.15.84:0/3891284618 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fec84004000 con 0x7fec8c108390 2024-04-26T18:36:47.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.498+0000 7fec89ffb700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fec7c0a3db0 0x7fec7c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:47.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.498+0000 7fec89ffb700 1 -- 172.21.15.84:0/3891284618 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fec8400f020 con 0x7fec8c108390 2024-04-26T18:36:47.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.498+0000 7fec90cb7700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fec7c0a3db0 0x7fec7c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:47.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.498+0000 7fec90cb7700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fec7c0a3db0 0x7fec7c0a6270 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec8c199ab0 tx=0x7fec80008040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:47.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.501+0000 7fec89ffb700 1 -- 172.21.15.84:0/3891284618 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec8402a020 con 0x7fec8c108390 2024-04-26T18:36:47.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:47 smithi084 ceph-mon[29141]: pgmap v273: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-26T18:36:47.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:47 smithi174 ceph-mon[32705]: pgmap v273: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-26T18:36:47.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.747+0000 7fec92f1b700 1 -- 172.21.15.84:0/3891284618 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec8c06a110 con 0x7fec8c108390 2024-04-26T18:36:47.747 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.748+0000 7fec89ffb700 1 -- 172.21.15.84:0/3891284618 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7fec8409ea00 con 0x7fec8c108390 2024-04-26T18:36:47.748 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:36:47.748 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:36:47.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.750+0000 7fec92f1b700 1 -- 172.21.15.84:0/3891284618 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fec7c0a3db0 msgr2=0x7fec7c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:47.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.750+0000 7fec92f1b700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fec7c0a3db0 0x7fec7c0a6270 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fec8c199ab0 tx=0x7fec80008040 comp rx=0 tx=0).stop 2024-04-26T18:36:47.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.750+0000 7fec92f1b700 1 -- 172.21.15.84:0/3891284618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c108390 msgr2=0x7fec8c198970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:47.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.750+0000 7fec92f1b700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c108390 0x7fec8c198970 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fec840062a0 tx=0x7fec84004ed0 comp rx=0 tx=0).stop 2024-04-26T18:36:47.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.750+0000 7fec92f1b700 1 -- 172.21.15.84:0/3891284618 shutdown_connections 2024-04-26T18:36:47.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.750+0000 7fec92f1b700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fec7c0a3db0 0x7fec7c0a6270 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:47.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.750+0000 7fec92f1b700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fec8c108390 0x7fec8c198970 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:47.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.750+0000 7fec92f1b700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fec8c102320 0x7fec8c19e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:47.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.750+0000 7fec92f1b700 1 --2- 172.21.15.84:0/3891284618 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fec8c102ce0 0x7fec8c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:47.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.750+0000 7fec92f1b700 1 -- 172.21.15.84:0/3891284618 >> 172.21.15.84:0/3891284618 conn(0x7fec8c0792d0 msgr2=0x7fec8c101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:47.750 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.750+0000 7fec92f1b700 1 -- 172.21.15.84:0/3891284618 shutdown_connections 2024-04-26T18:36:47.751 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:47.750+0000 7fec92f1b700 1 -- 172.21.15.84:0/3891284618 wait complete. 2024-04-26T18:36:47.751 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:36:47.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:47 smithi103 ceph-mon[32727]: pgmap v273: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-26T18:36:48.465 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id db6a0f03 --release --await' 2024-04-26T18:36:48.465 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db --set-id db6a0f03 --release --await 2024-04-26T18:36:48.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:48 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3891284618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:36:48.727 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:36:48.756 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:48 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3891284618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:36:48.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:48 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3891284618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:36:49.709 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:49 smithi084 ceph-mon[29141]: pgmap v274: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-26T18:36:49.778 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:49 smithi103 ceph-mon[32727]: pgmap v274: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-26T18:36:49.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:49 smithi174 ceph-mon[32705]: pgmap v274: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 682 B/s wr, 8 op/s 2024-04-26T18:36:50.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.490+0000 7fd46bbe6700 1 -- 172.21.15.84:0/2467328667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd464101aa0 msgr2=0x7fd464101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:50.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.490+0000 7fd46bbe6700 1 --2- 172.21.15.84:0/2467328667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd464101aa0 0x7fd464101f20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fd454009990 tx=0x7fd454009ca0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.490+0000 7fd46bbe6700 1 -- 172.21.15.84:0/2467328667 shutdown_connections 2024-04-26T18:36:50.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.490+0000 7fd46bbe6700 1 --2- 172.21.15.84:0/2467328667 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd464102460 0x7fd46410e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.490+0000 7fd46bbe6700 1 --2- 172.21.15.84:0/2467328667 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd464107aa0 0x7fd464107e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.490+0000 7fd46bbe6700 1 --2- 172.21.15.84:0/2467328667 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd464101aa0 0x7fd464101f20 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.490+0000 7fd46bbe6700 1 -- 172.21.15.84:0/2467328667 >> 172.21.15.84:0/2467328667 conn(0x7fd464078920 msgr2=0x7fd4640ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.490+0000 7fd46bbe6700 1 -- 172.21.15.84:0/2467328667 shutdown_connections 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.490+0000 7fd46bbe6700 1 -- 172.21.15.84:0/2467328667 wait complete. 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.490+0000 7fd46bbe6700 1 Processor -- start 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46bbe6700 1 -- start start 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46bbe6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd464101aa0 0x7fd46419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46bbe6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd464102460 0x7fd46419fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46bbe6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd464107aa0 0x7fd464199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46bbe6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd464111450 con 0x7fd464107aa0 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46bbe6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641112d0 con 0x7fd464101aa0 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46bbe6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641115d0 con 0x7fd464102460 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46a183700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd464107aa0 0x7fd464199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46a183700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd464107aa0 0x7fd464199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55568/0 (socket says 172.21.15.84:55568) 2024-04-26T18:36:50.495 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46a183700 1 -- 172.21.15.84:0/2362961289 learned_addr learned my addr 172.21.15.84:0/2362961289 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd469181700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd464102460 0x7fd46419fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd469982700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd464101aa0 0x7fd46419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46a183700 1 -- 172.21.15.84:0/2362961289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd464102460 msgr2=0x7fd46419fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46a183700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd464102460 0x7fd46419fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46a183700 1 -- 172.21.15.84:0/2362961289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd464101aa0 msgr2=0x7fd46419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46a183700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd464101aa0 0x7fd46419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46a183700 1 -- 172.21.15.84:0/2362961289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46419a0f0 con 0x7fd464107aa0 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd469982700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd464101aa0 0x7fd46419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd46a183700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd464107aa0 0x7fd464199990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00b970 tx=0x7fd45c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd45affd700 1 -- 172.21.15.84:0/2362961289 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c019070 con 0x7fd464107aa0 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.491+0000 7fd45affd700 1 -- 172.21.15.84:0/2362961289 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45c00cde0 con 0x7fd464107aa0 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.492+0000 7fd46bbe6700 1 -- 172.21.15.84:0/2362961289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46419a3e0 con 0x7fd464107aa0 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.492+0000 7fd45affd700 1 -- 172.21.15.84:0/2362961289 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45c007370 con 0x7fd464107aa0 2024-04-26T18:36:50.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.492+0000 7fd46bbe6700 1 -- 172.21.15.84:0/2362961289 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4641a6650 con 0x7fd464107aa0 2024-04-26T18:36:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.494+0000 7fd45affd700 1 -- 172.21.15.84:0/2362961289 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fd45c01c070 con 0x7fd464107aa0 2024-04-26T18:36:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.495+0000 7fd45affd700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd4500a3e30 0x7fd4500a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.495+0000 7fd469982700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd4500a3e30 0x7fd4500a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.495+0000 7fd45affd700 1 -- 172.21.15.84:0/2362961289 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fd45c0cfb60 con 0x7fd464107aa0 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.495+0000 7fd469982700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd4500a3e30 0x7fd4500a62f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd460005fd0 tx=0x7fd460005f00 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.495+0000 7fd476478b80 1 Processor -- start 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd476478b80 1 -- start start 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd476478b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 0x561463bdc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd476478b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdcbd0 0x561463bd6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd476478b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bd6c80 0x561463bd70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd476478b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561463b24930 con 0x561463bdc2b0 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd476478b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561463a8aa30 con 0x561463bdcbd0 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd476478b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561463b208f0 con 0x561463bd6c80 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd469982700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 0x561463bdc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd469982700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 0x561463bdc690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55584/0 (socket says 172.21.15.84:55584) 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd469982700 1 -- 172.21.15.84:0/326533429 learned_addr learned my addr 172.21.15.84:0/326533429 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd46a183700 1 --2- 172.21.15.84:0/326533429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bd6c80 0x561463bd70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd469181700 1 --2- 172.21.15.84:0/326533429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdcbd0 0x561463bd6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd469982700 1 -- 172.21.15.84:0/326533429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bd6c80 msgr2=0x561463bd70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd469982700 1 --2- 172.21.15.84:0/326533429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bd6c80 0x561463bd70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd469982700 1 -- 172.21.15.84:0/326533429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdcbd0 msgr2=0x561463bd6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd469982700 1 --2- 172.21.15.84:0/326533429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdcbd0 0x561463bd6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd469982700 1 -- 172.21.15.84:0/326533429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561463a719d0 con 0x561463bdc2b0 2024-04-26T18:36:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.496+0000 7fd469181700 1 --2- 172.21.15.84:0/326533429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdcbd0 0x561463bd6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:36:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.497+0000 7fd469982700 1 --2- 172.21.15.84:0/326533429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 0x561463bdc690 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd46000eed0 tx=0x7fd460011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a65ebb729151437 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.497+0000 7fd44ffff700 1 -- 172.21.15.84:0/326533429 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460019e30 con 0x561463bdc2b0 2024-04-26T18:36:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.497+0000 7fd44ffff700 1 -- 172.21.15.84:0/326533429 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4600044d0 con 0x561463bdc2b0 2024-04-26T18:36:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.497+0000 7fd44ffff700 1 -- 172.21.15.84:0/326533429 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4600176c0 con 0x561463bdc2b0 2024-04-26T18:36:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.497+0000 7fd476478b80 1 -- 172.21.15.84:0/326533429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 msgr2=0x561463bdc690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:50.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.497+0000 7fd476478b80 1 --2- 172.21.15.84:0/326533429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 0x561463bdc690 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fd46000eed0 tx=0x7fd460011000 comp rx=0 tx=0).stop 2024-04-26T18:36:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.497+0000 7fd476478b80 1 -- 172.21.15.84:0/326533429 shutdown_connections 2024-04-26T18:36:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.497+0000 7fd476478b80 1 --2- 172.21.15.84:0/326533429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 0x561463bdc690 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.497+0000 7fd476478b80 1 --2- 172.21.15.84:0/326533429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bd6c80 0x561463bd70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.497+0000 7fd476478b80 1 --2- 172.21.15.84:0/326533429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdcbd0 0x561463bd6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.498+0000 7fd476478b80 1 -- 172.21.15.84:0/326533429 >> 172.21.15.84:0/326533429 conn(0x561463b006c0 msgr2=0x561463bcd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.498+0000 7fd476478b80 1 -- 172.21.15.84:0/326533429 shutdown_connections 2024-04-26T18:36:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.498+0000 7fd476478b80 1 -- 172.21.15.84:0/326533429 wait complete. 2024-04-26T18:36:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.498+0000 7fd476478b80 1 -- start start 2024-04-26T18:36:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.498+0000 7fd476478b80 1 Processor -- start 2024-04-26T18:36:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.498+0000 7fd476478b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 0x561463be2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.498+0000 7fd476478b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdc2b0 0x561463bdfbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.498+0000 7fd476478b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bdcbd0 0x561463be0110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.498+0000 7fd476478b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x561463be3640 con 0x561463bd6c80 2024-04-26T18:36:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.498+0000 7fd476478b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x561463adeb80 con 0x561463bdc2b0 2024-04-26T18:36:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.498+0000 7fd476478b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x561463be37c0 con 0x561463bdcbd0 2024-04-26T18:36:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.498+0000 7fd476478b80 10 client.0 _ll_register_callbacks cb 0x561463abca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd469982700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 0x561463be2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd469982700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 0x561463be2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55586/0 (socket says 172.21.15.84:55586) 2024-04-26T18:36:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd469982700 1 -- 172.21.15.84:0/629990269 learned_addr learned my addr 172.21.15.84:0/629990269 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd469181700 1 --2- 172.21.15.84:0/629990269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdc2b0 0x561463bdfbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd469982700 1 -- 172.21.15.84:0/629990269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bdcbd0 msgr2=0x561463be0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd469982700 1 --2- 172.21.15.84:0/629990269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bdcbd0 0x561463be0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd469982700 1 -- 172.21.15.84:0/629990269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdc2b0 msgr2=0x561463bdfbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:50.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd469982700 1 --2- 172.21.15.84:0/629990269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdc2b0 0x561463bdfbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd469982700 1 -- 172.21.15.84:0/629990269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561463ae2580 con 0x561463bd6c80 2024-04-26T18:36:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd469181700 1 --2- 172.21.15.84:0/629990269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdc2b0 0x561463bdfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:36:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd469982700 1 --2- 172.21.15.84:0/629990269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 0x561463be2830 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd460004070 tx=0x7fd460004150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee4478d9e9bb0a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd44d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd44d7fa700 1 -- 172.21.15.84:0/629990269 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd460019e30 con 0x561463bd6c80 2024-04-26T18:36:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd44d7fa700 1 -- 172.21.15.84:0/629990269 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fd460005260 con 0x561463bd6c80 2024-04-26T18:36:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd476478b80 1 -- 172.21.15.84:0/629990269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd4600044d0 con 0x561463bd6c80 2024-04-26T18:36:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd44d7fa700 1 -- 172.21.15.84:0/629990269 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd460038050 con 0x561463bd6c80 2024-04-26T18:36:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd476478b80 10 client.16479 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.499+0000 7fd476478b80 1 -- 172.21.15.84:0/629990269 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561463bedd60 con 0x561463bd6c80 2024-04-26T18:36:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.500+0000 7fd44d7fa700 1 -- 172.21.15.84:0/629990269 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd4600044d0 con 0x561463bd6c80 2024-04-26T18:36:50.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.500+0000 7fd44d7fa700 1 -- 172.21.15.84:0/629990269 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd460015040 con 0x561463bd6c80 2024-04-26T18:36:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.500+0000 7fd476478b80 10 client.16479 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.500+0000 7fd476478b80 10 client.16479 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:36:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.500+0000 7fd476478b80 20 client.16479 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.500+0000 7fd476478b80 1 --2- 172.21.15.84:0/629990269 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561463bf0230 0x561463bf2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.500+0000 7fd476478b80 4 client.16479 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:36:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.500+0000 7fd476478b80 1 -- 172.21.15.84:0/629990269 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561463b32310 con 0x561463bf0230 2024-04-26T18:36:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.500+0000 7fd469181700 1 --2- 172.21.15.84:0/629990269 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561463bf0230 0x561463bf2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.500+0000 7fd44d7fa700 10 client.16479 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.500+0000 7fd469181700 1 --2- 172.21.15.84:0/629990269 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561463bf0230 0x561463bf2620 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d147881f9958dec3 server_cookie=7c1a2df94519be2a in_seq=0 out_seq=0 2024-04-26T18:36:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.501+0000 7fd44d7fa700 1 -- 172.21.15.84:0/629990269 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561463b32310 con 0x561463bf0230 2024-04-26T18:36:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.501+0000 7fd44d7fa700 10 client.16479 handle_command_reply: tid=0 2024-04-26T18:36:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.501+0000 7fd476478b80 1 client.16479 shutdown 2024-04-26T18:36:50.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.501+0000 7fd476478b80 1 -- 172.21.15.84:0/629990269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 msgr2=0x561463be2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.501+0000 7fd476478b80 1 --2- 172.21.15.84:0/629990269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 0x561463be2830 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fd460004070 tx=0x7fd460004150 comp rx=0 tx=0).stop 2024-04-26T18:36:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.501+0000 7fd476478b80 1 -- 172.21.15.84:0/629990269 shutdown_connections 2024-04-26T18:36:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.502+0000 7fd476478b80 1 --2- 172.21.15.84:0/629990269 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561463bf0230 0x561463bf2620 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.502+0000 7fd476478b80 1 --2- 172.21.15.84:0/629990269 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 0x561463be2830 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.502+0000 7fd476478b80 1 --2- 172.21.15.84:0/629990269 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bdcbd0 0x561463be0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.502+0000 7fd44d7fa700 0 client.16479 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.502+0000 7fd476478b80 1 --2- 172.21.15.84:0/629990269 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdc2b0 0x561463bdfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.502+0000 7fd476478b80 1 -- 172.21.15.84:0/629990269 >> 172.21.15.84:0/629990269 conn(0x561463b006c0 msgr2=0x561463bd5650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.502+0000 7fd476478b80 1 -- 172.21.15.84:0/629990269 shutdown_connections 2024-04-26T18:36:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.502+0000 7fd476478b80 1 -- 172.21.15.84:0/629990269 wait complete. 2024-04-26T18:36:50.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.502+0000 7fd476478b80 20 client.16479 trim_cache size 0 max 16384 2024-04-26T18:36:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd476478b80 1 Processor -- start 2024-04-26T18:36:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd476478b80 1 -- start start 2024-04-26T18:36:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd476478b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bd6c80 0x561463bd6660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd476478b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 0x561463bec570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd476478b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdcbd0 0x561463bdf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd476478b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641abb90 con 0x561463bdc2b0 2024-04-26T18:36:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd476478b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641ad6d0 con 0x561463bdcbd0 2024-04-26T18:36:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd476478b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641ac260 con 0x561463bd6c80 2024-04-26T18:36:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd469181700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 0x561463bec570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd469181700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 0x561463bec570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55600/0 (socket says 172.21.15.84:55600) 2024-04-26T18:36:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd469181700 1 -- 172.21.15.84:0/1307830745 learned_addr learned my addr 172.21.15.84:0/1307830745 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:50.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd469982700 1 --2- 172.21.15.84:0/1307830745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bd6c80 0x561463bd6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd469181700 1 -- 172.21.15.84:0/1307830745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bd6c80 msgr2=0x561463bd6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:50.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd469181700 1 --2- 172.21.15.84:0/1307830745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bd6c80 0x561463bd6660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd469181700 1 -- 172.21.15.84:0/1307830745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdcbd0 msgr2=0x561463bdf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:50.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd469181700 1 --2- 172.21.15.84:0/1307830745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdcbd0 0x561463bdf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd469181700 1 -- 172.21.15.84:0/1307830745 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561463bc97f0 con 0x561463bdc2b0 2024-04-26T18:36:50.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.521+0000 7fd469982700 1 --2- 172.21.15.84:0/1307830745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bd6c80 0x561463bd6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:36:50.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.522+0000 7fd469181700 1 --2- 172.21.15.84:0/1307830745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 0x561463bec570 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd45400b5c0 tx=0x7fd454005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=467f4c5904deeba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:50.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.522+0000 7fd44f7fe700 1 -- 172.21.15.84:0/1307830745 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd45402e070 con 0x561463bdc2b0 2024-04-26T18:36:50.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.522+0000 7fd44f7fe700 1 -- 172.21.15.84:0/1307830745 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45400bdc0 con 0x561463bdc2b0 2024-04-26T18:36:50.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.522+0000 7fd44f7fe700 1 -- 172.21.15.84:0/1307830745 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd454029070 con 0x561463bdc2b0 2024-04-26T18:36:50.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.522+0000 7fd476478b80 1 -- 172.21.15.84:0/1307830745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 msgr2=0x561463bec570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:50.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.522+0000 7fd476478b80 1 --2- 172.21.15.84:0/1307830745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 0x561463bec570 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fd45400b5c0 tx=0x7fd454005960 comp rx=0 tx=0).stop 2024-04-26T18:36:50.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.522+0000 7fd476478b80 1 -- 172.21.15.84:0/1307830745 shutdown_connections 2024-04-26T18:36:50.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.522+0000 7fd476478b80 1 --2- 172.21.15.84:0/1307830745 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdcbd0 0x561463bdf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.522+0000 7fd476478b80 1 --2- 172.21.15.84:0/1307830745 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bd6c80 0x561463bd6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.522+0000 7fd476478b80 1 --2- 172.21.15.84:0/1307830745 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bdc2b0 0x561463bec570 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.522+0000 7fd476478b80 1 -- 172.21.15.84:0/1307830745 >> 172.21.15.84:0/1307830745 conn(0x561463b006c0 msgr2=0x561463bc6f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:50.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.523+0000 7fd476478b80 1 -- 172.21.15.84:0/1307830745 shutdown_connections 2024-04-26T18:36:50.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.523+0000 7fd476478b80 1 -- 172.21.15.84:0/1307830745 wait complete. 2024-04-26T18:36:50.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.523+0000 7fd476478b80 1 -- start start 2024-04-26T18:36:50.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.523+0000 7fd476478b80 1 Processor -- start 2024-04-26T18:36:50.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.523+0000 7fd476478b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 0x561463be8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.523+0000 7fd476478b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdc2b0 0x561463be4f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.523+0000 7fd476478b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bdcbd0 0x561463be5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.523+0000 7fd476478b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641ad000 con 0x561463bd6c80 2024-04-26T18:36:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.523+0000 7fd476478b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641ac930 con 0x561463bdc2b0 2024-04-26T18:36:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.523+0000 7fd476478b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd4641a9b70 con 0x561463bdcbd0 2024-04-26T18:36:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.523+0000 7fd476478b80 10 client.0 _ll_register_callbacks cb 0x561463abca50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.523+0000 7fd469982700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 0x561463be8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.529 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd469982700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 0x561463be8ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55612/0 (socket says 172.21.15.84:55612) 2024-04-26T18:36:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd469982700 1 -- 172.21.15.84:0/3499166028 learned_addr learned my addr 172.21.15.84:0/3499166028 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd469181700 1 --2- 172.21.15.84:0/3499166028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdc2b0 0x561463be4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd469982700 1 -- 172.21.15.84:0/3499166028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bdcbd0 msgr2=0x561463be5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd469982700 1 --2- 172.21.15.84:0/3499166028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bdcbd0 0x561463be5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd469982700 1 -- 172.21.15.84:0/3499166028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdc2b0 msgr2=0x561463be4f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd469982700 1 --2- 172.21.15.84:0/3499166028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdc2b0 0x561463be4f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd469982700 1 -- 172.21.15.84:0/3499166028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561463be2ba0 con 0x561463bd6c80 2024-04-26T18:36:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd469982700 1 --2- 172.21.15.84:0/3499166028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 0x561463be8ba0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd46000a440 tx=0x7fd460004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585eeb189dbedfd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd44d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd44d7fa700 1 -- 172.21.15.84:0/3499166028 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd46001d070 con 0x561463bd6c80 2024-04-26T18:36:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd44d7fa700 1 -- 172.21.15.84:0/3499166028 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fd460005460 con 0x561463bd6c80 2024-04-26T18:36:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd476478b80 1 -- 172.21.15.84:0/3499166028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd45400bdc0 con 0x561463bd6c80 2024-04-26T18:36:50.530 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd44d7fa700 1 -- 172.21.15.84:0/3499166028 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd46002a7d0 con 0x561463bd6c80 2024-04-26T18:36:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd476478b80 10 client.16485 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd476478b80 1 -- 172.21.15.84:0/3499166028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561463c571d0 con 0x561463bd6c80 2024-04-26T18:36:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.524+0000 7fd44d7fa700 1 -- 172.21.15.84:0/3499166028 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd45400bdc0 con 0x561463bd6c80 2024-04-26T18:36:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.525+0000 7fd44d7fa700 1 -- 172.21.15.84:0/3499166028 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd460042070 con 0x561463bd6c80 2024-04-26T18:36:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.525+0000 7fd476478b80 10 client.16485 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.525+0000 7fd476478b80 10 client.16485 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:36:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.525+0000 7fd476478b80 20 client.16485 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.525+0000 7fd476478b80 1 --2- 172.21.15.84:0/3499166028 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561463c57880 0x561463c59c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.525+0000 7fd476478b80 4 client.16485 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "db6a0f03", "release": true, "await": true}] 2024-04-26T18:36:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.525+0000 7fd476478b80 1 -- 172.21.15.84:0/3499166028 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "set_id": "db6a0f03", "release": true, "await": true}) -- 0x561463b2c3b0 con 0x561463c57880 2024-04-26T18:36:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.525+0000 7fd469181700 1 --2- 172.21.15.84:0/3499166028 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561463c57880 0x561463c59c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.525+0000 7fd44d7fa700 10 client.16485 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:50.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.525+0000 7fd469181700 1 --2- 172.21.15.84:0/3499166028 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561463c57880 0x561463c59c70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33867f97b82c6217 server_cookie=600021cbd15798b in_seq=0 out_seq=0 2024-04-26T18:36:50.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd44d7fa700 1 -- 172.21.15.84:0/3499166028 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x561463b2c3b0 con 0x561463c57880 2024-04-26T18:36:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd44d7fa700 10 client.16485 handle_command_reply: tid=0 2024-04-26T18:36:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd476478b80 1 client.16485 shutdown 2024-04-26T18:36:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd476478b80 1 -- 172.21.15.84:0/3499166028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 msgr2=0x561463be8ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd476478b80 1 --2- 172.21.15.84:0/3499166028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 0x561463be8ba0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fd46000a440 tx=0x7fd460004320 comp rx=0 tx=0).stop 2024-04-26T18:36:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd476478b80 1 -- 172.21.15.84:0/3499166028 shutdown_connections 2024-04-26T18:36:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd476478b80 1 --2- 172.21.15.84:0/3499166028 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x561463c57880 0x561463c59c70 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd476478b80 1 --2- 172.21.15.84:0/3499166028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x561463bd6c80 0x561463be8ba0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd476478b80 1 --2- 172.21.15.84:0/3499166028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x561463bdcbd0 0x561463be5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd44d7fa700 0 client.16485 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd476478b80 1 --2- 172.21.15.84:0/3499166028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x561463bdc2b0 0x561463be4f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd476478b80 1 -- 172.21.15.84:0/3499166028 >> 172.21.15.84:0/3499166028 conn(0x561463b006c0 msgr2=0x561463be11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:50.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd476478b80 1 -- 172.21.15.84:0/3499166028 shutdown_connections 2024-04-26T18:36:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd476478b80 1 -- 172.21.15.84:0/3499166028 wait complete. 2024-04-26T18:36:50.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd476478b80 20 client.16485 trim_cache size 0 max 16384 2024-04-26T18:36:50.534 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:36:50.534 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:36:50.534 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:36:50.534 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 24, 2024-04-26T18:36:50.534 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:36:50.535 INFO:teuthology.orchestra.run.smithi084.stdout: "db6a0f03": { 2024-04-26T18:36:50.535 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 24, 2024-04-26T18:36:50.535 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:36:50.535 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:36:50.535 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:36:50.535 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:36:50.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:36:50.535 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:36:50.535 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 89.7, 2024-04-26T18:36:50.535 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:36:50.535 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:36:50.536 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:36:50.536 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:36:50.536 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:36:50.536 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:36:50.536 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:36:50.536 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:36:50.536 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:36:50.536 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:36:50.536 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:36:50.536 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:36:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.530+0000 7fd46bbe6700 1 -- 172.21.15.84:0/2362961289 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd4500a3e30 msgr2=0x7fd4500a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.531+0000 7fd46bbe6700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd4500a3e30 0x7fd4500a62f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fd460005fd0 tx=0x7fd460005f00 comp rx=0 tx=0).stop 2024-04-26T18:36:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.531+0000 7fd46bbe6700 1 -- 172.21.15.84:0/2362961289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd464107aa0 msgr2=0x7fd464199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.531+0000 7fd46bbe6700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd464107aa0 0x7fd464199990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00b970 tx=0x7fd45c00bc80 comp rx=0 tx=0).stop 2024-04-26T18:36:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.531+0000 7fd46bbe6700 1 -- 172.21.15.84:0/2362961289 shutdown_connections 2024-04-26T18:36:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.531+0000 7fd46bbe6700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd4500a3e30 0x7fd4500a62f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.531+0000 7fd46bbe6700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd464107aa0 0x7fd464199990 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.537 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.531+0000 7fd46bbe6700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd464101aa0 0x7fd46419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.531+0000 7fd46bbe6700 1 --2- 172.21.15.84:0/2362961289 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd464102460 0x7fd46419fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:50.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.531+0000 7fd46bbe6700 1 -- 172.21.15.84:0/2362961289 >> 172.21.15.84:0/2362961289 conn(0x7fd464078920 msgr2=0x7fd4641004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:50.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.531+0000 7fd46bbe6700 1 -- 172.21.15.84:0/2362961289 shutdown_connections 2024-04-26T18:36:50.538 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:36:50.531+0000 7fd46bbe6700 1 -- 172.21.15.84:0/2362961289 wait complete. 2024-04-26T18:36:51.312 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'db6a0f03', seconds elapsed: 41.3 2024-04-26T18:36:51.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:51 smithi084 ceph-mon[29141]: pgmap v275: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-26T18:36:51.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:51 smithi103 ceph-mon[32727]: pgmap v275: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-26T18:36:51.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:51 smithi174 ceph-mon[32705]: pgmap v275: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-26T18:36:53.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:53 smithi084 ceph-mon[29141]: pgmap v276: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-26T18:36:53.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:53 smithi103 ceph-mon[32727]: pgmap v276: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-26T18:36:53.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:53 smithi174 ceph-mon[32705]: pgmap v276: 97 pgs: 97 active+clean; 1.8 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-26T18:36:55.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:55 smithi084 ceph-mon[29141]: pgmap v277: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-26T18:36:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:55 smithi103 ceph-mon[32727]: pgmap v277: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-26T18:36:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:55 smithi174 ceph-mon[32705]: pgmap v277: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 15 op/s 2024-04-26T18:36:57.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:57 smithi084 ceph-mon[29141]: pgmap v278: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-04-26T18:36:57.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:57 smithi103 ceph-mon[32727]: pgmap v278: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-04-26T18:36:57.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:57 smithi174 ceph-mon[32705]: pgmap v278: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-04-26T18:36:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.989+0000 7fd89a0c8640 1 -- 172.21.15.84:0/405776104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd894102930 msgr2=0x7fd894102db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.989+0000 7fd89a0c8640 1 --2- 172.21.15.84:0/405776104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd894102930 0x7fd894102db0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fd888009930 tx=0x7fd88802a420 comp rx=0 tx=0).stop 2024-04-26T18:36:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.989+0000 7fd89a0c8640 1 -- 172.21.15.84:0/405776104 shutdown_connections 2024-04-26T18:36:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.990+0000 7fd89a0c8640 1 --2- 172.21.15.84:0/405776104 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8941032f0 0x7fd89410f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.990+0000 7fd89a0c8640 1 --2- 172.21.15.84:0/405776104 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd894102930 0x7fd894102db0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.990+0000 7fd89a0c8640 1 --2- 172.21.15.84:0/405776104 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd894108940 0x7fd894108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.990+0000 7fd89a0c8640 1 -- 172.21.15.84:0/405776104 >> 172.21.15.84:0/405776104 conn(0x7fd8940fe400 msgr2=0x7fd894100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.990+0000 7fd89a0c8640 1 -- 172.21.15.84:0/405776104 shutdown_connections 2024-04-26T18:36:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.990+0000 7fd89a0c8640 1 -- 172.21.15.84:0/405776104 wait complete. 2024-04-26T18:36:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.990+0000 7fd89a0c8640 1 Processor -- start 2024-04-26T18:36:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.990+0000 7fd89a0c8640 1 -- start start 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd89a0c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd894102930 0x7fd8941990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd89a0c8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8941032f0 0x7fd894199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd89a0c8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd894108940 0x7fd89419d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd89a0c8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd894112c00 con 0x7fd894108940 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd89a0c8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd894112a80 con 0x7fd894102930 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd89a0c8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd894112d80 con 0x7fd8941032f0 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd893fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd894108940 0x7fd89419d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd8937fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd894102930 0x7fd8941990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd893fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd894108940 0x7fd89419d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38168/0 (socket says 172.21.15.84:38168) 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd8937fe640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd894102930 0x7fd8941990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:35186/0 (socket says 172.21.15.84:35186) 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd893fff640 1 -- 172.21.15.84:0/3248558765 learned_addr learned my addr 172.21.15.84:0/3248558765 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd892ffd640 1 --2- 172.21.15.84:0/3248558765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8941032f0 0x7fd894199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd893fff640 1 -- 172.21.15.84:0/3248558765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8941032f0 msgr2=0x7fd894199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd893fff640 1 --2- 172.21.15.84:0/3248558765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8941032f0 0x7fd894199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd893fff640 1 -- 172.21.15.84:0/3248558765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd894102930 msgr2=0x7fd8941990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd893fff640 1 --2- 172.21.15.84:0/3248558765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd894102930 0x7fd8941990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd893fff640 1 -- 172.21.15.84:0/3248558765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd89419e0d0 con 0x7fd894108940 2024-04-26T18:36:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd893fff640 1 --2- 172.21.15.84:0/3248558765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd894108940 0x7fd89419d9a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00eaa0 tx=0x7fd88c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.991+0000 7fd890ff9640 1 -- 172.21.15.84:0/3248558765 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c00cdb0 con 0x7fd894108940 2024-04-26T18:36:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.992+0000 7fd890ff9640 1 -- 172.21.15.84:0/3248558765 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd88c005020 con 0x7fd894108940 2024-04-26T18:36:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.992+0000 7fd89a0c8640 1 -- 172.21.15.84:0/3248558765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd89419e3c0 con 0x7fd894108940 2024-04-26T18:36:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.992+0000 7fd890ff9640 1 -- 172.21.15.84:0/3248558765 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd88c0116d0 con 0x7fd894108940 2024-04-26T18:36:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.992+0000 7fd89a0c8640 1 -- 172.21.15.84:0/3248558765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8941a5c50 con 0x7fd894108940 2024-04-26T18:36:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.994+0000 7fd890ff9640 1 -- 172.21.15.84:0/3248558765 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fd88c0051d0 con 0x7fd894108940 2024-04-26T18:36:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd89cde3740 1 Processor -- start 2024-04-26T18:36:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd89cde3740 1 -- start start 2024-04-26T18:36:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd89cde3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d7f00 0x55e6001d82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd89cde3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001d8820 0x55e6001e2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd89cde3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 0x55e6001e54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd89cde3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e60013e460 con 0x55e6001e30e0 2024-04-26T18:36:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd89cde3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e6000f8590 con 0x55e6001d8820 2024-04-26T18:36:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd89cde3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e60013a420 con 0x55e6001d7f00 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd892ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 0x55e6001e54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd892ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 0x55e6001e54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38178/0 (socket says 172.21.15.84:38178) 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd892ffd640 1 -- 172.21.15.84:0/2935292249 learned_addr learned my addr 172.21.15.84:0/2935292249 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd8937fe640 1 --2- 172.21.15.84:0/2935292249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001d8820 0x55e6001e2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd892ffd640 1 --2- 172.21.15.84:0/2935292249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d7f00 0x55e6001d82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd890ff9640 1 --2- 172.21.15.84:0/3248558765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd87c0a3e40 0x7fd87c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd893fff640 1 --2- 172.21.15.84:0/3248558765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd87c0a3e40 0x7fd87c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd892ffd640 1 -- 172.21.15.84:0/2935292249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d7f00 msgr2=0x55e6001d82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd892ffd640 1 --2- 172.21.15.84:0/2935292249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d7f00 0x55e6001d82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd892ffd640 1 -- 172.21.15.84:0/2935292249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001d8820 msgr2=0x55e6001e2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd892ffd640 1 --2- 172.21.15.84:0/2935292249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001d8820 0x55e6001e2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.995+0000 7fd892ffd640 1 -- 172.21.15.84:0/2935292249 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6001050d0 con 0x55e6001e30e0 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.996+0000 7fd890ff9640 1 -- 172.21.15.84:0/3248558765 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fd88c0d0980 con 0x7fd894108940 2024-04-26T18:36:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.996+0000 7fd892ffd640 1 --2- 172.21.15.84:0/2935292249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 0x55e6001e54d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd88802a940 tx=0x7fd888005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219cb7f4696fae4e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.996+0000 7fd879ffb640 1 -- 172.21.15.84:0/2935292249 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888035070 con 0x55e6001e30e0 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.996+0000 7fd893fff640 1 --2- 172.21.15.84:0/3248558765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd87c0a3e40 0x7fd87c0a6300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0a4fc0 tx=0x7fd88c06e370 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.996+0000 7fd879ffb640 1 -- 172.21.15.84:0/2935292249 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd888005020 con 0x55e6001e30e0 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.996+0000 7fd879ffb640 1 -- 172.21.15.84:0/2935292249 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8880044e0 con 0x55e6001e30e0 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.997+0000 7fd89cde3740 1 -- 172.21.15.84:0/2935292249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 msgr2=0x55e6001e54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.997+0000 7fd89cde3740 1 --2- 172.21.15.84:0/2935292249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 0x55e6001e54d0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fd88802a940 tx=0x7fd888005fd0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.997+0000 7fd89cde3740 1 -- 172.21.15.84:0/2935292249 shutdown_connections 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.997+0000 7fd89cde3740 1 --2- 172.21.15.84:0/2935292249 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 0x55e6001e54d0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.997+0000 7fd89cde3740 1 --2- 172.21.15.84:0/2935292249 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001d8820 0x55e6001e2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.997+0000 7fd89cde3740 1 --2- 172.21.15.84:0/2935292249 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d7f00 0x55e6001d82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.997+0000 7fd89cde3740 1 -- 172.21.15.84:0/2935292249 >> 172.21.15.84:0/2935292249 conn(0x55e6001d4410 msgr2=0x55e6001d47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.997+0000 7fd89cde3740 1 -- 172.21.15.84:0/2935292249 shutdown_connections 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.997+0000 7fd89cde3740 1 -- 172.21.15.84:0/2935292249 wait complete. 2024-04-26T18:36:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.997+0000 7fd89cde3740 1 -- start start 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.997+0000 7fd89cde3740 1 Processor -- start 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.997+0000 7fd89cde3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 0x55e6001e0100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.997+0000 7fd89cde3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 0x55e6001e0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd89cde3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001e30e0 0x55e6001e0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd89cde3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e6001e5cb0 con 0x55e6001d7f00 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd89cde3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e6000c5b60 con 0x55e6001e30e0 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd89cde3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e6001e5e30 con 0x55e6001d8820 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd89cde3740 10 client.0 _ll_register_callbacks cb 0x55e6000ea2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd892ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 0x55e6001e0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd892ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 0x55e6001e0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38184/0 (socket says 172.21.15.84:38184) 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd892ffd640 1 -- 172.21.15.84:0/543294997 learned_addr learned my addr 172.21.15.84:0/543294997 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd892ffd640 1 --2- 172.21.15.84:0/543294997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001e30e0 0x55e6001e0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd8937fe640 1 --2- 172.21.15.84:0/543294997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 0x55e6001e0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd892ffd640 1 -- 172.21.15.84:0/543294997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 msgr2=0x55e6001e0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd892ffd640 1 --2- 172.21.15.84:0/543294997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 0x55e6001e0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd892ffd640 1 -- 172.21.15.84:0/543294997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001e30e0 msgr2=0x55e6001e0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd892ffd640 1 --2- 172.21.15.84:0/543294997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001e30e0 0x55e6001e0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd892ffd640 1 -- 172.21.15.84:0/543294997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5ffe50210 con 0x55e6001d7f00 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd892ffd640 1 --2- 172.21.15.84:0/543294997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 0x55e6001e0100 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd888005250 tx=0x7fd888005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef092aaec9a4498 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd872ffd640 1 -- 172.21.15.84:0/543294997 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888035070 con 0x55e6001d7f00 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd872ffd640 1 -- 172.21.15.84:0/543294997 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fd888004230 con 0x55e6001d7f00 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd89cde3740 1 -- 172.21.15.84:0/543294997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd888005020 con 0x55e6001d7f00 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd872ffd640 1 -- 172.21.15.84:0/543294997 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd888004430 con 0x55e6001d7f00 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd89cde3740 10 client.16497 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.998+0000 7fd89cde3740 1 -- 172.21.15.84:0/543294997 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5fff1c620 con 0x55e6001d7f00 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.999+0000 7fd872ffd640 1 -- 172.21.15.84:0/543294997 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd888005020 con 0x55e6001d7f00 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.999+0000 7fd872ffd640 1 -- 172.21.15.84:0/543294997 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd888054030 con 0x55e6001d7f00 2024-04-26T18:36:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.999+0000 7fd89cde3740 10 client.16497 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.999+0000 7fd89cde3740 10 client.16497 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.999+0000 7fd89cde3740 20 client.16497 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.999+0000 7fd89cde3740 1 --2- 172.21.15.84:0/543294997 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e6001f30e0 0x55e6001f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.999+0000 7fd89cde3740 4 client.16497 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.999+0000 7fd89cde3740 1 -- 172.21.15.84:0/543294997 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e60014c070 con 0x55e6001f30e0 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:58.999+0000 7fd8937fe640 1 --2- 172.21.15.84:0/543294997 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e6001f30e0 0x55e6001f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.000+0000 7fd872ffd640 10 client.16497 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.000+0000 7fd8937fe640 1 --2- 172.21.15.84:0/543294997 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e6001f30e0 0x55e6001f54d0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bdafff19e47302e server_cookie=6c432371268cc8f5 in_seq=0 out_seq=0 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.000+0000 7fd872ffd640 1 -- 172.21.15.84:0/543294997 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e60014c070 con 0x55e6001f30e0 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.000+0000 7fd872ffd640 10 client.16497 handle_command_reply: tid=0 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.000+0000 7fd89cde3740 1 client.16497 shutdown 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.000+0000 7fd89cde3740 1 -- 172.21.15.84:0/543294997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 msgr2=0x55e6001e0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.000+0000 7fd89cde3740 1 --2- 172.21.15.84:0/543294997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 0x55e6001e0100 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fd888005250 tx=0x7fd888005c90 comp rx=0 tx=0).stop 2024-04-26T18:36:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.000+0000 7fd89cde3740 1 -- 172.21.15.84:0/543294997 shutdown_connections 2024-04-26T18:36:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.000+0000 7fd89cde3740 1 --2- 172.21.15.84:0/543294997 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e6001f30e0 0x55e6001f54d0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.000+0000 7fd89cde3740 1 --2- 172.21.15.84:0/543294997 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001e30e0 0x55e6001e0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.000+0000 7fd89cde3740 1 --2- 172.21.15.84:0/543294997 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 0x55e6001e0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.000+0000 7fd89cde3740 1 --2- 172.21.15.84:0/543294997 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 0x55e6001e0100 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.001+0000 7fd89cde3740 1 -- 172.21.15.84:0/543294997 >> 172.21.15.84:0/543294997 conn(0x55e6001d4410 msgr2=0x55e6001da140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.000+0000 7fd872ffd640 0 client.16497 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.001+0000 7fd89cde3740 1 -- 172.21.15.84:0/543294997 shutdown_connections 2024-04-26T18:36:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.001+0000 7fd89cde3740 1 -- 172.21.15.84:0/543294997 wait complete. 2024-04-26T18:36:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.001+0000 7fd89cde3740 20 client.16497 trim_cache size 0 max 16384 2024-04-26T18:36:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.015+0000 7fd89cde3740 1 Processor -- start 2024-04-26T18:36:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd89cde3740 1 -- start start 2024-04-26T18:36:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd89cde3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 0x55e6001d3d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd89cde3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 0x55e6001dd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd89cde3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001e30e0 0x55e6001dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd89cde3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941ae610 con 0x55e6001d7f00 2024-04-26T18:36:59.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd89cde3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941b0150 con 0x55e6001e30e0 2024-04-26T18:36:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd89cde3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941aece0 con 0x55e6001d8820 2024-04-26T18:36:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd892ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 0x55e6001d3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd892ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 0x55e6001d3d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38196/0 (socket says 172.21.15.84:38196) 2024-04-26T18:36:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd892ffd640 1 -- 172.21.15.84:0/4173583075 learned_addr learned my addr 172.21.15.84:0/4173583075 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd892ffd640 1 -- 172.21.15.84:0/4173583075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 msgr2=0x55e6001dd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd892ffd640 1 --2- 172.21.15.84:0/4173583075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 0x55e6001dd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd892ffd640 1 -- 172.21.15.84:0/4173583075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001e30e0 msgr2=0x55e6001dd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd892ffd640 1 --2- 172.21.15.84:0/4173583075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001e30e0 0x55e6001dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd892ffd640 1 -- 172.21.15.84:0/4173583075 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6001e54d0 con 0x55e6001d7f00 2024-04-26T18:36:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd892ffd640 1 --2- 172.21.15.84:0/4173583075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 0x55e6001d3d30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd888000c00 tx=0x7fd888003d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3e7900d5fb3006 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd8797fa640 1 -- 172.21.15.84:0/4173583075 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888035040 con 0x55e6001d7f00 2024-04-26T18:36:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd8797fa640 1 -- 172.21.15.84:0/4173583075 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8880035f0 con 0x55e6001d7f00 2024-04-26T18:36:59.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.016+0000 7fd8797fa640 1 -- 172.21.15.84:0/4173583075 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888004770 con 0x55e6001d7f00 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.017+0000 7fd89cde3740 1 -- 172.21.15.84:0/4173583075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 msgr2=0x55e6001d3d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.017+0000 7fd89cde3740 1 --2- 172.21.15.84:0/4173583075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 0x55e6001d3d30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fd888000c00 tx=0x7fd888003d20 comp rx=0 tx=0).stop 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.017+0000 7fd89cde3740 1 -- 172.21.15.84:0/4173583075 shutdown_connections 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.017+0000 7fd89cde3740 1 --2- 172.21.15.84:0/4173583075 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001e30e0 0x55e6001dd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.017+0000 7fd89cde3740 1 --2- 172.21.15.84:0/4173583075 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 0x55e6001dd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.017+0000 7fd89cde3740 1 --2- 172.21.15.84:0/4173583075 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001d7f00 0x55e6001d3d30 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.017+0000 7fd89cde3740 1 -- 172.21.15.84:0/4173583075 >> 172.21.15.84:0/4173583075 conn(0x55e6001d4410 msgr2=0x55e6001db3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.017+0000 7fd89cde3740 1 -- 172.21.15.84:0/4173583075 shutdown_connections 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.017+0000 7fd89cde3740 1 -- 172.21.15.84:0/4173583075 wait complete. 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.017+0000 7fd89cde3740 1 -- start start 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd89cde3740 1 Processor -- start 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd89cde3740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001d7f00 0x55e6001f85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd89cde3740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 0x55e6001f5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd89cde3740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 0x55e6001f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd89cde3740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941afa80 con 0x55e6001e30e0 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd89cde3740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941af3b0 con 0x55e6001d7f00 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd89cde3740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8941ac610 con 0x55e6001d8820 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd89cde3740 10 client.0 _ll_register_callbacks cb 0x7fd88400a570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd892ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 0x55e6001f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd892ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 0x55e6001f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38206/0 (socket says 172.21.15.84:38206) 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd892ffd640 1 -- 172.21.15.84:0/2299291930 learned_addr learned my addr 172.21.15.84:0/2299291930 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd892ffd640 1 --2- 172.21.15.84:0/2299291930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001d7f00 0x55e6001f85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd8937fe640 1 --2- 172.21.15.84:0/2299291930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 0x55e6001f5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd892ffd640 1 -- 172.21.15.84:0/2299291930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 msgr2=0x55e6001f5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd892ffd640 1 --2- 172.21.15.84:0/2299291930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 0x55e6001f5940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd892ffd640 1 -- 172.21.15.84:0/2299291930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001d7f00 msgr2=0x55e6001f85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd892ffd640 1 --2- 172.21.15.84:0/2299291930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001d7f00 0x55e6001f85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd892ffd640 1 -- 172.21.15.84:0/2299291930 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6001f6750 con 0x55e6001e30e0 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd892ffd640 1 --2- 172.21.15.84:0/2299291930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 0x55e6001f5f50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd888004050 tx=0x7fd888005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6bdaa77c12525e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd872ffd640 1 -- 172.21.15.84:0/2299291930 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888004b80 con 0x55e6001e30e0 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd872ffd640 1 -- 172.21.15.84:0/2299291930 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 8579+0+0 (secure 0 0 0) 0x7fd888004d20 con 0x55e6001e30e0 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.018+0000 7fd89cde3740 1 -- 172.21.15.84:0/2299291930 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8880035f0 con 0x55e6001e30e0 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.019+0000 7fd872ffd640 1 -- 172.21.15.84:0/2299291930 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd888004f20 con 0x55e6001e30e0 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.019+0000 7fd89cde3740 10 client.16503 fetch_fsmap learned FSMap version 29 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.019+0000 7fd89cde3740 1 -- 172.21.15.84:0/2299291930 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e6001f6750 con 0x55e6001e30e0 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.019+0000 7fd872ffd640 1 -- 172.21.15.84:0/2299291930 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8880035f0 con 0x55e6001e30e0 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.019+0000 7fd872ffd640 1 -- 172.21.15.84:0/2299291930 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd88803d450 con 0x55e6001e30e0 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.019+0000 7fd89cde3740 10 client.16503 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.019+0000 7fd89cde3740 10 client.16503 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.019+0000 7fd89cde3740 20 client.16503 populate_metadata read hostname 'smithi084' 2024-04-26T18:36:59.024 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.019+0000 7fd89cde3740 1 --2- 172.21.15.84:0/2299291930 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e6002595f0 0x55e60025b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:36:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.019+0000 7fd89cde3740 4 client.16503 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:36:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.019+0000 7fd89cde3740 1 -- 172.21.15.84:0/2299291930 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e600145ee0 con 0x55e6002595f0 2024-04-26T18:36:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.019+0000 7fd8937fe640 1 --2- 172.21.15.84:0/2299291930 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e6002595f0 0x55e60025b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:36:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.020+0000 7fd872ffd640 10 client.16503 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.020+0000 7fd8937fe640 1 --2- 172.21.15.84:0/2299291930 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e6002595f0 0x55e60025b9e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=286dfc61f7f1c719 server_cookie=1a18dbac439921c9 in_seq=0 out_seq=0 2024-04-26T18:36:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd872ffd640 1 -- 172.21.15.84:0/2299291930 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e600145ee0 con 0x55e6002595f0 2024-04-26T18:36:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd872ffd640 10 client.16503 handle_command_reply: tid=0 2024-04-26T18:36:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd89cde3740 1 client.16503 shutdown 2024-04-26T18:36:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd89cde3740 1 -- 172.21.15.84:0/2299291930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 msgr2=0x55e6001f5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd89cde3740 1 --2- 172.21.15.84:0/2299291930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 0x55e6001f5f50 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd888004050 tx=0x7fd888005fd0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd89cde3740 1 -- 172.21.15.84:0/2299291930 shutdown_connections 2024-04-26T18:36:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd89cde3740 1 --2- 172.21.15.84:0/2299291930 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e6002595f0 0x55e60025b9e0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd89cde3740 1 --2- 172.21.15.84:0/2299291930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e6001e30e0 0x55e6001f5f50 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd89cde3740 1 --2- 172.21.15.84:0/2299291930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e6001d8820 0x55e6001f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd872ffd640 0 client.16503 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:36:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd89cde3740 1 --2- 172.21.15.84:0/2299291930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e6001d7f00 0x55e6001f85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd89cde3740 1 -- 172.21.15.84:0/2299291930 >> 172.21.15.84:0/2299291930 conn(0x55e6001d4410 msgr2=0x55e6001d9820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd89cde3740 1 -- 172.21.15.84:0/2299291930 shutdown_connections 2024-04-26T18:36:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.030+0000 7fd89cde3740 1 -- 172.21.15.84:0/2299291930 wait complete. 2024-04-26T18:36:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.031+0000 7fd89cde3740 20 client.16503 trim_cache size 0 max 16384 2024-04-26T18:36:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:36:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:36:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:36:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:36:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.031+0000 7fd89a0c8640 1 -- 172.21.15.84:0/3248558765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd87c0a3e40 msgr2=0x7fd87c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.031+0000 7fd89a0c8640 1 --2- 172.21.15.84:0/3248558765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd87c0a3e40 0x7fd87c0a6300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fd88c0a4fc0 tx=0x7fd88c06e370 comp rx=0 tx=0).stop 2024-04-26T18:36:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.031+0000 7fd89a0c8640 1 -- 172.21.15.84:0/3248558765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd894108940 msgr2=0x7fd89419d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:36:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.031+0000 7fd89a0c8640 1 --2- 172.21.15.84:0/3248558765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd894108940 0x7fd89419d9a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fd88c00eaa0 tx=0x7fd88c00ef60 comp rx=0 tx=0).stop 2024-04-26T18:36:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.031+0000 7fd89a0c8640 1 -- 172.21.15.84:0/3248558765 shutdown_connections 2024-04-26T18:36:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.031+0000 7fd89a0c8640 1 --2- 172.21.15.84:0/3248558765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd87c0a3e40 0x7fd87c0a6300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.031+0000 7fd89a0c8640 1 --2- 172.21.15.84:0/3248558765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd894108940 0x7fd89419d9a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.031+0000 7fd89a0c8640 1 --2- 172.21.15.84:0/3248558765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8941032f0 0x7fd894199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.031+0000 7fd89a0c8640 1 --2- 172.21.15.84:0/3248558765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd894102930 0x7fd8941990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:36:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.031+0000 7fd89a0c8640 1 -- 172.21.15.84:0/3248558765 >> 172.21.15.84:0/3248558765 conn(0x7fd8940fe400 msgr2=0x7fd8940feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:36:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.031+0000 7fd89a0c8640 1 -- 172.21.15.84:0/3248558765 shutdown_connections 2024-04-26T18:36:59.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:36:59.031+0000 7fd89a0c8640 1 -- 172.21.15.84:0/3248558765 wait complete. 2024-04-26T18:36:59.770 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:36:59 smithi084 ceph-mon[29141]: pgmap v279: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-04-26T18:36:59.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:36:59 smithi103 ceph-mon[32727]: pgmap v279: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-04-26T18:36:59.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:36:59 smithi174 ceph-mon[32705]: pgmap v279: 97 pgs: 97 active+clean; 2.0 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 11 op/s 2024-04-26T18:37:01.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:01 smithi084 ceph-mon[29141]: pgmap v280: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-26T18:37:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:01 smithi103 ceph-mon[32727]: pgmap v280: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-26T18:37:01.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:01 smithi174 ceph-mon[32705]: pgmap v280: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-26T18:37:03.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:03 smithi084 ceph-mon[29141]: pgmap v281: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-04-26T18:37:03.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:03 smithi103 ceph-mon[32727]: pgmap v281: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-04-26T18:37:03.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:03 smithi174 ceph-mon[32705]: pgmap v281: 97 pgs: 97 active+clean; 2.2 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 27 KiB/s wr, 11 op/s 2024-04-26T18:37:05.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:05 smithi084 ceph-mon[29141]: pgmap v282: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 37 op/s 2024-04-26T18:37:05.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:05 smithi084 ceph-mon[29141]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T18:37:05.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:05 smithi103 ceph-mon[32727]: pgmap v282: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 37 op/s 2024-04-26T18:37:05.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:05 smithi103 ceph-mon[32727]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T18:37:05.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:05 smithi174 ceph-mon[32705]: pgmap v282: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 95 KiB/s wr, 37 op/s 2024-04-26T18:37:05.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:05 smithi174 ceph-mon[32705]: osdmap e101: 12 total, 12 up, 12 in 2024-04-26T18:37:06.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:06 smithi084 ceph-mon[29141]: pgmap v284: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 101 KiB/s wr, 38 op/s 2024-04-26T18:37:06.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:06 smithi103 ceph-mon[32727]: pgmap v284: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 101 KiB/s wr, 38 op/s 2024-04-26T18:37:06.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:06 smithi174 ceph-mon[32705]: pgmap v284: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 101 KiB/s wr, 38 op/s 2024-04-26T18:37:08.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:08 smithi174 ceph-mon[32705]: pgmap v285: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 101 KiB/s wr, 38 op/s 2024-04-26T18:37:08.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:08 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:37:09.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:08 smithi084 ceph-mon[29141]: pgmap v285: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 101 KiB/s wr, 38 op/s 2024-04-26T18:37:09.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:08 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:37:09.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:08 smithi103 ceph-mon[32727]: pgmap v285: 97 pgs: 97 active+clean; 2.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 101 KiB/s wr, 38 op/s 2024-04-26T18:37:09.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:08 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:37:09.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:09 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:37:09.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:09 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:37:09.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:09 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:37:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:09 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:37:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:09 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:37:10.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:09 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:37:10.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:09 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:37:10.130 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:09 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:37:10.130 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:09 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:37:11.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:11 smithi084 ceph-mon[29141]: pgmap v286: 97 pgs: 97 active+clean; 2.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-26T18:37:11.627 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:11 smithi103 ceph-mon[32727]: pgmap v286: 97 pgs: 97 active+clean; 2.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-26T18:37:11.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:11 smithi174 ceph-mon[32705]: pgmap v286: 97 pgs: 97 active+clean; 2.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-26T18:37:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 -- 172.21.15.84:0/439635807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8114101820 msgr2=0x7f8114101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 --2- 172.21.15.84:0/439635807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8114101820 0x7f8114101ca0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8108009930 tx=0x7f810802a420 comp rx=0 tx=0).stop 2024-04-26T18:37:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 -- 172.21.15.84:0/439635807 shutdown_connections 2024-04-26T18:37:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 --2- 172.21.15.84:0/439635807 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f81141021e0 0x7f811410e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 --2- 172.21.15.84:0/439635807 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8114101820 0x7f8114101ca0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 --2- 172.21.15.84:0/439635807 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8114107830 0x7f8114107c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 -- 172.21.15.84:0/439635807 >> 172.21.15.84:0/439635807 conn(0x7f81140789e0 msgr2=0x7f8114078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 -- 172.21.15.84:0/439635807 shutdown_connections 2024-04-26T18:37:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 -- 172.21.15.84:0/439635807 wait complete. 2024-04-26T18:37:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 Processor -- start 2024-04-26T18:37:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 -- start start 2024-04-26T18:37:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8114101820 0x7f81141990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81141021e0 0x7f81141995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8114107830 0x7f811419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8114112bf0 con 0x7f8114107830 2024-04-26T18:37:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.127+0000 7f811ae07640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8114112a70 con 0x7f8114101820 2024-04-26T18:37:12.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f811ae07640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8114112d70 con 0x7f81141021e0 2024-04-26T18:37:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f81193a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8114107830 0x7f811419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f81193a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8114107830 0x7f811419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52814/0 (socket says 172.21.15.84:52814) 2024-04-26T18:37:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f81193a4640 1 -- 172.21.15.84:0/3857440153 learned_addr learned my addr 172.21.15.84:0/3857440153 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f8113fff640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81141021e0 0x7f81141995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f8118ba3640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8114101820 0x7f81141990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f81193a4640 1 -- 172.21.15.84:0/3857440153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81141021e0 msgr2=0x7f81141995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f81193a4640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81141021e0 0x7f81141995f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f81193a4640 1 -- 172.21.15.84:0/3857440153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8114101820 msgr2=0x7f81141990b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f81193a4640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8114101820 0x7f81141990b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f81193a4640 1 -- 172.21.15.84:0/3857440153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f811419e0a0 con 0x7f8114107830 2024-04-26T18:37:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f8118ba3640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8114101820 0x7f81141990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:37:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f81193a4640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8114107830 0x7f811419d970 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f810c00eb10 tx=0x7f810c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f8111ffb640 1 -- 172.21.15.84:0/3857440153 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c004b20 con 0x7f8114107830 2024-04-26T18:37:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f8111ffb640 1 -- 172.21.15.84:0/3857440153 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f810c004cc0 con 0x7f8114107830 2024-04-26T18:37:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f811ae07640 1 -- 172.21.15.84:0/3857440153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f811419e390 con 0x7f8114107830 2024-04-26T18:37:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.128+0000 7f8111ffb640 1 -- 172.21.15.84:0/3857440153 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c010670 con 0x7f8114107830 2024-04-26T18:37:12.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.129+0000 7f811ae07640 1 -- 172.21.15.84:0/3857440153 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81141a5c20 con 0x7f8114107830 2024-04-26T18:37:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.131+0000 7f8111ffb640 1 -- 172.21.15.84:0/3857440153 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f810c005020 con 0x7f8114107830 2024-04-26T18:37:12.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.131+0000 7f811dbf9740 1 Processor -- start 2024-04-26T18:37:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.131+0000 7f811dbf9740 1 -- start start 2024-04-26T18:37:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.131+0000 7f8111ffb640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f80fc0a3e40 0x7f80fc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.131+0000 7f811dbf9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e11f00 0x55f2d4e122e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.131+0000 7f8113fff640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f80fc0a3e40 0x7f80fc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.131+0000 7f811dbf9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e12820 0x55f2d4e1cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.131+0000 7f811dbf9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e1f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.131+0000 7f811dbf9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f2d4d78460 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.131+0000 7f8111ffb640 1 -- 172.21.15.84:0/3857440153 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 8623+0+0 (secure 0 0 0) 0x7f810c0d0e70 con 0x7f8114107830 2024-04-26T18:37:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.131+0000 7f811dbf9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f2d4d32590 con 0x55f2d4e11f00 2024-04-26T18:37:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.131+0000 7f811dbf9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f2d4d74420 con 0x55f2d4e12820 2024-04-26T18:37:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f81193a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e1f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.140 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f81193a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e1f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52822/0 (socket says 172.21.15.84:52822) 2024-04-26T18:37:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f81193a4640 1 -- 172.21.15.84:0/403792748 learned_addr learned my addr 172.21.15.84:0/403792748 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f8118ba3640 1 --2- 172.21.15.84:0/403792748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e11f00 0x55f2d4e122e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f8113fff640 1 --2- 172.21.15.84:0/403792748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e12820 0x55f2d4e1cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f81193a4640 1 -- 172.21.15.84:0/403792748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e12820 msgr2=0x55f2d4e1cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f81193a4640 1 --2- 172.21.15.84:0/403792748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e12820 0x55f2d4e1cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f81193a4640 1 -- 172.21.15.84:0/403792748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e11f00 msgr2=0x55f2d4e122e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f81193a4640 1 --2- 172.21.15.84:0/403792748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e11f00 0x55f2d4e122e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f8113fff640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f80fc0a3e40 0x7f80fc0a6300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8108000c00 tx=0x7f8108032bf0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f81193a4640 1 -- 172.21.15.84:0/403792748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2d4d3f0d0 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.141 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f81193a4640 1 --2- 172.21.15.84:0/403792748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e1f4d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f810c099780 tx=0x7f810c09c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa0214ff355b4a49 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f80faffd640 1 -- 172.21.15.84:0/403792748 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c09cdb0 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f80faffd640 1 -- 172.21.15.84:0/403792748 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f810c0a0070 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.132+0000 7f80faffd640 1 -- 172.21.15.84:0/403792748 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c099420 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.142 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.133+0000 7f811dbf9740 1 -- 172.21.15.84:0/403792748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 msgr2=0x55f2d4e1f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.133+0000 7f811dbf9740 1 --2- 172.21.15.84:0/403792748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e1f4d0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f810c099780 tx=0x7f810c09c9b0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.133+0000 7f811dbf9740 1 -- 172.21.15.84:0/403792748 shutdown_connections 2024-04-26T18:37:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.133+0000 7f811dbf9740 1 --2- 172.21.15.84:0/403792748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e1f4d0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.133+0000 7f811dbf9740 1 --2- 172.21.15.84:0/403792748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e12820 0x55f2d4e1cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.133+0000 7f811dbf9740 1 --2- 172.21.15.84:0/403792748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e11f00 0x55f2d4e122e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.133+0000 7f811dbf9740 1 -- 172.21.15.84:0/403792748 >> 172.21.15.84:0/403792748 conn(0x55f2d4e0e410 msgr2=0x55f2d4e0e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:12.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.133+0000 7f811dbf9740 1 -- 172.21.15.84:0/403792748 shutdown_connections 2024-04-26T18:37:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.133+0000 7f811dbf9740 1 -- 172.21.15.84:0/403792748 wait complete. 2024-04-26T18:37:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.133+0000 7f811dbf9740 1 -- start start 2024-04-26T18:37:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.133+0000 7f811dbf9740 1 Processor -- start 2024-04-26T18:37:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f811dbf9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e11f00 0x55f2d4e1a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f811dbf9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e12820 0x55f2d4e1a570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f811dbf9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e1aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f811dbf9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f2d4e1fd10 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f811dbf9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f2d4cffb60 con 0x55f2d4e12820 2024-04-26T18:37:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f811dbf9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f2d4e1fe90 con 0x55f2d4e11f00 2024-04-26T18:37:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f811dbf9740 10 client.0 _ll_register_callbacks cb 0x55f2d4d242b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:37:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f81193a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e1aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f81193a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e1aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52828/0 (socket says 172.21.15.84:52828) 2024-04-26T18:37:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f81193a4640 1 -- 172.21.15.84:0/1877079710 learned_addr learned my addr 172.21.15.84:0/1877079710 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f8113fff640 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e12820 0x55f2d4e1a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f8118ba3640 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e11f00 0x55f2d4e1a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f81193a4640 1 -- 172.21.15.84:0/1877079710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e11f00 msgr2=0x55f2d4e1a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f81193a4640 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e11f00 0x55f2d4e1a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f81193a4640 1 -- 172.21.15.84:0/1877079710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e12820 msgr2=0x55f2d4e1a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:12.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f81193a4640 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e12820 0x55f2d4e1a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f81193a4640 1 -- 172.21.15.84:0/1877079710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2d4a8a210 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f8118ba3640 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e11f00 0x55f2d4e1a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:37:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.134+0000 7f8113fff640 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e12820 0x55f2d4e1a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:37:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f81193a4640 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e1aab0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f810c0ceb60 tx=0x7f810c09d5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e68f1387dea6ed server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f80f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:37:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f80f3fff640 1 -- 172.21.15.84:0/1877079710 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810c0ba070 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f80f3fff640 1 -- 172.21.15.84:0/1877079710 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8623+0+0 (secure 0 0 0) 0x7f810c0a5590 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.148 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f811dbf9740 1 -- 172.21.15.84:0/1877079710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f810c0a0070 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f80f3fff640 1 -- 172.21.15.84:0/1877079710 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f810c0a1160 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f811dbf9740 10 client.16515 fetch_fsmap learned FSMap version 29 2024-04-26T18:37:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f811dbf9740 1 -- 172.21.15.84:0/1877079710 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f2d4b56620 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f80f3fff640 1 -- 172.21.15.84:0/1877079710 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f810c0a0070 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f80f3fff640 1 -- 172.21.15.84:0/1877079710 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f810c087040 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f811dbf9740 10 client.16515 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:37:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f811dbf9740 10 client.16515 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:37:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f811dbf9740 20 client.16515 populate_metadata read hostname 'smithi084' 2024-04-26T18:37:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f811dbf9740 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f2d4e2d060 0x55f2d4e2f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.149 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f811dbf9740 4 client.16515 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:37:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f811dbf9740 1 -- 172.21.15.84:0/1877079710 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f2d4d86070 con 0x55f2d4e2d060 2024-04-26T18:37:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.135+0000 7f8118ba3640 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f2d4e2d060 0x55f2d4e2f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.136+0000 7f8118ba3640 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f2d4e2d060 0x55f2d4e2f450 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f8067fed057fbdd server_cookie=cf0d8927c71c8270 in_seq=0 out_seq=0 2024-04-26T18:37:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.136+0000 7f80f3fff640 10 client.16515 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:12.150 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f80f3fff640 1 -- 172.21.15.84:0/1877079710 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f2d4d86070 con 0x55f2d4e2d060 2024-04-26T18:37:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f80f3fff640 10 client.16515 handle_command_reply: tid=0 2024-04-26T18:37:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f811dbf9740 1 client.16515 shutdown 2024-04-26T18:37:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f811dbf9740 1 -- 172.21.15.84:0/1877079710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 msgr2=0x55f2d4e1aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f811dbf9740 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e1aab0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f810c0ceb60 tx=0x7f810c09d5f0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f811dbf9740 1 -- 172.21.15.84:0/1877079710 shutdown_connections 2024-04-26T18:37:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f811dbf9740 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f2d4e2d060 0x55f2d4e2f450 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f811dbf9740 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e1aab0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f811dbf9740 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e12820 0x55f2d4e1a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f811dbf9740 1 --2- 172.21.15.84:0/1877079710 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e11f00 0x55f2d4e1a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f80f3fff640 0 client.16515 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f811dbf9740 1 -- 172.21.15.84:0/1877079710 >> 172.21.15.84:0/1877079710 conn(0x55f2d4e0e410 msgr2=0x55f2d4e14140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f811dbf9740 1 -- 172.21.15.84:0/1877079710 shutdown_connections 2024-04-26T18:37:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f811dbf9740 1 -- 172.21.15.84:0/1877079710 wait complete. 2024-04-26T18:37:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.137+0000 7f811dbf9740 20 client.16515 trim_cache size 0 max 16384 2024-04-26T18:37:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f811dbf9740 1 Processor -- start 2024-04-26T18:37:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f811dbf9740 1 -- start start 2024-04-26T18:37:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f811dbf9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e11f00 0x55f2d4e0dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f811dbf9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e12820 0x55f2d4e0e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f811dbf9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e34a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f811dbf9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f81141ae700 con 0x55f2d4e11f00 2024-04-26T18:37:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f811dbf9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f81141b0240 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f811dbf9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81141aedd0 con 0x55f2d4e12820 2024-04-26T18:37:12.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f8118ba3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e11f00 0x55f2d4e0dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f8118ba3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e11f00 0x55f2d4e0dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52838/0 (socket says 172.21.15.84:52838) 2024-04-26T18:37:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f8118ba3640 1 -- 172.21.15.84:0/2234230034 learned_addr learned my addr 172.21.15.84:0/2234230034 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f8118ba3640 1 -- 172.21.15.84:0/2234230034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e12820 msgr2=0x55f2d4e0e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f8118ba3640 1 --2- 172.21.15.84:0/2234230034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e12820 0x55f2d4e0e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f8118ba3640 1 -- 172.21.15.84:0/2234230034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e1d0e0 msgr2=0x55f2d4e34a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f8118ba3640 1 --2- 172.21.15.84:0/2234230034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e34a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f8118ba3640 1 -- 172.21.15.84:0/2234230034 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f2d4e19000 con 0x55f2d4e11f00 2024-04-26T18:37:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.152+0000 7f8118ba3640 1 --2- 172.21.15.84:0/2234230034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e11f00 0x55f2d4e0dd30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8104009b10 tx=0x7f8104009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddf045321b11ca7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.153+0000 7f80fa7fc640 1 -- 172.21.15.84:0/2234230034 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104013ce0 con 0x55f2d4e11f00 2024-04-26T18:37:12.160 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.153+0000 7f80fa7fc640 1 -- 172.21.15.84:0/2234230034 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f810401e030 con 0x55f2d4e11f00 2024-04-26T18:37:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.153+0000 7f80fa7fc640 1 -- 172.21.15.84:0/2234230034 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8104011980 con 0x55f2d4e11f00 2024-04-26T18:37:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.153+0000 7f811dbf9740 1 -- 172.21.15.84:0/2234230034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e11f00 msgr2=0x55f2d4e0dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.153+0000 7f811dbf9740 1 --2- 172.21.15.84:0/2234230034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e11f00 0x55f2d4e0dd30 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8104009b10 tx=0x7f8104009450 comp rx=0 tx=0).stop 2024-04-26T18:37:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.153+0000 7f811dbf9740 1 -- 172.21.15.84:0/2234230034 shutdown_connections 2024-04-26T18:37:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.153+0000 7f811dbf9740 1 --2- 172.21.15.84:0/2234230034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e34a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.153+0000 7f811dbf9740 1 --2- 172.21.15.84:0/2234230034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e12820 0x55f2d4e0e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.153+0000 7f811dbf9740 1 --2- 172.21.15.84:0/2234230034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e11f00 0x55f2d4e0dd30 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.153+0000 7f811dbf9740 1 -- 172.21.15.84:0/2234230034 >> 172.21.15.84:0/2234230034 conn(0x55f2d4e0e410 msgr2=0x55f2d4e14140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.153+0000 7f811dbf9740 1 -- 172.21.15.84:0/2234230034 shutdown_connections 2024-04-26T18:37:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.153+0000 7f811dbf9740 1 -- 172.21.15.84:0/2234230034 wait complete. 2024-04-26T18:37:12.161 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f811dbf9740 1 -- start start 2024-04-26T18:37:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f811dbf9740 1 Processor -- start 2024-04-26T18:37:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f811dbf9740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e11f00 0x55f2d4e250e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f811dbf9740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e12820 0x55f2d4e25620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f811dbf9740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e220b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f811dbf9740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f81141afb70 con 0x55f2d4e12820 2024-04-26T18:37:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f811dbf9740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f81141af4a0 con 0x55f2d4e1d0e0 2024-04-26T18:37:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f811dbf9740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f81141ac700 con 0x55f2d4e11f00 2024-04-26T18:37:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f811dbf9740 10 client.0 _ll_register_callbacks cb 0x7f8104009320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:37:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f8113fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e12820 0x55f2d4e25620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f8113fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e12820 0x55f2d4e25620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:52842/0 (socket says 172.21.15.84:52842) 2024-04-26T18:37:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f8113fff640 1 -- 172.21.15.84:0/2074401246 learned_addr learned my addr 172.21.15.84:0/2074401246 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f8118ba3640 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e11f00 0x55f2d4e250e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f81193a4640 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e220b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.154+0000 7f8113fff640 1 -- 172.21.15.84:0/2074401246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e11f00 msgr2=0x55f2d4e250e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f8113fff640 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e11f00 0x55f2d4e250e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f8113fff640 1 -- 172.21.15.84:0/2074401246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e1d0e0 msgr2=0x55f2d4e220b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f8113fff640 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e220b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f8113fff640 1 -- 172.21.15.84:0/2074401246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f2d4e12650 con 0x55f2d4e12820 2024-04-26T18:37:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f81193a4640 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e220b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:37:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f8113fff640 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e12820 0x55f2d4e25620 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f810802a940 tx=0x7f8108009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5048c49074f650 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f80f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:37:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f80f3fff640 1 -- 172.21.15.84:0/2074401246 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f810804d070 con 0x55f2d4e12820 2024-04-26T18:37:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f80f3fff640 1 -- 172.21.15.84:0/2074401246 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 8623+0+0 (secure 0 0 0) 0x7f8108005240 con 0x55f2d4e12820 2024-04-26T18:37:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f811dbf9740 1 -- 172.21.15.84:0/2074401246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f810401e030 con 0x55f2d4e12820 2024-04-26T18:37:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f80f3fff640 1 -- 172.21.15.84:0/2074401246 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8108059050 con 0x55f2d4e12820 2024-04-26T18:37:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f811dbf9740 10 client.16521 fetch_fsmap learned FSMap version 29 2024-04-26T18:37:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f811dbf9740 1 -- 172.21.15.84:0/2074401246 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f2d4e93b30 con 0x55f2d4e12820 2024-04-26T18:37:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f80f3fff640 1 -- 172.21.15.84:0/2074401246 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f810401e030 con 0x55f2d4e12820 2024-04-26T18:37:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f80f3fff640 1 -- 172.21.15.84:0/2074401246 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f81080035f0 con 0x55f2d4e12820 2024-04-26T18:37:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f811dbf9740 10 client.16521 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:37:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f811dbf9740 10 client.16521 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:37:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f811dbf9740 20 client.16521 populate_metadata read hostname 'smithi084' 2024-04-26T18:37:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f811dbf9740 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f2d4e942e0 0x55f2d4e96700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f811dbf9740 4 client.16521 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:37:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.155+0000 7f811dbf9740 1 -- 172.21.15.84:0/2074401246 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f2d4d7fee0 con 0x55f2d4e942e0 2024-04-26T18:37:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.156+0000 7f8118ba3640 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f2d4e942e0 0x55f2d4e96700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.156+0000 7f80f3fff640 10 client.16521 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.156+0000 7f8118ba3640 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f2d4e942e0 0x55f2d4e96700 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a712240413af2db8 server_cookie=6641c6f837d61cd2 in_seq=0 out_seq=0 2024-04-26T18:37:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.169+0000 7f80f3fff640 1 -- 172.21.15.84:0/2074401246 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f2d4d7fee0 con 0x55f2d4e942e0 2024-04-26T18:37:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.169+0000 7f80f3fff640 10 client.16521 handle_command_reply: tid=0 2024-04-26T18:37:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.169+0000 7f811dbf9740 1 client.16521 shutdown 2024-04-26T18:37:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.169+0000 7f811dbf9740 1 -- 172.21.15.84:0/2074401246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e12820 msgr2=0x55f2d4e25620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.169+0000 7f811dbf9740 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e12820 0x55f2d4e25620 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f810802a940 tx=0x7f8108009260 comp rx=0 tx=0).stop 2024-04-26T18:37:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.169+0000 7f811dbf9740 1 -- 172.21.15.84:0/2074401246 shutdown_connections 2024-04-26T18:37:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.169+0000 7f811dbf9740 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f2d4e942e0 0x55f2d4e96700 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.169+0000 7f811dbf9740 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f2d4e1d0e0 0x55f2d4e220b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.169+0000 7f811dbf9740 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f2d4e12820 0x55f2d4e25620 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.169+0000 7f811dbf9740 1 --2- 172.21.15.84:0/2074401246 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f2d4e11f00 0x55f2d4e250e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.169+0000 7f80f3fff640 0 client.16521 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.169+0000 7f811dbf9740 1 -- 172.21.15.84:0/2074401246 >> 172.21.15.84:0/2074401246 conn(0x55f2d4e0e410 msgr2=0x55f2d4e14140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:12.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811dbf9740 1 -- 172.21.15.84:0/2074401246 shutdown_connections 2024-04-26T18:37:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811dbf9740 1 -- 172.21.15.84:0/2074401246 wait complete. 2024-04-26T18:37:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811dbf9740 20 client.16521 trim_cache size 0 max 16384 2024-04-26T18:37:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:37:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:37:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:37:12.172 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:37:12.173 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811ae07640 1 -- 172.21.15.84:0/3857440153 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f80fc0a3e40 msgr2=0x7f80fc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811ae07640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f80fc0a3e40 0x7f80fc0a6300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8108000c00 tx=0x7f8108032bf0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811ae07640 1 -- 172.21.15.84:0/3857440153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8114107830 msgr2=0x7f811419d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811ae07640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8114107830 0x7f811419d970 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f810c00eb10 tx=0x7f810c00c2d0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811ae07640 1 -- 172.21.15.84:0/3857440153 shutdown_connections 2024-04-26T18:37:12.174 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811ae07640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f80fc0a3e40 0x7f80fc0a6300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811ae07640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8114107830 0x7f811419d970 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811ae07640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f81141021e0 0x7f81141995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811ae07640 1 --2- 172.21.15.84:0/3857440153 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8114101820 0x7f81141990b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:12.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811ae07640 1 -- 172.21.15.84:0/3857440153 >> 172.21.15.84:0/3857440153 conn(0x7f81140789e0 msgr2=0x7f8114104060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811ae07640 1 -- 172.21.15.84:0/3857440153 shutdown_connections 2024-04-26T18:37:12.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:12.170+0000 7f811ae07640 1 -- 172.21.15.84:0/3857440153 wait complete. 2024-04-26T18:37:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:13 smithi174 ceph-mon[32705]: pgmap v287: 97 pgs: 97 active+clean; 2.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-26T18:37:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:13 smithi174 ceph-mon[32705]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T18:37:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:13 smithi084 ceph-mon[29141]: pgmap v287: 97 pgs: 97 active+clean; 2.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-26T18:37:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:13 smithi084 ceph-mon[29141]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T18:37:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:13 smithi103 ceph-mon[32727]: pgmap v287: 97 pgs: 97 active+clean; 2.6 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 38 op/s 2024-04-26T18:37:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:13 smithi103 ceph-mon[32727]: osdmap e102: 12 total, 12 up, 12 in 2024-04-26T18:37:15.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:15 smithi174 ceph-mon[32705]: pgmap v289: 97 pgs: 97 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 20 op/s 2024-04-26T18:37:15.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:15 smithi084 ceph-mon[29141]: pgmap v289: 97 pgs: 97 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 20 op/s 2024-04-26T18:37:15.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:15 smithi103 ceph-mon[32727]: pgmap v289: 97 pgs: 97 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 20 op/s 2024-04-26T18:37:17.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:17 smithi174 ceph-mon[32705]: pgmap v290: 97 pgs: 97 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-26T18:37:17.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:17 smithi084 ceph-mon[29141]: pgmap v290: 97 pgs: 97 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-26T18:37:17.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:17 smithi103 ceph-mon[32727]: pgmap v290: 97 pgs: 97 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-26T18:37:19.679 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:19 smithi103 ceph-mon[32727]: pgmap v291: 97 pgs: 97 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-26T18:37:19.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:19 smithi174 ceph-mon[32705]: pgmap v291: 97 pgs: 97 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-26T18:37:19.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:19 smithi084 ceph-mon[29141]: pgmap v291: 97 pgs: 97 active+clean; 2.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-26T18:37:21.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:21 smithi174 ceph-mon[32705]: pgmap v292: 97 pgs: 97 active+clean; 3.0 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:37:21.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:21 smithi084 ceph-mon[29141]: pgmap v292: 97 pgs: 97 active+clean; 3.0 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:37:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:21 smithi103 ceph-mon[32727]: pgmap v292: 97 pgs: 97 active+clean; 3.0 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:37:23.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:23 smithi174 ceph-mon[32705]: pgmap v293: 97 pgs: 97 active+clean; 3.0 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:37:23.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:23 smithi084 ceph-mon[29141]: pgmap v293: 97 pgs: 97 active+clean; 3.0 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:37:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:23 smithi103 ceph-mon[32727]: pgmap v293: 97 pgs: 97 active+clean; 3.0 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:37:25.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.266+0000 7ffa91f40640 1 -- 172.21.15.84:0/1206093182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa8c108920 msgr2=0x7ffa8c108d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.266+0000 7ffa91f40640 1 --2- 172.21.15.84:0/1206093182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa8c108920 0x7ffa8c108d00 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009930 tx=0x7ffa7c02a420 comp rx=0 tx=0).stop 2024-04-26T18:37:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.266+0000 7ffa91f40640 1 -- 172.21.15.84:0/1206093182 shutdown_connections 2024-04-26T18:37:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.266+0000 7ffa91f40640 1 --2- 172.21.15.84:0/1206093182 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa8c1032d0 0x7ffa8c10f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.266+0000 7ffa91f40640 1 --2- 172.21.15.84:0/1206093182 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa8c102910 0x7ffa8c102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.266+0000 7ffa91f40640 1 --2- 172.21.15.84:0/1206093182 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa8c108920 0x7ffa8c108d00 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.266+0000 7ffa91f40640 1 -- 172.21.15.84:0/1206093182 >> 172.21.15.84:0/1206093182 conn(0x7ffa8c079dc0 msgr2=0x7ffa8c07a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.266+0000 7ffa91f40640 1 -- 172.21.15.84:0/1206093182 shutdown_connections 2024-04-26T18:37:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.266+0000 7ffa91f40640 1 -- 172.21.15.84:0/1206093182 wait complete. 2024-04-26T18:37:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa91f40640 1 Processor -- start 2024-04-26T18:37:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa91f40640 1 -- start start 2024-04-26T18:37:25.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa91f40640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa8c102910 0x7ffa8c199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa91f40640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa8c1032d0 0x7ffa8c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa91f40640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa8c108920 0x7ffa8c19d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa91f40640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c112be0 con 0x7ffa8c108920 2024-04-26T18:37:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa91f40640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c112a60 con 0x7ffa8c1032d0 2024-04-26T18:37:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa91f40640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c112d60 con 0x7ffa8c102910 2024-04-26T18:37:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa8bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa8c108920 0x7ffa8c19d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa8bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa8c108920 0x7ffa8c19d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35696/0 (socket says 172.21.15.84:35696) 2024-04-26T18:37:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa8bfff640 1 -- 172.21.15.84:0/1726294118 learned_addr learned my addr 172.21.15.84:0/1726294118 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa8b7fe640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa8c102910 0x7ffa8c199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa83fff640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa8c1032d0 0x7ffa8c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa8bfff640 1 -- 172.21.15.84:0/1726294118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa8c102910 msgr2=0x7ffa8c199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:25.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa8bfff640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa8c102910 0x7ffa8c199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa8bfff640 1 -- 172.21.15.84:0/1726294118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa8c1032d0 msgr2=0x7ffa8c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa8bfff640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa8c1032d0 0x7ffa8c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa8bfff640 1 -- 172.21.15.84:0/1726294118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa8c19e0c0 con 0x7ffa8c108920 2024-04-26T18:37:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa83fff640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa8c1032d0 0x7ffa8c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:37:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.267+0000 7ffa8bfff640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa8c108920 0x7ffa8c19d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400e960 tx=0x7ffa8400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.268+0000 7ffa897fa640 1 -- 172.21.15.84:0/1726294118 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa84004d20 con 0x7ffa8c108920 2024-04-26T18:37:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.268+0000 7ffa897fa640 1 -- 172.21.15.84:0/1726294118 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa84005020 con 0x7ffa8c108920 2024-04-26T18:37:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.268+0000 7ffa91f40640 1 -- 172.21.15.84:0/1726294118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa8c19e3b0 con 0x7ffa8c108920 2024-04-26T18:37:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.268+0000 7ffa897fa640 1 -- 172.21.15.84:0/1726294118 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa840116d0 con 0x7ffa8c108920 2024-04-26T18:37:25.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.268+0000 7ffa91f40640 1 -- 172.21.15.84:0/1726294118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa8c1a5c40 con 0x7ffa8c108920 2024-04-26T18:37:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.270+0000 7ffa897fa640 1 -- 172.21.15.84:0/1726294118 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ffa840051d0 con 0x7ffa8c108920 2024-04-26T18:37:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.270+0000 7ffa94c5a740 1 Processor -- start 2024-04-26T18:37:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.270+0000 7ffa94c5a740 1 -- start start 2024-04-26T18:37:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.270+0000 7ffa94c5a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 0x55cb813102e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.270+0000 7ffa94c5a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb8131aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.279 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.270+0000 7ffa94c5a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8131b0e0 0x55cb8131d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.270+0000 7ffa94c5a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cb81276460 con 0x55cb81310820 2024-04-26T18:37:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.270+0000 7ffa94c5a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cb81230590 con 0x55cb8130ff00 2024-04-26T18:37:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.270+0000 7ffa94c5a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cb81272420 con 0x55cb8131b0e0 2024-04-26T18:37:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb8131aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa897fa640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ffa780a3dc0 0x7ffa780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb8131aba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35710/0 (socket says 172.21.15.84:35710) 2024-04-26T18:37:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa8b7fe640 1 -- 172.21.15.84:0/3552618906 learned_addr learned my addr 172.21.15.84:0/3552618906 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa8bfff640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ffa780a3dc0 0x7ffa780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa83fff640 1 --2- 172.21.15.84:0/3552618906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 0x55cb813102e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa83fff640 1 --2- 172.21.15.84:0/3552618906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8131b0e0 0x55cb8131d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa8b7fe640 1 -- 172.21.15.84:0/3552618906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8131b0e0 msgr2=0x55cb8131d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa8b7fe640 1 --2- 172.21.15.84:0/3552618906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8131b0e0 0x55cb8131d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.280 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa8b7fe640 1 -- 172.21.15.84:0/3552618906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 msgr2=0x55cb813102e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa8b7fe640 1 --2- 172.21.15.84:0/3552618906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 0x55cb813102e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa8b7fe640 1 -- 172.21.15.84:0/3552618906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb8123d0d0 con 0x55cb81310820 2024-04-26T18:37:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa83fff640 1 --2- 172.21.15.84:0/3552618906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8131b0e0 0x55cb8131d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:37:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa897fa640 1 -- 172.21.15.84:0/1726294118 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7ffa840d0c20 con 0x7ffa8c108920 2024-04-26T18:37:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa8b7fe640 1 --2- 172.21.15.84:0/3552618906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb8131aba0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009930 tx=0x7ffa7c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7de790cbe4412ad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa8bfff640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ffa780a3dc0 0x7ffa780a6280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa84098fe0 tx=0x7ffa84088f90 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa81ffb640 1 -- 172.21.15.84:0/3552618906 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c040070 con 0x55cb81310820 2024-04-26T18:37:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa81ffb640 1 -- 172.21.15.84:0/3552618906 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa7c0316a0 con 0x55cb81310820 2024-04-26T18:37:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.271+0000 7ffa81ffb640 1 -- 172.21.15.84:0/3552618906 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c035040 con 0x55cb81310820 2024-04-26T18:37:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.272+0000 7ffa94c5a740 1 -- 172.21.15.84:0/3552618906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 msgr2=0x55cb8131aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.272+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/3552618906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb8131aba0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c009930 tx=0x7ffa7c005fd0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.281 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.272+0000 7ffa94c5a740 1 -- 172.21.15.84:0/3552618906 shutdown_connections 2024-04-26T18:37:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.272+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/3552618906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8131b0e0 0x55cb8131d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.272+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/3552618906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb8131aba0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.272+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/3552618906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 0x55cb813102e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.272+0000 7ffa94c5a740 1 -- 172.21.15.84:0/3552618906 >> 172.21.15.84:0/3552618906 conn(0x55cb8130c410 msgr2=0x55cb8130c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.272+0000 7ffa94c5a740 1 -- 172.21.15.84:0/3552618906 shutdown_connections 2024-04-26T18:37:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.272+0000 7ffa94c5a740 1 -- 172.21.15.84:0/3552618906 wait complete. 2024-04-26T18:37:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.272+0000 7ffa94c5a740 1 -- start start 2024-04-26T18:37:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa94c5a740 1 Processor -- start 2024-04-26T18:37:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa94c5a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 0x55cb81318000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa94c5a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb81318540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa94c5a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8131b0e0 0x55cb81318a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa94c5a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55cb8131dcb0 con 0x55cb81310820 2024-04-26T18:37:25.282 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa94c5a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55cb811fdb60 con 0x55cb8130ff00 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa94c5a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55cb8131de30 con 0x55cb8131b0e0 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa94c5a740 10 client.0 _ll_register_callbacks cb 0x55cb812222b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb81318540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb81318540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35714/0 (socket says 172.21.15.84:35714) 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa8b7fe640 1 -- 172.21.15.84:0/3849660628 learned_addr learned my addr 172.21.15.84:0/3849660628 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa83fff640 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8131b0e0 0x55cb81318a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa83fff640 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 0x55cb81318000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa8b7fe640 1 -- 172.21.15.84:0/3849660628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8131b0e0 msgr2=0x55cb81318a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa8b7fe640 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8131b0e0 0x55cb81318a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa8b7fe640 1 -- 172.21.15.84:0/3849660628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 msgr2=0x55cb81318000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa8b7fe640 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 0x55cb81318000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa8b7fe640 1 -- 172.21.15.84:0/3849660628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb80f88210 con 0x55cb81310820 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.273+0000 7ffa83fff640 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 0x55cb81318000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:37:25.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa8b7fe640 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb81318540 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c031440 tx=0x7ffa7c0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=569823ef625f5127 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa6affd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa6affd640 1 -- 172.21.15.84:0/3849660628 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c040030 con 0x55cb81310820 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa6affd640 1 -- 172.21.15.84:0/3849660628 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7ffa7c004250 con 0x55cb81310820 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa94c5a740 1 -- 172.21.15.84:0/3849660628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa7c0316a0 con 0x55cb81310820 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa6affd640 1 -- 172.21.15.84:0/3849660628 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa7c056050 con 0x55cb81310820 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa94c5a740 10 client.16533 fetch_fsmap learned FSMap version 29 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa94c5a740 1 -- 172.21.15.84:0/3849660628 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb81054620 con 0x55cb81310820 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa6affd640 1 -- 172.21.15.84:0/3849660628 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa7c0316a0 con 0x55cb81310820 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa6affd640 1 -- 172.21.15.84:0/3849660628 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ffa7c0035f0 con 0x55cb81310820 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa94c5a740 10 client.16533 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa94c5a740 10 client.16533 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa94c5a740 20 client.16533 populate_metadata read hostname 'smithi084' 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cb8132af60 0x55cb8132d350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa94c5a740 4 client.16533 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.274+0000 7ffa94c5a740 1 -- 172.21.15.84:0/3849660628 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cb81284070 con 0x55cb8132af60 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.275+0000 7ffa83fff640 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cb8132af60 0x55cb8132d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.275+0000 7ffa6affd640 10 client.16533 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.275+0000 7ffa83fff640 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cb8132af60 0x55cb8132d350 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e888c4edb45235cf server_cookie=4a92dc5cb5385b54 in_seq=0 out_seq=0 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa6affd640 1 -- 172.21.15.84:0/3849660628 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cb81284070 con 0x55cb8132af60 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa6affd640 10 client.16533 handle_command_reply: tid=0 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa94c5a740 1 client.16533 shutdown 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa94c5a740 1 -- 172.21.15.84:0/3849660628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 msgr2=0x55cb81318540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb81318540 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c031440 tx=0x7ffa7c0314a0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa94c5a740 1 -- 172.21.15.84:0/3849660628 shutdown_connections 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cb8132af60 0x55cb8132d350 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8131b0e0 0x55cb81318a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb81318540 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/3849660628 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 0x55cb81318000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa6affd640 0 client.16533 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa94c5a740 1 -- 172.21.15.84:0/3849660628 >> 172.21.15.84:0/3849660628 conn(0x55cb8130c410 msgr2=0x55cb81312190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa94c5a740 1 -- 172.21.15.84:0/3849660628 shutdown_connections 2024-04-26T18:37:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa94c5a740 1 -- 172.21.15.84:0/3849660628 wait complete. 2024-04-26T18:37:25.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.276+0000 7ffa94c5a740 20 client.16533 trim_cache size 0 max 16384 2024-04-26T18:37:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa94c5a740 1 Processor -- start 2024-04-26T18:37:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa94c5a740 1 -- start start 2024-04-26T18:37:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa94c5a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8130ff00 0x55cb81326b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa94c5a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb81323460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.295 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa94c5a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8131b0e0 0x55cb81317fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa94c5a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1ae5f0 con 0x55cb81310820 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa94c5a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1b0130 con 0x55cb8131b0e0 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa94c5a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1aecc0 con 0x55cb8130ff00 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb81323460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa8b7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb81323460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35728/0 (socket says 172.21.15.84:35728) 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa8b7fe640 1 -- 172.21.15.84:0/633969675 learned_addr learned my addr 172.21.15.84:0/633969675 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa83fff640 1 --2- 172.21.15.84:0/633969675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8131b0e0 0x55cb81317fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa83fff640 1 --2- 172.21.15.84:0/633969675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8130ff00 0x55cb81326b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa8b7fe640 1 -- 172.21.15.84:0/633969675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8130ff00 msgr2=0x55cb81326b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa8b7fe640 1 --2- 172.21.15.84:0/633969675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8130ff00 0x55cb81326b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa8b7fe640 1 -- 172.21.15.84:0/633969675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8131b0e0 msgr2=0x55cb81317fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa8b7fe640 1 --2- 172.21.15.84:0/633969675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8131b0e0 0x55cb81317fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa8b7fe640 1 -- 172.21.15.84:0/633969675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cb81310650 con 0x55cb81310820 2024-04-26T18:37:25.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.291+0000 7ffa83fff640 1 --2- 172.21.15.84:0/633969675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8130ff00 0x55cb81326b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.292+0000 7ffa8b7fe640 1 --2- 172.21.15.84:0/633969675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb81323460 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c02a940 tx=0x7ffa7c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca16a6a681d6bf69 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.292+0000 7ffa817fa640 1 -- 172.21.15.84:0/633969675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c05f070 con 0x55cb81310820 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.292+0000 7ffa817fa640 1 -- 172.21.15.84:0/633969675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa7c04d070 con 0x55cb81310820 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.292+0000 7ffa817fa640 1 -- 172.21.15.84:0/633969675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa7c040070 con 0x55cb81310820 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.292+0000 7ffa94c5a740 1 -- 172.21.15.84:0/633969675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 msgr2=0x55cb81323460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.292+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/633969675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb81323460 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c02a940 tx=0x7ffa7c005fd0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.292+0000 7ffa94c5a740 1 -- 172.21.15.84:0/633969675 shutdown_connections 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.292+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/633969675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8131b0e0 0x55cb81317fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.292+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/633969675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb81310820 0x55cb81323460 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.292+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/633969675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb8130ff00 0x55cb81326b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.292+0000 7ffa94c5a740 1 -- 172.21.15.84:0/633969675 >> 172.21.15.84:0/633969675 conn(0x55cb8130c410 msgr2=0x55cb8130c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.292+0000 7ffa94c5a740 1 -- 172.21.15.84:0/633969675 shutdown_connections 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.292+0000 7ffa94c5a740 1 -- 172.21.15.84:0/633969675 wait complete. 2024-04-26T18:37:25.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa94c5a740 1 -- start start 2024-04-26T18:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa94c5a740 1 Processor -- start 2024-04-26T18:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa94c5a740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 0x55cb81389180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa94c5a740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb81310820 0x55cb813896c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa94c5a740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb8131b0e0 0x55cb81394200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa94c5a740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1afa60 con 0x55cb8131b0e0 2024-04-26T18:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa94c5a740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1af390 con 0x55cb8130ff00 2024-04-26T18:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa94c5a740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ffa8c1ac5f0 con 0x55cb81310820 2024-04-26T18:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa94c5a740 10 client.0 _ll_register_callbacks cb 0x7ffa7c031d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa83fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb8131b0e0 0x55cb81394200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa83fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb8131b0e0 0x55cb81394200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35740/0 (socket says 172.21.15.84:35740) 2024-04-26T18:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa83fff640 1 -- 172.21.15.84:0/1636874080 learned_addr learned my addr 172.21.15.84:0/1636874080 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:25.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa8b7fe640 1 --2- 172.21.15.84:0/1636874080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb81310820 0x55cb813896c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.293+0000 7ffa83fff640 1 --2- 172.21.15.84:0/1636874080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 0x55cb81389180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa83fff640 1 -- 172.21.15.84:0/1636874080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb81310820 msgr2=0x55cb813896c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa83fff640 1 --2- 172.21.15.84:0/1636874080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb81310820 0x55cb813896c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa83fff640 1 -- 172.21.15.84:0/1636874080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 msgr2=0x55cb81389180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa83fff640 1 --2- 172.21.15.84:0/1636874080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 0x55cb81389180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa83fff640 1 -- 172.21.15.84:0/1636874080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cb81394740 con 0x55cb8131b0e0 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa83fff640 1 --2- 172.21.15.84:0/1636874080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb8131b0e0 0x55cb81394200 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x55cb8130d5a0 tx=0x7ffa74005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f34a063534a9f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa6affd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa6affd640 1 -- 172.21.15.84:0/1636874080 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa74028420 con 0x55cb8131b0e0 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa94c5a740 1 -- 172.21.15.84:0/1636874080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa7c04d070 con 0x55cb8131b0e0 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa6affd640 1 -- 172.21.15.84:0/1636874080 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7ffa740285c0 con 0x55cb8131b0e0 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa6affd640 1 -- 172.21.15.84:0/1636874080 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa740287c0 con 0x55cb8131b0e0 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa94c5a740 10 client.16539 fetch_fsmap learned FSMap version 29 2024-04-26T18:37:25.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa94c5a740 1 -- 172.21.15.84:0/1636874080 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cb813958d0 con 0x55cb8131b0e0 2024-04-26T18:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa6affd640 1 -- 172.21.15.84:0/1636874080 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa7c04d070 con 0x55cb8131b0e0 2024-04-26T18:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.294+0000 7ffa6affd640 1 -- 172.21.15.84:0/1636874080 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ffa740308b0 con 0x55cb8131b0e0 2024-04-26T18:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.295+0000 7ffa94c5a740 10 client.16539 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.295+0000 7ffa94c5a740 10 client.16539 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.295+0000 7ffa94c5a740 20 client.16539 populate_metadata read hostname 'smithi084' 2024-04-26T18:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.295+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/1636874080 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cb81396000 0x55cb81398420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.295+0000 7ffa94c5a740 4 client.16539 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.295+0000 7ffa94c5a740 1 -- 172.21.15.84:0/1636874080 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cb8127dee0 con 0x55cb81396000 2024-04-26T18:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.295+0000 7ffa8b7fe640 1 --2- 172.21.15.84:0/1636874080 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cb81396000 0x55cb81398420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.295+0000 7ffa8b7fe640 1 --2- 172.21.15.84:0/1636874080 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cb81396000 0x55cb81398420 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1e507e5bf421037 server_cookie=9272f20827167013 in_seq=0 out_seq=0 2024-04-26T18:37:25.300 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.295+0000 7ffa6affd640 10 client.16539 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa6affd640 1 -- 172.21.15.84:0/1636874080 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cb8127dee0 con 0x55cb81396000 2024-04-26T18:37:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa6affd640 10 client.16539 handle_command_reply: tid=0 2024-04-26T18:37:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa94c5a740 1 client.16539 shutdown 2024-04-26T18:37:25.309 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa94c5a740 1 -- 172.21.15.84:0/1636874080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb8131b0e0 msgr2=0x55cb81394200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/1636874080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb8131b0e0 0x55cb81394200 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x55cb8130d5a0 tx=0x7ffa74005b70 comp rx=0 tx=0).stop 2024-04-26T18:37:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa94c5a740 1 -- 172.21.15.84:0/1636874080 shutdown_connections 2024-04-26T18:37:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/1636874080 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55cb81396000 0x55cb81398420 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/1636874080 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55cb8131b0e0 0x55cb81394200 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa6affd640 0 client.16539 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/1636874080 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55cb81310820 0x55cb813896c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa94c5a740 1 --2- 172.21.15.84:0/1636874080 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55cb8130ff00 0x55cb81389180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa94c5a740 1 -- 172.21.15.84:0/1636874080 >> 172.21.15.84:0/1636874080 conn(0x55cb8130c410 msgr2=0x55cb81319830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa94c5a740 1 -- 172.21.15.84:0/1636874080 shutdown_connections 2024-04-26T18:37:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa94c5a740 1 -- 172.21.15.84:0/1636874080 wait complete. 2024-04-26T18:37:25.310 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.309+0000 7ffa94c5a740 20 client.16539 trim_cache size 0 max 16384 2024-04-26T18:37:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:37:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:37:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:37:25.311 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:37:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.310+0000 7ffa91f40640 1 -- 172.21.15.84:0/1726294118 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ffa780a3dc0 msgr2=0x7ffa780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.310+0000 7ffa91f40640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ffa780a3dc0 0x7ffa780a6280 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ffa84098fe0 tx=0x7ffa84088f90 comp rx=0 tx=0).stop 2024-04-26T18:37:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.310+0000 7ffa91f40640 1 -- 172.21.15.84:0/1726294118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa8c108920 msgr2=0x7ffa8c19d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.310+0000 7ffa91f40640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa8c108920 0x7ffa8c19d990 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffa8400e960 tx=0x7ffa8400ee20 comp rx=0 tx=0).stop 2024-04-26T18:37:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.310+0000 7ffa91f40640 1 -- 172.21.15.84:0/1726294118 shutdown_connections 2024-04-26T18:37:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.310+0000 7ffa91f40640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ffa780a3dc0 0x7ffa780a6280 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.310+0000 7ffa91f40640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ffa8c108920 0x7ffa8c19d990 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.310+0000 7ffa91f40640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ffa8c1032d0 0x7ffa8c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.310+0000 7ffa91f40640 1 --2- 172.21.15.84:0/1726294118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ffa8c102910 0x7ffa8c199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.310+0000 7ffa91f40640 1 -- 172.21.15.84:0/1726294118 >> 172.21.15.84:0/1726294118 conn(0x7ffa8c079dc0 msgr2=0x7ffa8c100390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:25.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.310+0000 7ffa91f40640 1 -- 172.21.15.84:0/1726294118 shutdown_connections 2024-04-26T18:37:25.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:25.310+0000 7ffa91f40640 1 -- 172.21.15.84:0/1726294118 wait complete. 2024-04-26T18:37:25.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:25 smithi174 ceph-mon[32705]: pgmap v294: 97 pgs: 97 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-04-26T18:37:25.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:25 smithi084 ceph-mon[29141]: pgmap v294: 97 pgs: 97 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-04-26T18:37:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:25 smithi103 ceph-mon[32727]: pgmap v294: 97 pgs: 97 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 64 KiB/s wr, 23 op/s 2024-04-26T18:37:27.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:27 smithi174 ceph-mon[32705]: pgmap v295: 97 pgs: 97 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:37:27.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:27 smithi084 ceph-mon[29141]: pgmap v295: 97 pgs: 97 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:37:27.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:27 smithi103 ceph-mon[32727]: pgmap v295: 97 pgs: 97 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:37:29.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:29 smithi174 ceph-mon[32705]: pgmap v296: 97 pgs: 97 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:37:29.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:29 smithi084 ceph-mon[29141]: pgmap v296: 97 pgs: 97 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:37:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:29 smithi103 ceph-mon[32727]: pgmap v296: 97 pgs: 97 active+clean; 3.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:37:31.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:31 smithi174 ceph-mon[32705]: pgmap v297: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:37:31.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:31 smithi084 ceph-mon[29141]: pgmap v297: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:37:31.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:31 smithi103 ceph-mon[32727]: pgmap v297: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:37:33.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:33 smithi174 ceph-mon[32705]: pgmap v298: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:37:33.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:33 smithi084 ceph-mon[29141]: pgmap v298: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:37:33.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:33 smithi103 ceph-mon[32727]: pgmap v298: 97 pgs: 97 active+clean; 3.5 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:37:35.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:35 smithi084 ceph-mon[29141]: pgmap v299: 97 pgs: 97 active+clean; 3.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:37:35.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:35 smithi103 ceph-mon[32727]: pgmap v299: 97 pgs: 97 active+clean; 3.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:37:35.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:35 smithi174 ceph-mon[32705]: pgmap v299: 97 pgs: 97 active+clean; 3.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:37:36.899 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.899 INFO:tasks.workunit.client.0.smithi084.stdout:251/600: open fd = 3 2024-04-26T18:37:36.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.900 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.900 INFO:tasks.workunit.client.0.smithi084.stdout:252/600: open fd = 3 2024-04-26T18:37:36.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.900 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.900 INFO:tasks.workunit.client.0.smithi084.stdout:253/600: open fd = 3 2024-04-26T18:37:36.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.900 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.900 INFO:tasks.workunit.client.0.smithi084.stdout:254/600: open fd = 3 2024-04-26T18:37:36.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.900 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:255/600: open fd = 3 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:256/600: open fd = 3 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:257/600: open fd = 3 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:258/600: open fd = 3 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:259/600: open fd = 3 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:260/600: open fd = 3 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:261/600: open fd = 3 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.902 INFO:tasks.workunit.client.0.smithi084.stdout:262/600: open fd = 3 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:263/600: open fd = 3 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:264/600: open fd = 3 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:265/600: open fd = 3 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.903 INFO:tasks.workunit.client.0.smithi084.stdout:266/600: open fd = 3 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:267/600: open fd = 3 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:268/600: open fd = 3 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:269/600: open fd = 3 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.904 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:270/600: open fd = 3 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:271/600: open fd = 3 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:272/600: open fd = 3 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:273/600: open fd = 3 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:274/600: open fd = 3 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:275/600: open fd = 3 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:276/600: open fd = 3 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:277/600: open fd = 3 2024-04-26T18:37:36.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:278/600: open fd = 3 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:279/600: open fd = 3 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:280/600: open fd = 3 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.907 INFO:tasks.workunit.client.0.smithi084.stdout:281/600: open fd = 3 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:282/600: open fd = 3 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:283/600: open fd = 3 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:284/600: open fd = 3 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.908 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:285/600: open fd = 3 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:286/600: open fd = 3 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:287/600: open fd = 3 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:288/600: open fd = 3 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.909 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.910 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.910 INFO:tasks.workunit.client.0.smithi084.stdout:289/600: open fd = 3 2024-04-26T18:37:36.910 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.910 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.910 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.910 INFO:tasks.workunit.client.0.smithi084.stdout:290/600: open fd = 3 2024-04-26T18:37:36.910 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.910 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.910 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.910 INFO:tasks.workunit.client.0.smithi084.stdout:291/600: open fd = 3 2024-04-26T18:37:36.910 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.910 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.911 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.911 INFO:tasks.workunit.client.0.smithi084.stdout:292/600: open fd = 3 2024-04-26T18:37:36.911 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.911 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.911 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.911 INFO:tasks.workunit.client.0.smithi084.stdout:293/600: open fd = 3 2024-04-26T18:37:36.911 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.911 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.911 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.911 INFO:tasks.workunit.client.0.smithi084.stdout:294/600: open fd = 3 2024-04-26T18:37:36.911 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.911 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:295/600: open fd = 3 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:296/600: open fd = 3 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:297/600: open fd = 3 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:298/600: open fd = 3 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.912 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:299/600: open fd = 3 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:300/600: open fd = 3 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:301/600: open fd = 3 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.913 INFO:tasks.workunit.client.0.smithi084.stdout:302/600: open fd = 3 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:303/600: open fd = 3 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:304/600: open fd = 3 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:305/600: open fd = 3 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.914 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:306/600: open fd = 3 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:307/600: open fd = 3 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:308/600: open fd = 3 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:309/600: open fd = 3 2024-04-26T18:37:36.915 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.916 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.916 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.916 INFO:tasks.workunit.client.0.smithi084.stdout:310/600: open fd = 3 2024-04-26T18:37:36.916 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.916 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.916 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.916 INFO:tasks.workunit.client.0.smithi084.stdout:311/600: open fd = 3 2024-04-26T18:37:36.916 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.916 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.916 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:37:36.916 INFO:tasks.workunit.client.0.smithi084.stdout:312/600: open fd = 3 2024-04-26T18:37:36.916 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:36.916 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:37:37.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:37 smithi084 ceph-mon[29141]: pgmap v300: 97 pgs: 97 active+clean; 3.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:37:37.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:37 smithi103 ceph-mon[32727]: pgmap v300: 97 pgs: 97 active+clean; 3.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:37:37.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:37 smithi174 ceph-mon[32705]: pgmap v300: 97 pgs: 97 active+clean; 3.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:37:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.406+0000 7fa6062c8640 1 -- 172.21.15.84:0/3720276806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa600101230 msgr2=0x7fa600101610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.406+0000 7fa6062c8640 1 --2- 172.21.15.84:0/3720276806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa600101230 0x7fa600101610 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009900 tx=0x7fa5f002a410 comp rx=0 tx=0).stop 2024-04-26T18:37:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.406+0000 7fa6062c8640 1 -- 172.21.15.84:0/3720276806 shutdown_connections 2024-04-26T18:37:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.406+0000 7fa6062c8640 1 --2- 172.21.15.84:0/3720276806 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa600106300 0x7fa60010ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.406+0000 7fa6062c8640 1 --2- 172.21.15.84:0/3720276806 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa600101be0 0x7fa600105c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.406+0000 7fa6062c8640 1 --2- 172.21.15.84:0/3720276806 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa600101230 0x7fa600101610 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.406+0000 7fa6062c8640 1 -- 172.21.15.84:0/3720276806 >> 172.21.15.84:0/3720276806 conn(0x7fa600077dc0 msgr2=0x7fa6000ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.406+0000 7fa6062c8640 1 -- 172.21.15.84:0/3720276806 shutdown_connections 2024-04-26T18:37:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.406+0000 7fa6062c8640 1 -- 172.21.15.84:0/3720276806 wait complete. 2024-04-26T18:37:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa6062c8640 1 Processor -- start 2024-04-26T18:37:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa6062c8640 1 -- start start 2024-04-26T18:37:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa6062c8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa600101230 0x7fa600196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.410 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa6062c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa600106300 0x7fa6001971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa6062c8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6001976e0 0x7fa6001a7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa6062c8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001109e0 con 0x7fa600101230 2024-04-26T18:37:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa6062c8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa600110860 con 0x7fa600106300 2024-04-26T18:37:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa6062c8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa600110b60 con 0x7fa6001976e0 2024-04-26T18:37:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa600101230 0x7fa600196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa5fffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa600101230 0x7fa600196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58612/0 (socket says 172.21.15.84:58612) 2024-04-26T18:37:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa5fffff640 1 -- 172.21.15.84:0/2830940272 learned_addr learned my addr 172.21.15.84:0/2830940272 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa5ff7fe640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa600106300 0x7fa6001971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa604865640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6001976e0 0x7fa6001a7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.411 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa5fffff640 1 -- 172.21.15.84:0/2830940272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6001976e0 msgr2=0x7fa6001a7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa5fffff640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6001976e0 0x7fa6001a7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa5fffff640 1 -- 172.21.15.84:0/2830940272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa600106300 msgr2=0x7fa6001971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa5fffff640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa600106300 0x7fa6001971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.407+0000 7fa5fffff640 1 -- 172.21.15.84:0/2830940272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6001a7920 con 0x7fa600101230 2024-04-26T18:37:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.408+0000 7fa604865640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6001976e0 0x7fa6001a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:37:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.408+0000 7fa5ff7fe640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa600106300 0x7fa6001971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:37:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.408+0000 7fa5fffff640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa600101230 0x7fa600196c60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0005940 tx=0x7fa5f00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.408+0000 7fa5fd7fa640 1 -- 172.21.15.84:0/2830940272 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f003f030 con 0x7fa600101230 2024-04-26T18:37:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.408+0000 7fa5fd7fa640 1 -- 172.21.15.84:0/2830940272 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f0003cb0 con 0x7fa600101230 2024-04-26T18:37:38.412 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.408+0000 7fa5fd7fa640 1 -- 172.21.15.84:0/2830940272 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0035070 con 0x7fa600101230 2024-04-26T18:37:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.408+0000 7fa6062c8640 1 -- 172.21.15.84:0/2830940272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6001a7bb0 con 0x7fa600101230 2024-04-26T18:37:38.413 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.408+0000 7fa6062c8640 1 -- 172.21.15.84:0/2830940272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6001a80a0 con 0x7fa600101230 2024-04-26T18:37:38.427 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.411+0000 7fa5fd7fa640 1 -- 172.21.15.84:0/2830940272 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa5f0042030 con 0x7fa600101230 2024-04-26T18:37:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.411+0000 7fa608fe7740 1 Processor -- start 2024-04-26T18:37:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.411+0000 7fa608fe7740 1 -- start start 2024-04-26T18:37:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.411+0000 7fa608fe7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65be22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.411+0000 7fa608fe7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 0x557d65becba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.428 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.411+0000 7fa608fe7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65bef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.411+0000 7fa608fe7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557d65b48460 con 0x557d65bed0e0 2024-04-26T18:37:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.411+0000 7fa608fe7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d65b02590 con 0x557d65be1f00 2024-04-26T18:37:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.411+0000 7fa608fe7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d65b44420 con 0x557d65be2820 2024-04-26T18:37:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.411+0000 7fa5ff7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65bef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.411+0000 7fa5fd7fa640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa5e80a3df0 0x7fa5e80a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.429 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.411+0000 7fa5ff7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65bef4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58624/0 (socket says 172.21.15.84:58624) 2024-04-26T18:37:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.411+0000 7fa5ff7fe640 1 -- 172.21.15.84:0/2498216739 learned_addr learned my addr 172.21.15.84:0/2498216739 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa5fffff640 1 --2- 172.21.15.84:0/2498216739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 0x557d65becba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa604865640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa5e80a3df0 0x7fa5e80a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa604865640 1 --2- 172.21.15.84:0/2498216739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65be22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa5fd7fa640 1 -- 172.21.15.84:0/2830940272 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fa5f00f0de0 con 0x7fa600101230 2024-04-26T18:37:38.430 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa5ff7fe640 1 -- 172.21.15.84:0/2498216739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 msgr2=0x557d65becba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa5ff7fe640 1 --2- 172.21.15.84:0/2498216739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 0x557d65becba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa5ff7fe640 1 -- 172.21.15.84:0/2498216739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 msgr2=0x557d65be22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa5ff7fe640 1 --2- 172.21.15.84:0/2498216739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65be22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa5ff7fe640 1 -- 172.21.15.84:0/2498216739 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d65b0f0d0 con 0x557d65bed0e0 2024-04-26T18:37:38.431 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa5ff7fe640 1 --2- 172.21.15.84:0/2498216739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65bef4d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400cc40 tx=0x7fa5f400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97cef9184b4df1d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa604865640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa5e80a3df0 0x7fa5e80a62b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800afd0 tx=0x7fa5f800a510 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa5e67fc640 1 -- 172.21.15.84:0/2498216739 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4014070 con 0x557d65bed0e0 2024-04-26T18:37:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa5e67fc640 1 -- 172.21.15.84:0/2498216739 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f40047a0 con 0x557d65bed0e0 2024-04-26T18:37:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.412+0000 7fa5e67fc640 1 -- 172.21.15.84:0/2498216739 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4010650 con 0x557d65bed0e0 2024-04-26T18:37:38.432 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.413+0000 7fa608fe7740 1 -- 172.21.15.84:0/2498216739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 msgr2=0x557d65bef4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.413+0000 7fa608fe7740 1 --2- 172.21.15.84:0/2498216739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65bef4d0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400cc40 tx=0x7fa5f400e720 comp rx=0 tx=0).stop 2024-04-26T18:37:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.413+0000 7fa608fe7740 1 -- 172.21.15.84:0/2498216739 shutdown_connections 2024-04-26T18:37:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.413+0000 7fa608fe7740 1 --2- 172.21.15.84:0/2498216739 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65bef4d0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.413+0000 7fa608fe7740 1 --2- 172.21.15.84:0/2498216739 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 0x557d65becba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.433 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.413+0000 7fa608fe7740 1 --2- 172.21.15.84:0/2498216739 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65be22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.413+0000 7fa608fe7740 1 -- 172.21.15.84:0/2498216739 >> 172.21.15.84:0/2498216739 conn(0x557d65bde410 msgr2=0x557d65bde7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.413+0000 7fa608fe7740 1 -- 172.21.15.84:0/2498216739 shutdown_connections 2024-04-26T18:37:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.413+0000 7fa608fe7740 1 -- 172.21.15.84:0/2498216739 wait complete. 2024-04-26T18:37:38.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.413+0000 7fa608fe7740 1 -- start start 2024-04-26T18:37:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa608fe7740 1 Processor -- start 2024-04-26T18:37:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa608fe7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65bea010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa608fe7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65be2820 0x557d65bea550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa608fe7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65bed0e0 0x557d65beaa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa608fe7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557d65befd10 con 0x557d65be2820 2024-04-26T18:37:38.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa608fe7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557d65acfb60 con 0x557d65be1f00 2024-04-26T18:37:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa608fe7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557d65befe90 con 0x557d65bed0e0 2024-04-26T18:37:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa608fe7740 10 client.0 _ll_register_callbacks cb 0x557d65af42b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:37:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa5ff7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65be2820 0x557d65bea550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa5ff7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65be2820 0x557d65bea550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58634/0 (socket says 172.21.15.84:58634) 2024-04-26T18:37:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa5ff7fe640 1 -- 172.21.15.84:0/921414388 learned_addr learned my addr 172.21.15.84:0/921414388 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:38.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa604865640 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65bed0e0 0x557d65beaa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa5fffff640 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65bea010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa5ff7fe640 1 -- 172.21.15.84:0/921414388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65bed0e0 msgr2=0x557d65beaa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa5ff7fe640 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65bed0e0 0x557d65beaa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa5ff7fe640 1 -- 172.21.15.84:0/921414388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 msgr2=0x557d65bea010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:38.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa5ff7fe640 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65bea010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa5ff7fe640 1 -- 172.21.15.84:0/921414388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6585a210 con 0x557d65be2820 2024-04-26T18:37:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa5fffff640 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65bea010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:37:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa5ff7fe640 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65be2820 0x557d65bea550 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400cc10 tx=0x7fa5f4010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb37d944d2563c3d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa5df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:37:38.438 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.414+0000 7fa5df7fe640 1 -- 172.21.15.84:0/921414388 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f4014070 con 0x557d65be2820 2024-04-26T18:37:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa5df7fe640 1 -- 172.21.15.84:0/921414388 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fa5f4004020 con 0x557d65be2820 2024-04-26T18:37:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa608fe7740 1 -- 172.21.15.84:0/921414388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f40047a0 con 0x557d65be2820 2024-04-26T18:37:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa5df7fe640 1 -- 172.21.15.84:0/921414388 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5f400c820 con 0x557d65be2820 2024-04-26T18:37:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa608fe7740 10 client.16551 fetch_fsmap learned FSMap version 29 2024-04-26T18:37:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa608fe7740 1 -- 172.21.15.84:0/921414388 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557d65926620 con 0x557d65be2820 2024-04-26T18:37:38.439 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa5df7fe640 1 -- 172.21.15.84:0/921414388 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f40047a0 con 0x557d65be2820 2024-04-26T18:37:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa5df7fe640 1 -- 172.21.15.84:0/921414388 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa5f4022410 con 0x557d65be2820 2024-04-26T18:37:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa608fe7740 10 client.16551 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:37:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa608fe7740 10 client.16551 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:37:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa608fe7740 20 client.16551 populate_metadata read hostname 'smithi084' 2024-04-26T18:37:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa608fe7740 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557d65bfd100 0x557d65bff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.440 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa608fe7740 4 client.16551 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:37:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa608fe7740 1 -- 172.21.15.84:0/921414388 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d65b56070 con 0x557d65bfd100 2024-04-26T18:37:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.415+0000 7fa5fffff640 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557d65bfd100 0x557d65bff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.416+0000 7fa5fffff640 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557d65bfd100 0x557d65bff4f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5b3a8c935515271 server_cookie=22efc160c942ee87 in_seq=0 out_seq=0 2024-04-26T18:37:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.416+0000 7fa5df7fe640 10 client.16551 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:38.441 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.416+0000 7fa5df7fe640 1 -- 172.21.15.84:0/921414388 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d65b56070 con 0x557d65bfd100 2024-04-26T18:37:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.416+0000 7fa5df7fe640 10 client.16551 handle_command_reply: tid=0 2024-04-26T18:37:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.416+0000 7fa608fe7740 1 client.16551 shutdown 2024-04-26T18:37:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.417+0000 7fa608fe7740 1 -- 172.21.15.84:0/921414388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65be2820 msgr2=0x557d65bea550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.417+0000 7fa608fe7740 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65be2820 0x557d65bea550 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400cc10 tx=0x7fa5f4010c60 comp rx=0 tx=0).stop 2024-04-26T18:37:38.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.417+0000 7fa608fe7740 1 -- 172.21.15.84:0/921414388 shutdown_connections 2024-04-26T18:37:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.417+0000 7fa608fe7740 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557d65bfd100 0x557d65bff4f0 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.417+0000 7fa608fe7740 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65bed0e0 0x557d65beaa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.417+0000 7fa608fe7740 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65be2820 0x557d65bea550 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.417+0000 7fa5df7fe640 0 client.16551 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.417+0000 7fa608fe7740 1 --2- 172.21.15.84:0/921414388 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65bea010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.417+0000 7fa608fe7740 1 -- 172.21.15.84:0/921414388 >> 172.21.15.84:0/921414388 conn(0x557d65bde410 msgr2=0x557d65be4140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.417+0000 7fa608fe7740 1 -- 172.21.15.84:0/921414388 shutdown_connections 2024-04-26T18:37:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.417+0000 7fa608fe7740 1 -- 172.21.15.84:0/921414388 wait complete. 2024-04-26T18:37:38.444 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.417+0000 7fa608fe7740 20 client.16551 trim_cache size 0 max 16384 2024-04-26T18:37:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa608fe7740 1 Processor -- start 2024-04-26T18:37:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa608fe7740 1 -- start start 2024-04-26T18:37:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa608fe7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65bddd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa608fe7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 0x557d65bebf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa608fe7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65c04af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa608fe7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001b07f0 con 0x557d65bed0e0 2024-04-26T18:37:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa608fe7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001b2330 con 0x557d65be1f00 2024-04-26T18:37:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa608fe7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001b0ec0 con 0x557d65be2820 2024-04-26T18:37:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa604865640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65c04af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa604865640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65c04af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58650/0 (socket says 172.21.15.84:58650) 2024-04-26T18:37:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa604865640 1 -- 172.21.15.84:0/543449495 learned_addr learned my addr 172.21.15.84:0/543449495 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa5fffff640 1 --2- 172.21.15.84:0/543449495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65bddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa604865640 1 -- 172.21.15.84:0/543449495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 msgr2=0x557d65bebf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa604865640 1 --2- 172.21.15.84:0/543449495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 0x557d65bebf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa604865640 1 -- 172.21.15.84:0/543449495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 msgr2=0x557d65bddd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa604865640 1 --2- 172.21.15.84:0/543449495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65bddd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa604865640 1 -- 172.21.15.84:0/543449495 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d65befe30 con 0x557d65bed0e0 2024-04-26T18:37:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa604865640 1 --2- 172.21.15.84:0/543449495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65c04af0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800ab90 tx=0x7fa5f800a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89043b03456deff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa5e5ffb640 1 -- 172.21.15.84:0/543449495 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f802a070 con 0x557d65bed0e0 2024-04-26T18:37:38.458 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa5e5ffb640 1 -- 172.21.15.84:0/543449495 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f801cdc0 con 0x557d65bed0e0 2024-04-26T18:37:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.432+0000 7fa5e5ffb640 1 -- 172.21.15.84:0/543449495 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8025c40 con 0x557d65bed0e0 2024-04-26T18:37:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.433+0000 7fa608fe7740 1 -- 172.21.15.84:0/543449495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 msgr2=0x557d65c04af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.433+0000 7fa608fe7740 1 --2- 172.21.15.84:0/543449495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65c04af0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800ab90 tx=0x7fa5f800a310 comp rx=0 tx=0).stop 2024-04-26T18:37:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.433+0000 7fa608fe7740 1 -- 172.21.15.84:0/543449495 shutdown_connections 2024-04-26T18:37:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.433+0000 7fa608fe7740 1 --2- 172.21.15.84:0/543449495 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65c04af0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.433+0000 7fa608fe7740 1 --2- 172.21.15.84:0/543449495 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 0x557d65bebf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.433+0000 7fa608fe7740 1 --2- 172.21.15.84:0/543449495 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65bddd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.459 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.433+0000 7fa608fe7740 1 -- 172.21.15.84:0/543449495 >> 172.21.15.84:0/543449495 conn(0x557d65bde410 msgr2=0x557d65beb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.433+0000 7fa608fe7740 1 -- 172.21.15.84:0/543449495 shutdown_connections 2024-04-26T18:37:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.433+0000 7fa608fe7740 1 -- 172.21.15.84:0/543449495 wait complete. 2024-04-26T18:37:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.433+0000 7fa608fe7740 1 -- start start 2024-04-26T18:37:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa608fe7740 1 Processor -- start 2024-04-26T18:37:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa608fe7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65bf0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa608fe7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 0x557d65bf1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa608fe7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65c62360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa608fe7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001b1c60 con 0x557d65bed0e0 2024-04-26T18:37:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa608fe7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001b1590 con 0x557d65be1f00 2024-04-26T18:37:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa608fe7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6001ae7f0 con 0x557d65be2820 2024-04-26T18:37:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa608fe7740 10 client.0 _ll_register_callbacks cb 0x7fa5f400ca90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:37:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa604865640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65c62360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.460 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa604865640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65c62360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:58664/0 (socket says 172.21.15.84:58664) 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa604865640 1 -- 172.21.15.84:0/3417551087 learned_addr learned my addr 172.21.15.84:0/3417551087 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa5ff7fe640 1 --2- 172.21.15.84:0/3417551087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 0x557d65bf1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa5fffff640 1 --2- 172.21.15.84:0/3417551087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65bf0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa604865640 1 -- 172.21.15.84:0/3417551087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 msgr2=0x557d65bf1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa604865640 1 --2- 172.21.15.84:0/3417551087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 0x557d65bf1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa604865640 1 -- 172.21.15.84:0/3417551087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 msgr2=0x557d65bf0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa604865640 1 --2- 172.21.15.84:0/3417551087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65bf0f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa604865640 1 -- 172.21.15.84:0/3417551087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d65bf1d10 con 0x557d65bed0e0 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa604865640 1 --2- 172.21.15.84:0/3417551087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65c62360 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8006010 tx=0x7fa5f8025df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb8e95aa2e0772d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.434+0000 7fa5df7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa5df7fe640 1 -- 172.21.15.84:0/3417551087 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f802a030 con 0x557d65bed0e0 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa5df7fe640 1 -- 172.21.15.84:0/3417551087 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fa5f8017070 con 0x557d65bed0e0 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa608fe7740 1 -- 172.21.15.84:0/3417551087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5f801cdc0 con 0x557d65bed0e0 2024-04-26T18:37:38.461 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa5df7fe640 1 -- 172.21.15.84:0/3417551087 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5f80256c0 con 0x557d65bed0e0 2024-04-26T18:37:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa608fe7740 10 client.16557 fetch_fsmap learned FSMap version 29 2024-04-26T18:37:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa608fe7740 1 -- 172.21.15.84:0/3417551087 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557d65c639b0 con 0x557d65bed0e0 2024-04-26T18:37:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa5df7fe640 1 -- 172.21.15.84:0/3417551087 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5f801cdc0 con 0x557d65bed0e0 2024-04-26T18:37:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa5df7fe640 1 -- 172.21.15.84:0/3417551087 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa5f8024070 con 0x557d65bed0e0 2024-04-26T18:37:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa608fe7740 10 client.16557 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:37:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa608fe7740 10 client.16557 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:37:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa608fe7740 20 client.16557 populate_metadata read hostname 'smithi084' 2024-04-26T18:37:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa608fe7740 1 --2- 172.21.15.84:0/3417551087 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557d65c640a0 0x557d65c664c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:38.462 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa608fe7740 4 client.16557 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:37:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa608fe7740 1 -- 172.21.15.84:0/3417551087 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557d65b4fee0 con 0x557d65c640a0 2024-04-26T18:37:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.435+0000 7fa5fffff640 1 --2- 172.21.15.84:0/3417551087 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557d65c640a0 0x557d65c664c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.436+0000 7fa5fffff640 1 --2- 172.21.15.84:0/3417551087 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557d65c640a0 0x557d65c664c0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17cbbb14a1a02d22 server_cookie=5d43569a6ac97e6c in_seq=0 out_seq=0 2024-04-26T18:37:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.436+0000 7fa5df7fe640 10 client.16557 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa5df7fe640 1 -- 172.21.15.84:0/3417551087 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557d65b4fee0 con 0x557d65c640a0 2024-04-26T18:37:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa5df7fe640 10 client.16557 handle_command_reply: tid=0 2024-04-26T18:37:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa608fe7740 1 client.16557 shutdown 2024-04-26T18:37:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa608fe7740 1 -- 172.21.15.84:0/3417551087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 msgr2=0x557d65c62360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:38.463 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa608fe7740 1 --2- 172.21.15.84:0/3417551087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65c62360 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8006010 tx=0x7fa5f8025df0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa608fe7740 1 -- 172.21.15.84:0/3417551087 shutdown_connections 2024-04-26T18:37:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa608fe7740 1 --2- 172.21.15.84:0/3417551087 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557d65c640a0 0x557d65c664c0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa608fe7740 1 --2- 172.21.15.84:0/3417551087 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557d65bed0e0 0x557d65c62360 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa608fe7740 1 --2- 172.21.15.84:0/3417551087 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557d65be2820 0x557d65bf1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa5df7fe640 0 client.16557 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa608fe7740 1 --2- 172.21.15.84:0/3417551087 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557d65be1f00 0x557d65bf0f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa608fe7740 1 -- 172.21.15.84:0/3417551087 >> 172.21.15.84:0/3417551087 conn(0x557d65bde410 msgr2=0x557d65bec0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa608fe7740 1 -- 172.21.15.84:0/3417551087 shutdown_connections 2024-04-26T18:37:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa608fe7740 1 -- 172.21.15.84:0/3417551087 wait complete. 2024-04-26T18:37:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.452+0000 7fa608fe7740 20 client.16557 trim_cache size 0 max 16384 2024-04-26T18:37:38.464 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:37:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:37:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:37:38.465 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:37:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.453+0000 7fa6062c8640 1 -- 172.21.15.84:0/2830940272 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa5e80a3df0 msgr2=0x7fa5e80a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.453+0000 7fa6062c8640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa5e80a3df0 0x7fa5e80a62b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa5f800afd0 tx=0x7fa5f800a510 comp rx=0 tx=0).stop 2024-04-26T18:37:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.453+0000 7fa6062c8640 1 -- 172.21.15.84:0/2830940272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa600101230 msgr2=0x7fa600196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.453+0000 7fa6062c8640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa600101230 0x7fa600196c60 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0005940 tx=0x7fa5f00042f0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.453+0000 7fa6062c8640 1 -- 172.21.15.84:0/2830940272 shutdown_connections 2024-04-26T18:37:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.453+0000 7fa6062c8640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa5e80a3df0 0x7fa5e80a62b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.453+0000 7fa6062c8640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6001976e0 0x7fa6001a7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.453+0000 7fa6062c8640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa600106300 0x7fa6001971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.453+0000 7fa6062c8640 1 --2- 172.21.15.84:0/2830940272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa600101230 0x7fa600196c60 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.453+0000 7fa6062c8640 1 -- 172.21.15.84:0/2830940272 >> 172.21.15.84:0/2830940272 conn(0x7fa600077dc0 msgr2=0x7fa6000ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.453+0000 7fa6062c8640 1 -- 172.21.15.84:0/2830940272 shutdown_connections 2024-04-26T18:37:38.466 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:38.453+0000 7fa6062c8640 1 -- 172.21.15.84:0/2830940272 wait complete. 2024-04-26T18:37:39.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:39 smithi084 ceph-mon[29141]: pgmap v301: 97 pgs: 97 active+clean; 3.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:37:39.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:39 smithi103 ceph-mon[32727]: pgmap v301: 97 pgs: 97 active+clean; 3.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:37:39.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:39 smithi174 ceph-mon[32705]: pgmap v301: 97 pgs: 97 active+clean; 3.6 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:37:41.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:41 smithi084 ceph-mon[29141]: pgmap v302: 97 pgs: 97 active+clean; 3.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-04-26T18:37:41.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:41 smithi103 ceph-mon[32727]: pgmap v302: 97 pgs: 97 active+clean; 3.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-04-26T18:37:41.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:41 smithi174 ceph-mon[32705]: pgmap v302: 97 pgs: 97 active+clean; 3.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-04-26T18:37:43.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:43 smithi084 ceph-mon[29141]: pgmap v303: 97 pgs: 97 active+clean; 3.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:37:43.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:43 smithi103 ceph-mon[32727]: pgmap v303: 97 pgs: 97 active+clean; 3.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:37:43.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:43 smithi174 ceph-mon[32705]: pgmap v303: 97 pgs: 97 active+clean; 3.9 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:37:45.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:45 smithi084 ceph-mon[29141]: pgmap v304: 97 pgs: 97 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-26T18:37:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:45 smithi103 ceph-mon[32727]: pgmap v304: 97 pgs: 97 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-26T18:37:45.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:45 smithi174 ceph-mon[32705]: pgmap v304: 97 pgs: 97 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-26T18:37:46.471 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-26T18:37:46.471 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.2 2024-04-26T18:37:46.472 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:37:46.723 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:37:47.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:47 smithi084 ceph-mon[29141]: pgmap v305: 97 pgs: 97 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:37:47.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:47 smithi103 ceph-mon[32727]: pgmap v305: 97 pgs: 97 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:37:47.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:47 smithi174 ceph-mon[32705]: pgmap v305: 97 pgs: 97 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:37:48.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.500+0000 7ff50f227700 1 -- 172.21.15.84:0/2894105552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff508108500 msgr2=0x7ff5081088e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:48.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.500+0000 7ff50f227700 1 --2- 172.21.15.84:0/2894105552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff508108500 0x7ff5081088e0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009990 tx=0x7ff4f0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:37:48.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.500+0000 7ff50f227700 1 -- 172.21.15.84:0/2894105552 shutdown_connections 2024-04-26T18:37:48.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.500+0000 7ff50f227700 1 --2- 172.21.15.84:0/2894105552 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff508108500 0x7ff5081088e0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:48.502 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.500+0000 7ff50f227700 1 --2- 172.21.15.84:0/2894105552 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5081027a0 0x7ff508110210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:48.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.500+0000 7ff50f227700 1 --2- 172.21.15.84:0/2894105552 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff508101de0 0x7ff508102260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:48.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.500+0000 7ff50f227700 1 -- 172.21.15.84:0/2894105552 >> 172.21.15.84:0/2894105552 conn(0x7ff508077640 msgr2=0x7ff508077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:48.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.500+0000 7ff50f227700 1 -- 172.21.15.84:0/2894105552 shutdown_connections 2024-04-26T18:37:48.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.501+0000 7ff50f227700 1 -- 172.21.15.84:0/2894105552 wait complete. 2024-04-26T18:37:48.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.501+0000 7ff50f227700 1 Processor -- start 2024-04-26T18:37:48.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.501+0000 7ff50f227700 1 -- start start 2024-04-26T18:37:48.504 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50f227700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff508101de0 0x7ff50819f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50f227700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5081027a0 0x7ff50819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50f227700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff508108500 0x7ff508199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50f227700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff5081136f0 con 0x7ff508108500 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50f227700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff508113570 con 0x7ff508101de0 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50f227700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff508113870 con 0x7ff5081027a0 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50d7c4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff508108500 0x7ff508199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50d7c4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff508108500 0x7ff508199a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37812/0 (socket says 172.21.15.84:37812) 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50d7c4700 1 -- 172.21.15.84:0/2234568230 learned_addr learned my addr 172.21.15.84:0/2234568230 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50cfc3700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff508101de0 0x7ff50819f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff507fff700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5081027a0 0x7ff50819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50d7c4700 1 -- 172.21.15.84:0/2234568230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5081027a0 msgr2=0x7ff50819fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50d7c4700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5081027a0 0x7ff50819fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50d7c4700 1 -- 172.21.15.84:0/2234568230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff508101de0 msgr2=0x7ff50819f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:48.505 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50d7c4700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff508101de0 0x7ff50819f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50d7c4700 1 -- 172.21.15.84:0/2234568230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff50819a190 con 0x7ff508108500 2024-04-26T18:37:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50d7c4700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff508108500 0x7ff508199a30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff50000b970 tx=0x7ff50000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff505ffb700 1 -- 172.21.15.84:0/2234568230 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500019070 con 0x7ff508108500 2024-04-26T18:37:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff505ffb700 1 -- 172.21.15.84:0/2234568230 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff50000cde0 con 0x7ff508108500 2024-04-26T18:37:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff50f227700 1 -- 172.21.15.84:0/2234568230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff50819a480 con 0x7ff508108500 2024-04-26T18:37:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.502+0000 7ff505ffb700 1 -- 172.21.15.84:0/2234568230 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff500007370 con 0x7ff508108500 2024-04-26T18:37:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.503+0000 7ff50f227700 1 -- 172.21.15.84:0/2234568230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5081a6740 con 0x7ff508108500 2024-04-26T18:37:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.504+0000 7ff50f227700 1 -- 172.21.15.84:0/2234568230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff50806a110 con 0x7ff508108500 2024-04-26T18:37:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.504+0000 7ff505ffb700 1 -- 172.21.15.84:0/2234568230 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff50001c070 con 0x7ff508108500 2024-04-26T18:37:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.505+0000 7ff505ffb700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff4f40a3db0 0x7ff4f40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.505+0000 7ff50cfc3700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff4f40a3db0 0x7ff4f40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.505+0000 7ff505ffb700 1 -- 172.21.15.84:0/2234568230 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7ff5000cfc10 con 0x7ff508108500 2024-04-26T18:37:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.506+0000 7ff50cfc3700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff4f40a3db0 0x7ff4f40a6270 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0006010 tx=0x7ff4f0005960 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.510+0000 7ff505ffb700 1 -- 172.21.15.84:0/2234568230 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff50009c380 con 0x7ff508108500 2024-04-26T18:37:48.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.762+0000 7ff50f227700 1 -- 172.21.15.84:0/2234568230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff50819af60 con 0x7ff508108500 2024-04-26T18:37:48.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.763+0000 7ff505ffb700 1 -- 172.21.15.84:0/2234568230 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7ff50009bc80 con 0x7ff508108500 2024-04-26T18:37:48.766 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:37:48.766 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:37:48.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.765+0000 7ff50f227700 1 -- 172.21.15.84:0/2234568230 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff4f40a3db0 msgr2=0x7ff4f40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:48.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.765+0000 7ff50f227700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff4f40a3db0 0x7ff4f40a6270 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0006010 tx=0x7ff4f0005960 comp rx=0 tx=0).stop 2024-04-26T18:37:48.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.765+0000 7ff50f227700 1 -- 172.21.15.84:0/2234568230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff508108500 msgr2=0x7ff508199a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:48.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.765+0000 7ff50f227700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff508108500 0x7ff508199a30 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff50000b970 tx=0x7ff50000bc80 comp rx=0 tx=0).stop 2024-04-26T18:37:48.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.765+0000 7ff50f227700 1 -- 172.21.15.84:0/2234568230 shutdown_connections 2024-04-26T18:37:48.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.765+0000 7ff50f227700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff4f40a3db0 0x7ff4f40a6270 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:48.767 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.765+0000 7ff50f227700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff508108500 0x7ff508199a30 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:48.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.765+0000 7ff50f227700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff508101de0 0x7ff50819f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:48.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.765+0000 7ff50f227700 1 --2- 172.21.15.84:0/2234568230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff5081027a0 0x7ff50819fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:48.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.765+0000 7ff50f227700 1 -- 172.21.15.84:0/2234568230 >> 172.21.15.84:0/2234568230 conn(0x7ff508077640 msgr2=0x7ff508100a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:48.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.765+0000 7ff50f227700 1 -- 172.21.15.84:0/2234568230 shutdown_connections 2024-04-26T18:37:48.768 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:48.765+0000 7ff50f227700 1 -- 172.21.15.84:0/2234568230 wait complete. 2024-04-26T18:37:48.768 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:37:49.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:49 smithi084 ceph-mon[29141]: pgmap v306: 97 pgs: 97 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:37:49.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:49 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2234568230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:37:49.689 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 60 --expiration 81.2 --await-for 42.0' 2024-04-26T18:37:49.690 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db / --timeout 60 --expiration 81.2 --await-for 42.0 2024-04-26T18:37:49.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:49 smithi174 ceph-mon[32705]: pgmap v306: 97 pgs: 97 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:37:49.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:49 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2234568230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:37:49.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:49 smithi103 ceph-mon[32727]: pgmap v306: 97 pgs: 97 active+clean; 4.1 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:37:49.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:49 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2234568230' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:37:49.945 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:37:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.541+0000 7fa569f4e640 1 -- 172.21.15.84:0/738974185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa564101830 msgr2=0x7fa564101cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.541+0000 7fa569f4e640 1 --2- 172.21.15.84:0/738974185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa564101830 0x7fa564101cb0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fa558009930 tx=0x7fa55802a420 comp rx=0 tx=0).stop 2024-04-26T18:37:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.541+0000 7fa569f4e640 1 -- 172.21.15.84:0/738974185 shutdown_connections 2024-04-26T18:37:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.541+0000 7fa569f4e640 1 --2- 172.21.15.84:0/738974185 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa5641021f0 0x7fa56410e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.541+0000 7fa569f4e640 1 --2- 172.21.15.84:0/738974185 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa564101830 0x7fa564101cb0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.541+0000 7fa569f4e640 1 --2- 172.21.15.84:0/738974185 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa564107840 0x7fa564107c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.541+0000 7fa569f4e640 1 -- 172.21.15.84:0/738974185 >> 172.21.15.84:0/738974185 conn(0x7fa564078950 msgr2=0x7fa564078d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.541+0000 7fa569f4e640 1 -- 172.21.15.84:0/738974185 shutdown_connections 2024-04-26T18:37:51.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.541+0000 7fa569f4e640 1 -- 172.21.15.84:0/738974185 wait complete. 2024-04-26T18:37:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.542+0000 7fa569f4e640 1 Processor -- start 2024-04-26T18:37:51.544 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.542+0000 7fa569f4e640 1 -- start start 2024-04-26T18:37:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.542+0000 7fa569f4e640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa564101830 0x7fa564199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.545 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.542+0000 7fa569f4e640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5641021f0 0x7fa56419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.542+0000 7fa569f4e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa564107840 0x7fa5641a5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.542+0000 7fa569f4e640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa564111b00 con 0x7fa564107840 2024-04-26T18:37:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.542+0000 7fa569f4e640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa564111980 con 0x7fa5641021f0 2024-04-26T18:37:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.542+0000 7fa569f4e640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa564111c80 con 0x7fa564101830 2024-04-26T18:37:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.542+0000 7fa563fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa564107840 0x7fa5641a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa563fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa564107840 0x7fa5641a5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37828/0 (socket says 172.21.15.84:37828) 2024-04-26T18:37:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa563fff640 1 -- 172.21.15.84:0/3428297351 learned_addr learned my addr 172.21.15.84:0/3428297351 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa5637fe640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa564101830 0x7fa564199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa562ffd640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5641021f0 0x7fa56419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa563fff640 1 -- 172.21.15.84:0/3428297351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa564101830 msgr2=0x7fa564199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa563fff640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa564101830 0x7fa564199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa563fff640 1 -- 172.21.15.84:0/3428297351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5641021f0 msgr2=0x7fa56419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa563fff640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5641021f0 0x7fa56419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa563fff640 1 -- 172.21.15.84:0/3428297351 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5641a5540 con 0x7fa564107840 2024-04-26T18:37:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa5637fe640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa564101830 0x7fa564199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:37:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa562ffd640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5641021f0 0x7fa56419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:37:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa563fff640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa564107840 0x7fa5641a5000 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00cad0 tx=0x7fa55c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa560ff9640 1 -- 172.21.15.84:0/3428297351 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00ae20 con 0x7fa564107840 2024-04-26T18:37:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa560ff9640 1 -- 172.21.15.84:0/3428297351 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa55c005020 con 0x7fa564107840 2024-04-26T18:37:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa569f4e640 1 -- 172.21.15.84:0/3428297351 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5641a5710 con 0x7fa564107840 2024-04-26T18:37:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa560ff9640 1 -- 172.21.15.84:0/3428297351 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55c00f6b0 con 0x7fa564107840 2024-04-26T18:37:51.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.543+0000 7fa569f4e640 1 -- 172.21.15.84:0/3428297351 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5641a5c80 con 0x7fa564107840 2024-04-26T18:37:51.553 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.545+0000 7fa560ff9640 1 -- 172.21.15.84:0/3428297351 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa55c0051d0 con 0x7fa564107840 2024-04-26T18:37:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.545+0000 7fa56cc60740 1 Processor -- start 2024-04-26T18:37:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa56cc60740 1 -- start start 2024-04-26T18:37:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa56cc60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43304f00 0x555b433052e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa56cc60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43305820 0x555b4330fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa56cc60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b433124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa56cc60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555b4326b460 con 0x555b433100e0 2024-04-26T18:37:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa56cc60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555b43225590 con 0x555b43305820 2024-04-26T18:37:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa56cc60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555b43267420 con 0x555b43304f00 2024-04-26T18:37:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa562ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b433124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa562ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b433124d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37832/0 (socket says 172.21.15.84:37832) 2024-04-26T18:37:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa562ffd640 1 -- 172.21.15.84:0/3111033634 learned_addr learned my addr 172.21.15.84:0/3111033634 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa560ff9640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa54c0a3e40 0x7fa54c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa562ffd640 1 --2- 172.21.15.84:0/3111033634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43304f00 0x555b433052e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa5637fe640 1 --2- 172.21.15.84:0/3111033634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43305820 0x555b4330fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa563fff640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa54c0a3e40 0x7fa54c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa562ffd640 1 -- 172.21.15.84:0/3111033634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43304f00 msgr2=0x555b433052e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa562ffd640 1 --2- 172.21.15.84:0/3111033634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43304f00 0x555b433052e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa560ff9640 1 -- 172.21.15.84:0/3428297351 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fa55c0d08b0 con 0x7fa564107840 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa562ffd640 1 -- 172.21.15.84:0/3111033634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43305820 msgr2=0x555b4330fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa562ffd640 1 --2- 172.21.15.84:0/3111033634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43305820 0x555b4330fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.546+0000 7fa562ffd640 1 -- 172.21.15.84:0/3111033634 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b432320d0 con 0x555b433100e0 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.547+0000 7fa562ffd640 1 --2- 172.21.15.84:0/3111033634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b433124d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa5580057c0 tx=0x7fa5580058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c6cf58a9949b2e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.547+0000 7fa563fff640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa54c0a3e40 0x7fa54c0a6300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0ce5c0 tx=0x7fa55c06e250 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.547+0000 7fa549ffb640 1 -- 172.21.15.84:0/3111033634 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558035040 con 0x555b433100e0 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.547+0000 7fa549ffb640 1 -- 172.21.15.84:0/3111033634 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5580035f0 con 0x555b433100e0 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.547+0000 7fa549ffb640 1 -- 172.21.15.84:0/3111033634 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5580044e0 con 0x555b433100e0 2024-04-26T18:37:51.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.547+0000 7fa56cc60740 1 -- 172.21.15.84:0/3111033634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 msgr2=0x555b433124d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 --2- 172.21.15.84:0/3111033634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b433124d0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fa5580057c0 tx=0x7fa5580058d0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 -- 172.21.15.84:0/3111033634 shutdown_connections 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 --2- 172.21.15.84:0/3111033634 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b433124d0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 --2- 172.21.15.84:0/3111033634 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43305820 0x555b4330fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 --2- 172.21.15.84:0/3111033634 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43304f00 0x555b433052e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 -- 172.21.15.84:0/3111033634 >> 172.21.15.84:0/3111033634 conn(0x555b43301410 msgr2=0x555b433017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 -- 172.21.15.84:0/3111033634 shutdown_connections 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 -- 172.21.15.84:0/3111033634 wait complete. 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 -- start start 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 Processor -- start 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b43304f00 0x555b4330d160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43305820 0x555b4330d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b433100e0 0x555b4330dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555b43312cb0 con 0x555b43304f00 2024-04-26T18:37:51.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555b431f2b60 con 0x555b433100e0 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555b43312e30 con 0x555b43305820 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.548+0000 7fa56cc60740 10 client.0 _ll_register_callbacks cb 0x555b432172b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa562ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b43304f00 0x555b4330d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa562ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b43304f00 0x555b4330d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37846/0 (socket says 172.21.15.84:37846) 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa562ffd640 1 -- 172.21.15.84:0/3912933950 learned_addr learned my addr 172.21.15.84:0/3912933950 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa562ffd640 1 --2- 172.21.15.84:0/3912933950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b433100e0 0x555b4330dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa5637fe640 1 --2- 172.21.15.84:0/3912933950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43305820 0x555b4330d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa562ffd640 1 -- 172.21.15.84:0/3912933950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43305820 msgr2=0x555b4330d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa562ffd640 1 --2- 172.21.15.84:0/3912933950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43305820 0x555b4330d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa562ffd640 1 -- 172.21.15.84:0/3912933950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b433100e0 msgr2=0x555b4330dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa562ffd640 1 --2- 172.21.15.84:0/3912933950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b433100e0 0x555b4330dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa562ffd640 1 -- 172.21.15.84:0/3912933950 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b42f7d210 con 0x555b43304f00 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa562ffd640 1 --2- 172.21.15.84:0/3912933950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b43304f00 0x555b4330d160 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa558006010 tx=0x7fa558003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4431b16ba428981f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:51.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa542ffd640 1 -- 172.21.15.84:0/3912933950 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558000e60 con 0x555b43304f00 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa542ffd640 1 -- 172.21.15.84:0/3912933950 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fa5580037a0 con 0x555b43304f00 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa56cc60740 1 -- 172.21.15.84:0/3912933950 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa5580035f0 con 0x555b43304f00 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa542ffd640 1 -- 172.21.15.84:0/3912933950 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa558041030 con 0x555b43304f00 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa56cc60740 10 client.16575 fetch_fsmap learned FSMap version 29 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.549+0000 7fa56cc60740 1 -- 172.21.15.84:0/3912933950 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555b43049620 con 0x555b43304f00 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.550+0000 7fa542ffd640 1 -- 172.21.15.84:0/3912933950 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5580035f0 con 0x555b43304f00 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.550+0000 7fa542ffd640 1 -- 172.21.15.84:0/3912933950 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa55803a040 con 0x555b43304f00 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.550+0000 7fa56cc60740 10 client.16575 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.550+0000 7fa56cc60740 10 client.16575 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.550+0000 7fa56cc60740 20 client.16575 populate_metadata read hostname 'smithi084' 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.550+0000 7fa56cc60740 1 --2- 172.21.15.84:0/3912933950 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555b43320140 0x555b43322530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.550+0000 7fa56cc60740 4 client.16575 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:37:51.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.550+0000 7fa56cc60740 1 -- 172.21.15.84:0/3912933950 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b43279070 con 0x555b43320140 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.550+0000 7fa5637fe640 1 --2- 172.21.15.84:0/3912933950 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555b43320140 0x555b43322530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.550+0000 7fa5637fe640 1 --2- 172.21.15.84:0/3912933950 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555b43320140 0x555b43322530 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f19d4b4555193e2 server_cookie=2c9d65d3709b5ab3 in_seq=0 out_seq=0 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.550+0000 7fa542ffd640 10 client.16575 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa542ffd640 1 -- 172.21.15.84:0/3912933950 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555b43279070 con 0x555b43320140 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa542ffd640 10 client.16575 handle_command_reply: tid=0 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa56cc60740 1 client.16575 shutdown 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa56cc60740 1 -- 172.21.15.84:0/3912933950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b43304f00 msgr2=0x555b4330d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa56cc60740 1 --2- 172.21.15.84:0/3912933950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b43304f00 0x555b4330d160 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fa558006010 tx=0x7fa558003ef0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa56cc60740 1 -- 172.21.15.84:0/3912933950 shutdown_connections 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa56cc60740 1 --2- 172.21.15.84:0/3912933950 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555b43320140 0x555b43322530 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa56cc60740 1 --2- 172.21.15.84:0/3912933950 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b433100e0 0x555b4330dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa56cc60740 1 --2- 172.21.15.84:0/3912933950 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43305820 0x555b4330d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa56cc60740 1 --2- 172.21.15.84:0/3912933950 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b43304f00 0x555b4330d160 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa56cc60740 1 -- 172.21.15.84:0/3912933950 >> 172.21.15.84:0/3912933950 conn(0x555b43301410 msgr2=0x555b433005b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:51.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa542ffd640 0 client.16575 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa56cc60740 1 -- 172.21.15.84:0/3912933950 shutdown_connections 2024-04-26T18:37:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.551+0000 7fa56cc60740 1 -- 172.21.15.84:0/3912933950 wait complete. 2024-04-26T18:37:51.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.552+0000 7fa56cc60740 20 client.16575 trim_cache size 0 max 16384 2024-04-26T18:37:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.566+0000 7fa56cc60740 1 Processor -- start 2024-04-26T18:37:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.566+0000 7fa56cc60740 1 -- start start 2024-04-26T18:37:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.566+0000 7fa56cc60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43304f00 0x555b43300d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.566+0000 7fa56cc60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43305820 0x555b4330ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.566+0000 7fa56cc60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b43307890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.566+0000 7fa56cc60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641ae690 con 0x555b433100e0 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.566+0000 7fa56cc60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641b01d0 con 0x555b43305820 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.566+0000 7fa56cc60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641aed60 con 0x555b43304f00 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.567+0000 7fa562ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b43307890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.567+0000 7fa562ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b43307890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37854/0 (socket says 172.21.15.84:37854) 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.567+0000 7fa562ffd640 1 -- 172.21.15.84:0/2040904408 learned_addr learned my addr 172.21.15.84:0/2040904408 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.567+0000 7fa5637fe640 1 --2- 172.21.15.84:0/2040904408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43305820 0x555b4330ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.567+0000 7fa562ffd640 1 -- 172.21.15.84:0/2040904408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43304f00 msgr2=0x555b43300d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.567+0000 7fa562ffd640 1 --2- 172.21.15.84:0/2040904408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43304f00 0x555b43300d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.567+0000 7fa562ffd640 1 -- 172.21.15.84:0/2040904408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43305820 msgr2=0x555b4330ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.567+0000 7fa562ffd640 1 --2- 172.21.15.84:0/2040904408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43305820 0x555b4330ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.567+0000 7fa562ffd640 1 -- 172.21.15.84:0/2040904408 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b4330c110 con 0x555b433100e0 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.567+0000 7fa562ffd640 1 --2- 172.21.15.84:0/2040904408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b43307890 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa558004650 tx=0x7fa55803d4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4969623d25bcee88 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:51.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.567+0000 7fa5497fa640 1 -- 172.21.15.84:0/2040904408 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558035040 con 0x555b433100e0 2024-04-26T18:37:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.567+0000 7fa5497fa640 1 -- 172.21.15.84:0/2040904408 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa558009780 con 0x555b433100e0 2024-04-26T18:37:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.567+0000 7fa5497fa640 1 -- 172.21.15.84:0/2040904408 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55803d800 con 0x555b433100e0 2024-04-26T18:37:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.568+0000 7fa56cc60740 1 -- 172.21.15.84:0/2040904408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 msgr2=0x555b43307890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.568+0000 7fa56cc60740 1 --2- 172.21.15.84:0/2040904408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b43307890 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fa558004650 tx=0x7fa55803d4c0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.568+0000 7fa56cc60740 1 -- 172.21.15.84:0/2040904408 shutdown_connections 2024-04-26T18:37:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.568+0000 7fa56cc60740 1 --2- 172.21.15.84:0/2040904408 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b43307890 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.568+0000 7fa56cc60740 1 --2- 172.21.15.84:0/2040904408 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43305820 0x555b4330ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.568+0000 7fa56cc60740 1 --2- 172.21.15.84:0/2040904408 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43304f00 0x555b43300d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.568+0000 7fa56cc60740 1 -- 172.21.15.84:0/2040904408 >> 172.21.15.84:0/2040904408 conn(0x555b43301410 msgr2=0x555b4331b910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.568+0000 7fa56cc60740 1 -- 172.21.15.84:0/2040904408 shutdown_connections 2024-04-26T18:37:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.568+0000 7fa56cc60740 1 -- 172.21.15.84:0/2040904408 wait complete. 2024-04-26T18:37:51.571 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.568+0000 7fa56cc60740 1 -- start start 2024-04-26T18:37:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa56cc60740 1 Processor -- start 2024-04-26T18:37:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa56cc60740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43304f00 0x555b43322b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa56cc60740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43305820 0x555b43323090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa56cc60740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b433235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa56cc60740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641afb00 con 0x555b433100e0 2024-04-26T18:37:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa56cc60740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641af430 con 0x555b43304f00 2024-04-26T18:37:51.574 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa56cc60740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa5641ac690 con 0x555b43305820 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa56cc60740 10 client.0 _ll_register_callbacks cb 0x7fa554009550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa562ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b433235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa562ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b433235d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37870/0 (socket says 172.21.15.84:37870) 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa562ffd640 1 -- 172.21.15.84:0/4176439744 learned_addr learned my addr 172.21.15.84:0/4176439744 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa562ffd640 1 --2- 172.21.15.84:0/4176439744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43304f00 0x555b43322b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa5637fe640 1 --2- 172.21.15.84:0/4176439744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43305820 0x555b43323090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa562ffd640 1 -- 172.21.15.84:0/4176439744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43305820 msgr2=0x555b43323090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa562ffd640 1 --2- 172.21.15.84:0/4176439744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43305820 0x555b43323090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa562ffd640 1 -- 172.21.15.84:0/4176439744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43304f00 msgr2=0x555b43322b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa562ffd640 1 --2- 172.21.15.84:0/4176439744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43304f00 0x555b43322b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa562ffd640 1 -- 172.21.15.84:0/4176439744 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b43385320 con 0x555b433100e0 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa562ffd640 1 --2- 172.21.15.84:0/4176439744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b433235d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa55802a940 tx=0x7fa5580045b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598b631db604631f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.569+0000 7fa542ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:37:51.575 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa542ffd640 1 -- 172.21.15.84:0/4176439744 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55803d7b0 con 0x555b433100e0 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa542ffd640 1 -- 172.21.15.84:0/4176439744 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fa5580035f0 con 0x555b433100e0 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa56cc60740 1 -- 172.21.15.84:0/4176439744 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa558009780 con 0x555b433100e0 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa542ffd640 1 -- 172.21.15.84:0/4176439744 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa5580037f0 con 0x555b433100e0 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa56cc60740 10 client.16581 fetch_fsmap learned FSMap version 29 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa56cc60740 1 -- 172.21.15.84:0/4176439744 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555b433865b0 con 0x555b433100e0 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa542ffd640 1 -- 172.21.15.84:0/4176439744 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa558009780 con 0x555b433100e0 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa542ffd640 1 -- 172.21.15.84:0/4176439744 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fa558049b00 con 0x555b433100e0 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa56cc60740 10 client.16581 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa56cc60740 10 client.16581 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa56cc60740 20 client.16581 populate_metadata read hostname 'smithi084' 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa56cc60740 1 --2- 172.21.15.84:0/4176439744 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555b43386a80 0x555b43388e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa56cc60740 4 client.16581 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa56cc60740 1 -- 172.21.15.84:0/4176439744 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555b43272ee0 con 0x555b43386a80 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.570+0000 7fa5637fe640 1 --2- 172.21.15.84:0/4176439744 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555b43386a80 0x555b43388e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.576 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.571+0000 7fa542ffd640 10 client.16581 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:51.577 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.571+0000 7fa5637fe640 1 --2- 172.21.15.84:0/4176439744 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555b43386a80 0x555b43388e70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a940f9dd2ca6887 server_cookie=6e73a8c39ccc8c8d in_seq=0 out_seq=0 2024-04-26T18:37:51.588 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa542ffd640 1 -- 172.21.15.84:0/4176439744 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555b43272ee0 con 0x555b43386a80 2024-04-26T18:37:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa542ffd640 10 client.16581 handle_command_reply: tid=0 2024-04-26T18:37:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa56cc60740 1 client.16581 shutdown 2024-04-26T18:37:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa56cc60740 1 -- 172.21.15.84:0/4176439744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 msgr2=0x555b433235d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa56cc60740 1 --2- 172.21.15.84:0/4176439744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b433235d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fa55802a940 tx=0x7fa5580045b0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa56cc60740 1 -- 172.21.15.84:0/4176439744 shutdown_connections 2024-04-26T18:37:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa56cc60740 1 --2- 172.21.15.84:0/4176439744 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555b43386a80 0x555b43388e70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa56cc60740 1 --2- 172.21.15.84:0/4176439744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555b433100e0 0x555b433235d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa56cc60740 1 --2- 172.21.15.84:0/4176439744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555b43305820 0x555b43323090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa56cc60740 1 --2- 172.21.15.84:0/4176439744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555b43304f00 0x555b43322b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa542ffd640 0 client.16581 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa56cc60740 1 -- 172.21.15.84:0/4176439744 >> 172.21.15.84:0/4176439744 conn(0x555b43301410 msgr2=0x555b4330f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa56cc60740 1 -- 172.21.15.84:0/4176439744 shutdown_connections 2024-04-26T18:37:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa56cc60740 1 -- 172.21.15.84:0/4176439744 wait complete. 2024-04-26T18:37:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.587+0000 7fa56cc60740 20 client.16581 trim_cache size 0 max 16384 2024-04-26T18:37:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:37:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:37:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:37:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:37:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.588+0000 7fa569f4e640 1 -- 172.21.15.84:0/3428297351 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa54c0a3e40 msgr2=0x7fa54c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.588+0000 7fa569f4e640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa54c0a3e40 0x7fa54c0a6300 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa55c0ce5c0 tx=0x7fa55c06e250 comp rx=0 tx=0).stop 2024-04-26T18:37:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.588+0000 7fa569f4e640 1 -- 172.21.15.84:0/3428297351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa564107840 msgr2=0x7fa5641a5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.588+0000 7fa569f4e640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa564107840 0x7fa5641a5000 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fa55c00cad0 tx=0x7fa55c00cf90 comp rx=0 tx=0).stop 2024-04-26T18:37:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.588+0000 7fa569f4e640 1 -- 172.21.15.84:0/3428297351 shutdown_connections 2024-04-26T18:37:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.588+0000 7fa569f4e640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa54c0a3e40 0x7fa54c0a6300 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.588+0000 7fa569f4e640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa564107840 0x7fa5641a5000 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.588+0000 7fa569f4e640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa5641021f0 0x7fa56419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.588+0000 7fa569f4e640 1 --2- 172.21.15.84:0/3428297351 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa564101830 0x7fa564199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.588+0000 7fa569f4e640 1 -- 172.21.15.84:0/3428297351 >> 172.21.15.84:0/3428297351 conn(0x7fa564078950 msgr2=0x7fa564104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.588+0000 7fa569f4e640 1 -- 172.21.15.84:0/3428297351 shutdown_connections 2024-04-26T18:37:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:37:51.588+0000 7fa569f4e640 1 -- 172.21.15.84:0/3428297351 wait complete. 2024-04-26T18:37:51.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.652+0000 7fc14b0df700 1 -- 172.21.15.84:0/3298761457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc144103e90 msgr2=0x7fc144110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.652+0000 7fc14b0df700 1 --2- 172.21.15.84:0/3298761457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc144103e90 0x7fc144110380 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0099c0 tx=0x7fc13c009cd0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.652+0000 7fc14b0df700 1 -- 172.21.15.84:0/3298761457 shutdown_connections 2024-04-26T18:37:51.653 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.652+0000 7fc14b0df700 1 --2- 172.21.15.84:0/3298761457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc144103e90 0x7fc144110380 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.652+0000 7fc14b0df700 1 --2- 172.21.15.84:0/3298761457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc144108480 0x7fc144108860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.652+0000 7fc14b0df700 1 --2- 172.21.15.84:0/3298761457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc1441034d0 0x7fc144103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.652+0000 7fc14b0df700 1 -- 172.21.15.84:0/3298761457 >> 172.21.15.84:0/3298761457 conn(0x7fc14407a5d0 msgr2=0x7fc14407a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:51.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.652+0000 7fc14b0df700 1 -- 172.21.15.84:0/3298761457 shutdown_connections 2024-04-26T18:37:51.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.652+0000 7fc14b0df700 1 -- 172.21.15.84:0/3298761457 wait complete. 2024-04-26T18:37:51.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.653+0000 7fc14b0df700 1 Processor -- start 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.653+0000 7fc14b0df700 1 -- start start 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.653+0000 7fc14b0df700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1441034d0 0x7fc14419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.653+0000 7fc14b0df700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc144103e90 0x7fc14419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.653+0000 7fc14b0df700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc144108480 0x7fc1441998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.653+0000 7fc14b0df700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc144113700 con 0x7fc144103e90 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.653+0000 7fc14b0df700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc144113580 con 0x7fc1441034d0 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.653+0000 7fc14b0df700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc144113880 con 0x7fc144108480 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.653+0000 7fc13bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc144103e90 0x7fc14419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.653+0000 7fc13bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc144103e90 0x7fc14419fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37884/0 (socket says 172.21.15.84:37884) 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.653+0000 7fc13bfff700 1 -- 172.21.15.84:0/2869303771 learned_addr learned my addr 172.21.15.84:0/2869303771 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.654+0000 7fc148e7b700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1441034d0 0x7fc14419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.654+0000 7fc14967c700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc144108480 0x7fc1441998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.654+0000 7fc13bfff700 1 -- 172.21.15.84:0/2869303771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc144108480 msgr2=0x7fc1441998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.654+0000 7fc13bfff700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc144108480 0x7fc1441998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.654+0000 7fc13bfff700 1 -- 172.21.15.84:0/2869303771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1441034d0 msgr2=0x7fc14419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.654+0000 7fc13bfff700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1441034d0 0x7fc14419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.654+0000 7fc13bfff700 1 -- 172.21.15.84:0/2869303771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc14419a000 con 0x7fc144103e90 2024-04-26T18:37:51.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.654+0000 7fc13bfff700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc144103e90 0x7fc14419fc20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc13400b530 tx=0x7fc13400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:51.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.654+0000 7fc139ffb700 1 -- 172.21.15.84:0/2869303771 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134015020 con 0x7fc144103e90 2024-04-26T18:37:51.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.654+0000 7fc139ffb700 1 -- 172.21.15.84:0/2869303771 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc134011ce0 con 0x7fc144103e90 2024-04-26T18:37:51.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.654+0000 7fc14b0df700 1 -- 172.21.15.84:0/2869303771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc14419a2f0 con 0x7fc144103e90 2024-04-26T18:37:51.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.654+0000 7fc139ffb700 1 -- 172.21.15.84:0/2869303771 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc134010dd0 con 0x7fc144103e90 2024-04-26T18:37:51.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.654+0000 7fc14b0df700 1 -- 172.21.15.84:0/2869303771 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1441a65b0 con 0x7fc144103e90 2024-04-26T18:37:51.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.657+0000 7fc139ffb700 1 -- 172.21.15.84:0/2869303771 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc134018020 con 0x7fc144103e90 2024-04-26T18:37:51.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.658+0000 7fc139ffb700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc12c0a3ed0 0x7fc12c0a6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.658+0000 7fc148e7b700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc12c0a3ed0 0x7fc12c0a6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.658+0000 7fc139ffb700 1 -- 172.21.15.84:0/2869303771 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fc1340cfdd0 con 0x7fc144103e90 2024-04-26T18:37:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.658+0000 7fc148e7b700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc12c0a3ed0 0x7fc12c0a6390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc14419a9e0 tx=0x7fc14000c040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc155971b80 1 Processor -- start 2024-04-26T18:37:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc155971b80 1 -- start start 2024-04-26T18:37:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc155971b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 0x55e1e7c64610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc155971b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64b50 0x55e1e7c6f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc155971b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c71df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc155971b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e1e7bb78d0 con 0x55e1e7c64230 2024-04-26T18:37:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc155971b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e1e7b56770 con 0x55e1e7c6fa00 2024-04-26T18:37:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc155971b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e1e7bb3890 con 0x55e1e7c64b50 2024-04-26T18:37:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc148e7b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 0x55e1e7c64610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc148e7b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 0x55e1e7c64610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37900/0 (socket says 172.21.15.84:37900) 2024-04-26T18:37:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc148e7b700 1 -- 172.21.15.84:0/3412859765 learned_addr learned my addr 172.21.15.84:0/3412859765 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:51.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc14967c700 1 --2- 172.21.15.84:0/3412859765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c71df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc148e7b700 1 -- 172.21.15.84:0/3412859765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64b50 msgr2=0x55e1e7c6f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc148e7b700 1 --2- 172.21.15.84:0/3412859765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64b50 0x55e1e7c6f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc148e7b700 1 -- 172.21.15.84:0/3412859765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c6fa00 msgr2=0x55e1e7c71df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc148e7b700 1 --2- 172.21.15.84:0/3412859765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c71df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.659+0000 7fc148e7b700 1 -- 172.21.15.84:0/3412859765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e7b04a60 con 0x55e1e7c64230 2024-04-26T18:37:51.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.660+0000 7fc148e7b700 1 --2- 172.21.15.84:0/3412859765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 0x55e1e7c64610 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc14000a420 tx=0x7fc140012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6acfc16930d4bb server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:51.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.660+0000 7fc12affd700 1 -- 172.21.15.84:0/3412859765 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140026070 con 0x55e1e7c64230 2024-04-26T18:37:51.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.660+0000 7fc12affd700 1 -- 172.21.15.84:0/3412859765 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc140016e40 con 0x55e1e7c64230 2024-04-26T18:37:51.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.660+0000 7fc12affd700 1 -- 172.21.15.84:0/3412859765 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1400216a0 con 0x55e1e7c64230 2024-04-26T18:37:51.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.660+0000 7fc155971b80 1 -- 172.21.15.84:0/3412859765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 msgr2=0x55e1e7c64610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.660+0000 7fc155971b80 1 --2- 172.21.15.84:0/3412859765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 0x55e1e7c64610 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fc14000a420 tx=0x7fc140012000 comp rx=0 tx=0).stop 2024-04-26T18:37:51.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.660+0000 7fc155971b80 1 -- 172.21.15.84:0/3412859765 shutdown_connections 2024-04-26T18:37:51.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.660+0000 7fc155971b80 1 --2- 172.21.15.84:0/3412859765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 0x55e1e7c64610 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.660+0000 7fc155971b80 1 --2- 172.21.15.84:0/3412859765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c71df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.660+0000 7fc155971b80 1 --2- 172.21.15.84:0/3412859765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64b50 0x55e1e7c6f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.660+0000 7fc155971b80 1 -- 172.21.15.84:0/3412859765 >> 172.21.15.84:0/3412859765 conn(0x55e1e7b91ed0 msgr2=0x55e1e7c60720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:51.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc155971b80 1 -- 172.21.15.84:0/3412859765 shutdown_connections 2024-04-26T18:37:51.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc155971b80 1 -- 172.21.15.84:0/3412859765 wait complete. 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc155971b80 1 -- start start 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc155971b80 1 Processor -- start 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc155971b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64230 0x55e1e7c6f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc155971b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c64b50 0x55e1e7c6c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc155971b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c6ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc155971b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e1e7c725d0 con 0x55e1e7c6fa00 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc155971b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e1e7b1dae0 con 0x55e1e7c64b50 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc155971b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e1e7c72750 con 0x55e1e7c64230 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc155971b80 10 client.0 _ll_register_callbacks cb 0x55e1e7b4b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc14967c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c6ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc14967c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c6ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37912/0 (socket says 172.21.15.84:37912) 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.661+0000 7fc14967c700 1 -- 172.21.15.84:0/3925771358 learned_addr learned my addr 172.21.15.84:0/3925771358 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc148e7b700 1 --2- 172.21.15.84:0/3925771358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64230 0x55e1e7c6f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc13bfff700 1 --2- 172.21.15.84:0/3925771358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c64b50 0x55e1e7c6c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc14967c700 1 -- 172.21.15.84:0/3925771358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64230 msgr2=0x55e1e7c6f1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc14967c700 1 --2- 172.21.15.84:0/3925771358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64230 0x55e1e7c6f1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc14967c700 1 -- 172.21.15.84:0/3925771358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c64b50 msgr2=0x55e1e7c6c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc14967c700 1 --2- 172.21.15.84:0/3925771358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c64b50 0x55e1e7c6c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc14967c700 1 -- 172.21.15.84:0/3925771358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e7c6d100 con 0x55e1e7c6fa00 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc14967c700 1 --2- 172.21.15.84:0/3925771358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c6ca00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0099c0 tx=0x7fc13c00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1531c67a992e5ae server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc123fff700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc123fff700 1 -- 172.21.15.84:0/3925771358 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c003c70 con 0x55e1e7c6fa00 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc123fff700 1 -- 172.21.15.84:0/3925771358 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fc13c005070 con 0x55e1e7c6fa00 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc155971b80 1 -- 172.21.15.84:0/3925771358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e1e7bc52b0 con 0x55e1e7c6fa00 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc123fff700 1 -- 172.21.15.84:0/3925771358 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc13c030050 con 0x55e1e7c6fa00 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc155971b80 10 client.16593 fetch_fsmap learned FSMap version 29 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.662+0000 7fc155971b80 1 -- 172.21.15.84:0/3925771358 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e1e7c7d0a0 con 0x55e1e7c6fa00 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.663+0000 7fc123fff700 1 -- 172.21.15.84:0/3925771358 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e1e7bc52b0 con 0x55e1e7c6fa00 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.663+0000 7fc123fff700 1 -- 172.21.15.84:0/3925771358 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc13c02b030 con 0x55e1e7c6fa00 2024-04-26T18:37:51.668 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.663+0000 7fc155971b80 10 client.16593 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.663+0000 7fc155971b80 10 client.16593 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.663+0000 7fc155971b80 20 client.16593 populate_metadata read hostname 'smithi084' 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.663+0000 7fc155971b80 1 --2- 172.21.15.84:0/3925771358 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e1e7c7f6e0 0x55e1e7c81b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.663+0000 7fc155971b80 4 client.16593 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.663+0000 7fc155971b80 1 -- 172.21.15.84:0/3925771358 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1e7bbf350 con 0x55e1e7c7f6e0 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.663+0000 7fc148e7b700 1 --2- 172.21.15.84:0/3925771358 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e1e7c7f6e0 0x55e1e7c81b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.663+0000 7fc123fff700 10 client.16593 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.663+0000 7fc148e7b700 1 --2- 172.21.15.84:0/3925771358 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e1e7c7f6e0 0x55e1e7c81b00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a25f985d5a9c55a0 server_cookie=ce6e54e6a159e25d in_seq=0 out_seq=0 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.664+0000 7fc123fff700 1 -- 172.21.15.84:0/3925771358 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e1e7bbf350 con 0x55e1e7c7f6e0 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.664+0000 7fc123fff700 10 client.16593 handle_command_reply: tid=0 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.664+0000 7fc155971b80 1 client.16593 shutdown 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.664+0000 7fc155971b80 1 -- 172.21.15.84:0/3925771358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 msgr2=0x55e1e7c6ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.664+0000 7fc155971b80 1 --2- 172.21.15.84:0/3925771358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c6ca00 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc13c0099c0 tx=0x7fc13c00bee0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.669 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.664+0000 7fc155971b80 1 -- 172.21.15.84:0/3925771358 shutdown_connections 2024-04-26T18:37:51.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.664+0000 7fc155971b80 1 --2- 172.21.15.84:0/3925771358 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e1e7c7f6e0 0x55e1e7c81b00 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.664+0000 7fc123fff700 0 client.16593 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:51.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.664+0000 7fc155971b80 1 --2- 172.21.15.84:0/3925771358 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c6ca00 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.664+0000 7fc155971b80 1 --2- 172.21.15.84:0/3925771358 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64230 0x55e1e7c6f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.664+0000 7fc155971b80 1 --2- 172.21.15.84:0/3925771358 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c64b50 0x55e1e7c6c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.664+0000 7fc155971b80 1 -- 172.21.15.84:0/3925771358 >> 172.21.15.84:0/3925771358 conn(0x55e1e7b91ed0 msgr2=0x55e1e7c60350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:51.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.665+0000 7fc155971b80 1 -- 172.21.15.84:0/3925771358 shutdown_connections 2024-04-26T18:37:51.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.665+0000 7fc155971b80 1 -- 172.21.15.84:0/3925771358 wait complete. 2024-04-26T18:37:51.670 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.665+0000 7fc155971b80 20 client.16593 trim_cache size 0 max 16384 2024-04-26T18:37:51.670 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:51 smithi084 ceph-mon[29141]: pgmap v307: 97 pgs: 97 active+clean; 4.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-26T18:37:51.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.684+0000 7fc155971b80 1 Processor -- start 2024-04-26T18:37:51.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.684+0000 7fc155971b80 1 -- start start 2024-04-26T18:37:51.686 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc155971b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 0x55e1e7c6e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc155971b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64b50 0x55e1e7c87390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc155971b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c878d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc155971b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441adcd0 con 0x55e1e7c64230 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc155971b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441af810 con 0x55e1e7c6fa00 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc155971b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441ae3a0 con 0x55e1e7c64b50 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc148e7b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 0x55e1e7c6e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc148e7b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 0x55e1e7c6e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37920/0 (socket says 172.21.15.84:37920) 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc148e7b700 1 -- 172.21.15.84:0/1546774748 learned_addr learned my addr 172.21.15.84:0/1546774748 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc14967c700 1 --2- 172.21.15.84:0/1546774748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c878d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc148e7b700 1 -- 172.21.15.84:0/1546774748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64b50 msgr2=0x55e1e7c87390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc148e7b700 1 --2- 172.21.15.84:0/1546774748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64b50 0x55e1e7c87390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc148e7b700 1 -- 172.21.15.84:0/1546774748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c6fa00 msgr2=0x55e1e7c878d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc148e7b700 1 --2- 172.21.15.84:0/1546774748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c878d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.687 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc148e7b700 1 -- 172.21.15.84:0/1546774748 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1e7c82160 con 0x55e1e7c64230 2024-04-26T18:37:51.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc14967c700 1 --2- 172.21.15.84:0/1546774748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c878d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:37:51.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc148e7b700 1 --2- 172.21.15.84:0/1546774748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 0x55e1e7c6e7c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc1400047b0 tx=0x7fc140004c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5445411e4c58cb5b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:51.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc12a7fc700 1 -- 172.21.15.84:0/1546774748 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc140026040 con 0x55e1e7c64230 2024-04-26T18:37:51.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc12a7fc700 1 -- 172.21.15.84:0/1546774748 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc140004dc0 con 0x55e1e7c64230 2024-04-26T18:37:51.688 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.685+0000 7fc12a7fc700 1 -- 172.21.15.84:0/1546774748 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1400216a0 con 0x55e1e7c64230 2024-04-26T18:37:51.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.686+0000 7fc155971b80 1 -- 172.21.15.84:0/1546774748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 msgr2=0x55e1e7c6e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.686+0000 7fc155971b80 1 --2- 172.21.15.84:0/1546774748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 0x55e1e7c6e7c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc1400047b0 tx=0x7fc140004c20 comp rx=0 tx=0).stop 2024-04-26T18:37:51.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.686+0000 7fc155971b80 1 -- 172.21.15.84:0/1546774748 shutdown_connections 2024-04-26T18:37:51.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.686+0000 7fc155971b80 1 --2- 172.21.15.84:0/1546774748 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c64230 0x55e1e7c6e7c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.686+0000 7fc155971b80 1 --2- 172.21.15.84:0/1546774748 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c878d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.686+0000 7fc155971b80 1 --2- 172.21.15.84:0/1546774748 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64b50 0x55e1e7c87390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.686+0000 7fc155971b80 1 -- 172.21.15.84:0/1546774748 >> 172.21.15.84:0/1546774748 conn(0x55e1e7b91ed0 msgr2=0x55e1e7c67b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:51.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.686+0000 7fc155971b80 1 -- 172.21.15.84:0/1546774748 shutdown_connections 2024-04-26T18:37:51.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.686+0000 7fc155971b80 1 -- 172.21.15.84:0/1546774748 wait complete. 2024-04-26T18:37:51.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc155971b80 1 -- start start 2024-04-26T18:37:51.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc155971b80 1 Processor -- start 2024-04-26T18:37:51.691 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc155971b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c64230 0x55e1e7c77670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc155971b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64b50 0x55e1e7c77bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc155971b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c75c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc155971b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441af140 con 0x55e1e7c6fa00 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc155971b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441aea70 con 0x55e1e7c64230 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc155971b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc1441abc90 con 0x55e1e7c64b50 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc14967c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c75c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc14967c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c75c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37926/0 (socket says 172.21.15.84:37926) 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc14967c700 1 -- 172.21.15.84:0/627171299 learned_addr learned my addr 172.21.15.84:0/627171299 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc155971b80 10 client.0 _ll_register_callbacks cb 0x55e1e7b4b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc148e7b700 1 --2- 172.21.15.84:0/627171299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c64230 0x55e1e7c77670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.687+0000 7fc13bfff700 1 --2- 172.21.15.84:0/627171299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64b50 0x55e1e7c77bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc14967c700 1 -- 172.21.15.84:0/627171299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64b50 msgr2=0x55e1e7c77bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc14967c700 1 --2- 172.21.15.84:0/627171299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64b50 0x55e1e7c77bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc14967c700 1 -- 172.21.15.84:0/627171299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c64230 msgr2=0x55e1e7c77670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:37:51.692 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc14967c700 1 --2- 172.21.15.84:0/627171299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c64230 0x55e1e7c77670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc14967c700 1 -- 172.21.15.84:0/627171299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1e7c76340 con 0x55e1e7c6fa00 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc14967c700 1 --2- 172.21.15.84:0/627171299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c75c90 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc13c023830 tx=0x7fc13c009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d707466061212673 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc123fff700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc123fff700 1 -- 172.21.15.84:0/627171299 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc13c038070 con 0x55e1e7c6fa00 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc123fff700 1 -- 172.21.15.84:0/627171299 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7fc13c003c70 con 0x55e1e7c6fa00 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc155971b80 1 -- 172.21.15.84:0/627171299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc140004dc0 con 0x55e1e7c6fa00 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc123fff700 1 -- 172.21.15.84:0/627171299 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc13c041050 con 0x55e1e7c6fa00 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc155971b80 10 client.16599 fetch_fsmap learned FSMap version 29 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc155971b80 1 -- 172.21.15.84:0/627171299 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e1e7ce66c0 con 0x55e1e7c6fa00 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc123fff700 1 -- 172.21.15.84:0/627171299 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc140004dc0 con 0x55e1e7c6fa00 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc123fff700 1 -- 172.21.15.84:0/627171299 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc13c03b020 con 0x55e1e7c6fa00 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.688+0000 7fc155971b80 10 client.16599 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.689+0000 7fc155971b80 10 client.16599 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.689+0000 7fc155971b80 20 client.16599 populate_metadata read hostname 'smithi084' 2024-04-26T18:37:51.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.689+0000 7fc155971b80 1 --2- 172.21.15.84:0/627171299 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e1e7ce6e00 0x55e1e7ce91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:37:51.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.689+0000 7fc155971b80 4 client.16599 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.2, "await_for": 42.0}] 2024-04-26T18:37:51.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.689+0000 7fc155971b80 1 -- 172.21.15.84:0/627171299 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.2, "await_for": 42.0}) -- 0x55e1e7a373e0 con 0x55e1e7ce6e00 2024-04-26T18:37:51.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.689+0000 7fc148e7b700 1 --2- 172.21.15.84:0/627171299 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e1e7ce6e00 0x55e1e7ce91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:37:51.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.689+0000 7fc123fff700 10 client.16599 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:51.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.689+0000 7fc148e7b700 1 --2- 172.21.15.84:0/627171299 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e1e7ce6e00 0x55e1e7ce91f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76c737e84733e4ca server_cookie=bce7adcb28ea2e49 in_seq=0 out_seq=0 2024-04-26T18:37:51.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.706+0000 7fc123fff700 1 -- 172.21.15.84:0/627171299 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55e1e7a373e0 con 0x55e1e7ce6e00 2024-04-26T18:37:51.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.706+0000 7fc123fff700 10 client.16599 handle_command_reply: tid=0 2024-04-26T18:37:51.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.706+0000 7fc155971b80 1 client.16599 shutdown 2024-04-26T18:37:51.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.706+0000 7fc155971b80 1 -- 172.21.15.84:0/627171299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 msgr2=0x55e1e7c75c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.706+0000 7fc155971b80 1 --2- 172.21.15.84:0/627171299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c75c90 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fc13c023830 tx=0x7fc13c009290 comp rx=0 tx=0).stop 2024-04-26T18:37:51.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.706+0000 7fc155971b80 1 -- 172.21.15.84:0/627171299 shutdown_connections 2024-04-26T18:37:51.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.706+0000 7fc155971b80 1 --2- 172.21.15.84:0/627171299 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e1e7ce6e00 0x55e1e7ce91f0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.706+0000 7fc155971b80 1 --2- 172.21.15.84:0/627171299 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e1e7c6fa00 0x55e1e7c75c90 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.708 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.706+0000 7fc123fff700 0 client.16599 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:37:51.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.706+0000 7fc155971b80 1 --2- 172.21.15.84:0/627171299 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e1e7c64230 0x55e1e7c77670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.706+0000 7fc155971b80 1 --2- 172.21.15.84:0/627171299 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e1e7c64b50 0x55e1e7c77bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.706+0000 7fc155971b80 1 -- 172.21.15.84:0/627171299 >> 172.21.15.84:0/627171299 conn(0x55e1e7b91ed0 msgr2=0x55e1e7c67260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:51.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.707+0000 7fc155971b80 1 -- 172.21.15.84:0/627171299 shutdown_connections 2024-04-26T18:37:51.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.707+0000 7fc155971b80 1 -- 172.21.15.84:0/627171299 wait complete. 2024-04-26T18:37:51.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.707+0000 7fc155971b80 20 client.16599 trim_cache size 0 max 16384 2024-04-26T18:37:51.709 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:37:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:37:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:37:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 26, 2024-04-26T18:37:51.709 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:37:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "400c61dd": { 2024-04-26T18:37:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 26, 2024-04-26T18:37:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:37:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:37:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:37:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:37:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:37:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:37:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 81.2, 2024-04-26T18:37:51.710 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:37:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:37:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:37:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:37:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:37:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:37:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:37:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:37:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:37:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:37:51.711 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:37:51.711 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:37:51.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.707+0000 7fc14b0df700 1 -- 172.21.15.84:0/2869303771 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc12c0a3ed0 msgr2=0x7fc12c0a6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.707+0000 7fc14b0df700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc12c0a3ed0 0x7fc12c0a6390 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc14419a9e0 tx=0x7fc14000c040 comp rx=0 tx=0).stop 2024-04-26T18:37:51.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.707+0000 7fc14b0df700 1 -- 172.21.15.84:0/2869303771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc144103e90 msgr2=0x7fc14419fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:37:51.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.707+0000 7fc14b0df700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc144103e90 0x7fc14419fc20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc13400b530 tx=0x7fc13400b840 comp rx=0 tx=0).stop 2024-04-26T18:37:51.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.707+0000 7fc14b0df700 1 -- 172.21.15.84:0/2869303771 shutdown_connections 2024-04-26T18:37:51.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.707+0000 7fc14b0df700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc12c0a3ed0 0x7fc12c0a6390 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.707+0000 7fc14b0df700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc144108480 0x7fc1441998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.707+0000 7fc14b0df700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc1441034d0 0x7fc14419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.707+0000 7fc14b0df700 1 --2- 172.21.15.84:0/2869303771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc144103e90 0x7fc14419fc20 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:37:51.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.707+0000 7fc14b0df700 1 -- 172.21.15.84:0/2869303771 >> 172.21.15.84:0/2869303771 conn(0x7fc14407a5d0 msgr2=0x7fc1441014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:37:51.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.708+0000 7fc14b0df700 1 -- 172.21.15.84:0/2869303771 shutdown_connections 2024-04-26T18:37:51.713 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:37:51.708+0000 7fc14b0df700 1 -- 172.21.15.84:0/2869303771 wait complete. 2024-04-26T18:37:51.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:51 smithi103 ceph-mon[32727]: pgmap v307: 97 pgs: 97 active+clean; 4.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-26T18:37:51.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:51 smithi174 ceph-mon[32705]: pgmap v307: 97 pgs: 97 active+clean; 4.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-26T18:37:52.439 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '400c61dd', quiesce took 6.0 seconds. Will release after: 21.2 2024-04-26T18:37:53.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:53 smithi084 ceph-mon[29141]: pgmap v308: 97 pgs: 97 active+clean; 4.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-26T18:37:53.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:53 smithi103 ceph-mon[32727]: pgmap v308: 97 pgs: 97 active+clean; 4.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-26T18:37:53.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:53 smithi174 ceph-mon[32705]: pgmap v308: 97 pgs: 97 active+clean; 4.3 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-26T18:37:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:55 smithi084 ceph-mon[29141]: pgmap v309: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 21 op/s 2024-04-26T18:37:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:55 smithi103 ceph-mon[32727]: pgmap v309: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 21 op/s 2024-04-26T18:37:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:55 smithi174 ceph-mon[32705]: pgmap v309: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 51 KiB/s wr, 21 op/s 2024-04-26T18:37:57.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:57 smithi084 ceph-mon[29141]: pgmap v310: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-26T18:37:57.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:57 smithi103 ceph-mon[32727]: pgmap v310: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-26T18:37:57.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:57 smithi174 ceph-mon[32705]: pgmap v310: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-26T18:37:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:37:59 smithi084 ceph-mon[29141]: pgmap v311: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-26T18:37:59.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:37:59 smithi103 ceph-mon[32727]: pgmap v311: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-26T18:37:59.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:37:59 smithi174 ceph-mon[32705]: pgmap v311: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-26T18:38:01.624 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:01 smithi084 ceph-mon[29141]: pgmap v312: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-04-26T18:38:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:01 smithi103 ceph-mon[32727]: pgmap v312: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-04-26T18:38:01.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:01 smithi174 ceph-mon[32705]: pgmap v312: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-04-26T18:38:03.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:03 smithi084 ceph-mon[29141]: pgmap v313: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-26T18:38:03.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:03 smithi103 ceph-mon[32727]: pgmap v313: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-26T18:38:03.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:03 smithi174 ceph-mon[32705]: pgmap v313: 97 pgs: 97 active+clean; 4.4 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 17 KiB/s wr, 12 op/s 2024-04-26T18:38:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.685+0000 7f455ba37640 1 -- 172.21.15.84:0/2187576315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4554107c60 msgr2=0x7f455410f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.685+0000 7f455ba37640 1 --2- 172.21.15.84:0/2187576315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4554107c60 0x7f455410f830 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f454c009900 tx=0x7f454c02a410 comp rx=0 tx=0).stop 2024-04-26T18:38:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.685+0000 7f455ba37640 1 -- 172.21.15.84:0/2187576315 shutdown_connections 2024-04-26T18:38:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.685+0000 7f455ba37640 1 --2- 172.21.15.84:0/2187576315 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4554107c60 0x7f455410f830 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.687 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.685+0000 7f455ba37640 1 --2- 172.21.15.84:0/2187576315 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4554101a50 0x7f4554107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.685+0000 7f455ba37640 1 --2- 172.21.15.84:0/2187576315 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f45541010a0 0x7f4554101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.685+0000 7f455ba37640 1 -- 172.21.15.84:0/2187576315 >> 172.21.15.84:0/2187576315 conn(0x7f4554077dc0 msgr2=0x7f45540ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.685+0000 7f455ba37640 1 -- 172.21.15.84:0/2187576315 shutdown_connections 2024-04-26T18:38:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.685+0000 7f455ba37640 1 -- 172.21.15.84:0/2187576315 wait complete. 2024-04-26T18:38:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f455ba37640 1 Processor -- start 2024-04-26T18:38:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f455ba37640 1 -- start start 2024-04-26T18:38:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f455ba37640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45541010a0 0x7f4554198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f455ba37640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4554107c60 0x7f45541994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f455ba37640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f455419d860 0x7f455419dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f455ba37640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4554112bc0 con 0x7f45541010a0 2024-04-26T18:38:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f455ba37640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4554112a40 con 0x7f4554107c60 2024-04-26T18:38:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f455ba37640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4554112d40 con 0x7f455419d860 2024-04-26T18:38:04.688 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f45597d3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45541010a0 0x7f4554198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f45597d3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45541010a0 0x7f4554198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36654/0 (socket says 172.21.15.84:36654) 2024-04-26T18:38:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f45597d3640 1 -- 172.21.15.84:0/118074098 learned_addr learned my addr 172.21.15.84:0/118074098 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f45597d3640 1 -- 172.21.15.84:0/118074098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f455419d860 msgr2=0x7f455419dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f45597d3640 1 --2- 172.21.15.84:0/118074098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f455419d860 0x7f455419dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f45597d3640 1 -- 172.21.15.84:0/118074098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4554107c60 msgr2=0x7f45541994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f45597d3640 1 --2- 172.21.15.84:0/118074098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4554107c60 0x7f45541994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f45597d3640 1 -- 172.21.15.84:0/118074098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f455419e350 con 0x7f45541010a0 2024-04-26T18:38:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.686+0000 7f45597d3640 1 --2- 172.21.15.84:0/118074098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45541010a0 0x7f4554198fa0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f455000cce0 tx=0x7f455000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.687+0000 7f454a7fc640 1 -- 172.21.15.84:0/118074098 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455000f650 con 0x7f45541010a0 2024-04-26T18:38:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.687+0000 7f454a7fc640 1 -- 172.21.15.84:0/118074098 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f455000fca0 con 0x7f45541010a0 2024-04-26T18:38:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.687+0000 7f455ba37640 1 -- 172.21.15.84:0/118074098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f455419e640 con 0x7f45541010a0 2024-04-26T18:38:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.687+0000 7f454a7fc640 1 -- 172.21.15.84:0/118074098 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4550018b40 con 0x7f45541010a0 2024-04-26T18:38:04.689 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.687+0000 7f455ba37640 1 -- 172.21.15.84:0/118074098 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45541a5e30 con 0x7f45541010a0 2024-04-26T18:38:04.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.689+0000 7f454a7fc640 1 -- 172.21.15.84:0/118074098 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f455000f7f0 con 0x7f45541010a0 2024-04-26T18:38:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f455e82b740 1 Processor -- start 2024-04-26T18:38:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f455e82b740 1 -- start start 2024-04-26T18:38:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f455e82b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f5144e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f455e82b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144e820 0x559f51458ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f454a7fc640 1 --2- 172.21.15.84:0/118074098 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f45400a3dc0 0x7f45400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f455e82b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f514590e0 0x559f5145b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f455e82b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559f513b4460 con 0x559f5144df00 2024-04-26T18:38:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f455e82b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559f5136e590 con 0x559f5144e820 2024-04-26T18:38:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f455e82b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559f513b0420 con 0x559f514590e0 2024-04-26T18:38:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f4559fd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f5144e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f4559fd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f5144e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36664/0 (socket says 172.21.15.84:36664) 2024-04-26T18:38:04.697 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f4559fd4640 1 -- 172.21.15.84:0/1520070028 learned_addr learned my addr 172.21.15.84:0/1520070028 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f45597d3640 1 --2- 172.21.15.84:0/118074098 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f45400a3dc0 0x7f45400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f4558fd2640 1 --2- 172.21.15.84:0/1520070028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144e820 0x559f51458ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f4559fd4640 1 --2- 172.21.15.84:0/1520070028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f514590e0 0x559f5145b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f454a7fc640 1 -- 172.21.15.84:0/118074098 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f45500d0dd0 con 0x7f45541010a0 2024-04-26T18:38:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f4559fd4640 1 -- 172.21.15.84:0/1520070028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f514590e0 msgr2=0x559f5145b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f4559fd4640 1 --2- 172.21.15.84:0/1520070028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f514590e0 0x559f5145b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f4559fd4640 1 -- 172.21.15.84:0/1520070028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144e820 msgr2=0x559f51458ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f4559fd4640 1 --2- 172.21.15.84:0/1520070028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144e820 0x559f51458ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.690+0000 7f4559fd4640 1 -- 172.21.15.84:0/1520070028 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f5137b0d0 con 0x559f5144df00 2024-04-26T18:38:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.691+0000 7f45597d3640 1 --2- 172.21.15.84:0/118074098 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f45400a3dc0 0x7f45400a6280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f455000ccb0 tx=0x7f455001c000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.691+0000 7f4559fd4640 1 --2- 172.21.15.84:0/1520070028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f5144e2e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f454c005c20 tx=0x7f454c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3079cf1ed83370fc server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.691+0000 7f453f7fe640 1 -- 172.21.15.84:0/1520070028 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c043020 con 0x559f5144df00 2024-04-26T18:38:04.698 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.691+0000 7f453f7fe640 1 -- 172.21.15.84:0/1520070028 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454c0316a0 con 0x559f5144df00 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.691+0000 7f453f7fe640 1 -- 172.21.15.84:0/1520070028 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c035040 con 0x559f5144df00 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.692+0000 7f455e82b740 1 -- 172.21.15.84:0/1520070028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 msgr2=0x559f5144e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.692+0000 7f455e82b740 1 --2- 172.21.15.84:0/1520070028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f5144e2e0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f454c005c20 tx=0x7f454c005b50 comp rx=0 tx=0).stop 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.692+0000 7f455e82b740 1 -- 172.21.15.84:0/1520070028 shutdown_connections 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.692+0000 7f455e82b740 1 --2- 172.21.15.84:0/1520070028 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f514590e0 0x559f5145b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.692+0000 7f455e82b740 1 --2- 172.21.15.84:0/1520070028 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144e820 0x559f51458ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.692+0000 7f455e82b740 1 --2- 172.21.15.84:0/1520070028 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f5144e2e0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.692+0000 7f455e82b740 1 -- 172.21.15.84:0/1520070028 >> 172.21.15.84:0/1520070028 conn(0x559f5144a410 msgr2=0x559f5144a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.692+0000 7f455e82b740 1 -- 172.21.15.84:0/1520070028 shutdown_connections 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.692+0000 7f455e82b740 1 -- 172.21.15.84:0/1520070028 wait complete. 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.692+0000 7f455e82b740 1 -- start start 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.692+0000 7f455e82b740 1 Processor -- start 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f455e82b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144df00 0x559f514560d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.699 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f455e82b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f5144e820 0x559f51456610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f455e82b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f514590e0 0x559f51456b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f455e82b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x559f5145bcb0 con 0x559f514590e0 2024-04-26T18:38:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f455e82b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x559f5133bb60 con 0x559f5144df00 2024-04-26T18:38:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f455e82b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x559f5145be30 con 0x559f5144e820 2024-04-26T18:38:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f455e82b740 10 client.0 _ll_register_callbacks cb 0x559f513602b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f4559fd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f514590e0 0x559f51456b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f4559fd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f514590e0 0x559f51456b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36672/0 (socket says 172.21.15.84:36672) 2024-04-26T18:38:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f4559fd4640 1 -- 172.21.15.84:0/1916193224 learned_addr learned my addr 172.21.15.84:0/1916193224 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f4558fd2640 1 --2- 172.21.15.84:0/1916193224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f5144e820 0x559f51456610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f4559fd4640 1 -- 172.21.15.84:0/1916193224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f5144e820 msgr2=0x559f51456610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f4559fd4640 1 --2- 172.21.15.84:0/1916193224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f5144e820 0x559f51456610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f4559fd4640 1 -- 172.21.15.84:0/1916193224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144df00 msgr2=0x559f514560d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:04.700 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f4559fd4640 1 --2- 172.21.15.84:0/1916193224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144df00 0x559f514560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f4559fd4640 1 -- 172.21.15.84:0/1916193224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f510c6210 con 0x559f514590e0 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f4559fd4640 1 --2- 172.21.15.84:0/1916193224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f514590e0 0x559f51456b50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f454c033000 tx=0x7f454c031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee84ed80f836bb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f453cff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f453cff9640 1 -- 172.21.15.84:0/1916193224 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c04e020 con 0x559f514590e0 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f453cff9640 1 -- 172.21.15.84:0/1916193224 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f454c031d10 con 0x559f514590e0 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f455e82b740 1 -- 172.21.15.84:0/1916193224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f454c0316a0 con 0x559f514590e0 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.693+0000 7f453cff9640 1 -- 172.21.15.84:0/1916193224 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f454c059050 con 0x559f514590e0 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.694+0000 7f455e82b740 10 client.16611 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.694+0000 7f455e82b740 1 -- 172.21.15.84:0/1916193224 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559f51192620 con 0x559f514590e0 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.694+0000 7f453cff9640 1 -- 172.21.15.84:0/1916193224 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454c0316a0 con 0x559f514590e0 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.694+0000 7f453cff9640 1 -- 172.21.15.84:0/1916193224 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f454c03e820 con 0x559f514590e0 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.694+0000 7f455e82b740 10 client.16611 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.694+0000 7f455e82b740 10 client.16611 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:38:04.701 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.694+0000 7f455e82b740 20 client.16611 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.694+0000 7f455e82b740 1 --2- 172.21.15.84:0/1916193224 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559f514690b0 0x559f5146b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.694+0000 7f455e82b740 4 client.16611 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:38:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.694+0000 7f455e82b740 1 -- 172.21.15.84:0/1916193224 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f513c2070 con 0x559f514690b0 2024-04-26T18:38:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.694+0000 7f4558fd2640 1 --2- 172.21.15.84:0/1916193224 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559f514690b0 0x559f5146b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.695+0000 7f4558fd2640 1 --2- 172.21.15.84:0/1916193224 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559f514690b0 0x559f5146b4a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f4a3c71e8662d7a server_cookie=e73b138b12178680 in_seq=0 out_seq=0 2024-04-26T18:38:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.695+0000 7f453cff9640 10 client.16611 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.695+0000 7f453cff9640 1 -- 172.21.15.84:0/1916193224 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f513c2070 con 0x559f514690b0 2024-04-26T18:38:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.695+0000 7f453cff9640 10 client.16611 handle_command_reply: tid=0 2024-04-26T18:38:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.696+0000 7f455e82b740 1 client.16611 shutdown 2024-04-26T18:38:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.696+0000 7f455e82b740 1 -- 172.21.15.84:0/1916193224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f514590e0 msgr2=0x559f51456b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.696+0000 7f455e82b740 1 --2- 172.21.15.84:0/1916193224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f514590e0 0x559f51456b50 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f454c033000 tx=0x7f454c031850 comp rx=0 tx=0).stop 2024-04-26T18:38:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.696+0000 7f455e82b740 1 -- 172.21.15.84:0/1916193224 shutdown_connections 2024-04-26T18:38:04.702 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.696+0000 7f455e82b740 1 --2- 172.21.15.84:0/1916193224 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559f514690b0 0x559f5146b4a0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.696+0000 7f455e82b740 1 --2- 172.21.15.84:0/1916193224 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f514590e0 0x559f51456b50 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.696+0000 7f453cff9640 0 client.16611 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.696+0000 7f455e82b740 1 --2- 172.21.15.84:0/1916193224 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f5144e820 0x559f51456610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.696+0000 7f455e82b740 1 --2- 172.21.15.84:0/1916193224 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144df00 0x559f514560d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.696+0000 7f455e82b740 1 -- 172.21.15.84:0/1916193224 >> 172.21.15.84:0/1916193224 conn(0x559f5144a410 msgr2=0x559f51450190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.696+0000 7f455e82b740 1 -- 172.21.15.84:0/1916193224 shutdown_connections 2024-04-26T18:38:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.696+0000 7f455e82b740 1 -- 172.21.15.84:0/1916193224 wait complete. 2024-04-26T18:38:04.703 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.696+0000 7f455e82b740 20 client.16611 trim_cache size 0 max 16384 2024-04-26T18:38:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f455e82b740 1 Processor -- start 2024-04-26T18:38:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f455e82b740 1 -- start start 2024-04-26T18:38:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f455e82b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f51464b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f455e82b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f5144e820 0x559f514567a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f455e82b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f514590e0 0x559f51470b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f455e82b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f45541ae430 con 0x559f5144df00 2024-04-26T18:38:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f455e82b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45541aff70 con 0x559f514590e0 2024-04-26T18:38:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f455e82b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f45541aeb00 con 0x559f5144e820 2024-04-26T18:38:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f4559fd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f51464b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.716 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f4559fd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f51464b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36686/0 (socket says 172.21.15.84:36686) 2024-04-26T18:38:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f4559fd4640 1 -- 172.21.15.84:0/2451660778 learned_addr learned my addr 172.21.15.84:0/2451660778 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f4558fd2640 1 --2- 172.21.15.84:0/2451660778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f5144e820 0x559f514567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f4559fd4640 1 --2- 172.21.15.84:0/2451660778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f514590e0 0x559f51470b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f4559fd4640 1 -- 172.21.15.84:0/2451660778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f5144e820 msgr2=0x559f514567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f4559fd4640 1 --2- 172.21.15.84:0/2451660778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f5144e820 0x559f514567a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f4559fd4640 1 -- 172.21.15.84:0/2451660778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f514590e0 msgr2=0x559f51470b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f4559fd4640 1 --2- 172.21.15.84:0/2451660778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f514590e0 0x559f51470b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f4559fd4640 1 -- 172.21.15.84:0/2451660778 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f51445d60 con 0x559f5144df00 2024-04-26T18:38:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.711+0000 7f4558fd2640 1 --2- 172.21.15.84:0/2451660778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f5144e820 0x559f514567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:38:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.712+0000 7f4559fd4640 1 --2- 172.21.15.84:0/2451660778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f51464b70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f454c0059d0 tx=0x7f454c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91ab21422d7ec40a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.712+0000 7f453effd640 1 -- 172.21.15.84:0/2451660778 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c04e040 con 0x559f5144df00 2024-04-26T18:38:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.712+0000 7f453effd640 1 -- 172.21.15.84:0/2451660778 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f454c032d40 con 0x559f5144df00 2024-04-26T18:38:04.717 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.712+0000 7f453effd640 1 -- 172.21.15.84:0/2451660778 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c043030 con 0x559f5144df00 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.712+0000 7f455e82b740 1 -- 172.21.15.84:0/2451660778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 msgr2=0x559f51464b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.712+0000 7f455e82b740 1 --2- 172.21.15.84:0/2451660778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f51464b70 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f454c0059d0 tx=0x7f454c001e50 comp rx=0 tx=0).stop 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.712+0000 7f455e82b740 1 -- 172.21.15.84:0/2451660778 shutdown_connections 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.712+0000 7f455e82b740 1 --2- 172.21.15.84:0/2451660778 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f514590e0 0x559f51470b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.712+0000 7f455e82b740 1 --2- 172.21.15.84:0/2451660778 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f5144e820 0x559f514567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.712+0000 7f455e82b740 1 --2- 172.21.15.84:0/2451660778 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f51464b70 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.712+0000 7f455e82b740 1 -- 172.21.15.84:0/2451660778 >> 172.21.15.84:0/2451660778 conn(0x559f5144a410 msgr2=0x559f51456090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.712+0000 7f455e82b740 1 -- 172.21.15.84:0/2451660778 shutdown_connections 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.712+0000 7f455e82b740 1 -- 172.21.15.84:0/2451660778 wait complete. 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.713+0000 7f455e82b740 1 -- start start 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.713+0000 7f455e82b740 1 Processor -- start 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.713+0000 7f455e82b740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f51461370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.713+0000 7f455e82b740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144e820 0x559f5145e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.718 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.713+0000 7f455e82b740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f514590e0 0x559f5145e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.713+0000 7f455e82b740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f45541af8a0 con 0x559f5144df00 2024-04-26T18:38:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.713+0000 7f455e82b740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f45541af1d0 con 0x559f5144e820 2024-04-26T18:38:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.713+0000 7f455e82b740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f45541ac430 con 0x559f514590e0 2024-04-26T18:38:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.713+0000 7f455e82b740 10 client.0 _ll_register_callbacks cb 0x7f4544005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.713+0000 7f4559fd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f51461370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.713+0000 7f4559fd4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f51461370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:36700/0 (socket says 172.21.15.84:36700) 2024-04-26T18:38:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.713+0000 7f4559fd4640 1 -- 172.21.15.84:0/3258940894 learned_addr learned my addr 172.21.15.84:0/3258940894 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.713+0000 7f4558fd2640 1 --2- 172.21.15.84:0/3258940894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144e820 0x559f5145e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f4559fd4640 1 -- 172.21.15.84:0/3258940894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f514590e0 msgr2=0x559f5145e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f4559fd4640 1 --2- 172.21.15.84:0/3258940894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f514590e0 0x559f5145e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f4559fd4640 1 -- 172.21.15.84:0/3258940894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144e820 msgr2=0x559f5145e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f4559fd4640 1 --2- 172.21.15.84:0/3258940894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144e820 0x559f5145e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f4559fd4640 1 -- 172.21.15.84:0/3258940894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f5145ecd0 con 0x559f5144df00 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f4559fd4640 1 --2- 172.21.15.84:0/3258940894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f51461370 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f454c005ec0 tx=0x7f454c03efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c98d698d9fe74aa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f453cff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f453cff9640 1 -- 172.21.15.84:0/3258940894 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f454c043070 con 0x559f5144df00 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f455e82b740 1 -- 172.21.15.84:0/3258940894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559f513bbee0 con 0x559f5144df00 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f453cff9640 1 -- 172.21.15.84:0/3258940894 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 8579+0+0 (secure 0 0 0) 0x7f454c005050 con 0x559f5144df00 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f453cff9640 1 -- 172.21.15.84:0/3258940894 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f454c06d050 con 0x559f5144df00 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f455e82b740 10 client.16617 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f455e82b740 1 -- 172.21.15.84:0/3258940894 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559f514d0300 con 0x559f5144df00 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f453cff9640 1 -- 172.21.15.84:0/3258940894 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559f513bbee0 con 0x559f5144df00 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f453cff9640 1 -- 172.21.15.84:0/3258940894 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f454c048720 con 0x559f5144df00 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f455e82b740 10 client.16617 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f455e82b740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f455e82b740 20 client.16617 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f455e82b740 1 --2- 172.21.15.84:0/3258940894 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559f514d0940 0x559f514d2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f455e82b740 4 client.16617 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:38:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.714+0000 7f455e82b740 1 -- 172.21.15.84:0/3258940894 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559f513375f0 con 0x559f514d0940 2024-04-26T18:38:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.715+0000 7f4558fd2640 1 --2- 172.21.15.84:0/3258940894 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559f514d0940 0x559f514d2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.715+0000 7f4558fd2640 1 --2- 172.21.15.84:0/3258940894 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559f514d0940 0x559f514d2d60 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd9c34645df9f5f2 server_cookie=4d3c48dd2cb77cb4 in_seq=0 out_seq=0 2024-04-26T18:38:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.715+0000 7f453cff9640 10 client.16617 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.730+0000 7f453cff9640 1 -- 172.21.15.84:0/3258940894 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559f513375f0 con 0x559f514d0940 2024-04-26T18:38:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.730+0000 7f453cff9640 10 client.16617 handle_command_reply: tid=0 2024-04-26T18:38:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.730+0000 7f455e82b740 1 client.16617 shutdown 2024-04-26T18:38:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.730+0000 7f455e82b740 1 -- 172.21.15.84:0/3258940894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 msgr2=0x559f51461370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.730+0000 7f455e82b740 1 --2- 172.21.15.84:0/3258940894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f51461370 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f454c005ec0 tx=0x7f454c03efa0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.730+0000 7f455e82b740 1 -- 172.21.15.84:0/3258940894 shutdown_connections 2024-04-26T18:38:04.731 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.730+0000 7f455e82b740 1 --2- 172.21.15.84:0/3258940894 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x559f514d0940 0x559f514d2d60 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.730+0000 7f455e82b740 1 --2- 172.21.15.84:0/3258940894 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x559f514590e0 0x559f5145e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.730+0000 7f455e82b740 1 --2- 172.21.15.84:0/3258940894 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x559f5144e820 0x559f5145e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.730+0000 7f453cff9640 0 client.16617 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.730+0000 7f455e82b740 1 --2- 172.21.15.84:0/3258940894 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x559f5144df00 0x559f51461370 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.730+0000 7f455e82b740 1 -- 172.21.15.84:0/3258940894 >> 172.21.15.84:0/3258940894 conn(0x559f5144a410 msgr2=0x559f51450140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455e82b740 1 -- 172.21.15.84:0/3258940894 shutdown_connections 2024-04-26T18:38:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455e82b740 1 -- 172.21.15.84:0/3258940894 wait complete. 2024-04-26T18:38:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455e82b740 20 client.16617 trim_cache size 0 max 16384 2024-04-26T18:38:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:38:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:38:04.732 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:38:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:38:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455ba37640 1 -- 172.21.15.84:0/118074098 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f45400a3dc0 msgr2=0x7f45400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455ba37640 1 --2- 172.21.15.84:0/118074098 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f45400a3dc0 0x7f45400a6280 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f455000ccb0 tx=0x7f455001c000 comp rx=0 tx=0).stop 2024-04-26T18:38:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455ba37640 1 -- 172.21.15.84:0/118074098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45541010a0 msgr2=0x7f4554198fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:04.733 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455ba37640 1 --2- 172.21.15.84:0/118074098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45541010a0 0x7f4554198fa0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f455000cce0 tx=0x7f455000a620 comp rx=0 tx=0).stop 2024-04-26T18:38:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455ba37640 1 -- 172.21.15.84:0/118074098 shutdown_connections 2024-04-26T18:38:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455ba37640 1 --2- 172.21.15.84:0/118074098 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f45400a3dc0 0x7f45400a6280 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455ba37640 1 --2- 172.21.15.84:0/118074098 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f455419d860 0x7f455419dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455ba37640 1 --2- 172.21.15.84:0/118074098 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4554107c60 0x7f45541994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455ba37640 1 --2- 172.21.15.84:0/118074098 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f45541010a0 0x7f4554198fa0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455ba37640 1 -- 172.21.15.84:0/118074098 >> 172.21.15.84:0/118074098 conn(0x7f4554077dc0 msgr2=0x7f45541063f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455ba37640 1 -- 172.21.15.84:0/118074098 shutdown_connections 2024-04-26T18:38:04.734 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:04.731+0000 7f455ba37640 1 -- 172.21.15.84:0/118074098 wait complete. 2024-04-26T18:38:04.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:04 smithi084 ceph-mon[29141]: pgmap v314: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-04-26T18:38:04.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:04 smithi103 ceph-mon[32727]: pgmap v314: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-04-26T18:38:04.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:04 smithi174 ceph-mon[32705]: pgmap v314: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-04-26T18:38:06.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:06 smithi174 ceph-mon[32705]: pgmap v315: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 25 op/s 2024-04-26T18:38:07.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:06 smithi084 ceph-mon[29141]: pgmap v315: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 25 op/s 2024-04-26T18:38:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:06 smithi103 ceph-mon[32727]: pgmap v315: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 25 op/s 2024-04-26T18:38:09.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:09 smithi174 ceph-mon[32705]: pgmap v316: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 25 op/s 2024-04-26T18:38:09.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:09 smithi084 ceph-mon[29141]: pgmap v316: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 25 op/s 2024-04-26T18:38:09.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:09 smithi103 ceph-mon[32727]: pgmap v316: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 48 KiB/s wr, 25 op/s 2024-04-26T18:38:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:10 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:38:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:10 smithi174 ceph-mon[32705]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T18:38:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:10 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:10 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:10 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:10 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:10 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.690 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:10 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:10 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:38:10.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:10 smithi084 ceph-mon[29141]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T18:38:10.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:10 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:10 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:10 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:10 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:10 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.776 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:10 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.778 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:10 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:38:10.778 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:10 smithi103 ceph-mon[32727]: osdmap e103: 12 total, 12 up, 12 in 2024-04-26T18:38:10.778 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:10 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.778 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:10 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.779 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:10 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.779 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:10 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.779 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:10 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:10.779 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:10 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:11.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:11 smithi174 ceph-mon[32705]: pgmap v318: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-04-26T18:38:11.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:11 smithi084 ceph-mon[29141]: pgmap v318: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-04-26T18:38:11.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:11 smithi103 ceph-mon[32727]: pgmap v318: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-04-26T18:38:12.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: pgmap v319: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-04-26T18:38:12.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:12.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:12.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:12.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi174 to 176.2M 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: Unable to set osd_memory_target on smithi174 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:12.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:12.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:12.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:12.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-04-26T18:38:12.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: Adjusting osd_memory_target on smithi103 to 176.2M 2024-04-26T18:38:12.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: Unable to set osd_memory_target on smithi084 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-26T18:38:12.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: Unable to set osd_memory_target on smithi103 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:38:12.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:38:12.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:38:12.941 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:12 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: pgmap v319: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-04-26T18:38:13.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi174 to 176.2M 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: Unable to set osd_memory_target on smithi174 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-04-26T18:38:13.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: Adjusting osd_memory_target on smithi103 to 176.2M 2024-04-26T18:38:13.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: Unable to set osd_memory_target on smithi084 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-26T18:38:13.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: Unable to set osd_memory_target on smithi103 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:38:13.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:38:13.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:38:13.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:12 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: pgmap v319: 97 pgs: 97 active+clean; 4.5 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 65 KiB/s wr, 34 op/s 2024-04-26T18:38:13.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi174 to 176.2M 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: Unable to set osd_memory_target on smithi174 to 184770252: error parsing value: Value '184770252' is below minimum 939524096 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.128 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.129 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.129 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.129 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-26T18:38:13.129 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi084 to 176.2M 2024-04-26T18:38:13.129 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: Adjusting osd_memory_target on smithi103 to 176.2M 2024-04-26T18:38:13.129 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: Unable to set osd_memory_target on smithi084 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-26T18:38:13.129 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: Unable to set osd_memory_target on smithi103 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-26T18:38:13.129 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:38:13.129 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:38:13.129 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:12 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:38:13.640 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:38:13.894 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:38:14.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:14 smithi174 ceph-mon[32705]: pgmap v320: 97 pgs: 97 active+clean; 4.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.6 KiB/s wr, 13 op/s 2024-04-26T18:38:15.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:14 smithi084 ceph-mon[29141]: pgmap v320: 97 pgs: 97 active+clean; 4.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.6 KiB/s wr, 13 op/s 2024-04-26T18:38:15.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:14 smithi103 ceph-mon[32727]: pgmap v320: 97 pgs: 97 active+clean; 4.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.6 KiB/s wr, 13 op/s 2024-04-26T18:38:15.706 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.706+0000 7fc468a6c700 1 -- 172.21.15.84:0/2735443527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc46410b650 msgr2=0x7fc46410ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:15.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.706+0000 7fc468a6c700 1 --2- 172.21.15.84:0/2735443527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc46410b650 0x7fc46410ba30 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fc45c009990 tx=0x7fc45c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:38:15.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.706+0000 7fc468a6c700 1 -- 172.21.15.84:0/2735443527 shutdown_connections 2024-04-26T18:38:15.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.706+0000 7fc468a6c700 1 --2- 172.21.15.84:0/2735443527 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc46410b650 0x7fc46410ba30 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:15.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.706+0000 7fc468a6c700 1 --2- 172.21.15.84:0/2735443527 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc464106010 0x7fc464112520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:15.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.706+0000 7fc468a6c700 1 --2- 172.21.15.84:0/2735443527 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc464105650 0x7fc464105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:15.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.706+0000 7fc468a6c700 1 -- 172.21.15.84:0/2735443527 >> 172.21.15.84:0/2735443527 conn(0x7fc464101140 msgr2=0x7fc464103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:15.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.706+0000 7fc468a6c700 1 -- 172.21.15.84:0/2735443527 shutdown_connections 2024-04-26T18:38:15.707 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.706+0000 7fc468a6c700 1 -- 172.21.15.84:0/2735443527 wait complete. 2024-04-26T18:38:15.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.707+0000 7fc468a6c700 1 Processor -- start 2024-04-26T18:38:15.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.707+0000 7fc468a6c700 1 -- start start 2024-04-26T18:38:15.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.707+0000 7fc468a6c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc464105650 0x7fc4641a18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:15.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.707+0000 7fc468a6c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc464106010 0x7fc4641a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:15.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.707+0000 7fc468a6c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc46410b650 0x7fc46419ba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:15.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.707+0000 7fc468a6c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc464115880 con 0x7fc464105650 2024-04-26T18:38:15.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.707+0000 7fc468a6c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc464115700 con 0x7fc46410b650 2024-04-26T18:38:15.709 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.707+0000 7fc468a6c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc464115a00 con 0x7fc464106010 2024-04-26T18:38:15.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.707+0000 7fc46259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc464105650 0x7fc4641a18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:15.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.707+0000 7fc46259c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc464105650 0x7fc4641a18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44992/0 (socket says 172.21.15.84:44992) 2024-04-26T18:38:15.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.707+0000 7fc46259c700 1 -- 172.21.15.84:0/592524708 learned_addr learned my addr 172.21.15.84:0/592524708 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:15.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.707+0000 7fc462d9d700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc46410b650 0x7fc46419ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:15.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.708+0000 7fc461d9b700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc464106010 0x7fc4641a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:15.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.708+0000 7fc46259c700 1 -- 172.21.15.84:0/592524708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc464106010 msgr2=0x7fc4641a1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:15.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.708+0000 7fc46259c700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc464106010 0x7fc4641a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:15.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.708+0000 7fc46259c700 1 -- 172.21.15.84:0/592524708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc46410b650 msgr2=0x7fc46419ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:15.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.708+0000 7fc46259c700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc46410b650 0x7fc46419ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:15.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.708+0000 7fc46259c700 1 -- 172.21.15.84:0/592524708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc46419c1e0 con 0x7fc464105650 2024-04-26T18:38:15.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.708+0000 7fc462d9d700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc46410b650 0x7fc46419ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:38:15.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.708+0000 7fc46259c700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc464105650 0x7fc4641a18c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc45c000c00 tx=0x7fc45c005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:15.710 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.708+0000 7fc45b7fe700 1 -- 172.21.15.84:0/592524708 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c027070 con 0x7fc464105650 2024-04-26T18:38:15.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.708+0000 7fc45b7fe700 1 -- 172.21.15.84:0/592524708 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc45c003f20 con 0x7fc464105650 2024-04-26T18:38:15.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.708+0000 7fc468a6c700 1 -- 172.21.15.84:0/592524708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc46419c470 con 0x7fc464105650 2024-04-26T18:38:15.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.708+0000 7fc45b7fe700 1 -- 172.21.15.84:0/592524708 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc45c022c70 con 0x7fc464105650 2024-04-26T18:38:15.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.708+0000 7fc468a6c700 1 -- 172.21.15.84:0/592524708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4641a8790 con 0x7fc464105650 2024-04-26T18:38:15.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.709+0000 7fc468a6c700 1 -- 172.21.15.84:0/592524708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc464106d90 con 0x7fc464105650 2024-04-26T18:38:15.711 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.710+0000 7fc45b7fe700 1 -- 172.21.15.84:0/592524708 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc45c02a070 con 0x7fc464105650 2024-04-26T18:38:15.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.711+0000 7fc45b7fe700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc4500a3db0 0x7fc4500a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:15.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.711+0000 7fc461d9b700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc4500a3db0 0x7fc4500a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:15.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.711+0000 7fc45b7fe700 1 -- 172.21.15.84:0/592524708 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 8623+0+0 (secure 0 0 0) 0x7fc45c0d1740 con 0x7fc464105650 2024-04-26T18:38:15.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.711+0000 7fc461d9b700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc4500a3db0 0x7fc4500a6270 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc46410ab80 tx=0x7fc44c005d50 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:15.715 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.715+0000 7fc45b7fe700 1 -- 172.21.15.84:0/592524708 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc45c05eff0 con 0x7fc464105650 2024-04-26T18:38:15.957 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.957+0000 7fc468a6c700 1 -- 172.21.15.84:0/592524708 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc46402e490 con 0x7fc464105650 2024-04-26T18:38:15.958 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.958+0000 7fc45b7fe700 1 -- 172.21.15.84:0/592524708 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7fc45c013070 con 0x7fc464105650 2024-04-26T18:38:15.958 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:38:15.958 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:38:15.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.960+0000 7fc468a6c700 1 -- 172.21.15.84:0/592524708 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc4500a3db0 msgr2=0x7fc4500a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:15.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.960+0000 7fc468a6c700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc4500a3db0 0x7fc4500a6270 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc46410ab80 tx=0x7fc44c005d50 comp rx=0 tx=0).stop 2024-04-26T18:38:15.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.960+0000 7fc468a6c700 1 -- 172.21.15.84:0/592524708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc464105650 msgr2=0x7fc4641a18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:15.960 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.960+0000 7fc468a6c700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc464105650 0x7fc4641a18c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fc45c000c00 tx=0x7fc45c005ef0 comp rx=0 tx=0).stop 2024-04-26T18:38:15.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.960+0000 7fc468a6c700 1 -- 172.21.15.84:0/592524708 shutdown_connections 2024-04-26T18:38:15.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.960+0000 7fc468a6c700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc4500a3db0 0x7fc4500a6270 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:15.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.960+0000 7fc468a6c700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc464105650 0x7fc4641a18c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:15.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.960+0000 7fc468a6c700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc46410b650 0x7fc46419ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:15.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.960+0000 7fc468a6c700 1 --2- 172.21.15.84:0/592524708 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc464106010 0x7fc4641a1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:15.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.960+0000 7fc468a6c700 1 -- 172.21.15.84:0/592524708 >> 172.21.15.84:0/592524708 conn(0x7fc464101140 msgr2=0x7fc464101660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:15.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.960+0000 7fc468a6c700 1 -- 172.21.15.84:0/592524708 shutdown_connections 2024-04-26T18:38:15.961 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:15.960+0000 7fc468a6c700 1 -- 172.21.15.84:0/592524708 wait complete. 2024-04-26T18:38:15.961 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:38:16.666 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id 400c61dd --release --await' 2024-04-26T18:38:16.667 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db --set-id 400c61dd --release --await 2024-04-26T18:38:16.907 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:38:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:16 smithi174 ceph-mon[32705]: pgmap v321: 97 pgs: 97 active+clean; 4.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.6 KiB/s wr, 13 op/s 2024-04-26T18:38:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:16 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/592524708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:38:17.030 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:16 smithi084 ceph-mon[29141]: pgmap v321: 97 pgs: 97 active+clean; 4.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.6 KiB/s wr, 13 op/s 2024-04-26T18:38:17.030 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:16 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/592524708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:38:17.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:16 smithi103 ceph-mon[32727]: pgmap v321: 97 pgs: 97 active+clean; 4.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 7.6 KiB/s wr, 13 op/s 2024-04-26T18:38:17.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:16 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/592524708' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:38:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.820+0000 7f571acf8640 1 -- 172.21.15.84:0/2720987719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5714101210 msgr2=0x7f57141015f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:17.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.820+0000 7f571acf8640 1 --2- 172.21.15.84:0/2720987719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5714101210 0x7f57141015f0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5710009900 tx=0x7f571002b620 comp rx=0 tx=0).stop 2024-04-26T18:38:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.820+0000 7f571acf8640 1 -- 172.21.15.84:0/2720987719 shutdown_connections 2024-04-26T18:38:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.820+0000 7f571acf8640 1 --2- 172.21.15.84:0/2720987719 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f57141062e0 0x7f571410deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.820+0000 7f571acf8640 1 --2- 172.21.15.84:0/2720987719 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5714101bc0 0x7f5714105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.820+0000 7f571acf8640 1 --2- 172.21.15.84:0/2720987719 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5714101210 0x7f57141015f0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.820+0000 7f571acf8640 1 -- 172.21.15.84:0/2720987719 >> 172.21.15.84:0/2720987719 conn(0x7f5714077d50 msgr2=0x7f57140ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.820+0000 7f571acf8640 1 -- 172.21.15.84:0/2720987719 shutdown_connections 2024-04-26T18:38:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.820+0000 7f571acf8640 1 -- 172.21.15.84:0/2720987719 wait complete. 2024-04-26T18:38:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f571acf8640 1 Processor -- start 2024-04-26T18:38:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f571acf8640 1 -- start start 2024-04-26T18:38:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f571acf8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5714101210 0x7f5714199270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f571acf8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57141062e0 0x7f57141997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f571acf8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f571419dae0 0x7f571419df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f571acf8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5714110aa0 con 0x7f5714101210 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f571acf8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5714110920 con 0x7f571419dae0 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f571acf8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5714110c20 con 0x7f57141062e0 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f5718a94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5714101210 0x7f5714199270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f5718a94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5714101210 0x7f5714199270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34228/0 (socket says 172.21.15.84:34228) 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f5718a94640 1 -- 172.21.15.84:0/1749095974 learned_addr learned my addr 172.21.15.84:0/1749095974 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f5718a94640 1 -- 172.21.15.84:0/1749095974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57141062e0 msgr2=0x7f57141997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f5718a94640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57141062e0 0x7f57141997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f5718a94640 1 -- 172.21.15.84:0/1749095974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f571419dae0 msgr2=0x7f571419df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f5719295640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f571419dae0 0x7f571419df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f5718a94640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f571419dae0 0x7f571419df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f5718a94640 1 -- 172.21.15.84:0/1749095974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f571419e5d0 con 0x7f5714101210 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.821+0000 7f5719295640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f571419dae0 0x7f571419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.822+0000 7f5718a94640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5714101210 0x7f5714199270 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5710006010 tx=0x7f57100315d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:17.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.822+0000 7f5709ffb640 1 -- 172.21.15.84:0/1749095974 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f571003f070 con 0x7f5714101210 2024-04-26T18:38:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.822+0000 7f5709ffb640 1 -- 172.21.15.84:0/1749095974 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5710031bd0 con 0x7f5714101210 2024-04-26T18:38:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.822+0000 7f571acf8640 1 -- 172.21.15.84:0/1749095974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f571419e860 con 0x7f5714101210 2024-04-26T18:38:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.822+0000 7f5709ffb640 1 -- 172.21.15.84:0/1749095974 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710036030 con 0x7f5714101210 2024-04-26T18:38:17.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.822+0000 7f571acf8640 1 -- 172.21.15.84:0/1749095974 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57141a6080 con 0x7f5714101210 2024-04-26T18:38:17.831 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.824+0000 7f5709ffb640 1 -- 172.21.15.84:0/1749095974 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5710042070 con 0x7f5714101210 2024-04-26T18:38:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.824+0000 7f571da0e740 1 Processor -- start 2024-04-26T18:38:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f571da0e740 1 -- start start 2024-04-26T18:38:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5709ffb640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f56fc0a3e40 0x7f56fc0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f571da0e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 0x56282b0de2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f571da0e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0de820 0x56282b0e8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f571da0e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0e90e0 0x56282b0eb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f571da0e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56282b044460 con 0x56282b0ddf00 2024-04-26T18:38:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f571da0e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56282affe590 con 0x56282b0de820 2024-04-26T18:38:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f571da0e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56282b040420 con 0x56282b0e90e0 2024-04-26T18:38:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5719295640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f56fc0a3e40 0x7f56fc0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5709ffb640 1 -- 172.21.15.84:0/1749095974 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f57100f1110 con 0x7f5714101210 2024-04-26T18:38:17.832 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5718a94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 0x56282b0de2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5718a94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 0x56282b0de2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34234/0 (socket says 172.21.15.84:34234) 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5718a94640 1 -- 172.21.15.84:0/1896034066 learned_addr learned my addr 172.21.15.84:0/1896034066 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f570bfff640 1 --2- 172.21.15.84:0/1896034066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0de820 0x56282b0e8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5719295640 1 --2- 172.21.15.84:0/1896034066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0e90e0 0x56282b0eb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5718a94640 1 -- 172.21.15.84:0/1896034066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0e90e0 msgr2=0x56282b0eb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5718a94640 1 --2- 172.21.15.84:0/1896034066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0e90e0 0x56282b0eb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5718a94640 1 -- 172.21.15.84:0/1896034066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0de820 msgr2=0x56282b0e8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5718a94640 1 --2- 172.21.15.84:0/1896034066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0de820 0x56282b0e8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5718a94640 1 -- 172.21.15.84:0/1896034066 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56282b00b0d0 con 0x56282b0ddf00 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5719295640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f56fc0a3e40 0x7f56fc0a6300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f570c0103b0 tx=0x7f570c010870 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.825+0000 7f5718a94640 1 --2- 172.21.15.84:0/1896034066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 0x56282b0de2e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f57100b9a40 tx=0x7f57100bc9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1a5affba321f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.826+0000 7f56faffd640 1 -- 172.21.15.84:0/1896034066 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710049070 con 0x56282b0ddf00 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.826+0000 7f56faffd640 1 -- 172.21.15.84:0/1896034066 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57100bd070 con 0x56282b0ddf00 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.826+0000 7f56faffd640 1 -- 172.21.15.84:0/1896034066 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57100b96e0 con 0x56282b0ddf00 2024-04-26T18:38:17.833 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.826+0000 7f571da0e740 1 -- 172.21.15.84:0/1896034066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 msgr2=0x56282b0de2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.826+0000 7f571da0e740 1 --2- 172.21.15.84:0/1896034066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 0x56282b0de2e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f57100b9a40 tx=0x7f57100bc9d0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.826+0000 7f571da0e740 1 -- 172.21.15.84:0/1896034066 shutdown_connections 2024-04-26T18:38:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.826+0000 7f571da0e740 1 --2- 172.21.15.84:0/1896034066 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0e90e0 0x56282b0eb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.826+0000 7f571da0e740 1 --2- 172.21.15.84:0/1896034066 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0de820 0x56282b0e8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.826+0000 7f571da0e740 1 --2- 172.21.15.84:0/1896034066 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 0x56282b0de2e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.826+0000 7f571da0e740 1 -- 172.21.15.84:0/1896034066 >> 172.21.15.84:0/1896034066 conn(0x56282b0da410 msgr2=0x56282b0da7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.826+0000 7f571da0e740 1 -- 172.21.15.84:0/1896034066 shutdown_connections 2024-04-26T18:38:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.826+0000 7f571da0e740 1 -- 172.21.15.84:0/1896034066 wait complete. 2024-04-26T18:38:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f571da0e740 1 -- start start 2024-04-26T18:38:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f571da0e740 1 Processor -- start 2024-04-26T18:38:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f571da0e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0ddf00 0x56282b0e6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f571da0e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0de820 0x56282b0e65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.834 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f571da0e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0e90e0 0x56282b0e6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f571da0e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56282b0ebad0 con 0x56282b0de820 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f571da0e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56282afcbb60 con 0x56282b0ddf00 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f571da0e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56282b0ebc50 con 0x56282b0e90e0 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f571da0e740 10 client.0 _ll_register_callbacks cb 0x56282aff02b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f570bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0de820 0x56282b0e65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f570bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0de820 0x56282b0e65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34238/0 (socket says 172.21.15.84:34238) 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f570bfff640 1 -- 172.21.15.84:0/2288944689 learned_addr learned my addr 172.21.15.84:0/2288944689 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f5718a94640 1 --2- 172.21.15.84:0/2288944689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0ddf00 0x56282b0e6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f570bfff640 1 -- 172.21.15.84:0/2288944689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0e90e0 msgr2=0x56282b0e6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f570bfff640 1 --2- 172.21.15.84:0/2288944689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0e90e0 0x56282b0e6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f570bfff640 1 -- 172.21.15.84:0/2288944689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0ddf00 msgr2=0x56282b0e6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f570bfff640 1 --2- 172.21.15.84:0/2288944689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0ddf00 0x56282b0e6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.827+0000 7f570bfff640 1 -- 172.21.15.84:0/2288944689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56282ad56210 con 0x56282b0de820 2024-04-26T18:38:17.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f570bfff640 1 --2- 172.21.15.84:0/2288944689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0de820 0x56282b0e65c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f570400d190 tx=0x7f570400d650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15f93bb46398f7de server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f56f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f56f3fff640 1 -- 172.21.15.84:0/2288944689 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570400de80 con 0x56282b0de820 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f56f3fff640 1 -- 172.21.15.84:0/2288944689 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f57040125f0 con 0x56282b0de820 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f571da0e740 1 -- 172.21.15.84:0/2288944689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57100bd070 con 0x56282b0de820 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f56f3fff640 1 -- 172.21.15.84:0/2288944689 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5704022050 con 0x56282b0de820 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f571da0e740 10 client.16635 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f571da0e740 1 -- 172.21.15.84:0/2288944689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56282ae22620 con 0x56282b0de820 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f56f3fff640 1 -- 172.21.15.84:0/2288944689 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57100bd070 con 0x56282b0de820 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f56f3fff640 1 -- 172.21.15.84:0/2288944689 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5704013070 con 0x56282b0de820 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f571da0e740 10 client.16635 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f571da0e740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f571da0e740 20 client.16635 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f571da0e740 1 --2- 172.21.15.84:0/2288944689 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56282b0f90b0 0x56282b0fb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f571da0e740 4 client.16635 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.828+0000 7f571da0e740 1 -- 172.21.15.84:0/2288944689 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56282b052070 con 0x56282b0f90b0 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.829+0000 7f5718a94640 1 --2- 172.21.15.84:0/2288944689 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56282b0f90b0 0x56282b0fb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.829+0000 7f5718a94640 1 --2- 172.21.15.84:0/2288944689 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56282b0f90b0 0x56282b0fb520 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a0882e721730c3f server_cookie=2ad4a429b1795fba in_seq=0 out_seq=0 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.829+0000 7f56f3fff640 10 client.16635 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f56f3fff640 1 -- 172.21.15.84:0/2288944689 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56282b052070 con 0x56282b0f90b0 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f56f3fff640 10 client.16635 handle_command_reply: tid=0 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f571da0e740 1 client.16635 shutdown 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f571da0e740 1 -- 172.21.15.84:0/2288944689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0de820 msgr2=0x56282b0e65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f571da0e740 1 --2- 172.21.15.84:0/2288944689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0de820 0x56282b0e65c0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f570400d190 tx=0x7f570400d650 comp rx=0 tx=0).stop 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f571da0e740 1 -- 172.21.15.84:0/2288944689 shutdown_connections 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f571da0e740 1 --2- 172.21.15.84:0/2288944689 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56282b0f90b0 0x56282b0fb520 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f571da0e740 1 --2- 172.21.15.84:0/2288944689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0e90e0 0x56282b0e6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f571da0e740 1 --2- 172.21.15.84:0/2288944689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0de820 0x56282b0e65c0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f56f3fff640 0 client.16635 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f571da0e740 1 --2- 172.21.15.84:0/2288944689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0ddf00 0x56282b0e6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f571da0e740 1 -- 172.21.15.84:0/2288944689 >> 172.21.15.84:0/2288944689 conn(0x56282b0da410 msgr2=0x56282b0e0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f571da0e740 1 -- 172.21.15.84:0/2288944689 shutdown_connections 2024-04-26T18:38:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f571da0e740 1 -- 172.21.15.84:0/2288944689 wait complete. 2024-04-26T18:38:17.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.830+0000 7f571da0e740 20 client.16635 trim_cache size 0 max 16384 2024-04-26T18:38:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.844+0000 7f571da0e740 1 Processor -- start 2024-04-26T18:38:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.844+0000 7f571da0e740 1 -- start start 2024-04-26T18:38:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f571da0e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 0x56282b0f4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.849 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f571da0e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0de820 0x56282b100e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f571da0e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0e90e0 0x56282b101370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f571da0e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f57141ae760 con 0x56282b0ddf00 2024-04-26T18:38:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f571da0e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57141b02a0 con 0x56282b0e90e0 2024-04-26T18:38:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f571da0e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57141aee30 con 0x56282b0de820 2024-04-26T18:38:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f5718a94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 0x56282b0f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f5718a94640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 0x56282b0f4870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34240/0 (socket says 172.21.15.84:34240) 2024-04-26T18:38:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f5718a94640 1 -- 172.21.15.84:0/754160646 learned_addr learned my addr 172.21.15.84:0/754160646 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f570bfff640 1 --2- 172.21.15.84:0/754160646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0de820 0x56282b100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f5718a94640 1 -- 172.21.15.84:0/754160646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0de820 msgr2=0x56282b100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f5718a94640 1 --2- 172.21.15.84:0/754160646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0de820 0x56282b100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f5718a94640 1 -- 172.21.15.84:0/754160646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0e90e0 msgr2=0x56282b101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:17.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f5718a94640 1 --2- 172.21.15.84:0/754160646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0e90e0 0x56282b101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f5718a94640 1 -- 172.21.15.84:0/754160646 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56282b0eb840 con 0x56282b0ddf00 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f5718a94640 1 --2- 172.21.15.84:0/754160646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 0x56282b0f4870 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f57100b94d0 tx=0x7f5710005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d4b487c3ddb6ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f56fa7fc640 1 -- 172.21.15.84:0/754160646 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57100d5070 con 0x56282b0ddf00 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f56fa7fc640 1 -- 172.21.15.84:0/754160646 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57100b7070 con 0x56282b0ddf00 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.845+0000 7f56fa7fc640 1 -- 172.21.15.84:0/754160646 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5710049040 con 0x56282b0ddf00 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.846+0000 7f571da0e740 1 -- 172.21.15.84:0/754160646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 msgr2=0x56282b0f4870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.846+0000 7f571da0e740 1 --2- 172.21.15.84:0/754160646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 0x56282b0f4870 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f57100b94d0 tx=0x7f5710005d50 comp rx=0 tx=0).stop 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.846+0000 7f571da0e740 1 -- 172.21.15.84:0/754160646 shutdown_connections 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.846+0000 7f571da0e740 1 --2- 172.21.15.84:0/754160646 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0e90e0 0x56282b101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.846+0000 7f571da0e740 1 --2- 172.21.15.84:0/754160646 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0de820 0x56282b100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.846+0000 7f571da0e740 1 --2- 172.21.15.84:0/754160646 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0ddf00 0x56282b0f4870 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.846+0000 7f571da0e740 1 -- 172.21.15.84:0/754160646 >> 172.21.15.84:0/754160646 conn(0x56282b0da410 msgr2=0x56282b0df8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.846+0000 7f571da0e740 1 -- 172.21.15.84:0/754160646 shutdown_connections 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.846+0000 7f571da0e740 1 -- 172.21.15.84:0/754160646 wait complete. 2024-04-26T18:38:17.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.846+0000 7f571da0e740 1 -- start start 2024-04-26T18:38:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.846+0000 7f571da0e740 1 Processor -- start 2024-04-26T18:38:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f571da0e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0ddf00 0x56282b0f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f571da0e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0de820 0x56282b0f12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f571da0e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0e90e0 0x56282b0edd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f571da0e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f57141afbd0 con 0x56282b0e90e0 2024-04-26T18:38:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f571da0e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f57141af500 con 0x56282b0de820 2024-04-26T18:38:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f571da0e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f57141ac760 con 0x56282b0ddf00 2024-04-26T18:38:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f571da0e740 10 client.0 _ll_register_callbacks cb 0x7f570400b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f5719295640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0e90e0 0x56282b0edd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f5718a94640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0ddf00 0x56282b0f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f5718a94640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0ddf00 0x56282b0f0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.174:3300/0 says I am v2:172.21.15.84:57520/0 (socket says 172.21.15.84:57520) 2024-04-26T18:38:17.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f5718a94640 1 -- 172.21.15.84:0/2473985088 learned_addr learned my addr 172.21.15.84:0/2473985088 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f5719295640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0e90e0 0x56282b0edd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34254/0 (socket says 172.21.15.84:34254) 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f570bfff640 1 --2- 172.21.15.84:0/2473985088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0de820 0x56282b0f12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f5719295640 1 -- 172.21.15.84:0/2473985088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0ddf00 msgr2=0x56282b0f0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f5719295640 1 --2- 172.21.15.84:0/2473985088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0ddf00 0x56282b0f0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f5719295640 1 -- 172.21.15.84:0/2473985088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0de820 msgr2=0x56282b0f12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f5719295640 1 --2- 172.21.15.84:0/2473985088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0de820 0x56282b0f12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f5719295640 1 -- 172.21.15.84:0/2473985088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56282b0ee550 con 0x56282b0e90e0 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f5719295640 1 --2- 172.21.15.84:0/2473985088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0e90e0 0x56282b0edd50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x56282b0db830 tx=0x7f570c012e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a00a602b92496ba server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f56f3fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f56f3fff640 1 -- 172.21.15.84:0/2473985088 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f570c022070 con 0x56282b0e90e0 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f56f3fff640 1 -- 172.21.15.84:0/2473985088 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f570c011c90 con 0x56282b0e90e0 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.847+0000 7f571da0e740 1 -- 172.21.15.84:0/2473985088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57100b7070 con 0x56282b0e90e0 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f56f3fff640 1 -- 172.21.15.84:0/2473985088 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f570c02e050 con 0x56282b0e90e0 2024-04-26T18:38:17.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f571da0e740 10 client.16641 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f571da0e740 1 -- 172.21.15.84:0/2473985088 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56282b160160 con 0x56282b0e90e0 2024-04-26T18:38:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f56f3fff640 1 -- 172.21.15.84:0/2473985088 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57100b7070 con 0x56282b0e90e0 2024-04-26T18:38:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f56f3fff640 1 -- 172.21.15.84:0/2473985088 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f570c01d3a0 con 0x56282b0e90e0 2024-04-26T18:38:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f571da0e740 10 client.16641 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f571da0e740 10 client.16641 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:38:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f571da0e740 20 client.16641 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f571da0e740 1 --2- 172.21.15.84:0/2473985088 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56282b1608e0 0x56282b162d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f571da0e740 4 client.16641 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:38:17.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f571da0e740 1 -- 172.21.15.84:0/2473985088 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56282b04bee0 con 0x56282b1608e0 2024-04-26T18:38:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f5718a94640 1 --2- 172.21.15.84:0/2473985088 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56282b1608e0 0x56282b162d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f5718a94640 1 --2- 172.21.15.84:0/2473985088 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56282b1608e0 0x56282b162d00 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=611e182f7914b672 server_cookie=d74a26f8ad641c4d in_seq=0 out_seq=0 2024-04-26T18:38:17.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.848+0000 7f56f3fff640 10 client.16641 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f56f3fff640 1 -- 172.21.15.84:0/2473985088 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56282b04bee0 con 0x56282b1608e0 2024-04-26T18:38:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f56f3fff640 10 client.16641 handle_command_reply: tid=0 2024-04-26T18:38:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f571da0e740 1 client.16641 shutdown 2024-04-26T18:38:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f571da0e740 1 -- 172.21.15.84:0/2473985088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0e90e0 msgr2=0x56282b0edd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f571da0e740 1 --2- 172.21.15.84:0/2473985088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0e90e0 0x56282b0edd50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x56282b0db830 tx=0x7f570c012e50 comp rx=0 tx=0).stop 2024-04-26T18:38:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f571da0e740 1 -- 172.21.15.84:0/2473985088 shutdown_connections 2024-04-26T18:38:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f571da0e740 1 --2- 172.21.15.84:0/2473985088 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56282b1608e0 0x56282b162d00 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f571da0e740 1 --2- 172.21.15.84:0/2473985088 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56282b0e90e0 0x56282b0edd50 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f571da0e740 1 --2- 172.21.15.84:0/2473985088 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56282b0de820 0x56282b0f12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f56f3fff640 0 client.16641 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:17.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f571da0e740 1 --2- 172.21.15.84:0/2473985088 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56282b0ddf00 0x56282b0f0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f571da0e740 1 -- 172.21.15.84:0/2473985088 >> 172.21.15.84:0/2473985088 conn(0x56282b0da410 msgr2=0x56282b0e0140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f571da0e740 1 -- 172.21.15.84:0/2473985088 shutdown_connections 2024-04-26T18:38:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.861+0000 7f571da0e740 1 -- 172.21.15.84:0/2473985088 wait complete. 2024-04-26T18:38:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.862+0000 7f571da0e740 20 client.16641 trim_cache size 0 max 16384 2024-04-26T18:38:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:38:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:38:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:38:17.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:38:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.862+0000 7f571acf8640 1 -- 172.21.15.84:0/1749095974 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f56fc0a3e40 msgr2=0x7f56fc0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.862+0000 7f571acf8640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f56fc0a3e40 0x7f56fc0a6300 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f570c0103b0 tx=0x7f570c010870 comp rx=0 tx=0).stop 2024-04-26T18:38:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.862+0000 7f571acf8640 1 -- 172.21.15.84:0/1749095974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5714101210 msgr2=0x7f5714199270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.862+0000 7f571acf8640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5714101210 0x7f5714199270 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f5710006010 tx=0x7f57100315d0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.862+0000 7f571acf8640 1 -- 172.21.15.84:0/1749095974 shutdown_connections 2024-04-26T18:38:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.862+0000 7f571acf8640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f56fc0a3e40 0x7f56fc0a6300 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.862+0000 7f571acf8640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f571419dae0 0x7f571419df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.862+0000 7f571acf8640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f57141062e0 0x7f57141997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.862+0000 7f571acf8640 1 --2- 172.21.15.84:0/1749095974 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5714101210 0x7f5714199270 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.862+0000 7f571acf8640 1 -- 172.21.15.84:0/1749095974 >> 172.21.15.84:0/1749095974 conn(0x7f5714077d50 msgr2=0x7f57140ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.862+0000 7f571acf8640 1 -- 172.21.15.84:0/1749095974 shutdown_connections 2024-04-26T18:38:17.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:17.862+0000 7f571acf8640 1 -- 172.21.15.84:0/1749095974 wait complete. 2024-04-26T18:38:18.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.518+0000 7f7ab847a700 1 -- 172.21.15.84:0/4183917771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ab0103740 msgr2=0x7f7ab0103bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:18.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.518+0000 7f7ab847a700 1 --2- 172.21.15.84:0/4183917771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ab0103740 0x7f7ab0103bc0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009990 tx=0x7f7aa4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.518+0000 7f7ab847a700 1 -- 172.21.15.84:0/4183917771 shutdown_connections 2024-04-26T18:38:18.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.518+0000 7f7ab847a700 1 --2- 172.21.15.84:0/4183917771 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab0104100 0x7f7ab0110370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.518+0000 7f7ab847a700 1 --2- 172.21.15.84:0/4183917771 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ab01095e0 0x7f7ab01099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.520 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.518+0000 7f7ab847a700 1 --2- 172.21.15.84:0/4183917771 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ab0103740 0x7f7ab0103bc0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.518+0000 7f7ab847a700 1 -- 172.21.15.84:0/4183917771 >> 172.21.15.84:0/4183917771 conn(0x7f7ab007b3b0 msgr2=0x7f7ab007b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.518+0000 7f7ab847a700 1 -- 172.21.15.84:0/4183917771 shutdown_connections 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.518+0000 7f7ab847a700 1 -- 172.21.15.84:0/4183917771 wait complete. 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.519+0000 7f7ab847a700 1 Processor -- start 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.519+0000 7f7ab847a700 1 -- start start 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.519+0000 7f7ab847a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ab0103740 0x7f7ab019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.519+0000 7f7ab847a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab0104100 0x7f7ab019fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.519+0000 7f7ab847a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ab01095e0 0x7f7ab01999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.519+0000 7f7ab847a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab0113760 con 0x7f7ab01095e0 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.519+0000 7f7ab847a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01135e0 con 0x7f7ab0103740 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.519+0000 7f7ab847a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01138e0 con 0x7f7ab0104100 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.519+0000 7f7ab6a17700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ab01095e0 0x7f7ab01999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.519+0000 7f7ab6a17700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ab01095e0 0x7f7ab01999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34268/0 (socket says 172.21.15.84:34268) 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.519+0000 7f7ab6a17700 1 -- 172.21.15.84:0/4061052767 learned_addr learned my addr 172.21.15.84:0/4061052767 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:18.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.519+0000 7f7ab6216700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ab0103740 0x7f7ab019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.520+0000 7f7ab5a15700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab0104100 0x7f7ab019fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.520+0000 7f7ab6a17700 1 -- 172.21.15.84:0/4061052767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab0104100 msgr2=0x7f7ab019fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.520+0000 7f7ab6a17700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab0104100 0x7f7ab019fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.520+0000 7f7ab6a17700 1 -- 172.21.15.84:0/4061052767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ab0103740 msgr2=0x7f7ab019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.520+0000 7f7ab6a17700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ab0103740 0x7f7ab019f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.520+0000 7f7ab6a17700 1 -- 172.21.15.84:0/4061052767 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab019a120 con 0x7f7ab01095e0 2024-04-26T18:38:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.520+0000 7f7ab6a17700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ab01095e0 0x7f7ab01999c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000b410 tx=0x7f7aa000b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.520+0000 7f7aaf7fe700 1 -- 172.21.15.84:0/4061052767 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0015020 con 0x7f7ab01095e0 2024-04-26T18:38:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.520+0000 7f7ab847a700 1 -- 172.21.15.84:0/4061052767 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab019a410 con 0x7f7ab01095e0 2024-04-26T18:38:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.520+0000 7f7aaf7fe700 1 -- 172.21.15.84:0/4061052767 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa000be20 con 0x7f7ab01095e0 2024-04-26T18:38:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.520+0000 7f7aaf7fe700 1 -- 172.21.15.84:0/4061052767 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0010ba0 con 0x7f7ab01095e0 2024-04-26T18:38:18.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.520+0000 7f7ab847a700 1 -- 172.21.15.84:0/4061052767 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab01a66d0 con 0x7f7ab01095e0 2024-04-26T18:38:18.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.522+0000 7f7aaf7fe700 1 -- 172.21.15.84:0/4061052767 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f7aa0018020 con 0x7f7ab01095e0 2024-04-26T18:38:18.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.523+0000 7f7aaf7fe700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7a980a3db0 0x7f7a980a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.523+0000 7f7ab6216700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7a980a3db0 0x7f7a980a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.523+0000 7f7aaf7fe700 1 -- 172.21.15.84:0/4061052767 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f7aa00cfd00 con 0x7f7ab01095e0 2024-04-26T18:38:18.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.523+0000 7f7ab6216700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7a980a3db0 0x7f7a980a6270 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7ab019ab00 tx=0x7f7aa800d040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:18.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.524+0000 7f7ac2d0cb80 1 Processor -- start 2024-04-26T18:38:18.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.524+0000 7f7ac2d0cb80 1 -- start start 2024-04-26T18:38:18.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.524+0000 7f7ac2d0cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddaea0 0x562acfddb280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.524+0000 7f7ac2d0cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfddb7c0 0x562acfdd5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.524+0000 7f7ac2d0cb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 0x562acfdd5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.524+0000 7f7ac2d0cb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562acfd27930 con 0x562acfdd5870 2024-04-26T18:38:18.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.524+0000 7f7ac2d0cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562acfc8da30 con 0x562acfddb7c0 2024-04-26T18:38:18.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.524+0000 7f7ac2d0cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562acfd238f0 con 0x562acfddaea0 2024-04-26T18:38:18.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.524+0000 7f7ab6a17700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 0x562acfdd5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.526 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.524+0000 7f7ab6a17700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 0x562acfdd5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34278/0 (socket says 172.21.15.84:34278) 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.524+0000 7f7ab6a17700 1 -- 172.21.15.84:0/2081410568 learned_addr learned my addr 172.21.15.84:0/2081410568 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.524+0000 7f7ab5a15700 1 --2- 172.21.15.84:0/2081410568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfddb7c0 0x562acfdd5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.524+0000 7f7ab6216700 1 --2- 172.21.15.84:0/2081410568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddaea0 0x562acfddb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.525+0000 7f7ab6a17700 1 -- 172.21.15.84:0/2081410568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddaea0 msgr2=0x562acfddb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.525+0000 7f7ab6a17700 1 --2- 172.21.15.84:0/2081410568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddaea0 0x562acfddb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.525+0000 7f7ab6a17700 1 -- 172.21.15.84:0/2081410568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfddb7c0 msgr2=0x562acfdd5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.525+0000 7f7ab6a17700 1 --2- 172.21.15.84:0/2081410568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfddb7c0 0x562acfdd5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.525+0000 7f7ab6a17700 1 -- 172.21.15.84:0/2081410568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562acfc749d0 con 0x562acfdd5870 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.525+0000 7f7ab5a15700 1 --2- 172.21.15.84:0/2081410568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfddb7c0 0x562acfdd5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.525+0000 7f7ab6a17700 1 --2- 172.21.15.84:0/2081410568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 0x562acfdd5cd0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00981d0 tx=0x7f7aa009c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f57783629ccb2fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.525+0000 7f7aacd70700 1 -- 172.21.15.84:0/2081410568 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa00b5070 con 0x562acfdd5870 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.525+0000 7f7aacd70700 1 -- 172.21.15.84:0/2081410568 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa00a49e0 con 0x562acfdd5870 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.525+0000 7f7aacd70700 1 -- 172.21.15.84:0/2081410568 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0018510 con 0x562acfdd5870 2024-04-26T18:38:18.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/2081410568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 msgr2=0x562acfdd5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:18.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/2081410568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 0x562acfdd5cd0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00981d0 tx=0x7f7aa009c4f0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/2081410568 shutdown_connections 2024-04-26T18:38:18.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/2081410568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 0x562acfdd5cd0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/2081410568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddaea0 0x562acfddb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/2081410568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfddb7c0 0x562acfdd5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/2081410568 >> 172.21.15.84:0/2081410568 conn(0x562acfd036c0 msgr2=0x562acfdd0000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:18.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/2081410568 shutdown_connections 2024-04-26T18:38:18.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/2081410568 wait complete. 2024-04-26T18:38:18.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 -- start start 2024-04-26T18:38:18.528 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 Processor -- start 2024-04-26T18:38:18.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 0x562acfde1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfddaea0 0x562acfdde920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddb7c0 0x562acfddee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.531 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562acfde2230 con 0x562acfdd5870 2024-04-26T18:38:18.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562acfce1b80 con 0x562acfddaea0 2024-04-26T18:38:18.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.526+0000 7f7ac2d0cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562acfde23b0 con 0x562acfddb7c0 2024-04-26T18:38:18.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ac2d0cb80 10 client.0 _ll_register_callbacks cb 0x562acfcbfa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:18.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ab6216700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 0x562acfde1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ab6216700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 0x562acfde1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34286/0 (socket says 172.21.15.84:34286) 2024-04-26T18:38:18.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ab6216700 1 -- 172.21.15.84:0/361463403 learned_addr learned my addr 172.21.15.84:0/361463403 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:18.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ab6a17700 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddb7c0 0x562acfddee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ab5a15700 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfddaea0 0x562acfdde920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ab6216700 1 -- 172.21.15.84:0/361463403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddb7c0 msgr2=0x562acfddee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:18.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ab6216700 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddb7c0 0x562acfddee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ab6216700 1 -- 172.21.15.84:0/361463403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfddaea0 msgr2=0x562acfdde920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:18.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ab6216700 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfddaea0 0x562acfdde920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.532 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ab6216700 1 -- 172.21.15.84:0/361463403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562acfce5580 con 0x562acfdd5870 2024-04-26T18:38:18.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ab5a15700 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfddaea0 0x562acfdde920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:38:18.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ab6216700 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 0x562acfde1580 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8016df0 tx=0x7f7aa8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f300aaf795c3d0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:18.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7a95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:18.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7a95ffb700 1 -- 172.21.15.84:0/361463403 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8016e20 con 0x562acfdd5870 2024-04-26T18:38:18.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/361463403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa00a49e0 con 0x562acfdd5870 2024-04-26T18:38:18.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7a95ffb700 1 -- 172.21.15.84:0/361463403 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f7aa8022470 con 0x562acfdd5870 2024-04-26T18:38:18.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7a95ffb700 1 -- 172.21.15.84:0/361463403 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7aa8030050 con 0x562acfdd5870 2024-04-26T18:38:18.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ac2d0cb80 10 client.16653 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:18.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.527+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/361463403 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562acfdecb60 con 0x562acfdd5870 2024-04-26T18:38:18.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.528+0000 7f7a95ffb700 1 -- 172.21.15.84:0/361463403 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa00a49e0 con 0x562acfdd5870 2024-04-26T18:38:18.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.528+0000 7f7a95ffb700 1 -- 172.21.15.84:0/361463403 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7aa802b750 con 0x562acfdd5870 2024-04-26T18:38:18.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.528+0000 7f7ac2d0cb80 10 client.16653 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:18.533 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.528+0000 7f7ac2d0cb80 10 client.16653 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.528+0000 7f7ac2d0cb80 20 client.16653 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.528+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562acfdef510 0x562acfdf1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.528+0000 7f7ac2d0cb80 4 client.16653 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.528+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/361463403 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562acfd35310 con 0x562acfdef510 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.528+0000 7f7ab5a15700 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562acfdef510 0x562acfdf1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7ab5a15700 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562acfdef510 0x562acfdf1950 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67779bcf9d4feadb server_cookie=524c41474f73d02f in_seq=0 out_seq=0 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7a95ffb700 10 client.16653 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7a95ffb700 1 -- 172.21.15.84:0/361463403 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562acfd35310 con 0x562acfdef510 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7a95ffb700 10 client.16653 handle_command_reply: tid=0 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7ac2d0cb80 1 client.16653 shutdown 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/361463403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 msgr2=0x562acfde1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 0x562acfde1580 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7aa8016df0 tx=0x7f7aa8004ad0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/361463403 shutdown_connections 2024-04-26T18:38:18.534 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562acfdef510 0x562acfdf1950 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfdd5870 0x562acfde1580 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddb7c0 0x562acfddee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7a95ffb700 0 client.16653 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:18.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/361463403 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfddaea0 0x562acfdde920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.529+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/361463403 >> 172.21.15.84:0/361463403 conn(0x562acfd036c0 msgr2=0x562acfdd4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:18.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.530+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/361463403 shutdown_connections 2024-04-26T18:38:18.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.530+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/361463403 wait complete. 2024-04-26T18:38:18.535 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.530+0000 7f7ac2d0cb80 20 client.16653 trim_cache size 0 max 16384 2024-04-26T18:38:18.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.549+0000 7f7ac2d0cb80 1 Processor -- start 2024-04-26T18:38:18.549 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.549+0000 7f7ac2d0cb80 1 -- start start 2024-04-26T18:38:18.549 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:18 smithi084 ceph-mon[29141]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T18:38:18.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.549+0000 7f7ac2d0cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 0x562acfdd5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.550 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.549+0000 7f7ac2d0cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddaea0 0x562acfde7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.549+0000 7f7ac2d0cb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfddb7c0 0x562acfdeb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.549+0000 7f7ac2d0cb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01adf40 con 0x562acfddb7c0 2024-04-26T18:38:18.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.549+0000 7f7ac2d0cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01afa80 con 0x562acfdd5870 2024-04-26T18:38:18.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.549+0000 7f7ac2d0cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01ae610 con 0x562acfddaea0 2024-04-26T18:38:18.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7ab6216700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 0x562acfdd5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7ab6216700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 0x562acfdd5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:60254/0 (socket says 172.21.15.84:60254) 2024-04-26T18:38:18.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7ab6216700 1 -- 172.21.15.84:0/1233337444 learned_addr learned my addr 172.21.15.84:0/1233337444 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:18.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7ab6a17700 1 --2- 172.21.15.84:0/1233337444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfddb7c0 0x562acfdeb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7ab5a15700 1 --2- 172.21.15.84:0/1233337444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddaea0 0x562acfde7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7ab6216700 1 -- 172.21.15.84:0/1233337444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddaea0 msgr2=0x562acfde7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:18.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7ab6216700 1 --2- 172.21.15.84:0/1233337444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddaea0 0x562acfde7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.551 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7ab6216700 1 -- 172.21.15.84:0/1233337444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfddb7c0 msgr2=0x562acfdeb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:18.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7ab6216700 1 --2- 172.21.15.84:0/1233337444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfddb7c0 0x562acfdeb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7ab6216700 1 -- 172.21.15.84:0/1233337444 --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562acfdf1cc0 con 0x562acfdd5870 2024-04-26T18:38:18.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7ab5a15700 1 --2- 172.21.15.84:0/1233337444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddaea0 0x562acfde7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:38:18.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7ab6216700 1 --2- 172.21.15.84:0/1233337444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 0x562acfdd5250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7aa802bb40 tx=0x7f7aa800ad30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2234196a8c450a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:18.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7a97fff700 1 -- 172.21.15.84:0/1233337444 <== mon.2 v2:172.21.15.103:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa803a020 con 0x562acfdd5870 2024-04-26T18:38:18.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7a97fff700 1 -- 172.21.15.84:0/1233337444 <== mon.2 v2:172.21.15.103:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa8027070 con 0x562acfdd5870 2024-04-26T18:38:18.552 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.550+0000 7f7a97fff700 1 -- 172.21.15.84:0/1233337444 <== mon.2 v2:172.21.15.103:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80226f0 con 0x562acfdd5870 2024-04-26T18:38:18.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.551+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/1233337444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 msgr2=0x562acfdd5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:18.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.551+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/1233337444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 0x562acfdd5250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7aa802bb40 tx=0x7f7aa800ad30 comp rx=0 tx=0).stop 2024-04-26T18:38:18.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.551+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/1233337444 shutdown_connections 2024-04-26T18:38:18.555 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.551+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/1233337444 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfddb7c0 0x562acfdeb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.551+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/1233337444 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 0x562acfdd5250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.551+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/1233337444 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddaea0 0x562acfde7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.551+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/1233337444 >> 172.21.15.84:0/1233337444 conn(0x562acfd036c0 msgr2=0x562acfdc9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:18.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.551+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/1233337444 shutdown_connections 2024-04-26T18:38:18.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.551+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/1233337444 wait complete. 2024-04-26T18:38:18.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.552+0000 7f7ac2d0cb80 1 -- start start 2024-04-26T18:38:18.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.552+0000 7f7ac2d0cb80 1 Processor -- start 2024-04-26T18:38:18.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.552+0000 7f7ac2d0cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 0x562acfdf6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.552+0000 7f7ac2d0cb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfddaea0 0x562acfdf6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.552+0000 7f7ac2d0cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddb7c0 0x562acfdf6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.552+0000 7f7ac2d0cb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01af3b0 con 0x562acfddaea0 2024-04-26T18:38:18.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.552+0000 7f7ac2d0cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01aece0 con 0x562acfdd5870 2024-04-26T18:38:18.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.552+0000 7f7ac2d0cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab01abf00 con 0x562acfddb7c0 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.552+0000 7f7ac2d0cb80 10 client.0 _ll_register_callbacks cb 0x562acfcbfa50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.552+0000 7f7ab6216700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 0x562acfdf6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.552+0000 7f7ab6216700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 0x562acfdf6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:60268/0 (socket says 172.21.15.84:60268) 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.552+0000 7f7ab6216700 1 -- 172.21.15.84:0/2513295118 learned_addr learned my addr 172.21.15.84:0/2513295118 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.552+0000 7f7ab5a15700 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfddaea0 0x562acfdf6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7ab6a17700 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddb7c0 0x562acfdf6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7ab5a15700 1 -- 172.21.15.84:0/2513295118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddb7c0 msgr2=0x562acfdf6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7ab5a15700 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddb7c0 0x562acfdf6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7ab5a15700 1 -- 172.21.15.84:0/2513295118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 msgr2=0x562acfdf6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7ab5a15700 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 0x562acfdf6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7ab5a15700 1 -- 172.21.15.84:0/2513295118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562acfe59c20 con 0x562acfddaea0 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7ab6216700 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 0x562acfdf6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7ab5a15700 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfddaea0 0x562acfdf6790 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40059d0 tx=0x7f7aa400bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b64495abcb0ae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:18.557 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7a95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7a95ffb700 1 -- 172.21.15.84:0/2513295118 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4003a60 con 0x562acfddaea0 2024-04-26T18:38:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/2513295118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa8027070 con 0x562acfddaea0 2024-04-26T18:38:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7a95ffb700 1 -- 172.21.15.84:0/2513295118 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f7aa4003c00 con 0x562acfddaea0 2024-04-26T18:38:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7a95ffb700 1 -- 172.21.15.84:0/2513295118 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7aa4003e00 con 0x562acfddaea0 2024-04-26T18:38:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7ac2d0cb80 10 client.16659 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.553+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/2513295118 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562acfe5af60 con 0x562acfddaea0 2024-04-26T18:38:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.554+0000 7f7a95ffb700 1 -- 172.21.15.84:0/2513295118 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa8027070 con 0x562acfddaea0 2024-04-26T18:38:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.554+0000 7f7a95ffb700 1 -- 172.21.15.84:0/2513295118 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7aa4034030 con 0x562acfddaea0 2024-04-26T18:38:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.554+0000 7f7ac2d0cb80 10 client.16659 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.554+0000 7f7ac2d0cb80 10 client.16659 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:38:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.554+0000 7f7ac2d0cb80 20 client.16659 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.554+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562acfe5b670 0x562acfe5da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:18.558 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.554+0000 7f7ac2d0cb80 4 client.16659 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "400c61dd", "release": true, "await": true}] 2024-04-26T18:38:18.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.554+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/2513295118 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "set_id": "400c61dd", "release": true, "await": true}) -- 0x562acfd2f3b0 con 0x562acfe5b670 2024-04-26T18:38:18.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.554+0000 7f7ab6216700 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562acfe5b670 0x562acfe5da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:18.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.555+0000 7f7ab6216700 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562acfe5b670 0x562acfe5da90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6790c84d865b4161 server_cookie=3e4eabae4498af27 in_seq=0 out_seq=0 2024-04-26T18:38:18.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.555+0000 7f7a95ffb700 10 client.16659 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:18.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7a95ffb700 1 -- 172.21.15.84:0/2513295118 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x562acfd2f3b0 con 0x562acfe5b670 2024-04-26T18:38:18.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7a95ffb700 10 client.16659 handle_command_reply: tid=0 2024-04-26T18:38:18.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7ac2d0cb80 1 client.16659 shutdown 2024-04-26T18:38:18.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/2513295118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfddaea0 msgr2=0x562acfdf6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:18.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfddaea0 0x562acfdf6790 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7aa40059d0 tx=0x7f7aa400bc20 comp rx=0 tx=0).stop 2024-04-26T18:38:18.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/2513295118 shutdown_connections 2024-04-26T18:38:18.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562acfe5b670 0x562acfe5da90 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562acfddb7c0 0x562acfdf6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562acfdd5870 0x562acfdf6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7ac2d0cb80 1 --2- 172.21.15.84:0/2513295118 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562acfddaea0 0x562acfdf6790 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/2513295118 >> 172.21.15.84:0/2513295118 conn(0x562acfd036c0 msgr2=0x562acfdc9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:18.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7a95ffb700 0 client.16659 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:18.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/2513295118 shutdown_connections 2024-04-26T18:38:18.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7ac2d0cb80 1 -- 172.21.15.84:0/2513295118 wait complete. 2024-04-26T18:38:18.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.559+0000 7f7ac2d0cb80 20 client.16659 trim_cache size 0 max 16384 2024-04-26T18:38:18.561 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:38:18.561 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:38:18.561 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:38:18.561 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 28, 2024-04-26T18:38:18.561 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:38:18.562 INFO:teuthology.orchestra.run.smithi084.stdout: "400c61dd": { 2024-04-26T18:38:18.562 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 28, 2024-04-26T18:38:18.562 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:38:18.562 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:38:18.562 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:38:18.562 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:38:18.562 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:38:18.562 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:38:18.562 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 81.2, 2024-04-26T18:38:18.562 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:38:18.562 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:38:18.562 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:38:18.562 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:38:18.563 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:38:18.563 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:38:18.563 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:18.563 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:18.563 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:18.563 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:18.563 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:18.563 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:38:18.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.560+0000 7f7ab847a700 1 -- 172.21.15.84:0/4061052767 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7a980a3db0 msgr2=0x7f7a980a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:18.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.560+0000 7f7ab847a700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7a980a3db0 0x7f7a980a6270 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7ab019ab00 tx=0x7f7aa800d040 comp rx=0 tx=0).stop 2024-04-26T18:38:18.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.560+0000 7f7ab847a700 1 -- 172.21.15.84:0/4061052767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ab01095e0 msgr2=0x7f7ab01999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:18.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.560+0000 7f7ab847a700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ab01095e0 0x7f7ab01999c0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f7aa000b410 tx=0x7f7aa000b720 comp rx=0 tx=0).stop 2024-04-26T18:38:18.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.560+0000 7f7ab847a700 1 -- 172.21.15.84:0/4061052767 shutdown_connections 2024-04-26T18:38:18.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.560+0000 7f7ab847a700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7a980a3db0 0x7f7a980a6270 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.560+0000 7f7ab847a700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ab01095e0 0x7f7ab01999c0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.560+0000 7f7ab847a700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ab0103740 0x7f7ab019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.560+0000 7f7ab847a700 1 --2- 172.21.15.84:0/4061052767 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ab0104100 0x7f7ab019fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:18.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.560+0000 7f7ab847a700 1 -- 172.21.15.84:0/4061052767 >> 172.21.15.84:0/4061052767 conn(0x7f7ab007b3b0 msgr2=0x7f7ab01021b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:18.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.560+0000 7f7ab847a700 1 -- 172.21.15.84:0/4061052767 shutdown_connections 2024-04-26T18:38:18.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:18.560+0000 7f7ab847a700 1 -- 172.21.15.84:0/4061052767 wait complete. 2024-04-26T18:38:18.688 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:18 smithi174 ceph-mon[32705]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T18:38:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:18 smithi103 ceph-mon[32727]: osdmap e104: 12 total, 12 up, 12 in 2024-04-26T18:38:19.316 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '400c61dd', seconds elapsed: 32.8 2024-04-26T18:38:19.316 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.2 2024-04-26T18:38:19.317 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:38:19.578 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:38:19.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:19 smithi084 ceph-mon[29141]: pgmap v323: 97 pgs: 97 active+clean; 4.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-26T18:38:19.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:19 smithi174 ceph-mon[32705]: pgmap v323: 97 pgs: 97 active+clean; 4.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-26T18:38:19.778 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:19 smithi103 ceph-mon[32727]: pgmap v323: 97 pgs: 97 active+clean; 4.5 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-04-26T18:38:21.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.357+0000 7f16a8641700 1 -- 172.21.15.84:0/504645298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a01094c0 msgr2=0x7f16a01098a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:21.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.357+0000 7f16a8641700 1 --2- 172.21.15.84:0/504645298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a01094c0 0x7f16a01098a0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f1690009990 tx=0x7f1690009ca0 comp rx=0 tx=0).stop 2024-04-26T18:38:21.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 -- 172.21.15.84:0/504645298 shutdown_connections 2024-04-26T18:38:21.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 --2- 172.21.15.84:0/504645298 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a01094c0 0x7f16a01098a0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:21.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 --2- 172.21.15.84:0/504645298 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a0103e30 0x7f16a0110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:21.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 --2- 172.21.15.84:0/504645298 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a0103470 0x7f16a01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:21.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 -- 172.21.15.84:0/504645298 >> 172.21.15.84:0/504645298 conn(0x7f16a007a580 msgr2=0x7f16a007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:21.360 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 -- 172.21.15.84:0/504645298 shutdown_connections 2024-04-26T18:38:21.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 -- 172.21.15.84:0/504645298 wait complete. 2024-04-26T18:38:21.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 Processor -- start 2024-04-26T18:38:21.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 -- start start 2024-04-26T18:38:21.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a0103470 0x7f16a0075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:21.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a0103e30 0x7f16a0075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:21.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a01094c0 0x7f16a00761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:21.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0113610 con 0x7f16a0103470 2024-04-26T18:38:21.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0113490 con 0x7f16a01094c0 2024-04-26T18:38:21.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.358+0000 7f16a8641700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f16a0113790 con 0x7f16a0103e30 2024-04-26T18:38:21.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a63dd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a0103470 0x7f16a0075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:21.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a63dd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a0103470 0x7f16a0075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34322/0 (socket says 172.21.15.84:34322) 2024-04-26T18:38:21.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a63dd700 1 -- 172.21.15.84:0/2505317497 learned_addr learned my addr 172.21.15.84:0/2505317497 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:21.361 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a6bde700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a01094c0 0x7f16a00761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a5bdc700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a0103e30 0x7f16a0075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a63dd700 1 -- 172.21.15.84:0/2505317497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a0103e30 msgr2=0x7f16a0075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a63dd700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a0103e30 0x7f16a0075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a63dd700 1 -- 172.21.15.84:0/2505317497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a01094c0 msgr2=0x7f16a00761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a63dd700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a01094c0 0x7f16a00761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a63dd700 1 -- 172.21.15.84:0/2505317497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16a01a7ff0 con 0x7f16a0103470 2024-04-26T18:38:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a6bde700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a01094c0 0x7f16a00761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:38:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a63dd700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a0103470 0x7f16a0075720 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1690005910 tx=0x7f1690003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16977fe700 1 -- 172.21.15.84:0/2505317497 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690027070 con 0x7f16a0103470 2024-04-26T18:38:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a8641700 1 -- 172.21.15.84:0/2505317497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16a01a8280 con 0x7f16a0103470 2024-04-26T18:38:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16977fe700 1 -- 172.21.15.84:0/2505317497 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1690004140 con 0x7f16a0103470 2024-04-26T18:38:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16977fe700 1 -- 172.21.15.84:0/2505317497 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169000f460 con 0x7f16a0103470 2024-04-26T18:38:21.362 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.359+0000 7f16a8641700 1 -- 172.21.15.84:0/2505317497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16a01a8760 con 0x7f16a0103470 2024-04-26T18:38:21.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.360+0000 7f16a8641700 1 -- 172.21.15.84:0/2505317497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16a006a110 con 0x7f16a0103470 2024-04-26T18:38:21.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.362+0000 7f16977fe700 1 -- 172.21.15.84:0/2505317497 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f169002a070 con 0x7f16a0103470 2024-04-26T18:38:21.363 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.362+0000 7f16977fe700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f168c0a3db0 0x7f168c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:21.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.362+0000 7f16a5bdc700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f168c0a3db0 0x7f168c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:21.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.362+0000 7f16977fe700 1 -- 172.21.15.84:0/2505317497 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f16900d1f10 con 0x7f16a0103470 2024-04-26T18:38:21.364 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.363+0000 7f16a5bdc700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f168c0a3db0 0x7f168c0a6270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f169c005ea0 tx=0x7f169c005e30 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:21.366 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.366+0000 7f16977fe700 1 -- 172.21.15.84:0/2505317497 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1690031070 con 0x7f16a0103470 2024-04-26T18:38:21.618 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.617+0000 7f16a8641700 1 -- 172.21.15.84:0/2505317497 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f16a002e490 con 0x7f16a0103470 2024-04-26T18:38:21.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.618+0000 7f16977fe700 1 -- 172.21.15.84:0/2505317497 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f1690013070 con 0x7f16a0103470 2024-04-26T18:38:21.619 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:38:21.619 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:38:21.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.621+0000 7f16a8641700 1 -- 172.21.15.84:0/2505317497 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f168c0a3db0 msgr2=0x7f168c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:21.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.621+0000 7f16a8641700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f168c0a3db0 0x7f168c0a6270 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f169c005ea0 tx=0x7f169c005e30 comp rx=0 tx=0).stop 2024-04-26T18:38:21.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.621+0000 7f16a8641700 1 -- 172.21.15.84:0/2505317497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a0103470 msgr2=0x7f16a0075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.621+0000 7f16a8641700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a0103470 0x7f16a0075720 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f1690005910 tx=0x7f1690003730 comp rx=0 tx=0).stop 2024-04-26T18:38:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.621+0000 7f16a8641700 1 -- 172.21.15.84:0/2505317497 shutdown_connections 2024-04-26T18:38:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.621+0000 7f16a8641700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f168c0a3db0 0x7f168c0a6270 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.621+0000 7f16a8641700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f16a0103470 0x7f16a0075720 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.621+0000 7f16a8641700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f16a01094c0 0x7f16a00761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.621+0000 7f16a8641700 1 --2- 172.21.15.84:0/2505317497 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f16a0103e30 0x7f16a0075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.621+0000 7f16a8641700 1 -- 172.21.15.84:0/2505317497 >> 172.21.15.84:0/2505317497 conn(0x7f16a007a580 msgr2=0x7f16a0102250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.621+0000 7f16a8641700 1 -- 172.21.15.84:0/2505317497 shutdown_connections 2024-04-26T18:38:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:21.621+0000 7f16a8641700 1 -- 172.21.15.84:0/2505317497 wait complete. 2024-04-26T18:38:21.622 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:38:21.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:21 smithi174 ceph-mon[32705]: pgmap v324: 97 pgs: 97 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-04-26T18:38:21.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:21 smithi084 ceph-mon[29141]: pgmap v324: 97 pgs: 97 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-04-26T18:38:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:21 smithi103 ceph-mon[32727]: pgmap v324: 97 pgs: 97 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-04-26T18:38:22.318 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 60 --expiration 81.2 --await-for 42.0' 2024-04-26T18:38:22.318 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db / --timeout 60 --expiration 81.2 --await-for 42.0 2024-04-26T18:38:22.565 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:38:22.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:22 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2505317497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:38:22.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:22 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2505317497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:38:22.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:22 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2505317497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:38:23.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:23 smithi174 ceph-mon[32705]: pgmap v325: 97 pgs: 97 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-04-26T18:38:23.745 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:23 smithi084 ceph-mon[29141]: pgmap v325: 97 pgs: 97 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-04-26T18:38:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:23 smithi103 ceph-mon[32727]: pgmap v325: 97 pgs: 97 active+clean; 4.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 7.3 KiB/s wr, 12 op/s 2024-04-26T18:38:24.278 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.276+0000 7f9d8543f700 1 -- 172.21.15.84:0/3630973253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d80102700 msgr2=0x7f9d80102b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:24.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.276+0000 7f9d8543f700 1 --2- 172.21.15.84:0/3630973253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d80102700 0x7f9d80102b80 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009990 tx=0x7f9d74009ca0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.277+0000 7f9d8543f700 1 -- 172.21.15.84:0/3630973253 shutdown_connections 2024-04-26T18:38:24.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.277+0000 7f9d8543f700 1 --2- 172.21.15.84:0/3630973253 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d801030c0 0x7f9d8010f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.277+0000 7f9d8543f700 1 --2- 172.21.15.84:0/3630973253 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d801084c0 0x7f9d801088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.277+0000 7f9d8543f700 1 --2- 172.21.15.84:0/3630973253 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d80102700 0x7f9d80102b80 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.277+0000 7f9d8543f700 1 -- 172.21.15.84:0/3630973253 >> 172.21.15.84:0/3630973253 conn(0x7f9d800792d0 msgr2=0x7f9d800796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:24.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.277+0000 7f9d8543f700 1 -- 172.21.15.84:0/3630973253 shutdown_connections 2024-04-26T18:38:24.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.277+0000 7f9d8543f700 1 -- 172.21.15.84:0/3630973253 wait complete. 2024-04-26T18:38:24.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.277+0000 7f9d8543f700 1 Processor -- start 2024-04-26T18:38:24.279 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.277+0000 7f9d8543f700 1 -- start start 2024-04-26T18:38:24.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d8543f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d80102700 0x7f9d801041b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d8543f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d801030c0 0x7f9d801046f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d8543f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d801084c0 0x7f9d80104c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d8543f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9d80112660 con 0x7f9d801030c0 2024-04-26T18:38:24.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d8543f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801124e0 con 0x7f9d80102700 2024-04-26T18:38:24.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d8543f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801127e0 con 0x7f9d801084c0 2024-04-26T18:38:24.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d7e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d801030c0 0x7f9d801046f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d7e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d801030c0 0x7f9d801046f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34342/0 (socket says 172.21.15.84:34342) 2024-04-26T18:38:24.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d7e7fc700 1 -- 172.21.15.84:0/3537655107 learned_addr learned my addr 172.21.15.84:0/3537655107 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:24.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d7f7fe700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d801084c0 0x7f9d80104c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d7effd700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d80102700 0x7f9d801041b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.280 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d7e7fc700 1 -- 172.21.15.84:0/3537655107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d801084c0 msgr2=0x7f9d80104c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:24.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d801084c0 0x7f9d80104c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d7e7fc700 1 -- 172.21.15.84:0/3537655107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d80102700 msgr2=0x7f9d801041b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:24.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d80102700 0x7f9d801041b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d7e7fc700 1 -- 172.21.15.84:0/3537655107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d801a7f90 con 0x7f9d801030c0 2024-04-26T18:38:24.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d801030c0 0x7f9d801046f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9d74003ac0 tx=0x7f9d74003af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:24.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d67fff700 1 -- 172.21.15.84:0/3537655107 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74027070 con 0x7f9d801030c0 2024-04-26T18:38:24.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d8543f700 1 -- 172.21.15.84:0/3537655107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d801a8220 con 0x7f9d801030c0 2024-04-26T18:38:24.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d67fff700 1 -- 172.21.15.84:0/3537655107 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d74003680 con 0x7f9d801030c0 2024-04-26T18:38:24.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.278+0000 7f9d67fff700 1 -- 172.21.15.84:0/3537655107 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d74022d40 con 0x7f9d801030c0 2024-04-26T18:38:24.281 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.279+0000 7f9d8543f700 1 -- 172.21.15.84:0/3537655107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d801a8700 con 0x7f9d801030c0 2024-04-26T18:38:24.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.280+0000 7f9d67fff700 1 -- 172.21.15.84:0/3537655107 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9d7402a070 con 0x7f9d801030c0 2024-04-26T18:38:24.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.281+0000 7f9d67fff700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9d680a3e50 0x7f9d680a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.281+0000 7f9d7effd700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9d680a3e50 0x7f9d680a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.281+0000 7f9d67fff700 1 -- 172.21.15.84:0/3537655107 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f9d740d15a0 con 0x7f9d801030c0 2024-04-26T18:38:24.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.282+0000 7f9d7effd700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9d680a3e50 0x7f9d680a6310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9d70005ea0 tx=0x7f9d70005e30 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:24.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.282+0000 7f9d8fcd1b80 1 Processor -- start 2024-04-26T18:38:24.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.282+0000 7f9d8fcd1b80 1 -- start start 2024-04-26T18:38:24.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.282+0000 7f9d8fcd1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70a230 0x55f3ac70a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.282+0000 7f9d8fcd1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 0x55f3ac7154c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.284 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.282+0000 7f9d8fcd1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 0x55f3ac717df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.282+0000 7f9d8fcd1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f3ac65d8d0 con 0x55f3ac70ab50 2024-04-26T18:38:24.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.282+0000 7f9d8fcd1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f3ac5fc770 con 0x55f3ac715a00 2024-04-26T18:38:24.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.282+0000 7f9d8fcd1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f3ac659890 con 0x55f3ac70a230 2024-04-26T18:38:24.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d7e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 0x55f3ac7154c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d7e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 0x55f3ac7154c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34358/0 (socket says 172.21.15.84:34358) 2024-04-26T18:38:24.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d7e7fc700 1 -- 172.21.15.84:0/3885269942 learned_addr learned my addr 172.21.15.84:0/3885269942 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:24.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d7f7fe700 1 --2- 172.21.15.84:0/3885269942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 0x55f3ac717df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d7effd700 1 --2- 172.21.15.84:0/3885269942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70a230 0x55f3ac70a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d7e7fc700 1 -- 172.21.15.84:0/3885269942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70a230 msgr2=0x55f3ac70a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:24.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/3885269942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70a230 0x55f3ac70a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d7e7fc700 1 -- 172.21.15.84:0/3885269942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 msgr2=0x55f3ac717df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:24.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/3885269942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 0x55f3ac717df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.285 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d7e7fc700 1 -- 172.21.15.84:0/3885269942 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ac5aaa60 con 0x55f3ac70ab50 2024-04-26T18:38:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/3885269942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 0x55f3ac7154c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9d74099f60 tx=0x7f9d74022810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e36f0b8e9106fe1e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d657fa700 1 -- 172.21.15.84:0/3885269942 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7409d590 con 0x55f3ac70ab50 2024-04-26T18:38:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d657fa700 1 -- 172.21.15.84:0/3885269942 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d740a1940 con 0x55f3ac70ab50 2024-04-26T18:38:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.283+0000 7f9d657fa700 1 -- 172.21.15.84:0/3885269942 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d740a5ce0 con 0x55f3ac70ab50 2024-04-26T18:38:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.284+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/3885269942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 msgr2=0x55f3ac7154c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.284+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/3885269942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 0x55f3ac7154c0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f9d74099f60 tx=0x7f9d74022810 comp rx=0 tx=0).stop 2024-04-26T18:38:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.284+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/3885269942 shutdown_connections 2024-04-26T18:38:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.284+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/3885269942 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 0x55f3ac717df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.284+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/3885269942 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70a230 0x55f3ac70a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.284+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/3885269942 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 0x55f3ac7154c0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.284+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/3885269942 >> 172.21.15.84:0/3885269942 conn(0x55f3ac637ed0 msgr2=0x55f3ac706720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.284+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/3885269942 shutdown_connections 2024-04-26T18:38:24.286 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.284+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/3885269942 wait complete. 2024-04-26T18:38:24.290 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.284+0000 7f9d8fcd1b80 1 -- start start 2024-04-26T18:38:24.290 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d8fcd1b80 1 Processor -- start 2024-04-26T18:38:24.290 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d8fcd1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70a230 0x55f3ac715270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.290 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d8fcd1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 0x55f3ac712570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.290 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d8fcd1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 0x55f3ac712ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.290 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d8fcd1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f3ac718390 con 0x55f3ac70ab50 2024-04-26T18:38:24.290 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d8fcd1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f3ac5c3ae0 con 0x55f3ac715a00 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d8fcd1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f3ac718510 con 0x55f3ac70a230 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d8fcd1b80 10 client.0 _ll_register_callbacks cb 0x55f3ac5f1710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d7e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 0x55f3ac712570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d7e7fc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 0x55f3ac712570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34370/0 (socket says 172.21.15.84:34370) 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d7e7fc700 1 -- 172.21.15.84:0/2780444765 learned_addr learned my addr 172.21.15.84:0/2780444765 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d7effd700 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70a230 0x55f3ac715270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d7e7fc700 1 -- 172.21.15.84:0/2780444765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70a230 msgr2=0x55f3ac715270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d7f7fe700 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 0x55f3ac712ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70a230 0x55f3ac715270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d7e7fc700 1 -- 172.21.15.84:0/2780444765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 msgr2=0x55f3ac712ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 0x55f3ac712ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d7e7fc700 1 -- 172.21.15.84:0/2780444765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ac7132a0 con 0x55f3ac70ab50 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d7effd700 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70a230 0x55f3ac715270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:38:24.291 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 0x55f3ac712570 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9d740cf230 tx=0x7f9d74088040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e18c14dde3b2c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d62ff5700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.285+0000 7f9d62ff5700 1 -- 172.21.15.84:0/2780444765 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d740a6170 con 0x55f3ac70ab50 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d62ff5700 1 -- 172.21.15.84:0/2780444765 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f9d740a5ef0 con 0x55f3ac70ab50 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2780444765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d740a1940 con 0x55f3ac70ab50 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d62ff5700 1 -- 172.21.15.84:0/2780444765 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9d7409d590 con 0x55f3ac70ab50 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d8fcd1b80 10 client.16677 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2780444765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3ac718390 con 0x55f3ac70ab50 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d62ff5700 1 -- 172.21.15.84:0/2780444765 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d740a1940 con 0x55f3ac70ab50 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d62ff5700 1 -- 172.21.15.84:0/2780444765 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9d740bc020 con 0x55f3ac70ab50 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d8fcd1b80 10 client.16677 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d8fcd1b80 10 client.16677 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d8fcd1b80 20 client.16677 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f3ac725160 0x55f3ac727550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.292 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d8fcd1b80 4 client.16677 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2780444765 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3ac66b2b0 con 0x55f3ac725160 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.286+0000 7f9d7effd700 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f3ac725160 0x55f3ac727550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.287+0000 7f9d62ff5700 10 client.16677 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.287+0000 7f9d7effd700 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f3ac725160 0x55f3ac727550 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e01f0fa7ae00f4f2 server_cookie=fc77223e70f9b6ab in_seq=0 out_seq=0 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.287+0000 7f9d62ff5700 1 -- 172.21.15.84:0/2780444765 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3ac66b2b0 con 0x55f3ac725160 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.287+0000 7f9d62ff5700 10 client.16677 handle_command_reply: tid=0 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.288+0000 7f9d8fcd1b80 1 client.16677 shutdown 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.288+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2780444765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 msgr2=0x55f3ac712570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.288+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 0x55f3ac712570 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f9d740cf230 tx=0x7f9d74088040 comp rx=0 tx=0).stop 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.288+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2780444765 shutdown_connections 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.288+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f3ac725160 0x55f3ac727550 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.288+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 0x55f3ac712ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.288+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70a230 0x55f3ac715270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.288+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/2780444765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70ab50 0x55f3ac712570 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.293 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.288+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2780444765 >> 172.21.15.84:0/2780444765 conn(0x55f3ac637ed0 msgr2=0x55f3ac706350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:24.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.288+0000 7f9d62ff5700 0 client.16677 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:24.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.288+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2780444765 shutdown_connections 2024-04-26T18:38:24.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.288+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2780444765 wait complete. 2024-04-26T18:38:24.294 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.288+0000 7f9d8fcd1b80 20 client.16677 trim_cache size 0 max 16384 2024-04-26T18:38:24.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.307+0000 7f9d8fcd1b80 1 Processor -- start 2024-04-26T18:38:24.308 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.307+0000 7f9d8fcd1b80 1 -- start start 2024-04-26T18:38:24.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.307+0000 7f9d8fcd1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 0x55f3ac711440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.307+0000 7f9d8fcd1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac70ab50 0x55f3ac721250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.307+0000 7f9d8fcd1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac715a00 0x55f3ac72ce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.307+0000 7f9d8fcd1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801afe30 con 0x55f3ac70a230 2024-04-26T18:38:24.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.307+0000 7f9d8fcd1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801b1970 con 0x55f3ac70ab50 2024-04-26T18:38:24.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.307+0000 7f9d8fcd1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801b0500 con 0x55f3ac715a00 2024-04-26T18:38:24.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.307+0000 7f9d7effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 0x55f3ac711440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.307+0000 7f9d7effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 0x55f3ac711440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34384/0 (socket says 172.21.15.84:34384) 2024-04-26T18:38:24.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.307+0000 7f9d7effd700 1 -- 172.21.15.84:0/1507323632 learned_addr learned my addr 172.21.15.84:0/1507323632 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:24.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.308+0000 7f9d7effd700 1 -- 172.21.15.84:0/1507323632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac715a00 msgr2=0x55f3ac72ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:24.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.308+0000 7f9d7effd700 1 --2- 172.21.15.84:0/1507323632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac715a00 0x55f3ac72ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.308+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/1507323632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac70ab50 0x55f3ac721250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.309 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.308+0000 7f9d7effd700 1 -- 172.21.15.84:0/1507323632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac70ab50 msgr2=0x55f3ac721250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.308+0000 7f9d7effd700 1 --2- 172.21.15.84:0/1507323632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac70ab50 0x55f3ac721250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.308+0000 7f9d7effd700 1 -- 172.21.15.84:0/1507323632 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3ac712340 con 0x55f3ac70a230 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.308+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/1507323632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac70ab50 0x55f3ac721250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.308+0000 7f9d7effd700 1 --2- 172.21.15.84:0/1507323632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 0x55f3ac711440 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9d70005d60 tx=0x7f9d7000af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6be5d17dcad4ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.308+0000 7f9d64ff9700 1 -- 172.21.15.84:0/1507323632 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d7001fdb0 con 0x55f3ac70a230 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.308+0000 7f9d64ff9700 1 -- 172.21.15.84:0/1507323632 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d70028850 con 0x55f3ac70a230 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.308+0000 7f9d64ff9700 1 -- 172.21.15.84:0/1507323632 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70010d60 con 0x55f3ac70a230 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/1507323632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 msgr2=0x55f3ac711440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/1507323632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 0x55f3ac711440 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f9d70005d60 tx=0x7f9d7000af90 comp rx=0 tx=0).stop 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/1507323632 shutdown_connections 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/1507323632 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 0x55f3ac711440 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/1507323632 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac715a00 0x55f3ac72ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/1507323632 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac70ab50 0x55f3ac721250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/1507323632 >> 172.21.15.84:0/1507323632 conn(0x55f3ac637ed0 msgr2=0x55f3ac710d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:24.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/1507323632 shutdown_connections 2024-04-26T18:38:24.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/1507323632 wait complete. 2024-04-26T18:38:24.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 -- start start 2024-04-26T18:38:24.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 Processor -- start 2024-04-26T18:38:24.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 0x55f3ac719cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70ab50 0x55f3ac71a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 0x55f3ac783650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801b12a0 con 0x55f3ac70a230 2024-04-26T18:38:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801b0bd0 con 0x55f3ac715a00 2024-04-26T18:38:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9d801addf0 con 0x55f3ac70ab50 2024-04-26T18:38:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.309+0000 7f9d8fcd1b80 10 client.0 _ll_register_callbacks cb 0x55f3ac5f1710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d7effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 0x55f3ac719cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d7effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 0x55f3ac719cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34392/0 (socket says 172.21.15.84:34392) 2024-04-26T18:38:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d7effd700 1 -- 172.21.15.84:0/2787195732 learned_addr learned my addr 172.21.15.84:0/2787195732 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70ab50 0x55f3ac71a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d7effd700 1 -- 172.21.15.84:0/2787195732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70ab50 msgr2=0x55f3ac71a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d7effd700 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70ab50 0x55f3ac71a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d7f7fe700 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 0x55f3ac783650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d7effd700 1 -- 172.21.15.84:0/2787195732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 msgr2=0x55f3ac783650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:24.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d7effd700 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 0x55f3ac783650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d7effd700 1 -- 172.21.15.84:0/2787195732 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ac783d50 con 0x55f3ac70a230 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d7f7fe700 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 0x55f3ac783650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d7effd700 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 0x55f3ac719cf0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000f010 tx=0x7f9d70010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58324c3ebb8fa847 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d62ff5700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d62ff5700 1 -- 172.21.15.84:0/2787195732 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d70034070 con 0x55f3ac70a230 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d62ff5700 1 -- 172.21.15.84:0/2787195732 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f9d70010be0 con 0x55f3ac70a230 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2787195732 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d70028850 con 0x55f3ac70a230 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d62ff5700 1 -- 172.21.15.84:0/2787195732 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9d70028670 con 0x55f3ac70a230 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d8fcd1b80 10 client.16683 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2787195732 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3ac78c140 con 0x55f3ac70a230 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.310+0000 7f9d62ff5700 1 -- 172.21.15.84:0/2787195732 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d70028850 con 0x55f3ac70a230 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.311+0000 7f9d62ff5700 1 -- 172.21.15.84:0/2787195732 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f9d70030410 con 0x55f3ac70a230 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.311+0000 7f9d8fcd1b80 10 client.16683 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:24.315 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.311+0000 7f9d8fcd1b80 10 client.16683 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:38:24.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.311+0000 7f9d8fcd1b80 20 client.16683 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:24.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.311+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f3ac78c7d0 0x55f3ac78ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:24.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.311+0000 7f9d8fcd1b80 4 client.16683 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.2, "await_for": 42.0}] 2024-04-26T18:38:24.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.311+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2787195732 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.2, "await_for": 42.0}) -- 0x55f3ac665350 con 0x55f3ac78c7d0 2024-04-26T18:38:24.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.311+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f3ac78c7d0 0x55f3ac78ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:24.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.311+0000 7f9d7e7fc700 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f3ac78c7d0 0x55f3ac78ebf0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2bf687ffdd16fb9 server_cookie=c9615c91088653f8 in_seq=0 out_seq=0 2024-04-26T18:38:24.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.311+0000 7f9d62ff5700 10 client.16683 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d62ff5700 1 -- 172.21.15.84:0/2787195732 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f3ac665350 con 0x55f3ac78c7d0 2024-04-26T18:38:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d62ff5700 10 client.16683 handle_command_reply: tid=0 2024-04-26T18:38:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d8fcd1b80 1 client.16683 shutdown 2024-04-26T18:38:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2787195732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 msgr2=0x55f3ac719cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 0x55f3ac719cf0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000f010 tx=0x7f9d70010660 comp rx=0 tx=0).stop 2024-04-26T18:38:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2787195732 shutdown_connections 2024-04-26T18:38:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f3ac78c7d0 0x55f3ac78ebf0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f3ac70a230 0x55f3ac719cf0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f3ac715a00 0x55f3ac783650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d8fcd1b80 1 --2- 172.21.15.84:0/2787195732 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f3ac70ab50 0x55f3ac71a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2787195732 >> 172.21.15.84:0/2787195732 conn(0x55f3ac637ed0 msgr2=0x55f3ac7115c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d62ff5700 0 client.16683 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:24.325 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2787195732 shutdown_connections 2024-04-26T18:38:24.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d8fcd1b80 1 -- 172.21.15.84:0/2787195732 wait complete. 2024-04-26T18:38:24.326 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.324+0000 7f9d8fcd1b80 20 client.16683 trim_cache size 0 max 16384 2024-04-26T18:38:24.326 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:38:24.326 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:38:24.326 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:38:24.326 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 30, 2024-04-26T18:38:24.326 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:38:24.326 INFO:teuthology.orchestra.run.smithi084.stdout: "a8dba540": { 2024-04-26T18:38:24.326 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 30, 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 81.2, 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:38:24.327 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:24.328 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:24.328 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:24.328 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:24.328 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:24.328 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:38:24.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.325+0000 7f9d8543f700 1 -- 172.21.15.84:0/3537655107 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9d680a3e50 msgr2=0x7f9d680a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:24.328 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.325+0000 7f9d8543f700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9d680a3e50 0x7f9d680a6310 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f9d70005ea0 tx=0x7f9d70005e30 comp rx=0 tx=0).stop 2024-04-26T18:38:24.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.325+0000 7f9d8543f700 1 -- 172.21.15.84:0/3537655107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d801030c0 msgr2=0x7f9d801046f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:24.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.325+0000 7f9d8543f700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d801030c0 0x7f9d801046f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f9d74003ac0 tx=0x7f9d74003af0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.325+0000 7f9d8543f700 1 -- 172.21.15.84:0/3537655107 shutdown_connections 2024-04-26T18:38:24.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.325+0000 7f9d8543f700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9d680a3e50 0x7f9d680a6310 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.325+0000 7f9d8543f700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9d801084c0 0x7f9d80104c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.325+0000 7f9d8543f700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9d80102700 0x7f9d801041b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.325+0000 7f9d8543f700 1 --2- 172.21.15.84:0/3537655107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9d801030c0 0x7f9d801046f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:24.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.325+0000 7f9d8543f700 1 -- 172.21.15.84:0/3537655107 >> 172.21.15.84:0/3537655107 conn(0x7f9d800792d0 msgr2=0x7f9d801062f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:24.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.325+0000 7f9d8543f700 1 -- 172.21.15.84:0/3537655107 shutdown_connections 2024-04-26T18:38:24.329 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:24.325+0000 7f9d8543f700 1 -- 172.21.15.84:0/3537655107 wait complete. 2024-04-26T18:38:24.980 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'a8dba540', quiesce took 5.7 seconds. Will release after: 21.2 2024-04-26T18:38:25.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:25 smithi084 ceph-mon[29141]: pgmap v326: 97 pgs: 97 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-26T18:38:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:25 smithi103 ceph-mon[32727]: pgmap v326: 97 pgs: 97 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-26T18:38:25.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:25 smithi174 ceph-mon[32705]: pgmap v326: 97 pgs: 97 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-26T18:38:27.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:27 smithi084 ceph-mon[29141]: pgmap v327: 97 pgs: 97 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-26T18:38:27.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:27 smithi103 ceph-mon[32727]: pgmap v327: 97 pgs: 97 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-26T18:38:27.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:27 smithi174 ceph-mon[32705]: pgmap v327: 97 pgs: 97 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 15 op/s 2024-04-26T18:38:29.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:29 smithi084 ceph-mon[29141]: pgmap v328: 97 pgs: 97 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-26T18:38:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:29 smithi103 ceph-mon[32727]: pgmap v328: 97 pgs: 97 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-26T18:38:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:29 smithi174 ceph-mon[32705]: pgmap v328: 97 pgs: 97 active+clean; 4.7 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 27 KiB/s wr, 14 op/s 2024-04-26T18:38:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.958+0000 7fcaa8437640 1 -- 172.21.15.84:0/962868704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcaa0102940 msgr2=0x7fcaa0102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.958+0000 7fcaa8437640 1 --2- 172.21.15.84:0/962868704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcaa0102940 0x7fcaa0102dc0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fca90009930 tx=0x7fca9002a420 comp rx=0 tx=0).stop 2024-04-26T18:38:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.958+0000 7fcaa8437640 1 -- 172.21.15.84:0/962868704 shutdown_connections 2024-04-26T18:38:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.958+0000 7fcaa8437640 1 --2- 172.21.15.84:0/962868704 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa0103300 0x7fcaa010f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.958+0000 7fcaa8437640 1 --2- 172.21.15.84:0/962868704 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcaa0102940 0x7fcaa0102dc0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.958+0000 7fcaa8437640 1 --2- 172.21.15.84:0/962868704 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa0108950 0x7fcaa0108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.958+0000 7fcaa8437640 1 -- 172.21.15.84:0/962868704 >> 172.21.15.84:0/962868704 conn(0x7fcaa0079dc0 msgr2=0x7fcaa007a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:30.961 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.958+0000 7fcaa8437640 1 -- 172.21.15.84:0/962868704 shutdown_connections 2024-04-26T18:38:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.958+0000 7fcaa8437640 1 -- 172.21.15.84:0/962868704 wait complete. 2024-04-26T18:38:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.959+0000 7fcaa8437640 1 Processor -- start 2024-04-26T18:38:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.959+0000 7fcaa8437640 1 -- start start 2024-04-26T18:38:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.959+0000 7fcaa8437640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa0102940 0x7fcaa0199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.959+0000 7fcaa8437640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa0103300 0x7fcaa01996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.959+0000 7fcaa8437640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcaa0108950 0x7fcaa019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.959+0000 7fcaa8437640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0112c30 con 0x7fcaa0108950 2024-04-26T18:38:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.959+0000 7fcaa8437640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0112ab0 con 0x7fcaa0102940 2024-04-26T18:38:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.959+0000 7fcaa8437640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0112db0 con 0x7fcaa0103300 2024-04-26T18:38:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.959+0000 7fcaa69d4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcaa0108950 0x7fcaa019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.959+0000 7fcaa69d4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcaa0108950 0x7fcaa019d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51366/0 (socket says 172.21.15.84:51366) 2024-04-26T18:38:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.959+0000 7fcaa69d4640 1 -- 172.21.15.84:0/3209553671 learned_addr learned my addr 172.21.15.84:0/3209553671 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:30.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.959+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa0103300 0x7fcaa01996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.959+0000 7fcaa61d3640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa0102940 0x7fcaa0199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.960+0000 7fcaa69d4640 1 -- 172.21.15.84:0/3209553671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa0103300 msgr2=0x7fcaa01996b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.960+0000 7fcaa69d4640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa0103300 0x7fcaa01996b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.960+0000 7fcaa69d4640 1 -- 172.21.15.84:0/3209553671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa0102940 msgr2=0x7fcaa0199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.960+0000 7fcaa69d4640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa0102940 0x7fcaa0199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.960+0000 7fcaa69d4640 1 -- 172.21.15.84:0/3209553671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa019e110 con 0x7fcaa0108950 2024-04-26T18:38:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.960+0000 7fcaa61d3640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa0102940 0x7fcaa0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:38:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.960+0000 7fcaa69d4640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcaa0108950 0x7fcaa019d9e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fca9800b9e0 tx=0x7fca9800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.960+0000 7fca977fe640 1 -- 172.21.15.84:0/3209553671 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98019070 con 0x7fcaa0108950 2024-04-26T18:38:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.960+0000 7fca977fe640 1 -- 172.21.15.84:0/3209553671 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca98014440 con 0x7fcaa0108950 2024-04-26T18:38:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.960+0000 7fcaa8437640 1 -- 172.21.15.84:0/3209553671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa019e400 con 0x7fcaa0108950 2024-04-26T18:38:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.960+0000 7fca977fe640 1 -- 172.21.15.84:0/3209553671 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca980075d0 con 0x7fcaa0108950 2024-04-26T18:38:30.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.960+0000 7fcaa8437640 1 -- 172.21.15.84:0/3209553671 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa01a5c90 con 0x7fcaa0108950 2024-04-26T18:38:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.962+0000 7fca977fe640 1 -- 172.21.15.84:0/3209553671 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fca9801c070 con 0x7fcaa0108950 2024-04-26T18:38:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.963+0000 7fcaab22c740 1 Processor -- start 2024-04-26T18:38:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.963+0000 7fcaab22c740 1 -- start start 2024-04-26T18:38:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.963+0000 7fcaab22c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 0x5558fc1f72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.963+0000 7fcaab22c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc1f7820 0x5558fc201ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.963+0000 7fcaab22c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc2020e0 0x5558fc2044d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.963+0000 7fcaab22c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5558fc15d460 con 0x5558fc1f7820 2024-04-26T18:38:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.963+0000 7fcaab22c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5558fc117590 con 0x5558fc1f6f00 2024-04-26T18:38:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.963+0000 7fcaab22c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5558fc159420 con 0x5558fc2020e0 2024-04-26T18:38:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.963+0000 7fca977fe640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fca8c0a3dc0 0x7fca8c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.971 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fcaa61d3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc1f7820 0x5558fc201ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fcaa61d3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc1f7820 0x5558fc201ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51368/0 (socket says 172.21.15.84:51368) 2024-04-26T18:38:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fcaa61d3640 1 -- 172.21.15.84:0/4068309506 learned_addr learned my addr 172.21.15.84:0/4068309506 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fcaa69d4640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fca8c0a3dc0 0x7fca8c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/4068309506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 0x5558fc1f72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/4068309506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc2020e0 0x5558fc2044d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fca977fe640 1 -- 172.21.15.84:0/3209553671 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7fca980cfd40 con 0x7fcaa0108950 2024-04-26T18:38:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fcaa61d3640 1 -- 172.21.15.84:0/4068309506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc2020e0 msgr2=0x5558fc2044d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fcaa61d3640 1 --2- 172.21.15.84:0/4068309506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc2020e0 0x5558fc2044d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fcaa61d3640 1 -- 172.21.15.84:0/4068309506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 msgr2=0x5558fc1f72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fcaa61d3640 1 --2- 172.21.15.84:0/4068309506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 0x5558fc1f72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.972 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fcaa61d3640 1 -- 172.21.15.84:0/4068309506 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558fc1240d0 con 0x5558fc1f7820 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fcaa69d4640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fca8c0a3dc0 0x7fca8c0a6280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fca980981c0 tx=0x7fca9800e040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fcaa61d3640 1 --2- 172.21.15.84:0/4068309506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc1f7820 0x5558fc201ba0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00bd40 tx=0x7fca9c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c9756a0951587a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fca94bfe640 1 -- 172.21.15.84:0/4068309506 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c015070 con 0x5558fc1f7820 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fca94bfe640 1 -- 172.21.15.84:0/4068309506 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca9c00be30 con 0x5558fc1f7820 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.964+0000 7fca94bfe640 1 -- 172.21.15.84:0/4068309506 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c0119a0 con 0x5558fc1f7820 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.965+0000 7fcaab22c740 1 -- 172.21.15.84:0/4068309506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc1f7820 msgr2=0x5558fc201ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.965+0000 7fcaab22c740 1 --2- 172.21.15.84:0/4068309506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc1f7820 0x5558fc201ba0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00bd40 tx=0x7fca9c00f730 comp rx=0 tx=0).stop 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.965+0000 7fcaab22c740 1 -- 172.21.15.84:0/4068309506 shutdown_connections 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.965+0000 7fcaab22c740 1 --2- 172.21.15.84:0/4068309506 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc2020e0 0x5558fc2044d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.965+0000 7fcaab22c740 1 --2- 172.21.15.84:0/4068309506 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc1f7820 0x5558fc201ba0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.965+0000 7fcaab22c740 1 --2- 172.21.15.84:0/4068309506 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 0x5558fc1f72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.965+0000 7fcaab22c740 1 -- 172.21.15.84:0/4068309506 >> 172.21.15.84:0/4068309506 conn(0x5558fc1f3410 msgr2=0x5558fc1f37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.965+0000 7fcaab22c740 1 -- 172.21.15.84:0/4068309506 shutdown_connections 2024-04-26T18:38:30.973 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.965+0000 7fcaab22c740 1 -- 172.21.15.84:0/4068309506 wait complete. 2024-04-26T18:38:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaab22c740 1 -- start start 2024-04-26T18:38:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaab22c740 1 Processor -- start 2024-04-26T18:38:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaab22c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 0x5558fc1ff0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaab22c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f7820 0x5558fc1ff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaab22c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaab22c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5558fc204d10 con 0x5558fc2020e0 2024-04-26T18:38:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaab22c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5558fc0e4b60 con 0x5558fc1f6f00 2024-04-26T18:38:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaab22c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5558fc204e90 con 0x5558fc1f7820 2024-04-26T18:38:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaab22c740 10 client.0 _ll_register_callbacks cb 0x5558fc1092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaa59d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaa59d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1ffb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51372/0 (socket says 172.21.15.84:51372) 2024-04-26T18:38:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaa59d2640 1 -- 172.21.15.84:0/1725353870 learned_addr learned my addr 172.21.15.84:0/1725353870 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:30.974 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaa61d3640 1 --2- 172.21.15.84:0/1725353870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f7820 0x5558fc1ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/1725353870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 0x5558fc1ff0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaa59d2640 1 -- 172.21.15.84:0/1725353870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f7820 msgr2=0x5558fc1ff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/1725353870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f7820 0x5558fc1ff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaa59d2640 1 -- 172.21.15.84:0/1725353870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 msgr2=0x5558fc1ff0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/1725353870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 0x5558fc1ff0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaa59d2640 1 -- 172.21.15.84:0/1725353870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558fbe6f210 con 0x5558fc2020e0 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/1725353870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1ffb40 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fca90009930 tx=0x7fca9002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4f28ad90b2c883c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.966+0000 7fca89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fca89ffb640 1 -- 172.21.15.84:0/1725353870 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90044070 con 0x5558fc2020e0 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fca89ffb640 1 -- 172.21.15.84:0/1725353870 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7fca900318b0 con 0x5558fc2020e0 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fcaab22c740 1 -- 172.21.15.84:0/1725353870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca9c00be30 con 0x5558fc2020e0 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fca89ffb640 1 -- 172.21.15.84:0/1725353870 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca90035040 con 0x5558fc2020e0 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fcaab22c740 10 client.16695 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:30.975 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fcaab22c740 1 -- 172.21.15.84:0/1725353870 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558fbf3b620 con 0x5558fc2020e0 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fca89ffb640 1 -- 172.21.15.84:0/1725353870 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca9c00be30 con 0x5558fc2020e0 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fca89ffb640 1 -- 172.21.15.84:0/1725353870 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fca9003d570 con 0x5558fc2020e0 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fcaab22c740 10 client.16695 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fcaab22c740 10 client.16695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fcaab22c740 20 client.16695 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fcaab22c740 1 --2- 172.21.15.84:0/1725353870 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5558fc2121e0 0x5558fc2145d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fcaab22c740 4 client.16695 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.967+0000 7fcaab22c740 1 -- 172.21.15.84:0/1725353870 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558fc16b070 con 0x5558fc2121e0 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.968+0000 7fcaa61d3640 1 --2- 172.21.15.84:0/1725353870 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5558fc2121e0 0x5558fc2145d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.968+0000 7fcaa61d3640 1 --2- 172.21.15.84:0/1725353870 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5558fc2121e0 0x5558fc2145d0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3f5e031d6cf2a3d server_cookie=252041b93cf553db in_seq=0 out_seq=0 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.968+0000 7fca89ffb640 10 client.16695 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fca89ffb640 1 -- 172.21.15.84:0/1725353870 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5558fc16b070 con 0x5558fc2121e0 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fca89ffb640 10 client.16695 handle_command_reply: tid=0 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fcaab22c740 1 client.16695 shutdown 2024-04-26T18:38:30.976 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fcaab22c740 1 -- 172.21.15.84:0/1725353870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 msgr2=0x5558fc1ffb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fcaab22c740 1 --2- 172.21.15.84:0/1725353870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1ffb40 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fca90009930 tx=0x7fca9002a420 comp rx=0 tx=0).stop 2024-04-26T18:38:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fcaab22c740 1 -- 172.21.15.84:0/1725353870 shutdown_connections 2024-04-26T18:38:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fcaab22c740 1 --2- 172.21.15.84:0/1725353870 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5558fc2121e0 0x5558fc2145d0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fcaab22c740 1 --2- 172.21.15.84:0/1725353870 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1ffb40 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fcaab22c740 1 --2- 172.21.15.84:0/1725353870 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f7820 0x5558fc1ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fca89ffb640 0 client.16695 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fcaab22c740 1 --2- 172.21.15.84:0/1725353870 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 0x5558fc1ff0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fcaab22c740 1 -- 172.21.15.84:0/1725353870 >> 172.21.15.84:0/1725353870 conn(0x5558fc1f3410 msgr2=0x5558fc1f9190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fcaab22c740 1 -- 172.21.15.84:0/1725353870 shutdown_connections 2024-04-26T18:38:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fcaab22c740 1 -- 172.21.15.84:0/1725353870 wait complete. 2024-04-26T18:38:30.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.969+0000 7fcaab22c740 20 client.16695 trim_cache size 0 max 16384 2024-04-26T18:38:30.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaab22c740 1 Processor -- start 2024-04-26T18:38:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaab22c740 1 -- start start 2024-04-26T18:38:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaab22c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 0x5558fc20a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaab22c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f7820 0x5558fc1f9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaab22c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1f2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaab22c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01ae5f0 con 0x5558fc2020e0 2024-04-26T18:38:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaab22c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01b0130 con 0x5558fc1f6f00 2024-04-26T18:38:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaab22c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01aecc0 con 0x5558fc1f7820 2024-04-26T18:38:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaa59d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1f2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaa59d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1f2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51378/0 (socket says 172.21.15.84:51378) 2024-04-26T18:38:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaa59d2640 1 -- 172.21.15.84:0/3237798984 learned_addr learned my addr 172.21.15.84:0/3237798984 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:30.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaa59d2640 1 -- 172.21.15.84:0/3237798984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f7820 msgr2=0x5558fc1f9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/3237798984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f7820 0x5558fc1f9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaa59d2640 1 -- 172.21.15.84:0/3237798984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 msgr2=0x5558fc20a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/3237798984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 0x5558fc20a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaa59d2640 1 -- 172.21.15.84:0/3237798984 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558fc204700 con 0x5558fc2020e0 2024-04-26T18:38:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.984+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/3237798984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1f2430 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fca90009900 tx=0x7fca90000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1123cb6247e6f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.985+0000 7fca8bfff640 1 -- 172.21.15.84:0/3237798984 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90044030 con 0x5558fc2020e0 2024-04-26T18:38:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.985+0000 7fca8bfff640 1 -- 172.21.15.84:0/3237798984 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca9003db80 con 0x5558fc2020e0 2024-04-26T18:38:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.985+0000 7fca8bfff640 1 -- 172.21.15.84:0/3237798984 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90031910 con 0x5558fc2020e0 2024-04-26T18:38:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.985+0000 7fcaab22c740 1 -- 172.21.15.84:0/3237798984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 msgr2=0x5558fc1f2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.985+0000 7fcaab22c740 1 --2- 172.21.15.84:0/3237798984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1f2430 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fca90009900 tx=0x7fca90000bc0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.985+0000 7fcaab22c740 1 -- 172.21.15.84:0/3237798984 shutdown_connections 2024-04-26T18:38:30.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.985+0000 7fcaab22c740 1 --2- 172.21.15.84:0/3237798984 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1f2430 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.985+0000 7fcaab22c740 1 --2- 172.21.15.84:0/3237798984 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f7820 0x5558fc1f9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.985+0000 7fcaab22c740 1 --2- 172.21.15.84:0/3237798984 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f6f00 0x5558fc20a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.985+0000 7fcaab22c740 1 -- 172.21.15.84:0/3237798984 >> 172.21.15.84:0/3237798984 conn(0x5558fc1f3410 msgr2=0x5558fc1f1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.985+0000 7fcaab22c740 1 -- 172.21.15.84:0/3237798984 shutdown_connections 2024-04-26T18:38:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.985+0000 7fcaab22c740 1 -- 172.21.15.84:0/3237798984 wait complete. 2024-04-26T18:38:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.985+0000 7fcaab22c740 1 -- start start 2024-04-26T18:38:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaab22c740 1 Processor -- start 2024-04-26T18:38:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaab22c740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f6f00 0x5558fc1fc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaab22c740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f7820 0x5558fc1fcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaab22c740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1fb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaab22c740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01afa60 con 0x5558fc2020e0 2024-04-26T18:38:30.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaab22c740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01af390 con 0x5558fc1f7820 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaab22c740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01ac5f0 con 0x5558fc1f6f00 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaab22c740 10 client.0 _ll_register_callbacks cb 0x7fca90031ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaa59d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1fb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaa59d2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1fb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51382/0 (socket says 172.21.15.84:51382) 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaa59d2640 1 -- 172.21.15.84:0/469942389 learned_addr learned my addr 172.21.15.84:0/469942389 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f6f00 0x5558fc1fc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaa61d3640 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f7820 0x5558fc1fcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaa59d2640 1 -- 172.21.15.84:0/469942389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f6f00 msgr2=0x5558fc1fc730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f6f00 0x5558fc1fc730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaa59d2640 1 -- 172.21.15.84:0/469942389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f7820 msgr2=0x5558fc1fcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f7820 0x5558fc1fcc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaa59d2640 1 -- 172.21.15.84:0/469942389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558fc1f9e50 con 0x5558fc2020e0 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaa61d3640 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f7820 0x5558fc1fcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:38:30.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fcaa59d2640 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1fb390 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fca90009340 tx=0x7fca900326a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e85b14fdc9cdf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fca89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.986+0000 7fca89ffb640 1 -- 172.21.15.84:0/469942389 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90044070 con 0x5558fc2020e0 2024-04-26T18:38:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fca89ffb640 1 -- 172.21.15.84:0/469942389 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7fca900316b0 con 0x5558fc2020e0 2024-04-26T18:38:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fcaab22c740 1 -- 172.21.15.84:0/469942389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca9003db80 con 0x5558fc2020e0 2024-04-26T18:38:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fca89ffb640 1 -- 172.21.15.84:0/469942389 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca9003d720 con 0x5558fc2020e0 2024-04-26T18:38:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fcaab22c740 10 client.16701 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fcaab22c740 1 -- 172.21.15.84:0/469942389 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5558fc27d160 con 0x5558fc2020e0 2024-04-26T18:38:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fca89ffb640 1 -- 172.21.15.84:0/469942389 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca9003db80 con 0x5558fc2020e0 2024-04-26T18:38:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fca89ffb640 1 -- 172.21.15.84:0/469942389 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fca90049610 con 0x5558fc2020e0 2024-04-26T18:38:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fcaab22c740 10 client.16701 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fcaab22c740 10 client.16701 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:38:30.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fcaab22c740 20 client.16701 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fcaab22c740 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5558fc27d7c0 0x5558fc27fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fcaab22c740 4 client.16701 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:38:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fcaab22c740 1 -- 172.21.15.84:0/469942389 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5558fc164ee0 con 0x5558fc27d7c0 2024-04-26T18:38:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.987+0000 7fcaa61d3640 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5558fc27d7c0 0x5558fc27fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.988+0000 7fca89ffb640 10 client.16701 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:30.995 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:30.988+0000 7fcaa61d3640 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5558fc27d7c0 0x5558fc27fbb0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca3fa791092a098 server_cookie=7c8426321dc0aa11 in_seq=0 out_seq=0 2024-04-26T18:38:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fca89ffb640 1 -- 172.21.15.84:0/469942389 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5558fc164ee0 con 0x5558fc27d7c0 2024-04-26T18:38:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fca89ffb640 10 client.16701 handle_command_reply: tid=0 2024-04-26T18:38:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fcaab22c740 1 client.16701 shutdown 2024-04-26T18:38:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fcaab22c740 1 -- 172.21.15.84:0/469942389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 msgr2=0x5558fc1fb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fcaab22c740 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1fb390 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fca90009340 tx=0x7fca900326a0 comp rx=0 tx=0).stop 2024-04-26T18:38:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fcaab22c740 1 -- 172.21.15.84:0/469942389 shutdown_connections 2024-04-26T18:38:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fcaab22c740 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5558fc27d7c0 0x5558fc27fbb0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fcaab22c740 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5558fc2020e0 0x5558fc1fb390 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:31.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fcaab22c740 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5558fc1f7820 0x5558fc1fcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fca89ffb640 0 client.16701 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fcaab22c740 1 --2- 172.21.15.84:0/469942389 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5558fc1f6f00 0x5558fc1fc730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fcaab22c740 1 -- 172.21.15.84:0/469942389 >> 172.21.15.84:0/469942389 conn(0x5558fc1f3410 msgr2=0x5558fc1f25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fcaab22c740 1 -- 172.21.15.84:0/469942389 shutdown_connections 2024-04-26T18:38:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.000+0000 7fcaab22c740 1 -- 172.21.15.84:0/469942389 wait complete. 2024-04-26T18:38:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.001+0000 7fcaab22c740 20 client.16701 trim_cache size 0 max 16384 2024-04-26T18:38:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:38:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:38:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:38:31.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:38:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.001+0000 7fcaa8437640 1 -- 172.21.15.84:0/3209553671 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fca8c0a3dc0 msgr2=0x7fca8c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.001+0000 7fcaa8437640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fca8c0a3dc0 0x7fca8c0a6280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fca980981c0 tx=0x7fca9800e040 comp rx=0 tx=0).stop 2024-04-26T18:38:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.001+0000 7fcaa8437640 1 -- 172.21.15.84:0/3209553671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcaa0108950 msgr2=0x7fcaa019d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.001+0000 7fcaa8437640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcaa0108950 0x7fcaa019d9e0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fca9800b9e0 tx=0x7fca9800bea0 comp rx=0 tx=0).stop 2024-04-26T18:38:31.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.001+0000 7fcaa8437640 1 -- 172.21.15.84:0/3209553671 shutdown_connections 2024-04-26T18:38:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.001+0000 7fcaa8437640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fca8c0a3dc0 0x7fca8c0a6280 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.001+0000 7fcaa8437640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcaa0108950 0x7fcaa019d9e0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.001+0000 7fcaa8437640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcaa0103300 0x7fcaa01996b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.001+0000 7fcaa8437640 1 --2- 172.21.15.84:0/3209553671 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcaa0102940 0x7fcaa0199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.001+0000 7fcaa8437640 1 -- 172.21.15.84:0/3209553671 >> 172.21.15.84:0/3209553671 conn(0x7fcaa0079dc0 msgr2=0x7fcaa0100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.001+0000 7fcaa8437640 1 -- 172.21.15.84:0/3209553671 shutdown_connections 2024-04-26T18:38:31.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:31.001+0000 7fcaa8437640 1 -- 172.21.15.84:0/3209553671 wait complete. 2024-04-26T18:38:31.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:31 smithi084 ceph-mon[29141]: pgmap v329: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-26T18:38:31.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:31 smithi103 ceph-mon[32727]: pgmap v329: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-26T18:38:31.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:31 smithi174 ceph-mon[32705]: pgmap v329: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 28 KiB/s wr, 17 op/s 2024-04-26T18:38:33.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:33 smithi084 ceph-mon[29141]: pgmap v330: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-04-26T18:38:33.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:33 smithi103 ceph-mon[32727]: pgmap v330: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-04-26T18:38:33.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:33 smithi174 ceph-mon[32705]: pgmap v330: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 11 op/s 2024-04-26T18:38:35.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:35 smithi084 ceph-mon[29141]: pgmap v331: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 17 op/s 2024-04-26T18:38:35.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:35 smithi103 ceph-mon[32727]: pgmap v331: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 17 op/s 2024-04-26T18:38:35.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:35 smithi174 ceph-mon[32705]: pgmap v331: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 26 KiB/s wr, 17 op/s 2024-04-26T18:38:37.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:37 smithi084 ceph-mon[29141]: pgmap v332: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-04-26T18:38:37.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:37 smithi103 ceph-mon[32727]: pgmap v332: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-04-26T18:38:37.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:37 smithi174 ceph-mon[32705]: pgmap v332: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-04-26T18:38:39.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:39 smithi084 ceph-mon[29141]: pgmap v333: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-04-26T18:38:39.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:39 smithi103 ceph-mon[32727]: pgmap v333: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-04-26T18:38:39.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:39 smithi174 ceph-mon[32705]: pgmap v333: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.9 KiB/s wr, 11 op/s 2024-04-26T18:38:41.785 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:41 smithi084 ceph-mon[29141]: pgmap v334: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-04-26T18:38:41.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:41 smithi103 ceph-mon[32727]: pgmap v334: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-04-26T18:38:41.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:41 smithi174 ceph-mon[32705]: pgmap v334: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.9 KiB/s wr, 15 op/s 2024-04-26T18:38:43.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:43 smithi084 ceph-mon[29141]: pgmap v335: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-26T18:38:43.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:43 smithi103 ceph-mon[32727]: pgmap v335: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-26T18:38:43.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:43 smithi174 ceph-mon[32705]: pgmap v335: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 4.6 KiB/s wr, 10 op/s 2024-04-26T18:38:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.097+0000 7f692ee93640 1 -- 172.21.15.84:0/1329353291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6928103be0 msgr2=0x7f6928103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.097+0000 7f692ee93640 1 --2- 172.21.15.84:0/1329353291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6928103be0 0x7f6928103fc0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f691c009900 tx=0x7f691c02a480 comp rx=0 tx=0).stop 2024-04-26T18:38:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.097+0000 7f692ee93640 1 -- 172.21.15.84:0/1329353291 shutdown_connections 2024-04-26T18:38:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.097+0000 7f692ee93640 1 --2- 172.21.15.84:0/1329353291 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f692810cf20 0x7f692810f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.097+0000 7f692ee93640 1 --2- 172.21.15.84:0/1329353291 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6928104500 0x7f692810c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.097+0000 7f692ee93640 1 --2- 172.21.15.84:0/1329353291 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6928103be0 0x7f6928103fc0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.097+0000 7f692ee93640 1 -- 172.21.15.84:0/1329353291 >> 172.21.15.84:0/1329353291 conn(0x7f69280789b0 msgr2=0x7f6928078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.097+0000 7f692ee93640 1 -- 172.21.15.84:0/1329353291 shutdown_connections 2024-04-26T18:38:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.097+0000 7f692ee93640 1 -- 172.21.15.84:0/1329353291 wait complete. 2024-04-26T18:38:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692ee93640 1 Processor -- start 2024-04-26T18:38:44.100 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692ee93640 1 -- start start 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692ee93640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6928103be0 0x7f69281990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692ee93640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6928104500 0x7f69281995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692ee93640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f692810cf20 0x7f692819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692ee93640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6928112ba0 con 0x7f692810cf20 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692ee93640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6928112a20 con 0x7f6928104500 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692ee93640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6928112d20 con 0x7f6928103be0 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692d430640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f692810cf20 0x7f692819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692d430640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f692810cf20 0x7f692819d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35486/0 (socket says 172.21.15.84:35486) 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692d430640 1 -- 172.21.15.84:0/1561240563 learned_addr learned my addr 172.21.15.84:0/1561240563 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692cc2f640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6928103be0 0x7f69281990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f6927fff640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6928104500 0x7f69281995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692d430640 1 -- 172.21.15.84:0/1561240563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6928103be0 msgr2=0x7f69281990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692d430640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6928103be0 0x7f69281990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.101 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692d430640 1 -- 172.21.15.84:0/1561240563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6928104500 msgr2=0x7f69281995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.098+0000 7f692d430640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6928104500 0x7f69281995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.099+0000 7f692d430640 1 -- 172.21.15.84:0/1561240563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f692819e090 con 0x7f692810cf20 2024-04-26T18:38:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.099+0000 7f692cc2f640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6928103be0 0x7f69281990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:38:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.099+0000 7f692d430640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f692810cf20 0x7f692819d960 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f692000eaa0 tx=0x7f692000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.099+0000 7f6925ffb640 1 -- 172.21.15.84:0/1561240563 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692000cdb0 con 0x7f692810cf20 2024-04-26T18:38:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.099+0000 7f692ee93640 1 -- 172.21.15.84:0/1561240563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f692819e380 con 0x7f692810cf20 2024-04-26T18:38:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.099+0000 7f6925ffb640 1 -- 172.21.15.84:0/1561240563 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6920005020 con 0x7f692810cf20 2024-04-26T18:38:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.099+0000 7f6925ffb640 1 -- 172.21.15.84:0/1561240563 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69200116d0 con 0x7f692810cf20 2024-04-26T18:38:44.102 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.099+0000 7f692ee93640 1 -- 172.21.15.84:0/1561240563 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69281a5c10 con 0x7f692810cf20 2024-04-26T18:38:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.101+0000 7f6925ffb640 1 -- 172.21.15.84:0/1561240563 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f69200054a0 con 0x7f692810cf20 2024-04-26T18:38:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6931bad740 1 Processor -- start 2024-04-26T18:38:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6931bad740 1 -- start start 2024-04-26T18:38:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6925ffb640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f69180a3dc0 0x7f69180a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6931bad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 0x55e5802241e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6931bad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580224720 0x55e580232fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6931bad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5802334f0 0x55e5802358e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6931bad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e58018a460 con 0x55e580224720 2024-04-26T18:38:44.109 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6925ffb640 1 -- 172.21.15.84:0/1561240563 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f69200d0a20 con 0x7f692810cf20 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6931bad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e580144590 con 0x55e580223e00 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6931bad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e580186420 con 0x55e5802334f0 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6927fff640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f69180a3dc0 0x7f69180a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6927fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580224720 0x55e580232fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6927fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580224720 0x55e580232fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35490/0 (socket says 172.21.15.84:35490) 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6927fff640 1 -- 172.21.15.84:0/237975111 learned_addr learned my addr 172.21.15.84:0/237975111 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f692cc2f640 1 --2- 172.21.15.84:0/237975111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 0x55e5802241e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f692d430640 1 --2- 172.21.15.84:0/237975111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5802334f0 0x55e5802358e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6927fff640 1 -- 172.21.15.84:0/237975111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5802334f0 msgr2=0x55e5802358e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6927fff640 1 --2- 172.21.15.84:0/237975111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5802334f0 0x55e5802358e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6927fff640 1 -- 172.21.15.84:0/237975111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 msgr2=0x55e5802241e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6927fff640 1 --2- 172.21.15.84:0/237975111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 0x55e5802241e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.102+0000 7f6927fff640 1 -- 172.21.15.84:0/237975111 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5801510d0 con 0x55e580224720 2024-04-26T18:38:44.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.103+0000 7f6927fff640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f69180a3dc0 0x7f69180a6280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6914013910 tx=0x7f691400bae0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.103+0000 7f6927fff640 1 --2- 172.21.15.84:0/237975111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580224720 0x55e580232fb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6914010c90 tx=0x7f691401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de40c4ef6382f032 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.103+0000 7f6912ffd640 1 -- 172.21.15.84:0/237975111 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914021070 con 0x55e580224720 2024-04-26T18:38:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.103+0000 7f6912ffd640 1 -- 172.21.15.84:0/237975111 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691400b360 con 0x55e580224720 2024-04-26T18:38:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.103+0000 7f6912ffd640 1 -- 172.21.15.84:0/237975111 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6914011b50 con 0x55e580224720 2024-04-26T18:38:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.103+0000 7f6931bad740 1 -- 172.21.15.84:0/237975111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580224720 msgr2=0x55e580232fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.103+0000 7f6931bad740 1 --2- 172.21.15.84:0/237975111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580224720 0x55e580232fb0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f6914010c90 tx=0x7f691401a000 comp rx=0 tx=0).stop 2024-04-26T18:38:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 -- 172.21.15.84:0/237975111 shutdown_connections 2024-04-26T18:38:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 --2- 172.21.15.84:0/237975111 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e5802334f0 0x55e5802358e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 --2- 172.21.15.84:0/237975111 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580224720 0x55e580232fb0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 --2- 172.21.15.84:0/237975111 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 0x55e5802241e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 -- 172.21.15.84:0/237975111 >> 172.21.15.84:0/237975111 conn(0x55e580220410 msgr2=0x55e5802207f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 -- 172.21.15.84:0/237975111 shutdown_connections 2024-04-26T18:38:44.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 -- 172.21.15.84:0/237975111 wait complete. 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 -- start start 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 Processor -- start 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580223e00 0x55e580230410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 0x55e580230950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5802334f0 0x55e580230e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e5802360c0 con 0x55e580223e00 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e580111b60 con 0x55e5802334f0 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e580236240 con 0x55e580224720 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.104+0000 7f6931bad740 10 client.0 _ll_register_callbacks cb 0x55e5801362b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f692cc2f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580223e00 0x55e580230410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f692cc2f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580223e00 0x55e580230410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35504/0 (socket says 172.21.15.84:35504) 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f692cc2f640 1 -- 172.21.15.84:0/2431631265 learned_addr learned my addr 172.21.15.84:0/2431631265 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f692d430640 1 --2- 172.21.15.84:0/2431631265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5802334f0 0x55e580230e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f6927fff640 1 --2- 172.21.15.84:0/2431631265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 0x55e580230950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f692cc2f640 1 -- 172.21.15.84:0/2431631265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 msgr2=0x55e580230950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f692cc2f640 1 --2- 172.21.15.84:0/2431631265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 0x55e580230950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f692cc2f640 1 -- 172.21.15.84:0/2431631265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5802334f0 msgr2=0x55e580230e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f692cc2f640 1 --2- 172.21.15.84:0/2431631265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5802334f0 0x55e580230e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f692cc2f640 1 -- 172.21.15.84:0/2431631265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e57fe9c210 con 0x55e580223e00 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f692cc2f640 1 --2- 172.21.15.84:0/2431631265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580223e00 0x55e580230410 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f691c009900 tx=0x7f691c031950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e10b840ef1eb50aa server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f6907fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f6907fff640 1 -- 172.21.15.84:0/2431631265 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691c035040 con 0x55e580223e00 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f6907fff640 1 -- 172.21.15.84:0/2431631265 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f691c004380 con 0x55e580223e00 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f6931bad740 1 -- 172.21.15.84:0/2431631265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f691400b360 con 0x55e580223e00 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f6907fff640 1 -- 172.21.15.84:0/2431631265 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f691c005030 con 0x55e580223e00 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f6931bad740 10 client.16713 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.105+0000 7f6931bad740 1 -- 172.21.15.84:0/2431631265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e57ff68620 con 0x55e580223e00 2024-04-26T18:38:44.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.106+0000 7f6907fff640 1 -- 172.21.15.84:0/2431631265 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f691400b360 con 0x55e580223e00 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.106+0000 7f6907fff640 1 -- 172.21.15.84:0/2431631265 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f691c004c90 con 0x55e580223e00 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.106+0000 7f6931bad740 10 client.16713 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.106+0000 7f6931bad740 10 client.16713 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.106+0000 7f6931bad740 20 client.16713 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.106+0000 7f6931bad740 1 --2- 172.21.15.84:0/2431631265 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e580243680 0x55e580245a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.106+0000 7f6931bad740 4 client.16713 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.106+0000 7f6931bad740 1 -- 172.21.15.84:0/2431631265 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e580198070 con 0x55e580243680 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.106+0000 7f6927fff640 1 --2- 172.21.15.84:0/2431631265 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e580243680 0x55e580245a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.106+0000 7f6907fff640 10 client.16713 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.106+0000 7f6927fff640 1 --2- 172.21.15.84:0/2431631265 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e580243680 0x55e580245a70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16b72603c965621c server_cookie=98e6db4bd4f5b05a in_seq=0 out_seq=0 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.107+0000 7f6907fff640 1 -- 172.21.15.84:0/2431631265 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e580198070 con 0x55e580243680 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.107+0000 7f6907fff640 10 client.16713 handle_command_reply: tid=0 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.107+0000 7f6931bad740 1 client.16713 shutdown 2024-04-26T18:38:44.114 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.107+0000 7f6931bad740 1 -- 172.21.15.84:0/2431631265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580223e00 msgr2=0x55e580230410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.107+0000 7f6931bad740 1 --2- 172.21.15.84:0/2431631265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580223e00 0x55e580230410 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f691c009900 tx=0x7f691c031950 comp rx=0 tx=0).stop 2024-04-26T18:38:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.107+0000 7f6931bad740 1 -- 172.21.15.84:0/2431631265 shutdown_connections 2024-04-26T18:38:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.107+0000 7f6931bad740 1 --2- 172.21.15.84:0/2431631265 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e580243680 0x55e580245a70 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.107+0000 7f6931bad740 1 --2- 172.21.15.84:0/2431631265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e5802334f0 0x55e580230e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.107+0000 7f6931bad740 1 --2- 172.21.15.84:0/2431631265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 0x55e580230950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.107+0000 7f6931bad740 1 --2- 172.21.15.84:0/2431631265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e580223e00 0x55e580230410 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.107+0000 7f6907fff640 0 client.16713 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.107+0000 7f6931bad740 1 -- 172.21.15.84:0/2431631265 >> 172.21.15.84:0/2431631265 conn(0x55e580220410 msgr2=0x55e580226280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.108+0000 7f6931bad740 1 -- 172.21.15.84:0/2431631265 shutdown_connections 2024-04-26T18:38:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.108+0000 7f6931bad740 1 -- 172.21.15.84:0/2431631265 wait complete. 2024-04-26T18:38:44.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.108+0000 7f6931bad740 20 client.16713 trim_cache size 0 max 16384 2024-04-26T18:38:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.122+0000 7f6931bad740 1 Processor -- start 2024-04-26T18:38:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.122+0000 7f6931bad740 1 -- start start 2024-04-26T18:38:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f6931bad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 0x55e58023ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f6931bad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 0x55e58021f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f6931bad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 0x55e58024b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f6931bad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69281ac490 con 0x55e5802334f0 2024-04-26T18:38:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f6931bad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69281adfd0 con 0x55e580223e00 2024-04-26T18:38:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f6931bad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69281acb60 con 0x55e580224720 2024-04-26T18:38:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f692d430640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 0x55e58024b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f692d430640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 0x55e58024b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35520/0 (socket says 172.21.15.84:35520) 2024-04-26T18:38:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f692d430640 1 -- 172.21.15.84:0/296745058 learned_addr learned my addr 172.21.15.84:0/296745058 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f692d430640 1 -- 172.21.15.84:0/296745058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 msgr2=0x55e58021f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:44.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f692d430640 1 --2- 172.21.15.84:0/296745058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 0x55e58021f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f692d430640 1 -- 172.21.15.84:0/296745058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 msgr2=0x55e58023ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f692d430640 1 --2- 172.21.15.84:0/296745058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 0x55e58023ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f692d430640 1 -- 172.21.15.84:0/296745058 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e580224550 con 0x55e5802334f0 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f692d430640 1 --2- 172.21.15.84:0/296745058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 0x55e58024b360 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f692009c910 tx=0x7f69200057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b969a16ba625b899 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f69127fc640 1 -- 172.21.15.84:0/296745058 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692001f040 con 0x55e5802334f0 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f69127fc640 1 -- 172.21.15.84:0/296745058 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69200a55f0 con 0x55e5802334f0 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.123+0000 7f69127fc640 1 -- 172.21.15.84:0/296745058 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69200870b0 con 0x55e5802334f0 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.124+0000 7f6931bad740 1 -- 172.21.15.84:0/296745058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 msgr2=0x55e58024b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.124+0000 7f6931bad740 1 --2- 172.21.15.84:0/296745058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 0x55e58024b360 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f692009c910 tx=0x7f69200057b0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.124+0000 7f6931bad740 1 -- 172.21.15.84:0/296745058 shutdown_connections 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.124+0000 7f6931bad740 1 --2- 172.21.15.84:0/296745058 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 0x55e58024b360 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.124+0000 7f6931bad740 1 --2- 172.21.15.84:0/296745058 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 0x55e58021f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.124+0000 7f6931bad740 1 --2- 172.21.15.84:0/296745058 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 0x55e58023ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.124+0000 7f6931bad740 1 -- 172.21.15.84:0/296745058 >> 172.21.15.84:0/296745058 conn(0x55e580220410 msgr2=0x55e58021ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:44.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.124+0000 7f6931bad740 1 -- 172.21.15.84:0/296745058 shutdown_connections 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.124+0000 7f6931bad740 1 -- 172.21.15.84:0/296745058 wait complete. 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.124+0000 7f6931bad740 1 -- start start 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.124+0000 7f6931bad740 1 Processor -- start 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6931bad740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 0x55e580237310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6931bad740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 0x55e580237850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6931bad740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 0x55e5802a8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6931bad740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f69281ad900 con 0x55e5802334f0 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6931bad740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f69281ad230 con 0x55e580223e00 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6931bad740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f69281aa490 con 0x55e580224720 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6931bad740 10 client.0 _ll_register_callbacks cb 0x7f691c032b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f692d430640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 0x55e5802a8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f692d430640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 0x55e5802a8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35532/0 (socket says 172.21.15.84:35532) 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f692d430640 1 -- 172.21.15.84:0/3920862747 learned_addr learned my addr 172.21.15.84:0/3920862747 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6927fff640 1 --2- 172.21.15.84:0/3920862747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 0x55e580237850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f692cc2f640 1 --2- 172.21.15.84:0/3920862747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 0x55e580237310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f692d430640 1 -- 172.21.15.84:0/3920862747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 msgr2=0x55e580237850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f692d430640 1 --2- 172.21.15.84:0/3920862747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 0x55e580237850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f692d430640 1 -- 172.21.15.84:0/3920862747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 msgr2=0x55e580237310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f692d430640 1 --2- 172.21.15.84:0/3920862747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 0x55e580237310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f692d430640 1 -- 172.21.15.84:0/3920862747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5802a9110 con 0x55e5802334f0 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f692d430640 1 --2- 172.21.15.84:0/3920862747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 0x55e5802a8bd0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f692009cb40 tx=0x7f6920005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7489db08baf0ab63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6907fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6907fff640 1 -- 172.21.15.84:0/3920862747 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f692001f070 con 0x55e5802334f0 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6907fff640 1 -- 172.21.15.84:0/3920862747 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f69200872c0 con 0x55e5802334f0 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6931bad740 1 -- 172.21.15.84:0/3920862747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f69200a55f0 con 0x55e5802334f0 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6907fff640 1 -- 172.21.15.84:0/3920862747 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6920067350 con 0x55e5802334f0 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6931bad740 10 client.16719 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.125+0000 7f6931bad740 1 -- 172.21.15.84:0/3920862747 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e5802aa2e0 con 0x55e5802334f0 2024-04-26T18:38:44.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.126+0000 7f6907fff640 1 -- 172.21.15.84:0/3920862747 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69200a55f0 con 0x55e5802334f0 2024-04-26T18:38:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.126+0000 7f6907fff640 1 -- 172.21.15.84:0/3920862747 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f692000f040 con 0x55e5802334f0 2024-04-26T18:38:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.126+0000 7f6931bad740 10 client.16719 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.126+0000 7f6931bad740 10 client.16719 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:38:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.126+0000 7f6931bad740 20 client.16719 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.126+0000 7f6931bad740 1 --2- 172.21.15.84:0/3920862747 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e5802aa950 0x55e5802acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.126+0000 7f6931bad740 4 client.16719 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:38:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.126+0000 7f6931bad740 1 -- 172.21.15.84:0/3920862747 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e580191ee0 con 0x55e5802aa950 2024-04-26T18:38:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.126+0000 7f692cc2f640 1 --2- 172.21.15.84:0/3920862747 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e5802aa950 0x55e5802acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.126+0000 7f6907fff640 10 client.16719 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:44.131 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.126+0000 7f692cc2f640 1 --2- 172.21.15.84:0/3920862747 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e5802aa950 0x55e5802acd40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7bc2cbd474c5ee0 server_cookie=80665d627f62b539 in_seq=0 out_seq=0 2024-04-26T18:38:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6907fff640 1 -- 172.21.15.84:0/3920862747 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e580191ee0 con 0x55e5802aa950 2024-04-26T18:38:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6907fff640 10 client.16719 handle_command_reply: tid=0 2024-04-26T18:38:44.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6931bad740 1 client.16719 shutdown 2024-04-26T18:38:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6931bad740 1 -- 172.21.15.84:0/3920862747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 msgr2=0x55e5802a8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6931bad740 1 --2- 172.21.15.84:0/3920862747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 0x55e5802a8bd0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f692009cb40 tx=0x7f6920005400 comp rx=0 tx=0).stop 2024-04-26T18:38:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6931bad740 1 -- 172.21.15.84:0/3920862747 shutdown_connections 2024-04-26T18:38:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6931bad740 1 --2- 172.21.15.84:0/3920862747 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e5802aa950 0x55e5802acd40 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6931bad740 1 --2- 172.21.15.84:0/3920862747 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e5802334f0 0x55e5802a8bd0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6931bad740 1 --2- 172.21.15.84:0/3920862747 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e580224720 0x55e580237850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6931bad740 1 --2- 172.21.15.84:0/3920862747 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e580223e00 0x55e580237310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6907fff640 0 client.16719 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6931bad740 1 -- 172.21.15.84:0/3920862747 >> 172.21.15.84:0/3920862747 conn(0x55e580220410 msgr2=0x55e5802303b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6931bad740 1 -- 172.21.15.84:0/3920862747 shutdown_connections 2024-04-26T18:38:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6931bad740 1 -- 172.21.15.84:0/3920862747 wait complete. 2024-04-26T18:38:44.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.136+0000 7f6931bad740 20 client.16719 trim_cache size 0 max 16384 2024-04-26T18:38:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:38:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:38:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:38:44.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:38:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.137+0000 7f692ee93640 1 -- 172.21.15.84:0/1561240563 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f69180a3dc0 msgr2=0x7f69180a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.137+0000 7f692ee93640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f69180a3dc0 0x7f69180a6280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6914013910 tx=0x7f691400bae0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.137+0000 7f692ee93640 1 -- 172.21.15.84:0/1561240563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f692810cf20 msgr2=0x7f692819d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.137+0000 7f692ee93640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f692810cf20 0x7f692819d960 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f692000eaa0 tx=0x7f692000ef60 comp rx=0 tx=0).stop 2024-04-26T18:38:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.137+0000 7f692ee93640 1 -- 172.21.15.84:0/1561240563 shutdown_connections 2024-04-26T18:38:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.137+0000 7f692ee93640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f69180a3dc0 0x7f69180a6280 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.137+0000 7f692ee93640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f692810cf20 0x7f692819d960 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.137+0000 7f692ee93640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6928104500 0x7f69281995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.137+0000 7f692ee93640 1 --2- 172.21.15.84:0/1561240563 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6928103be0 0x7f69281990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.137+0000 7f692ee93640 1 -- 172.21.15.84:0/1561240563 >> 172.21.15.84:0/1561240563 conn(0x7f69280789b0 msgr2=0x7f69280fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.137+0000 7f692ee93640 1 -- 172.21.15.84:0/1561240563 shutdown_connections 2024-04-26T18:38:44.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:44.137+0000 7f692ee93640 1 -- 172.21.15.84:0/1561240563 wait complete. 2024-04-26T18:38:45.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:45 smithi084 ceph-mon[29141]: pgmap v336: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.8 KiB/s wr, 15 op/s 2024-04-26T18:38:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:45 smithi103 ceph-mon[32727]: pgmap v336: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.8 KiB/s wr, 15 op/s 2024-04-26T18:38:45.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:45 smithi174 ceph-mon[32705]: pgmap v336: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 5.8 KiB/s wr, 15 op/s 2024-04-26T18:38:46.182 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:38:46.437 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:38:47.797 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:47 smithi084 ceph-mon[29141]: pgmap v337: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:38:47.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:47 smithi103 ceph-mon[32727]: pgmap v337: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:38:47.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:47 smithi174 ceph-mon[32705]: pgmap v337: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:38:48.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.175+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/3071025718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f406bad0 msgr2=0x7fa6f41102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:48.178 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.175+0000 7fa6fb5a8700 1 --2- 172.21.15.84:0/3071025718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f406bad0 0x7fa6f41102e0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00b0a0 tx=0x7fa6ec00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:38:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.175+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/3071025718 shutdown_connections 2024-04-26T18:38:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.175+0000 7fa6fb5a8700 1 --2- 172.21.15.84:0/3071025718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f406bad0 0x7fa6f41102e0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.175+0000 7fa6fb5a8700 1 --2- 172.21.15.84:0/3071025718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6f41094a0 0x7fa6f4109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.175+0000 7fa6fb5a8700 1 --2- 172.21.15.84:0/3071025718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f406b130 0x7fa6f406b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.175+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/3071025718 >> 172.21.15.84:0/3071025718 conn(0x7fa6f40792d0 msgr2=0x7fa6f40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.175+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/3071025718 shutdown_connections 2024-04-26T18:38:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.175+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/3071025718 wait complete. 2024-04-26T18:38:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.176+0000 7fa6fb5a8700 1 Processor -- start 2024-04-26T18:38:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.176+0000 7fa6fb5a8700 1 -- start start 2024-04-26T18:38:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.176+0000 7fa6fb5a8700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6f406b130 0x7fa6f419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.176+0000 7fa6fb5a8700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f406bad0 0x7fa6f419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:48.179 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.176+0000 7fa6fb5a8700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f41094a0 0x7fa6f41998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.176+0000 7fa6fb5a8700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f41135c0 con 0x7fa6f41094a0 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.176+0000 7fa6fb5a8700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4113440 con 0x7fa6f406bad0 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.176+0000 7fa6fb5a8700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f4113740 con 0x7fa6f406b130 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.176+0000 7fa6f9b45700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f41094a0 0x7fa6f41998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.176+0000 7fa6f9b45700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f41094a0 0x7fa6f41998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48460/0 (socket says 172.21.15.84:48460) 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.176+0000 7fa6f9b45700 1 -- 172.21.15.84:0/2581880365 learned_addr learned my addr 172.21.15.84:0/2581880365 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6f9344700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6f406b130 0x7fa6f419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6f8b43700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f406bad0 0x7fa6f419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6f9b45700 1 -- 172.21.15.84:0/2581880365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6f406b130 msgr2=0x7fa6f419f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6f9b45700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6f406b130 0x7fa6f419f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6f9b45700 1 -- 172.21.15.84:0/2581880365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f406bad0 msgr2=0x7fa6f419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6f9b45700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f406bad0 0x7fa6f419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6f9b45700 1 -- 172.21.15.84:0/2581880365 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f419a040 con 0x7fa6f41094a0 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6f8b43700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f406bad0 0x7fa6f419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:38:48.180 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6f9b45700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f41094a0 0x7fa6f41998e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00b830 tx=0x7fa6ec004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:48.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6ea7fc700 1 -- 172.21.15.84:0/2581880365 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec021070 con 0x7fa6f41094a0 2024-04-26T18:38:48.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/2581880365 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f419a2d0 con 0x7fa6f41094a0 2024-04-26T18:38:48.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6ea7fc700 1 -- 172.21.15.84:0/2581880365 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6ec007e30 con 0x7fa6f41094a0 2024-04-26T18:38:48.181 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6ea7fc700 1 -- 172.21.15.84:0/2581880365 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6ec0047d0 con 0x7fa6f41094a0 2024-04-26T18:38:48.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.177+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/2581880365 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f41a65f0 con 0x7fa6f41094a0 2024-04-26T18:38:48.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.178+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/2581880365 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f406a110 con 0x7fa6f41094a0 2024-04-26T18:38:48.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.179+0000 7fa6ea7fc700 1 -- 172.21.15.84:0/2581880365 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fa6ec007980 con 0x7fa6f41094a0 2024-04-26T18:38:48.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.180+0000 7fa6ea7fc700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa6e00a3db0 0x7fa6e00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:48.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.180+0000 7fa6f9344700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa6e00a3db0 0x7fa6e00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:48.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.180+0000 7fa6ea7fc700 1 -- 172.21.15.84:0/2581880365 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7fa6ec00f020 con 0x7fa6f41094a0 2024-04-26T18:38:48.182 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.180+0000 7fa6f9344700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa6e00a3db0 0x7fa6e00a6270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa6f419aa20 tx=0x7fa6f000b3c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:48.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.185+0000 7fa6ea7fc700 1 -- 172.21.15.84:0/2581880365 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa6ec02a020 con 0x7fa6f41094a0 2024-04-26T18:38:48.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.434+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/2581880365 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6f402e490 con 0x7fa6f41094a0 2024-04-26T18:38:48.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.435+0000 7fa6ea7fc700 1 -- 172.21.15.84:0/2581880365 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7fa6ec09e4a0 con 0x7fa6f41094a0 2024-04-26T18:38:48.436 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:38:48.436 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:38:48.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.437+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/2581880365 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa6e00a3db0 msgr2=0x7fa6e00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:48.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.437+0000 7fa6fb5a8700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa6e00a3db0 0x7fa6e00a6270 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fa6f419aa20 tx=0x7fa6f000b3c0 comp rx=0 tx=0).stop 2024-04-26T18:38:48.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.437+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/2581880365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f41094a0 msgr2=0x7fa6f41998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:48.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.437+0000 7fa6fb5a8700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f41094a0 0x7fa6f41998e0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec00b830 tx=0x7fa6ec004ce0 comp rx=0 tx=0).stop 2024-04-26T18:38:48.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.437+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/2581880365 shutdown_connections 2024-04-26T18:38:48.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.437+0000 7fa6fb5a8700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fa6e00a3db0 0x7fa6e00a6270 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:48.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.437+0000 7fa6fb5a8700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fa6f41094a0 0x7fa6f41998e0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:48.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.437+0000 7fa6fb5a8700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fa6f406b130 0x7fa6f419f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:48.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.437+0000 7fa6fb5a8700 1 --2- 172.21.15.84:0/2581880365 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fa6f406bad0 0x7fa6f419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:48.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.437+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/2581880365 >> 172.21.15.84:0/2581880365 conn(0x7fa6f40792d0 msgr2=0x7fa6f4100330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:48.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.437+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/2581880365 shutdown_connections 2024-04-26T18:38:48.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:48.437+0000 7fa6fb5a8700 1 -- 172.21.15.84:0/2581880365 wait complete. 2024-04-26T18:38:48.439 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:38:48.752 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:48 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2581880365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:38:48.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:48 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2581880365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:38:48.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:48 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2581880365' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:38:49.097 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id a8dba540 --release --await' 2024-04-26T18:38:49.097 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db --set-id a8dba540 --release --await 2024-04-26T18:38:49.345 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:38:49.778 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:49 smithi103 ceph-mon[32727]: pgmap v338: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:38:49.798 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:49 smithi084 ceph-mon[29141]: pgmap v338: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:38:49.840 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:49 smithi174 ceph-mon[32705]: pgmap v338: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:38:50.738 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:50 smithi084 ceph-mon[29141]: pgmap v339: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-04-26T18:38:50.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:50 smithi103 ceph-mon[32727]: pgmap v339: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-04-26T18:38:50.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:50 smithi174 ceph-mon[32705]: pgmap v339: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 3.2 KiB/s wr, 14 op/s 2024-04-26T18:38:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.978+0000 7f6e30c98700 1 -- 172.21.15.84:0/290960255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e2c102350 msgr2=0x7f6e2c1027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.978+0000 7f6e30c98700 1 --2- 172.21.15.84:0/290960255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e2c102350 0x7f6e2c1027b0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c009990 tx=0x7f6e1c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.978+0000 7f6e30c98700 1 -- 172.21.15.84:0/290960255 shutdown_connections 2024-04-26T18:38:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.978+0000 7f6e30c98700 1 --2- 172.21.15.84:0/290960255 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e2c102cf0 0x7f6e2c10f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.978+0000 7f6e30c98700 1 --2- 172.21.15.84:0/290960255 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e2c108390 0x7f6e2c108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.978+0000 7f6e30c98700 1 --2- 172.21.15.84:0/290960255 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e2c102350 0x7f6e2c1027b0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.978+0000 7f6e30c98700 1 -- 172.21.15.84:0/290960255 >> 172.21.15.84:0/290960255 conn(0x7f6e2c0792c0 msgr2=0x7f6e2c0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.978+0000 7f6e30c98700 1 -- 172.21.15.84:0/290960255 shutdown_connections 2024-04-26T18:38:50.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.978+0000 7f6e30c98700 1 -- 172.21.15.84:0/290960255 wait complete. 2024-04-26T18:38:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.980+0000 7f6e30c98700 1 Processor -- start 2024-04-26T18:38:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.980+0000 7f6e30c98700 1 -- start start 2024-04-26T18:38:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.980+0000 7f6e30c98700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e2c102350 0x7f6e2c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.980+0000 7f6e30c98700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e2c102cf0 0x7f6e2c198cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:50.982 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.980+0000 7f6e30c98700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e2c108390 0x7f6e2c199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.980+0000 7f6e30c98700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c112540 con 0x7f6e2c102cf0 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.980+0000 7f6e30c98700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c1123c0 con 0x7f6e2c102350 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.980+0000 7f6e30c98700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c1126c0 con 0x7f6e2c108390 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e29d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e2c102cf0 0x7f6e2c198cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e29d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e2c102cf0 0x7f6e2c198cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48474/0 (socket says 172.21.15.84:48474) 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e29d9b700 1 -- 172.21.15.84:0/3584517020 learned_addr learned my addr 172.21.15.84:0/3584517020 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e2ad9d700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e2c108390 0x7f6e2c199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e2a59c700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e2c102350 0x7f6e2c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e29d9b700 1 -- 172.21.15.84:0/3584517020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e2c108390 msgr2=0x7f6e2c199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e29d9b700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e2c108390 0x7f6e2c199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e29d9b700 1 -- 172.21.15.84:0/3584517020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e2c102350 msgr2=0x7f6e2c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e29d9b700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e2c102350 0x7f6e2c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e29d9b700 1 -- 172.21.15.84:0/3584517020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e2c1a8000 con 0x7f6e2c102cf0 2024-04-26T18:38:50.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e29d9b700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e2c102cf0 0x7f6e2c198cd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000c00 tx=0x7f6e1c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:50.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e1b7fe700 1 -- 172.21.15.84:0/3584517020 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c027070 con 0x7f6e2c102cf0 2024-04-26T18:38:50.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e1b7fe700 1 -- 172.21.15.84:0/3584517020 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e1c0041d0 con 0x7f6e2c102cf0 2024-04-26T18:38:50.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e30c98700 1 -- 172.21.15.84:0/3584517020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e2c1a8290 con 0x7f6e2c102cf0 2024-04-26T18:38:50.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e1b7fe700 1 -- 172.21.15.84:0/3584517020 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00f460 con 0x7f6e2c102cf0 2024-04-26T18:38:50.984 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.981+0000 7f6e30c98700 1 -- 172.21.15.84:0/3584517020 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e2c1a86e0 con 0x7f6e2c102cf0 2024-04-26T18:38:50.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.983+0000 7f6e1b7fe700 1 -- 172.21.15.84:0/3584517020 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f6e1c02a070 con 0x7f6e2c102cf0 2024-04-26T18:38:50.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.984+0000 7f6e1b7fe700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e140a3db0 0x7f6e140a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:50.986 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.984+0000 7f6e2a59c700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e140a3db0 0x7f6e140a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.984+0000 7f6e1b7fe700 1 -- 172.21.15.84:0/3584517020 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f6e1c0d1830 con 0x7f6e2c102cf0 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.984+0000 7f6e2a59c700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e140a3db0 0x7f6e140a6270 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e20005fd0 tx=0x7f6e2000a560 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.985+0000 7f6e3b52ab80 1 Processor -- start 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.985+0000 7f6e3b52ab80 1 -- start start 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.985+0000 7f6e3b52ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f2ea0 0x55841f6f3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.985+0000 7f6e3b52ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f37c0 0x55841f6ed250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.985+0000 7f6e3b52ab80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 0x55841f6edcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.985+0000 7f6e3b52ab80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55841f63f930 con 0x55841f6ed870 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.985+0000 7f6e3b52ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55841f5a5a30 con 0x55841f6f37c0 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.985+0000 7f6e3b52ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55841f63b8f0 con 0x55841f6f2ea0 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.985+0000 7f6e2ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 0x55841f6edcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.985+0000 7f6e2ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 0x55841f6edcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48478/0 (socket says 172.21.15.84:48478) 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.985+0000 7f6e2ad9d700 1 -- 172.21.15.84:0/2786603221 learned_addr learned my addr 172.21.15.84:0/2786603221 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.985+0000 7f6e29d9b700 1 --2- 172.21.15.84:0/2786603221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f37c0 0x55841f6ed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:50.987 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.986+0000 7f6e2a59c700 1 --2- 172.21.15.84:0/2786603221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f2ea0 0x55841f6f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.986+0000 7f6e2ad9d700 1 -- 172.21.15.84:0/2786603221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f2ea0 msgr2=0x55841f6f3280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.986+0000 7f6e2ad9d700 1 --2- 172.21.15.84:0/2786603221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f2ea0 0x55841f6f3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.986+0000 7f6e2ad9d700 1 -- 172.21.15.84:0/2786603221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f37c0 msgr2=0x55841f6ed250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.986+0000 7f6e2ad9d700 1 --2- 172.21.15.84:0/2786603221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f37c0 0x55841f6ed250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.986+0000 7f6e2ad9d700 1 -- 172.21.15.84:0/2786603221 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55841f58c9d0 con 0x55841f6ed870 2024-04-26T18:38:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.986+0000 7f6e2ad9d700 1 --2- 172.21.15.84:0/2786603221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 0x55841f6edcd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009f20 tx=0x7f6e24009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c502689d2e3deef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.986+0000 7f6e18ff9700 1 -- 172.21.15.84:0/2786603221 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2401a070 con 0x55841f6ed870 2024-04-26T18:38:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.986+0000 7f6e18ff9700 1 -- 172.21.15.84:0/2786603221 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e24015410 con 0x55841f6ed870 2024-04-26T18:38:50.988 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.986+0000 7f6e18ff9700 1 -- 172.21.15.84:0/2786603221 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2400a660 con 0x55841f6ed870 2024-04-26T18:38:50.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.987+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/2786603221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 msgr2=0x55841f6edcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:50.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.987+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/2786603221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 0x55841f6edcd0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009f20 tx=0x7f6e24009f50 comp rx=0 tx=0).stop 2024-04-26T18:38:50.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.987+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/2786603221 shutdown_connections 2024-04-26T18:38:50.992 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.987+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/2786603221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 0x55841f6edcd0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.987+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/2786603221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f2ea0 0x55841f6f3280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.987+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/2786603221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f37c0 0x55841f6ed250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.987+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/2786603221 >> 172.21.15.84:0/2786603221 conn(0x55841f61b6c0 msgr2=0x55841f6e8000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.987+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/2786603221 shutdown_connections 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.987+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/2786603221 wait complete. 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.987+0000 7f6e3b52ab80 1 -- start start 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.987+0000 7f6e3b52ab80 1 Processor -- start 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e3b52ab80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 0x55841f6f94b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e3b52ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f2ea0 0x55841f6f6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e3b52ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f37c0 0x55841f6f6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e3b52ab80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55841f6fa230 con 0x55841f6ed870 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e3b52ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55841f5f9b80 con 0x55841f6f2ea0 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e3b52ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55841f6fa3b0 con 0x55841f6f37c0 2024-04-26T18:38:50.993 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e2a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 0x55841f6f94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e3b52ab80 10 client.0 _ll_register_callbacks cb 0x55841f5d7a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e2a59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 0x55841f6f94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48492/0 (socket says 172.21.15.84:48492) 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e2a59c700 1 -- 172.21.15.84:0/1048723821 learned_addr learned my addr 172.21.15.84:0/1048723821 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e2ad9d700 1 --2- 172.21.15.84:0/1048723821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f37c0 0x55841f6f6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e29d9b700 1 --2- 172.21.15.84:0/1048723821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f2ea0 0x55841f6f6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e2a59c700 1 -- 172.21.15.84:0/1048723821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f37c0 msgr2=0x55841f6f6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e2a59c700 1 --2- 172.21.15.84:0/1048723821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f37c0 0x55841f6f6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e2a59c700 1 -- 172.21.15.84:0/1048723821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f2ea0 msgr2=0x55841f6f6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e2a59c700 1 --2- 172.21.15.84:0/1048723821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f2ea0 0x55841f6f6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e2a59c700 1 -- 172.21.15.84:0/1048723821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55841f5fd580 con 0x55841f6ed870 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e2a59c700 1 --2- 172.21.15.84:0/1048723821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 0x55841f6f94b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6e200106b0 tx=0x7f6e20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53899c7790d9bd54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.988+0000 7f6e11ffb700 1 -- 172.21.15.84:0/1048723821 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2001d830 con 0x55841f6ed870 2024-04-26T18:38:50.994 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.989+0000 7f6e11ffb700 1 -- 172.21.15.84:0/1048723821 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f6e2001d9d0 con 0x55841f6ed870 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.989+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1048723821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e24015410 con 0x55841f6ed870 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.989+0000 7f6e11ffb700 1 -- 172.21.15.84:0/1048723821 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6e2001dbd0 con 0x55841f6ed870 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.989+0000 7f6e3b52ab80 10 client.16737 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.989+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1048723821 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55841f704a70 con 0x55841f6ed870 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.989+0000 7f6e11ffb700 1 -- 172.21.15.84:0/1048723821 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e24015410 con 0x55841f6ed870 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.989+0000 7f6e11ffb700 1 -- 172.21.15.84:0/1048723821 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6e2002d3f0 con 0x55841f6ed870 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.989+0000 7f6e3b52ab80 10 client.16737 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.989+0000 7f6e3b52ab80 10 client.16737 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.989+0000 7f6e3b52ab80 20 client.16737 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.989+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1048723821 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55841f7070b0 0x55841f7094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.989+0000 7f6e3b52ab80 4 client.16737 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.989+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1048723821 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55841f64d310 con 0x55841f7070b0 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.990+0000 7f6e29d9b700 1 --2- 172.21.15.84:0/1048723821 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55841f7070b0 0x55841f7094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:50.995 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.990+0000 7f6e29d9b700 1 --2- 172.21.15.84:0/1048723821 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55841f7070b0 0x55841f7094e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ff67fb9009e5191 server_cookie=4274b93df0894f0a in_seq=0 out_seq=0 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.990+0000 7f6e11ffb700 10 client.16737 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e11ffb700 1 -- 172.21.15.84:0/1048723821 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55841f64d310 con 0x55841f7070b0 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e11ffb700 10 client.16737 handle_command_reply: tid=0 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e3b52ab80 1 client.16737 shutdown 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1048723821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 msgr2=0x55841f6f94b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1048723821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 0x55841f6f94b0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f6e200106b0 tx=0x7f6e20004750 comp rx=0 tx=0).stop 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1048723821 shutdown_connections 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1048723821 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55841f7070b0 0x55841f7094e0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1048723821 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6ed870 0x55841f6f94b0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1048723821 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f37c0 0x55841f6f6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1048723821 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f2ea0 0x55841f6f6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1048723821 >> 172.21.15.84:0/1048723821 conn(0x55841f61b6c0 msgr2=0x55841f6ec240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e11ffb700 0 client.16737 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1048723821 shutdown_connections 2024-04-26T18:38:50.996 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1048723821 wait complete. 2024-04-26T18:38:50.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:50.991+0000 7f6e3b52ab80 20 client.16737 trim_cache size 0 max 16384 2024-04-26T18:38:51.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.011+0000 7f6e3b52ab80 1 Processor -- start 2024-04-26T18:38:51.012 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.011+0000 7f6e3b52ab80 1 -- start start 2024-04-26T18:38:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e3b52ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6ed870 0x55841f6ed250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e3b52ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f2ea0 0x55841f6f4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e3b52ab80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 0x55841f703130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e3b52ab80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c1afeb0 con 0x55841f6f37c0 2024-04-26T18:38:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e3b52ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c1b19f0 con 0x55841f6ed870 2024-04-26T18:38:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e3b52ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c1b0580 con 0x55841f6f2ea0 2024-04-26T18:38:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e2ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 0x55841f703130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e2ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 0x55841f703130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48494/0 (socket says 172.21.15.84:48494) 2024-04-26T18:38:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e2ad9d700 1 -- 172.21.15.84:0/1846542716 learned_addr learned my addr 172.21.15.84:0/1846542716 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e29d9b700 1 --2- 172.21.15.84:0/1846542716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f2ea0 0x55841f6f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e2a59c700 1 --2- 172.21.15.84:0/1846542716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6ed870 0x55841f6ed250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e2ad9d700 1 -- 172.21.15.84:0/1846542716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f2ea0 msgr2=0x55841f6f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:51.013 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e2ad9d700 1 --2- 172.21.15.84:0/1846542716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f2ea0 0x55841f6f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e2ad9d700 1 -- 172.21.15.84:0/1846542716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6ed870 msgr2=0x55841f6ed250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e2ad9d700 1 --2- 172.21.15.84:0/1846542716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6ed870 0x55841f6ed250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e2ad9d700 1 -- 172.21.15.84:0/1846542716 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55841f6e47f0 con 0x55841f6f37c0 2024-04-26T18:38:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e29d9b700 1 --2- 172.21.15.84:0/1846542716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f2ea0 0x55841f6f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:38:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e2ad9d700 1 --2- 172.21.15.84:0/1846542716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 0x55841f703130 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6e24015d90 tx=0x7f6e24009730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1948afb600ed4832 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e13fff700 1 -- 172.21.15.84:0/1846542716 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e24024070 con 0x55841f6f37c0 2024-04-26T18:38:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e13fff700 1 -- 172.21.15.84:0/1846542716 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e24015410 con 0x55841f6f37c0 2024-04-26T18:38:51.014 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.012+0000 7f6e13fff700 1 -- 172.21.15.84:0/1846542716 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2401a040 con 0x55841f6f37c0 2024-04-26T18:38:51.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.013+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1846542716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 msgr2=0x55841f703130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:51.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.013+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1846542716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 0x55841f703130 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f6e24015d90 tx=0x7f6e24009730 comp rx=0 tx=0).stop 2024-04-26T18:38:51.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.013+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1846542716 shutdown_connections 2024-04-26T18:38:51.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.013+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1846542716 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 0x55841f703130 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.013+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1846542716 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6ed870 0x55841f6ed250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.013+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1846542716 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6f2ea0 0x55841f6f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.017 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.013+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1846542716 >> 172.21.15.84:0/1846542716 conn(0x55841f61b6c0 msgr2=0x55841f6e1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.013+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1846542716 shutdown_connections 2024-04-26T18:38:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.013+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1846542716 wait complete. 2024-04-26T18:38:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.013+0000 7f6e3b52ab80 1 -- start start 2024-04-26T18:38:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e3b52ab80 1 Processor -- start 2024-04-26T18:38:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e3b52ab80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6ed870 0x55841f6ffa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e3b52ab80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f2ea0 0x55841f6fc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e3b52ab80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 0x55841f6fcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e3b52ab80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c1b1320 con 0x55841f6f37c0 2024-04-26T18:38:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e3b52ab80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c1b0c50 con 0x55841f6f2ea0 2024-04-26T18:38:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e3b52ab80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e2c1ade70 con 0x55841f6ed870 2024-04-26T18:38:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e3b52ab80 10 client.0 _ll_register_callbacks cb 0x55841f5d7a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e2ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 0x55841f6fcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e2ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 0x55841f6fcb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48502/0 (socket says 172.21.15.84:48502) 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e2ad9d700 1 -- 172.21.15.84:0/1199183209 learned_addr learned my addr 172.21.15.84:0/1199183209 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e2a59c700 1 --2- 172.21.15.84:0/1199183209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6ed870 0x55841f6ffa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e2ad9d700 1 -- 172.21.15.84:0/1199183209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6ed870 msgr2=0x55841f6ffa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e2ad9d700 1 --2- 172.21.15.84:0/1199183209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6ed870 0x55841f6ffa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e2ad9d700 1 -- 172.21.15.84:0/1199183209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f2ea0 msgr2=0x55841f6fc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e2ad9d700 1 --2- 172.21.15.84:0/1199183209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f2ea0 0x55841f6fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e2ad9d700 1 -- 172.21.15.84:0/1199183209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55841f6e0230 con 0x55841f6f37c0 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e2ad9d700 1 --2- 172.21.15.84:0/1199183209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 0x55841f6fcb50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6e24018040 tx=0x7f6e24009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2b6559ff68864c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e11ffb700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e11ffb700 1 -- 172.21.15.84:0/1199183209 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e2401a070 con 0x55841f6f37c0 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e11ffb700 1 -- 172.21.15.84:0/1199183209 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f6e24015780 con 0x55841f6f37c0 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.014+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1199183209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e24015410 con 0x55841f6f37c0 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.015+0000 7f6e11ffb700 1 -- 172.21.15.84:0/1199183209 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6e24024040 con 0x55841f6f37c0 2024-04-26T18:38:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.015+0000 7f6e3b52ab80 10 client.16743 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.015+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1199183209 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55841f76e460 con 0x55841f6f37c0 2024-04-26T18:38:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.015+0000 7f6e11ffb700 1 -- 172.21.15.84:0/1199183209 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e24015410 con 0x55841f6f37c0 2024-04-26T18:38:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.015+0000 7f6e11ffb700 1 -- 172.21.15.84:0/1199183209 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6e2401d030 con 0x55841f6f37c0 2024-04-26T18:38:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.015+0000 7f6e3b52ab80 10 client.16743 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.015+0000 7f6e3b52ab80 10 client.16743 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:38:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.015+0000 7f6e3b52ab80 20 client.16743 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.015+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1199183209 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55841f76e8c0 0x55841f770cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.015+0000 7f6e3b52ab80 4 client.16743 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "a8dba540", "release": true, "await": true}] 2024-04-26T18:38:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.015+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1199183209 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "set_id": "a8dba540", "release": true, "await": true}) -- 0x55841f6473b0 con 0x55841f76e8c0 2024-04-26T18:38:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.015+0000 7f6e2a59c700 1 --2- 172.21.15.84:0/1199183209 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55841f76e8c0 0x55841f770cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.016+0000 7f6e2a59c700 1 --2- 172.21.15.84:0/1199183209 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55841f76e8c0 0x55841f770cb0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2ec0d9e4333f5b29 server_cookie=b003185c765f21f9 in_seq=0 out_seq=0 2024-04-26T18:38:51.020 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.016+0000 7f6e11ffb700 10 client.16743 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e11ffb700 1 -- 172.21.15.84:0/1199183209 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55841f6473b0 con 0x55841f76e8c0 2024-04-26T18:38:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e11ffb700 10 client.16743 handle_command_reply: tid=0 2024-04-26T18:38:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e3b52ab80 1 client.16743 shutdown 2024-04-26T18:38:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1199183209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 msgr2=0x55841f6fcb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1199183209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 0x55841f6fcb50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f6e24018040 tx=0x7f6e24009b00 comp rx=0 tx=0).stop 2024-04-26T18:38:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1199183209 shutdown_connections 2024-04-26T18:38:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1199183209 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55841f76e8c0 0x55841f770cb0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1199183209 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55841f6f37c0 0x55841f6fcb50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1199183209 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55841f6ed870 0x55841f6ffa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e11ffb700 0 client.16743 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e3b52ab80 1 --2- 172.21.15.84:0/1199183209 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55841f6f2ea0 0x55841f6fc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1199183209 >> 172.21.15.84:0/1199183209 conn(0x55841f61b6c0 msgr2=0x55841f6f2290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1199183209 shutdown_connections 2024-04-26T18:38:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e3b52ab80 1 -- 172.21.15.84:0/1199183209 wait complete. 2024-04-26T18:38:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.020+0000 7f6e3b52ab80 20 client.16743 trim_cache size 0 max 16384 2024-04-26T18:38:51.023 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:38:51.023 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:38:51.023 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:38:51.023 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 32, 2024-04-26T18:38:51.023 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:38:51.023 INFO:teuthology.orchestra.run.smithi084.stdout: "a8dba540": { 2024-04-26T18:38:51.023 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 32, 2024-04-26T18:38:51.023 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:38:51.023 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:38:51.023 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:38:51.023 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 81.2, 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:51.024 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:38:51.025 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:38:51.025 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.021+0000 7f6e30c98700 1 -- 172.21.15.84:0/3584517020 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e140a3db0 msgr2=0x7f6e140a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.021+0000 7f6e30c98700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e140a3db0 0x7f6e140a6270 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6e20005fd0 tx=0x7f6e2000a560 comp rx=0 tx=0).stop 2024-04-26T18:38:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.021+0000 7f6e30c98700 1 -- 172.21.15.84:0/3584517020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e2c102cf0 msgr2=0x7f6e2c198cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.021+0000 7f6e30c98700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e2c102cf0 0x7f6e2c198cd0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c000c00 tx=0x7f6e1c0047b0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.021+0000 7f6e30c98700 1 -- 172.21.15.84:0/3584517020 shutdown_connections 2024-04-26T18:38:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.021+0000 7f6e30c98700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e140a3db0 0x7f6e140a6270 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.021+0000 7f6e30c98700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e2c108390 0x7f6e2c199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.021+0000 7f6e30c98700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e2c102350 0x7f6e2c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.021+0000 7f6e30c98700 1 --2- 172.21.15.84:0/3584517020 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e2c102cf0 0x7f6e2c198cd0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.021+0000 7f6e30c98700 1 -- 172.21.15.84:0/3584517020 >> 172.21.15.84:0/3584517020 conn(0x7f6e2c0792c0 msgr2=0x7f6e2c101180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.021+0000 7f6e30c98700 1 -- 172.21.15.84:0/3584517020 shutdown_connections 2024-04-26T18:38:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:38:51.021+0000 7f6e30c98700 1 -- 172.21.15.84:0/3584517020 wait complete. 2024-04-26T18:38:51.697 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'a8dba540', seconds elapsed: 32.4 2024-04-26T18:38:52.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:52 smithi174 ceph-mon[32705]: pgmap v340: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-04-26T18:38:53.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:52 smithi084 ceph-mon[29141]: pgmap v340: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-04-26T18:38:53.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:52 smithi103 ceph-mon[32727]: pgmap v340: 97 pgs: 97 active+clean; 4.8 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.2 KiB/s wr, 9 op/s 2024-04-26T18:38:55.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:54 smithi084 ceph-mon[29141]: pgmap v341: 97 pgs: 97 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-26T18:38:55.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:54 smithi103 ceph-mon[32727]: pgmap v341: 97 pgs: 97 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-26T18:38:55.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:54 smithi174 ceph-mon[32705]: pgmap v341: 97 pgs: 97 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-26T18:38:57.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:56 smithi084 ceph-mon[29141]: pgmap v342: 97 pgs: 97 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-26T18:38:57.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:56 smithi103 ceph-mon[32727]: pgmap v342: 97 pgs: 97 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-26T18:38:57.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:56 smithi174 ceph-mon[32705]: pgmap v342: 97 pgs: 97 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-26T18:38:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.234+0000 7f06c92a2640 1 -- 172.21.15.84:0/4125293417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c4103be0 msgr2=0x7f06c4103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:57.234 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.234+0000 7f06c92a2640 1 --2- 172.21.15.84:0/4125293417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c4103be0 0x7f06c4103fc0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009900 tx=0x7f06b402a480 comp rx=0 tx=0).stop 2024-04-26T18:38:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.234+0000 7f06c92a2640 1 -- 172.21.15.84:0/4125293417 shutdown_connections 2024-04-26T18:38:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.234+0000 7f06c92a2640 1 --2- 172.21.15.84:0/4125293417 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06c410cf20 0x7f06c410f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.234+0000 7f06c92a2640 1 --2- 172.21.15.84:0/4125293417 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4104500 0x7f06c410c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.234+0000 7f06c92a2640 1 --2- 172.21.15.84:0/4125293417 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c4103be0 0x7f06c4103fc0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.234+0000 7f06c92a2640 1 -- 172.21.15.84:0/4125293417 >> 172.21.15.84:0/4125293417 conn(0x7f06c40789b0 msgr2=0x7f06c4078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.234+0000 7f06c92a2640 1 -- 172.21.15.84:0/4125293417 shutdown_connections 2024-04-26T18:38:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.234+0000 7f06c92a2640 1 -- 172.21.15.84:0/4125293417 wait complete. 2024-04-26T18:38:57.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.235+0000 7f06c92a2640 1 Processor -- start 2024-04-26T18:38:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.235+0000 7f06c92a2640 1 -- start start 2024-04-26T18:38:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.235+0000 7f06c92a2640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4103be0 0x7f06c4199120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.235+0000 7f06c92a2640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06c4104500 0x7f06c4199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.235+0000 7f06c92a2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c410cf20 0x7f06c419d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.235+0000 7f06c92a2640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4112ba0 con 0x7f06c410cf20 2024-04-26T18:38:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.235+0000 7f06c92a2640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4112a20 con 0x7f06c4103be0 2024-04-26T18:38:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.235+0000 7f06c92a2640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4112d20 con 0x7f06c4104500 2024-04-26T18:38:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.235+0000 7f06c37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c410cf20 0x7f06c419d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.235+0000 7f06c37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c410cf20 0x7f06c419d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60794/0 (socket says 172.21.15.84:60794) 2024-04-26T18:38:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.235+0000 7f06c37fe640 1 -- 172.21.15.84:0/455475500 learned_addr learned my addr 172.21.15.84:0/455475500 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:57.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.236+0000 7f06c2ffd640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4103be0 0x7f06c4199120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.236+0000 7f06c27fc640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06c4104500 0x7f06c4199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.236+0000 7f06c37fe640 1 -- 172.21.15.84:0/455475500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06c4104500 msgr2=0x7f06c4199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.236+0000 7f06c37fe640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06c4104500 0x7f06c4199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.236+0000 7f06c37fe640 1 -- 172.21.15.84:0/455475500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4103be0 msgr2=0x7f06c4199120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.236+0000 7f06c37fe640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4103be0 0x7f06c4199120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.236+0000 7f06c37fe640 1 -- 172.21.15.84:0/455475500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c419e110 con 0x7f06c410cf20 2024-04-26T18:38:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.236+0000 7f06c37fe640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c410cf20 0x7f06c419d9e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00eaa0 tx=0x7f06bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.236+0000 7f06abfff640 1 -- 172.21.15.84:0/455475500 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc00cdb0 con 0x7f06c410cf20 2024-04-26T18:38:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.236+0000 7f06c92a2640 1 -- 172.21.15.84:0/455475500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c419e400 con 0x7f06c410cf20 2024-04-26T18:38:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.236+0000 7f06abfff640 1 -- 172.21.15.84:0/455475500 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06bc005020 con 0x7f06c410cf20 2024-04-26T18:38:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.236+0000 7f06abfff640 1 -- 172.21.15.84:0/455475500 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc0116d0 con 0x7f06c410cf20 2024-04-26T18:38:57.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.236+0000 7f06c92a2640 1 -- 172.21.15.84:0/455475500 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c41a5c90 con 0x7f06c410cf20 2024-04-26T18:38:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.238+0000 7f06abfff640 1 -- 172.21.15.84:0/455475500 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f06bc0051d0 con 0x7f06c410cf20 2024-04-26T18:38:57.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06cbfbd740 1 Processor -- start 2024-04-26T18:38:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06cbfbd740 1 -- start start 2024-04-26T18:38:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06abfff640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f06ac0a3dc0 0x7f06ac0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06cbfbd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420e832e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06cbfbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e83820 0x55f420e8dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06cbfbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e8e0e0 0x55f420e904d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06cbfbd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f420de9460 con 0x55f420e82f00 2024-04-26T18:38:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06cbfbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f420da3590 con 0x55f420e83820 2024-04-26T18:38:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06cbfbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f420de5420 con 0x55f420e8e0e0 2024-04-26T18:38:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06c27fc640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f06ac0a3dc0 0x7f06ac0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06c2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420e832e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06abfff640 1 -- 172.21.15.84:0/455475500 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f06bc0d08e0 con 0x7f06c410cf20 2024-04-26T18:38:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06c2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420e832e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60806/0 (socket says 172.21.15.84:60806) 2024-04-26T18:38:57.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06c2ffd640 1 -- 172.21.15.84:0/2386288019 learned_addr learned my addr 172.21.15.84:0/2386288019 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06c27fc640 1 --2- 172.21.15.84:0/2386288019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e83820 0x55f420e8dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.239+0000 7f06c37fe640 1 --2- 172.21.15.84:0/2386288019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e8e0e0 0x55f420e904d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.240+0000 7f06c2ffd640 1 -- 172.21.15.84:0/2386288019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e8e0e0 msgr2=0x55f420e904d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.240+0000 7f06c2ffd640 1 --2- 172.21.15.84:0/2386288019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e8e0e0 0x55f420e904d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.240+0000 7f06c2ffd640 1 -- 172.21.15.84:0/2386288019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e83820 msgr2=0x55f420e8dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.240+0000 7f06c2ffd640 1 --2- 172.21.15.84:0/2386288019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e83820 0x55f420e8dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.240+0000 7f06c2ffd640 1 -- 172.21.15.84:0/2386288019 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f420db00d0 con 0x55f420e82f00 2024-04-26T18:38:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.240+0000 7f06c27fc640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f06ac0a3dc0 0x7f06ac0a6280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f06b800bfd0 tx=0x7f06b800bb40 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.240+0000 7f06c2ffd640 1 --2- 172.21.15.84:0/2386288019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420e832e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009900 tx=0x7f06b4032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752b5b2f8e336aa9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.240+0000 7f06a97fa640 1 -- 172.21.15.84:0/2386288019 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4040070 con 0x55f420e82f00 2024-04-26T18:38:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.240+0000 7f06a97fa640 1 -- 172.21.15.84:0/2386288019 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b4004030 con 0x55f420e82f00 2024-04-26T18:38:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.240+0000 7f06a97fa640 1 -- 172.21.15.84:0/2386288019 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4035040 con 0x55f420e82f00 2024-04-26T18:38:57.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 -- 172.21.15.84:0/2386288019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 msgr2=0x55f420e832e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/2386288019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420e832e0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009900 tx=0x7f06b4032bd0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 -- 172.21.15.84:0/2386288019 shutdown_connections 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/2386288019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e8e0e0 0x55f420e904d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/2386288019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e83820 0x55f420e8dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/2386288019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420e832e0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 -- 172.21.15.84:0/2386288019 >> 172.21.15.84:0/2386288019 conn(0x55f420e7f410 msgr2=0x55f420e7f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 -- 172.21.15.84:0/2386288019 shutdown_connections 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 -- 172.21.15.84:0/2386288019 wait complete. 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 -- start start 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 Processor -- start 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420e8b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e83820 0x55f420e8b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e8e0e0 0x55f420e8ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f420e90cb0 con 0x55f420e82f00 2024-04-26T18:38:57.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f420d70b60 con 0x55f420e8e0e0 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f420e90e30 con 0x55f420e83820 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.241+0000 7f06cbfbd740 10 client.0 _ll_register_callbacks cb 0x55f420d952b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06c2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420e8b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06c2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420e8b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60816/0 (socket says 172.21.15.84:60816) 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06c2ffd640 1 -- 172.21.15.84:0/2008824777 learned_addr learned my addr 172.21.15.84:0/2008824777 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06c27fc640 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e83820 0x55f420e8b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06c37fe640 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e8e0e0 0x55f420e8ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06c2ffd640 1 -- 172.21.15.84:0/2008824777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e83820 msgr2=0x55f420e8b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06c2ffd640 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e83820 0x55f420e8b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06c2ffd640 1 -- 172.21.15.84:0/2008824777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e8e0e0 msgr2=0x55f420e8ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06c2ffd640 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e8e0e0 0x55f420e8ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06c2ffd640 1 -- 172.21.15.84:0/2008824777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f420afb210 con 0x55f420e82f00 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06c27fc640 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e83820 0x55f420e8b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06c2ffd640 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420e8b010 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f06b4032f70 tx=0x7f06b402a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67e07933e517385 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:57.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06a6ff5640 1 -- 172.21.15.84:0/2008824777 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4040070 con 0x55f420e82f00 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06a6ff5640 1 -- 172.21.15.84:0/2008824777 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f06b4005280 con 0x55f420e82f00 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06cbfbd740 1 -- 172.21.15.84:0/2008824777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06b4004030 con 0x55f420e82f00 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06a6ff5640 1 -- 172.21.15.84:0/2008824777 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06b4056050 con 0x55f420e82f00 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06cbfbd740 10 client.16755 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06cbfbd740 1 -- 172.21.15.84:0/2008824777 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f420bc7620 con 0x55f420e82f00 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.242+0000 7f06a6ff5640 1 -- 172.21.15.84:0/2008824777 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b4004030 con 0x55f420e82f00 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.243+0000 7f06a6ff5640 1 -- 172.21.15.84:0/2008824777 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f06b4004be0 con 0x55f420e82f00 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.243+0000 7f06cbfbd740 10 client.16755 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.243+0000 7f06cbfbd740 10 client.16755 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.243+0000 7f06cbfbd740 20 client.16755 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.243+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f420e9dff0 0x55f420ea03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.243+0000 7f06cbfbd740 4 client.16755 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.243+0000 7f06cbfbd740 1 -- 172.21.15.84:0/2008824777 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f420df7070 con 0x55f420e9dff0 2024-04-26T18:38:57.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.243+0000 7f06c27fc640 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f420e9dff0 0x55f420ea03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.243+0000 7f06c27fc640 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f420e9dff0 0x55f420ea03e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=281c6ace22def5fd server_cookie=a8fae42c3b4b333d in_seq=0 out_seq=0 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.243+0000 7f06a6ff5640 10 client.16755 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06a6ff5640 1 -- 172.21.15.84:0/2008824777 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f420df7070 con 0x55f420e9dff0 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06a6ff5640 10 client.16755 handle_command_reply: tid=0 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06cbfbd740 1 client.16755 shutdown 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06cbfbd740 1 -- 172.21.15.84:0/2008824777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 msgr2=0x55f420e8b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420e8b010 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f06b4032f70 tx=0x7f06b402a480 comp rx=0 tx=0).stop 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06cbfbd740 1 -- 172.21.15.84:0/2008824777 shutdown_connections 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f420e9dff0 0x55f420ea03e0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e8e0e0 0x55f420e8ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e83820 0x55f420e8b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/2008824777 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420e8b010 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06a6ff5640 0 client.16755 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06cbfbd740 1 -- 172.21.15.84:0/2008824777 >> 172.21.15.84:0/2008824777 conn(0x55f420e7f410 msgr2=0x55f420e85190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06cbfbd740 1 -- 172.21.15.84:0/2008824777 shutdown_connections 2024-04-26T18:38:57.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06cbfbd740 1 -- 172.21.15.84:0/2008824777 wait complete. 2024-04-26T18:38:57.252 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.244+0000 7f06cbfbd740 20 client.16755 trim_cache size 0 max 16384 2024-04-26T18:38:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.259+0000 7f06cbfbd740 1 Processor -- start 2024-04-26T18:38:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.259+0000 7f06cbfbd740 1 -- start start 2024-04-26T18:38:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.259+0000 7f06cbfbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e82f00 0x55f420e99b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.259+0000 7f06cbfbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e83820 0x55f420e96460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.259+0000 7f06cbfbd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e8e0e0 0x55f420e84fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.259+0000 7f06cbfbd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41ae630 con 0x55f420e8e0e0 2024-04-26T18:38:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.259+0000 7f06cbfbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41b0170 con 0x55f420e83820 2024-04-26T18:38:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.259+0000 7f06cbfbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41aed00 con 0x55f420e82f00 2024-04-26T18:38:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.259+0000 7f06c37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e8e0e0 0x55f420e84fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.264 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.259+0000 7f06c37fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e8e0e0 0x55f420e84fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60822/0 (socket says 172.21.15.84:60822) 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.259+0000 7f06c37fe640 1 -- 172.21.15.84:0/3321711025 learned_addr learned my addr 172.21.15.84:0/3321711025 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.260+0000 7f06c27fc640 1 --2- 172.21.15.84:0/3321711025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e83820 0x55f420e96460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.260+0000 7f06c37fe640 1 -- 172.21.15.84:0/3321711025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e82f00 msgr2=0x55f420e99b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.260+0000 7f06c37fe640 1 --2- 172.21.15.84:0/3321711025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e82f00 0x55f420e99b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.260+0000 7f06c37fe640 1 -- 172.21.15.84:0/3321711025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e83820 msgr2=0x55f420e96460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.260+0000 7f06c37fe640 1 --2- 172.21.15.84:0/3321711025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e83820 0x55f420e96460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.260+0000 7f06c37fe640 1 -- 172.21.15.84:0/3321711025 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f420e83650 con 0x55f420e8e0e0 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.260+0000 7f06c37fe640 1 --2- 172.21.15.84:0/3321711025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e8e0e0 0x55f420e84fc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0ce0a0 tx=0x7f06bc09c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ebd2966eef3faf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.260+0000 7f06a8ff9640 1 -- 172.21.15.84:0/3321711025 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc01f040 con 0x55f420e8e0e0 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.260+0000 7f06a8ff9640 1 -- 172.21.15.84:0/3321711025 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06bc0ce270 con 0x55f420e8e0e0 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.260+0000 7f06a8ff9640 1 -- 172.21.15.84:0/3321711025 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06bc09cdb0 con 0x55f420e8e0e0 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 -- 172.21.15.84:0/3321711025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e8e0e0 msgr2=0x55f420e84fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/3321711025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e8e0e0 0x55f420e84fc0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0ce0a0 tx=0x7f06bc09c8f0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.265 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 -- 172.21.15.84:0/3321711025 shutdown_connections 2024-04-26T18:38:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/3321711025 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e8e0e0 0x55f420e84fc0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/3321711025 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e83820 0x55f420e96460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/3321711025 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e82f00 0x55f420e99b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 -- 172.21.15.84:0/3321711025 >> 172.21.15.84:0/3321711025 conn(0x55f420e7f410 msgr2=0x55f420e848b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 -- 172.21.15.84:0/3321711025 shutdown_connections 2024-04-26T18:38:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 -- 172.21.15.84:0/3321711025 wait complete. 2024-04-26T18:38:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 -- start start 2024-04-26T18:38:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 Processor -- start 2024-04-26T18:38:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420efc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e83820 0x55f420efc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e8e0e0 0x55f420f07340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.266 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41afaa0 con 0x55f420e82f00 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41af3d0 con 0x55f420e8e0e0 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41ac630 con 0x55f420e83820 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06cbfbd740 10 client.0 _ll_register_callbacks cb 0x7f06b8012490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06c2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420efc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06c2ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420efc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60830/0 (socket says 172.21.15.84:60830) 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06c2ffd640 1 -- 172.21.15.84:0/18590922 learned_addr learned my addr 172.21.15.84:0/18590922 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.261+0000 7f06c27fc640 1 --2- 172.21.15.84:0/18590922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e83820 0x55f420efc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06c2ffd640 1 -- 172.21.15.84:0/18590922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e83820 msgr2=0x55f420efc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06c2ffd640 1 --2- 172.21.15.84:0/18590922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e83820 0x55f420efc800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06c2ffd640 1 -- 172.21.15.84:0/18590922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e8e0e0 msgr2=0x55f420f07340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06c2ffd640 1 --2- 172.21.15.84:0/18590922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e8e0e0 0x55f420f07340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06c2ffd640 1 -- 172.21.15.84:0/18590922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f420efd0c0 con 0x55f420e82f00 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06c2ffd640 1 --2- 172.21.15.84:0/18590922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420efc2c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f06b402a9a0 tx=0x7f06b4032ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6f48d70a726826 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:38:57.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06a6ff5640 1 -- 172.21.15.84:0/18590922 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b405f060 con 0x55f420e82f00 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06a6ff5640 1 -- 172.21.15.84:0/18590922 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 8579+0+0 (secure 0 0 0) 0x7f06b40053e0 con 0x55f420e82f00 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06cbfbd740 1 -- 172.21.15.84:0/18590922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06bc0ce270 con 0x55f420e82f00 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06a6ff5640 1 -- 172.21.15.84:0/18590922 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06b4040070 con 0x55f420e82f00 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06cbfbd740 10 client.16761 fetch_fsmap learned FSMap version 29 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06cbfbd740 1 -- 172.21.15.84:0/18590922 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f420f08990 con 0x55f420e82f00 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06a6ff5640 1 -- 172.21.15.84:0/18590922 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06bc0ce270 con 0x55f420e82f00 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06a6ff5640 1 -- 172.21.15.84:0/18590922 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f06b4003ea0 con 0x55f420e82f00 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06cbfbd740 10 client.16761 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06cbfbd740 10 client.16761 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06cbfbd740 20 client.16761 populate_metadata read hostname 'smithi084' 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/18590922 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f420f090c0 0x55f420f0b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06cbfbd740 4 client.16761 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.262+0000 7f06cbfbd740 1 -- 172.21.15.84:0/18590922 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f420df0ee0 con 0x55f420f090c0 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.263+0000 7f06c27fc640 1 --2- 172.21.15.84:0/18590922 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f420f090c0 0x55f420f0b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:38:57.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.263+0000 7f06a6ff5640 10 client.16761 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:57.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.263+0000 7f06c27fc640 1 --2- 172.21.15.84:0/18590922 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f420f090c0 0x55f420f0b4e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0217c560c51bb4f server_cookie=2bfa52b5ceaf795f in_seq=0 out_seq=0 2024-04-26T18:38:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.272+0000 7f06a6ff5640 1 -- 172.21.15.84:0/18590922 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f420df0ee0 con 0x55f420f090c0 2024-04-26T18:38:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.272+0000 7f06a6ff5640 10 client.16761 handle_command_reply: tid=0 2024-04-26T18:38:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.272+0000 7f06cbfbd740 1 client.16761 shutdown 2024-04-26T18:38:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.272+0000 7f06cbfbd740 1 -- 172.21.15.84:0/18590922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 msgr2=0x55f420efc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.272+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/18590922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420efc2c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f06b402a9a0 tx=0x7f06b4032ba0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.272+0000 7f06cbfbd740 1 -- 172.21.15.84:0/18590922 shutdown_connections 2024-04-26T18:38:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.272+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/18590922 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f420f090c0 0x55f420f0b4e0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.272+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/18590922 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f420e8e0e0 0x55f420f07340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.272+0000 7f06a6ff5640 0 client.16761 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:38:57.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.272+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/18590922 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f420e83820 0x55f420efc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.272+0000 7f06cbfbd740 1 --2- 172.21.15.84:0/18590922 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f420e82f00 0x55f420efc2c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06cbfbd740 1 -- 172.21.15.84:0/18590922 >> 172.21.15.84:0/18590922 conn(0x55f420e7f410 msgr2=0x55f420e85140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06cbfbd740 1 -- 172.21.15.84:0/18590922 shutdown_connections 2024-04-26T18:38:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06cbfbd740 1 -- 172.21.15.84:0/18590922 wait complete. 2024-04-26T18:38:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06cbfbd740 20 client.16761 trim_cache size 0 max 16384 2024-04-26T18:38:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:38:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:38:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:38:57.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:38:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06c92a2640 1 -- 172.21.15.84:0/455475500 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f06ac0a3dc0 msgr2=0x7f06ac0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06c92a2640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f06ac0a3dc0 0x7f06ac0a6280 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f06b800bfd0 tx=0x7f06b800bb40 comp rx=0 tx=0).stop 2024-04-26T18:38:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06c92a2640 1 -- 172.21.15.84:0/455475500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c410cf20 msgr2=0x7f06c419d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:38:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06c92a2640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c410cf20 0x7f06c419d9e0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f06bc00eaa0 tx=0x7f06bc00ef60 comp rx=0 tx=0).stop 2024-04-26T18:38:57.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06c92a2640 1 -- 172.21.15.84:0/455475500 shutdown_connections 2024-04-26T18:38:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06c92a2640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f06ac0a3dc0 0x7f06ac0a6280 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06c92a2640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c410cf20 0x7f06c419d9e0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06c92a2640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06c4104500 0x7f06c4199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06c92a2640 1 --2- 172.21.15.84:0/455475500 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4103be0 0x7f06c4199120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:38:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06c92a2640 1 -- 172.21.15.84:0/455475500 >> 172.21.15.84:0/455475500 conn(0x7f06c40789b0 msgr2=0x7f06c40fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:38:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06c92a2640 1 -- 172.21.15.84:0/455475500 shutdown_connections 2024-04-26T18:38:57.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:38:57.273+0000 7f06c92a2640 1 -- 172.21.15.84:0/455475500 wait complete. 2024-04-26T18:38:59.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:38:58 smithi084 ceph-mon[29141]: pgmap v343: 97 pgs: 97 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-26T18:38:59.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:38:58 smithi103 ceph-mon[32727]: pgmap v343: 97 pgs: 97 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-26T18:38:59.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:38:58 smithi174 ceph-mon[32705]: pgmap v343: 97 pgs: 97 active+clean; 5.0 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 11 op/s 2024-04-26T18:39:01.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:00 smithi084 ceph-mon[29141]: pgmap v344: 97 pgs: 97 active+clean; 5.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-04-26T18:39:01.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:00 smithi103 ceph-mon[32727]: pgmap v344: 97 pgs: 97 active+clean; 5.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-04-26T18:39:01.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:00 smithi174 ceph-mon[32705]: pgmap v344: 97 pgs: 97 active+clean; 5.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 19 op/s 2024-04-26T18:39:03.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:02 smithi084 ceph-mon[29141]: pgmap v345: 97 pgs: 97 active+clean; 5.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 14 op/s 2024-04-26T18:39:03.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:02 smithi103 ceph-mon[32727]: pgmap v345: 97 pgs: 97 active+clean; 5.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 14 op/s 2024-04-26T18:39:03.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:02 smithi174 ceph-mon[32705]: pgmap v345: 97 pgs: 97 active+clean; 5.1 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 30 KiB/s wr, 14 op/s 2024-04-26T18:39:05.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:04 smithi084 ceph-mon[29141]: pgmap v346: 97 pgs: 97 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 94 KiB/s wr, 38 op/s 2024-04-26T18:39:05.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:04 smithi103 ceph-mon[32727]: pgmap v346: 97 pgs: 97 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 94 KiB/s wr, 38 op/s 2024-04-26T18:39:05.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:04 smithi174 ceph-mon[32705]: pgmap v346: 97 pgs: 97 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 94 KiB/s wr, 38 op/s 2024-04-26T18:39:06.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:05 smithi084 ceph-mon[29141]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T18:39:06.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:05 smithi103 ceph-mon[32727]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T18:39:06.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:05 smithi174 ceph-mon[32705]: osdmap e105: 12 total, 12 up, 12 in 2024-04-26T18:39:07.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:06 smithi084 ceph-mon[29141]: pgmap v348: 97 pgs: 97 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 101 KiB/s wr, 39 op/s 2024-04-26T18:39:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:06 smithi103 ceph-mon[32727]: pgmap v348: 97 pgs: 97 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 101 KiB/s wr, 39 op/s 2024-04-26T18:39:07.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:06 smithi174 ceph-mon[32705]: pgmap v348: 97 pgs: 97 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 101 KiB/s wr, 39 op/s 2024-04-26T18:39:09.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:08 smithi084 ceph-mon[29141]: pgmap v349: 97 pgs: 97 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 101 KiB/s wr, 39 op/s 2024-04-26T18:39:09.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:08 smithi103 ceph-mon[32727]: pgmap v349: 97 pgs: 97 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 101 KiB/s wr, 39 op/s 2024-04-26T18:39:09.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:08 smithi174 ceph-mon[32705]: pgmap v349: 97 pgs: 97 active+clean; 5.4 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 101 KiB/s wr, 39 op/s 2024-04-26T18:39:10.371 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.369+0000 7f58240ea640 1 -- 172.21.15.84:0/1323089542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f581c103300 msgr2=0x7f581c10f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.369+0000 7f58240ea640 1 --2- 172.21.15.84:0/1323089542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f581c103300 0x7f581c10f810 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f5814009cd0 tx=0x7f581402a8e0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.369+0000 7f58240ea640 1 -- 172.21.15.84:0/1323089542 shutdown_connections 2024-04-26T18:39:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.369+0000 7f58240ea640 1 --2- 172.21.15.84:0/1323089542 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f581c103300 0x7f581c10f810 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.369+0000 7f58240ea640 1 --2- 172.21.15.84:0/1323089542 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f581c102940 0x7f581c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.369+0000 7f58240ea640 1 --2- 172.21.15.84:0/1323089542 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f581c108950 0x7f581c108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.369+0000 7f58240ea640 1 -- 172.21.15.84:0/1323089542 >> 172.21.15.84:0/1323089542 conn(0x7f581c079dc0 msgr2=0x7f581c07a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.369+0000 7f58240ea640 1 -- 172.21.15.84:0/1323089542 shutdown_connections 2024-04-26T18:39:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.369+0000 7f58240ea640 1 -- 172.21.15.84:0/1323089542 wait complete. 2024-04-26T18:39:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.370+0000 7f58240ea640 1 Processor -- start 2024-04-26T18:39:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.370+0000 7f58240ea640 1 -- start start 2024-04-26T18:39:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.370+0000 7f58240ea640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f581c102940 0x7f581c074c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.372 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.370+0000 7f58240ea640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f581c103300 0x7f581c075170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.370+0000 7f58240ea640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f581c108950 0x7f581c0756b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.370+0000 7f58240ea640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f581c112c10 con 0x7f581c102940 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.370+0000 7f58240ea640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f581c112a90 con 0x7f581c103300 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.370+0000 7f58240ea640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f581c112d90 con 0x7f581c108950 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.370+0000 7f5821e86640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f581c102940 0x7f581c074c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.370+0000 7f5821e86640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f581c102940 0x7f581c074c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55656/0 (socket says 172.21.15.84:55656) 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.370+0000 7f5821e86640 1 -- 172.21.15.84:0/3892528580 learned_addr learned my addr 172.21.15.84:0/3892528580 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.370+0000 7f5822687640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f581c108950 0x7f581c0756b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.370+0000 7f5821685640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f581c103300 0x7f581c075170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.371+0000 7f5821e86640 1 -- 172.21.15.84:0/3892528580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f581c108950 msgr2=0x7f581c0756b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.371+0000 7f5821e86640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f581c108950 0x7f581c0756b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.371+0000 7f5821e86640 1 -- 172.21.15.84:0/3892528580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f581c103300 msgr2=0x7f581c075170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.371+0000 7f5821e86640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f581c103300 0x7f581c075170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.371+0000 7f5821e86640 1 -- 172.21.15.84:0/3892528580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f581c1a7570 con 0x7f581c102940 2024-04-26T18:39:10.373 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.371+0000 7f5821e86640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f581c102940 0x7f581c074c30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f581800e9f0 tx=0x7f581800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.371+0000 7f5812ffd640 1 -- 172.21.15.84:0/3892528580 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f581800cd80 con 0x7f581c102940 2024-04-26T18:39:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.371+0000 7f5812ffd640 1 -- 172.21.15.84:0/3892528580 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58180108e0 con 0x7f581c102940 2024-04-26T18:39:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.371+0000 7f58240ea640 1 -- 172.21.15.84:0/3892528580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f581c1a7860 con 0x7f581c102940 2024-04-26T18:39:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.371+0000 7f5812ffd640 1 -- 172.21.15.84:0/3892528580 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5818018980 con 0x7f581c102940 2024-04-26T18:39:10.374 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.371+0000 7f58240ea640 1 -- 172.21.15.84:0/3892528580 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f581c1a7db0 con 0x7f581c102940 2024-04-26T18:39:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.373+0000 7f5812ffd640 1 -- 172.21.15.84:0/3892528580 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5818010430 con 0x7f581c102940 2024-04-26T18:39:10.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.373+0000 7f5826e00740 1 Processor -- start 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5826e00740 1 -- start start 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5826e00740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 0x5581032572e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5826e00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558103257820 0x558103261ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5826e00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581032620e0 0x5581032644d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5826e00740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5581031bd460 con 0x558103256f00 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5826e00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558103177590 con 0x5581032620e0 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5826e00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5581031b9420 con 0x558103257820 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5812ffd640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f58080a3dc0 0x7f58080a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5822687640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 0x5581032572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5822687640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 0x5581032572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55672/0 (socket says 172.21.15.84:55672) 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5822687640 1 -- 172.21.15.84:0/3763474811 learned_addr learned my addr 172.21.15.84:0/3763474811 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5821685640 1 --2- 172.21.15.84:0/3763474811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581032620e0 0x5581032644d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5822687640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f58080a3dc0 0x7f58080a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5812ffd640 1 -- 172.21.15.84:0/3892528580 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8623+0+0 (secure 0 0 0) 0x7f58180d0ad0 con 0x7f581c102940 2024-04-26T18:39:10.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5821e86640 1 --2- 172.21.15.84:0/3763474811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558103257820 0x558103261ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5822687640 1 -- 172.21.15.84:0/3763474811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558103257820 msgr2=0x558103261ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5822687640 1 --2- 172.21.15.84:0/3763474811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558103257820 0x558103261ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.374+0000 7f5822687640 1 -- 172.21.15.84:0/3763474811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581032620e0 msgr2=0x5581032644d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.375+0000 7f5822687640 1 --2- 172.21.15.84:0/3763474811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581032620e0 0x5581032644d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.375+0000 7f5822687640 1 -- 172.21.15.84:0/3763474811 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581031840d0 con 0x558103256f00 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.375+0000 7f5821685640 1 --2- 172.21.15.84:0/3763474811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581032620e0 0x5581032644d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.375+0000 7f5822687640 1 --2- 172.21.15.84:0/3763474811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 0x5581032572e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5814004af0 tx=0x7f5814031850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=277eb325c04ca526 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.375+0000 7f5807fff640 1 -- 172.21.15.84:0/3763474811 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814004410 con 0x558103256f00 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.375+0000 7f5807fff640 1 -- 172.21.15.84:0/3763474811 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5814050070 con 0x558103256f00 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.375+0000 7f5807fff640 1 -- 172.21.15.84:0/3763474811 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5814030cb0 con 0x558103256f00 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.375+0000 7f5822687640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f58080a3dc0 0x7f58080a6280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f581404f000 tx=0x7f581404d000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.376+0000 7f5826e00740 1 -- 172.21.15.84:0/3763474811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 msgr2=0x5581032572e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.376+0000 7f5826e00740 1 --2- 172.21.15.84:0/3763474811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 0x5581032572e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f5814004af0 tx=0x7f5814031850 comp rx=0 tx=0).stop 2024-04-26T18:39:10.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.376+0000 7f5826e00740 1 -- 172.21.15.84:0/3763474811 shutdown_connections 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.376+0000 7f5826e00740 1 --2- 172.21.15.84:0/3763474811 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581032620e0 0x5581032644d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.376+0000 7f5826e00740 1 --2- 172.21.15.84:0/3763474811 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558103257820 0x558103261ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.376+0000 7f5826e00740 1 --2- 172.21.15.84:0/3763474811 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 0x5581032572e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.376+0000 7f5826e00740 1 -- 172.21.15.84:0/3763474811 >> 172.21.15.84:0/3763474811 conn(0x558103253410 msgr2=0x5581032537f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.376+0000 7f5826e00740 1 -- 172.21.15.84:0/3763474811 shutdown_connections 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.376+0000 7f5826e00740 1 -- 172.21.15.84:0/3763474811 wait complete. 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.376+0000 7f5826e00740 1 -- start start 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.376+0000 7f5826e00740 1 Processor -- start 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5826e00740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 0x55810325f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5826e00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558103257820 0x55810325f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5826e00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581032620e0 0x55810325fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5826e00740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558103264d10 con 0x558103256f00 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5826e00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558103144b60 con 0x558103257820 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5826e00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558103264e90 con 0x5581032620e0 2024-04-26T18:39:10.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5826e00740 10 client.0 _ll_register_callbacks cb 0x5581031692b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5821e86640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 0x55810325f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5821e86640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 0x55810325f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55678/0 (socket says 172.21.15.84:55678) 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5821e86640 1 -- 172.21.15.84:0/153078901 learned_addr learned my addr 172.21.15.84:0/153078901 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5822687640 1 --2- 172.21.15.84:0/153078901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581032620e0 0x55810325fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5821685640 1 --2- 172.21.15.84:0/153078901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558103257820 0x55810325f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5821e86640 1 -- 172.21.15.84:0/153078901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581032620e0 msgr2=0x55810325fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5821e86640 1 --2- 172.21.15.84:0/153078901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581032620e0 0x55810325fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5821e86640 1 -- 172.21.15.84:0/153078901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558103257820 msgr2=0x55810325f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5821e86640 1 --2- 172.21.15.84:0/153078901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558103257820 0x55810325f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5821e86640 1 -- 172.21.15.84:0/153078901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558102ecf210 con 0x558103256f00 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5821e86640 1 --2- 172.21.15.84:0/153078901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 0x55810325f040 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f581800c6a0 tx=0x7f5818010720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679cd6a3352d26a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f58057fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f58057fa640 1 -- 172.21.15.84:0/153078901 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58180a0dc0 con 0x558103256f00 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f58057fa640 1 -- 172.21.15.84:0/153078901 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8623+0+0 (secure 0 0 0) 0x7f58180bbbd0 con 0x558103256f00 2024-04-26T18:39:10.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.377+0000 7f5826e00740 1 -- 172.21.15.84:0/153078901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5814050070 con 0x558103256f00 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.378+0000 7f58057fa640 1 -- 172.21.15.84:0/153078901 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f58180a5680 con 0x558103256f00 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.378+0000 7f5826e00740 10 client.16773 fetch_fsmap learned FSMap version 29 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.378+0000 7f5826e00740 1 -- 172.21.15.84:0/153078901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558102f9b620 con 0x558103256f00 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.378+0000 7f58057fa640 1 -- 172.21.15.84:0/153078901 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5814050070 con 0x558103256f00 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.378+0000 7f58057fa640 1 -- 172.21.15.84:0/153078901 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5818098810 con 0x558103256f00 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.378+0000 7f5826e00740 10 client.16773 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.378+0000 7f5826e00740 10 client.16773 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.378+0000 7f5826e00740 20 client.16773 populate_metadata read hostname 'smithi084' 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.378+0000 7f5826e00740 1 --2- 172.21.15.84:0/153078901 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5581032721e0 0x5581032745f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.378+0000 7f5826e00740 4 client.16773 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.378+0000 7f5826e00740 1 -- 172.21.15.84:0/153078901 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5581031cb070 con 0x5581032721e0 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.378+0000 7f5821685640 1 --2- 172.21.15.84:0/153078901 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5581032721e0 0x5581032745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.379+0000 7f58057fa640 10 client.16773 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:10.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.379+0000 7f5821685640 1 --2- 172.21.15.84:0/153078901 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5581032721e0 0x5581032745f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e612e7f12e2b1ccc server_cookie=e87313f994803331 in_seq=0 out_seq=0 2024-04-26T18:39:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.379+0000 7f58057fa640 1 -- 172.21.15.84:0/153078901 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5581031cb070 con 0x5581032721e0 2024-04-26T18:39:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.379+0000 7f58057fa640 10 client.16773 handle_command_reply: tid=0 2024-04-26T18:39:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.380+0000 7f5826e00740 1 client.16773 shutdown 2024-04-26T18:39:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.380+0000 7f5826e00740 1 -- 172.21.15.84:0/153078901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 msgr2=0x55810325f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.380+0000 7f5826e00740 1 --2- 172.21.15.84:0/153078901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 0x55810325f040 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f581800c6a0 tx=0x7f5818010720 comp rx=0 tx=0).stop 2024-04-26T18:39:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.380+0000 7f5826e00740 1 -- 172.21.15.84:0/153078901 shutdown_connections 2024-04-26T18:39:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.380+0000 7f5826e00740 1 --2- 172.21.15.84:0/153078901 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5581032721e0 0x5581032745f0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.380+0000 7f5826e00740 1 --2- 172.21.15.84:0/153078901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581032620e0 0x55810325fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.380+0000 7f5826e00740 1 --2- 172.21.15.84:0/153078901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558103257820 0x55810325f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.380+0000 7f5826e00740 1 --2- 172.21.15.84:0/153078901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103256f00 0x55810325f040 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.380+0000 7f58057fa640 0 client.16773 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.380+0000 7f5826e00740 1 -- 172.21.15.84:0/153078901 >> 172.21.15.84:0/153078901 conn(0x558103253410 msgr2=0x558103259140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:10.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.380+0000 7f5826e00740 1 -- 172.21.15.84:0/153078901 shutdown_connections 2024-04-26T18:39:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.380+0000 7f5826e00740 1 -- 172.21.15.84:0/153078901 wait complete. 2024-04-26T18:39:10.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.380+0000 7f5826e00740 20 client.16773 trim_cache size 0 max 16384 2024-04-26T18:39:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5826e00740 1 Processor -- start 2024-04-26T18:39:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5826e00740 1 -- start start 2024-04-26T18:39:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5826e00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558103256f00 0x55810326dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5826e00740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 0x55810325f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5826e00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581032620e0 0x558103279f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5826e00740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f581c1b0820 con 0x558103257820 2024-04-26T18:39:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5826e00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f581c1b2360 con 0x5581032620e0 2024-04-26T18:39:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5826e00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f581c1b0ef0 con 0x558103256f00 2024-04-26T18:39:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5821685640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 0x55810325f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5821685640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 0x55810325f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55686/0 (socket says 172.21.15.84:55686) 2024-04-26T18:39:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5821685640 1 -- 172.21.15.84:0/3769506774 learned_addr learned my addr 172.21.15.84:0/3769506774 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:10.400 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5822687640 1 --2- 172.21.15.84:0/3769506774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581032620e0 0x558103279f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5821685640 1 -- 172.21.15.84:0/3769506774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558103256f00 msgr2=0x55810326dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5821685640 1 --2- 172.21.15.84:0/3769506774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558103256f00 0x55810326dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5821685640 1 -- 172.21.15.84:0/3769506774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581032620e0 msgr2=0x558103279f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5821685640 1 --2- 172.21.15.84:0/3769506774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581032620e0 0x558103279f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5821685640 1 -- 172.21.15.84:0/3769506774 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55810324ed60 con 0x558103257820 2024-04-26T18:39:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5822687640 1 --2- 172.21.15.84:0/3769506774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581032620e0 0x558103279f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:39:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.395+0000 7f5821685640 1 --2- 172.21.15.84:0/3769506774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 0x55810325f710 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b780 tx=0x7f580c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c602c0fea5d65d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.396+0000 7f58077fe640 1 -- 172.21.15.84:0/3769506774 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c003ca0 con 0x558103257820 2024-04-26T18:39:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.396+0000 7f58077fe640 1 -- 172.21.15.84:0/3769506774 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580c023020 con 0x558103257820 2024-04-26T18:39:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.396+0000 7f58077fe640 1 -- 172.21.15.84:0/3769506774 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c011730 con 0x558103257820 2024-04-26T18:39:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.396+0000 7f5826e00740 1 -- 172.21.15.84:0/3769506774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 msgr2=0x55810325f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:10.401 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.396+0000 7f5826e00740 1 --2- 172.21.15.84:0/3769506774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 0x55810325f710 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f580c00b780 tx=0x7f580c00b390 comp rx=0 tx=0).stop 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.396+0000 7f5826e00740 1 -- 172.21.15.84:0/3769506774 shutdown_connections 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.396+0000 7f5826e00740 1 --2- 172.21.15.84:0/3769506774 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5581032620e0 0x558103279f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.396+0000 7f5826e00740 1 --2- 172.21.15.84:0/3769506774 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 0x55810325f710 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.396+0000 7f5826e00740 1 --2- 172.21.15.84:0/3769506774 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558103256f00 0x55810326dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.396+0000 7f5826e00740 1 -- 172.21.15.84:0/3769506774 >> 172.21.15.84:0/3769506774 conn(0x558103253410 msgr2=0x55810325f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.396+0000 7f5826e00740 1 -- 172.21.15.84:0/3769506774 shutdown_connections 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.396+0000 7f5826e00740 1 -- 172.21.15.84:0/3769506774 wait complete. 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5826e00740 1 -- start start 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5826e00740 1 Processor -- start 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5826e00740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558103256f00 0x55810326a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5826e00740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 0x558103266660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5826e00740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581032620e0 0x558103266ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5826e00740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f581c1b1c90 con 0x558103257820 2024-04-26T18:39:10.402 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5826e00740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f581c1b15c0 con 0x558103256f00 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5826e00740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f581c1ae820 con 0x5581032620e0 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5826e00740 10 client.0 _ll_register_callbacks cb 0x7f5818009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5821685640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 0x558103266660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5821685640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 0x558103266660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55690/0 (socket says 172.21.15.84:55690) 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5821685640 1 -- 172.21.15.84:0/3043771238 learned_addr learned my addr 172.21.15.84:0/3043771238 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5821e86640 1 --2- 172.21.15.84:0/3043771238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558103256f00 0x55810326a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.397+0000 7f5822687640 1 --2- 172.21.15.84:0/3043771238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581032620e0 0x558103266ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5821685640 1 -- 172.21.15.84:0/3043771238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581032620e0 msgr2=0x558103266ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5821685640 1 --2- 172.21.15.84:0/3043771238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581032620e0 0x558103266ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5821685640 1 -- 172.21.15.84:0/3043771238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558103256f00 msgr2=0x55810326a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5821685640 1 --2- 172.21.15.84:0/3043771238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558103256f00 0x55810326a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5821685640 1 -- 172.21.15.84:0/3043771238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5581032671e0 con 0x558103257820 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5821685640 1 --2- 172.21.15.84:0/3043771238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 0x558103266660 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f580c005fa0 tx=0x7f580c003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd78de5a3ef2981 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:10.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f58057fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f58057fa640 1 -- 172.21.15.84:0/3043771238 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f580c005500 con 0x558103257820 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f58057fa640 1 -- 172.21.15.84:0/3043771238 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(105..105 src has 1..105) ==== 8623+0+0 (secure 0 0 0) 0x7f580c004730 con 0x558103257820 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5826e00740 1 -- 172.21.15.84:0/3043771238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f580c023020 con 0x558103257820 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f58057fa640 1 -- 172.21.15.84:0/3043771238 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f580c035050 con 0x558103257820 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5826e00740 10 client.16779 fetch_fsmap learned FSMap version 29 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5826e00740 1 -- 172.21.15.84:0/3043771238 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5581032d9530 con 0x558103257820 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f58057fa640 1 -- 172.21.15.84:0/3043771238 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f580c023020 con 0x558103257820 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f58057fa640 1 -- 172.21.15.84:0/3043771238 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f580c030030 con 0x558103257820 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5826e00740 10 client.16779 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5826e00740 10 client.16779 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5826e00740 20 client.16779 populate_metadata read hostname 'smithi084' 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5826e00740 1 --2- 172.21.15.84:0/3043771238 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5581032d9bf0 0x5581032dc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5826e00740 4 client.16779 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.398+0000 7f5826e00740 1 -- 172.21.15.84:0/3043771238 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5581031c4ee0 con 0x5581032d9bf0 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.399+0000 7f5821e86640 1 --2- 172.21.15.84:0/3043771238 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5581032d9bf0 0x5581032dc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:10.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.399+0000 7f5821e86640 1 --2- 172.21.15.84:0/3043771238 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5581032d9bf0 0x5581032dc010 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=328b1099c5b251a9 server_cookie=f5e8248b5fd1c133 in_seq=0 out_seq=0 2024-04-26T18:39:10.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.399+0000 7f58057fa640 10 client.16779 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.413+0000 7f58057fa640 1 -- 172.21.15.84:0/3043771238 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5581031c4ee0 con 0x5581032d9bf0 2024-04-26T18:39:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.413+0000 7f58057fa640 10 client.16779 handle_command_reply: tid=0 2024-04-26T18:39:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.413+0000 7f5826e00740 1 client.16779 shutdown 2024-04-26T18:39:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.413+0000 7f5826e00740 1 -- 172.21.15.84:0/3043771238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 msgr2=0x558103266660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.413+0000 7f5826e00740 1 --2- 172.21.15.84:0/3043771238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 0x558103266660 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f580c005fa0 tx=0x7f580c003e00 comp rx=0 tx=0).stop 2024-04-26T18:39:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.413+0000 7f5826e00740 1 -- 172.21.15.84:0/3043771238 shutdown_connections 2024-04-26T18:39:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.413+0000 7f5826e00740 1 --2- 172.21.15.84:0/3043771238 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5581032d9bf0 0x5581032dc010 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.413+0000 7f5826e00740 1 --2- 172.21.15.84:0/3043771238 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5581032620e0 0x558103266ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.413+0000 7f58057fa640 0 client.16779 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.413+0000 7f5826e00740 1 --2- 172.21.15.84:0/3043771238 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558103257820 0x558103266660 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.413+0000 7f5826e00740 1 --2- 172.21.15.84:0/3043771238 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558103256f00 0x55810326a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.414 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.413+0000 7f5826e00740 1 -- 172.21.15.84:0/3043771238 >> 172.21.15.84:0/3043771238 conn(0x558103253410 msgr2=0x558103259140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f5826e00740 1 -- 172.21.15.84:0/3043771238 shutdown_connections 2024-04-26T18:39:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f5826e00740 1 -- 172.21.15.84:0/3043771238 wait complete. 2024-04-26T18:39:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f5826e00740 20 client.16779 trim_cache size 0 max 16384 2024-04-26T18:39:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:39:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:39:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:39:10.415 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:39:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f58240ea640 1 -- 172.21.15.84:0/3892528580 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f58080a3dc0 msgr2=0x7f58080a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f58240ea640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f58080a3dc0 0x7f58080a6280 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f581404f000 tx=0x7f581404d000 comp rx=0 tx=0).stop 2024-04-26T18:39:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f58240ea640 1 -- 172.21.15.84:0/3892528580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f581c102940 msgr2=0x7f581c074c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f58240ea640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f581c102940 0x7f581c074c30 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f581800e9f0 tx=0x7f581800eeb0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f58240ea640 1 -- 172.21.15.84:0/3892528580 shutdown_connections 2024-04-26T18:39:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f58240ea640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f58080a3dc0 0x7f58080a6280 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f58240ea640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f581c108950 0x7f581c0756b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f58240ea640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f581c103300 0x7f581c075170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.416 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f58240ea640 1 --2- 172.21.15.84:0/3892528580 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f581c102940 0x7f581c074c30 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f58240ea640 1 -- 172.21.15.84:0/3892528580 >> 172.21.15.84:0/3892528580 conn(0x7f581c079dc0 msgr2=0x7f581c100150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f58240ea640 1 -- 172.21.15.84:0/3892528580 shutdown_connections 2024-04-26T18:39:10.417 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:10.414+0000 7f58240ea640 1 -- 172.21.15.84:0/3892528580 wait complete. 2024-04-26T18:39:11.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:10 smithi084 ceph-mon[29141]: pgmap v350: 97 pgs: 97 active+clean; 5.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 100 KiB/s wr, 37 op/s 2024-04-26T18:39:11.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:10 smithi103 ceph-mon[32727]: pgmap v350: 97 pgs: 97 active+clean; 5.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 100 KiB/s wr, 37 op/s 2024-04-26T18:39:11.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:10 smithi174 ceph-mon[32705]: pgmap v350: 97 pgs: 97 active+clean; 5.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 100 KiB/s wr, 37 op/s 2024-04-26T18:39:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:13 smithi174 ceph-mon[32705]: pgmap v351: 97 pgs: 97 active+clean; 5.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 100 KiB/s wr, 37 op/s 2024-04-26T18:39:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:13 smithi174 ceph-mon[32705]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T18:39:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:13 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:39:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:13 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:39:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:13 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:39:13.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:13 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:39:13.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:13 smithi084 ceph-mon[29141]: pgmap v351: 97 pgs: 97 active+clean; 5.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 100 KiB/s wr, 37 op/s 2024-04-26T18:39:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:13 smithi084 ceph-mon[29141]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T18:39:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:13 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:39:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:13 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:39:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:13 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:39:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:13 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:39:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:13 smithi103 ceph-mon[32727]: pgmap v351: 97 pgs: 97 active+clean; 5.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 100 KiB/s wr, 37 op/s 2024-04-26T18:39:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:13 smithi103 ceph-mon[32727]: osdmap e106: 12 total, 12 up, 12 in 2024-04-26T18:39:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:13 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:39:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:13 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:39:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:13 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:39:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:13 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:39:15.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:15 smithi174 ceph-mon[32705]: pgmap v353: 97 pgs: 97 active+clean; 1.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:39:15.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:15 smithi084 ceph-mon[29141]: pgmap v353: 97 pgs: 97 active+clean; 1.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:39:15.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:15 smithi103 ceph-mon[32727]: pgmap v353: 97 pgs: 97 active+clean; 1.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:39:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:17 smithi084 ceph-mon[29141]: pgmap v354: 97 pgs: 97 active+clean; 1.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-26T18:39:17.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:17 smithi103 ceph-mon[32727]: pgmap v354: 97 pgs: 97 active+clean; 1.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-26T18:39:17.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:17 smithi174 ceph-mon[32705]: pgmap v354: 97 pgs: 97 active+clean; 1.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-26T18:39:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:19 smithi174 ceph-mon[32705]: pgmap v355: 97 pgs: 97 active+clean; 1.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-26T18:39:19.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:19 smithi084 ceph-mon[29141]: pgmap v355: 97 pgs: 97 active+clean; 1.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-26T18:39:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:19 smithi103 ceph-mon[32727]: pgmap v355: 97 pgs: 97 active+clean; 1.8 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 52 KiB/s wr, 19 op/s 2024-04-26T18:39:21.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:21 smithi084 ceph-mon[29141]: pgmap v356: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:39:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:21 smithi103 ceph-mon[32727]: pgmap v356: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:39:21.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:21 smithi174 ceph-mon[32705]: pgmap v356: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:39:23.513 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.511+0000 7f4e9ab9d640 1 -- 172.21.15.84:0/898174812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e94101850 msgr2=0x7f4e94101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.511+0000 7f4e9ab9d640 1 --2- 172.21.15.84:0/898174812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e94101850 0x7f4e94101cd0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009930 tx=0x7f4e8002a420 comp rx=0 tx=0).stop 2024-04-26T18:39:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.511+0000 7f4e9ab9d640 1 -- 172.21.15.84:0/898174812 shutdown_connections 2024-04-26T18:39:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.511+0000 7f4e9ab9d640 1 --2- 172.21.15.84:0/898174812 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e94102210 0x7f4e9410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.511+0000 7f4e9ab9d640 1 --2- 172.21.15.84:0/898174812 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e94101850 0x7f4e94101cd0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.511+0000 7f4e9ab9d640 1 --2- 172.21.15.84:0/898174812 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e94107860 0x7f4e94107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.511+0000 7f4e9ab9d640 1 -- 172.21.15.84:0/898174812 >> 172.21.15.84:0/898174812 conn(0x7f4e94077bb0 msgr2=0x7f4e94077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.511+0000 7f4e9ab9d640 1 -- 172.21.15.84:0/898174812 shutdown_connections 2024-04-26T18:39:23.514 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.511+0000 7f4e9ab9d640 1 -- 172.21.15.84:0/898174812 wait complete. 2024-04-26T18:39:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.512+0000 7f4e9ab9d640 1 Processor -- start 2024-04-26T18:39:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.512+0000 7f4e9ab9d640 1 -- start start 2024-04-26T18:39:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.512+0000 7f4e9ab9d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e94101850 0x7f4e94199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.512+0000 7f4e9ab9d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e94102210 0x7f4e9419a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.512+0000 7f4e9ab9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e94107860 0x7f4e941a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.512+0000 7f4e9ab9d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4e94111b20 con 0x7f4e94107860 2024-04-26T18:39:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.512+0000 7f4e9ab9d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e941119a0 con 0x7f4e94101850 2024-04-26T18:39:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.512+0000 7f4e9ab9d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4e94111ca0 con 0x7f4e94102210 2024-04-26T18:39:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.512+0000 7f4e9913a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e94107860 0x7f4e941a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.512+0000 7f4e9913a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e94107860 0x7f4e941a4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37954/0 (socket says 172.21.15.84:37954) 2024-04-26T18:39:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.512+0000 7f4e9913a640 1 -- 172.21.15.84:0/3024248614 learned_addr learned my addr 172.21.15.84:0/3024248614 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:23.515 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.512+0000 7f4e93fff640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e94102210 0x7f4e9419a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.512+0000 7f4e98939640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e94101850 0x7f4e94199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.513+0000 7f4e9913a640 1 -- 172.21.15.84:0/3024248614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e94102210 msgr2=0x7f4e9419a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.513+0000 7f4e9913a640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e94102210 0x7f4e9419a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.513+0000 7f4e9913a640 1 -- 172.21.15.84:0/3024248614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e94101850 msgr2=0x7f4e94199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.513+0000 7f4e9913a640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e94101850 0x7f4e94199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.513+0000 7f4e9913a640 1 -- 172.21.15.84:0/3024248614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e941a5500 con 0x7f4e94107860 2024-04-26T18:39:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.513+0000 7f4e9913a640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e94107860 0x7f4e941a4fc0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00cad0 tx=0x7f4e8c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.513+0000 7f4e91ffb640 1 -- 172.21.15.84:0/3024248614 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00ae20 con 0x7f4e94107860 2024-04-26T18:39:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.513+0000 7f4e91ffb640 1 -- 172.21.15.84:0/3024248614 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e8c005020 con 0x7f4e94107860 2024-04-26T18:39:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.513+0000 7f4e9ab9d640 1 -- 172.21.15.84:0/3024248614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e941a56d0 con 0x7f4e94107860 2024-04-26T18:39:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.513+0000 7f4e91ffb640 1 -- 172.21.15.84:0/3024248614 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c00f6b0 con 0x7f4e94107860 2024-04-26T18:39:23.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.513+0000 7f4e9ab9d640 1 -- 172.21.15.84:0/3024248614 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e941a5bf0 con 0x7f4e94107860 2024-04-26T18:39:23.523 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.516+0000 7f4e91ffb640 1 -- 172.21.15.84:0/3024248614 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4e8c0051d0 con 0x7f4e94107860 2024-04-26T18:39:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.516+0000 7f4e9d991740 1 Processor -- start 2024-04-26T18:39:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.516+0000 7f4e9d991740 1 -- start start 2024-04-26T18:39:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.516+0000 7f4e9d991740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b7f00 0x5615b84b82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.516+0000 7f4e9d991740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84b8820 0x5615b84c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.516+0000 7f4e9d991740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84c30e0 0x5615b84c54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.516+0000 7f4e9d991740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5615b841e460 con 0x5615b84c30e0 2024-04-26T18:39:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.516+0000 7f4e9d991740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615b83d8590 con 0x5615b84b7f00 2024-04-26T18:39:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.516+0000 7f4e9d991740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615b841a420 con 0x5615b84b8820 2024-04-26T18:39:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84c30e0 0x5615b84c54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84c30e0 0x5615b84c54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37960/0 (socket says 172.21.15.84:37960) 2024-04-26T18:39:23.524 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e93fff640 1 -- 172.21.15.84:0/240200043 learned_addr learned my addr 172.21.15.84:0/240200043 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e91ffb640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4e840a3e40 0x7f4e840a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e93fff640 1 --2- 172.21.15.84:0/240200043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b7f00 0x5615b84b82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e98939640 1 --2- 172.21.15.84:0/240200043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84b8820 0x5615b84c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e9913a640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4e840a3e40 0x7f4e840a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e93fff640 1 -- 172.21.15.84:0/240200043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84b8820 msgr2=0x5615b84c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e93fff640 1 --2- 172.21.15.84:0/240200043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84b8820 0x5615b84c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e93fff640 1 -- 172.21.15.84:0/240200043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b7f00 msgr2=0x5615b84b82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e93fff640 1 --2- 172.21.15.84:0/240200043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b7f00 0x5615b84b82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e93fff640 1 -- 172.21.15.84:0/240200043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615b83e50d0 con 0x5615b84c30e0 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e91ffb640 1 -- 172.21.15.84:0/3024248614 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f4e8c0d0890 con 0x7f4e94107860 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e93fff640 1 --2- 172.21.15.84:0/240200043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84c30e0 0x5615b84c54d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009930 tx=0x7f4e80009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3aa8740b2842146 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e7effd640 1 -- 172.21.15.84:0/240200043 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80035070 con 0x5615b84c30e0 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e7effd640 1 -- 172.21.15.84:0/240200043 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e80031780 con 0x5615b84c30e0 2024-04-26T18:39:23.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e7effd640 1 -- 172.21.15.84:0/240200043 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e800047a0 con 0x5615b84c30e0 2024-04-26T18:39:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.517+0000 7f4e9913a640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4e840a3e40 0x7f4e840a6300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0a4f30 tx=0x7f4e8c005890 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.518+0000 7f4e9d991740 1 -- 172.21.15.84:0/240200043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84c30e0 msgr2=0x5615b84c54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.518+0000 7f4e9d991740 1 --2- 172.21.15.84:0/240200043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84c30e0 0x5615b84c54d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009930 tx=0x7f4e80009560 comp rx=0 tx=0).stop 2024-04-26T18:39:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.518+0000 7f4e9d991740 1 -- 172.21.15.84:0/240200043 shutdown_connections 2024-04-26T18:39:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.518+0000 7f4e9d991740 1 --2- 172.21.15.84:0/240200043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84c30e0 0x5615b84c54d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.518+0000 7f4e9d991740 1 --2- 172.21.15.84:0/240200043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84b8820 0x5615b84c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.518+0000 7f4e9d991740 1 --2- 172.21.15.84:0/240200043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b7f00 0x5615b84b82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.518+0000 7f4e9d991740 1 -- 172.21.15.84:0/240200043 >> 172.21.15.84:0/240200043 conn(0x5615b84b4410 msgr2=0x5615b84b47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.518+0000 7f4e9d991740 1 -- 172.21.15.84:0/240200043 shutdown_connections 2024-04-26T18:39:23.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.518+0000 7f4e9d991740 1 -- 172.21.15.84:0/240200043 wait complete. 2024-04-26T18:39:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.518+0000 7f4e9d991740 1 -- start start 2024-04-26T18:39:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e9d991740 1 Processor -- start 2024-04-26T18:39:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e9d991740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84c0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e9d991740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 0x5615b84c05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e9d991740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 0x5615b84c0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e9d991740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5615b84c5cb0 con 0x5615b84b7f00 2024-04-26T18:39:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e9d991740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5615b83a5b60 con 0x5615b84b8820 2024-04-26T18:39:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e9d991740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5615b84c5e30 con 0x5615b84c30e0 2024-04-26T18:39:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e9d991740 10 client.0 _ll_register_callbacks cb 0x5615b83ca2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:39:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84c0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84c0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37976/0 (socket says 172.21.15.84:37976) 2024-04-26T18:39:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e93fff640 1 -- 172.21.15.84:0/2799892030 learned_addr learned my addr 172.21.15.84:0/2799892030 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:23.527 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e98939640 1 --2- 172.21.15.84:0/2799892030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 0x5615b84c05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e93fff640 1 --2- 172.21.15.84:0/2799892030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 0x5615b84c0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e93fff640 1 -- 172.21.15.84:0/2799892030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 msgr2=0x5615b84c0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e93fff640 1 --2- 172.21.15.84:0/2799892030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 0x5615b84c0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e93fff640 1 -- 172.21.15.84:0/2799892030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 msgr2=0x5615b84c05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e93fff640 1 --2- 172.21.15.84:0/2799892030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 0x5615b84c05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.519+0000 7f4e93fff640 1 -- 172.21.15.84:0/2799892030 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615b8130210 con 0x5615b84b7f00 2024-04-26T18:39:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e93fff640 1 --2- 172.21.15.84:0/2799892030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84c0080 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4e8002a940 tx=0x7f4e800054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978a860630a2785e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e73fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:39:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e73fff640 1 -- 172.21.15.84:0/2799892030 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80000e60 con 0x5615b84b7f00 2024-04-26T18:39:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e73fff640 1 -- 172.21.15.84:0/2799892030 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f4e80004250 con 0x5615b84b7f00 2024-04-26T18:39:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e9d991740 1 -- 172.21.15.84:0/2799892030 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e80031780 con 0x5615b84b7f00 2024-04-26T18:39:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e73fff640 1 -- 172.21.15.84:0/2799892030 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4e8005b050 con 0x5615b84b7f00 2024-04-26T18:39:23.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e9d991740 10 client.16791 fetch_fsmap learned FSMap version 29 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e9d991740 1 -- 172.21.15.84:0/2799892030 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5615b81fc620 con 0x5615b84b7f00 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e73fff640 1 -- 172.21.15.84:0/2799892030 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e80031780 con 0x5615b84b7f00 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e73fff640 1 -- 172.21.15.84:0/2799892030 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4e80003710 con 0x5615b84b7f00 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e9d991740 10 client.16791 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e9d991740 10 client.16791 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e9d991740 20 client.16791 populate_metadata read hostname 'smithi084' 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e9d991740 1 --2- 172.21.15.84:0/2799892030 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5615b84d3070 0x5615b84d5460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e9d991740 4 client.16791 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.520+0000 7f4e9d991740 1 -- 172.21.15.84:0/2799892030 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5615b842c070 con 0x5615b84d3070 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.521+0000 7f4e98939640 1 --2- 172.21.15.84:0/2799892030 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5615b84d3070 0x5615b84d5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.521+0000 7f4e98939640 1 --2- 172.21.15.84:0/2799892030 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5615b84d3070 0x5615b84d5460 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49df1b5d057fd241 server_cookie=67ce64491efc0ad3 in_seq=0 out_seq=0 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.521+0000 7f4e73fff640 10 client.16791 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e73fff640 1 -- 172.21.15.84:0/2799892030 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5615b842c070 con 0x5615b84d3070 2024-04-26T18:39:23.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e73fff640 10 client.16791 handle_command_reply: tid=0 2024-04-26T18:39:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e9d991740 1 client.16791 shutdown 2024-04-26T18:39:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e9d991740 1 -- 172.21.15.84:0/2799892030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 msgr2=0x5615b84c0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e9d991740 1 --2- 172.21.15.84:0/2799892030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84c0080 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f4e8002a940 tx=0x7f4e800054d0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e9d991740 1 -- 172.21.15.84:0/2799892030 shutdown_connections 2024-04-26T18:39:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e9d991740 1 --2- 172.21.15.84:0/2799892030 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5615b84d3070 0x5615b84d5460 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e9d991740 1 --2- 172.21.15.84:0/2799892030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 0x5615b84c0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e9d991740 1 --2- 172.21.15.84:0/2799892030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 0x5615b84c05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e73fff640 0 client.16791 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e9d991740 1 --2- 172.21.15.84:0/2799892030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84c0080 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e9d991740 1 -- 172.21.15.84:0/2799892030 >> 172.21.15.84:0/2799892030 conn(0x5615b84b4410 msgr2=0x5615b84ba140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e9d991740 1 -- 172.21.15.84:0/2799892030 shutdown_connections 2024-04-26T18:39:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e9d991740 1 -- 172.21.15.84:0/2799892030 wait complete. 2024-04-26T18:39:23.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.522+0000 7f4e9d991740 20 client.16791 trim_cache size 0 max 16384 2024-04-26T18:39:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.537+0000 7f4e9d991740 1 Processor -- start 2024-04-26T18:39:23.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.537+0000 7f4e9d991740 1 -- start start 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e9d991740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84ceb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e9d991740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 0x5615b84ba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e9d991740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 0x5615b84daa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e9d991740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4e941ae5b0 con 0x5615b84b7f00 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e9d991740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e941b00f0 con 0x5615b84b8820 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e9d991740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4e941aec80 con 0x5615b84c30e0 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84ceb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84ceb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37988/0 (socket says 172.21.15.84:37988) 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e93fff640 1 -- 172.21.15.84:0/4092725256 learned_addr learned my addr 172.21.15.84:0/4092725256 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e98939640 1 --2- 172.21.15.84:0/4092725256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 0x5615b84ba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e93fff640 1 -- 172.21.15.84:0/4092725256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 msgr2=0x5615b84daa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e93fff640 1 --2- 172.21.15.84:0/4092725256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 0x5615b84daa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e93fff640 1 -- 172.21.15.84:0/4092725256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 msgr2=0x5615b84ba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:23.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e93fff640 1 --2- 172.21.15.84:0/4092725256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 0x5615b84ba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e93fff640 1 -- 172.21.15.84:0/4092725256 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615b84bae00 con 0x5615b84b7f00 2024-04-26T18:39:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e93fff640 1 --2- 172.21.15.84:0/4092725256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84ceb70 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009850 tx=0x7f4e80031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7c7a69fc9fcd3f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e7e7fc640 1 -- 172.21.15.84:0/4092725256 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80050070 con 0x5615b84b7f00 2024-04-26T18:39:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e7e7fc640 1 -- 172.21.15.84:0/4092725256 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e80031700 con 0x5615b84b7f00 2024-04-26T18:39:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.538+0000 7f4e7e7fc640 1 -- 172.21.15.84:0/4092725256 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80035040 con 0x5615b84b7f00 2024-04-26T18:39:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.539+0000 7f4e9d991740 1 -- 172.21.15.84:0/4092725256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 msgr2=0x5615b84ceb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.539+0000 7f4e9d991740 1 --2- 172.21.15.84:0/4092725256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84ceb70 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f4e80009850 tx=0x7f4e80031400 comp rx=0 tx=0).stop 2024-04-26T18:39:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.539+0000 7f4e9d991740 1 -- 172.21.15.84:0/4092725256 shutdown_connections 2024-04-26T18:39:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.539+0000 7f4e9d991740 1 --2- 172.21.15.84:0/4092725256 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 0x5615b84daa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.539+0000 7f4e9d991740 1 --2- 172.21.15.84:0/4092725256 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 0x5615b84ba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.539+0000 7f4e9d991740 1 --2- 172.21.15.84:0/4092725256 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84ceb70 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.539+0000 7f4e9d991740 1 -- 172.21.15.84:0/4092725256 >> 172.21.15.84:0/4092725256 conn(0x5615b84b4410 msgr2=0x5615b84b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.539+0000 7f4e9d991740 1 -- 172.21.15.84:0/4092725256 shutdown_connections 2024-04-26T18:39:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.539+0000 7f4e9d991740 1 -- 172.21.15.84:0/4092725256 wait complete. 2024-04-26T18:39:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.539+0000 7f4e9d991740 1 -- start start 2024-04-26T18:39:23.542 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.539+0000 7f4e9d991740 1 Processor -- start 2024-04-26T18:39:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e9d991740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e9d991740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 0x5615b84cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e9d991740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 0x5615b84c8050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e9d991740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4e941afa20 con 0x5615b84b7f00 2024-04-26T18:39:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e9d991740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4e941af350 con 0x5615b84b8820 2024-04-26T18:39:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e9d991740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4e941ac5b0 con 0x5615b84c30e0 2024-04-26T18:39:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e9d991740 10 client.0 _ll_register_callbacks cb 0x7f4e80031530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:39:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e93fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84cb080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37998/0 (socket says 172.21.15.84:37998) 2024-04-26T18:39:23.545 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e93fff640 1 -- 172.21.15.84:0/2436587543 learned_addr learned my addr 172.21.15.84:0/2436587543 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e98939640 1 --2- 172.21.15.84:0/2436587543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 0x5615b84cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e93fff640 1 --2- 172.21.15.84:0/2436587543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 0x5615b84c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e93fff640 1 -- 172.21.15.84:0/2436587543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 msgr2=0x5615b84c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e93fff640 1 --2- 172.21.15.84:0/2436587543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 0x5615b84c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e93fff640 1 -- 172.21.15.84:0/2436587543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 msgr2=0x5615b84cb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e93fff640 1 --2- 172.21.15.84:0/2436587543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 0x5615b84cb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e93fff640 1 -- 172.21.15.84:0/2436587543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615b84c8690 con 0x5615b84b7f00 2024-04-26T18:39:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e93fff640 1 --2- 172.21.15.84:0/2436587543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84cb080 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4e80003ea0 tx=0x7f4e8002a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97f75c0a819c61f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.540+0000 7f4e73fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:39:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e73fff640 1 -- 172.21.15.84:0/2436587543 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e80058070 con 0x5615b84b7f00 2024-04-26T18:39:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e73fff640 1 -- 172.21.15.84:0/2436587543 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f4e80003800 con 0x5615b84b7f00 2024-04-26T18:39:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e9d991740 1 -- 172.21.15.84:0/2436587543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e80031700 con 0x5615b84b7f00 2024-04-26T18:39:23.546 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e73fff640 1 -- 172.21.15.84:0/2436587543 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4e80035070 con 0x5615b84b7f00 2024-04-26T18:39:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e9d991740 10 client.16797 fetch_fsmap learned FSMap version 29 2024-04-26T18:39:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e9d991740 1 -- 172.21.15.84:0/2436587543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5615b8539c00 con 0x5615b84b7f00 2024-04-26T18:39:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e73fff640 1 -- 172.21.15.84:0/2436587543 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e80031700 con 0x5615b84b7f00 2024-04-26T18:39:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e73fff640 1 -- 172.21.15.84:0/2436587543 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4e80069070 con 0x5615b84b7f00 2024-04-26T18:39:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e9d991740 10 client.16797 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:39:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e9d991740 10 client.16797 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:39:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e9d991740 20 client.16797 populate_metadata read hostname 'smithi084' 2024-04-26T18:39:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e9d991740 1 --2- 172.21.15.84:0/2436587543 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5615b853a2a0 0x5615b853c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e9d991740 4 client.16797 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:39:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e9d991740 1 -- 172.21.15.84:0/2436587543 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5615b8425ee0 con 0x5615b853a2a0 2024-04-26T18:39:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.541+0000 7f4e98939640 1 --2- 172.21.15.84:0/2436587543 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5615b853a2a0 0x5615b853c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.542+0000 7f4e73fff640 10 client.16797 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:23.547 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.542+0000 7f4e98939640 1 --2- 172.21.15.84:0/2436587543 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5615b853a2a0 0x5615b853c6c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa371b4486e5c218 server_cookie=79bdfe75ea23b53b in_seq=0 out_seq=0 2024-04-26T18:39:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e73fff640 1 -- 172.21.15.84:0/2436587543 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5615b8425ee0 con 0x5615b853a2a0 2024-04-26T18:39:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e73fff640 10 client.16797 handle_command_reply: tid=0 2024-04-26T18:39:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e9d991740 1 client.16797 shutdown 2024-04-26T18:39:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e9d991740 1 -- 172.21.15.84:0/2436587543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 msgr2=0x5615b84cb080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e9d991740 1 --2- 172.21.15.84:0/2436587543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84cb080 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f4e80003ea0 tx=0x7f4e8002a420 comp rx=0 tx=0).stop 2024-04-26T18:39:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e9d991740 1 -- 172.21.15.84:0/2436587543 shutdown_connections 2024-04-26T18:39:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e9d991740 1 --2- 172.21.15.84:0/2436587543 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5615b853a2a0 0x5615b853c6c0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e9d991740 1 --2- 172.21.15.84:0/2436587543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5615b84c30e0 0x5615b84c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e9d991740 1 --2- 172.21.15.84:0/2436587543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5615b84b8820 0x5615b84cb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e9d991740 1 --2- 172.21.15.84:0/2436587543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5615b84b7f00 0x5615b84cb080 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e73fff640 0 client.16797 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e9d991740 1 -- 172.21.15.84:0/2436587543 >> 172.21.15.84:0/2436587543 conn(0x5615b84b4410 msgr2=0x5615b84c0040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e9d991740 1 -- 172.21.15.84:0/2436587543 shutdown_connections 2024-04-26T18:39:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.555+0000 7f4e9d991740 1 -- 172.21.15.84:0/2436587543 wait complete. 2024-04-26T18:39:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.556+0000 7f4e9d991740 20 client.16797 trim_cache size 0 max 16384 2024-04-26T18:39:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:39:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:39:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:39:23.557 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:39:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.556+0000 7f4e9ab9d640 1 -- 172.21.15.84:0/3024248614 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4e840a3e40 msgr2=0x7f4e840a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.556+0000 7f4e9ab9d640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4e840a3e40 0x7f4e840a6300 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c0a4f30 tx=0x7f4e8c005890 comp rx=0 tx=0).stop 2024-04-26T18:39:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.556+0000 7f4e9ab9d640 1 -- 172.21.15.84:0/3024248614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e94107860 msgr2=0x7f4e941a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.556+0000 7f4e9ab9d640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e94107860 0x7f4e941a4fc0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00cad0 tx=0x7f4e8c00cf90 comp rx=0 tx=0).stop 2024-04-26T18:39:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.556+0000 7f4e9ab9d640 1 -- 172.21.15.84:0/3024248614 shutdown_connections 2024-04-26T18:39:23.558 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.556+0000 7f4e9ab9d640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4e840a3e40 0x7f4e840a6300 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.556+0000 7f4e9ab9d640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4e94107860 0x7f4e941a4fc0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.556+0000 7f4e9ab9d640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4e94102210 0x7f4e9419a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.556+0000 7f4e9ab9d640 1 --2- 172.21.15.84:0/3024248614 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4e94101850 0x7f4e94199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.556+0000 7f4e9ab9d640 1 -- 172.21.15.84:0/3024248614 >> 172.21.15.84:0/3024248614 conn(0x7f4e94077bb0 msgr2=0x7f4e940ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.556+0000 7f4e9ab9d640 1 -- 172.21.15.84:0/3024248614 shutdown_connections 2024-04-26T18:39:23.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:23.556+0000 7f4e9ab9d640 1 -- 172.21.15.84:0/3024248614 wait complete. 2024-04-26T18:39:23.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:23 smithi084 ceph-mon[29141]: pgmap v357: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:39:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:23 smithi103 ceph-mon[32727]: pgmap v357: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:39:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:23 smithi174 ceph-mon[32705]: pgmap v357: 97 pgs: 97 active+clean; 2.0 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:39:25.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:25 smithi084 ceph-mon[29141]: pgmap v358: 97 pgs: 97 active+clean; 2.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-26T18:39:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:25 smithi103 ceph-mon[32727]: pgmap v358: 97 pgs: 97 active+clean; 2.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-26T18:39:25.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:25 smithi174 ceph-mon[32705]: pgmap v358: 97 pgs: 97 active+clean; 2.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-26T18:39:27.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:27 smithi084 ceph-mon[29141]: pgmap v359: 97 pgs: 97 active+clean; 2.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:39:27.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:27 smithi103 ceph-mon[32727]: pgmap v359: 97 pgs: 97 active+clean; 2.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:39:27.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:27 smithi174 ceph-mon[32705]: pgmap v359: 97 pgs: 97 active+clean; 2.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:39:29.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:29 smithi084 ceph-mon[29141]: pgmap v360: 97 pgs: 97 active+clean; 2.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:39:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:29 smithi103 ceph-mon[32727]: pgmap v360: 97 pgs: 97 active+clean; 2.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:39:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:29 smithi174 ceph-mon[32705]: pgmap v360: 97 pgs: 97 active+clean; 2.2 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:39:31.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:31 smithi084 ceph-mon[29141]: pgmap v361: 97 pgs: 97 active+clean; 2.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-26T18:39:31.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:31 smithi103 ceph-mon[32727]: pgmap v361: 97 pgs: 97 active+clean; 2.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-26T18:39:31.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:31 smithi174 ceph-mon[32705]: pgmap v361: 97 pgs: 97 active+clean; 2.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-26T18:39:32.375 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.375 INFO:tasks.workunit.client.0.smithi084.stdout:313/600: open fd = 3 2024-04-26T18:39:32.375 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.375 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.375 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.375 INFO:tasks.workunit.client.0.smithi084.stdout:314/600: open fd = 3 2024-04-26T18:39:32.375 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.375 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.375 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.375 INFO:tasks.workunit.client.0.smithi084.stdout:315/600: open fd = 3 2024-04-26T18:39:32.375 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:316/600: open fd = 3 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:317/600: open fd = 3 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:318/600: open fd = 3 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.376 INFO:tasks.workunit.client.0.smithi084.stdout:319/600: open fd = 3 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:320/600: open fd = 3 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:321/600: open fd = 3 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:322/600: open fd = 3 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.377 INFO:tasks.workunit.client.0.smithi084.stdout:323/600: open fd = 3 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:324/600: open fd = 3 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:325/600: open fd = 3 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:326/600: open fd = 3 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.378 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.379 INFO:tasks.workunit.client.0.smithi084.stdout:327/600: open fd = 3 2024-04-26T18:39:32.379 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.379 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.379 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.379 INFO:tasks.workunit.client.0.smithi084.stdout:328/600: open fd = 3 2024-04-26T18:39:32.379 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.379 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.379 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.379 INFO:tasks.workunit.client.0.smithi084.stdout:329/600: open fd = 3 2024-04-26T18:39:32.379 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.379 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:330/600: open fd = 3 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:331/600: open fd = 3 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:332/600: open fd = 3 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.380 INFO:tasks.workunit.client.0.smithi084.stdout:333/600: open fd = 3 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:334/600: open fd = 3 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:335/600: open fd = 3 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:336/600: open fd = 3 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.381 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:337/600: open fd = 3 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:338/600: open fd = 3 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:339/600: open fd = 3 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:340/600: open fd = 3 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.382 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:341/600: open fd = 3 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:342/600: open fd = 3 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:343/600: open fd = 3 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:344/600: open fd = 3 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.383 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:345/600: open fd = 3 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:346/600: open fd = 3 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:347/600: open fd = 3 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:348/600: open fd = 3 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.384 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:349/600: open fd = 3 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:350/600: open fd = 3 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:351/600: open fd = 3 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:352/600: open fd = 3 2024-04-26T18:39:32.385 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:353/600: open fd = 3 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:354/600: open fd = 3 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:355/600: open fd = 3 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.386 INFO:tasks.workunit.client.0.smithi084.stdout:356/600: open fd = 3 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:357/600: open fd = 3 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:358/600: open fd = 3 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:359/600: open fd = 3 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.387 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:360/600: open fd = 3 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:361/600: open fd = 3 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:362/600: open fd = 3 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:363/600: open fd = 3 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.388 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:364/600: open fd = 3 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:365/600: open fd = 3 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:366/600: open fd = 3 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:367/600: open fd = 3 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.389 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:368/600: open fd = 3 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:369/600: open fd = 3 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:370/600: open fd = 3 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:371/600: open fd = 3 2024-04-26T18:39:32.390 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:372/600: open fd = 3 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:373/600: open fd = 3 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:374/600: open fd = 3 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:39:32.391 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:39:33.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:33 smithi084 ceph-mon[29141]: pgmap v362: 97 pgs: 97 active+clean; 2.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:39:33.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:33 smithi103 ceph-mon[32727]: pgmap v362: 97 pgs: 97 active+clean; 2.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:39:33.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:33 smithi174 ceph-mon[32705]: pgmap v362: 97 pgs: 97 active+clean; 2.4 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:39:35.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:35 smithi084 ceph-mon[29141]: pgmap v363: 97 pgs: 97 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:39:35.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:35 smithi103 ceph-mon[32727]: pgmap v363: 97 pgs: 97 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:39:35.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:35 smithi174 ceph-mon[32705]: pgmap v363: 97 pgs: 97 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.649+0000 7f3efc0c8640 1 -- 172.21.15.84:0/4042256829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef4101850 msgr2=0x7f3ef4101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.649+0000 7f3efc0c8640 1 --2- 172.21.15.84:0/4042256829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef4101850 0x7f3ef4101cd0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0009930 tx=0x7f3ef002a420 comp rx=0 tx=0).stop 2024-04-26T18:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.649+0000 7f3efc0c8640 1 -- 172.21.15.84:0/4042256829 shutdown_connections 2024-04-26T18:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.649+0000 7f3efc0c8640 1 --2- 172.21.15.84:0/4042256829 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ef4102210 0x7f3ef410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.649+0000 7f3efc0c8640 1 --2- 172.21.15.84:0/4042256829 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef4101850 0x7f3ef4101cd0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.649+0000 7f3efc0c8640 1 --2- 172.21.15.84:0/4042256829 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ef4107860 0x7f3ef4107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.649+0000 7f3efc0c8640 1 -- 172.21.15.84:0/4042256829 >> 172.21.15.84:0/4042256829 conn(0x7f3ef4077bb0 msgr2=0x7f3ef4077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.649+0000 7f3efc0c8640 1 -- 172.21.15.84:0/4042256829 shutdown_connections 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.649+0000 7f3efc0c8640 1 -- 172.21.15.84:0/4042256829 wait complete. 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.649+0000 7f3efc0c8640 1 Processor -- start 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3efc0c8640 1 -- start start 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3efc0c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ef4101850 0x7f3ef410f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3efc0c8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef4102210 0x7f3ef410f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3efc0c8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ef4107860 0x7f3ef410fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3efc0c8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4112c20 con 0x7f3ef4102210 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3efc0c8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4112aa0 con 0x7f3ef4101850 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3efc0c8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef4112da0 con 0x7f3ef4107860 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3ef9663640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef4102210 0x7f3ef410f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3ef9663640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef4102210 0x7f3ef410f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45058/0 (socket says 172.21.15.84:45058) 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3ef9663640 1 -- 172.21.15.84:0/3541623686 learned_addr learned my addr 172.21.15.84:0/3541623686 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3efa665640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ef4107860 0x7f3ef410fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3ef9e64640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ef4101850 0x7f3ef410f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3ef9663640 1 -- 172.21.15.84:0/3541623686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ef4107860 msgr2=0x7f3ef410fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3ef9663640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ef4107860 0x7f3ef410fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3ef9663640 1 -- 172.21.15.84:0/3541623686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ef4101850 msgr2=0x7f3ef410f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3ef9663640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ef4101850 0x7f3ef410f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3ef9663640 1 -- 172.21.15.84:0/3541623686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ef41a7560 con 0x7f3ef4102210 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3efa665640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ef4107860 0x7f3ef410fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.650+0000 7f3ef9663640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef4102210 0x7f3ef410f6f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0005940 tx=0x7f3ef0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.651+0000 7f3ee6ffd640 1 -- 172.21.15.84:0/3541623686 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef003f030 con 0x7f3ef4102210 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.651+0000 7f3ee6ffd640 1 -- 172.21.15.84:0/3541623686 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ef0003aa0 con 0x7f3ef4102210 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.651+0000 7f3efc0c8640 1 -- 172.21.15.84:0/3541623686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ef41a77f0 con 0x7f3ef4102210 2024-04-26T18:39:36.653 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.651+0000 7f3ee6ffd640 1 -- 172.21.15.84:0/3541623686 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0035070 con 0x7f3ef4102210 2024-04-26T18:39:36.654 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.651+0000 7f3efc0c8640 1 -- 172.21.15.84:0/3541623686 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ef41a7ce0 con 0x7f3ef4102210 2024-04-26T18:39:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.653+0000 7f3ee6ffd640 1 -- 172.21.15.84:0/3541623686 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3ef0042030 con 0x7f3ef4102210 2024-04-26T18:39:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.653+0000 7f3efede7740 1 Processor -- start 2024-04-26T18:39:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.653+0000 7f3efede7740 1 -- start start 2024-04-26T18:39:36.660 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.653+0000 7f3efede7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 0x55dd97a5b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.653+0000 7f3efede7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5b820 0x55dd97a65ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.653+0000 7f3efede7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a660e0 0x55dd97a684d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3efede7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dd979c1460 con 0x55dd97a5af00 2024-04-26T18:39:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3efede7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dd9797b590 con 0x55dd97a5b820 2024-04-26T18:39:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3efede7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dd979bd420 con 0x55dd97a660e0 2024-04-26T18:39:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3efa665640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 0x55dd97a5b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3efa665640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 0x55dd97a5b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45068/0 (socket says 172.21.15.84:45068) 2024-04-26T18:39:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3efa665640 1 -- 172.21.15.84:0/1197595467 learned_addr learned my addr 172.21.15.84:0/1197595467 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3ef9663640 1 --2- 172.21.15.84:0/1197595467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a660e0 0x55dd97a684d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3ee6ffd640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3ee00a3df0 0x7f3ee00a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3ef9e64640 1 --2- 172.21.15.84:0/1197595467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5b820 0x55dd97a65ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3efa665640 1 -- 172.21.15.84:0/1197595467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a660e0 msgr2=0x55dd97a684d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:36.661 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3efa665640 1 --2- 172.21.15.84:0/1197595467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a660e0 0x55dd97a684d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3efa665640 1 -- 172.21.15.84:0/1197595467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5b820 msgr2=0x55dd97a65ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3efa665640 1 --2- 172.21.15.84:0/1197595467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5b820 0x55dd97a65ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3efa665640 1 -- 172.21.15.84:0/1197595467 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd979880d0 con 0x55dd97a5af00 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3ef9663640 1 --2- 172.21.15.84:0/1197595467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a660e0 0x55dd97a684d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3efa665640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3ee00a3df0 0x7f3ee00a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3ee6ffd640 1 -- 172.21.15.84:0/3541623686 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f3ef00f10a0 con 0x7f3ef4102210 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3efa665640 1 --2- 172.21.15.84:0/1197595467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 0x55dd97a5b2e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c550 tx=0x7f3eec00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe8c5d05e830ccf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3edffff640 1 -- 172.21.15.84:0/1197595467 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec00a600 con 0x55dd97a5af00 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.654+0000 7f3edffff640 1 -- 172.21.15.84:0/1197595467 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eec00ac50 con 0x55dd97a5af00 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.655+0000 7f3edffff640 1 -- 172.21.15.84:0/1197595467 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec0179e0 con 0x55dd97a5af00 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.655+0000 7f3efa665640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3ee00a3df0 0x7f3ee00a62b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3eec01f000 tx=0x7f3eec018000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.655+0000 7f3efede7740 1 -- 172.21.15.84:0/1197595467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 msgr2=0x55dd97a5b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.655+0000 7f3efede7740 1 --2- 172.21.15.84:0/1197595467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 0x55dd97a5b2e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00c550 tx=0x7f3eec00ca10 comp rx=0 tx=0).stop 2024-04-26T18:39:36.662 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.655+0000 7f3efede7740 1 -- 172.21.15.84:0/1197595467 shutdown_connections 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.655+0000 7f3efede7740 1 --2- 172.21.15.84:0/1197595467 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a660e0 0x55dd97a684d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.655+0000 7f3efede7740 1 --2- 172.21.15.84:0/1197595467 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5b820 0x55dd97a65ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.655+0000 7f3efede7740 1 --2- 172.21.15.84:0/1197595467 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 0x55dd97a5b2e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.655+0000 7f3efede7740 1 -- 172.21.15.84:0/1197595467 >> 172.21.15.84:0/1197595467 conn(0x55dd97a57410 msgr2=0x55dd97a577f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.655+0000 7f3efede7740 1 -- 172.21.15.84:0/1197595467 shutdown_connections 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.655+0000 7f3efede7740 1 -- 172.21.15.84:0/1197595467 wait complete. 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3efede7740 1 -- start start 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3efede7740 1 Processor -- start 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3efede7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5af00 0x55dd97a62fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3efede7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a5b820 0x55dd97a634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3efede7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a660e0 0x55dd97a63a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3efede7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55dd97a68d10 con 0x55dd97a660e0 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3efede7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55dd97948b60 con 0x55dd97a5af00 2024-04-26T18:39:36.663 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3efede7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55dd97a68e90 con 0x55dd97a5b820 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3efede7740 10 client.0 _ll_register_callbacks cb 0x55dd9796d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3efa665640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a660e0 0x55dd97a63a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3efa665640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a660e0 0x55dd97a63a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45078/0 (socket says 172.21.15.84:45078) 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3efa665640 1 -- 172.21.15.84:0/2985405451 learned_addr learned my addr 172.21.15.84:0/2985405451 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3ef9e64640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5af00 0x55dd97a62fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.656+0000 7f3ef9663640 1 --2- 172.21.15.84:0/2985405451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a5b820 0x55dd97a634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3efa665640 1 -- 172.21.15.84:0/2985405451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a5b820 msgr2=0x55dd97a634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3efa665640 1 --2- 172.21.15.84:0/2985405451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a5b820 0x55dd97a634f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3efa665640 1 -- 172.21.15.84:0/2985405451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5af00 msgr2=0x55dd97a62fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3efa665640 1 --2- 172.21.15.84:0/2985405451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5af00 0x55dd97a62fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3efa665640 1 -- 172.21.15.84:0/2985405451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd976d3210 con 0x55dd97a660e0 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3efa665640 1 --2- 172.21.15.84:0/2985405451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a660e0 0x55dd97a63a30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0094f0 tx=0x7f3eec00c2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1026b70c7b94d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3edd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:39:36.664 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3edd7fa640 1 -- 172.21.15.84:0/2985405451 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eec02d020 con 0x55dd97a660e0 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3edd7fa640 1 -- 172.21.15.84:0/2985405451 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f3eec017980 con 0x55dd97a660e0 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3efede7740 1 -- 172.21.15.84:0/2985405451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3eec00ac50 con 0x55dd97a660e0 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3edd7fa640 1 -- 172.21.15.84:0/2985405451 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3eec00a400 con 0x55dd97a660e0 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3efede7740 10 client.16809 fetch_fsmap learned FSMap version 29 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3efede7740 1 -- 172.21.15.84:0/2985405451 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dd9779f620 con 0x55dd97a660e0 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3edd7fa640 1 -- 172.21.15.84:0/2985405451 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3eec00ac50 con 0x55dd97a660e0 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3edd7fa640 1 -- 172.21.15.84:0/2985405451 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3eec018810 con 0x55dd97a660e0 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3efede7740 10 client.16809 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.657+0000 7f3efede7740 10 client.16809 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.658+0000 7f3efede7740 20 client.16809 populate_metadata read hostname 'smithi084' 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.658+0000 7f3efede7740 1 --2- 172.21.15.84:0/2985405451 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55dd97a75fe0 0x55dd97a783d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.658+0000 7f3efede7740 4 client.16809 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.658+0000 7f3efede7740 1 -- 172.21.15.84:0/2985405451 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dd979cf070 con 0x55dd97a75fe0 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.658+0000 7f3ef9e64640 1 --2- 172.21.15.84:0/2985405451 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55dd97a75fe0 0x55dd97a783d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.665 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.658+0000 7f3ef9e64640 1 --2- 172.21.15.84:0/2985405451 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55dd97a75fe0 0x55dd97a783d0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb64dd6d487d6eb server_cookie=5b5015235ff8ab06 in_seq=0 out_seq=0 2024-04-26T18:39:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.658+0000 7f3edd7fa640 10 client.16809 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3edd7fa640 1 -- 172.21.15.84:0/2985405451 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55dd979cf070 con 0x55dd97a75fe0 2024-04-26T18:39:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3edd7fa640 10 client.16809 handle_command_reply: tid=0 2024-04-26T18:39:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3efede7740 1 client.16809 shutdown 2024-04-26T18:39:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3efede7740 1 -- 172.21.15.84:0/2985405451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a660e0 msgr2=0x55dd97a63a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3efede7740 1 --2- 172.21.15.84:0/2985405451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a660e0 0x55dd97a63a30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3eec0094f0 tx=0x7f3eec00c2e0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3efede7740 1 -- 172.21.15.84:0/2985405451 shutdown_connections 2024-04-26T18:39:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3efede7740 1 --2- 172.21.15.84:0/2985405451 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55dd97a75fe0 0x55dd97a783d0 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3efede7740 1 --2- 172.21.15.84:0/2985405451 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a660e0 0x55dd97a63a30 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3efede7740 1 --2- 172.21.15.84:0/2985405451 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a5b820 0x55dd97a634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3edd7fa640 0 client.16809 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3efede7740 1 --2- 172.21.15.84:0/2985405451 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5af00 0x55dd97a62fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.666 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3efede7740 1 -- 172.21.15.84:0/2985405451 >> 172.21.15.84:0/2985405451 conn(0x55dd97a57410 msgr2=0x55dd97a5d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3efede7740 1 -- 172.21.15.84:0/2985405451 shutdown_connections 2024-04-26T18:39:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3efede7740 1 -- 172.21.15.84:0/2985405451 wait complete. 2024-04-26T18:39:36.667 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.659+0000 7f3efede7740 20 client.16809 trim_cache size 0 max 16384 2024-04-26T18:39:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3efede7740 1 Processor -- start 2024-04-26T18:39:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3efede7740 1 -- start start 2024-04-26T18:39:36.678 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3efede7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 0x55dd97a5d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3efede7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5b820 0x55dd97a7dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3efede7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a660e0 0x55dd97a7e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3efede7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41b0800 con 0x55dd97a5af00 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3efede7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41b2340 con 0x55dd97a5b820 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3efede7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41b0ed0 con 0x55dd97a660e0 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3ef9e64640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 0x55dd97a5d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3ef9e64640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 0x55dd97a5d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45086/0 (socket says 172.21.15.84:45086) 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3ef9e64640 1 -- 172.21.15.84:0/1279853715 learned_addr learned my addr 172.21.15.84:0/1279853715 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3ef9663640 1 --2- 172.21.15.84:0/1279853715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5b820 0x55dd97a7dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3ef9e64640 1 -- 172.21.15.84:0/1279853715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a660e0 msgr2=0x55dd97a7e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3efa665640 1 --2- 172.21.15.84:0/1279853715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a660e0 0x55dd97a7e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3ef9e64640 1 --2- 172.21.15.84:0/1279853715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a660e0 0x55dd97a7e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3ef9e64640 1 -- 172.21.15.84:0/1279853715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5b820 msgr2=0x55dd97a7dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:36.679 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.674+0000 7f3ef9e64640 1 --2- 172.21.15.84:0/1279853715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5b820 0x55dd97a7dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.675+0000 7f3ef9e64640 1 -- 172.21.15.84:0/1279853715 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dd97a5a700 con 0x55dd97a5af00 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.675+0000 7f3ef9663640 1 --2- 172.21.15.84:0/1279853715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5b820 0x55dd97a7dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.675+0000 7f3ef9e64640 1 --2- 172.21.15.84:0/1279853715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 0x55dd97a5d8c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8004410 tx=0x7f3ee80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4a66435fd9de86 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.675+0000 7f3edf7fe640 1 -- 172.21.15.84:0/1279853715 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee801d050 con 0x55dd97a5af00 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.675+0000 7f3edf7fe640 1 -- 172.21.15.84:0/1279853715 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee801d6a0 con 0x55dd97a5af00 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.675+0000 7f3edf7fe640 1 -- 172.21.15.84:0/1279853715 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8011a80 con 0x55dd97a5af00 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.675+0000 7f3efede7740 1 -- 172.21.15.84:0/1279853715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 msgr2=0x55dd97a5d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.675+0000 7f3efede7740 1 --2- 172.21.15.84:0/1279853715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 0x55dd97a5d8c0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8004410 tx=0x7f3ee80039c0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.675+0000 7f3efede7740 1 -- 172.21.15.84:0/1279853715 shutdown_connections 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.675+0000 7f3efede7740 1 --2- 172.21.15.84:0/1279853715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a660e0 0x55dd97a7e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.675+0000 7f3efede7740 1 --2- 172.21.15.84:0/1279853715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a5b820 0x55dd97a7dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.675+0000 7f3efede7740 1 --2- 172.21.15.84:0/1279853715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5af00 0x55dd97a5d8c0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.675+0000 7f3efede7740 1 -- 172.21.15.84:0/1279853715 >> 172.21.15.84:0/1279853715 conn(0x55dd97a57410 msgr2=0x55dd97a65070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:36.680 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3efede7740 1 -- 172.21.15.84:0/1279853715 shutdown_connections 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3efede7740 1 -- 172.21.15.84:0/1279853715 wait complete. 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3efede7740 1 -- start start 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3efede7740 1 Processor -- start 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3efede7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a5af00 0x55dd97a6e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3efede7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5b820 0x55dd97a5f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3efede7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a660e0 0x55dd97a5ff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3efede7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41b1c70 con 0x55dd97a5b820 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3efede7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41b15a0 con 0x55dd97a660e0 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3efede7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3ef41ae800 con 0x55dd97a5af00 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3efede7740 10 client.0 _ll_register_callbacks cb 0x7f3eec00c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3ef9663640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5b820 0x55dd97a5f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3ef9663640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5b820 0x55dd97a5f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45096/0 (socket says 172.21.15.84:45096) 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.676+0000 7f3ef9663640 1 -- 172.21.15.84:0/1947893554 learned_addr learned my addr 172.21.15.84:0/1947893554 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:36.681 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3ef9e64640 1 --2- 172.21.15.84:0/1947893554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a5af00 0x55dd97a6e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3efa665640 1 --2- 172.21.15.84:0/1947893554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a660e0 0x55dd97a5ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3ef9663640 1 -- 172.21.15.84:0/1947893554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a5af00 msgr2=0x55dd97a6e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3ef9663640 1 --2- 172.21.15.84:0/1947893554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a5af00 0x55dd97a6e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3ef9663640 1 -- 172.21.15.84:0/1947893554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a660e0 msgr2=0x55dd97a5ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3ef9663640 1 --2- 172.21.15.84:0/1947893554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a660e0 0x55dd97a5ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3ef9663640 1 -- 172.21.15.84:0/1947893554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dd97a60790 con 0x55dd97a5b820 2024-04-26T18:39:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3ef9663640 1 --2- 172.21.15.84:0/1947893554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5b820 0x55dd97a5f9e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00b9400 tx=0x7f3ef00bc900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5525fdf70e1cbe0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3edd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:39:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3edd7fa640 1 -- 172.21.15.84:0/1947893554 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0049040 con 0x55dd97a5b820 2024-04-26T18:39:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3edd7fa640 1 -- 172.21.15.84:0/1947893554 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f3ef00b9610 con 0x55dd97a5b820 2024-04-26T18:39:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3efede7740 1 -- 172.21.15.84:0/1947893554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3ee801d6a0 con 0x55dd97a5b820 2024-04-26T18:39:36.682 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3edd7fa640 1 -- 172.21.15.84:0/1947893554 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3ef00bcfa0 con 0x55dd97a5b820 2024-04-26T18:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3efede7740 10 client.16815 fetch_fsmap learned FSMap version 29 2024-04-26T18:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3efede7740 1 -- 172.21.15.84:0/1947893554 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55dd97adc530 con 0x55dd97a5b820 2024-04-26T18:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3edd7fa640 1 -- 172.21.15.84:0/1947893554 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ee801d6a0 con 0x55dd97a5b820 2024-04-26T18:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3edd7fa640 1 -- 172.21.15.84:0/1947893554 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3ef00ee9d0 con 0x55dd97a5b820 2024-04-26T18:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3efede7740 10 client.16815 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3efede7740 10 client.16815 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3efede7740 20 client.16815 populate_metadata read hostname 'smithi084' 2024-04-26T18:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.677+0000 7f3efede7740 1 --2- 172.21.15.84:0/1947893554 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55dd97adccd0 0x55dd97adf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.678+0000 7f3efede7740 4 client.16815 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.678+0000 7f3efede7740 1 -- 172.21.15.84:0/1947893554 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dd979c8ee0 con 0x55dd97adccd0 2024-04-26T18:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.678+0000 7f3ef9e64640 1 --2- 172.21.15.84:0/1947893554 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55dd97adccd0 0x55dd97adf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.678+0000 7f3edd7fa640 10 client.16815 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:36.683 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.678+0000 7f3ef9e64640 1 --2- 172.21.15.84:0/1947893554 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55dd97adccd0 0x55dd97adf0f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c26da60a504af0 server_cookie=e7e4b66b91797d93 in_seq=0 out_seq=0 2024-04-26T18:39:36.692 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3edd7fa640 1 -- 172.21.15.84:0/1947893554 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55dd979c8ee0 con 0x55dd97adccd0 2024-04-26T18:39:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3edd7fa640 10 client.16815 handle_command_reply: tid=0 2024-04-26T18:39:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3efede7740 1 client.16815 shutdown 2024-04-26T18:39:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3efede7740 1 -- 172.21.15.84:0/1947893554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5b820 msgr2=0x55dd97a5f9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3efede7740 1 --2- 172.21.15.84:0/1947893554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5b820 0x55dd97a5f9e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3ef00b9400 tx=0x7f3ef00bc900 comp rx=0 tx=0).stop 2024-04-26T18:39:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3efede7740 1 -- 172.21.15.84:0/1947893554 shutdown_connections 2024-04-26T18:39:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3efede7740 1 --2- 172.21.15.84:0/1947893554 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55dd97adccd0 0x55dd97adf0f0 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3efede7740 1 --2- 172.21.15.84:0/1947893554 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55dd97a660e0 0x55dd97a5ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3efede7740 1 --2- 172.21.15.84:0/1947893554 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55dd97a5b820 0x55dd97a5f9e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3edd7fa640 0 client.16815 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3efede7740 1 --2- 172.21.15.84:0/1947893554 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55dd97a5af00 0x55dd97a6e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3efede7740 1 -- 172.21.15.84:0/1947893554 >> 172.21.15.84:0/1947893554 conn(0x55dd97a57410 msgr2=0x55dd97a565b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3efede7740 1 -- 172.21.15.84:0/1947893554 shutdown_connections 2024-04-26T18:39:36.693 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3efede7740 1 -- 172.21.15.84:0/1947893554 wait complete. 2024-04-26T18:39:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.692+0000 7f3efede7740 20 client.16815 trim_cache size 0 max 16384 2024-04-26T18:39:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:39:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:39:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:39:36.694 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:39:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.693+0000 7f3efc0c8640 1 -- 172.21.15.84:0/3541623686 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3ee00a3df0 msgr2=0x7f3ee00a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.693+0000 7f3efc0c8640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3ee00a3df0 0x7f3ee00a62b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f3eec01f000 tx=0x7f3eec018000 comp rx=0 tx=0).stop 2024-04-26T18:39:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.693+0000 7f3efc0c8640 1 -- 172.21.15.84:0/3541623686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef4102210 msgr2=0x7f3ef410f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.693+0000 7f3efc0c8640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef4102210 0x7f3ef410f6f0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0005940 tx=0x7f3ef0004270 comp rx=0 tx=0).stop 2024-04-26T18:39:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.693+0000 7f3efc0c8640 1 -- 172.21.15.84:0/3541623686 shutdown_connections 2024-04-26T18:39:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.693+0000 7f3efc0c8640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3ee00a3df0 0x7f3ee00a62b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.693+0000 7f3efc0c8640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3ef4107860 0x7f3ef410fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.695 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.693+0000 7f3efc0c8640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3ef4102210 0x7f3ef410f6f0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.693+0000 7f3efc0c8640 1 --2- 172.21.15.84:0/3541623686 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3ef4101850 0x7f3ef410f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.693+0000 7f3efc0c8640 1 -- 172.21.15.84:0/3541623686 >> 172.21.15.84:0/3541623686 conn(0x7f3ef4077bb0 msgr2=0x7f3ef40ff5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.693+0000 7f3efc0c8640 1 -- 172.21.15.84:0/3541623686 shutdown_connections 2024-04-26T18:39:36.696 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:36.693+0000 7f3efc0c8640 1 -- 172.21.15.84:0/3541623686 wait complete. 2024-04-26T18:39:37.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:37 smithi084 ceph-mon[29141]: pgmap v364: 97 pgs: 97 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:39:37.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:37 smithi103 ceph-mon[32727]: pgmap v364: 97 pgs: 97 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:39:37.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:37 smithi174 ceph-mon[32705]: pgmap v364: 97 pgs: 97 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:39:39.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:39 smithi084 ceph-mon[29141]: pgmap v365: 97 pgs: 97 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:39:39.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:39 smithi103 ceph-mon[32727]: pgmap v365: 97 pgs: 97 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:39:39.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:39 smithi174 ceph-mon[32705]: pgmap v365: 97 pgs: 97 active+clean; 2.6 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:39:41.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:41 smithi084 ceph-mon[29141]: pgmap v366: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-26T18:39:41.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:41 smithi103 ceph-mon[32727]: pgmap v366: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-26T18:39:41.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:41 smithi174 ceph-mon[32705]: pgmap v366: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 64 KiB/s wr, 24 op/s 2024-04-26T18:39:43.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:43 smithi084 ceph-mon[29141]: pgmap v367: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:39:43.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:43 smithi103 ceph-mon[32727]: pgmap v367: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:39:43.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:43 smithi174 ceph-mon[32705]: pgmap v367: 97 pgs: 97 active+clean; 2.9 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:39:45.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:45 smithi084 ceph-mon[29141]: pgmap v368: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-26T18:39:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:45 smithi103 ceph-mon[32727]: pgmap v368: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-26T18:39:45.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:45 smithi174 ceph-mon[32705]: pgmap v368: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-26T18:39:47.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:47 smithi084 ceph-mon[29141]: pgmap v369: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:39:47.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:47 smithi103 ceph-mon[32727]: pgmap v369: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:39:47.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:47 smithi174 ceph-mon[32705]: pgmap v369: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:39:49.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:49 smithi174 ceph-mon[32705]: pgmap v370: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:39:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.789+0000 7f7edf446640 1 -- 172.21.15.84:0/732668187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ed8102080 msgr2=0x7f7ed810edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.789+0000 7f7edf446640 1 --2- 172.21.15.84:0/732668187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ed8102080 0x7f7ed810edd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009900 tx=0x7f7ec802a410 comp rx=0 tx=0).stop 2024-04-26T18:39:49.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.789+0000 7f7edf446640 1 -- 172.21.15.84:0/732668187 shutdown_connections 2024-04-26T18:39:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.789+0000 7f7edf446640 1 --2- 172.21.15.84:0/732668187 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ed810f310 0x7f7ed8111700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.789+0000 7f7edf446640 1 --2- 172.21.15.84:0/732668187 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ed8102080 0x7f7ed810edd0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.789+0000 7f7edf446640 1 --2- 172.21.15.84:0/732668187 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8101760 0x7f7ed8101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.790+0000 7f7edf446640 1 -- 172.21.15.84:0/732668187 >> 172.21.15.84:0/732668187 conn(0x7f7ed8077bb0 msgr2=0x7f7ed8077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.790+0000 7f7edf446640 1 -- 172.21.15.84:0/732668187 shutdown_connections 2024-04-26T18:39:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.790+0000 7f7edf446640 1 -- 172.21.15.84:0/732668187 wait complete. 2024-04-26T18:39:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.790+0000 7f7edf446640 1 Processor -- start 2024-04-26T18:39:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.790+0000 7f7edf446640 1 -- start start 2024-04-26T18:39:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.790+0000 7f7edf446640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8101760 0x7f7ed819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.790+0000 7f7edf446640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ed8102080 0x7f7ed819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.790+0000 7f7edf446640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ed810f310 0x7f7ed819fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.790+0000 7f7edf446640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8114e20 con 0x7f7ed810f310 2024-04-26T18:39:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.790+0000 7f7edf446640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8114ca0 con 0x7f7ed8101760 2024-04-26T18:39:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.790+0000 7f7edf446640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8114fa0 con 0x7f7ed8102080 2024-04-26T18:39:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7edd9e3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ed810f310 0x7f7ed819fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7edd9e3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ed810f310 0x7f7ed819fb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37604/0 (socket says 172.21.15.84:37604) 2024-04-26T18:39:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7edd9e3640 1 -- 172.21.15.84:0/4174993044 learned_addr learned my addr 172.21.15.84:0/4174993044 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7edd1e2640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8101760 0x7f7ed819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ed8102080 0x7f7ed819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7edd9e3640 1 -- 172.21.15.84:0/4174993044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ed8102080 msgr2=0x7f7ed819b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7edd9e3640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ed8102080 0x7f7ed819b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7edd9e3640 1 -- 172.21.15.84:0/4174993044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8101760 msgr2=0x7f7ed819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7edd9e3640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8101760 0x7f7ed819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.794 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7edd9e3640 1 -- 172.21.15.84:0/4174993044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed81a02b0 con 0x7f7ed810f310 2024-04-26T18:39:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7edd9e3640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ed810f310 0x7f7ed819fb80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000dac0 tx=0x7f7ed000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7ece7fc640 1 -- 172.21.15.84:0/4174993044 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0017030 con 0x7f7ed810f310 2024-04-26T18:39:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7ece7fc640 1 -- 172.21.15.84:0/4174993044 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed0003e20 con 0x7f7ed810f310 2024-04-26T18:39:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7edf446640 1 -- 172.21.15.84:0/4174993044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed81a05a0 con 0x7f7ed810f310 2024-04-26T18:39:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7ece7fc640 1 -- 172.21.15.84:0/4174993044 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0010460 con 0x7f7ed810f310 2024-04-26T18:39:49.795 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.791+0000 7f7edf446640 1 -- 172.21.15.84:0/4174993044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed81a7e30 con 0x7f7ed810f310 2024-04-26T18:39:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.793+0000 7f7ece7fc640 1 -- 172.21.15.84:0/4174993044 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f7ed001a070 con 0x7f7ed810f310 2024-04-26T18:39:49.802 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.794+0000 7f7ee2235740 1 Processor -- start 2024-04-26T18:39:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.794+0000 7f7ee2235740 1 -- start start 2024-04-26T18:39:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.794+0000 7f7ee2235740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad2722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.794+0000 7f7ee2235740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad27cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.794+0000 7f7ee2235740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 0x5635ad27f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.794+0000 7f7ee2235740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5635ad1d8460 con 0x5635ad272820 2024-04-26T18:39:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.794+0000 7f7ee2235740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5635ad192590 con 0x5635ad271f00 2024-04-26T18:39:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.794+0000 7f7ee2235740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5635ad1d4420 con 0x5635ad27d0e0 2024-04-26T18:39:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.794+0000 7f7ece7fc640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7ec40a3dc0 0x7f7ec40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.794+0000 7f7edd1e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad27cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.794+0000 7f7edd1e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad27cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37608/0 (socket says 172.21.15.84:37608) 2024-04-26T18:39:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7edd1e2640 1 -- 172.21.15.84:0/1444321190 learned_addr learned my addr 172.21.15.84:0/1444321190 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:49.803 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7edd9e3640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7ec40a3dc0 0x7f7ec40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/1444321190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad2722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7ece7fc640 1 -- 172.21.15.84:0/4174993044 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f7ed00d0620 con 0x7f7ed810f310 2024-04-26T18:39:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/1444321190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 0x5635ad27f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7edd1e2640 1 -- 172.21.15.84:0/1444321190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 msgr2=0x5635ad27f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7edd1e2640 1 --2- 172.21.15.84:0/1444321190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 0x5635ad27f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7edd1e2640 1 -- 172.21.15.84:0/1444321190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 msgr2=0x5635ad2722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7edd1e2640 1 --2- 172.21.15.84:0/1444321190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad2722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7edd1e2640 1 -- 172.21.15.84:0/1444321190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ad19f0d0 con 0x5635ad272820 2024-04-26T18:39:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7edd9e3640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7ec40a3dc0 0x7f7ec40a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00cdde0 tx=0x7f7ed006dfc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7edd1e2640 1 --2- 172.21.15.84:0/1444321190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad27cba0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400bcb0 tx=0x7f7ed400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6c5e38c98d3905 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7ec37fe640 1 -- 172.21.15.84:0/1444321190 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4015070 con 0x5635ad272820 2024-04-26T18:39:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7ec37fe640 1 -- 172.21.15.84:0/1444321190 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed40118f0 con 0x5635ad272820 2024-04-26T18:39:49.804 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.795+0000 7f7ec37fe640 1 -- 172.21.15.84:0/1444321190 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed40108b0 con 0x5635ad272820 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.796+0000 7f7ee2235740 1 -- 172.21.15.84:0/1444321190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 msgr2=0x5635ad27cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.796+0000 7f7ee2235740 1 --2- 172.21.15.84:0/1444321190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad27cba0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f7ed400bcb0 tx=0x7f7ed400f730 comp rx=0 tx=0).stop 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.796+0000 7f7ee2235740 1 -- 172.21.15.84:0/1444321190 shutdown_connections 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.796+0000 7f7ee2235740 1 --2- 172.21.15.84:0/1444321190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 0x5635ad27f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.796+0000 7f7ee2235740 1 --2- 172.21.15.84:0/1444321190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad27cba0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.796+0000 7f7ee2235740 1 --2- 172.21.15.84:0/1444321190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad2722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.796+0000 7f7ee2235740 1 -- 172.21.15.84:0/1444321190 >> 172.21.15.84:0/1444321190 conn(0x5635ad26e410 msgr2=0x5635ad26e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.796+0000 7f7ee2235740 1 -- 172.21.15.84:0/1444321190 shutdown_connections 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.796+0000 7f7ee2235740 1 -- 172.21.15.84:0/1444321190 wait complete. 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.796+0000 7f7ee2235740 1 -- start start 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7ee2235740 1 Processor -- start 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7ee2235740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad279fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7ee2235740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad27a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.805 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7ee2235740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 0x5635ad27aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7ee2235740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5635ad27fd10 con 0x5635ad272820 2024-04-26T18:39:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7ee2235740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5635ad15fb60 con 0x5635ad271f00 2024-04-26T18:39:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7ee2235740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5635ad27fe90 con 0x5635ad27d0e0 2024-04-26T18:39:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7ee2235740 10 client.0 _ll_register_callbacks cb 0x5635ad1842b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:39:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7edd1e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad27a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7edd1e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad27a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37622/0 (socket says 172.21.15.84:37622) 2024-04-26T18:39:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7edd1e2640 1 -- 172.21.15.84:0/644663027 learned_addr learned my addr 172.21.15.84:0/644663027 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 0x5635ad27aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad279fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7edd1e2640 1 -- 172.21.15.84:0/644663027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 msgr2=0x5635ad27aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7edd1e2640 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 0x5635ad27aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.806 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7edd1e2640 1 -- 172.21.15.84:0/644663027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 msgr2=0x5635ad279fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7edd1e2640 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad279fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7edd1e2640 1 -- 172.21.15.84:0/644663027 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635aceea210 con 0x5635ad272820 2024-04-26T18:39:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.797+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad279fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:39:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7edd1e2640 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad27a510 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4005760 tx=0x7f7ed400bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60418686dac9615b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ec0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:39:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ec0ff9640 1 -- 172.21.15.84:0/644663027 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4029060 con 0x5635ad272820 2024-04-26T18:39:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ec0ff9640 1 -- 172.21.15.84:0/644663027 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f7ed4011aa0 con 0x5635ad272820 2024-04-26T18:39:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ee2235740 1 -- 172.21.15.84:0/644663027 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ed40118f0 con 0x5635ad272820 2024-04-26T18:39:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ec0ff9640 1 -- 172.21.15.84:0/644663027 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7ed4015050 con 0x5635ad272820 2024-04-26T18:39:49.807 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ee2235740 10 client.16827 fetch_fsmap learned FSMap version 29 2024-04-26T18:39:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ee2235740 1 -- 172.21.15.84:0/644663027 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5635acfb6620 con 0x5635ad272820 2024-04-26T18:39:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ec0ff9640 1 -- 172.21.15.84:0/644663027 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ed40118f0 con 0x5635ad272820 2024-04-26T18:39:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ec0ff9640 1 -- 172.21.15.84:0/644663027 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7ed4003960 con 0x5635ad272820 2024-04-26T18:39:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ee2235740 10 client.16827 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:39:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ee2235740 10 client.16827 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:39:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ee2235740 20 client.16827 populate_metadata read hostname 'smithi084' 2024-04-26T18:39:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ee2235740 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5635ad28d070 0x5635ad28f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ee2235740 4 client.16827 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:39:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.798+0000 7f7ee2235740 1 -- 172.21.15.84:0/644663027 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635ad1e6070 con 0x5635ad28d070 2024-04-26T18:39:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.799+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5635ad28d070 0x5635ad28f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.799+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5635ad28d070 0x5635ad28f460 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5ab231db6739a46 server_cookie=7571dcbcb6e04659 in_seq=0 out_seq=0 2024-04-26T18:39:49.808 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.799+0000 7f7ec0ff9640 10 client.16827 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ec0ff9640 1 -- 172.21.15.84:0/644663027 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5635ad1e6070 con 0x5635ad28d070 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ec0ff9640 10 client.16827 handle_command_reply: tid=0 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ee2235740 1 client.16827 shutdown 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ee2235740 1 -- 172.21.15.84:0/644663027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 msgr2=0x5635ad27a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ee2235740 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad27a510 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7ed4005760 tx=0x7f7ed400bee0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ee2235740 1 -- 172.21.15.84:0/644663027 shutdown_connections 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ee2235740 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5635ad28d070 0x5635ad28f460 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ee2235740 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 0x5635ad27aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ee2235740 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad27a510 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ec0ff9640 0 client.16827 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ee2235740 1 --2- 172.21.15.84:0/644663027 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad279fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ee2235740 1 -- 172.21.15.84:0/644663027 >> 172.21.15.84:0/644663027 conn(0x5635ad26e410 msgr2=0x5635ad274190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ee2235740 1 -- 172.21.15.84:0/644663027 shutdown_connections 2024-04-26T18:39:49.809 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ee2235740 1 -- 172.21.15.84:0/644663027 wait complete. 2024-04-26T18:39:49.810 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.800+0000 7f7ee2235740 20 client.16827 trim_cache size 0 max 16384 2024-04-26T18:39:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7ee2235740 1 Processor -- start 2024-04-26T18:39:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7ee2235740 1 -- start start 2024-04-26T18:39:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7ee2235740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad2887f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7ee2235740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad272820 0x5635ad292480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7ee2235740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad27d0e0 0x5635ad2929c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7ee2235740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81b08d0 con 0x5635ad27d0e0 2024-04-26T18:39:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7ee2235740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81b2410 con 0x5635ad271f00 2024-04-26T18:39:49.820 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7ee2235740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81b0fa0 con 0x5635ad272820 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7edc9e1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad27d0e0 0x5635ad2929c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7edc9e1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad27d0e0 0x5635ad2929c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37630/0 (socket says 172.21.15.84:37630) 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7edc9e1640 1 -- 172.21.15.84:0/1529759471 learned_addr learned my addr 172.21.15.84:0/1529759471 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/1529759471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad2887f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7edd1e2640 1 --2- 172.21.15.84:0/1529759471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad272820 0x5635ad292480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7edc9e1640 1 -- 172.21.15.84:0/1529759471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad272820 msgr2=0x5635ad292480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/1529759471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad272820 0x5635ad292480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7edc9e1640 1 -- 172.21.15.84:0/1529759471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 msgr2=0x5635ad2887f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/1529759471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad2887f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7edc9e1640 1 -- 172.21.15.84:0/1529759471 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635ad2952a0 con 0x5635ad27d0e0 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.815+0000 7f7edd1e2640 1 --2- 172.21.15.84:0/1529759471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad272820 0x5635ad292480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.816+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/1529759471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad27d0e0 0x5635ad2929c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8031440 tx=0x7f7ec802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d997f0cba79a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.816+0000 7f7ec2ffd640 1 -- 172.21.15.84:0/1529759471 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8052070 con 0x5635ad27d0e0 2024-04-26T18:39:49.821 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.816+0000 7f7ec2ffd640 1 -- 172.21.15.84:0/1529759471 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ec8031640 con 0x5635ad27d0e0 2024-04-26T18:39:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.816+0000 7f7ec2ffd640 1 -- 172.21.15.84:0/1529759471 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8035040 con 0x5635ad27d0e0 2024-04-26T18:39:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.816+0000 7f7ee2235740 1 -- 172.21.15.84:0/1529759471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad27d0e0 msgr2=0x5635ad2929c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.816+0000 7f7ee2235740 1 --2- 172.21.15.84:0/1529759471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad27d0e0 0x5635ad2929c0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8031440 tx=0x7f7ec802a410 comp rx=0 tx=0).stop 2024-04-26T18:39:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.816+0000 7f7ee2235740 1 -- 172.21.15.84:0/1529759471 shutdown_connections 2024-04-26T18:39:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.816+0000 7f7ee2235740 1 --2- 172.21.15.84:0/1529759471 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad27d0e0 0x5635ad2929c0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.816+0000 7f7ee2235740 1 --2- 172.21.15.84:0/1529759471 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad272820 0x5635ad292480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.816+0000 7f7ee2235740 1 --2- 172.21.15.84:0/1529759471 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad2887f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.816+0000 7f7ee2235740 1 -- 172.21.15.84:0/1529759471 >> 172.21.15.84:0/1529759471 conn(0x5635ad26e410 msgr2=0x5635ad2880e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.816+0000 7f7ee2235740 1 -- 172.21.15.84:0/1529759471 shutdown_connections 2024-04-26T18:39:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.816+0000 7f7ee2235740 1 -- 172.21.15.84:0/1529759471 wait complete. 2024-04-26T18:39:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.817+0000 7f7ee2235740 1 -- start start 2024-04-26T18:39:49.822 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.817+0000 7f7ee2235740 1 Processor -- start 2024-04-26T18:39:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.817+0000 7f7ee2235740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad292290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.817+0000 7f7ee2235740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad2927d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.817+0000 7f7ee2235740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 0x5635ad290710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.817+0000 7f7ee2235740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81b1d40 con 0x5635ad272820 2024-04-26T18:39:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.817+0000 7f7ee2235740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81b1670 con 0x5635ad271f00 2024-04-26T18:39:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.817+0000 7f7ee2235740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81ae8d0 con 0x5635ad27d0e0 2024-04-26T18:39:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.817+0000 7f7ee2235740 10 client.0 _ll_register_callbacks cb 0x7f7ec8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:39:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.817+0000 7f7edd1e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad2927d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.817+0000 7f7edd1e2640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad2927d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:37638/0 (socket says 172.21.15.84:37638) 2024-04-26T18:39:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.817+0000 7f7edd1e2640 1 -- 172.21.15.84:0/880050337 learned_addr learned my addr 172.21.15.84:0/880050337 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:39:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.817+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/880050337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad292290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.823 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/880050337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 0x5635ad290710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7edd1e2640 1 -- 172.21.15.84:0/880050337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 msgr2=0x5635ad290710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7edd1e2640 1 --2- 172.21.15.84:0/880050337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 0x5635ad290710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7edd1e2640 1 -- 172.21.15.84:0/880050337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 msgr2=0x5635ad292290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7edd1e2640 1 --2- 172.21.15.84:0/880050337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad292290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7edd1e2640 1 -- 172.21.15.84:0/880050337 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635ad290ef0 con 0x5635ad272820 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7edd1e2640 1 --2- 172.21.15.84:0/880050337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad2927d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x5635ad279830 tx=0x7f7ed4011d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780b420af60e7b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7ec0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7ec0ff9640 1 -- 172.21.15.84:0/880050337 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ed4029040 con 0x5635ad272820 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7ec0ff9640 1 -- 172.21.15.84:0/880050337 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f7ed4005070 con 0x5635ad272820 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7ee2235740 1 -- 172.21.15.84:0/880050337 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ec8031640 con 0x5635ad272820 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7ec0ff9640 1 -- 172.21.15.84:0/880050337 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7ed40105e0 con 0x5635ad272820 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7ee2235740 10 client.16833 fetch_fsmap learned FSMap version 29 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7ee2235740 1 -- 172.21.15.84:0/880050337 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5635ad2f3de0 con 0x5635ad272820 2024-04-26T18:39:49.824 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7ec0ff9640 1 -- 172.21.15.84:0/880050337 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ec8031640 con 0x5635ad272820 2024-04-26T18:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7ec0ff9640 1 -- 172.21.15.84:0/880050337 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f7ed4003780 con 0x5635ad272820 2024-04-26T18:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7ee2235740 10 client.16833 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7ee2235740 10 client.16833 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7ee2235740 20 client.16833 populate_metadata read hostname 'smithi084' 2024-04-26T18:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.818+0000 7f7ee2235740 1 --2- 172.21.15.84:0/880050337 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5635ad2f44d0 0x5635ad2f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.819+0000 7f7ee2235740 4 client.16833 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.819+0000 7f7ee2235740 1 -- 172.21.15.84:0/880050337 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5635ad1dfee0 con 0x5635ad2f44d0 2024-04-26T18:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.819+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/880050337 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5635ad2f44d0 0x5635ad2f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.819+0000 7f7edc9e1640 1 --2- 172.21.15.84:0/880050337 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5635ad2f44d0 0x5635ad2f68f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb39acdca224e2b7 server_cookie=c377353f00b7fa33 in_seq=0 out_seq=0 2024-04-26T18:39:49.825 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.819+0000 7f7ec0ff9640 10 client.16833 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ec0ff9640 1 -- 172.21.15.84:0/880050337 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5635ad1dfee0 con 0x5635ad2f44d0 2024-04-26T18:39:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ec0ff9640 10 client.16833 handle_command_reply: tid=0 2024-04-26T18:39:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ee2235740 1 client.16833 shutdown 2024-04-26T18:39:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ee2235740 1 -- 172.21.15.84:0/880050337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 msgr2=0x5635ad2927d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ee2235740 1 --2- 172.21.15.84:0/880050337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad2927d0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x5635ad279830 tx=0x7f7ed4011d40 comp rx=0 tx=0).stop 2024-04-26T18:39:49.835 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ee2235740 1 -- 172.21.15.84:0/880050337 shutdown_connections 2024-04-26T18:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ee2235740 1 --2- 172.21.15.84:0/880050337 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5635ad2f44d0 0x5635ad2f68f0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ee2235740 1 --2- 172.21.15.84:0/880050337 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5635ad27d0e0 0x5635ad290710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ee2235740 1 --2- 172.21.15.84:0/880050337 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5635ad272820 0x5635ad2927d0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ec0ff9640 0 client.16833 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ee2235740 1 --2- 172.21.15.84:0/880050337 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5635ad271f00 0x5635ad292290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ee2235740 1 -- 172.21.15.84:0/880050337 >> 172.21.15.84:0/880050337 conn(0x5635ad26e410 msgr2=0x5635ad26d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ee2235740 1 -- 172.21.15.84:0/880050337 shutdown_connections 2024-04-26T18:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ee2235740 1 -- 172.21.15.84:0/880050337 wait complete. 2024-04-26T18:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.834+0000 7f7ee2235740 20 client.16833 trim_cache size 0 max 16384 2024-04-26T18:39:49.836 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:39:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:39:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:39:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:39:49.837 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.835+0000 7f7edf446640 1 -- 172.21.15.84:0/4174993044 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7ec40a3dc0 msgr2=0x7f7ec40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.835+0000 7f7edf446640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7ec40a3dc0 0x7f7ec40a6280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7ed00cdde0 tx=0x7f7ed006dfc0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.835+0000 7f7edf446640 1 -- 172.21.15.84:0/4174993044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ed810f310 msgr2=0x7f7ed819fb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.835+0000 7f7edf446640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ed810f310 0x7f7ed819fb80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000dac0 tx=0x7f7ed000df80 comp rx=0 tx=0).stop 2024-04-26T18:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.835+0000 7f7edf446640 1 -- 172.21.15.84:0/4174993044 shutdown_connections 2024-04-26T18:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.835+0000 7f7edf446640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7ec40a3dc0 0x7f7ec40a6280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.835+0000 7f7edf446640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7ed810f310 0x7f7ed819fb80 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.835+0000 7f7edf446640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7ed8102080 0x7f7ed819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.835+0000 7f7edf446640 1 --2- 172.21.15.84:0/4174993044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7ed8101760 0x7f7ed819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.835+0000 7f7edf446640 1 -- 172.21.15.84:0/4174993044 >> 172.21.15.84:0/4174993044 conn(0x7f7ed8077bb0 msgr2=0x7f7ed81107d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.835+0000 7f7edf446640 1 -- 172.21.15.84:0/4174993044 shutdown_connections 2024-04-26T18:39:49.838 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:39:49.835+0000 7f7edf446640 1 -- 172.21.15.84:0/4174993044 wait complete. 2024-04-26T18:39:49.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:49 smithi084 ceph-mon[29141]: pgmap v370: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:39:49.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:49 smithi103 ceph-mon[32727]: pgmap v370: 97 pgs: 97 active+clean; 3.0 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:39:51.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:51 smithi084 ceph-mon[29141]: pgmap v371: 97 pgs: 97 active+clean; 3.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-26T18:39:51.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:51 smithi103 ceph-mon[32727]: pgmap v371: 97 pgs: 97 active+clean; 3.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-26T18:39:51.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:51 smithi174 ceph-mon[32705]: pgmap v371: 97 pgs: 97 active+clean; 3.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-26T18:39:53.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:53 smithi084 ceph-mon[29141]: pgmap v372: 97 pgs: 97 active+clean; 3.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:39:53.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:53 smithi103 ceph-mon[32727]: pgmap v372: 97 pgs: 97 active+clean; 3.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:39:53.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:53 smithi174 ceph-mon[32705]: pgmap v372: 97 pgs: 97 active+clean; 3.2 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:39:55.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:55 smithi084 ceph-mon[29141]: pgmap v373: 97 pgs: 97 active+clean; 3.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-26T18:39:55.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:55 smithi103 ceph-mon[32727]: pgmap v373: 97 pgs: 97 active+clean; 3.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-26T18:39:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:55 smithi174 ceph-mon[32705]: pgmap v373: 97 pgs: 97 active+clean; 3.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-26T18:39:56.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:56 smithi084 ceph-mon[29141]: pgmap v374: 97 pgs: 97 active+clean; 3.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:39:56.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:56 smithi103 ceph-mon[32727]: pgmap v374: 97 pgs: 97 active+clean; 3.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:39:56.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:56 smithi174 ceph-mon[32705]: pgmap v374: 97 pgs: 97 active+clean; 3.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:39:59.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:39:58 smithi084 ceph-mon[29141]: pgmap v375: 97 pgs: 97 active+clean; 3.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:39:59.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:39:58 smithi103 ceph-mon[32727]: pgmap v375: 97 pgs: 97 active+clean; 3.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:39:59.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:39:58 smithi174 ceph-mon[32705]: pgmap v375: 97 pgs: 97 active+clean; 3.4 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:40:01.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:00 smithi084 ceph-mon[29141]: pgmap v376: 97 pgs: 97 active+clean; 3.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-26T18:40:01.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:00 smithi084 ceph-mon[29141]: overall HEALTH_OK 2024-04-26T18:40:01.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:00 smithi103 ceph-mon[32727]: pgmap v376: 97 pgs: 97 active+clean; 3.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-26T18:40:01.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:00 smithi103 ceph-mon[32727]: overall HEALTH_OK 2024-04-26T18:40:01.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:00 smithi174 ceph-mon[32705]: pgmap v376: 97 pgs: 97 active+clean; 3.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-26T18:40:01.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:00 smithi174 ceph-mon[32705]: overall HEALTH_OK 2024-04-26T18:40:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.931+0000 7f3a0bef1640 1 -- 172.21.15.84:0/4292007681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a041022f0 msgr2=0x7f3a041067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.931+0000 7f3a0bef1640 1 --2- 172.21.15.84:0/4292007681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a041022f0 0x7f3a041067b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f39f8009930 tx=0x7f39f802a4d0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.934 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.931+0000 7f3a0bef1640 1 -- 172.21.15.84:0/4292007681 shutdown_connections 2024-04-26T18:40:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.931+0000 7f3a0bef1640 1 --2- 172.21.15.84:0/4292007681 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a04106dd0 0x7f3a04107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.931+0000 7f3a0bef1640 1 --2- 172.21.15.84:0/4292007681 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a041022f0 0x7f3a041067b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.931+0000 7f3a0bef1640 1 --2- 172.21.15.84:0/4292007681 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a041019d0 0x7f3a04101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.931+0000 7f3a0bef1640 1 -- 172.21.15.84:0/4292007681 >> 172.21.15.84:0/4292007681 conn(0x7f3a04077bb0 msgr2=0x7f3a04077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.931+0000 7f3a0bef1640 1 -- 172.21.15.84:0/4292007681 shutdown_connections 2024-04-26T18:40:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.931+0000 7f3a0bef1640 1 -- 172.21.15.84:0/4292007681 wait complete. 2024-04-26T18:40:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.932+0000 7f3a0bef1640 1 Processor -- start 2024-04-26T18:40:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.932+0000 7f3a0bef1640 1 -- start start 2024-04-26T18:40:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.932+0000 7f3a0bef1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a041019d0 0x7f3a0419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.932+0000 7f3a0bef1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a041022f0 0x7f3a0419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.935 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.932+0000 7f3a0bef1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a04106dd0 0x7f3a041a3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.932+0000 7f3a0bef1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041191d0 con 0x7f3a041022f0 2024-04-26T18:40:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.932+0000 7f3a0bef1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04119050 con 0x7f3a04106dd0 2024-04-26T18:40:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.932+0000 7f3a0bef1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a04119350 con 0x7f3a041019d0 2024-04-26T18:40:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.932+0000 7f3a0948c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a041022f0 0x7f3a0419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.932+0000 7f3a0948c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a041022f0 0x7f3a0419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46568/0 (socket says 172.21.15.84:46568) 2024-04-26T18:40:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.932+0000 7f3a0948c640 1 -- 172.21.15.84:0/760002737 learned_addr learned my addr 172.21.15.84:0/760002737 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f3a09c8d640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a041019d0 0x7f3a0419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a04106dd0 0x7f3a041a3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f3a0948c640 1 -- 172.21.15.84:0/760002737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a041019d0 msgr2=0x7f3a0419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:02.936 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f3a0948c640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a041019d0 0x7f3a0419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f3a0948c640 1 -- 172.21.15.84:0/760002737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a04106dd0 msgr2=0x7f3a041a3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f3a0948c640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a04106dd0 0x7f3a041a3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f3a0948c640 1 -- 172.21.15.84:0/760002737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a041a46d0 con 0x7f3a041022f0 2024-04-26T18:40:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f3a09c8d640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a041019d0 0x7f3a0419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:40:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f3a0948c640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a041022f0 0x7f3a0419fc70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f39f8005940 tx=0x7f39f8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f39f6ffd640 1 -- 172.21.15.84:0/760002737 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f803f030 con 0x7f3a041022f0 2024-04-26T18:40:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f39f6ffd640 1 -- 172.21.15.84:0/760002737 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39f8003aa0 con 0x7f3a041022f0 2024-04-26T18:40:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f3a0bef1640 1 -- 172.21.15.84:0/760002737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a041a4960 con 0x7f3a041022f0 2024-04-26T18:40:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f39f6ffd640 1 -- 172.21.15.84:0/760002737 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39f8035070 con 0x7f3a041022f0 2024-04-26T18:40:02.937 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.933+0000 7f3a0bef1640 1 -- 172.21.15.84:0/760002737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a041ac250 con 0x7f3a041022f0 2024-04-26T18:40:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.935+0000 7f39f6ffd640 1 -- 172.21.15.84:0/760002737 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f39f8042030 con 0x7f3a041022f0 2024-04-26T18:40:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.936+0000 7f3a0ec07740 1 Processor -- start 2024-04-26T18:40:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.936+0000 7f3a0ec07740 1 -- start start 2024-04-26T18:40:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.936+0000 7f3a0ec07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 0x560bfa0322e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.936+0000 7f3a0ec07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 0x560bfa03cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.936+0000 7f3a0ec07740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa03d0e0 0x560bfa03f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.936+0000 7f3a0ec07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560bf9f98460 con 0x560bfa031f00 2024-04-26T18:40:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.936+0000 7f3a0ec07740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560bf9f52590 con 0x560bfa03d0e0 2024-04-26T18:40:02.945 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.936+0000 7f3a0ec07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560bf9f94420 con 0x560bfa032820 2024-04-26T18:40:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.936+0000 7f39f6ffd640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f39f00a3dc0 0x7f39f00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.936+0000 7f3a0a48e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 0x560bfa0322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f3a0a48e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 0x560bfa0322e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46584/0 (socket says 172.21.15.84:46584) 2024-04-26T18:40:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f3a0a48e640 1 -- 172.21.15.84:0/1926875044 learned_addr learned my addr 172.21.15.84:0/1926875044 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f3a0948c640 1 --2- 172.21.15.84:0/1926875044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa03d0e0 0x560bfa03f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f39f00a3dc0 0x7f39f00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f3a09c8d640 1 --2- 172.21.15.84:0/1926875044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 0x560bfa03cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f39f6ffd640 1 -- 172.21.15.84:0/760002737 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f39f80f0ea0 con 0x7f3a041022f0 2024-04-26T18:40:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f3a0a48e640 1 -- 172.21.15.84:0/1926875044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 msgr2=0x560bfa03cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/1926875044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 0x560bfa03cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f3a0a48e640 1 -- 172.21.15.84:0/1926875044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa03d0e0 msgr2=0x560bfa03f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:02.946 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/1926875044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa03d0e0 0x560bfa03f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f3a0a48e640 1 -- 172.21.15.84:0/1926875044 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bf9f5f0d0 con 0x560bfa031f00 2024-04-26T18:40:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/1926875044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 0x560bfa0322e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f39fc013e70 tx=0x7f39fc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a0fa759eb2ac095 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f39f00a3dc0 0x7f39f00a6280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f39fc010ca0 tx=0x7f39fc015000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f39effff640 1 -- 172.21.15.84:0/1926875044 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc00ae10 con 0x560bfa031f00 2024-04-26T18:40:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f39effff640 1 -- 172.21.15.84:0/1926875044 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39fc00bb70 con 0x560bfa031f00 2024-04-26T18:40:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.937+0000 7f39effff640 1 -- 172.21.15.84:0/1926875044 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc004780 con 0x560bfa031f00 2024-04-26T18:40:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.938+0000 7f3a0ec07740 1 -- 172.21.15.84:0/1926875044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 msgr2=0x560bfa0322e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.938+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/1926875044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 0x560bfa0322e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f39fc013e70 tx=0x7f39fc0107a0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.938+0000 7f3a0ec07740 1 -- 172.21.15.84:0/1926875044 shutdown_connections 2024-04-26T18:40:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.938+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/1926875044 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa03d0e0 0x560bfa03f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.938+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/1926875044 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 0x560bfa03cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.947 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.938+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/1926875044 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 0x560bfa0322e0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.938+0000 7f3a0ec07740 1 -- 172.21.15.84:0/1926875044 >> 172.21.15.84:0/1926875044 conn(0x560bfa02e410 msgr2=0x560bfa02e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.938+0000 7f3a0ec07740 1 -- 172.21.15.84:0/1926875044 shutdown_connections 2024-04-26T18:40:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.938+0000 7f3a0ec07740 1 -- 172.21.15.84:0/1926875044 wait complete. 2024-04-26T18:40:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.938+0000 7f3a0ec07740 1 -- start start 2024-04-26T18:40:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0ec07740 1 Processor -- start 2024-04-26T18:40:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0ec07740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa031f00 0x560bfa03a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0ec07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 0x560bfa03a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0ec07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 0x560bfa03ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0ec07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560bfa03fcb0 con 0x560bfa03d0e0 2024-04-26T18:40:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0ec07740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560bf9f1fb60 con 0x560bfa031f00 2024-04-26T18:40:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0ec07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560bfa03fe30 con 0x560bfa032820 2024-04-26T18:40:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0ec07740 10 client.0 _ll_register_callbacks cb 0x560bf9f442b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:02.948 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0a48e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 0x560bfa03ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0a48e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 0x560bfa03ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46600/0 (socket says 172.21.15.84:46600) 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0a48e640 1 -- 172.21.15.84:0/2495272000 learned_addr learned my addr 172.21.15.84:0/2495272000 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0948c640 1 --2- 172.21.15.84:0/2495272000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 0x560bfa03a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a09c8d640 1 --2- 172.21.15.84:0/2495272000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa031f00 0x560bfa03a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0a48e640 1 -- 172.21.15.84:0/2495272000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 msgr2=0x560bfa03a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/2495272000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 0x560bfa03a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0a48e640 1 -- 172.21.15.84:0/2495272000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa031f00 msgr2=0x560bfa03a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/2495272000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa031f00 0x560bfa03a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.939+0000 7f3a0a48e640 1 -- 172.21.15.84:0/2495272000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bf9caa210 con 0x560bfa03d0e0 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/2495272000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 0x560bfa03ab90 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f39fc013630 tx=0x7f39fc0046b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fd537f10f68728 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f39ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f39ed7fa640 1 -- 172.21.15.84:0/2495272000 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc013250 con 0x560bfa03d0e0 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f39ed7fa640 1 -- 172.21.15.84:0/2495272000 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f39fc003970 con 0x560bfa03d0e0 2024-04-26T18:40:02.949 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f3a0ec07740 1 -- 172.21.15.84:0/2495272000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560bf9fa6070 con 0x560bfa03d0e0 2024-04-26T18:40:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f39ed7fa640 1 -- 172.21.15.84:0/2495272000 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39fc003b70 con 0x560bfa03d0e0 2024-04-26T18:40:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f3a0ec07740 10 client.16845 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f3a0ec07740 1 -- 172.21.15.84:0/2495272000 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560bf9d76620 con 0x560bfa03d0e0 2024-04-26T18:40:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f39ed7fa640 1 -- 172.21.15.84:0/2495272000 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560bf9fa6070 con 0x560bfa03d0e0 2024-04-26T18:40:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f39ed7fa640 1 -- 172.21.15.84:0/2495272000 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f39fc039c10 con 0x560bfa03d0e0 2024-04-26T18:40:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f3a0ec07740 10 client.16845 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f3a0ec07740 10 client.16845 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:40:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f3a0ec07740 20 client.16845 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/2495272000 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560bfa04d250 0x560bfa04f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f3a0ec07740 4 client.16845 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:40:02.950 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.940+0000 7f3a0ec07740 1 -- 172.21.15.84:0/2495272000 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bf9f9fee0 con 0x560bfa04d250 2024-04-26T18:40:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.941+0000 7f3a09c8d640 1 --2- 172.21.15.84:0/2495272000 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560bfa04d250 0x560bfa04f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.941+0000 7f3a09c8d640 1 --2- 172.21.15.84:0/2495272000 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560bfa04d250 0x560bfa04f640 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b87b77cc8221a27 server_cookie=af23baf46901e904 in_seq=0 out_seq=0 2024-04-26T18:40:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.941+0000 7f39ed7fa640 10 client.16845 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f39ed7fa640 1 -- 172.21.15.84:0/2495272000 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bf9f9fee0 con 0x560bfa04d250 2024-04-26T18:40:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f39ed7fa640 10 client.16845 handle_command_reply: tid=0 2024-04-26T18:40:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f3a0ec07740 1 client.16845 shutdown 2024-04-26T18:40:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f3a0ec07740 1 -- 172.21.15.84:0/2495272000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 msgr2=0x560bfa03ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/2495272000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 0x560bfa03ab90 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f39fc013630 tx=0x7f39fc0046b0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f3a0ec07740 1 -- 172.21.15.84:0/2495272000 shutdown_connections 2024-04-26T18:40:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/2495272000 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560bfa04d250 0x560bfa04f640 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/2495272000 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 0x560bfa03ab90 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f39ed7fa640 0 client.16845 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:02.951 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/2495272000 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 0x560bfa03a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/2495272000 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa031f00 0x560bfa03a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f3a0ec07740 1 -- 172.21.15.84:0/2495272000 >> 172.21.15.84:0/2495272000 conn(0x560bfa02e410 msgr2=0x560bfa034140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f3a0ec07740 1 -- 172.21.15.84:0/2495272000 shutdown_connections 2024-04-26T18:40:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f3a0ec07740 1 -- 172.21.15.84:0/2495272000 wait complete. 2024-04-26T18:40:02.952 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.942+0000 7f3a0ec07740 20 client.16845 trim_cache size 0 max 16384 2024-04-26T18:40:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a0ec07740 1 Processor -- start 2024-04-26T18:40:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a0ec07740 1 -- start start 2024-04-26T18:40:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a0ec07740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa031f00 0x560bfa02dd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a0ec07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 0x560bfa054e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.962 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a0ec07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 0x560bfa055340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a0ec07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041b4c70 con 0x560bfa03d0e0 2024-04-26T18:40:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a0ec07740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041b67b0 con 0x560bfa031f00 2024-04-26T18:40:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a0ec07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041b5340 con 0x560bfa032820 2024-04-26T18:40:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a0a48e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 0x560bfa055340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a0a48e640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 0x560bfa055340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46606/0 (socket says 172.21.15.84:46606) 2024-04-26T18:40:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a0a48e640 1 -- 172.21.15.84:0/552824543 learned_addr learned my addr 172.21.15.84:0/552824543 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a09c8d640 1 --2- 172.21.15.84:0/552824543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa031f00 0x560bfa02dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a0948c640 1 --2- 172.21.15.84:0/552824543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 0x560bfa054e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.957+0000 7f3a0a48e640 1 -- 172.21.15.84:0/552824543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 msgr2=0x560bfa054e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/552824543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 0x560bfa054e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f3a0a48e640 1 -- 172.21.15.84:0/552824543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa031f00 msgr2=0x560bfa02dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/552824543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa031f00 0x560bfa02dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.963 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f3a0a48e640 1 -- 172.21.15.84:0/552824543 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bfa038660 con 0x560bfa03d0e0 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/552824543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 0x560bfa055340 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f39fc004210 tx=0x7f39fc0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23aa6d9205560a16 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f39ef7fe640 1 -- 172.21.15.84:0/552824543 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc0139d0 con 0x560bfa03d0e0 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f39ef7fe640 1 -- 172.21.15.84:0/552824543 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39fc0126a0 con 0x560bfa03d0e0 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f39ef7fe640 1 -- 172.21.15.84:0/552824543 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39fc012e50 con 0x560bfa03d0e0 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f3a0ec07740 1 -- 172.21.15.84:0/552824543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 msgr2=0x560bfa055340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/552824543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 0x560bfa055340 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f39fc004210 tx=0x7f39fc0107a0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f3a0ec07740 1 -- 172.21.15.84:0/552824543 shutdown_connections 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/552824543 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa03d0e0 0x560bfa055340 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/552824543 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa032820 0x560bfa054e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/552824543 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa031f00 0x560bfa02dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f3a0ec07740 1 -- 172.21.15.84:0/552824543 >> 172.21.15.84:0/552824543 conn(0x560bfa02e410 msgr2=0x560bfa03c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.958+0000 7f3a0ec07740 1 -- 172.21.15.84:0/552824543 shutdown_connections 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a0ec07740 1 -- 172.21.15.84:0/552824543 wait complete. 2024-04-26T18:40:02.964 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a0ec07740 1 -- start start 2024-04-26T18:40:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a0ec07740 1 Processor -- start 2024-04-26T18:40:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a0ec07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 0x560bfa045260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a0ec07740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa032820 0x560bfa040620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a0ec07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa03d0e0 0x560bfa040b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a0ec07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041b60e0 con 0x560bfa031f00 2024-04-26T18:40:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a0ec07740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041b5a10 con 0x560bfa032820 2024-04-26T18:40:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a0ec07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3a041b2c70 con 0x560bfa03d0e0 2024-04-26T18:40:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a0ec07740 10 client.0 _ll_register_callbacks cb 0x7f39f8005b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a09c8d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 0x560bfa045260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a09c8d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 0x560bfa045260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:46620/0 (socket says 172.21.15.84:46620) 2024-04-26T18:40:02.965 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a09c8d640 1 -- 172.21.15.84:0/280008464 learned_addr learned my addr 172.21.15.84:0/280008464 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.959+0000 7f3a0948c640 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa032820 0x560bfa040620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa03d0e0 0x560bfa040b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f3a09c8d640 1 -- 172.21.15.84:0/280008464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa03d0e0 msgr2=0x560bfa040b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f3a09c8d640 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa03d0e0 0x560bfa040b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f3a09c8d640 1 -- 172.21.15.84:0/280008464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa032820 msgr2=0x560bfa040620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f3a09c8d640 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa032820 0x560bfa040620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f3a09c8d640 1 -- 172.21.15.84:0/280008464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bfa0411a0 con 0x560bfa031f00 2024-04-26T18:40:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f3a0a48e640 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa03d0e0 0x560bfa040b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:40:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f3a09c8d640 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 0x560bfa045260 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000cc40 tx=0x7f3a0001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e80b89e27ae88321 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f39ed7fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f39ed7fa640 1 -- 172.21.15.84:0/280008464 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3a00004020 con 0x560bfa031f00 2024-04-26T18:40:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f39ed7fa640 1 -- 172.21.15.84:0/280008464 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 8579+0+0 (secure 0 0 0) 0x7f3a000041c0 con 0x560bfa031f00 2024-04-26T18:40:02.966 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f3a0ec07740 1 -- 172.21.15.84:0/280008464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39fc0126a0 con 0x560bfa031f00 2024-04-26T18:40:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f39ed7fa640 1 -- 172.21.15.84:0/280008464 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3a000043c0 con 0x560bfa031f00 2024-04-26T18:40:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f3a0ec07740 10 client.16851 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f3a0ec07740 1 -- 172.21.15.84:0/280008464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560bfa0b34f0 con 0x560bfa031f00 2024-04-26T18:40:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f39ed7fa640 1 -- 172.21.15.84:0/280008464 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39fc0126a0 con 0x560bfa031f00 2024-04-26T18:40:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f39ed7fa640 1 -- 172.21.15.84:0/280008464 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f3a00024680 con 0x560bfa031f00 2024-04-26T18:40:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f3a0ec07740 10 client.16851 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.960+0000 7f3a0ec07740 10 client.16851 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:40:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.961+0000 7f3a0ec07740 20 client.16851 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.961+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560bfa0b3c20 0x560bfa0b6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.961+0000 7f3a0ec07740 4 client.16851 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:40:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.961+0000 7f3a0ec07740 1 -- 172.21.15.84:0/280008464 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560bf9f1b5f0 con 0x560bfa0b3c20 2024-04-26T18:40:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.961+0000 7f3a0948c640 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560bfa0b3c20 0x560bfa0b6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:02.967 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.961+0000 7f39ed7fa640 10 client.16851 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:02.968 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.961+0000 7f3a0948c640 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560bfa0b3c20 0x560bfa0b6040 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb0df38a35e33ba server_cookie=92e21071a3c53784 in_seq=0 out_seq=0 2024-04-26T18:40:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f39ed7fa640 1 -- 172.21.15.84:0/280008464 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560bf9f1b5f0 con 0x560bfa0b3c20 2024-04-26T18:40:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f39ed7fa640 10 client.16851 handle_command_reply: tid=0 2024-04-26T18:40:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f3a0ec07740 1 client.16851 shutdown 2024-04-26T18:40:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f3a0ec07740 1 -- 172.21.15.84:0/280008464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 msgr2=0x560bfa045260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 0x560bfa045260 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000cc40 tx=0x7f3a0001ca20 comp rx=0 tx=0).stop 2024-04-26T18:40:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f3a0ec07740 1 -- 172.21.15.84:0/280008464 shutdown_connections 2024-04-26T18:40:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560bfa0b3c20 0x560bfa0b6040 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560bfa03d0e0 0x560bfa040b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560bfa032820 0x560bfa040620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.977 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f39ed7fa640 0 client.16851 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f3a0ec07740 1 --2- 172.21.15.84:0/280008464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560bfa031f00 0x560bfa045260 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f3a0ec07740 1 -- 172.21.15.84:0/280008464 >> 172.21.15.84:0/280008464 conn(0x560bfa02e410 msgr2=0x560bfa03a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f3a0ec07740 1 -- 172.21.15.84:0/280008464 shutdown_connections 2024-04-26T18:40:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f3a0ec07740 1 -- 172.21.15.84:0/280008464 wait complete. 2024-04-26T18:40:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.976+0000 7f3a0ec07740 20 client.16851 trim_cache size 0 max 16384 2024-04-26T18:40:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:40:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:40:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:40:02.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:40:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.977+0000 7f3a0bef1640 1 -- 172.21.15.84:0/760002737 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f39f00a3dc0 msgr2=0x7f39f00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.977+0000 7f3a0bef1640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f39f00a3dc0 0x7f39f00a6280 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f39fc010ca0 tx=0x7f39fc015000 comp rx=0 tx=0).stop 2024-04-26T18:40:02.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.977+0000 7f3a0bef1640 1 -- 172.21.15.84:0/760002737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a041022f0 msgr2=0x7f3a0419fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.977+0000 7f3a0bef1640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a041022f0 0x7f3a0419fc70 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f39f8005940 tx=0x7f39f8004270 comp rx=0 tx=0).stop 2024-04-26T18:40:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.977+0000 7f3a0bef1640 1 -- 172.21.15.84:0/760002737 shutdown_connections 2024-04-26T18:40:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.977+0000 7f3a0bef1640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f39f00a3dc0 0x7f39f00a6280 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.977+0000 7f3a0bef1640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3a04106dd0 0x7f3a041a3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.977+0000 7f3a0bef1640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3a041022f0 0x7f3a0419fc70 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.977+0000 7f3a0bef1640 1 --2- 172.21.15.84:0/760002737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3a041019d0 0x7f3a0419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.977+0000 7f3a0bef1640 1 -- 172.21.15.84:0/760002737 >> 172.21.15.84:0/760002737 conn(0x7f3a04077bb0 msgr2=0x7f3a04104780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.977+0000 7f3a0bef1640 1 -- 172.21.15.84:0/760002737 shutdown_connections 2024-04-26T18:40:02.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:02.977+0000 7f3a0bef1640 1 -- 172.21.15.84:0/760002737 wait complete. 2024-04-26T18:40:03.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:02 smithi084 ceph-mon[29141]: pgmap v377: 97 pgs: 97 active+clean; 3.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-26T18:40:03.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:02 smithi103 ceph-mon[32727]: pgmap v377: 97 pgs: 97 active+clean; 3.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-26T18:40:03.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:02 smithi174 ceph-mon[32705]: pgmap v377: 97 pgs: 97 active+clean; 3.6 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-26T18:40:05.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:04 smithi084 ceph-mon[29141]: pgmap v378: 97 pgs: 97 active+clean; 3.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 110 KiB/s wr, 41 op/s 2024-04-26T18:40:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:04 smithi103 ceph-mon[32727]: pgmap v378: 97 pgs: 97 active+clean; 3.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 110 KiB/s wr, 41 op/s 2024-04-26T18:40:05.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:04 smithi174 ceph-mon[32705]: pgmap v378: 97 pgs: 97 active+clean; 3.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 110 KiB/s wr, 41 op/s 2024-04-26T18:40:06.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:05 smithi084 ceph-mon[29141]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T18:40:06.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:05 smithi103 ceph-mon[32727]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T18:40:06.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:05 smithi174 ceph-mon[32705]: osdmap e107: 12 total, 12 up, 12 in 2024-04-26T18:40:07.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:06 smithi084 ceph-mon[29141]: pgmap v380: 97 pgs: 97 active+clean; 3.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:40:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:06 smithi103 ceph-mon[32727]: pgmap v380: 97 pgs: 97 active+clean; 3.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:40:07.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:06 smithi174 ceph-mon[32705]: pgmap v380: 97 pgs: 97 active+clean; 3.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:40:09.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:09 smithi084 ceph-mon[29141]: pgmap v381: 97 pgs: 97 active+clean; 3.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:40:09.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:09 smithi103 ceph-mon[32727]: pgmap v381: 97 pgs: 97 active+clean; 3.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:40:09.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:09 smithi174 ceph-mon[32705]: pgmap v381: 97 pgs: 97 active+clean; 3.9 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:40:10.442 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 26.4 2024-04-26T18:40:10.442 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:40:10.700 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:40:11.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:11 smithi084 ceph-mon[29141]: pgmap v382: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:40:11.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:11 smithi103 ceph-mon[32727]: pgmap v382: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:40:11.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:11 smithi174 ceph-mon[32705]: pgmap v382: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:40:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.502+0000 7fcef5b31700 1 -- 172.21.15.84:0/569053566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef01094f0 msgr2=0x7fcef01098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.502+0000 7fcef5b31700 1 --2- 172.21.15.84:0/569053566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef01094f0 0x7fcef01098d0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fcee8009990 tx=0x7fcee8009ca0 comp rx=0 tx=0).stop 2024-04-26T18:40:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.502+0000 7fcef5b31700 1 -- 172.21.15.84:0/569053566 shutdown_connections 2024-04-26T18:40:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.502+0000 7fcef5b31700 1 --2- 172.21.15.84:0/569053566 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef01094f0 0x7fcef01098d0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.502+0000 7fcef5b31700 1 --2- 172.21.15.84:0/569053566 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef0103e90 0x7fcef01102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.502+0000 7fcef5b31700 1 --2- 172.21.15.84:0/569053566 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcef01034d0 0x7fcef0103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.502+0000 7fcef5b31700 1 -- 172.21.15.84:0/569053566 >> 172.21.15.84:0/569053566 conn(0x7fcef007a500 msgr2=0x7fcef007a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.502+0000 7fcef5b31700 1 -- 172.21.15.84:0/569053566 shutdown_connections 2024-04-26T18:40:12.515 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.502+0000 7fcef5b31700 1 -- 172.21.15.84:0/569053566 wait complete. 2024-04-26T18:40:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fcef5b31700 1 Processor -- start 2024-04-26T18:40:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fcef5b31700 1 -- start start 2024-04-26T18:40:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fcef5b31700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef01034d0 0x7fcef019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fcef5b31700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcef0103e90 0x7fcef019fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fcef5b31700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef01094f0 0x7fcef0199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fcef5b31700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcef01136f0 con 0x7fcef01034d0 2024-04-26T18:40:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fcef5b31700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcef0113570 con 0x7fcef0103e90 2024-04-26T18:40:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fcef5b31700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcef0113870 con 0x7fcef01094f0 2024-04-26T18:40:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef01034d0 0x7fcef019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fceef7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef01034d0 0x7fcef019f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44570/0 (socket says 172.21.15.84:44570) 2024-04-26T18:40:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fceef7fe700 1 -- 172.21.15.84:0/3328665528 learned_addr learned my addr 172.21.15.84:0/3328665528 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fceeeffd700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcef0103e90 0x7fcef019fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:12.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fceeffff700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef01094f0 0x7fcef0199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fceef7fe700 1 -- 172.21.15.84:0/3328665528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef01094f0 msgr2=0x7fcef0199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fceef7fe700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef01094f0 0x7fcef0199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fceef7fe700 1 -- 172.21.15.84:0/3328665528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcef0103e90 msgr2=0x7fcef019fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fceef7fe700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcef0103e90 0x7fcef019fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fceef7fe700 1 -- 172.21.15.84:0/3328665528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcef019a1d0 con 0x7fcef01034d0 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.503+0000 7fceeeffd700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcef0103e90 0x7fcef019fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.504+0000 7fceef7fe700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef01034d0 0x7fcef019f8b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcee8000c00 tx=0x7fcee80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.504+0000 7fceecff9700 1 -- 172.21.15.84:0/3328665528 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee8027070 con 0x7fcef01034d0 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.504+0000 7fcef5b31700 1 -- 172.21.15.84:0/3328665528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcef019a460 con 0x7fcef01034d0 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.504+0000 7fceecff9700 1 -- 172.21.15.84:0/3328665528 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcee80041d0 con 0x7fcef01034d0 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.504+0000 7fceecff9700 1 -- 172.21.15.84:0/3328665528 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcee800f460 con 0x7fcef01034d0 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.504+0000 7fcef5b31700 1 -- 172.21.15.84:0/3328665528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcef01a6780 con 0x7fcef01034d0 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.505+0000 7fcef5b31700 1 -- 172.21.15.84:0/3328665528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcef006a110 con 0x7fcef01034d0 2024-04-26T18:40:12.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.506+0000 7fceecff9700 1 -- 172.21.15.84:0/3328665528 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fcee802a070 con 0x7fcef01034d0 2024-04-26T18:40:12.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.506+0000 7fceecff9700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fced80a3db0 0x7fced80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:12.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.506+0000 7fceeeffd700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fced80a3db0 0x7fced80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:12.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.507+0000 7fceecff9700 1 -- 172.21.15.84:0/3328665528 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fcee80d1c30 con 0x7fcef01034d0 2024-04-26T18:40:12.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.507+0000 7fceeeffd700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fced80a3db0 0x7fced80a6270 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcef007ac50 tx=0x7fcee4008040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:12.518 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.510+0000 7fceecff9700 1 -- 172.21.15.84:0/3328665528 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcee8031070 con 0x7fcef01034d0 2024-04-26T18:40:12.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.762+0000 7fcef5b31700 1 -- 172.21.15.84:0/3328665528 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcef002e490 con 0x7fcef01034d0 2024-04-26T18:40:12.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.763+0000 7fceecff9700 1 -- 172.21.15.84:0/3328665528 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7fcee8013070 con 0x7fcef01034d0 2024-04-26T18:40:12.763 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:40:12.763 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:40:12.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.765+0000 7fcef5b31700 1 -- 172.21.15.84:0/3328665528 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fced80a3db0 msgr2=0x7fced80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:12.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.765+0000 7fcef5b31700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fced80a3db0 0x7fced80a6270 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fcef007ac50 tx=0x7fcee4008040 comp rx=0 tx=0).stop 2024-04-26T18:40:12.765 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.765+0000 7fcef5b31700 1 -- 172.21.15.84:0/3328665528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef01034d0 msgr2=0x7fcef019f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:12.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.765+0000 7fcef5b31700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef01034d0 0x7fcef019f8b0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fcee8000c00 tx=0x7fcee80047b0 comp rx=0 tx=0).stop 2024-04-26T18:40:12.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.765+0000 7fcef5b31700 1 -- 172.21.15.84:0/3328665528 shutdown_connections 2024-04-26T18:40:12.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.765+0000 7fcef5b31700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fced80a3db0 0x7fced80a6270 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:12.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.765+0000 7fcef5b31700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcef01034d0 0x7fcef019f8b0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:12.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.765+0000 7fcef5b31700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcef01094f0 0x7fcef0199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:12.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.765+0000 7fcef5b31700 1 --2- 172.21.15.84:0/3328665528 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcef0103e90 0x7fcef019fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:12.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.765+0000 7fcef5b31700 1 -- 172.21.15.84:0/3328665528 >> 172.21.15.84:0/3328665528 conn(0x7fcef007a500 msgr2=0x7fcef01023c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:12.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.765+0000 7fcef5b31700 1 -- 172.21.15.84:0/3328665528 shutdown_connections 2024-04-26T18:40:12.766 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:12.765+0000 7fcef5b31700 1 -- 172.21.15.84:0/3328665528 wait complete. 2024-04-26T18:40:12.766 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:40:13.630 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 60 --expiration 86.4 --await-for 42.0' 2024-04-26T18:40:13.630 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db / --timeout 60 --expiration 86.4 --await-for 42.0 2024-04-26T18:40:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:13 smithi084 ceph-mon[29141]: pgmap v383: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:40:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:13 smithi084 ceph-mon[29141]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T18:40:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:13 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3328665528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:40:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:13 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:40:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:13 smithi103 ceph-mon[32727]: pgmap v383: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:40:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:13 smithi103 ceph-mon[32727]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T18:40:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:13 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3328665528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:40:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:13 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:40:13.901 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:40:13.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:13 smithi174 ceph-mon[32705]: pgmap v383: 97 pgs: 97 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:40:13.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:13 smithi174 ceph-mon[32705]: osdmap e108: 12 total, 12 up, 12 in 2024-04-26T18:40:13.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:13 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3328665528' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:40:13.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:13 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:40:14.537 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:14 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:40:14.537 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:14 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:40:14.537 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:14 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:40:14.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:14 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:40:14.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:14 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:40:14.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:14 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:40:14.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:14 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:40:14.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:14 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:40:14.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:14 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:40:15.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.618+0000 7f27a6f85700 1 -- 172.21.15.84:0/2104492314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27a0103e90 msgr2=0x7f27a01102c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.618+0000 7f27a6f85700 1 --2- 172.21.15.84:0/2104492314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27a0103e90 0x7f27a01102c0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f279800b0d0 tx=0x7f279800b3e0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.618+0000 7f27a6f85700 1 -- 172.21.15.84:0/2104492314 shutdown_connections 2024-04-26T18:40:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.618+0000 7f27a6f85700 1 --2- 172.21.15.84:0/2104492314 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27a0103e90 0x7f27a01102c0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.618+0000 7f27a6f85700 1 --2- 172.21.15.84:0/2104492314 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27a01094a0 0x7f27a0109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.618+0000 7f27a6f85700 1 --2- 172.21.15.84:0/2104492314 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27a01034d0 0x7f27a0103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.618+0000 7f27a6f85700 1 -- 172.21.15.84:0/2104492314 >> 172.21.15.84:0/2104492314 conn(0x7f27a007a4f0 msgr2=0x7f27a007a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.618+0000 7f27a6f85700 1 -- 172.21.15.84:0/2104492314 shutdown_connections 2024-04-26T18:40:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.618+0000 7f27a6f85700 1 -- 172.21.15.84:0/2104492314 wait complete. 2024-04-26T18:40:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.619+0000 7f27a6f85700 1 Processor -- start 2024-04-26T18:40:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.619+0000 7f27a6f85700 1 -- start start 2024-04-26T18:40:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.619+0000 7f27a6f85700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27a01034d0 0x7f27a019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.621 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.619+0000 7f27a6f85700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27a0103e90 0x7f27a019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.619+0000 7f27a6f85700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27a01094a0 0x7f27a0199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.619+0000 7f27a6f85700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01136a0 con 0x7f27a0103e90 2024-04-26T18:40:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.619+0000 7f27a6f85700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0113520 con 0x7f27a01094a0 2024-04-26T18:40:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.619+0000 7f27a6f85700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0113820 con 0x7f27a01034d0 2024-04-26T18:40:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.619+0000 7f279ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27a0103e90 0x7f27a019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.619+0000 7f279ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27a0103e90 0x7f27a019fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44578/0 (socket says 172.21.15.84:44578) 2024-04-26T18:40:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.619+0000 7f279ffff700 1 -- 172.21.15.84:0/3576166930 learned_addr learned my addr 172.21.15.84:0/3576166930 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f27a4d21700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27a01034d0 0x7f27a019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f27a5522700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27a01094a0 0x7f27a0199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f279ffff700 1 -- 172.21.15.84:0/3576166930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27a01034d0 msgr2=0x7f27a019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:15.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f279ffff700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27a01034d0 0x7f27a019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f279ffff700 1 -- 172.21.15.84:0/3576166930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27a01094a0 msgr2=0x7f27a0199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f279ffff700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27a01094a0 0x7f27a0199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f279ffff700 1 -- 172.21.15.84:0/3576166930 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a0199fc0 con 0x7f27a0103e90 2024-04-26T18:40:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f27a4d21700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27a01034d0 0x7f27a019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f27a5522700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27a01094a0 0x7f27a0199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f279ffff700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27a0103e90 0x7f27a019fbe0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f278c00b530 tx=0x7f278c00b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f279dffb700 1 -- 172.21.15.84:0/3576166930 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c015020 con 0x7f27a0103e90 2024-04-26T18:40:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f279dffb700 1 -- 172.21.15.84:0/3576166930 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278c011ce0 con 0x7f27a0103e90 2024-04-26T18:40:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f27a6f85700 1 -- 172.21.15.84:0/3576166930 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a019a2b0 con 0x7f27a0103e90 2024-04-26T18:40:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f279dffb700 1 -- 172.21.15.84:0/3576166930 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c010d20 con 0x7f27a0103e90 2024-04-26T18:40:15.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.620+0000 7f27a6f85700 1 -- 172.21.15.84:0/3576166930 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a01a6570 con 0x7f27a0103e90 2024-04-26T18:40:15.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.622+0000 7f279dffb700 1 -- 172.21.15.84:0/3576166930 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f278c018020 con 0x7f27a0103e90 2024-04-26T18:40:15.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.623+0000 7f279dffb700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f27900a3e50 0x7f27900a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.623+0000 7f27a4d21700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f27900a3e50 0x7f27900a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.623+0000 7f279dffb700 1 -- 172.21.15.84:0/3576166930 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f278c0cffe0 con 0x7f27a0103e90 2024-04-26T18:40:15.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.623+0000 7f27a4d21700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f27900a3e50 0x7f27900a6310 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2794005fd0 tx=0x7f2794005de0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:15.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.624+0000 7f27b1817b80 1 Processor -- start 2024-04-26T18:40:15.625 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.624+0000 7f27b1817b80 1 -- start start 2024-04-26T18:40:15.626 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:15 smithi084 ceph-mon[29141]: pgmap v385: 97 pgs: 97 active+clean; 4.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 17 op/s 2024-04-26T18:40:15.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.624+0000 7f27b1817b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748abbf30 0x563748abc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.624+0000 7f27b1817b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abc850 0x563748ab62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.631 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.624+0000 7f27b1817b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748ab6900 0x563748ab6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.624+0000 7f27b1817b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563748a089c0 con 0x563748ab6900 2024-04-26T18:40:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.624+0000 7f27b1817b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56374896eac0 con 0x563748abc850 2024-04-26T18:40:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.624+0000 7f27b1817b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563748a04980 con 0x563748abbf30 2024-04-26T18:40:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.624+0000 7f27a5522700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748ab6900 0x563748ab6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.624+0000 7f27a5522700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748ab6900 0x563748ab6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44592/0 (socket says 172.21.15.84:44592) 2024-04-26T18:40:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.624+0000 7f27a5522700 1 -- 172.21.15.84:0/3918013844 learned_addr learned my addr 172.21.15.84:0/3918013844 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.625+0000 7f279ffff700 1 --2- 172.21.15.84:0/3918013844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abc850 0x563748ab62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.625+0000 7f27a4d21700 1 --2- 172.21.15.84:0/3918013844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748abbf30 0x563748abc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.625+0000 7f27a5522700 1 -- 172.21.15.84:0/3918013844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748abbf30 msgr2=0x563748abc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:15.632 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.625+0000 7f27a5522700 1 --2- 172.21.15.84:0/3918013844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748abbf30 0x563748abc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.625+0000 7f27a5522700 1 -- 172.21.15.84:0/3918013844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abc850 msgr2=0x563748ab62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:15.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.625+0000 7f27a5522700 1 --2- 172.21.15.84:0/3918013844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abc850 0x563748ab62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.625+0000 7f27a5522700 1 -- 172.21.15.84:0/3918013844 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563748955a60 con 0x563748ab6900 2024-04-26T18:40:15.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.625+0000 7f279ffff700 1 --2- 172.21.15.84:0/3918013844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abc850 0x563748ab62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:15.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.625+0000 7f27a5522700 1 --2- 172.21.15.84:0/3918013844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748ab6900 0x563748ab6d60 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2798009d10 tx=0x7f2798009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba440e5b2b07c534 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:15.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.625+0000 7f278affd700 1 -- 172.21.15.84:0/3918013844 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798020070 con 0x563748ab6900 2024-04-26T18:40:15.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.625+0000 7f278affd700 1 -- 172.21.15.84:0/3918013844 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27980093d0 con 0x563748ab6900 2024-04-26T18:40:15.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.625+0000 7f278affd700 1 -- 172.21.15.84:0/3918013844 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279801c440 con 0x563748ab6900 2024-04-26T18:40:15.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.626+0000 7f27b1817b80 1 -- 172.21.15.84:0/3918013844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748ab6900 msgr2=0x563748ab6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:15.633 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.626+0000 7f27b1817b80 1 --2- 172.21.15.84:0/3918013844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748ab6900 0x563748ab6d60 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2798009d10 tx=0x7f2798009d40 comp rx=0 tx=0).stop 2024-04-26T18:40:15.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.626+0000 7f27b1817b80 1 -- 172.21.15.84:0/3918013844 shutdown_connections 2024-04-26T18:40:15.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.626+0000 7f27b1817b80 1 --2- 172.21.15.84:0/3918013844 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748ab6900 0x563748ab6d60 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.626+0000 7f27b1817b80 1 --2- 172.21.15.84:0/3918013844 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748abbf30 0x563748abc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.626+0000 7f27b1817b80 1 --2- 172.21.15.84:0/3918013844 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abc850 0x563748ab62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.634 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.626+0000 7f27b1817b80 1 -- 172.21.15.84:0/3918013844 >> 172.21.15.84:0/3918013844 conn(0x5637489e4750 msgr2=0x563748ab1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:15.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.626+0000 7f27b1817b80 1 -- 172.21.15.84:0/3918013844 shutdown_connections 2024-04-26T18:40:15.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.626+0000 7f27b1817b80 1 -- 172.21.15.84:0/3918013844 wait complete. 2024-04-26T18:40:15.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.626+0000 7f27b1817b80 1 -- start start 2024-04-26T18:40:15.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.626+0000 7f27b1817b80 1 Processor -- start 2024-04-26T18:40:15.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f27b1817b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748ab6900 0x563748ac24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f27b1817b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abbf30 0x563748abf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f27b1817b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abc850 0x563748abfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f27b1817b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563748ac3260 con 0x563748abbf30 2024-04-26T18:40:15.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f27b1817b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5637489c2c10 con 0x563748abc850 2024-04-26T18:40:15.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f27b1817b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563748ac33e0 con 0x563748ab6900 2024-04-26T18:40:15.635 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f27b1817b80 10 client.0 _ll_register_callbacks cb 0x5637489a0ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f279ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abbf30 0x563748abf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f279ffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abbf30 0x563748abf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44604/0 (socket says 172.21.15.84:44604) 2024-04-26T18:40:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f279ffff700 1 -- 172.21.15.84:0/1745727379 learned_addr learned my addr 172.21.15.84:0/1745727379 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f27a4d21700 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748ab6900 0x563748ac24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f27a5522700 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abc850 0x563748abfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f279ffff700 1 -- 172.21.15.84:0/1745727379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748ab6900 msgr2=0x563748ac24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f279ffff700 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748ab6900 0x563748ac24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f279ffff700 1 -- 172.21.15.84:0/1745727379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abc850 msgr2=0x563748abfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f279ffff700 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abc850 0x563748abfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f279ffff700 1 -- 172.21.15.84:0/1745727379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637489c6610 con 0x563748abbf30 2024-04-26T18:40:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f27a4d21700 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748ab6900 0x563748ac24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:40:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.627+0000 7f27a5522700 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abc850 0x563748abfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:15.636 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f279ffff700 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abbf30 0x563748abf840 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f278c09bcf0 tx=0x7f278c09c0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1068830ab9449304 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:15.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f277ffff700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:15.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f277ffff700 1 -- 172.21.15.84:0/1745727379 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278c09c450 con 0x563748abbf30 2024-04-26T18:40:15.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f277ffff700 1 -- 172.21.15.84:0/1745727379 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f278c0b5300 con 0x563748abbf30 2024-04-26T18:40:15.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f27b1817b80 1 -- 172.21.15.84:0/1745727379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27980093d0 con 0x563748abbf30 2024-04-26T18:40:15.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f277ffff700 1 -- 172.21.15.84:0/1745727379 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f278c0b5500 con 0x563748abbf30 2024-04-26T18:40:15.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f27b1817b80 10 client.16869 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:15.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f27b1817b80 1 -- 172.21.15.84:0/1745727379 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563748acd9e0 con 0x563748abbf30 2024-04-26T18:40:15.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f277ffff700 1 -- 172.21.15.84:0/1745727379 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27980093d0 con 0x563748abbf30 2024-04-26T18:40:15.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f277ffff700 1 -- 172.21.15.84:0/1745727379 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f278c096c80 con 0x563748abbf30 2024-04-26T18:40:15.637 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f27b1817b80 10 client.16869 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:15.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f27b1817b80 10 client.16869 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:40:15.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f27b1817b80 20 client.16869 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:15.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f27b1817b80 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563748acff70 0x563748ad2360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f27b1817b80 4 client.16869 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:40:15.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.628+0000 7f27b1817b80 1 -- 172.21.15.84:0/1745727379 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563748a163a0 con 0x563748acff70 2024-04-26T18:40:15.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.629+0000 7f27a4d21700 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563748acff70 0x563748ad2360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.629+0000 7f277ffff700 10 client.16869 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:15.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.629+0000 7f27a4d21700 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563748acff70 0x563748ad2360 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d8d234523daddaf server_cookie=b74de10c56af3c54 in_seq=0 out_seq=0 2024-04-26T18:40:15.638 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f277ffff700 1 -- 172.21.15.84:0/1745727379 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563748a163a0 con 0x563748acff70 2024-04-26T18:40:15.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f277ffff700 10 client.16869 handle_command_reply: tid=0 2024-04-26T18:40:15.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f27b1817b80 1 client.16869 shutdown 2024-04-26T18:40:15.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f27b1817b80 1 -- 172.21.15.84:0/1745727379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abbf30 msgr2=0x563748abf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:15.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f27b1817b80 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abbf30 0x563748abf840 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f278c09bcf0 tx=0x7f278c09c0a0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f27b1817b80 1 -- 172.21.15.84:0/1745727379 shutdown_connections 2024-04-26T18:40:15.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f27b1817b80 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563748acff70 0x563748ad2360 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f27b1817b80 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abc850 0x563748abfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f277ffff700 0 client.16869 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:15.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f27b1817b80 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748ab6900 0x563748ac24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f27b1817b80 1 --2- 172.21.15.84:0/1745727379 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abbf30 0x563748abf840 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f27b1817b80 1 -- 172.21.15.84:0/1745727379 >> 172.21.15.84:0/1745727379 conn(0x5637489e4750 msgr2=0x563748ab52d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:15.639 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f27b1817b80 1 -- 172.21.15.84:0/1745727379 shutdown_connections 2024-04-26T18:40:15.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f27b1817b80 1 -- 172.21.15.84:0/1745727379 wait complete. 2024-04-26T18:40:15.640 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.630+0000 7f27b1817b80 20 client.16869 trim_cache size 0 max 16384 2024-04-26T18:40:15.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.649+0000 7f27b1817b80 1 Processor -- start 2024-04-26T18:40:15.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.649+0000 7f27b1817b80 1 -- start start 2024-04-26T18:40:15.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.649+0000 7f27b1817b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748ab6900 0x563748ab62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.649+0000 7f27b1817b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748abbf30 0x563748acc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.649+0000 7f27b1817b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 0x563748abd360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.649+0000 7f27b1817b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01add90 con 0x563748abc850 2024-04-26T18:40:15.654 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.649+0000 7f27b1817b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01af8d0 con 0x563748ab6900 2024-04-26T18:40:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.649+0000 7f27b1817b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01ae460 con 0x563748abbf30 2024-04-26T18:40:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f27a5522700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 0x563748abd360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f27a5522700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 0x563748abd360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44616/0 (socket says 172.21.15.84:44616) 2024-04-26T18:40:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f27a5522700 1 -- 172.21.15.84:0/3021874386 learned_addr learned my addr 172.21.15.84:0/3021874386 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f279ffff700 1 --2- 172.21.15.84:0/3021874386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748abbf30 0x563748acc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f27a4d21700 1 --2- 172.21.15.84:0/3021874386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748ab6900 0x563748ab62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f27a5522700 1 -- 172.21.15.84:0/3021874386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748abbf30 msgr2=0x563748acc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f27a5522700 1 --2- 172.21.15.84:0/3021874386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748abbf30 0x563748acc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f27a5522700 1 -- 172.21.15.84:0/3021874386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748ab6900 msgr2=0x563748ab62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f27a5522700 1 --2- 172.21.15.84:0/3021874386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748ab6900 0x563748ab62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f27a5522700 1 -- 172.21.15.84:0/3021874386 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563748aad880 con 0x563748abc850 2024-04-26T18:40:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f27a4d21700 1 --2- 172.21.15.84:0/3021874386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748ab6900 0x563748ab62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:40:15.655 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f27a5522700 1 --2- 172.21.15.84:0/3021874386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 0x563748abd360 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f279800d010 tx=0x7f279800b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f908d881839bc2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:15.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f278a7fc700 1 -- 172.21.15.84:0/3021874386 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279802c070 con 0x563748abc850 2024-04-26T18:40:15.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f278a7fc700 1 -- 172.21.15.84:0/3021874386 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2798009250 con 0x563748abc850 2024-04-26T18:40:15.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.650+0000 7f278a7fc700 1 -- 172.21.15.84:0/3021874386 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798020050 con 0x563748abc850 2024-04-26T18:40:15.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.651+0000 7f27b1817b80 1 -- 172.21.15.84:0/3021874386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 msgr2=0x563748abd360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:15.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.651+0000 7f27b1817b80 1 --2- 172.21.15.84:0/3021874386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 0x563748abd360 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f279800d010 tx=0x7f279800b3e0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.651+0000 7f27b1817b80 1 -- 172.21.15.84:0/3021874386 shutdown_connections 2024-04-26T18:40:15.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.651+0000 7f27b1817b80 1 --2- 172.21.15.84:0/3021874386 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 0x563748abd360 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.651+0000 7f27b1817b80 1 --2- 172.21.15.84:0/3021874386 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748ab6900 0x563748ab62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.656 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.651+0000 7f27b1817b80 1 --2- 172.21.15.84:0/3021874386 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748abbf30 0x563748acc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.651+0000 7f27b1817b80 1 -- 172.21.15.84:0/3021874386 >> 172.21.15.84:0/3021874386 conn(0x5637489e4750 msgr2=0x563748aaaf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:15.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.651+0000 7f27b1817b80 1 -- 172.21.15.84:0/3021874386 shutdown_connections 2024-04-26T18:40:15.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.651+0000 7f27b1817b80 1 -- 172.21.15.84:0/3021874386 wait complete. 2024-04-26T18:40:15.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.651+0000 7f27b1817b80 1 -- start start 2024-04-26T18:40:15.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.651+0000 7f27b1817b80 1 Processor -- start 2024-04-26T18:40:15.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27b1817b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748ab6900 0x563748ac8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27b1817b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abbf30 0x563748ac5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27b1817b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 0x563748ac61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27b1817b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01af200 con 0x563748abc850 2024-04-26T18:40:15.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27b1817b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01aeb30 con 0x563748abbf30 2024-04-26T18:40:15.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27b1817b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01abd50 con 0x563748ab6900 2024-04-26T18:40:15.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27b1817b80 10 client.0 _ll_register_callbacks cb 0x5637489a0ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:15.657 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27a5522700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 0x563748ac61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27a5522700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 0x563748ac61b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44628/0 (socket says 172.21.15.84:44628) 2024-04-26T18:40:15.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27a5522700 1 -- 172.21.15.84:0/1717197164 learned_addr learned my addr 172.21.15.84:0/1717197164 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:15.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f279ffff700 1 --2- 172.21.15.84:0/1717197164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abbf30 0x563748ac5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27a4d21700 1 --2- 172.21.15.84:0/1717197164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748ab6900 0x563748ac8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27a5522700 1 -- 172.21.15.84:0/1717197164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748ab6900 msgr2=0x563748ac8730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:15.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27a5522700 1 --2- 172.21.15.84:0/1717197164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748ab6900 0x563748ac8730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27a5522700 1 -- 172.21.15.84:0/1717197164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abbf30 msgr2=0x563748ac5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:15.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27a5522700 1 --2- 172.21.15.84:0/1717197164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abbf30 0x563748ac5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27a5522700 1 -- 172.21.15.84:0/1717197164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563748aa92c0 con 0x563748abc850 2024-04-26T18:40:15.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f27a5522700 1 --2- 172.21.15.84:0/1717197164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 0x563748ac61b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f279800b080 tx=0x7f2798004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47da41ee74d56abe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:15.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f277ffff700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:15.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.652+0000 7f277ffff700 1 -- 172.21.15.84:0/1717197164 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2798035020 con 0x563748abc850 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f277ffff700 1 -- 172.21.15.84:0/1717197164 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f2798007ad0 con 0x563748abc850 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f27b1817b80 1 -- 172.21.15.84:0/1717197164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2798009250 con 0x563748abc850 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f277ffff700 1 -- 172.21.15.84:0/1717197164 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2798020020 con 0x563748abc850 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f27b1817b80 10 client.16875 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f27b1817b80 1 -- 172.21.15.84:0/1717197164 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563748b37f90 con 0x563748abc850 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f277ffff700 1 -- 172.21.15.84:0/1717197164 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2798009250 con 0x563748abc850 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f277ffff700 1 -- 172.21.15.84:0/1717197164 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f2798011070 con 0x563748abc850 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f27b1817b80 10 client.16875 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f27b1817b80 10 client.16875 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f27b1817b80 20 client.16875 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f27b1817b80 1 --2- 172.21.15.84:0/1717197164 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563748acff70 0x563748b3a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f27b1817b80 4 client.16875 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.4, "await_for": 42.0}] 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f27b1817b80 1 -- 172.21.15.84:0/1717197164 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 86.4, "await_for": 42.0}) -- 0x563748a10440 con 0x563748acff70 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.653+0000 7f27a4d21700 1 --2- 172.21.15.84:0/1717197164 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563748acff70 0x563748b3a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:15.659 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.654+0000 7f277ffff700 10 client.16875 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:15.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.654+0000 7f27a4d21700 1 --2- 172.21.15.84:0/1717197164 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563748acff70 0x563748b3a270 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=642017d6dd7b5673 server_cookie=d9402807210ad801 in_seq=0 out_seq=0 2024-04-26T18:40:15.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f277ffff700 1 -- 172.21.15.84:0/1717197164 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563748a10440 con 0x563748acff70 2024-04-26T18:40:15.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f277ffff700 10 client.16875 handle_command_reply: tid=0 2024-04-26T18:40:15.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f27b1817b80 1 client.16875 shutdown 2024-04-26T18:40:15.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f27b1817b80 1 -- 172.21.15.84:0/1717197164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 msgr2=0x563748ac61b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:15.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f27b1817b80 1 --2- 172.21.15.84:0/1717197164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 0x563748ac61b0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f279800b080 tx=0x7f2798004d00 comp rx=0 tx=0).stop 2024-04-26T18:40:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f27b1817b80 1 -- 172.21.15.84:0/1717197164 shutdown_connections 2024-04-26T18:40:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f27b1817b80 1 --2- 172.21.15.84:0/1717197164 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563748acff70 0x563748b3a270 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f27b1817b80 1 --2- 172.21.15.84:0/1717197164 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563748abc850 0x563748ac61b0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f27b1817b80 1 --2- 172.21.15.84:0/1717197164 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563748ab6900 0x563748ac8730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f27b1817b80 1 --2- 172.21.15.84:0/1717197164 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563748abbf30 0x563748ac5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f27b1817b80 1 -- 172.21.15.84:0/1717197164 >> 172.21.15.84:0/1717197164 conn(0x5637489e4750 msgr2=0x563748ac0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f277ffff700 0 client.16875 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f27b1817b80 1 -- 172.21.15.84:0/1717197164 shutdown_connections 2024-04-26T18:40:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.670+0000 7f27b1817b80 1 -- 172.21.15.84:0/1717197164 wait complete. 2024-04-26T18:40:15.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.671+0000 7f27b1817b80 20 client.16875 trim_cache size 0 max 16384 2024-04-26T18:40:15.672 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 34, 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: "2a546377": { 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 34, 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 86.4, 2024-04-26T18:40:15.673 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:40:15.674 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:40:15.674 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:40:15.674 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:40:15.674 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:40:15.674 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:40:15.674 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:40:15.674 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:40:15.674 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:40:15.674 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:40:15.674 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:40:15.674 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:40:15.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.671+0000 7f27a6f85700 1 -- 172.21.15.84:0/3576166930 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f27900a3e50 msgr2=0x7f27900a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:15.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.671+0000 7f27a6f85700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f27900a3e50 0x7f27900a6310 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2794005fd0 tx=0x7f2794005de0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.671+0000 7f27a6f85700 1 -- 172.21.15.84:0/3576166930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27a0103e90 msgr2=0x7f27a019fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:15.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.671+0000 7f27a6f85700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27a0103e90 0x7f27a019fbe0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f278c00b530 tx=0x7f278c00b8f0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.671+0000 7f27a6f85700 1 -- 172.21.15.84:0/3576166930 shutdown_connections 2024-04-26T18:40:15.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.671+0000 7f27a6f85700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f27900a3e50 0x7f27900a6310 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.671+0000 7f27a6f85700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f27a01094a0 0x7f27a0199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.671+0000 7f27a6f85700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f27a01034d0 0x7f27a019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.671+0000 7f27a6f85700 1 --2- 172.21.15.84:0/3576166930 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f27a0103e90 0x7f27a019fbe0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:15.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.671+0000 7f27a6f85700 1 -- 172.21.15.84:0/3576166930 >> 172.21.15.84:0/3576166930 conn(0x7f27a007a4f0 msgr2=0x7f27a0101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:15.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.671+0000 7f27a6f85700 1 -- 172.21.15.84:0/3576166930 shutdown_connections 2024-04-26T18:40:15.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:15.671+0000 7f27a6f85700 1 -- 172.21.15.84:0/3576166930 wait complete. 2024-04-26T18:40:15.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:15 smithi103 ceph-mon[32727]: pgmap v385: 97 pgs: 97 active+clean; 4.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 17 op/s 2024-04-26T18:40:15.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:15 smithi174 ceph-mon[32705]: pgmap v385: 97 pgs: 97 active+clean; 4.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 47 KiB/s wr, 17 op/s 2024-04-26T18:40:16.066 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.064+0000 7fb400178640 1 -- 172.21.15.84:0/3613658726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8107860 msgr2=0x7fb3f8107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.064+0000 7fb400178640 1 --2- 172.21.15.84:0/3613658726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8107860 0x7fb3f8107c40 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009930 tx=0x7fb3f402a420 comp rx=0 tx=0).stop 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.064+0000 7fb400178640 1 -- 172.21.15.84:0/3613658726 shutdown_connections 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.064+0000 7fb400178640 1 --2- 172.21.15.84:0/3613658726 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3f8102210 0x7fb3f810e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.064+0000 7fb400178640 1 --2- 172.21.15.84:0/3613658726 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3f8101850 0x7fb3f8101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.064+0000 7fb400178640 1 --2- 172.21.15.84:0/3613658726 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8107860 0x7fb3f8107c40 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.064+0000 7fb400178640 1 -- 172.21.15.84:0/3613658726 >> 172.21.15.84:0/3613658726 conn(0x7fb3f8077bb0 msgr2=0x7fb3f8077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.064+0000 7fb400178640 1 -- 172.21.15.84:0/3613658726 shutdown_connections 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.064+0000 7fb400178640 1 -- 172.21.15.84:0/3613658726 wait complete. 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb400178640 1 Processor -- start 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb400178640 1 -- start start 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb400178640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8101850 0x7fb3f819a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb400178640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3f8102210 0x7fb3f819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb400178640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3f8107860 0x7fb3f819ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.067 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb400178640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f8111bf0 con 0x7fb3f8101850 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb400178640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f8111a70 con 0x7fb3f8107860 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb400178640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f8111d70 con 0x7fb3f8102210 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb3fdf14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8101850 0x7fb3f819a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb3fdf14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8101850 0x7fb3f819a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44640/0 (socket says 172.21.15.84:44640) 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb3fdf14640 1 -- 172.21.15.84:0/1445011776 learned_addr learned my addr 172.21.15.84:0/1445011776 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb3fdf14640 1 -- 172.21.15.84:0/1445011776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3f8102210 msgr2=0x7fb3f819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb3fd713640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3f8102210 0x7fb3f819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3f8102210 0x7fb3f819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb3fe715640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3f8107860 0x7fb3f819ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb3fdf14640 1 -- 172.21.15.84:0/1445011776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3f8107860 msgr2=0x7fb3f819ab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3f8107860 0x7fb3f819ab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb3fdf14640 1 -- 172.21.15.84:0/1445011776 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3f81a5470 con 0x7fb3f8101850 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb3fd713640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3f8102210 0x7fb3f819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:16.068 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.065+0000 7fb3fe715640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3f8107860 0x7fb3f819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.066+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8101850 0x7fb3f819a070 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4031440 tx=0x7fb3f4003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.066+0000 7fb3eeffd640 1 -- 172.21.15.84:0/1445011776 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f403f070 con 0x7fb3f8101850 2024-04-26T18:40:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.066+0000 7fb3eeffd640 1 -- 172.21.15.84:0/1445011776 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f4031d60 con 0x7fb3f8101850 2024-04-26T18:40:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.066+0000 7fb3eeffd640 1 -- 172.21.15.84:0/1445011776 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4035030 con 0x7fb3f8101850 2024-04-26T18:40:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.066+0000 7fb400178640 1 -- 172.21.15.84:0/1445011776 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3f81a5700 con 0x7fb3f8101850 2024-04-26T18:40:16.069 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.066+0000 7fb400178640 1 -- 172.21.15.84:0/1445011776 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3f81a5bf0 con 0x7fb3f8101850 2024-04-26T18:40:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.068+0000 7fb3eeffd640 1 -- 172.21.15.84:0/1445011776 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fb3f4042070 con 0x7fb3f8101850 2024-04-26T18:40:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.068+0000 7fb402f6e740 1 Processor -- start 2024-04-26T18:40:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.068+0000 7fb402f6e740 1 -- start start 2024-04-26T18:40:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.068+0000 7fb402f6e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2c22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb402f6e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c2820 0x55f6cd2ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb402f6e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2cf4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb402f6e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd228460 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.076 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb402f6e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd1e2590 con 0x55f6cd2c2820 2024-04-26T18:40:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb402f6e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd224420 con 0x55f6cd2c1f00 2024-04-26T18:40:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3fd713640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2cf4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3fd713640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2cf4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44654/0 (socket says 172.21.15.84:44654) 2024-04-26T18:40:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3fd713640 1 -- 172.21.15.84:0/1227015476 learned_addr learned my addr 172.21.15.84:0/1227015476 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3eeffd640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fb3e40a3dc0 0x7fb3e40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/1227015476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c2820 0x55f6cd2ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3fe715640 1 --2- 172.21.15.84:0/1227015476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2c22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3fe715640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fb3e40a3dc0 0x7fb3e40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3fd713640 1 -- 172.21.15.84:0/1227015476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c1f00 msgr2=0x55f6cd2c22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3fd713640 1 --2- 172.21.15.84:0/1227015476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2c22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3fd713640 1 -- 172.21.15.84:0/1227015476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c2820 msgr2=0x55f6cd2ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3fd713640 1 --2- 172.21.15.84:0/1227015476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c2820 0x55f6cd2ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.077 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3fd713640 1 -- 172.21.15.84:0/1227015476 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6cd1ef0d0 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3eeffd640 1 -- 172.21.15.84:0/1445011776 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fb3f40f1200 con 0x7fb3f8101850 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3fd713640 1 --2- 172.21.15.84:0/1227015476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2cf4d0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800a9d0 tx=0x7fb3e800ae90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8202d24f0a5e85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3fe715640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fb3e40a3dc0 0x7fb3e40a6280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000e5d0 tx=0x7fb3f0008220 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3e3fff640 1 -- 172.21.15.84:0/1227015476 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e8015070 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3e3fff640 1 -- 172.21.15.84:0/1227015476 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e800cc60 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.069+0000 7fb3e3fff640 1 -- 172.21.15.84:0/1227015476 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3e80185c0 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.070+0000 7fb402f6e740 1 -- 172.21.15.84:0/1227015476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 msgr2=0x55f6cd2cf4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.070+0000 7fb402f6e740 1 --2- 172.21.15.84:0/1227015476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2cf4d0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb3e800a9d0 tx=0x7fb3e800ae90 comp rx=0 tx=0).stop 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.070+0000 7fb402f6e740 1 -- 172.21.15.84:0/1227015476 shutdown_connections 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.070+0000 7fb402f6e740 1 --2- 172.21.15.84:0/1227015476 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2cf4d0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.070+0000 7fb402f6e740 1 --2- 172.21.15.84:0/1227015476 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c2820 0x55f6cd2ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.070+0000 7fb402f6e740 1 --2- 172.21.15.84:0/1227015476 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2c22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.070+0000 7fb402f6e740 1 -- 172.21.15.84:0/1227015476 >> 172.21.15.84:0/1227015476 conn(0x55f6cd2be410 msgr2=0x55f6cd2be7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:16.078 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.070+0000 7fb402f6e740 1 -- 172.21.15.84:0/1227015476 shutdown_connections 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.070+0000 7fb402f6e740 1 -- 172.21.15.84:0/1227015476 wait complete. 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb402f6e740 1 -- start start 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb402f6e740 1 Processor -- start 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb402f6e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb402f6e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2ca550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb402f6e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2caa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb402f6e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd2cfd10 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb402f6e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd1afb60 con 0x55f6cd2c1f00 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb402f6e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f6cd2cfe90 con 0x55f6cd2c2820 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb402f6e740 10 client.0 _ll_register_callbacks cb 0x55f6cd1d42b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb3fe715640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb3fe715640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44662/0 (socket says 172.21.15.84:44662) 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb3fe715640 1 -- 172.21.15.84:0/2798698292 learned_addr learned my addr 172.21.15.84:0/2798698292 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:16.079 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb3fd713640 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb3fe715640 1 -- 172.21.15.84:0/2798698292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 msgr2=0x55f6cd2ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb3fe715640 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb3fe715640 1 -- 172.21.15.84:0/2798698292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c1f00 msgr2=0x55f6cd2ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb3fe715640 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2ca010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb3fe715640 1 -- 172.21.15.84:0/2798698292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6ccf3a210 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.071+0000 7fb3fd713640 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb3fe715640 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2caa90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000cc20 tx=0x7fb3f0004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e243a62b859a054d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb3e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb3e17fa640 1 -- 172.21.15.84:0/2798698292 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0020070 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb3e17fa640 1 -- 172.21.15.84:0/2798698292 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fb3f0025660 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.080 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb402f6e740 1 -- 172.21.15.84:0/2798698292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3e800cc60 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb3e17fa640 1 -- 172.21.15.84:0/2798698292 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb3f000ce60 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb402f6e740 10 client.16887 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb402f6e740 1 -- 172.21.15.84:0/2798698292 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6cd006620 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb3e17fa640 1 -- 172.21.15.84:0/2798698292 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3e800cc60 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb3e17fa640 1 -- 172.21.15.84:0/2798698292 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb3f001c8e0 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb402f6e740 10 client.16887 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb402f6e740 10 client.16887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb402f6e740 20 client.16887 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb402f6e740 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f6cd2dd270 0x55f6cd2df6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb402f6e740 4 client.16887 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.072+0000 7fb402f6e740 1 -- 172.21.15.84:0/2798698292 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6cd236070 con 0x55f6cd2dd270 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.073+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f6cd2dd270 0x55f6cd2df6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.073+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f6cd2dd270 0x55f6cd2df6e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd4709d83095c37d server_cookie=f761b5b3823d3046 in_seq=0 out_seq=0 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.073+0000 7fb3e17fa640 10 client.16887 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:16.081 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb3e17fa640 1 -- 172.21.15.84:0/2798698292 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f6cd236070 con 0x55f6cd2dd270 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb3e17fa640 10 client.16887 handle_command_reply: tid=0 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb402f6e740 1 client.16887 shutdown 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb402f6e740 1 -- 172.21.15.84:0/2798698292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 msgr2=0x55f6cd2caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb402f6e740 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2caa90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000cc20 tx=0x7fb3f0004b50 comp rx=0 tx=0).stop 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb402f6e740 1 -- 172.21.15.84:0/2798698292 shutdown_connections 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb402f6e740 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f6cd2dd270 0x55f6cd2df6e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb402f6e740 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2caa90 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb402f6e740 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb3e17fa640 0 client.16887 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb402f6e740 1 --2- 172.21.15.84:0/2798698292 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2ca010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb402f6e740 1 -- 172.21.15.84:0/2798698292 >> 172.21.15.84:0/2798698292 conn(0x55f6cd2be410 msgr2=0x55f6cd2bd5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb402f6e740 1 -- 172.21.15.84:0/2798698292 shutdown_connections 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb402f6e740 1 -- 172.21.15.84:0/2798698292 wait complete. 2024-04-26T18:40:16.082 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.074+0000 7fb402f6e740 20 client.16887 trim_cache size 0 max 16384 2024-04-26T18:40:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb402f6e740 1 Processor -- start 2024-04-26T18:40:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb402f6e740 1 -- start start 2024-04-26T18:40:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb402f6e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2d8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb402f6e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2e4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb402f6e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2e5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.094 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb402f6e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f81ae630 con 0x55f6cd2c1f00 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb402f6e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f81b0170 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb402f6e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f81aed00 con 0x55f6cd2c2820 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3fdf14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2d8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3fdf14640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2d8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44672/0 (socket says 172.21.15.84:44672) 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3fdf14640 1 -- 172.21.15.84:0/305240190 learned_addr learned my addr 172.21.15.84:0/305240190 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3fe715640 1 --2- 172.21.15.84:0/305240190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2e5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3fdf14640 1 -- 172.21.15.84:0/305240190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 msgr2=0x55f6cd2e4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3fd713640 1 --2- 172.21.15.84:0/305240190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2e4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/305240190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2e4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3fdf14640 1 -- 172.21.15.84:0/305240190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2cd0e0 msgr2=0x55f6cd2e5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/305240190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2e5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3fdf14640 1 -- 172.21.15.84:0/305240190 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6cd2dbe10 con 0x55f6cd2c1f00 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3fd713640 1 --2- 172.21.15.84:0/305240190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2e4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3fe715640 1 --2- 172.21.15.84:0/305240190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2e5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:16.095 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/305240190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2d8bd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40b95c0 tx=0x7fb3f408eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32066f0140aa918c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3e37fe640 1 -- 172.21.15.84:0/305240190 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4049040 con 0x55f6cd2c1f00 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3e37fe640 1 -- 172.21.15.84:0/305240190 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f40c1510 con 0x55f6cd2c1f00 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.090+0000 7fb3e37fe640 1 -- 172.21.15.84:0/305240190 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f40b97d0 con 0x55f6cd2c1f00 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.091+0000 7fb402f6e740 1 -- 172.21.15.84:0/305240190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2c1f00 msgr2=0x55f6cd2d8bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.091+0000 7fb402f6e740 1 --2- 172.21.15.84:0/305240190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2d8bd0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40b95c0 tx=0x7fb3f408eba0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.091+0000 7fb402f6e740 1 -- 172.21.15.84:0/305240190 shutdown_connections 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.091+0000 7fb402f6e740 1 --2- 172.21.15.84:0/305240190 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd2e5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.091+0000 7fb402f6e740 1 --2- 172.21.15.84:0/305240190 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2e4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.091+0000 7fb402f6e740 1 --2- 172.21.15.84:0/305240190 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2d8bd0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.091+0000 7fb402f6e740 1 -- 172.21.15.84:0/305240190 >> 172.21.15.84:0/305240190 conn(0x55f6cd2be410 msgr2=0x55f6cd2be7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.091+0000 7fb402f6e740 1 -- 172.21.15.84:0/305240190 shutdown_connections 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.091+0000 7fb402f6e740 1 -- 172.21.15.84:0/305240190 wait complete. 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.091+0000 7fb402f6e740 1 -- start start 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.091+0000 7fb402f6e740 1 Processor -- start 2024-04-26T18:40:16.096 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb402f6e740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2d1690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb402f6e740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2d1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb402f6e740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd33b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb402f6e740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f81afaa0 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb402f6e740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f81af3d0 con 0x55f6cd2c1f00 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb402f6e740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb3f81ac630 con 0x55f6cd2c2820 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb402f6e740 10 client.0 _ll_register_callbacks cb 0x7fb3f0009b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb3fe715640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd33b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb3fe715640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd33b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:44682/0 (socket says 172.21.15.84:44682) 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb3fe715640 1 -- 172.21.15.84:0/2912455635 learned_addr learned my addr 172.21.15.84:0/2912455635 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb3fd713640 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2d1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2d1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb3fe715640 1 -- 172.21.15.84:0/2912455635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 msgr2=0x55f6cd2d1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb3fe715640 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2d1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.097 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb3fe715640 1 -- 172.21.15.84:0/2912455635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c1f00 msgr2=0x55f6cd2d1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb3fe715640 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2d1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb3fe715640 1 -- 172.21.15.84:0/2912455635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6cd33c050 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2d1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb3fe715640 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd33b820 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb3f001cef0 tx=0x7fb3f000e300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53107a88f178e267 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.092+0000 7fb3e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb3e17fa640 1 -- 172.21.15.84:0/2912455635 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f0020040 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb3e17fa640 1 -- 172.21.15.84:0/2912455635 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fb3f001c810 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb402f6e740 1 -- 172.21.15.84:0/2912455635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3f40c1510 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb3e17fa640 1 -- 172.21.15.84:0/2912455635 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb3f001ca10 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb402f6e740 10 client.16893 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb402f6e740 1 -- 172.21.15.84:0/2912455635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6cd344340 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb3e17fa640 1 -- 172.21.15.84:0/2912455635 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3f40c1510 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.098 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb3e17fa640 1 -- 172.21.15.84:0/2912455635 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fb3f003b030 con 0x55f6cd2cd0e0 2024-04-26T18:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb402f6e740 10 client.16893 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb402f6e740 10 client.16893 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb402f6e740 20 client.16893 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb402f6e740 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f6cd344ae0 0x55f6cd346f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb402f6e740 4 client.16893 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb402f6e740 1 -- 172.21.15.84:0/2912455635 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f6cd22fee0 con 0x55f6cd344ae0 2024-04-26T18:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.093+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f6cd344ae0 0x55f6cd346f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.094+0000 7fb3e17fa640 10 client.16893 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:16.099 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.094+0000 7fb3fdf14640 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f6cd344ae0 0x55f6cd346f00 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a9fe31fa37174a0 server_cookie=9a1699d89b44ed54 in_seq=0 out_seq=0 2024-04-26T18:40:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.109+0000 7fb3e17fa640 1 -- 172.21.15.84:0/2912455635 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f6cd22fee0 con 0x55f6cd344ae0 2024-04-26T18:40:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.109+0000 7fb3e17fa640 10 client.16893 handle_command_reply: tid=0 2024-04-26T18:40:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.109+0000 7fb402f6e740 1 client.16893 shutdown 2024-04-26T18:40:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.109+0000 7fb402f6e740 1 -- 172.21.15.84:0/2912455635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 msgr2=0x55f6cd33b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.109+0000 7fb402f6e740 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd33b820 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fb3f001cef0 tx=0x7fb3f000e300 comp rx=0 tx=0).stop 2024-04-26T18:40:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb402f6e740 1 -- 172.21.15.84:0/2912455635 shutdown_connections 2024-04-26T18:40:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb402f6e740 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f6cd344ae0 0x55f6cd346f00 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb402f6e740 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f6cd2cd0e0 0x55f6cd33b820 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.110 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb402f6e740 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f6cd2c2820 0x55f6cd2d1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb3e17fa640 0 client.16893 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb402f6e740 1 --2- 172.21.15.84:0/2912455635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f6cd2c1f00 0x55f6cd2d1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb402f6e740 1 -- 172.21.15.84:0/2912455635 >> 172.21.15.84:0/2912455635 conn(0x55f6cd2be410 msgr2=0x55f6cd2c9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb402f6e740 1 -- 172.21.15.84:0/2912455635 shutdown_connections 2024-04-26T18:40:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb402f6e740 1 -- 172.21.15.84:0/2912455635 wait complete. 2024-04-26T18:40:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb402f6e740 20 client.16893 trim_cache size 0 max 16384 2024-04-26T18:40:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:40:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:40:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:40:16.111 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:40:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb400178640 1 -- 172.21.15.84:0/1445011776 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fb3e40a3dc0 msgr2=0x7fb3e40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb400178640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fb3e40a3dc0 0x7fb3e40a6280 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3f000e5d0 tx=0x7fb3f0008220 comp rx=0 tx=0).stop 2024-04-26T18:40:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb400178640 1 -- 172.21.15.84:0/1445011776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8101850 msgr2=0x7fb3f819a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:16.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb400178640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8101850 0x7fb3f819a070 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4031440 tx=0x7fb3f4003d50 comp rx=0 tx=0).stop 2024-04-26T18:40:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb400178640 1 -- 172.21.15.84:0/1445011776 shutdown_connections 2024-04-26T18:40:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb400178640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fb3e40a3dc0 0x7fb3e40a6280 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb400178640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb3f8107860 0x7fb3f819ab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb400178640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb3f8102210 0x7fb3f819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb400178640 1 --2- 172.21.15.84:0/1445011776 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb3f8101850 0x7fb3f819a070 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb400178640 1 -- 172.21.15.84:0/1445011776 >> 172.21.15.84:0/1445011776 conn(0x7fb3f8077bb0 msgr2=0x7fb3f80ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb400178640 1 -- 172.21.15.84:0/1445011776 shutdown_connections 2024-04-26T18:40:16.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:16.110+0000 7fb400178640 1 -- 172.21.15.84:0/1445011776 wait complete. 2024-04-26T18:40:16.423 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '2a546377', quiesce took 6.0 seconds. Will release after: 26.4 2024-04-26T18:40:17.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:17 smithi084 ceph-mon[29141]: pgmap v386: 97 pgs: 97 active+clean; 4.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:40:17.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:17 smithi103 ceph-mon[32727]: pgmap v386: 97 pgs: 97 active+clean; 4.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:40:17.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:17 smithi174 ceph-mon[32705]: pgmap v386: 97 pgs: 97 active+clean; 4.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:40:19.741 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:19 smithi174 ceph-mon[32705]: pgmap v387: 97 pgs: 97 active+clean; 4.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:40:19.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:19 smithi084 ceph-mon[29141]: pgmap v387: 97 pgs: 97 active+clean; 4.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:40:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:19 smithi103 ceph-mon[32727]: pgmap v387: 97 pgs: 97 active+clean; 4.3 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:40:21.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:21 smithi084 ceph-mon[29141]: pgmap v388: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-26T18:40:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:21 smithi103 ceph-mon[32727]: pgmap v388: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-26T18:40:21.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:21 smithi174 ceph-mon[32705]: pgmap v388: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-26T18:40:23.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:23 smithi084 ceph-mon[29141]: pgmap v389: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-26T18:40:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:23 smithi103 ceph-mon[32727]: pgmap v389: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-26T18:40:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:23 smithi174 ceph-mon[32705]: pgmap v389: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 17 op/s 2024-04-26T18:40:25.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:25 smithi084 ceph-mon[29141]: pgmap v390: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 20 op/s 2024-04-26T18:40:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:25 smithi103 ceph-mon[32727]: pgmap v390: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 20 op/s 2024-04-26T18:40:25.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:25 smithi174 ceph-mon[32705]: pgmap v390: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 20 op/s 2024-04-26T18:40:27.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:27 smithi084 ceph-mon[29141]: pgmap v391: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-26T18:40:27.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:27 smithi103 ceph-mon[32727]: pgmap v391: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-26T18:40:27.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:27 smithi174 ceph-mon[32705]: pgmap v391: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-26T18:40:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.206+0000 7fbd32f19640 1 -- 172.21.15.84:0/2385396874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd2c1032f0 msgr2=0x7fbd2c10f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:29.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.206+0000 7fbd32f19640 1 --2- 172.21.15.84:0/2385396874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd2c1032f0 0x7fbd2c10f800 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009cd0 tx=0x7fbd2402a8e0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.206+0000 7fbd32f19640 1 -- 172.21.15.84:0/2385396874 shutdown_connections 2024-04-26T18:40:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.206+0000 7fbd32f19640 1 --2- 172.21.15.84:0/2385396874 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd2c1032f0 0x7fbd2c10f800 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.206+0000 7fbd32f19640 1 --2- 172.21.15.84:0/2385396874 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd2c102930 0x7fbd2c102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.206+0000 7fbd32f19640 1 --2- 172.21.15.84:0/2385396874 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd2c108940 0x7fbd2c108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.206+0000 7fbd32f19640 1 -- 172.21.15.84:0/2385396874 >> 172.21.15.84:0/2385396874 conn(0x7fbd2c0fe400 msgr2=0x7fbd2c100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.206+0000 7fbd32f19640 1 -- 172.21.15.84:0/2385396874 shutdown_connections 2024-04-26T18:40:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.206+0000 7fbd32f19640 1 -- 172.21.15.84:0/2385396874 wait complete. 2024-04-26T18:40:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.206+0000 7fbd32f19640 1 Processor -- start 2024-04-26T18:40:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.206+0000 7fbd32f19640 1 -- start start 2024-04-26T18:40:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.206+0000 7fbd32f19640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd2c102930 0x7fbd2c199020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd32f19640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd2c1032f0 0x7fbd2c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.209 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd32f19640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd2c108940 0x7fbd2c19d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd32f19640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c112c00 con 0x7fbd2c102930 2024-04-26T18:40:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd32f19640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c112a80 con 0x7fbd2c1032f0 2024-04-26T18:40:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd32f19640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c112d80 con 0x7fbd2c108940 2024-04-26T18:40:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd30cb5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd2c102930 0x7fbd2c199020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd30cb5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd2c102930 0x7fbd2c199020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54204/0 (socket says 172.21.15.84:54204) 2024-04-26T18:40:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd30cb5640 1 -- 172.21.15.84:0/65679581 learned_addr learned my addr 172.21.15.84:0/65679581 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd2bfff640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd2c1032f0 0x7fbd2c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd314b6640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd2c108940 0x7fbd2c19d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd30cb5640 1 -- 172.21.15.84:0/65679581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd2c108940 msgr2=0x7fbd2c19d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd2c108940 0x7fbd2c19d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd30cb5640 1 -- 172.21.15.84:0/65679581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd2c1032f0 msgr2=0x7fbd2c199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd2c1032f0 0x7fbd2c199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.210 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd30cb5640 1 -- 172.21.15.84:0/65679581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd2c19e010 con 0x7fbd2c102930 2024-04-26T18:40:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd314b6640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd2c108940 0x7fbd2c19d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd2c102930 0x7fbd2c199020 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000c920 tx=0x7fbd2000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd29ffb640 1 -- 172.21.15.84:0/65679581 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20007c70 con 0x7fbd2c102930 2024-04-26T18:40:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd29ffb640 1 -- 172.21.15.84:0/65679581 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd20007e10 con 0x7fbd2c102930 2024-04-26T18:40:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd32f19640 1 -- 172.21.15.84:0/65679581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd2c19e300 con 0x7fbd2c102930 2024-04-26T18:40:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.207+0000 7fbd29ffb640 1 -- 172.21.15.84:0/65679581 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20018820 con 0x7fbd2c102930 2024-04-26T18:40:29.211 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.208+0000 7fbd32f19640 1 -- 172.21.15.84:0/65679581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd2c1a5b90 con 0x7fbd2c102930 2024-04-26T18:40:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.210+0000 7fbd29ffb640 1 -- 172.21.15.84:0/65679581 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fbd2000f450 con 0x7fbd2c102930 2024-04-26T18:40:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.210+0000 7fbd35c38740 1 Processor -- start 2024-04-26T18:40:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.210+0000 7fbd35c38740 1 -- start start 2024-04-26T18:40:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.210+0000 7fbd29ffb640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbd1c0a3dc0 0x7fbd1c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.210+0000 7fbd35c38740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154df00 0x556de154e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.210+0000 7fbd35c38740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154e820 0x556de1558ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.218 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.210+0000 7fbd35c38740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de15590e0 0x556de155b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.210+0000 7fbd35c38740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556de14b4460 con 0x556de154e820 2024-04-26T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.210+0000 7fbd35c38740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556de146e590 con 0x556de154df00 2024-04-26T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.210+0000 7fbd35c38740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556de14b0420 con 0x556de15590e0 2024-04-26T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.210+0000 7fbd29ffb640 1 -- 172.21.15.84:0/65679581 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fbd200d0970 con 0x7fbd2c102930 2024-04-26T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd314b6640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbd1c0a3dc0 0x7fbd1c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd2bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154e820 0x556de1558ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd2bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154e820 0x556de1558ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54210/0 (socket says 172.21.15.84:54210) 2024-04-26T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd2bfff640 1 -- 172.21.15.84:0/3844609786 learned_addr learned my addr 172.21.15.84:0/3844609786 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd314b6640 1 --2- 172.21.15.84:0/3844609786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de15590e0 0x556de155b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/3844609786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154df00 0x556de154e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd2bfff640 1 -- 172.21.15.84:0/3844609786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de15590e0 msgr2=0x556de155b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd2bfff640 1 --2- 172.21.15.84:0/3844609786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de15590e0 0x556de155b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.219 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd2bfff640 1 -- 172.21.15.84:0/3844609786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154df00 msgr2=0x556de154e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd2bfff640 1 --2- 172.21.15.84:0/3844609786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154df00 0x556de154e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd2bfff640 1 -- 172.21.15.84:0/3844609786 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556de147b0d0 con 0x556de154e820 2024-04-26T18:40:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/3844609786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154df00 0x556de154e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:40:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd314b6640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbd1c0a3dc0 0x7fbd1c0a6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbd24006010 tx=0x7fbd24031720 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd2bfff640 1 --2- 172.21.15.84:0/3844609786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154e820 0x556de1558ba0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009c30 tx=0x7fbd180117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c00665a9c7e274f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd16ffd640 1 -- 172.21.15.84:0/3844609786 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd18012450 con 0x556de154e820 2024-04-26T18:40:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd16ffd640 1 -- 172.21.15.84:0/3844609786 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd18012aa0 con 0x556de154e820 2024-04-26T18:40:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.211+0000 7fbd16ffd640 1 -- 172.21.15.84:0/3844609786 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd1800f9e0 con 0x556de154e820 2024-04-26T18:40:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.212+0000 7fbd35c38740 1 -- 172.21.15.84:0/3844609786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154e820 msgr2=0x556de1558ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.212+0000 7fbd35c38740 1 --2- 172.21.15.84:0/3844609786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154e820 0x556de1558ba0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009c30 tx=0x7fbd180117b0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.212+0000 7fbd35c38740 1 -- 172.21.15.84:0/3844609786 shutdown_connections 2024-04-26T18:40:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.212+0000 7fbd35c38740 1 --2- 172.21.15.84:0/3844609786 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de15590e0 0x556de155b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.220 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.212+0000 7fbd35c38740 1 --2- 172.21.15.84:0/3844609786 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154e820 0x556de1558ba0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.212+0000 7fbd35c38740 1 --2- 172.21.15.84:0/3844609786 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154df00 0x556de154e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.212+0000 7fbd35c38740 1 -- 172.21.15.84:0/3844609786 >> 172.21.15.84:0/3844609786 conn(0x556de154a410 msgr2=0x556de154a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.212+0000 7fbd35c38740 1 -- 172.21.15.84:0/3844609786 shutdown_connections 2024-04-26T18:40:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.212+0000 7fbd35c38740 1 -- 172.21.15.84:0/3844609786 wait complete. 2024-04-26T18:40:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.212+0000 7fbd35c38740 1 -- start start 2024-04-26T18:40:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.212+0000 7fbd35c38740 1 Processor -- start 2024-04-26T18:40:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd35c38740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 0x556de1556060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd35c38740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154e820 0x556de15565a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd35c38740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de15590e0 0x556de1556ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd35c38740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556de155bd10 con 0x556de154df00 2024-04-26T18:40:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd35c38740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556de143bb60 con 0x556de154e820 2024-04-26T18:40:29.221 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd35c38740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556de155be90 con 0x556de15590e0 2024-04-26T18:40:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd35c38740 10 client.0 _ll_register_callbacks cb 0x556de14602b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd30cb5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 0x556de1556060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd30cb5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 0x556de1556060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54216/0 (socket says 172.21.15.84:54216) 2024-04-26T18:40:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd30cb5640 1 -- 172.21.15.84:0/4289478082 learned_addr learned my addr 172.21.15.84:0/4289478082 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd314b6640 1 --2- 172.21.15.84:0/4289478082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de15590e0 0x556de1556ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd2bfff640 1 --2- 172.21.15.84:0/4289478082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154e820 0x556de15565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd30cb5640 1 -- 172.21.15.84:0/4289478082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de15590e0 msgr2=0x556de1556ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/4289478082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de15590e0 0x556de1556ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd30cb5640 1 -- 172.21.15.84:0/4289478082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154e820 msgr2=0x556de15565a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/4289478082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154e820 0x556de15565a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.213+0000 7fbd30cb5640 1 -- 172.21.15.84:0/4289478082 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556de11c6210 con 0x556de154df00 2024-04-26T18:40:29.222 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/4289478082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 0x556de1556060 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fbd2009c860 tx=0x7fbd2009d090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4184648aafb93956 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd0bfff640 1 -- 172.21.15.84:0/4289478082 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd200a5540 con 0x556de154df00 2024-04-26T18:40:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd0bfff640 1 -- 172.21.15.84:0/4289478082 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fbd2000fa90 con 0x556de154df00 2024-04-26T18:40:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd35c38740 1 -- 172.21.15.84:0/4289478082 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd18012aa0 con 0x556de154df00 2024-04-26T18:40:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd0bfff640 1 -- 172.21.15.84:0/4289478082 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd2000f740 con 0x556de154df00 2024-04-26T18:40:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd35c38740 10 client.16905 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd35c38740 1 -- 172.21.15.84:0/4289478082 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556de1292620 con 0x556de154df00 2024-04-26T18:40:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd0bfff640 1 -- 172.21.15.84:0/4289478082 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd18012aa0 con 0x556de154df00 2024-04-26T18:40:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd0bfff640 1 -- 172.21.15.84:0/4289478082 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbd200965d0 con 0x556de154df00 2024-04-26T18:40:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd35c38740 10 client.16905 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd35c38740 10 client.16905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:40:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd35c38740 20 client.16905 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:29.223 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd35c38740 1 --2- 172.21.15.84:0/4289478082 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556de1569210 0x556de156b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd35c38740 4 client.16905 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:40:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.214+0000 7fbd35c38740 1 -- 172.21.15.84:0/4289478082 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556de14c2070 con 0x556de1569210 2024-04-26T18:40:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.215+0000 7fbd2bfff640 1 --2- 172.21.15.84:0/4289478082 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556de1569210 0x556de156b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.215+0000 7fbd2bfff640 1 --2- 172.21.15.84:0/4289478082 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556de1569210 0x556de156b600 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6606c5dcf56d98f server_cookie=ed9621ab00a40245 in_seq=0 out_seq=0 2024-04-26T18:40:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.215+0000 7fbd0bfff640 10 client.16905 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd0bfff640 1 -- 172.21.15.84:0/4289478082 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556de14c2070 con 0x556de1569210 2024-04-26T18:40:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd0bfff640 10 client.16905 handle_command_reply: tid=0 2024-04-26T18:40:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd35c38740 1 client.16905 shutdown 2024-04-26T18:40:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd35c38740 1 -- 172.21.15.84:0/4289478082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 msgr2=0x556de1556060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd35c38740 1 --2- 172.21.15.84:0/4289478082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 0x556de1556060 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fbd2009c860 tx=0x7fbd2009d090 comp rx=0 tx=0).stop 2024-04-26T18:40:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd35c38740 1 -- 172.21.15.84:0/4289478082 shutdown_connections 2024-04-26T18:40:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd35c38740 1 --2- 172.21.15.84:0/4289478082 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556de1569210 0x556de156b600 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.224 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd35c38740 1 --2- 172.21.15.84:0/4289478082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de15590e0 0x556de1556ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd35c38740 1 --2- 172.21.15.84:0/4289478082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154e820 0x556de15565a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd35c38740 1 --2- 172.21.15.84:0/4289478082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 0x556de1556060 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd0bfff640 0 client.16905 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd35c38740 1 -- 172.21.15.84:0/4289478082 >> 172.21.15.84:0/4289478082 conn(0x556de154a410 msgr2=0x556de1550190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd35c38740 1 -- 172.21.15.84:0/4289478082 shutdown_connections 2024-04-26T18:40:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd35c38740 1 -- 172.21.15.84:0/4289478082 wait complete. 2024-04-26T18:40:29.225 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.216+0000 7fbd35c38740 20 client.16905 trim_cache size 0 max 16384 2024-04-26T18:40:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd35c38740 1 Processor -- start 2024-04-26T18:40:29.235 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd35c38740 1 -- start start 2024-04-26T18:40:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd35c38740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 0x556de1564bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd35c38740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de154e820 0x556de154ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd35c38740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de15590e0 0x556de1570f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd35c38740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1ae570 con 0x556de154df00 2024-04-26T18:40:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd35c38740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1b00b0 con 0x556de15590e0 2024-04-26T18:40:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd35c38740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1aec40 con 0x556de154e820 2024-04-26T18:40:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd30cb5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 0x556de1564bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd30cb5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 0x556de1564bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54224/0 (socket says 172.21.15.84:54224) 2024-04-26T18:40:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd30cb5640 1 -- 172.21.15.84:0/1855561861 learned_addr learned my addr 172.21.15.84:0/1855561861 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd30cb5640 1 -- 172.21.15.84:0/1855561861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de154e820 msgr2=0x556de154ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/1855561861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de154e820 0x556de154ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd30cb5640 1 -- 172.21.15.84:0/1855561861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de15590e0 msgr2=0x556de1570f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:29.236 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/1855561861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de15590e0 0x556de1570f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd30cb5640 1 -- 172.21.15.84:0/1855561861 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556de15545d0 con 0x556de154df00 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.231+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/1855561861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 0x556de1564bd0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fbd20007590 tx=0x7fbd200a0da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4b1aaee25d5f955 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.232+0000 7fbd167fc640 1 -- 172.21.15.84:0/1855561861 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2009c230 con 0x556de154df00 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.232+0000 7fbd167fc640 1 -- 172.21.15.84:0/1855561861 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd200a5540 con 0x556de154df00 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.232+0000 7fbd167fc640 1 -- 172.21.15.84:0/1855561861 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd20098b70 con 0x556de154df00 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.232+0000 7fbd35c38740 1 -- 172.21.15.84:0/1855561861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 msgr2=0x556de1564bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.232+0000 7fbd35c38740 1 --2- 172.21.15.84:0/1855561861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 0x556de1564bd0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fbd20007590 tx=0x7fbd200a0da0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.232+0000 7fbd35c38740 1 -- 172.21.15.84:0/1855561861 shutdown_connections 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.232+0000 7fbd35c38740 1 --2- 172.21.15.84:0/1855561861 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de15590e0 0x556de1570f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.232+0000 7fbd35c38740 1 --2- 172.21.15.84:0/1855561861 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de154e820 0x556de154ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.232+0000 7fbd35c38740 1 --2- 172.21.15.84:0/1855561861 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de154df00 0x556de1564bd0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.232+0000 7fbd35c38740 1 -- 172.21.15.84:0/1855561861 >> 172.21.15.84:0/1855561861 conn(0x556de154a410 msgr2=0x556de154f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.232+0000 7fbd35c38740 1 -- 172.21.15.84:0/1855561861 shutdown_connections 2024-04-26T18:40:29.237 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.232+0000 7fbd35c38740 1 -- 172.21.15.84:0/1855561861 wait complete. 2024-04-26T18:40:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd35c38740 1 -- start start 2024-04-26T18:40:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd35c38740 1 Processor -- start 2024-04-26T18:40:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd35c38740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154df00 0x556de15612d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd35c38740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de154e820 0x556de155e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd35c38740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de15590e0 0x556de155e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd35c38740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1af9e0 con 0x556de15590e0 2024-04-26T18:40:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd35c38740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1af310 con 0x556de154df00 2024-04-26T18:40:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd35c38740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd2c1ac570 con 0x556de154e820 2024-04-26T18:40:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd35c38740 10 client.0 _ll_register_callbacks cb 0x7fbd2000a580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd314b6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de15590e0 0x556de155e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd314b6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de15590e0 0x556de155e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54234/0 (socket says 172.21.15.84:54234) 2024-04-26T18:40:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd314b6640 1 -- 172.21.15.84:0/3149969968 learned_addr learned my addr 172.21.15.84:0/3149969968 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:29.238 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/3149969968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154df00 0x556de15612d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd2bfff640 1 --2- 172.21.15.84:0/3149969968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de154e820 0x556de155e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd314b6640 1 -- 172.21.15.84:0/3149969968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de154e820 msgr2=0x556de155e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd314b6640 1 --2- 172.21.15.84:0/3149969968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de154e820 0x556de155e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd314b6640 1 -- 172.21.15.84:0/3149969968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154df00 msgr2=0x556de15612d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.233+0000 7fbd314b6640 1 --2- 172.21.15.84:0/3149969968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154df00 0x556de15612d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd314b6640 1 -- 172.21.15.84:0/3149969968 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556de155eb30 con 0x556de15590e0 2024-04-26T18:40:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd314b6640 1 --2- 172.21.15.84:0/3149969968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de15590e0 0x556de155e5f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009cd0 tx=0x7fbd240095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55d7b9c7b46e6a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd0bfff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd0bfff640 1 -- 172.21.15.84:0/3149969968 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd24050030 con 0x556de15590e0 2024-04-26T18:40:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd0bfff640 1 -- 172.21.15.84:0/3149969968 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fbd24003e00 con 0x556de15590e0 2024-04-26T18:40:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd35c38740 1 -- 172.21.15.84:0/3149969968 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbd200a5540 con 0x556de15590e0 2024-04-26T18:40:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd0bfff640 1 -- 172.21.15.84:0/3149969968 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbd24004710 con 0x556de15590e0 2024-04-26T18:40:29.239 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd35c38740 10 client.16911 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd35c38740 1 -- 172.21.15.84:0/3149969968 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556de15d0510 con 0x556de15590e0 2024-04-26T18:40:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd0bfff640 1 -- 172.21.15.84:0/3149969968 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd200a5540 con 0x556de15590e0 2024-04-26T18:40:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd0bfff640 1 -- 172.21.15.84:0/3149969968 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fbd24049050 con 0x556de15590e0 2024-04-26T18:40:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd35c38740 10 client.16911 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd35c38740 10 client.16911 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:40:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd35c38740 20 client.16911 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd35c38740 1 --2- 172.21.15.84:0/3149969968 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556de15d0c90 0x556de15d30b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd35c38740 4 client.16911 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:40:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.234+0000 7fbd35c38740 1 -- 172.21.15.84:0/3149969968 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556de14bbee0 con 0x556de15d0c90 2024-04-26T18:40:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.235+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/3149969968 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556de15d0c90 0x556de15d30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.235+0000 7fbd0bfff640 10 client.16911 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:29.240 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.235+0000 7fbd30cb5640 1 --2- 172.21.15.84:0/3149969968 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556de15d0c90 0x556de15d30b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=722177c51df82b42 server_cookie=13fc2e8052b188cc in_seq=0 out_seq=0 2024-04-26T18:40:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.247+0000 7fbd0bfff640 1 -- 172.21.15.84:0/3149969968 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556de14bbee0 con 0x556de15d0c90 2024-04-26T18:40:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.247+0000 7fbd0bfff640 10 client.16911 handle_command_reply: tid=0 2024-04-26T18:40:29.249 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.247+0000 7fbd35c38740 1 client.16911 shutdown 2024-04-26T18:40:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.247+0000 7fbd35c38740 1 -- 172.21.15.84:0/3149969968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de15590e0 msgr2=0x556de155e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.247+0000 7fbd35c38740 1 --2- 172.21.15.84:0/3149969968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de15590e0 0x556de155e5f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fbd24009cd0 tx=0x7fbd240095c0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd35c38740 1 -- 172.21.15.84:0/3149969968 shutdown_connections 2024-04-26T18:40:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd35c38740 1 --2- 172.21.15.84:0/3149969968 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556de15d0c90 0x556de15d30b0 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd35c38740 1 --2- 172.21.15.84:0/3149969968 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556de15590e0 0x556de155e5f0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd35c38740 1 --2- 172.21.15.84:0/3149969968 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556de154e820 0x556de155e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd0bfff640 0 client.16911 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd35c38740 1 --2- 172.21.15.84:0/3149969968 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556de154df00 0x556de15612d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd35c38740 1 -- 172.21.15.84:0/3149969968 >> 172.21.15.84:0/3149969968 conn(0x556de154a410 msgr2=0x556de1556000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd35c38740 1 -- 172.21.15.84:0/3149969968 shutdown_connections 2024-04-26T18:40:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd35c38740 1 -- 172.21.15.84:0/3149969968 wait complete. 2024-04-26T18:40:29.250 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd35c38740 20 client.16911 trim_cache size 0 max 16384 2024-04-26T18:40:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:40:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:40:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:40:29.251 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:40:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd32f19640 1 -- 172.21.15.84:0/65679581 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbd1c0a3dc0 msgr2=0x7fbd1c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd32f19640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbd1c0a3dc0 0x7fbd1c0a6280 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fbd24006010 tx=0x7fbd24031720 comp rx=0 tx=0).stop 2024-04-26T18:40:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd32f19640 1 -- 172.21.15.84:0/65679581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd2c102930 msgr2=0x7fbd2c199020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.248+0000 7fbd32f19640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd2c102930 0x7fbd2c199020 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000c920 tx=0x7fbd2000cde0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.249+0000 7fbd32f19640 1 -- 172.21.15.84:0/65679581 shutdown_connections 2024-04-26T18:40:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.249+0000 7fbd32f19640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbd1c0a3dc0 0x7fbd1c0a6280 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.249+0000 7fbd32f19640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd2c108940 0x7fbd2c19d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.249+0000 7fbd32f19640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd2c1032f0 0x7fbd2c199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.249+0000 7fbd32f19640 1 --2- 172.21.15.84:0/65679581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd2c102930 0x7fbd2c199020 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.249+0000 7fbd32f19640 1 -- 172.21.15.84:0/65679581 >> 172.21.15.84:0/65679581 conn(0x7fbd2c0fe400 msgr2=0x7fbd2c0fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.249+0000 7fbd32f19640 1 -- 172.21.15.84:0/65679581 shutdown_connections 2024-04-26T18:40:29.252 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:29.249+0000 7fbd32f19640 1 -- 172.21.15.84:0/65679581 wait complete. 2024-04-26T18:40:29.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:29 smithi084 ceph-mon[29141]: pgmap v392: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-26T18:40:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:29 smithi103 ceph-mon[32727]: pgmap v392: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-26T18:40:29.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:29 smithi174 ceph-mon[32705]: pgmap v392: 97 pgs: 97 active+clean; 4.4 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-26T18:40:31.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:31 smithi084 ceph-mon[29141]: pgmap v393: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-26T18:40:31.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:31 smithi103 ceph-mon[32727]: pgmap v393: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-26T18:40:31.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:31 smithi174 ceph-mon[32705]: pgmap v393: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 16 op/s 2024-04-26T18:40:33.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:33 smithi084 ceph-mon[29141]: pgmap v394: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-26T18:40:33.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:33 smithi103 ceph-mon[32727]: pgmap v394: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-26T18:40:33.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:33 smithi174 ceph-mon[32705]: pgmap v394: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-26T18:40:35.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:35 smithi084 ceph-mon[29141]: pgmap v395: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-04-26T18:40:35.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:35 smithi103 ceph-mon[32727]: pgmap v395: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-04-26T18:40:35.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:35 smithi174 ceph-mon[32705]: pgmap v395: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.4 KiB/s wr, 15 op/s 2024-04-26T18:40:37.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:37 smithi084 ceph-mon[29141]: pgmap v396: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-26T18:40:37.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:37 smithi103 ceph-mon[32727]: pgmap v396: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-26T18:40:37.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:37 smithi174 ceph-mon[32705]: pgmap v396: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-26T18:40:39.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:39 smithi084 ceph-mon[29141]: pgmap v397: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-26T18:40:39.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:39 smithi103 ceph-mon[32727]: pgmap v397: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-26T18:40:39.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:39 smithi174 ceph-mon[32705]: pgmap v397: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.3 KiB/s wr, 10 op/s 2024-04-26T18:40:41.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:41 smithi084 ceph-mon[29141]: pgmap v398: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-04-26T18:40:41.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:41 smithi103 ceph-mon[32727]: pgmap v398: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-04-26T18:40:41.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:41 smithi174 ceph-mon[32705]: pgmap v398: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.3 KiB/s wr, 14 op/s 2024-04-26T18:40:42.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.343+0000 7f766f0c8640 1 -- 172.21.15.84:0/728389785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7668103300 msgr2=0x7f766810f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.343+0000 7f766f0c8640 1 --2- 172.21.15.84:0/728389785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7668103300 0x7f766810f810 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f7660009cd0 tx=0x7f766002a8e0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.343+0000 7f766f0c8640 1 -- 172.21.15.84:0/728389785 shutdown_connections 2024-04-26T18:40:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.343+0000 7f766f0c8640 1 --2- 172.21.15.84:0/728389785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7668103300 0x7f766810f810 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.343+0000 7f766f0c8640 1 --2- 172.21.15.84:0/728389785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7668102940 0x7f7668102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.343+0000 7f766f0c8640 1 --2- 172.21.15.84:0/728389785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7668108950 0x7f7668108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.343+0000 7f766f0c8640 1 -- 172.21.15.84:0/728389785 >> 172.21.15.84:0/728389785 conn(0x7f7668079dc0 msgr2=0x7f766807a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.343+0000 7f766f0c8640 1 -- 172.21.15.84:0/728389785 shutdown_connections 2024-04-26T18:40:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.343+0000 7f766f0c8640 1 -- 172.21.15.84:0/728389785 wait complete. 2024-04-26T18:40:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f766f0c8640 1 Processor -- start 2024-04-26T18:40:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f766f0c8640 1 -- start start 2024-04-26T18:40:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f766f0c8640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7668102940 0x7f7668199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f766f0c8640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7668103300 0x7f76681995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.347 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f766f0c8640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7668108950 0x7f766819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f766f0c8640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7668112c10 con 0x7f7668103300 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f766f0c8640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7668112a90 con 0x7f7668108950 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f766f0c8640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7668112d90 con 0x7f7668102940 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f765ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7668103300 0x7f76681995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f765ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7668103300 0x7f76681995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35736/0 (socket says 172.21.15.84:35736) 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f765ffff640 1 -- 172.21.15.84:0/2911186370 learned_addr learned my addr 172.21.15.84:0/2911186370 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f766ce64640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7668102940 0x7f7668199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f766d665640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7668108950 0x7f766819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f765ffff640 1 -- 172.21.15.84:0/2911186370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7668102940 msgr2=0x7f7668199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f765ffff640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7668102940 0x7f7668199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f765ffff640 1 -- 172.21.15.84:0/2911186370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7668108950 msgr2=0x7f766819d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f765ffff640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7668108950 0x7f766819d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f765ffff640 1 -- 172.21.15.84:0/2911186370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f766819e030 con 0x7f7668103300 2024-04-26T18:40:42.348 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.344+0000 7f766d665640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7668108950 0x7f766819d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.345+0000 7f765ffff640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7668103300 0x7f76681995d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f765800d770 tx=0x7f765800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.345+0000 7f765dffb640 1 -- 172.21.15.84:0/2911186370 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658017070 con 0x7f7668103300 2024-04-26T18:40:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.345+0000 7f765dffb640 1 -- 172.21.15.84:0/2911186370 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76580054d0 con 0x7f7668103300 2024-04-26T18:40:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.345+0000 7f766f0c8640 1 -- 172.21.15.84:0/2911186370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f766819e320 con 0x7f7668103300 2024-04-26T18:40:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.345+0000 7f765dffb640 1 -- 172.21.15.84:0/2911186370 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658004c30 con 0x7f7668103300 2024-04-26T18:40:42.349 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.345+0000 7f766f0c8640 1 -- 172.21.15.84:0/2911186370 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76681a5bb0 con 0x7f7668103300 2024-04-26T18:40:42.355 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.347+0000 7f765dffb640 1 -- 172.21.15.84:0/2911186370 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f7658004e10 con 0x7f7668103300 2024-04-26T18:40:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f765dffb640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f76500a3dc0 0x7f76500a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f7671de8740 1 Processor -- start 2024-04-26T18:40:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f7671de8740 1 -- start start 2024-04-26T18:40:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f765dffb640 1 -- 172.21.15.84:0/2911186370 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f76580d0310 con 0x7f7668103300 2024-04-26T18:40:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f766ce64640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f76500a3dc0 0x7f76500a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f7671de8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7cf00 0x5631a8c7d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f7671de8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7d820 0x5631a8c87ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f7671de8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c880e0 0x5631a8c8a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f7671de8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5631a8be3460 con 0x5631a8c7cf00 2024-04-26T18:40:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f7671de8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5631a8b9d590 con 0x5631a8c7d820 2024-04-26T18:40:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f7671de8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5631a8bdf420 con 0x5631a8c880e0 2024-04-26T18:40:42.356 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f766ce64640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7cf00 0x5631a8c7d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f766ce64640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7cf00 0x5631a8c7d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35740/0 (socket says 172.21.15.84:35740) 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f766ce64640 1 -- 172.21.15.84:0/2477980699 learned_addr learned my addr 172.21.15.84:0/2477980699 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f766ce64640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f76500a3dc0 0x7f76500a6280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7664005d00 tx=0x7f76640103d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f765ffff640 1 --2- 172.21.15.84:0/2477980699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7d820 0x5631a8c87ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f766d665640 1 --2- 172.21.15.84:0/2477980699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c880e0 0x5631a8c8a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f766ce64640 1 -- 172.21.15.84:0/2477980699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c880e0 msgr2=0x5631a8c8a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f766ce64640 1 --2- 172.21.15.84:0/2477980699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c880e0 0x5631a8c8a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.348+0000 7f766ce64640 1 -- 172.21.15.84:0/2477980699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7d820 msgr2=0x5631a8c87ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.349+0000 7f766ce64640 1 --2- 172.21.15.84:0/2477980699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7d820 0x5631a8c87ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.349+0000 7f766ce64640 1 -- 172.21.15.84:0/2477980699 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a8baa0d0 con 0x5631a8c7cf00 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.349+0000 7f766ce64640 1 --2- 172.21.15.84:0/2477980699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7cf00 0x5631a8c7d2e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7664013510 tx=0x7f766401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe12d56714bc773 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.349+0000 7f764effd640 1 -- 172.21.15.84:0/2477980699 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7664011440 con 0x5631a8c7cf00 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.349+0000 7f764effd640 1 -- 172.21.15.84:0/2477980699 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7664011a90 con 0x5631a8c7cf00 2024-04-26T18:40:42.357 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.349+0000 7f764effd640 1 -- 172.21.15.84:0/2477980699 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f766401ea50 con 0x5631a8c7cf00 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.350+0000 7f7671de8740 1 -- 172.21.15.84:0/2477980699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7cf00 msgr2=0x5631a8c7d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.350+0000 7f7671de8740 1 --2- 172.21.15.84:0/2477980699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7cf00 0x5631a8c7d2e0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7664013510 tx=0x7f766401b000 comp rx=0 tx=0).stop 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.350+0000 7f7671de8740 1 -- 172.21.15.84:0/2477980699 shutdown_connections 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.350+0000 7f7671de8740 1 --2- 172.21.15.84:0/2477980699 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c880e0 0x5631a8c8a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.350+0000 7f7671de8740 1 --2- 172.21.15.84:0/2477980699 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7d820 0x5631a8c87ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.350+0000 7f7671de8740 1 --2- 172.21.15.84:0/2477980699 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7cf00 0x5631a8c7d2e0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.350+0000 7f7671de8740 1 -- 172.21.15.84:0/2477980699 >> 172.21.15.84:0/2477980699 conn(0x5631a8c79410 msgr2=0x5631a8c797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.350+0000 7f7671de8740 1 -- 172.21.15.84:0/2477980699 shutdown_connections 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.350+0000 7f7671de8740 1 -- 172.21.15.84:0/2477980699 wait complete. 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.350+0000 7f7671de8740 1 -- start start 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.350+0000 7f7671de8740 1 Processor -- start 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f7671de8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7cf00 0x5631a8c85090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f7671de8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c7d820 0x5631a8c855d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.358 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f7671de8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 0x5631a8c85b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f7671de8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5631a8c8ad10 con 0x5631a8c880e0 2024-04-26T18:40:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f7671de8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5631a8b6ab60 con 0x5631a8c7cf00 2024-04-26T18:40:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f7671de8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5631a8c8ae90 con 0x5631a8c7d820 2024-04-26T18:40:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f7671de8740 10 client.0 _ll_register_callbacks cb 0x5631a8b8f2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f766d665640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 0x5631a8c85b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f766d665640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 0x5631a8c85b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35750/0 (socket says 172.21.15.84:35750) 2024-04-26T18:40:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f766d665640 1 -- 172.21.15.84:0/4042625384 learned_addr learned my addr 172.21.15.84:0/4042625384 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f765ffff640 1 --2- 172.21.15.84:0/4042625384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c7d820 0x5631a8c855d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f766ce64640 1 --2- 172.21.15.84:0/4042625384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7cf00 0x5631a8c85090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f766d665640 1 -- 172.21.15.84:0/4042625384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c7d820 msgr2=0x5631a8c855d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f766d665640 1 --2- 172.21.15.84:0/4042625384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c7d820 0x5631a8c855d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f766d665640 1 -- 172.21.15.84:0/4042625384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7cf00 msgr2=0x5631a8c85090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:42.359 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f766d665640 1 --2- 172.21.15.84:0/4042625384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7cf00 0x5631a8c85090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f766d665640 1 -- 172.21.15.84:0/4042625384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a88f5210 con 0x5631a8c880e0 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f766d665640 1 --2- 172.21.15.84:0/4042625384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 0x5631a8c85b10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7660031a80 tx=0x7f7660031ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd9d16e4b25dd9b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f7647fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f7647fff640 1 -- 172.21.15.84:0/4042625384 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660003b70 con 0x5631a8c880e0 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f7647fff640 1 -- 172.21.15.84:0/4042625384 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f7660003d10 con 0x5631a8c880e0 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.351+0000 7f7671de8740 1 -- 172.21.15.84:0/4042625384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7664011a90 con 0x5631a8c880e0 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.352+0000 7f7647fff640 1 -- 172.21.15.84:0/4042625384 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7660004040 con 0x5631a8c880e0 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.352+0000 7f7671de8740 10 client.16923 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.352+0000 7f7671de8740 1 -- 172.21.15.84:0/4042625384 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5631a89c1620 con 0x5631a8c880e0 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.352+0000 7f7647fff640 1 -- 172.21.15.84:0/4042625384 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7664011a90 con 0x5631a8c880e0 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.352+0000 7f7647fff640 1 -- 172.21.15.84:0/4042625384 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f766004b020 con 0x5631a8c880e0 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.352+0000 7f7671de8740 10 client.16923 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.352+0000 7f7671de8740 10 client.16923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:40:42.360 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.352+0000 7f7671de8740 20 client.16923 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.352+0000 7f7671de8740 1 --2- 172.21.15.84:0/4042625384 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5631a8c98290 0x5631a8c9a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.352+0000 7f7671de8740 4 client.16923 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.352+0000 7f7671de8740 1 -- 172.21.15.84:0/4042625384 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631a8bf1070 con 0x5631a8c98290 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.352+0000 7f766ce64640 1 --2- 172.21.15.84:0/4042625384 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5631a8c98290 0x5631a8c9a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.353+0000 7f766ce64640 1 --2- 172.21.15.84:0/4042625384 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5631a8c98290 0x5631a8c9a680 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b9a2e7d205c6937 server_cookie=8d52e7bcc68ece7e in_seq=0 out_seq=0 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.353+0000 7f7647fff640 10 client.16923 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.353+0000 7f7647fff640 1 -- 172.21.15.84:0/4042625384 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631a8bf1070 con 0x5631a8c98290 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.353+0000 7f7647fff640 10 client.16923 handle_command_reply: tid=0 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.353+0000 7f7671de8740 1 client.16923 shutdown 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.353+0000 7f7671de8740 1 -- 172.21.15.84:0/4042625384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 msgr2=0x5631a8c85b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.353+0000 7f7671de8740 1 --2- 172.21.15.84:0/4042625384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 0x5631a8c85b10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7660031a80 tx=0x7f7660031ab0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.354+0000 7f7671de8740 1 -- 172.21.15.84:0/4042625384 shutdown_connections 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.354+0000 7f7671de8740 1 --2- 172.21.15.84:0/4042625384 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5631a8c98290 0x5631a8c9a680 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.361 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.354+0000 7f7671de8740 1 --2- 172.21.15.84:0/4042625384 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 0x5631a8c85b10 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.354+0000 7f7647fff640 0 client.16923 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.354+0000 7f7671de8740 1 --2- 172.21.15.84:0/4042625384 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c7d820 0x5631a8c855d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.354+0000 7f7671de8740 1 --2- 172.21.15.84:0/4042625384 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7cf00 0x5631a8c85090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.354+0000 7f7671de8740 1 -- 172.21.15.84:0/4042625384 >> 172.21.15.84:0/4042625384 conn(0x5631a8c79410 msgr2=0x5631a8c7f190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.354+0000 7f7671de8740 1 -- 172.21.15.84:0/4042625384 shutdown_connections 2024-04-26T18:40:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.354+0000 7f7671de8740 1 -- 172.21.15.84:0/4042625384 wait complete. 2024-04-26T18:40:42.362 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.354+0000 7f7671de8740 20 client.16923 trim_cache size 0 max 16384 2024-04-26T18:40:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.368+0000 7f7671de8740 1 Processor -- start 2024-04-26T18:40:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.368+0000 7f7671de8740 1 -- start start 2024-04-26T18:40:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f7671de8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7cf00 0x5631a8c93bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f7671de8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7d820 0x5631a8c7f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f7671de8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c880e0 0x5631a8c9ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f7671de8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f76681ae550 con 0x5631a8c7d820 2024-04-26T18:40:42.378 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f7671de8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76681b0090 con 0x5631a8c7cf00 2024-04-26T18:40:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f7671de8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76681aec20 con 0x5631a8c880e0 2024-04-26T18:40:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f765ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7d820 0x5631a8c7f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f765ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7d820 0x5631a8c7f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35766/0 (socket says 172.21.15.84:35766) 2024-04-26T18:40:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f765ffff640 1 -- 172.21.15.84:0/3273460781 learned_addr learned my addr 172.21.15.84:0/3273460781 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f766ce64640 1 --2- 172.21.15.84:0/3273460781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7cf00 0x5631a8c93bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f766d665640 1 --2- 172.21.15.84:0/3273460781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c880e0 0x5631a8c9ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f765ffff640 1 -- 172.21.15.84:0/3273460781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c880e0 msgr2=0x5631a8c9ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f765ffff640 1 --2- 172.21.15.84:0/3273460781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c880e0 0x5631a8c9ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f765ffff640 1 -- 172.21.15.84:0/3273460781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7cf00 msgr2=0x5631a8c93bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f765ffff640 1 --2- 172.21.15.84:0/3273460781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7cf00 0x5631a8c93bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f765ffff640 1 -- 172.21.15.84:0/3273460781 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a8c9a9f0 con 0x5631a8c7d820 2024-04-26T18:40:42.379 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f765ffff640 1 --2- 172.21.15.84:0/3273460781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7d820 0x5631a8c7f910 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7658021000 tx=0x7f765809c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a277a0842dbfc70 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f764e7fc640 1 -- 172.21.15.84:0/3273460781 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76580cf040 con 0x5631a8c7d820 2024-04-26T18:40:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f764e7fc640 1 -- 172.21.15.84:0/3273460781 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76580a4ee0 con 0x5631a8c7d820 2024-04-26T18:40:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.369+0000 7f764e7fc640 1 -- 172.21.15.84:0/3273460781 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658020040 con 0x5631a8c7d820 2024-04-26T18:40:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.370+0000 7f7671de8740 1 -- 172.21.15.84:0/3273460781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7d820 msgr2=0x5631a8c7f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.370+0000 7f7671de8740 1 --2- 172.21.15.84:0/3273460781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7d820 0x5631a8c7f910 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f7658021000 tx=0x7f765809c9f0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.370+0000 7f7671de8740 1 -- 172.21.15.84:0/3273460781 shutdown_connections 2024-04-26T18:40:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.370+0000 7f7671de8740 1 --2- 172.21.15.84:0/3273460781 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c880e0 0x5631a8c9ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.370+0000 7f7671de8740 1 --2- 172.21.15.84:0/3273460781 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c7d820 0x5631a8c7f910 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.370+0000 7f7671de8740 1 --2- 172.21.15.84:0/3273460781 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7cf00 0x5631a8c93bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.370+0000 7f7671de8740 1 -- 172.21.15.84:0/3273460781 >> 172.21.15.84:0/3273460781 conn(0x5631a8c79410 msgr2=0x5631a8c858e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:42.380 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.370+0000 7f7671de8740 1 -- 172.21.15.84:0/3273460781 shutdown_connections 2024-04-26T18:40:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.370+0000 7f7671de8740 1 -- 172.21.15.84:0/3273460781 wait complete. 2024-04-26T18:40:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.370+0000 7f7671de8740 1 -- start start 2024-04-26T18:40:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.370+0000 7f7671de8740 1 Processor -- start 2024-04-26T18:40:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f7671de8740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c7cf00 0x5631a8c81f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f7671de8740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7d820 0x5631a8c824c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f7671de8740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 0x5631a8c82a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f7671de8740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f76681af9c0 con 0x5631a8c880e0 2024-04-26T18:40:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f7671de8740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76681af2f0 con 0x5631a8c7d820 2024-04-26T18:40:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f7671de8740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76681ac550 con 0x5631a8c7cf00 2024-04-26T18:40:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f7671de8740 10 client.0 _ll_register_callbacks cb 0x7f766806c1b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f766d665640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 0x5631a8c82a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.381 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f766d665640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 0x5631a8c82a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35770/0 (socket says 172.21.15.84:35770) 2024-04-26T18:40:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f766d665640 1 -- 172.21.15.84:0/2261399573 learned_addr learned my addr 172.21.15.84:0/2261399573 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f765ffff640 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7d820 0x5631a8c824c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f766ce64640 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c7cf00 0x5631a8c81f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f766d665640 1 -- 172.21.15.84:0/2261399573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c7cf00 msgr2=0x5631a8c81f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f766d665640 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c7cf00 0x5631a8c81f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f766d665640 1 -- 172.21.15.84:0/2261399573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7d820 msgr2=0x5631a8c824c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f766d665640 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7d820 0x5631a8c824c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f766d665640 1 -- 172.21.15.84:0/2261399573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a8cfda10 con 0x5631a8c880e0 2024-04-26T18:40:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f766ce64640 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c7cf00 0x5631a8c81f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:40:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f766d665640 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 0x5631a8c82a00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f7660030cc0 tx=0x7f76600095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc4e895e9f3c156 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:42.382 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.371+0000 7f7647fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7647fff640 1 -- 172.21.15.84:0/2261399573 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7660030eb0 con 0x5631a8c880e0 2024-04-26T18:40:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7647fff640 1 -- 172.21.15.84:0/2261399573 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f7660003b80 con 0x5631a8c880e0 2024-04-26T18:40:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7671de8740 1 -- 172.21.15.84:0/2261399573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76580a4ee0 con 0x5631a8c880e0 2024-04-26T18:40:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7647fff640 1 -- 172.21.15.84:0/2261399573 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f766005b050 con 0x5631a8c880e0 2024-04-26T18:40:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7671de8740 10 client.16929 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7671de8740 1 -- 172.21.15.84:0/2261399573 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5631a8cfed00 con 0x5631a8c880e0 2024-04-26T18:40:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7647fff640 1 -- 172.21.15.84:0/2261399573 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76580a4ee0 con 0x5631a8c880e0 2024-04-26T18:40:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7647fff640 1 -- 172.21.15.84:0/2261399573 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f766004b040 con 0x5631a8c880e0 2024-04-26T18:40:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7671de8740 10 client.16929 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7671de8740 10 client.16929 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:40:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7671de8740 20 client.16929 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:42.383 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7671de8740 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5631a8cff4d0 0x5631a8d018f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7671de8740 4 client.16929 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:40:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f7671de8740 1 -- 172.21.15.84:0/2261399573 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5631a8beaee0 con 0x5631a8cff4d0 2024-04-26T18:40:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.372+0000 7f766ce64640 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5631a8cff4d0 0x5631a8d018f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.373+0000 7f7647fff640 10 client.16929 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:42.384 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.373+0000 7f766ce64640 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5631a8cff4d0 0x5631a8d018f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=960ff7150c0f908a server_cookie=94046aea8bff4aa8 in_seq=0 out_seq=0 2024-04-26T18:40:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7647fff640 1 -- 172.21.15.84:0/2261399573 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5631a8beaee0 con 0x5631a8cff4d0 2024-04-26T18:40:42.385 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7647fff640 10 client.16929 handle_command_reply: tid=0 2024-04-26T18:40:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7671de8740 1 client.16929 shutdown 2024-04-26T18:40:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7671de8740 1 -- 172.21.15.84:0/2261399573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 msgr2=0x5631a8c82a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7671de8740 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 0x5631a8c82a00 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f7660030cc0 tx=0x7f76600095c0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7671de8740 1 -- 172.21.15.84:0/2261399573 shutdown_connections 2024-04-26T18:40:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7671de8740 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5631a8cff4d0 0x5631a8d018f0 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7671de8740 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5631a8c880e0 0x5631a8c82a00 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7647fff640 0 client.16929 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7671de8740 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5631a8c7d820 0x5631a8c824c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7671de8740 1 --2- 172.21.15.84:0/2261399573 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5631a8c7cf00 0x5631a8c81f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7671de8740 1 -- 172.21.15.84:0/2261399573 >> 172.21.15.84:0/2261399573 conn(0x5631a8c79410 msgr2=0x5631a8c77d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:42.386 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7671de8740 1 -- 172.21.15.84:0/2261399573 shutdown_connections 2024-04-26T18:40:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7671de8740 1 -- 172.21.15.84:0/2261399573 wait complete. 2024-04-26T18:40:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.383+0000 7f7671de8740 20 client.16929 trim_cache size 0 max 16384 2024-04-26T18:40:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:40:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:40:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:40:42.387 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:40:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.384+0000 7f766f0c8640 1 -- 172.21.15.84:0/2911186370 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f76500a3dc0 msgr2=0x7f76500a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.384+0000 7f766f0c8640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f76500a3dc0 0x7f76500a6280 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f7664005d00 tx=0x7f76640103d0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.384+0000 7f766f0c8640 1 -- 172.21.15.84:0/2911186370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7668103300 msgr2=0x7f76681995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:42.388 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.384+0000 7f766f0c8640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7668103300 0x7f76681995d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f765800d770 tx=0x7f765800dc30 comp rx=0 tx=0).stop 2024-04-26T18:40:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.384+0000 7f766f0c8640 1 -- 172.21.15.84:0/2911186370 shutdown_connections 2024-04-26T18:40:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.384+0000 7f766f0c8640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f76500a3dc0 0x7f76500a6280 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.384+0000 7f766f0c8640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7668108950 0x7f766819d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.384+0000 7f766f0c8640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7668103300 0x7f76681995d0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.384+0000 7f766f0c8640 1 --2- 172.21.15.84:0/2911186370 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7668102940 0x7f7668199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.384+0000 7f766f0c8640 1 -- 172.21.15.84:0/2911186370 >> 172.21.15.84:0/2911186370 conn(0x7f7668079dc0 msgr2=0x7f7668100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.384+0000 7f766f0c8640 1 -- 172.21.15.84:0/2911186370 shutdown_connections 2024-04-26T18:40:42.389 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:42.384+0000 7f766f0c8640 1 -- 172.21.15.84:0/2911186370 wait complete. 2024-04-26T18:40:42.824 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:40:43.071 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:40:43.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:43 smithi084 ceph-mon[29141]: pgmap v399: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-26T18:40:43.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:43 smithi103 ceph-mon[32727]: pgmap v399: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-26T18:40:43.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:43 smithi174 ceph-mon[32705]: pgmap v399: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 5.3 KiB/s wr, 10 op/s 2024-04-26T18:40:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/1481270908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd641094e0 msgr2=0x7fbd641098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 --2- 172.21.15.84:0/1481270908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd641094e0 0x7fbd641098c0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c009990 tx=0x7fbd5c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:40:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/1481270908 shutdown_connections 2024-04-26T18:40:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 --2- 172.21.15.84:0/1481270908 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd641094e0 0x7fbd641098c0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 --2- 172.21.15.84:0/1481270908 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd64103e40 0x7fbd64110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 --2- 172.21.15.84:0/1481270908 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd641034a0 0x7fbd64103900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/1481270908 >> 172.21.15.84:0/1481270908 conn(0x7fbd6407a4f0 msgr2=0x7fbd6407a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/1481270908 shutdown_connections 2024-04-26T18:40:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/1481270908 wait complete. 2024-04-26T18:40:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 Processor -- start 2024-04-26T18:40:44.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 -- start start 2024-04-26T18:40:44.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd641034a0 0x7fbd64075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:44.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd64103e40 0x7fbd64075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:44.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd641094e0 0x7fbd640761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:44.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbd641135d0 con 0x7fbd641034a0 2024-04-26T18:40:44.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbd64113450 con 0x7fbd641094e0 2024-04-26T18:40:44.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.721+0000 7fbd6bc7c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbd64113750 con 0x7fbd64103e40 2024-04-26T18:40:44.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd69a18700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd641034a0 0x7fbd64075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:44.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd69a18700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd641034a0 0x7fbd64075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:35780/0 (socket says 172.21.15.84:35780) 2024-04-26T18:40:44.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd69a18700 1 -- 172.21.15.84:0/351838333 learned_addr learned my addr 172.21.15.84:0/351838333 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:44.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd69217700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd64103e40 0x7fbd64075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:44.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd6a219700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd641094e0 0x7fbd640761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:44.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd69a18700 1 -- 172.21.15.84:0/351838333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd64103e40 msgr2=0x7fbd64075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:44.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd69a18700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd64103e40 0x7fbd64075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:44.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd69a18700 1 -- 172.21.15.84:0/351838333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd641094e0 msgr2=0x7fbd640761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:44.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd69a18700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd641094e0 0x7fbd640761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:44.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd69a18700 1 -- 172.21.15.84:0/351838333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd641a8010 con 0x7fbd641034a0 2024-04-26T18:40:44.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd6a219700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd641094e0 0x7fbd640761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:44.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd69a18700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd641034a0 0x7fbd64075720 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c000c00 tx=0x7fbd5c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:44.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd62ffd700 1 -- 172.21.15.84:0/351838333 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c027070 con 0x7fbd641034a0 2024-04-26T18:40:44.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/351838333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd641a82a0 con 0x7fbd641034a0 2024-04-26T18:40:44.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd62ffd700 1 -- 172.21.15.84:0/351838333 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd5c003f20 con 0x7fbd641034a0 2024-04-26T18:40:44.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd62ffd700 1 -- 172.21.15.84:0/351838333 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd5c022cf0 con 0x7fbd641034a0 2024-04-26T18:40:44.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.722+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/351838333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd641a8700 con 0x7fbd641034a0 2024-04-26T18:40:44.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.723+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/351838333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd6406a110 con 0x7fbd641034a0 2024-04-26T18:40:44.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.724+0000 7fbd62ffd700 1 -- 172.21.15.84:0/351838333 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fbd5c02a070 con 0x7fbd641034a0 2024-04-26T18:40:44.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.725+0000 7fbd62ffd700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbd4c0a3db0 0x7fbd4c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:44.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.725+0000 7fbd69217700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbd4c0a3db0 0x7fbd4c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:44.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.725+0000 7fbd62ffd700 1 -- 172.21.15.84:0/351838333 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fbd5c0d15e0 con 0x7fbd641034a0 2024-04-26T18:40:44.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.725+0000 7fbd69217700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbd4c0a3db0 0x7fbd4c0a6270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd58009550 tx=0x7fbd580091b0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:44.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.730+0000 7fbd62ffd700 1 -- 172.21.15.84:0/351838333 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd5c013070 con 0x7fbd641034a0 2024-04-26T18:40:44.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.980+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/351838333 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbd6402e490 con 0x7fbd641034a0 2024-04-26T18:40:44.980 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.980+0000 7fbd62ffd700 1 -- 172.21.15.84:0/351838333 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7fbd5c09dcd0 con 0x7fbd641034a0 2024-04-26T18:40:44.981 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:40:44.981 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:40:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.982+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/351838333 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbd4c0a3db0 msgr2=0x7fbd4c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.982+0000 7fbd6bc7c700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbd4c0a3db0 0x7fbd4c0a6270 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fbd58009550 tx=0x7fbd580091b0 comp rx=0 tx=0).stop 2024-04-26T18:40:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.982+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/351838333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd641034a0 msgr2=0x7fbd64075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.982+0000 7fbd6bc7c700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd641034a0 0x7fbd64075720 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c000c00 tx=0x7fbd5c0047b0 comp rx=0 tx=0).stop 2024-04-26T18:40:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.983+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/351838333 shutdown_connections 2024-04-26T18:40:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.983+0000 7fbd6bc7c700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbd4c0a3db0 0x7fbd4c0a6270 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.983+0000 7fbd6bc7c700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbd641034a0 0x7fbd64075720 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.983+0000 7fbd6bc7c700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbd641094e0 0x7fbd640761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.983+0000 7fbd6bc7c700 1 --2- 172.21.15.84:0/351838333 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbd64103e40 0x7fbd64075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.983+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/351838333 >> 172.21.15.84:0/351838333 conn(0x7fbd6407a4f0 msgr2=0x7fbd641015f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.983+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/351838333 shutdown_connections 2024-04-26T18:40:44.983 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:44.983+0000 7fbd6bc7c700 1 -- 172.21.15.84:0/351838333 wait complete. 2024-04-26T18:40:44.984 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:40:45.660 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:45 smithi084 ceph-mon[29141]: pgmap v400: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 16 op/s 2024-04-26T18:40:45.660 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:45 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/351838333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:40:45.690 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id 2a546377 --release --await' 2024-04-26T18:40:45.690 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db --set-id 2a546377 --release --await 2024-04-26T18:40:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:45 smithi103 ceph-mon[32727]: pgmap v400: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 16 op/s 2024-04-26T18:40:45.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:45 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/351838333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:40:45.932 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:40:45.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:45 smithi174 ceph-mon[32705]: pgmap v400: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.6 KiB/s wr, 16 op/s 2024-04-26T18:40:45.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:45 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/351838333' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:40:47.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.657+0000 7fd8ce8c9700 1 -- 172.21.15.84:0/2458310032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c8103e80 msgr2=0x7fd8c8110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:47.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.657+0000 7fd8ce8c9700 1 --2- 172.21.15.84:0/2458310032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c8103e80 0x7fd8c8110330 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fd8c000b0d0 tx=0x7fd8c000b3e0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.657+0000 7fd8ce8c9700 1 -- 172.21.15.84:0/2458310032 shutdown_connections 2024-04-26T18:40:47.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.657+0000 7fd8ce8c9700 1 --2- 172.21.15.84:0/2458310032 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c8103e80 0x7fd8c8110330 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.657+0000 7fd8ce8c9700 1 --2- 172.21.15.84:0/2458310032 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c8109560 0x7fd8c8109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.657+0000 7fd8ce8c9700 1 --2- 172.21.15.84:0/2458310032 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8c81034c0 0x7fd8c8103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.657+0000 7fd8ce8c9700 1 -- 172.21.15.84:0/2458310032 >> 172.21.15.84:0/2458310032 conn(0x7fd8c807a5d0 msgr2=0x7fd8c807a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:47.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.657+0000 7fd8ce8c9700 1 -- 172.21.15.84:0/2458310032 shutdown_connections 2024-04-26T18:40:47.658 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.657+0000 7fd8ce8c9700 1 -- 172.21.15.84:0/2458310032 wait complete. 2024-04-26T18:40:47.659 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:47 smithi084 ceph-mon[29141]: pgmap v401: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-26T18:40:47.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.658+0000 7fd8ce8c9700 1 Processor -- start 2024-04-26T18:40:47.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.658+0000 7fd8ce8c9700 1 -- start start 2024-04-26T18:40:47.660 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8ce8c9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c81034c0 0x7fd8c819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8ce8c9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c8103e80 0x7fd8c819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8ce8c9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8c8109560 0x7fd8c8199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8ce8c9700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81136c0 con 0x7fd8c81034c0 2024-04-26T18:40:47.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8ce8c9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8113540 con 0x7fd8c8109560 2024-04-26T18:40:47.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8ce8c9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c8113840 con 0x7fd8c8103e80 2024-04-26T18:40:47.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8c7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c81034c0 0x7fd8c819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8c7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c81034c0 0x7fd8c819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57524/0 (socket says 172.21.15.84:57524) 2024-04-26T18:40:47.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8c7fff700 1 -- 172.21.15.84:0/967575851 learned_addr learned my addr 172.21.15.84:0/967575851 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:47.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8cce66700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8c8109560 0x7fd8c8199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c8103e80 0x7fd8c819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8c7fff700 1 -- 172.21.15.84:0/967575851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c8103e80 msgr2=0x7fd8c819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:47.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8c7fff700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c8103e80 0x7fd8c819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.661 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8c7fff700 1 -- 172.21.15.84:0/967575851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8c8109560 msgr2=0x7fd8c8199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8c7fff700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8c8109560 0x7fd8c8199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8c7fff700 1 -- 172.21.15.84:0/967575851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8c819a0c0 con 0x7fd8c81034c0 2024-04-26T18:40:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.659+0000 7fd8c7fff700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c81034c0 0x7fd8c819f7a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800b530 tx=0x7fd8b800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.660+0000 7fd8c57fa700 1 -- 172.21.15.84:0/967575851 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8015020 con 0x7fd8c81034c0 2024-04-26T18:40:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.660+0000 7fd8ce8c9700 1 -- 172.21.15.84:0/967575851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8c819a3b0 con 0x7fd8c81034c0 2024-04-26T18:40:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.660+0000 7fd8c57fa700 1 -- 172.21.15.84:0/967575851 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8b8011ce0 con 0x7fd8c81034c0 2024-04-26T18:40:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.660+0000 7fd8c57fa700 1 -- 172.21.15.84:0/967575851 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b8010dd0 con 0x7fd8c81034c0 2024-04-26T18:40:47.662 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.660+0000 7fd8ce8c9700 1 -- 172.21.15.84:0/967575851 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8c81a6670 con 0x7fd8c81034c0 2024-04-26T18:40:47.664 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.662+0000 7fd8c57fa700 1 -- 172.21.15.84:0/967575851 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fd8b8018020 con 0x7fd8c81034c0 2024-04-26T18:40:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.662+0000 7fd8c57fa700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd8b00a3db0 0x7fd8b00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.662+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd8b00a3db0 0x7fd8b00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.662+0000 7fd8c57fa700 1 -- 172.21.15.84:0/967575851 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fd8b80cfec0 con 0x7fd8c81034c0 2024-04-26T18:40:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.663+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd8b00a3db0 0x7fd8b00a6270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd8c807aa80 tx=0x7fd8bc00a380 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.663+0000 7fd8d915bb80 1 Processor -- start 2024-04-26T18:40:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.663+0000 7fd8d915bb80 1 -- start start 2024-04-26T18:40:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.663+0000 7fd8d915bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f41a0 0x56327d2f4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8d915bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d2f4ac0 0x56327d303840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8d915bb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d303d80 0x56327d306170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8d915bb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56327d247840 con 0x56327d303d80 2024-04-26T18:40:47.665 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8d915bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56327d1e66e0 con 0x56327d2f41a0 2024-04-26T18:40:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8d915bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56327d243800 con 0x56327d2f4ac0 2024-04-26T18:40:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8cce66700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d303d80 0x56327d306170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8cce66700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d303d80 0x56327d306170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57540/0 (socket says 172.21.15.84:57540) 2024-04-26T18:40:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8cce66700 1 -- 172.21.15.84:0/2963793906 learned_addr learned my addr 172.21.15.84:0/2963793906 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/2963793906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d2f4ac0 0x56327d303840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8c7fff700 1 --2- 172.21.15.84:0/2963793906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f41a0 0x56327d2f4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8cce66700 1 -- 172.21.15.84:0/2963793906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d2f4ac0 msgr2=0x56327d303840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8cce66700 1 --2- 172.21.15.84:0/2963793906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d2f4ac0 0x56327d303840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8cce66700 1 -- 172.21.15.84:0/2963793906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f41a0 msgr2=0x56327d2f4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8cce66700 1 --2- 172.21.15.84:0/2963793906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f41a0 0x56327d2f4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8cce66700 1 -- 172.21.15.84:0/2963793906 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56327d1949d0 con 0x56327d303d80 2024-04-26T18:40:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/2963793906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d2f4ac0 0x56327d303840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:40:47.666 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8cce66700 1 --2- 172.21.15.84:0/2963793906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d303d80 0x56327d306170 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009180 tx=0x7fd8c00091b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a63712aaba7b5c78 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:47.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8ae7fc700 1 -- 172.21.15.84:0/2963793906 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0003c00 con 0x56327d303d80 2024-04-26T18:40:47.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8ae7fc700 1 -- 172.21.15.84:0/2963793906 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8c0003da0 con 0x56327d303d80 2024-04-26T18:40:47.667 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.664+0000 7fd8ae7fc700 1 -- 172.21.15.84:0/2963793906 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8c0005490 con 0x56327d303d80 2024-04-26T18:40:47.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.665+0000 7fd8d915bb80 1 -- 172.21.15.84:0/2963793906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d303d80 msgr2=0x56327d306170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:47.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.665+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/2963793906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d303d80 0x56327d306170 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fd8c0009180 tx=0x7fd8c00091b0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.665+0000 7fd8d915bb80 1 -- 172.21.15.84:0/2963793906 shutdown_connections 2024-04-26T18:40:47.671 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.665+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/2963793906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d303d80 0x56327d306170 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.665+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/2963793906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f41a0 0x56327d2f4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.665+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/2963793906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d2f4ac0 0x56327d303840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.665+0000 7fd8d915bb80 1 -- 172.21.15.84:0/2963793906 >> 172.21.15.84:0/2963793906 conn(0x56327d221e40 msgr2=0x56327d2f0690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.665+0000 7fd8d915bb80 1 -- 172.21.15.84:0/2963793906 shutdown_connections 2024-04-26T18:40:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.665+0000 7fd8d915bb80 1 -- 172.21.15.84:0/2963793906 wait complete. 2024-04-26T18:40:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.666+0000 7fd8d915bb80 1 -- start start 2024-04-26T18:40:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.666+0000 7fd8d915bb80 1 Processor -- start 2024-04-26T18:40:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.666+0000 7fd8d915bb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f41a0 0x56327d3034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.666+0000 7fd8d915bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f4ac0 0x56327d3007f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.666+0000 7fd8d915bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d303d80 0x56327d300d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.666+0000 7fd8d915bb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56327d3068f0 con 0x56327d2f41a0 2024-04-26T18:40:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.666+0000 7fd8d915bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56327d1ada50 con 0x56327d2f4ac0 2024-04-26T18:40:47.672 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.666+0000 7fd8d915bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56327d306a70 con 0x56327d303d80 2024-04-26T18:40:47.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.666+0000 7fd8d915bb80 10 client.0 _ll_register_callbacks cb 0x56327d1db680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:47.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.666+0000 7fd8c7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f41a0 0x56327d3034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.666+0000 7fd8c7fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f41a0 0x56327d3034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57550/0 (socket says 172.21.15.84:57550) 2024-04-26T18:40:47.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.666+0000 7fd8c7fff700 1 -- 172.21.15.84:0/516222286 learned_addr learned my addr 172.21.15.84:0/516222286 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:47.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.666+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f4ac0 0x56327d3007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8c7fff700 1 -- 172.21.15.84:0/516222286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d303d80 msgr2=0x56327d300d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:47.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8cce66700 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d303d80 0x56327d300d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8c7fff700 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d303d80 0x56327d300d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.673 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8c7fff700 1 -- 172.21.15.84:0/516222286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f4ac0 msgr2=0x56327d3007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8c7fff700 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f4ac0 0x56327d3007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8c7fff700 1 -- 172.21.15.84:0/516222286 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56327d3013e0 con 0x56327d2f41a0 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f4ac0 0x56327d3007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8c7fff700 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f41a0 0x56327d3034f0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80cdbd0 tx=0x7fd8b80cdb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=431f0fefcc23752d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8a77fe700 1 -- 172.21.15.84:0/516222286 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8b80b4020 con 0x56327d2f41a0 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8a77fe700 1 -- 172.21.15.84:0/516222286 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fd8b809c390 con 0x56327d2f41a0 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8d915bb80 1 -- 172.21.15.84:0/516222286 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8c0003da0 con 0x56327d2f41a0 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8a77fe700 1 -- 172.21.15.84:0/516222286 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8b8018520 con 0x56327d2f41a0 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8d915bb80 10 client.16947 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8d915bb80 1 -- 172.21.15.84:0/516222286 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56327d311420 con 0x56327d2f41a0 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8a77fe700 1 -- 172.21.15.84:0/516222286 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8c0003da0 con 0x56327d2f41a0 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8a77fe700 1 -- 172.21.15.84:0/516222286 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd8b80a4710 con 0x56327d2f41a0 2024-04-26T18:40:47.674 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8d915bb80 10 client.16947 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:47.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.667+0000 7fd8d915bb80 10 client.16947 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:40:47.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.668+0000 7fd8d915bb80 20 client.16947 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:47.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.668+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56327d3139b0 0x56327d315da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.668+0000 7fd8d915bb80 4 client.16947 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:40:47.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.668+0000 7fd8d915bb80 1 -- 172.21.15.84:0/516222286 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56327d255220 con 0x56327d3139b0 2024-04-26T18:40:47.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.668+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56327d3139b0 0x56327d315da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.668+0000 7fd8a77fe700 10 client.16947 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:47.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.668+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56327d3139b0 0x56327d315da0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3d9ad6440a9002a server_cookie=cf90421c518b1723 in_seq=0 out_seq=0 2024-04-26T18:40:47.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8a77fe700 1 -- 172.21.15.84:0/516222286 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56327d255220 con 0x56327d3139b0 2024-04-26T18:40:47.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8a77fe700 10 client.16947 handle_command_reply: tid=0 2024-04-26T18:40:47.675 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8d915bb80 1 client.16947 shutdown 2024-04-26T18:40:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8d915bb80 1 -- 172.21.15.84:0/516222286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f41a0 msgr2=0x56327d3034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f41a0 0x56327d3034f0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7fd8b80cdbd0 tx=0x7fd8b80cdb00 comp rx=0 tx=0).stop 2024-04-26T18:40:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8d915bb80 1 -- 172.21.15.84:0/516222286 shutdown_connections 2024-04-26T18:40:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56327d3139b0 0x56327d315da0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f41a0 0x56327d3034f0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d303d80 0x56327d300d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/516222286 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f4ac0 0x56327d3007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8d915bb80 1 -- 172.21.15.84:0/516222286 >> 172.21.15.84:0/516222286 conn(0x56327d221e40 msgr2=0x56327d2f02c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8a77fe700 0 client.16947 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8d915bb80 1 -- 172.21.15.84:0/516222286 shutdown_connections 2024-04-26T18:40:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8d915bb80 1 -- 172.21.15.84:0/516222286 wait complete. 2024-04-26T18:40:47.676 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.669+0000 7fd8d915bb80 20 client.16947 trim_cache size 0 max 16384 2024-04-26T18:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.688+0000 7fd8d915bb80 1 Processor -- start 2024-04-26T18:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.688+0000 7fd8d915bb80 1 -- start start 2024-04-26T18:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8d915bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d2f41a0 0x56327d2f0840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8d915bb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 0x56327d2fc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8d915bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d303d80 0x56327d2eed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8d915bb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81ab970 con 0x56327d2f4ac0 2024-04-26T18:40:47.693 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8d915bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81ad4b0 con 0x56327d303d80 2024-04-26T18:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8d915bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81ac040 con 0x56327d2f41a0 2024-04-26T18:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8c77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 0x56327d2fc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8c77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 0x56327d2fc070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57558/0 (socket says 172.21.15.84:57558) 2024-04-26T18:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8c77fe700 1 -- 172.21.15.84:0/2632978054 learned_addr learned my addr 172.21.15.84:0/2632978054 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8c7fff700 1 --2- 172.21.15.84:0/2632978054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d2f41a0 0x56327d2f0840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8c77fe700 1 -- 172.21.15.84:0/2632978054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d2f41a0 msgr2=0x56327d2f0840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/2632978054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d2f41a0 0x56327d2f0840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8c77fe700 1 -- 172.21.15.84:0/2632978054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d303d80 msgr2=0x56327d2eed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8cce66700 1 --2- 172.21.15.84:0/2632978054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d303d80 0x56327d2eed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/2632978054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d303d80 0x56327d2eed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8c77fe700 1 -- 172.21.15.84:0/2632978054 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56327d2f1b80 con 0x56327d2f4ac0 2024-04-26T18:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/2632978054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 0x56327d2fc070 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00f750 tx=0x7fd8bc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765625e9ca4f7545 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:47.694 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8adffb700 1 -- 172.21.15.84:0/2632978054 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc016070 con 0x56327d2f4ac0 2024-04-26T18:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8adffb700 1 -- 172.21.15.84:0/2632978054 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8bc014070 con 0x56327d2f4ac0 2024-04-26T18:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.689+0000 7fd8adffb700 1 -- 172.21.15.84:0/2632978054 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc0277e0 con 0x56327d2f4ac0 2024-04-26T18:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.690+0000 7fd8d915bb80 1 -- 172.21.15.84:0/2632978054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 msgr2=0x56327d2fc070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.690+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/2632978054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 0x56327d2fc070 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00f750 tx=0x7fd8bc012000 comp rx=0 tx=0).stop 2024-04-26T18:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.690+0000 7fd8d915bb80 1 -- 172.21.15.84:0/2632978054 shutdown_connections 2024-04-26T18:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.690+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/2632978054 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d303d80 0x56327d2eed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.690+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/2632978054 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d2f41a0 0x56327d2f0840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.690+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/2632978054 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 0x56327d2fc070 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.690+0000 7fd8d915bb80 1 -- 172.21.15.84:0/2632978054 >> 172.21.15.84:0/2632978054 conn(0x56327d221e40 msgr2=0x56327d2f6110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.690+0000 7fd8d915bb80 1 -- 172.21.15.84:0/2632978054 shutdown_connections 2024-04-26T18:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.690+0000 7fd8d915bb80 1 -- 172.21.15.84:0/2632978054 wait complete. 2024-04-26T18:40:47.695 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.690+0000 7fd8d915bb80 1 -- start start 2024-04-26T18:40:47.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8d915bb80 1 Processor -- start 2024-04-26T18:40:47.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8d915bb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f41a0 0x56327d30be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8d915bb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 0x56327d3072c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8d915bb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d303d80 0x56327d307800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8d915bb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81acde0 con 0x56327d2f4ac0 2024-04-26T18:40:47.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8d915bb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81ac710 con 0x56327d2f41a0 2024-04-26T18:40:47.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8d915bb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd8c81a9930 con 0x56327d303d80 2024-04-26T18:40:47.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8d915bb80 10 client.0 _ll_register_callbacks cb 0x56327d1db680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:47.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8c77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 0x56327d3072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.696 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8c77fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 0x56327d3072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57574/0 (socket says 172.21.15.84:57574) 2024-04-26T18:40:47.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8c77fe700 1 -- 172.21.15.84:0/577819940 learned_addr learned my addr 172.21.15.84:0/577819940 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:47.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8cce66700 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d303d80 0x56327d307800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8c7fff700 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f41a0 0x56327d30be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8c77fe700 1 -- 172.21.15.84:0/577819940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d303d80 msgr2=0x56327d307800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:47.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d303d80 0x56327d307800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8c77fe700 1 -- 172.21.15.84:0/577819940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f41a0 msgr2=0x56327d30be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:47.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f41a0 0x56327d30be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8c77fe700 1 -- 172.21.15.84:0/577819940 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56327d307dc0 con 0x56327d2f4ac0 2024-04-26T18:40:47.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8c7fff700 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f41a0 0x56327d30be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:40:47.697 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8c77fe700 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 0x56327d3072c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00a470 tx=0x7fd8bc004190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b311f815f8d24f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:47.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.691+0000 7fd8a77fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:47.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8a77fe700 1 -- 172.21.15.84:0/577819940 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8bc035060 con 0x56327d2f4ac0 2024-04-26T18:40:47.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8a77fe700 1 -- 172.21.15.84:0/577819940 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7fd8bc005250 con 0x56327d2f4ac0 2024-04-26T18:40:47.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8d915bb80 1 -- 172.21.15.84:0/577819940 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd8bc014070 con 0x56327d2f4ac0 2024-04-26T18:40:47.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8a77fe700 1 -- 172.21.15.84:0/577819940 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd8bc005450 con 0x56327d2f4ac0 2024-04-26T18:40:47.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8d915bb80 10 client.16953 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:47.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8d915bb80 1 -- 172.21.15.84:0/577819940 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56327d379cd0 con 0x56327d2f4ac0 2024-04-26T18:40:47.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8a77fe700 1 -- 172.21.15.84:0/577819940 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8bc014070 con 0x56327d2f4ac0 2024-04-26T18:40:47.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8a77fe700 1 -- 172.21.15.84:0/577819940 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd8bc019040 con 0x56327d2f4ac0 2024-04-26T18:40:47.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8d915bb80 10 client.16953 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:47.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8d915bb80 10 client.16953 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:40:47.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8d915bb80 20 client.16953 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:47.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56327d37a360 0x56327d37c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:47.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8d915bb80 4 client.16953 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "2a546377", "release": true, "await": true}] 2024-04-26T18:40:47.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8d915bb80 1 -- 172.21.15.84:0/577819940 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "set_id": "2a546377", "release": true, "await": true}) -- 0x56327d24f2c0 con 0x56327d37a360 2024-04-26T18:40:47.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.692+0000 7fd8c7fff700 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56327d37a360 0x56327d37c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:47.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.693+0000 7fd8a77fe700 10 client.16953 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:47.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.693+0000 7fd8c7fff700 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56327d37a360 0x56327d37c780 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec64e2aff52d5b7e server_cookie=507f1a854f515442 in_seq=0 out_seq=0 2024-04-26T18:40:47.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8a77fe700 1 -- 172.21.15.84:0/577819940 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56327d24f2c0 con 0x56327d37a360 2024-04-26T18:40:47.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8a77fe700 10 client.16953 handle_command_reply: tid=0 2024-04-26T18:40:47.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8d915bb80 1 client.16953 shutdown 2024-04-26T18:40:47.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8d915bb80 1 -- 172.21.15.84:0/577819940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 msgr2=0x56327d3072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:47.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 0x56327d3072c0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fd8bc00a470 tx=0x7fd8bc004190 comp rx=0 tx=0).stop 2024-04-26T18:40:47.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8d915bb80 1 -- 172.21.15.84:0/577819940 shutdown_connections 2024-04-26T18:40:47.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56327d37a360 0x56327d37c780 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56327d303d80 0x56327d307800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56327d2f41a0 0x56327d30be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8d915bb80 1 --2- 172.21.15.84:0/577819940 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56327d2f4ac0 0x56327d3072c0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8d915bb80 1 -- 172.21.15.84:0/577819940 >> 172.21.15.84:0/577819940 conn(0x56327d221e40 msgr2=0x56327d2f6930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:47.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8a77fe700 0 client.16953 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:47.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8d915bb80 1 -- 172.21.15.84:0/577819940 shutdown_connections 2024-04-26T18:40:47.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.697+0000 7fd8d915bb80 1 -- 172.21.15.84:0/577819940 wait complete. 2024-04-26T18:40:47.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.698+0000 7fd8d915bb80 20 client.16953 trim_cache size 0 max 16384 2024-04-26T18:40:47.701 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:40:47.702 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:40:47.702 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:40:47.702 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 36, 2024-04-26T18:40:47.702 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:40:47.702 INFO:teuthology.orchestra.run.smithi084.stdout: "2a546377": { 2024-04-26T18:40:47.702 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 36, 2024-04-26T18:40:47.702 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:40:47.702 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:40:47.702 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:40:47.702 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:40:47.702 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:40:47.702 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:40:47.702 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 86.4, 2024-04-26T18:40:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:40:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:40:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:40:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:40:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:40:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:40:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:40:47.703 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:40:47.704 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:40:47.704 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:40:47.704 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:40:47.704 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:40:47.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.698+0000 7fd8ce8c9700 1 -- 172.21.15.84:0/967575851 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd8b00a3db0 msgr2=0x7fd8b00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:47.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.698+0000 7fd8ce8c9700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd8b00a3db0 0x7fd8b00a6270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd8c807aa80 tx=0x7fd8bc00a380 comp rx=0 tx=0).stop 2024-04-26T18:40:47.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.698+0000 7fd8ce8c9700 1 -- 172.21.15.84:0/967575851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c81034c0 msgr2=0x7fd8c819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:47.704 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.698+0000 7fd8ce8c9700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c81034c0 0x7fd8c819f7a0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fd8b800b530 tx=0x7fd8b800b840 comp rx=0 tx=0).stop 2024-04-26T18:40:47.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.698+0000 7fd8ce8c9700 1 -- 172.21.15.84:0/967575851 shutdown_connections 2024-04-26T18:40:47.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.698+0000 7fd8ce8c9700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd8b00a3db0 0x7fd8b00a6270 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.698+0000 7fd8ce8c9700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd8c81034c0 0x7fd8c819f7a0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.698+0000 7fd8ce8c9700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd8c8109560 0x7fd8c8199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.698+0000 7fd8ce8c9700 1 --2- 172.21.15.84:0/967575851 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd8c8103e80 0x7fd8c819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:47.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.698+0000 7fd8ce8c9700 1 -- 172.21.15.84:0/967575851 >> 172.21.15.84:0/967575851 conn(0x7fd8c807a5d0 msgr2=0x7fd8c8101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:47.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.698+0000 7fd8ce8c9700 1 -- 172.21.15.84:0/967575851 shutdown_connections 2024-04-26T18:40:47.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:40:47.698+0000 7fd8ce8c9700 1 -- 172.21.15.84:0/967575851 wait complete. 2024-04-26T18:40:47.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:47 smithi103 ceph-mon[32727]: pgmap v401: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-26T18:40:47.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:47 smithi174 ceph-mon[32705]: pgmap v401: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-26T18:40:48.353 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '2a546377', seconds elapsed: 37.9 2024-04-26T18:40:48.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:48 smithi084 ceph-mon[29141]: pgmap v402: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-26T18:40:48.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:48 smithi103 ceph-mon[32727]: pgmap v402: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-26T18:40:48.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:48 smithi174 ceph-mon[32705]: pgmap v402: 97 pgs: 97 active+clean; 4.5 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-26T18:40:51.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:50 smithi084 ceph-mon[29141]: pgmap v403: 97 pgs: 97 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 14 op/s 2024-04-26T18:40:51.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:50 smithi103 ceph-mon[32727]: pgmap v403: 97 pgs: 97 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 14 op/s 2024-04-26T18:40:51.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:50 smithi174 ceph-mon[32705]: pgmap v403: 97 pgs: 97 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.6 KiB/s wr, 14 op/s 2024-04-26T18:40:53.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:52 smithi084 ceph-mon[29141]: pgmap v404: 97 pgs: 97 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-04-26T18:40:53.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:52 smithi103 ceph-mon[32727]: pgmap v404: 97 pgs: 97 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-04-26T18:40:53.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:52 smithi174 ceph-mon[32705]: pgmap v404: 97 pgs: 97 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 7.6 KiB/s wr, 10 op/s 2024-04-26T18:40:55.055 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:54 smithi084 ceph-mon[29141]: pgmap v405: 97 pgs: 97 active+clean; 4.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-26T18:40:55.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:54 smithi103 ceph-mon[32727]: pgmap v405: 97 pgs: 97 active+clean; 4.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-26T18:40:55.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:54 smithi174 ceph-mon[32705]: pgmap v405: 97 pgs: 97 active+clean; 4.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 25 KiB/s wr, 17 op/s 2024-04-26T18:40:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.474+0000 7f5992df7640 1 -- 172.21.15.84:0/2058321835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f598c10cf20 msgr2=0x7f598c10f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.474+0000 7f5992df7640 1 --2- 172.21.15.84:0/2058321835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f598c10cf20 0x7f598c10f390 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f5984009ca0 tx=0x7f598402a950 comp rx=0 tx=0).stop 2024-04-26T18:40:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.474+0000 7f5992df7640 1 -- 172.21.15.84:0/2058321835 shutdown_connections 2024-04-26T18:40:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.474+0000 7f5992df7640 1 --2- 172.21.15.84:0/2058321835 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f598c10cf20 0x7f598c10f390 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.474+0000 7f5992df7640 1 --2- 172.21.15.84:0/2058321835 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f598c104500 0x7f598c10c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.474+0000 7f5992df7640 1 --2- 172.21.15.84:0/2058321835 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f598c103be0 0x7f598c103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.474+0000 7f5992df7640 1 -- 172.21.15.84:0/2058321835 >> 172.21.15.84:0/2058321835 conn(0x7f598c0789b0 msgr2=0x7f598c078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.474+0000 7f5992df7640 1 -- 172.21.15.84:0/2058321835 shutdown_connections 2024-04-26T18:40:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.474+0000 7f5992df7640 1 -- 172.21.15.84:0/2058321835 wait complete. 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f5992df7640 1 Processor -- start 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f5992df7640 1 -- start start 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f5992df7640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f598c103be0 0x7f598c199040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f5992df7640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f598c104500 0x7f598c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f5992df7640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f598c10cf20 0x7f598c19d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f5992df7640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f598c112be0 con 0x7f598c104500 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f5992df7640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f598c112a60 con 0x7f598c10cf20 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f5992df7640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f598c112d60 con 0x7f598c103be0 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f598bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f598c104500 0x7f598c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f598bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f598c104500 0x7f598c199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57590/0 (socket says 172.21.15.84:57590) 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f598bfff640 1 -- 172.21.15.84:0/3876286107 learned_addr learned my addr 172.21.15.84:0/3876286107 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f5990b93640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f598c103be0 0x7f598c199040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f5991394640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f598c10cf20 0x7f598c19d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f598bfff640 1 -- 172.21.15.84:0/3876286107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f598c103be0 msgr2=0x7f598c199040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f598bfff640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f598c103be0 0x7f598c199040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f598bfff640 1 -- 172.21.15.84:0/3876286107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f598c10cf20 msgr2=0x7f598c19d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f598bfff640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f598c10cf20 0x7f598c19d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.475+0000 7f598bfff640 1 -- 172.21.15.84:0/3876286107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f598c19e030 con 0x7f598c104500 2024-04-26T18:40:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.476+0000 7f598bfff640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f598c104500 0x7f598c199580 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f597800b4a0 tx=0x7f597800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.476+0000 7f5989ffb640 1 -- 172.21.15.84:0/3876286107 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5978014070 con 0x7f598c104500 2024-04-26T18:40:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.476+0000 7f5989ffb640 1 -- 172.21.15.84:0/3876286107 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5978010ac0 con 0x7f598c104500 2024-04-26T18:40:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.476+0000 7f5992df7640 1 -- 172.21.15.84:0/3876286107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f598c19e320 con 0x7f598c104500 2024-04-26T18:40:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.476+0000 7f5989ffb640 1 -- 172.21.15.84:0/3876286107 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5978017c30 con 0x7f598c104500 2024-04-26T18:40:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.476+0000 7f5992df7640 1 -- 172.21.15.84:0/3876286107 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f598c1a5bb0 con 0x7f598c104500 2024-04-26T18:40:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.478+0000 7f5989ffb640 1 -- 172.21.15.84:0/3876286107 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5978017070 con 0x7f598c104500 2024-04-26T18:40:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.478+0000 7f5995bec740 1 Processor -- start 2024-04-26T18:40:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.478+0000 7f5995bec740 1 -- start start 2024-04-26T18:40:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5995bec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 0x55e60fbd42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5995bec740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbd4820 0x55e60fbdeba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5995bec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbdf0e0 0x55e60fbe14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5995bec740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e60fb3a460 con 0x55e60fbd4820 2024-04-26T18:40:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5995bec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e60faf4590 con 0x55e60fbdf0e0 2024-04-26T18:40:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5995bec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e60fb36420 con 0x55e60fbd3f00 2024-04-26T18:40:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5989ffb640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f597c0a3e40 0x7f597c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5990b93640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbd4820 0x55e60fbdeba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5990b93640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbd4820 0x55e60fbdeba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57602/0 (socket says 172.21.15.84:57602) 2024-04-26T18:40:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5990b93640 1 -- 172.21.15.84:0/2055456560 learned_addr learned my addr 172.21.15.84:0/2055456560 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5991394640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f597c0a3e40 0x7f597c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5989ffb640 1 -- 172.21.15.84:0/3876286107 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f59780cfec0 con 0x7f598c104500 2024-04-26T18:40:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f598bfff640 1 --2- 172.21.15.84:0/2055456560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbdf0e0 0x55e60fbe14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5990b93640 1 -- 172.21.15.84:0/2055456560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 msgr2=0x55e60fbd42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5990b93640 1 --2- 172.21.15.84:0/2055456560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 0x55e60fbd42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5990b93640 1 -- 172.21.15.84:0/2055456560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbdf0e0 msgr2=0x55e60fbe14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5990b93640 1 --2- 172.21.15.84:0/2055456560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbdf0e0 0x55e60fbe14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5990b93640 1 -- 172.21.15.84:0/2055456560 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60fb010d0 con 0x55e60fbd4820 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f598bfff640 1 --2- 172.21.15.84:0/2055456560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbdf0e0 0x55e60fbe14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5990b93640 1 --2- 172.21.15.84:0/2055456560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbd4820 0x55e60fbdeba0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f598000cc60 tx=0x7f598000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfe99789423c5ff server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5991394640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f597c0a3e40 0x7f597c0a6300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5984005e30 tx=0x7f5984005d80 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5976ffd640 1 -- 172.21.15.84:0/2055456560 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980014070 con 0x55e60fbd4820 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5976ffd640 1 -- 172.21.15.84:0/2055456560 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59800047a0 con 0x55e60fbd4820 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.479+0000 7f5976ffd640 1 -- 172.21.15.84:0/2055456560 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5980010690 con 0x55e60fbd4820 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.480+0000 7f5995bec740 1 -- 172.21.15.84:0/2055456560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbd4820 msgr2=0x55e60fbdeba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.480+0000 7f5995bec740 1 --2- 172.21.15.84:0/2055456560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbd4820 0x55e60fbdeba0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f598000cc60 tx=0x7f598000e720 comp rx=0 tx=0).stop 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.480+0000 7f5995bec740 1 -- 172.21.15.84:0/2055456560 shutdown_connections 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.480+0000 7f5995bec740 1 --2- 172.21.15.84:0/2055456560 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbdf0e0 0x55e60fbe14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.480+0000 7f5995bec740 1 --2- 172.21.15.84:0/2055456560 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbd4820 0x55e60fbdeba0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.480+0000 7f5995bec740 1 --2- 172.21.15.84:0/2055456560 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 0x55e60fbd42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.480+0000 7f5995bec740 1 -- 172.21.15.84:0/2055456560 >> 172.21.15.84:0/2055456560 conn(0x55e60fbd0410 msgr2=0x55e60fbd07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.480+0000 7f5995bec740 1 -- 172.21.15.84:0/2055456560 shutdown_connections 2024-04-26T18:40:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.480+0000 7f5995bec740 1 -- 172.21.15.84:0/2055456560 wait complete. 2024-04-26T18:40:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.480+0000 7f5995bec740 1 -- start start 2024-04-26T18:40:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5995bec740 1 Processor -- start 2024-04-26T18:40:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5995bec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 0x55e60fbdc070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5995bec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd4820 0x55e60fbdc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5995bec740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbdcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5995bec740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55e60fbe1d10 con 0x55e60fbdf0e0 2024-04-26T18:40:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5995bec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55e60fac1b60 con 0x55e60fbd4820 2024-04-26T18:40:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5995bec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55e60fbe1e90 con 0x55e60fbd3f00 2024-04-26T18:40:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5995bec740 10 client.0 _ll_register_callbacks cb 0x55e60fae62b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5991394640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbdcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5991394640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbdcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57612/0 (socket says 172.21.15.84:57612) 2024-04-26T18:40:55.489 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5991394640 1 -- 172.21.15.84:0/2168697952 learned_addr learned my addr 172.21.15.84:0/2168697952 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5990b93640 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 0x55e60fbdc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5991394640 1 -- 172.21.15.84:0/2168697952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 msgr2=0x55e60fbdc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f598bfff640 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd4820 0x55e60fbdc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5991394640 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 0x55e60fbdc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5991394640 1 -- 172.21.15.84:0/2168697952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd4820 msgr2=0x55e60fbdc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5991394640 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd4820 0x55e60fbdc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5991394640 1 -- 172.21.15.84:0/2168697952 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e60f84c210 con 0x55e60fbdf0e0 2024-04-26T18:40:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.481+0000 7f5990b93640 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 0x55e60fbdc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f5991394640 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbdcaf0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f598402f630 tx=0x7f598402a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e280d7db67dce32 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f596bfff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f596bfff640 1 -- 172.21.15.84:0/2168697952 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5984034040 con 0x55e60fbdf0e0 2024-04-26T18:40:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f596bfff640 1 -- 172.21.15.84:0/2168697952 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f5984004040 con 0x55e60fbdf0e0 2024-04-26T18:40:55.490 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f5995bec740 1 -- 172.21.15.84:0/2168697952 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59800047a0 con 0x55e60fbdf0e0 2024-04-26T18:40:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f596bfff640 1 -- 172.21.15.84:0/2168697952 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5984003e40 con 0x55e60fbdf0e0 2024-04-26T18:40:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f5995bec740 10 client.16965 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f5995bec740 1 -- 172.21.15.84:0/2168697952 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e60f918620 con 0x55e60fbdf0e0 2024-04-26T18:40:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f596bfff640 1 -- 172.21.15.84:0/2168697952 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59800047a0 con 0x55e60fbdf0e0 2024-04-26T18:40:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f596bfff640 1 -- 172.21.15.84:0/2168697952 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f598404fa80 con 0x55e60fbdf0e0 2024-04-26T18:40:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f5995bec740 10 client.16965 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f5995bec740 10 client.16965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:40:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f5995bec740 20 client.16965 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f5995bec740 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e60fbef190 0x55e60fbf15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f5995bec740 4 client.16965 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:40:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f5995bec740 1 -- 172.21.15.84:0/2168697952 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e60fb48070 con 0x55e60fbef190 2024-04-26T18:40:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.482+0000 7f5990b93640 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e60fbef190 0x55e60fbf15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.491 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.483+0000 7f5990b93640 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e60fbef190 0x55e60fbf15c0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f767e093abf676b2 server_cookie=253fdd2b72a3da95 in_seq=0 out_seq=0 2024-04-26T18:40:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.483+0000 7f596bfff640 10 client.16965 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f596bfff640 1 -- 172.21.15.84:0/2168697952 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e60fb48070 con 0x55e60fbef190 2024-04-26T18:40:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f596bfff640 10 client.16965 handle_command_reply: tid=0 2024-04-26T18:40:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f5995bec740 1 client.16965 shutdown 2024-04-26T18:40:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f5995bec740 1 -- 172.21.15.84:0/2168697952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 msgr2=0x55e60fbdcaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f5995bec740 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbdcaf0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f598402f630 tx=0x7f598402a950 comp rx=0 tx=0).stop 2024-04-26T18:40:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f5995bec740 1 -- 172.21.15.84:0/2168697952 shutdown_connections 2024-04-26T18:40:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f5995bec740 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e60fbef190 0x55e60fbf15c0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f5995bec740 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbdcaf0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f5995bec740 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd4820 0x55e60fbdc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f5995bec740 1 --2- 172.21.15.84:0/2168697952 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 0x55e60fbdc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.492 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f596bfff640 0 client.16965 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f5995bec740 1 -- 172.21.15.84:0/2168697952 >> 172.21.15.84:0/2168697952 conn(0x55e60fbd0410 msgr2=0x55e60fbd6140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f5995bec740 1 -- 172.21.15.84:0/2168697952 shutdown_connections 2024-04-26T18:40:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f5995bec740 1 -- 172.21.15.84:0/2168697952 wait complete. 2024-04-26T18:40:55.493 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.484+0000 7f5995bec740 20 client.16965 trim_cache size 0 max 16384 2024-04-26T18:40:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.498+0000 7f5995bec740 1 Processor -- start 2024-04-26T18:40:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.498+0000 7f5995bec740 1 -- start start 2024-04-26T18:40:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.498+0000 7f5995bec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 0x55e60fbeabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.498+0000 7f5995bec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd4820 0x55e60fbe74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.498+0000 7f5995bec740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbf6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.498+0000 7f5995bec740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f598c1ae500 con 0x55e60fbdf0e0 2024-04-26T18:40:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.498+0000 7f5995bec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f598c1b0040 con 0x55e60fbd4820 2024-04-26T18:40:55.504 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.498+0000 7f5995bec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f598c1aebd0 con 0x55e60fbd3f00 2024-04-26T18:40:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f5991394640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbf6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f5991394640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbf6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57620/0 (socket says 172.21.15.84:57620) 2024-04-26T18:40:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f5991394640 1 -- 172.21.15.84:0/2691780689 learned_addr learned my addr 172.21.15.84:0/2691780689 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f598bfff640 1 --2- 172.21.15.84:0/2691780689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd4820 0x55e60fbe74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f5991394640 1 -- 172.21.15.84:0/2691780689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 msgr2=0x55e60fbeabd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f5990b93640 1 --2- 172.21.15.84:0/2691780689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 0x55e60fbeabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f5991394640 1 --2- 172.21.15.84:0/2691780689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 0x55e60fbeabd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f5991394640 1 -- 172.21.15.84:0/2691780689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd4820 msgr2=0x55e60fbe74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f5991394640 1 --2- 172.21.15.84:0/2691780689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd4820 0x55e60fbe74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f5991394640 1 -- 172.21.15.84:0/2691780689 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e60fbd4650 con 0x55e60fbdf0e0 2024-04-26T18:40:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f598bfff640 1 --2- 172.21.15.84:0/2691780689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd4820 0x55e60fbe74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:55.505 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f5990b93640 1 --2- 172.21.15.84:0/2691780689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 0x55e60fbeabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:40:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f5991394640 1 --2- 172.21.15.84:0/2691780689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbf6ed0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f59840096a0 tx=0x7f5984003dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8750f5458aeb49b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f59767fc640 1 -- 172.21.15.84:0/2691780689 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598405b070 con 0x55e60fbdf0e0 2024-04-26T18:40:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f59767fc640 1 -- 172.21.15.84:0/2691780689 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f598404a070 con 0x55e60fbdf0e0 2024-04-26T18:40:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.499+0000 7f59767fc640 1 -- 172.21.15.84:0/2691780689 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598404fb60 con 0x55e60fbdf0e0 2024-04-26T18:40:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.500+0000 7f5995bec740 1 -- 172.21.15.84:0/2691780689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 msgr2=0x55e60fbf6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.500+0000 7f5995bec740 1 --2- 172.21.15.84:0/2691780689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbf6ed0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f59840096a0 tx=0x7f5984003dd0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.500+0000 7f5995bec740 1 -- 172.21.15.84:0/2691780689 shutdown_connections 2024-04-26T18:40:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.500+0000 7f5995bec740 1 --2- 172.21.15.84:0/2691780689 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbf6ed0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.500+0000 7f5995bec740 1 --2- 172.21.15.84:0/2691780689 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd4820 0x55e60fbe74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.500+0000 7f5995bec740 1 --2- 172.21.15.84:0/2691780689 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd3f00 0x55e60fbeabd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.500+0000 7f5995bec740 1 -- 172.21.15.84:0/2691780689 >> 172.21.15.84:0/2691780689 conn(0x55e60fbd0410 msgr2=0x55e60fbd07f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.500+0000 7f5995bec740 1 -- 172.21.15.84:0/2691780689 shutdown_connections 2024-04-26T18:40:55.506 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.500+0000 7f5995bec740 1 -- 172.21.15.84:0/2691780689 wait complete. 2024-04-26T18:40:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.500+0000 7f5995bec740 1 -- start start 2024-04-26T18:40:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.500+0000 7f5995bec740 1 Processor -- start 2024-04-26T18:40:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5995bec740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd3f00 0x55e60fbd9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5995bec740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd4820 0x55e60fbd9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5995bec740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbd8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5995bec740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f598c1af970 con 0x55e60fbdf0e0 2024-04-26T18:40:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5995bec740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f598c1af2a0 con 0x55e60fbd3f00 2024-04-26T18:40:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5995bec740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f598c1ac500 con 0x55e60fbd4820 2024-04-26T18:40:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5995bec740 10 client.0 _ll_register_callbacks cb 0x7f5984009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:40:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5991394640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbd8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5991394640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbd8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:57634/0 (socket says 172.21.15.84:57634) 2024-04-26T18:40:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5991394640 1 -- 172.21.15.84:0/1123987724 learned_addr learned my addr 172.21.15.84:0/1123987724 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:40:55.507 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f598bfff640 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd4820 0x55e60fbd9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5990b93640 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd3f00 0x55e60fbd9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5991394640 1 -- 172.21.15.84:0/1123987724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd4820 msgr2=0x55e60fbd9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5991394640 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd4820 0x55e60fbd9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5991394640 1 -- 172.21.15.84:0/1123987724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd3f00 msgr2=0x55e60fbd9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:40:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5991394640 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd3f00 0x55e60fbd9600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5991394640 1 -- 172.21.15.84:0/1123987724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e60fbd89f0 con 0x55e60fbdf0e0 2024-04-26T18:40:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f598bfff640 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd4820 0x55e60fbd9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:40:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5991394640 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbd8300 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f598402ff70 tx=0x7f598404f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a656fe4a994858d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:40:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f596bfff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:40:55.508 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f596bfff640 1 -- 172.21.15.84:0/1123987724 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598405b070 con 0x55e60fbdf0e0 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f596bfff640 1 -- 172.21.15.84:0/1123987724 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 8579+0+0 (secure 0 0 0) 0x7f5984030930 con 0x55e60fbdf0e0 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5995bec740 1 -- 172.21.15.84:0/1123987724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f598404a070 con 0x55e60fbdf0e0 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f596bfff640 1 -- 172.21.15.84:0/1123987724 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5984030b70 con 0x55e60fbdf0e0 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5995bec740 10 client.16971 fetch_fsmap learned FSMap version 29 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.501+0000 7f5995bec740 1 -- 172.21.15.84:0/1123987724 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e60fc5a0e0 con 0x55e60fbdf0e0 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.502+0000 7f596bfff640 1 -- 172.21.15.84:0/1123987724 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f598404a070 con 0x55e60fbdf0e0 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.502+0000 7f596bfff640 1 -- 172.21.15.84:0/1123987724 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5984065030 con 0x55e60fbdf0e0 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.502+0000 7f5995bec740 10 client.16971 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.502+0000 7f5995bec740 10 client.16971 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.502+0000 7f5995bec740 20 client.16971 populate_metadata read hostname 'smithi084' 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.502+0000 7f5995bec740 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e60fc5a6a0 0x55e60fc5cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.502+0000 7f5995bec740 4 client.16971 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.502+0000 7f5995bec740 1 -- 172.21.15.84:0/1123987724 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e60fb41ee0 con 0x55e60fc5a6a0 2024-04-26T18:40:55.509 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.502+0000 7f5990b93640 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e60fc5a6a0 0x55e60fc5cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:40:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.502+0000 7f5990b93640 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e60fc5a6a0 0x55e60fc5cac0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da9ff1258aa55615 server_cookie=8ce6056def4737de in_seq=0 out_seq=0 2024-04-26T18:40:55.510 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.502+0000 7f596bfff640 10 client.16971 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.515+0000 7f596bfff640 1 -- 172.21.15.84:0/1123987724 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e60fb41ee0 con 0x55e60fc5a6a0 2024-04-26T18:40:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.515+0000 7f596bfff640 10 client.16971 handle_command_reply: tid=0 2024-04-26T18:40:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.515+0000 7f5995bec740 1 client.16971 shutdown 2024-04-26T18:40:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.515+0000 7f5995bec740 1 -- 172.21.15.84:0/1123987724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 msgr2=0x55e60fbd8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.515+0000 7f5995bec740 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbd8300 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f598402ff70 tx=0x7f598404f470 comp rx=0 tx=0).stop 2024-04-26T18:40:55.516 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.515+0000 7f5995bec740 1 -- 172.21.15.84:0/1123987724 shutdown_connections 2024-04-26T18:40:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.515+0000 7f5995bec740 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55e60fc5a6a0 0x55e60fc5cac0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.515+0000 7f5995bec740 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55e60fbdf0e0 0x55e60fbd8300 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.515+0000 7f5995bec740 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55e60fbd4820 0x55e60fbd9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.515+0000 7f596bfff640 0 client.16971 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:40:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.515+0000 7f5995bec740 1 --2- 172.21.15.84:0/1123987724 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55e60fbd3f00 0x55e60fbd9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.515+0000 7f5995bec740 1 -- 172.21.15.84:0/1123987724 >> 172.21.15.84:0/1123987724 conn(0x55e60fbd0410 msgr2=0x55e60fbd58b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5995bec740 1 -- 172.21.15.84:0/1123987724 shutdown_connections 2024-04-26T18:40:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5995bec740 1 -- 172.21.15.84:0/1123987724 wait complete. 2024-04-26T18:40:55.517 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5995bec740 20 client.16971 trim_cache size 0 max 16384 2024-04-26T18:40:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:40:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:40:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:40:55.518 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:40:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5992df7640 1 -- 172.21.15.84:0/3876286107 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f597c0a3e40 msgr2=0x7f597c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5992df7640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f597c0a3e40 0x7f597c0a6300 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5984005e30 tx=0x7f5984005d80 comp rx=0 tx=0).stop 2024-04-26T18:40:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5992df7640 1 -- 172.21.15.84:0/3876286107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f598c104500 msgr2=0x7f598c199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:40:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5992df7640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f598c104500 0x7f598c199580 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f597800b4a0 tx=0x7f597800b960 comp rx=0 tx=0).stop 2024-04-26T18:40:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5992df7640 1 -- 172.21.15.84:0/3876286107 shutdown_connections 2024-04-26T18:40:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5992df7640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f597c0a3e40 0x7f597c0a6300 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5992df7640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f598c10cf20 0x7f598c19d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.519 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5992df7640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f598c104500 0x7f598c199580 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5992df7640 1 --2- 172.21.15.84:0/3876286107 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f598c103be0 0x7f598c199040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:40:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5992df7640 1 -- 172.21.15.84:0/3876286107 >> 172.21.15.84:0/3876286107 conn(0x7f598c0789b0 msgr2=0x7f598c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:40:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5992df7640 1 -- 172.21.15.84:0/3876286107 shutdown_connections 2024-04-26T18:40:55.520 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:40:55.516+0000 7f5992df7640 1 -- 172.21.15.84:0/3876286107 wait complete. 2024-04-26T18:40:57.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:56 smithi084 ceph-mon[29141]: pgmap v406: 97 pgs: 97 active+clean; 4.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-04-26T18:40:57.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:56 smithi103 ceph-mon[32727]: pgmap v406: 97 pgs: 97 active+clean; 4.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-04-26T18:40:57.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:56 smithi174 ceph-mon[32705]: pgmap v406: 97 pgs: 97 active+clean; 4.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-04-26T18:40:59.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:40:58 smithi084 ceph-mon[29141]: pgmap v407: 97 pgs: 97 active+clean; 4.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-04-26T18:40:59.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:40:58 smithi103 ceph-mon[32727]: pgmap v407: 97 pgs: 97 active+clean; 4.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-04-26T18:40:59.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:40:58 smithi174 ceph-mon[32705]: pgmap v407: 97 pgs: 97 active+clean; 4.7 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-04-26T18:41:01.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:00 smithi084 ceph-mon[29141]: pgmap v408: 97 pgs: 97 active+clean; 5.0 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-26T18:41:01.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:00 smithi103 ceph-mon[32727]: pgmap v408: 97 pgs: 97 active+clean; 5.0 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-26T18:41:01.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:00 smithi174 ceph-mon[32705]: pgmap v408: 97 pgs: 97 active+clean; 5.0 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-26T18:41:03.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:02 smithi084 ceph-mon[29141]: pgmap v409: 97 pgs: 97 active+clean; 5.0 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:41:03.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:02 smithi103 ceph-mon[32727]: pgmap v409: 97 pgs: 97 active+clean; 5.0 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:41:03.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:02 smithi174 ceph-mon[32705]: pgmap v409: 97 pgs: 97 active+clean; 5.0 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:41:05.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:04 smithi084 ceph-mon[29141]: pgmap v410: 97 pgs: 97 active+clean; 5.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-26T18:41:05.097 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:04 smithi084 ceph-mon[29141]: osdmap e109: 12 total, 12 up, 12 in 2024-04-26T18:41:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:04 smithi103 ceph-mon[32727]: pgmap v410: 97 pgs: 97 active+clean; 5.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-26T18:41:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:04 smithi103 ceph-mon[32727]: osdmap e109: 12 total, 12 up, 12 in 2024-04-26T18:41:05.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:04 smithi174 ceph-mon[32705]: pgmap v410: 97 pgs: 97 active+clean; 5.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-26T18:41:05.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:04 smithi174 ceph-mon[32705]: osdmap e109: 12 total, 12 up, 12 in 2024-04-26T18:41:06.878 INFO:tasks.workunit.client.0.smithi084.stdout:375/600: open fd = 3 2024-04-26T18:41:06.879 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.879 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.879 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.879 INFO:tasks.workunit.client.0.smithi084.stdout:376/600: open fd = 3 2024-04-26T18:41:06.879 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.879 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.880 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.880 INFO:tasks.workunit.client.0.smithi084.stdout:377/600: open fd = 3 2024-04-26T18:41:06.880 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.880 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.880 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.880 INFO:tasks.workunit.client.0.smithi084.stdout:378/600: open fd = 3 2024-04-26T18:41:06.881 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.881 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.881 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.881 INFO:tasks.workunit.client.0.smithi084.stdout:379/600: open fd = 3 2024-04-26T18:41:06.881 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.881 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.881 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.882 INFO:tasks.workunit.client.0.smithi084.stdout:380/600: open fd = 3 2024-04-26T18:41:06.882 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.882 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.882 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.882 INFO:tasks.workunit.client.0.smithi084.stdout:381/600: open fd = 3 2024-04-26T18:41:06.882 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.882 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.883 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.883 INFO:tasks.workunit.client.0.smithi084.stdout:382/600: open fd = 3 2024-04-26T18:41:06.883 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.883 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.883 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.883 INFO:tasks.workunit.client.0.smithi084.stdout:383/600: open fd = 3 2024-04-26T18:41:06.883 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.884 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.884 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.884 INFO:tasks.workunit.client.0.smithi084.stdout:384/600: open fd = 3 2024-04-26T18:41:06.884 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.884 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.884 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.884 INFO:tasks.workunit.client.0.smithi084.stdout:385/600: open fd = 3 2024-04-26T18:41:06.885 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.885 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.885 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.885 INFO:tasks.workunit.client.0.smithi084.stdout:386/600: open fd = 3 2024-04-26T18:41:06.885 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.885 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.886 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.886 INFO:tasks.workunit.client.0.smithi084.stdout:387/600: open fd = 3 2024-04-26T18:41:06.886 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.886 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.886 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.886 INFO:tasks.workunit.client.0.smithi084.stdout:388/600: open fd = 3 2024-04-26T18:41:06.886 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.886 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.886 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.886 INFO:tasks.workunit.client.0.smithi084.stdout:389/600: open fd = 3 2024-04-26T18:41:06.887 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.887 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.887 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.887 INFO:tasks.workunit.client.0.smithi084.stdout:390/600: open fd = 3 2024-04-26T18:41:06.887 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.887 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.887 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.887 INFO:tasks.workunit.client.0.smithi084.stdout:391/600: open fd = 3 2024-04-26T18:41:06.887 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.887 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.887 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.887 INFO:tasks.workunit.client.0.smithi084.stdout:392/600: open fd = 3 2024-04-26T18:41:06.887 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.888 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.888 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.888 INFO:tasks.workunit.client.0.smithi084.stdout:393/600: open fd = 3 2024-04-26T18:41:06.888 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.888 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.888 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.888 INFO:tasks.workunit.client.0.smithi084.stdout:394/600: open fd = 3 2024-04-26T18:41:06.889 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.889 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.889 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.889 INFO:tasks.workunit.client.0.smithi084.stdout:395/600: open fd = 3 2024-04-26T18:41:06.889 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.889 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.890 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.890 INFO:tasks.workunit.client.0.smithi084.stdout:396/600: open fd = 3 2024-04-26T18:41:06.890 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.890 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.890 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.890 INFO:tasks.workunit.client.0.smithi084.stdout:397/600: open fd = 3 2024-04-26T18:41:06.891 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.891 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.891 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.891 INFO:tasks.workunit.client.0.smithi084.stdout:398/600: open fd = 3 2024-04-26T18:41:06.891 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.891 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.892 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.892 INFO:tasks.workunit.client.0.smithi084.stdout:399/600: open fd = 3 2024-04-26T18:41:06.892 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.892 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.892 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.892 INFO:tasks.workunit.client.0.smithi084.stdout:400/600: open fd = 3 2024-04-26T18:41:06.892 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.893 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.893 INFO:tasks.workunit.client.0.smithi084.stdout:401/600: open fd = 3 2024-04-26T18:41:06.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.893 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.893 INFO:tasks.workunit.client.0.smithi084.stdout:402/600: open fd = 3 2024-04-26T18:41:06.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.893 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.893 INFO:tasks.workunit.client.0.smithi084.stdout:403/600: open fd = 3 2024-04-26T18:41:06.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.893 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:404/600: open fd = 3 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:405/600: open fd = 3 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:406/600: open fd = 3 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.894 INFO:tasks.workunit.client.0.smithi084.stdout:407/600: open fd = 3 2024-04-26T18:41:06.895 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.895 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.895 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.895 INFO:tasks.workunit.client.0.smithi084.stdout:408/600: open fd = 3 2024-04-26T18:41:06.895 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.895 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.895 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.896 INFO:tasks.workunit.client.0.smithi084.stdout:409/600: open fd = 3 2024-04-26T18:41:06.896 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.896 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.896 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.896 INFO:tasks.workunit.client.0.smithi084.stdout:410/600: open fd = 3 2024-04-26T18:41:06.896 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.897 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.897 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.897 INFO:tasks.workunit.client.0.smithi084.stdout:411/600: open fd = 3 2024-04-26T18:41:06.897 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.897 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.897 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.897 INFO:tasks.workunit.client.0.smithi084.stdout:412/600: open fd = 3 2024-04-26T18:41:06.898 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.898 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.898 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.898 INFO:tasks.workunit.client.0.smithi084.stdout:413/600: open fd = 3 2024-04-26T18:41:06.898 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.898 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.898 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.898 INFO:tasks.workunit.client.0.smithi084.stdout:414/600: open fd = 3 2024-04-26T18:41:06.898 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.899 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.899 INFO:tasks.workunit.client.0.smithi084.stdout:415/600: open fd = 3 2024-04-26T18:41:06.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.899 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.899 INFO:tasks.workunit.client.0.smithi084.stdout:416/600: open fd = 3 2024-04-26T18:41:06.899 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.900 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.900 INFO:tasks.workunit.client.0.smithi084.stdout:417/600: open fd = 3 2024-04-26T18:41:06.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.900 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.900 INFO:tasks.workunit.client.0.smithi084.stdout:418/600: open fd = 3 2024-04-26T18:41:06.900 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.901 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.901 INFO:tasks.workunit.client.0.smithi084.stdout:419/600: open fd = 3 2024-04-26T18:41:06.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.901 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.901 INFO:tasks.workunit.client.0.smithi084.stdout:420/600: open fd = 3 2024-04-26T18:41:06.901 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.902 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.902 INFO:tasks.workunit.client.0.smithi084.stdout:421/600: open fd = 3 2024-04-26T18:41:06.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.902 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.902 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.903 INFO:tasks.workunit.client.0.smithi084.stdout:422/600: open fd = 3 2024-04-26T18:41:06.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.903 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.903 INFO:tasks.workunit.client.0.smithi084.stdout:423/600: open fd = 3 2024-04-26T18:41:06.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.903 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.904 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.904 INFO:tasks.workunit.client.0.smithi084.stdout:424/600: open fd = 3 2024-04-26T18:41:06.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.904 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.904 INFO:tasks.workunit.client.0.smithi084.stdout:425/600: open fd = 3 2024-04-26T18:41:06.904 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.905 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.905 INFO:tasks.workunit.client.0.smithi084.stdout:426/600: open fd = 3 2024-04-26T18:41:06.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.905 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.905 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.905 INFO:tasks.workunit.client.0.smithi084.stdout:427/600: open fd = 3 2024-04-26T18:41:06.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.906 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.906 INFO:tasks.workunit.client.0.smithi084.stdout:428/600: open fd = 3 2024-04-26T18:41:06.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.906 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.907 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.907 INFO:tasks.workunit.client.0.smithi084.stdout:429/600: open fd = 3 2024-04-26T18:41:06.907 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.907 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.907 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.907 INFO:tasks.workunit.client.0.smithi084.stdout:430/600: open fd = 3 2024-04-26T18:41:06.907 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.907 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.908 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.908 INFO:tasks.workunit.client.0.smithi084.stdout:431/600: open fd = 3 2024-04-26T18:41:06.908 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.908 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.908 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.908 INFO:tasks.workunit.client.0.smithi084.stdout:432/600: open fd = 3 2024-04-26T18:41:06.908 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.909 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.909 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.909 INFO:tasks.workunit.client.0.smithi084.stdout:433/600: open fd = 3 2024-04-26T18:41:06.909 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.909 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.909 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.909 INFO:tasks.workunit.client.0.smithi084.stdout:434/600: open fd = 3 2024-04-26T18:41:06.909 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.909 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.909 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.909 INFO:tasks.workunit.client.0.smithi084.stdout:435/600: open fd = 3 2024-04-26T18:41:06.910 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.910 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.910 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:06.910 INFO:tasks.workunit.client.0.smithi084.stdout:436/600: open fd = 3 2024-04-26T18:41:06.910 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.910 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:41:06.910 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:41:07.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:06 smithi084 ceph-mon[29141]: pgmap v412: 97 pgs: 97 active+clean; 5.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-26T18:41:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:06 smithi103 ceph-mon[32727]: pgmap v412: 97 pgs: 97 active+clean; 5.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-26T18:41:07.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:06 smithi174 ceph-mon[32705]: pgmap v412: 97 pgs: 97 active+clean; 5.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-26T18:41:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.604+0000 7fdeb079d640 1 -- 172.21.15.84:0/3952276222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdea8102940 msgr2=0x7fdea8102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.604+0000 7fdeb079d640 1 --2- 172.21.15.84:0/3952276222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdea8102940 0x7fdea8102dc0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fdea4009930 tx=0x7fdea402a420 comp rx=0 tx=0).stop 2024-04-26T18:41:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.604+0000 7fdeb079d640 1 -- 172.21.15.84:0/3952276222 shutdown_connections 2024-04-26T18:41:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.604+0000 7fdeb079d640 1 --2- 172.21.15.84:0/3952276222 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdea8103300 0x7fdea810f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.604+0000 7fdeb079d640 1 --2- 172.21.15.84:0/3952276222 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdea8102940 0x7fdea8102dc0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.607 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.604+0000 7fdeb079d640 1 --2- 172.21.15.84:0/3952276222 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8108950 0x7fdea8108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.604+0000 7fdeb079d640 1 -- 172.21.15.84:0/3952276222 >> 172.21.15.84:0/3952276222 conn(0x7fdea8079dc0 msgr2=0x7fdea807a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.604+0000 7fdeb079d640 1 -- 172.21.15.84:0/3952276222 shutdown_connections 2024-04-26T18:41:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.604+0000 7fdeb079d640 1 -- 172.21.15.84:0/3952276222 wait complete. 2024-04-26T18:41:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeb079d640 1 Processor -- start 2024-04-26T18:41:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeb079d640 1 -- start start 2024-04-26T18:41:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeb079d640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8102940 0x7fdea8199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeb079d640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdea8103300 0x7fdea8199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeb079d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdea8108950 0x7fdea819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeb079d640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8112c60 con 0x7fdea8108950 2024-04-26T18:41:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeb079d640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8112ae0 con 0x7fdea8102940 2024-04-26T18:41:08.608 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeb079d640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdea8112de0 con 0x7fdea8103300 2024-04-26T18:41:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeaed3a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdea8108950 0x7fdea819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeaed3a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdea8108950 0x7fdea819d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43258/0 (socket says 172.21.15.84:43258) 2024-04-26T18:41:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeaed3a640 1 -- 172.21.15.84:0/1705099529 learned_addr learned my addr 172.21.15.84:0/1705099529 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeadd38640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdea8103300 0x7fdea8199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeae539640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8102940 0x7fdea8199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeaed3a640 1 -- 172.21.15.84:0/1705099529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdea8103300 msgr2=0x7fdea8199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeaed3a640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdea8103300 0x7fdea8199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeaed3a640 1 -- 172.21.15.84:0/1705099529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8102940 msgr2=0x7fdea8199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:08.609 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeaed3a640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8102940 0x7fdea8199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeaed3a640 1 -- 172.21.15.84:0/1705099529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdea819e0e0 con 0x7fdea8108950 2024-04-26T18:41:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeadd38640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdea8103300 0x7fdea8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:41:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fdeaed3a640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdea8108950 0x7fdea819d9b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fdea000e960 tx=0x7fdea000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.605+0000 7fde9f7fe640 1 -- 172.21.15.84:0/1705099529 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea0004d20 con 0x7fdea8108950 2024-04-26T18:41:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.606+0000 7fde9f7fe640 1 -- 172.21.15.84:0/1705099529 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea0005020 con 0x7fdea8108950 2024-04-26T18:41:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.606+0000 7fde9f7fe640 1 -- 172.21.15.84:0/1705099529 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea00116d0 con 0x7fdea8108950 2024-04-26T18:41:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.606+0000 7fdeb079d640 1 -- 172.21.15.84:0/1705099529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdea819e3d0 con 0x7fdea8108950 2024-04-26T18:41:08.610 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.606+0000 7fdeb079d640 1 -- 172.21.15.84:0/1705099529 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdea81a5c60 con 0x7fdea8108950 2024-04-26T18:41:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.608+0000 7fdeb3594740 1 Processor -- start 2024-04-26T18:41:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.608+0000 7fde9f7fe640 1 -- 172.21.15.84:0/1705099529 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fdea00051d0 con 0x7fdea8108950 2024-04-26T18:41:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeb3594740 1 -- start start 2024-04-26T18:41:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeb3594740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77def00 0x5650d77df2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeb3594740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77df820 0x5650d77e9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeb3594740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77ea0e0 0x5650d77ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeb3594740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5650d7745460 con 0x5650d77def00 2024-04-26T18:41:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeb3594740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5650d76ff590 con 0x5650d77ea0e0 2024-04-26T18:41:08.618 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeb3594740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5650d7741420 con 0x5650d77df820 2024-04-26T18:41:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeadd38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77def00 0x5650d77df2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeadd38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77def00 0x5650d77df2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43262/0 (socket says 172.21.15.84:43262) 2024-04-26T18:41:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeadd38640 1 -- 172.21.15.84:0/3597254429 learned_addr learned my addr 172.21.15.84:0/3597254429 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeae539640 1 --2- 172.21.15.84:0/3597254429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77df820 0x5650d77e9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fde9f7fe640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fde940a3e40 0x7fde940a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeadd38640 1 -- 172.21.15.84:0/3597254429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77df820 msgr2=0x5650d77e9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeadd38640 1 --2- 172.21.15.84:0/3597254429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77df820 0x5650d77e9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeadd38640 1 -- 172.21.15.84:0/3597254429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77ea0e0 msgr2=0x5650d77ec4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeadd38640 1 --2- 172.21.15.84:0/3597254429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77ea0e0 0x5650d77ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeadd38640 1 -- 172.21.15.84:0/3597254429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650d770c0d0 con 0x5650d77def00 2024-04-26T18:41:08.619 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeaed3a640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fde940a3e40 0x7fde940a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.609+0000 7fdeadd38640 1 --2- 172.21.15.84:0/3597254429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77def00 0x5650d77df2e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fdea4005ec0 tx=0x7fdea4004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65082e1963f5ce1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.610+0000 7fde9f7fe640 1 -- 172.21.15.84:0/1705099529 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 8623+0+0 (secure 0 0 0) 0x7fdea00d09d0 con 0x7fdea8108950 2024-04-26T18:41:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.610+0000 7fde9cff9640 1 -- 172.21.15.84:0/3597254429 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4041020 con 0x5650d77def00 2024-04-26T18:41:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.610+0000 7fde9cff9640 1 -- 172.21.15.84:0/3597254429 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea40046c0 con 0x5650d77def00 2024-04-26T18:41:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.610+0000 7fde9cff9640 1 -- 172.21.15.84:0/3597254429 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4035070 con 0x5650d77def00 2024-04-26T18:41:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.610+0000 7fdeb3594740 1 -- 172.21.15.84:0/3597254429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77def00 msgr2=0x5650d77df2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.610+0000 7fdeaed3a640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fde940a3e40 0x7fde940a6300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdea00ce170 tx=0x7fdea006f400 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.610+0000 7fdeb3594740 1 --2- 172.21.15.84:0/3597254429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77def00 0x5650d77df2e0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7fdea4005ec0 tx=0x7fdea4004350 comp rx=0 tx=0).stop 2024-04-26T18:41:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.610+0000 7fdeb3594740 1 -- 172.21.15.84:0/3597254429 shutdown_connections 2024-04-26T18:41:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.610+0000 7fdeb3594740 1 --2- 172.21.15.84:0/3597254429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77ea0e0 0x5650d77ec4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.610+0000 7fdeb3594740 1 --2- 172.21.15.84:0/3597254429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77df820 0x5650d77e9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.610+0000 7fdeb3594740 1 --2- 172.21.15.84:0/3597254429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77def00 0x5650d77df2e0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.620 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.610+0000 7fdeb3594740 1 -- 172.21.15.84:0/3597254429 >> 172.21.15.84:0/3597254429 conn(0x5650d77db410 msgr2=0x5650d77db7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeb3594740 1 -- 172.21.15.84:0/3597254429 shutdown_connections 2024-04-26T18:41:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeb3594740 1 -- 172.21.15.84:0/3597254429 wait complete. 2024-04-26T18:41:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeb3594740 1 -- start start 2024-04-26T18:41:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeb3594740 1 Processor -- start 2024-04-26T18:41:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeb3594740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77def00 0x5650d77e7060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeb3594740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 0x5650d77e75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeb3594740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77ea0e0 0x5650d77e7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeb3594740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5650d77eccb0 con 0x5650d77df820 2024-04-26T18:41:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeb3594740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5650d76ccb60 con 0x5650d77def00 2024-04-26T18:41:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeb3594740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5650d77ece30 con 0x5650d77ea0e0 2024-04-26T18:41:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeb3594740 10 client.0 _ll_register_callbacks cb 0x5650d76f12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:41:08.621 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeae539640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 0x5650d77e75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeae539640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 0x5650d77e75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43264/0 (socket says 172.21.15.84:43264) 2024-04-26T18:41:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeae539640 1 -- 172.21.15.84:0/3907634252 learned_addr learned my addr 172.21.15.84:0/3907634252 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeadd38640 1 --2- 172.21.15.84:0/3907634252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77def00 0x5650d77e7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeadd38640 1 --2- 172.21.15.84:0/3907634252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77ea0e0 0x5650d77e7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeae539640 1 -- 172.21.15.84:0/3907634252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77ea0e0 msgr2=0x5650d77e7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeae539640 1 --2- 172.21.15.84:0/3907634252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77ea0e0 0x5650d77e7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeae539640 1 -- 172.21.15.84:0/3907634252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77def00 msgr2=0x5650d77e7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeae539640 1 --2- 172.21.15.84:0/3907634252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77def00 0x5650d77e7060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.611+0000 7fdeae539640 1 -- 172.21.15.84:0/3907634252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650d7457210 con 0x5650d77df820 2024-04-26T18:41:08.622 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fdeae539640 1 --2- 172.21.15.84:0/3907634252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 0x5650d77e75a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fde9800bfc0 tx=0x7fde9800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c8ce1983866d18 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fde91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fde91ffb640 1 -- 172.21.15.84:0/3907634252 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98018020 con 0x5650d77df820 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fde91ffb640 1 -- 172.21.15.84:0/3907634252 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8623+0+0 (secure 0 0 0) 0x7fde98014870 con 0x5650d77df820 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fdeb3594740 1 -- 172.21.15.84:0/3907634252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdea40046c0 con 0x5650d77df820 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fde91ffb640 1 -- 172.21.15.84:0/3907634252 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fde98023050 con 0x5650d77df820 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fdeb3594740 10 client.16983 fetch_fsmap learned FSMap version 29 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fdeb3594740 1 -- 172.21.15.84:0/3907634252 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650d7523620 con 0x5650d77df820 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fde91ffb640 1 -- 172.21.15.84:0/3907634252 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdea40046c0 con 0x5650d77df820 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fde91ffb640 1 -- 172.21.15.84:0/3907634252 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fde98013a10 con 0x5650d77df820 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fdeb3594740 10 client.16983 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fdeb3594740 10 client.16983 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fdeb3594740 20 client.16983 populate_metadata read hostname 'smithi084' 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fdeb3594740 1 --2- 172.21.15.84:0/3907634252 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5650d77fa200 0x5650d77fc630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.623 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fdeb3594740 4 client.16983 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.612+0000 7fdeb3594740 1 -- 172.21.15.84:0/3907634252 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5650d7753070 con 0x5650d77fa200 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.613+0000 7fdeadd38640 1 --2- 172.21.15.84:0/3907634252 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5650d77fa200 0x5650d77fc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.613+0000 7fdeadd38640 1 --2- 172.21.15.84:0/3907634252 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5650d77fa200 0x5650d77fc630 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47ec7e474efc25e5 server_cookie=83f1f339f3694fa7 in_seq=0 out_seq=0 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.613+0000 7fde91ffb640 10 client.16983 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fde91ffb640 1 -- 172.21.15.84:0/3907634252 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5650d7753070 con 0x5650d77fa200 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fde91ffb640 10 client.16983 handle_command_reply: tid=0 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fdeb3594740 1 client.16983 shutdown 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fdeb3594740 1 -- 172.21.15.84:0/3907634252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 msgr2=0x5650d77e75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fdeb3594740 1 --2- 172.21.15.84:0/3907634252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 0x5650d77e75a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fde9800bfc0 tx=0x7fde9800f670 comp rx=0 tx=0).stop 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fdeb3594740 1 -- 172.21.15.84:0/3907634252 shutdown_connections 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fdeb3594740 1 --2- 172.21.15.84:0/3907634252 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5650d77fa200 0x5650d77fc630 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fdeb3594740 1 --2- 172.21.15.84:0/3907634252 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77ea0e0 0x5650d77e7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fdeb3594740 1 --2- 172.21.15.84:0/3907634252 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 0x5650d77e75a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.624 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fde91ffb640 0 client.16983 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fdeb3594740 1 --2- 172.21.15.84:0/3907634252 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77def00 0x5650d77e7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fdeb3594740 1 -- 172.21.15.84:0/3907634252 >> 172.21.15.84:0/3907634252 conn(0x5650d77db410 msgr2=0x5650d77e1190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fdeb3594740 1 -- 172.21.15.84:0/3907634252 shutdown_connections 2024-04-26T18:41:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fdeb3594740 1 -- 172.21.15.84:0/3907634252 wait complete. 2024-04-26T18:41:08.625 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.614+0000 7fdeb3594740 20 client.16983 trim_cache size 0 max 16384 2024-04-26T18:41:08.633 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeb3594740 1 Processor -- start 2024-04-26T18:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeb3594740 1 -- start start 2024-04-26T18:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeb3594740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77def00 0x5650d77f5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeb3594740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 0x5650d77e18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeb3594740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77ea0e0 0x5650d7801f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeb3594740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81ae600 con 0x5650d77df820 2024-04-26T18:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeb3594740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81b0140 con 0x5650d77ea0e0 2024-04-26T18:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeb3594740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81aecd0 con 0x5650d77def00 2024-04-26T18:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeae539640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 0x5650d77e18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeae539640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 0x5650d77e18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43266/0 (socket says 172.21.15.84:43266) 2024-04-26T18:41:08.634 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeae539640 1 -- 172.21.15.84:0/4164414694 learned_addr learned my addr 172.21.15.84:0/4164414694 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeae539640 1 -- 172.21.15.84:0/4164414694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77def00 msgr2=0x5650d77f5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeae539640 1 --2- 172.21.15.84:0/4164414694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77def00 0x5650d77f5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeae539640 1 -- 172.21.15.84:0/4164414694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77ea0e0 msgr2=0x5650d7801f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeae539640 1 --2- 172.21.15.84:0/4164414694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77ea0e0 0x5650d7801f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeae539640 1 -- 172.21.15.84:0/4164414694 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5650d77e1e00 con 0x5650d77df820 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.629+0000 7fdeae539640 1 --2- 172.21.15.84:0/4164414694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 0x5650d77e18c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fde98004810 tx=0x7fde9800b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aba771c44da7138 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.630+0000 7fde93fff640 1 -- 172.21.15.84:0/4164414694 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9802b070 con 0x5650d77df820 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.630+0000 7fde93fff640 1 -- 172.21.15.84:0/4164414694 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde9800fb70 con 0x5650d77df820 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.630+0000 7fde93fff640 1 -- 172.21.15.84:0/4164414694 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde98018040 con 0x5650d77df820 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.630+0000 7fdeb3594740 1 -- 172.21.15.84:0/4164414694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 msgr2=0x5650d77e18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.630+0000 7fdeb3594740 1 --2- 172.21.15.84:0/4164414694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 0x5650d77e18c0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fde98004810 tx=0x7fde9800b140 comp rx=0 tx=0).stop 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.630+0000 7fdeb3594740 1 -- 172.21.15.84:0/4164414694 shutdown_connections 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.630+0000 7fdeb3594740 1 --2- 172.21.15.84:0/4164414694 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77ea0e0 0x5650d7801f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.635 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.630+0000 7fdeb3594740 1 --2- 172.21.15.84:0/4164414694 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77df820 0x5650d77e18c0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.630+0000 7fdeb3594740 1 --2- 172.21.15.84:0/4164414694 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77def00 0x5650d77f5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.630+0000 7fdeb3594740 1 -- 172.21.15.84:0/4164414694 >> 172.21.15.84:0/4164414694 conn(0x5650d77db410 msgr2=0x5650d77e9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.630+0000 7fdeb3594740 1 -- 172.21.15.84:0/4164414694 shutdown_connections 2024-04-26T18:41:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.630+0000 7fdeb3594740 1 -- 172.21.15.84:0/4164414694 wait complete. 2024-04-26T18:41:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.630+0000 7fdeb3594740 1 -- start start 2024-04-26T18:41:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeb3594740 1 Processor -- start 2024-04-26T18:41:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeb3594740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77def00 0x5650d77f21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeb3594740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77df820 0x5650d77ee5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeb3594740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77ea0e0 0x5650d77eeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeb3594740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81afa70 con 0x5650d77ea0e0 2024-04-26T18:41:08.636 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeb3594740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81af3a0 con 0x5650d77def00 2024-04-26T18:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeb3594740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdea81ac600 con 0x5650d77df820 2024-04-26T18:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeb3594740 10 client.0 _ll_register_callbacks cb 0x7fde9800b2e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeadd38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77ea0e0 0x5650d77eeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeadd38640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77ea0e0 0x5650d77eeb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:43272/0 (socket says 172.21.15.84:43272) 2024-04-26T18:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeadd38640 1 -- 172.21.15.84:0/2844048857 learned_addr learned my addr 172.21.15.84:0/2844048857 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeadd38640 1 -- 172.21.15.84:0/2844048857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77df820 msgr2=0x5650d77ee5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeadd38640 1 --2- 172.21.15.84:0/2844048857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77df820 0x5650d77ee5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeadd38640 1 -- 172.21.15.84:0/2844048857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77def00 msgr2=0x5650d77f21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeadd38640 1 --2- 172.21.15.84:0/2844048857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77def00 0x5650d77f21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeadd38640 1 -- 172.21.15.84:0/2844048857 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5650d77ef040 con 0x5650d77ea0e0 2024-04-26T18:41:08.637 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeadd38640 1 --2- 172.21.15.84:0/2844048857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77ea0e0 0x5650d77eeb00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fdea4031440 tx=0x7fdea4009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b3674fbbb9c2901 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fde91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fde91ffb640 1 -- 172.21.15.84:0/2844048857 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdea4041070 con 0x5650d77ea0e0 2024-04-26T18:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fde91ffb640 1 -- 172.21.15.84:0/2844048857 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(109..109 src has 1..109) ==== 8623+0+0 (secure 0 0 0) 0x7fdea4005040 con 0x5650d77ea0e0 2024-04-26T18:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeb3594740 1 -- 172.21.15.84:0/2844048857 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde9800fb70 con 0x5650d77ea0e0 2024-04-26T18:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fde91ffb640 1 -- 172.21.15.84:0/2844048857 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdea4035040 con 0x5650d77ea0e0 2024-04-26T18:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.631+0000 7fdeb3594740 10 client.16989 fetch_fsmap learned FSMap version 29 2024-04-26T18:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.632+0000 7fdeb3594740 1 -- 172.21.15.84:0/2844048857 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5650d7861520 con 0x5650d77ea0e0 2024-04-26T18:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.632+0000 7fde91ffb640 1 -- 172.21.15.84:0/2844048857 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde9800fb70 con 0x5650d77ea0e0 2024-04-26T18:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.632+0000 7fde91ffb640 1 -- 172.21.15.84:0/2844048857 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdea40314a0 con 0x5650d77ea0e0 2024-04-26T18:41:08.638 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.632+0000 7fdeb3594740 10 client.16989 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:41:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.632+0000 7fdeb3594740 10 client.16989 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:41:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.632+0000 7fdeb3594740 20 client.16989 populate_metadata read hostname 'smithi084' 2024-04-26T18:41:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.632+0000 7fdeb3594740 1 --2- 172.21.15.84:0/2844048857 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5650d77fa200 0x5650d7863910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.632+0000 7fdeb3594740 4 client.16989 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:41:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.632+0000 7fdeb3594740 1 -- 172.21.15.84:0/2844048857 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5650d774cee0 con 0x5650d77fa200 2024-04-26T18:41:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.632+0000 7fdeae539640 1 --2- 172.21.15.84:0/2844048857 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5650d77fa200 0x5650d7863910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.632+0000 7fde91ffb640 10 client.16989 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:08.639 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.632+0000 7fdeae539640 1 --2- 172.21.15.84:0/2844048857 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5650d77fa200 0x5650d7863910 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a61a9d2f22233428 server_cookie=3c2dcda55579e35e in_seq=0 out_seq=0 2024-04-26T18:41:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fde91ffb640 1 -- 172.21.15.84:0/2844048857 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5650d774cee0 con 0x5650d77fa200 2024-04-26T18:41:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fde91ffb640 10 client.16989 handle_command_reply: tid=0 2024-04-26T18:41:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fdeb3594740 1 client.16989 shutdown 2024-04-26T18:41:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fdeb3594740 1 -- 172.21.15.84:0/2844048857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77ea0e0 msgr2=0x5650d77eeb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fdeb3594740 1 --2- 172.21.15.84:0/2844048857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77ea0e0 0x5650d77eeb00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fdea4031440 tx=0x7fdea4009810 comp rx=0 tx=0).stop 2024-04-26T18:41:08.648 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fdeb3594740 1 -- 172.21.15.84:0/2844048857 shutdown_connections 2024-04-26T18:41:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fdeb3594740 1 --2- 172.21.15.84:0/2844048857 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5650d77fa200 0x5650d7863910 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fdeb3594740 1 --2- 172.21.15.84:0/2844048857 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5650d77ea0e0 0x5650d77eeb00 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fde91ffb640 0 client.16989 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fdeb3594740 1 --2- 172.21.15.84:0/2844048857 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5650d77df820 0x5650d77ee5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fdeb3594740 1 --2- 172.21.15.84:0/2844048857 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5650d77def00 0x5650d77f21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fdeb3594740 1 -- 172.21.15.84:0/2844048857 >> 172.21.15.84:0/2844048857 conn(0x5650d77db410 msgr2=0x5650d77e7000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fdeb3594740 1 -- 172.21.15.84:0/2844048857 shutdown_connections 2024-04-26T18:41:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fdeb3594740 1 -- 172.21.15.84:0/2844048857 wait complete. 2024-04-26T18:41:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.647+0000 7fdeb3594740 20 client.16989 trim_cache size 0 max 16384 2024-04-26T18:41:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:41:08.649 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:41:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:41:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:41:08.650 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.648+0000 7fdeb079d640 1 -- 172.21.15.84:0/1705099529 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fde940a3e40 msgr2=0x7fde940a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.648+0000 7fdeb079d640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fde940a3e40 0x7fde940a6300 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fdea00ce170 tx=0x7fdea006f400 comp rx=0 tx=0).stop 2024-04-26T18:41:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.648+0000 7fdeb079d640 1 -- 172.21.15.84:0/1705099529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdea8108950 msgr2=0x7fdea819d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.648+0000 7fdeb079d640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdea8108950 0x7fdea819d9b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fdea000e960 tx=0x7fdea000ee20 comp rx=0 tx=0).stop 2024-04-26T18:41:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.648+0000 7fdeb079d640 1 -- 172.21.15.84:0/1705099529 shutdown_connections 2024-04-26T18:41:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.648+0000 7fdeb079d640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fde940a3e40 0x7fde940a6300 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.648+0000 7fdeb079d640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdea8108950 0x7fdea819d9b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.648+0000 7fdeb079d640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdea8103300 0x7fdea8199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.648+0000 7fdeb079d640 1 --2- 172.21.15.84:0/1705099529 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdea8102940 0x7fdea8199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:08.651 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.648+0000 7fdeb079d640 1 -- 172.21.15.84:0/1705099529 >> 172.21.15.84:0/1705099529 conn(0x7fdea8079dc0 msgr2=0x7fdea8100200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.648+0000 7fdeb079d640 1 -- 172.21.15.84:0/1705099529 shutdown_connections 2024-04-26T18:41:08.652 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:08.648+0000 7fdeb079d640 1 -- 172.21.15.84:0/1705099529 wait complete. 2024-04-26T18:41:09.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:08 smithi084 ceph-mon[29141]: pgmap v413: 97 pgs: 97 active+clean; 5.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-26T18:41:09.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:08 smithi103 ceph-mon[32727]: pgmap v413: 97 pgs: 97 active+clean; 5.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-26T18:41:09.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:08 smithi174 ceph-mon[32705]: pgmap v413: 97 pgs: 97 active+clean; 5.2 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-26T18:41:11.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:10 smithi084 ceph-mon[29141]: pgmap v414: 97 pgs: 97 active+clean; 5.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-04-26T18:41:11.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:10 smithi103 ceph-mon[32727]: pgmap v414: 97 pgs: 97 active+clean; 5.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-04-26T18:41:11.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:10 smithi174 ceph-mon[32705]: pgmap v414: 97 pgs: 97 active+clean; 5.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-04-26T18:41:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:13 smithi084 ceph-mon[29141]: pgmap v415: 97 pgs: 97 active+clean; 5.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-04-26T18:41:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:13 smithi084 ceph-mon[29141]: osdmap e110: 12 total, 12 up, 12 in 2024-04-26T18:41:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:13 smithi103 ceph-mon[32727]: pgmap v415: 97 pgs: 97 active+clean; 5.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-04-26T18:41:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:13 smithi103 ceph-mon[32727]: osdmap e110: 12 total, 12 up, 12 in 2024-04-26T18:41:13.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:13 smithi174 ceph-mon[32705]: pgmap v415: 97 pgs: 97 active+clean; 5.4 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 100 KiB/s wr, 36 op/s 2024-04-26T18:41:13.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:13 smithi174 ceph-mon[32705]: osdmap e110: 12 total, 12 up, 12 in 2024-04-26T18:41:14.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:14 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:41:14.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:14 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:41:14.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:14 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:41:15.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:15 smithi084 ceph-mon[29141]: pgmap v417: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-04-26T18:41:15.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:15 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:41:15.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:15 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:41:15.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:15 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:41:15.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:15 smithi103 ceph-mon[32727]: pgmap v417: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-04-26T18:41:15.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:15 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:41:15.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:15 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:41:15.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:15 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:41:15.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:15 smithi174 ceph-mon[32705]: pgmap v417: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 54 KiB/s wr, 20 op/s 2024-04-26T18:41:15.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:15 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:41:15.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:15 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:41:15.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:15 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:41:17.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:17 smithi084 ceph-mon[29141]: pgmap v418: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-26T18:41:17.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:17 smithi103 ceph-mon[32727]: pgmap v418: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-26T18:41:17.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:17 smithi174 ceph-mon[32705]: pgmap v418: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-26T18:41:19.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:19 smithi084 ceph-mon[29141]: pgmap v419: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-26T18:41:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:19 smithi103 ceph-mon[32727]: pgmap v419: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-26T18:41:19.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:19 smithi174 ceph-mon[32705]: pgmap v419: 97 pgs: 97 active+clean; 1.6 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-26T18:41:21.742 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:21 smithi174 ceph-mon[32705]: pgmap v420: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:41:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.744+0000 7f76fbc52640 1 -- 172.21.15.84:0/2509561819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76f41044e0 msgr2=0x7f76f410c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.744+0000 7f76fbc52640 1 --2- 172.21.15.84:0/2509561819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76f41044e0 0x7f76f410c9c0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009900 tx=0x7f76e802a510 comp rx=0 tx=0).stop 2024-04-26T18:41:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.744+0000 7f76fbc52640 1 -- 172.21.15.84:0/2509561819 shutdown_connections 2024-04-26T18:41:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.744+0000 7f76fbc52640 1 --2- 172.21.15.84:0/2509561819 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f410cf00 0x7f76f410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.744+0000 7f76fbc52640 1 --2- 172.21.15.84:0/2509561819 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76f41044e0 0x7f76f410c9c0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.746 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.744+0000 7f76fbc52640 1 --2- 172.21.15.84:0/2509561819 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76f4103bc0 0x7f76f4103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.744+0000 7f76fbc52640 1 -- 172.21.15.84:0/2509561819 >> 172.21.15.84:0/2509561819 conn(0x7f76f40789b0 msgr2=0x7f76f4078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.744+0000 7f76fbc52640 1 -- 172.21.15.84:0/2509561819 shutdown_connections 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.744+0000 7f76fbc52640 1 -- 172.21.15.84:0/2509561819 wait complete. 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76fbc52640 1 Processor -- start 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76fbc52640 1 -- start start 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76fbc52640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76f4103bc0 0x7f76f4101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76fbc52640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76f41044e0 0x7f76f4101d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76fbc52640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f410cf00 0x7f76f4102290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76fbc52640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4112bb0 con 0x7f76f4103bc0 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76fbc52640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4112a30 con 0x7f76f41044e0 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76fbc52640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76f4112d30 con 0x7f76f410cf00 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76f99ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76f4103bc0 0x7f76f4101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76f99ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76f4103bc0 0x7f76f4101810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51418/0 (socket says 172.21.15.84:51418) 2024-04-26T18:41:21.747 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76f99ee640 1 -- 172.21.15.84:0/1174743765 learned_addr learned my addr 172.21.15.84:0/1174743765 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76f91ed640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76f41044e0 0x7f76f4101d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76fa1ef640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f410cf00 0x7f76f4102290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76f99ee640 1 -- 172.21.15.84:0/1174743765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f410cf00 msgr2=0x7f76f4102290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76f99ee640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f410cf00 0x7f76f4102290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76f99ee640 1 -- 172.21.15.84:0/1174743765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76f41044e0 msgr2=0x7f76f4101d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76f99ee640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76f41044e0 0x7f76f4101d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.745+0000 7f76f99ee640 1 -- 172.21.15.84:0/1174743765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76f41a75b0 con 0x7f76f4103bc0 2024-04-26T18:41:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.746+0000 7f76f99ee640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76f4103bc0 0x7f76f4101810 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f76e400eab0 tx=0x7f76e400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.746+0000 7f76f2ffd640 1 -- 172.21.15.84:0/1174743765 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e400ce40 con 0x7f76f4103bc0 2024-04-26T18:41:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.746+0000 7f76f2ffd640 1 -- 172.21.15.84:0/1174743765 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76e4011920 con 0x7f76f4103bc0 2024-04-26T18:41:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.746+0000 7f76fbc52640 1 -- 172.21.15.84:0/1174743765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76f41a78a0 con 0x7f76f4103bc0 2024-04-26T18:41:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.746+0000 7f76f2ffd640 1 -- 172.21.15.84:0/1174743765 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4010a30 con 0x7f76f4103bc0 2024-04-26T18:41:21.748 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.746+0000 7f76fbc52640 1 -- 172.21.15.84:0/1174743765 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76f41a7dc0 con 0x7f76f4103bc0 2024-04-26T18:41:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.749+0000 7f76f2ffd640 1 -- 172.21.15.84:0/1174743765 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f76e4010bd0 con 0x7f76f4103bc0 2024-04-26T18:41:21.749 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.749+0000 7f76fe969740 1 Processor -- start 2024-04-26T18:41:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.749+0000 7f76fe969740 1 -- start start 2024-04-26T18:41:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fe969740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa552e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fe969740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa5fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fe969740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa624d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fe969740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5566ef9bb460 con 0x5566efa54f00 2024-04-26T18:41:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fe969740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5566ef975590 con 0x5566efa600e0 2024-04-26T18:41:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fe969740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5566ef9b7420 con 0x5566efa55820 2024-04-26T18:41:21.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76f2ffd640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f76dc0a3dc0 0x7f76dc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fa1ef640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fa1ef640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51422/0 (socket says 172.21.15.84:51422) 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fa1ef640 1 -- 172.21.15.84:0/4182928609 learned_addr learned my addr 172.21.15.84:0/4182928609 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76f99ee640 1 --2- 172.21.15.84:0/4182928609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa5fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fa1ef640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f76dc0a3dc0 0x7f76dc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76f91ed640 1 --2- 172.21.15.84:0/4182928609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa624d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76f2ffd640 1 -- 172.21.15.84:0/1174743765 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f76e40d0e70 con 0x7f76f4103bc0 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fa1ef640 1 -- 172.21.15.84:0/4182928609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 msgr2=0x5566efa5fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fa1ef640 1 --2- 172.21.15.84:0/4182928609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa5fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fa1ef640 1 -- 172.21.15.84:0/4182928609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 msgr2=0x5566efa624d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fa1ef640 1 --2- 172.21.15.84:0/4182928609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa624d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.750+0000 7f76fa1ef640 1 -- 172.21.15.84:0/4182928609 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566ef9820d0 con 0x5566efa54f00 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.751+0000 7f76fa1ef640 1 --2- 172.21.15.84:0/4182928609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa552e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0129d0 tx=0x7f76ec012dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bb55ed5921538 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:21.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.751+0000 7f76fa1ef640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f76dc0a3dc0 0x7f76dc0a6280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0107a0 tx=0x7f76ec01d000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.751+0000 7f76dbfff640 1 -- 172.21.15.84:0/4182928609 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec022070 con 0x5566efa54f00 2024-04-26T18:41:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.751+0000 7f76dbfff640 1 -- 172.21.15.84:0/4182928609 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76ec009e00 con 0x5566efa54f00 2024-04-26T18:41:21.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.751+0000 7f76dbfff640 1 -- 172.21.15.84:0/4182928609 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76ec00dc00 con 0x5566efa54f00 2024-04-26T18:41:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.751+0000 7f76fe969740 1 -- 172.21.15.84:0/4182928609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 msgr2=0x5566efa552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.751+0000 7f76fe969740 1 --2- 172.21.15.84:0/4182928609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa552e0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0129d0 tx=0x7f76ec012dd0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.752+0000 7f76fe969740 1 -- 172.21.15.84:0/4182928609 shutdown_connections 2024-04-26T18:41:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.752+0000 7f76fe969740 1 --2- 172.21.15.84:0/4182928609 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa624d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.752+0000 7f76fe969740 1 --2- 172.21.15.84:0/4182928609 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa5fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.752+0000 7f76fe969740 1 --2- 172.21.15.84:0/4182928609 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa552e0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.752+0000 7f76fe969740 1 -- 172.21.15.84:0/4182928609 >> 172.21.15.84:0/4182928609 conn(0x5566efa51410 msgr2=0x5566efa517f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:21.757 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.752+0000 7f76fe969740 1 -- 172.21.15.84:0/4182928609 shutdown_connections 2024-04-26T18:41:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.752+0000 7f76fe969740 1 -- 172.21.15.84:0/4182928609 wait complete. 2024-04-26T18:41:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.752+0000 7f76fe969740 1 -- start start 2024-04-26T18:41:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.752+0000 7f76fe969740 1 Processor -- start 2024-04-26T18:41:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.752+0000 7f76fe969740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa5d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.752+0000 7f76fe969740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa5d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76fe969740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa5dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76fe969740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5566efa62c50 con 0x5566efa54f00 2024-04-26T18:41:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76fe969740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5566ef942b60 con 0x5566efa600e0 2024-04-26T18:41:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76fe969740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5566efa62dd0 con 0x5566efa55820 2024-04-26T18:41:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76fe969740 10 client.0 _ll_register_callbacks cb 0x5566ef9672b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:41:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76f99ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa5d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.758 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76f99ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa5d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51424/0 (socket says 172.21.15.84:51424) 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76f99ee640 1 -- 172.21.15.84:0/70931194 learned_addr learned my addr 172.21.15.84:0/70931194 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76fa1ef640 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa5dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76f91ed640 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa5d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76f99ee640 1 -- 172.21.15.84:0/70931194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 msgr2=0x5566efa5d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76f99ee640 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa5d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76f99ee640 1 -- 172.21.15.84:0/70931194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 msgr2=0x5566efa5dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76f99ee640 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa5dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76f99ee640 1 -- 172.21.15.84:0/70931194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566ef6cd210 con 0x5566efa54f00 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76fa1ef640 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa5dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76f91ed640 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa5d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76f99ee640 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa5d170 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f76e4019000 tx=0x7f76e4018f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f5cd749532582e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76d97fa640 1 -- 172.21.15.84:0/70931194 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e409d4d0 con 0x5566efa54f00 2024-04-26T18:41:21.759 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76d97fa640 1 -- 172.21.15.84:0/70931194 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f76e40b6260 con 0x5566efa54f00 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76fe969740 1 -- 172.21.15.84:0/70931194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76ec009e00 con 0x5566efa54f00 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76d97fa640 1 -- 172.21.15.84:0/70931194 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f76e409cdd0 con 0x5566efa54f00 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76fe969740 10 client.17001 fetch_fsmap learned FSMap version 29 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.753+0000 7f76fe969740 1 -- 172.21.15.84:0/70931194 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5566ef799620 con 0x5566efa54f00 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.754+0000 7f76d97fa640 1 -- 172.21.15.84:0/70931194 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76ec009e00 con 0x5566efa54f00 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.754+0000 7f76d97fa640 1 -- 172.21.15.84:0/70931194 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f76e400f040 con 0x5566efa54f00 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.754+0000 7f76fe969740 10 client.17001 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.754+0000 7f76fe969740 10 client.17001 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.754+0000 7f76fe969740 20 client.17001 populate_metadata read hostname 'smithi084' 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.754+0000 7f76fe969740 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5566efa702d0 0x5566efa726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.754+0000 7f76fe969740 4 client.17001 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.754+0000 7f76fe969740 1 -- 172.21.15.84:0/70931194 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566ef9c9070 con 0x5566efa702d0 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.754+0000 7f76f91ed640 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5566efa702d0 0x5566efa726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.755+0000 7f76f91ed640 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5566efa702d0 0x5566efa726c0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=855e6652dba81fb1 server_cookie=a87d6519defe4189 in_seq=0 out_seq=0 2024-04-26T18:41:21.760 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.755+0000 7f76d97fa640 10 client.17001 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.755+0000 7f76d97fa640 1 -- 172.21.15.84:0/70931194 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566ef9c9070 con 0x5566efa702d0 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.755+0000 7f76d97fa640 10 client.17001 handle_command_reply: tid=0 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.755+0000 7f76fe969740 1 client.17001 shutdown 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.755+0000 7f76fe969740 1 -- 172.21.15.84:0/70931194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 msgr2=0x5566efa5d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.755+0000 7f76fe969740 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa5d170 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f76e4019000 tx=0x7f76e4018f90 comp rx=0 tx=0).stop 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.756+0000 7f76fe969740 1 -- 172.21.15.84:0/70931194 shutdown_connections 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.756+0000 7f76fe969740 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5566efa702d0 0x5566efa726c0 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.756+0000 7f76fe969740 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa5dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.756+0000 7f76fe969740 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa5d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.756+0000 7f76fe969740 1 --2- 172.21.15.84:0/70931194 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa5d170 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.756+0000 7f76d97fa640 0 client.17001 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.756+0000 7f76fe969740 1 -- 172.21.15.84:0/70931194 >> 172.21.15.84:0/70931194 conn(0x5566efa51410 msgr2=0x5566efa57140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.756+0000 7f76fe969740 1 -- 172.21.15.84:0/70931194 shutdown_connections 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.756+0000 7f76fe969740 1 -- 172.21.15.84:0/70931194 wait complete. 2024-04-26T18:41:21.761 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.756+0000 7f76fe969740 20 client.17001 trim_cache size 0 max 16384 2024-04-26T18:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76fe969740 1 Processor -- start 2024-04-26T18:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76fe969740 1 -- start start 2024-04-26T18:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76fe969740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa50430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76fe969740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa77f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76fe969740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa784c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76fe969740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b0780 con 0x5566efa54f00 2024-04-26T18:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76fe969740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b22c0 con 0x5566efa600e0 2024-04-26T18:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76fe969740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b0e50 con 0x5566efa55820 2024-04-26T18:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76f99ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa50430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.776 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76f99ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa50430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51428/0 (socket says 172.21.15.84:51428) 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76f99ee640 1 -- 172.21.15.84:0/348786737 learned_addr learned my addr 172.21.15.84:0/348786737 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76f91ed640 1 --2- 172.21.15.84:0/348786737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa77f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76fa1ef640 1 --2- 172.21.15.84:0/348786737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa784c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76f99ee640 1 -- 172.21.15.84:0/348786737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 msgr2=0x5566efa77f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76f99ee640 1 --2- 172.21.15.84:0/348786737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa77f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76f99ee640 1 -- 172.21.15.84:0/348786737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 msgr2=0x5566efa784c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76f99ee640 1 --2- 172.21.15.84:0/348786737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa784c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76f99ee640 1 -- 172.21.15.84:0/348786737 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566efa72a30 con 0x5566efa54f00 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76f91ed640 1 --2- 172.21.15.84:0/348786737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa77f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.771+0000 7f76fa1ef640 1 --2- 172.21.15.84:0/348786737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa784c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.772+0000 7f76f99ee640 1 --2- 172.21.15.84:0/348786737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa50430 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f76e409cad0 tx=0x7f76e40104d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2bf8375cba478b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.772+0000 7f76db7fe640 1 -- 172.21.15.84:0/348786737 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e409cba0 con 0x5566efa54f00 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.772+0000 7f76db7fe640 1 -- 172.21.15.84:0/348786737 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76e409d4d0 con 0x5566efa54f00 2024-04-26T18:41:21.777 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.772+0000 7f76db7fe640 1 -- 172.21.15.84:0/348786737 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e40b6280 con 0x5566efa54f00 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.772+0000 7f76fe969740 1 -- 172.21.15.84:0/348786737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 msgr2=0x5566efa50430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.772+0000 7f76fe969740 1 --2- 172.21.15.84:0/348786737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa50430 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f76e409cad0 tx=0x7f76e40104d0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.772+0000 7f76fe969740 1 -- 172.21.15.84:0/348786737 shutdown_connections 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.772+0000 7f76fe969740 1 --2- 172.21.15.84:0/348786737 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa784c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.772+0000 7f76fe969740 1 --2- 172.21.15.84:0/348786737 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa77f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.772+0000 7f76fe969740 1 --2- 172.21.15.84:0/348786737 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa50430 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.772+0000 7f76fe969740 1 -- 172.21.15.84:0/348786737 >> 172.21.15.84:0/348786737 conn(0x5566efa51410 msgr2=0x5566efa4fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.772+0000 7f76fe969740 1 -- 172.21.15.84:0/348786737 shutdown_connections 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.772+0000 7f76fe969740 1 -- 172.21.15.84:0/348786737 wait complete. 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.773+0000 7f76fe969740 1 -- start start 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.773+0000 7f76fe969740 1 Processor -- start 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.773+0000 7f76fe969740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa68020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.773+0000 7f76fe969740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa68560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.773+0000 7f76fe969740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa64ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.778 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.773+0000 7f76fe969740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b1bf0 con 0x5566efa54f00 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.773+0000 7f76fe969740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41b1520 con 0x5566efa600e0 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.773+0000 7f76fe969740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f76f41ae780 con 0x5566efa55820 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.773+0000 7f76fe969740 10 client.0 _ll_register_callbacks cb 0x7f76e4015ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.773+0000 7f76f99ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa68020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.773+0000 7f76f99ee640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa68020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51440/0 (socket says 172.21.15.84:51440) 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.773+0000 7f76f99ee640 1 -- 172.21.15.84:0/2597950148 learned_addr learned my addr 172.21.15.84:0/2597950148 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76fa1ef640 1 --2- 172.21.15.84:0/2597950148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa64ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76f99ee640 1 -- 172.21.15.84:0/2597950148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 msgr2=0x5566efa68560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76f99ee640 1 --2- 172.21.15.84:0/2597950148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa68560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76f99ee640 1 -- 172.21.15.84:0/2597950148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 msgr2=0x5566efa64ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76f99ee640 1 --2- 172.21.15.84:0/2597950148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa64ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76f99ee640 1 -- 172.21.15.84:0/2597950148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566efa656b0 con 0x5566efa54f00 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76fa1ef640 1 --2- 172.21.15.84:0/2597950148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa64ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76f99ee640 1 --2- 172.21.15.84:0/2597950148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa68020 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f76e4099230 tx=0x7f76e4099070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1185b4887c4357c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:21.779 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76d97fa640 1 -- 172.21.15.84:0/2597950148 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e40ba070 con 0x5566efa54f00 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76d97fa640 1 -- 172.21.15.84:0/2597950148 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f76e409c220 con 0x5566efa54f00 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76fe969740 1 -- 172.21.15.84:0/2597950148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76e409d4d0 con 0x5566efa54f00 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76d97fa640 1 -- 172.21.15.84:0/2597950148 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f76e409cba0 con 0x5566efa54f00 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76fe969740 10 client.17007 fetch_fsmap learned FSMap version 29 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76fe969740 1 -- 172.21.15.84:0/2597950148 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5566efad7140 con 0x5566efa54f00 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76d97fa640 1 -- 172.21.15.84:0/2597950148 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76e409d4d0 con 0x5566efa54f00 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76d97fa640 1 -- 172.21.15.84:0/2597950148 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f76e409bc90 con 0x5566efa54f00 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76fe969740 10 client.17007 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76fe969740 10 client.17007 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76fe969740 20 client.17007 populate_metadata read hostname 'smithi084' 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76fe969740 1 --2- 172.21.15.84:0/2597950148 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5566efad75c0 0x5566efad99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76fe969740 4 client.17007 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.774+0000 7f76fe969740 1 -- 172.21.15.84:0/2597950148 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566ef9c2ee0 con 0x5566efad75c0 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.775+0000 7f76f91ed640 1 --2- 172.21.15.84:0/2597950148 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5566efad75c0 0x5566efad99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:21.780 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.775+0000 7f76f91ed640 1 --2- 172.21.15.84:0/2597950148 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5566efad75c0 0x5566efad99b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c606b4b02c69a67d server_cookie=19fcb11151a77fde in_seq=0 out_seq=0 2024-04-26T18:41:21.781 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.775+0000 7f76d97fa640 10 client.17007 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:21.789 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.788+0000 7f76d97fa640 1 -- 172.21.15.84:0/2597950148 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566ef9c2ee0 con 0x5566efad75c0 2024-04-26T18:41:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.788+0000 7f76d97fa640 10 client.17007 handle_command_reply: tid=0 2024-04-26T18:41:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.788+0000 7f76fe969740 1 client.17007 shutdown 2024-04-26T18:41:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fe969740 1 -- 172.21.15.84:0/2597950148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 msgr2=0x5566efa68020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fe969740 1 --2- 172.21.15.84:0/2597950148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa68020 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f76e4099230 tx=0x7f76e4099070 comp rx=0 tx=0).stop 2024-04-26T18:41:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fe969740 1 -- 172.21.15.84:0/2597950148 shutdown_connections 2024-04-26T18:41:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fe969740 1 --2- 172.21.15.84:0/2597950148 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5566efad75c0 0x5566efad99b0 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fe969740 1 --2- 172.21.15.84:0/2597950148 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5566efa600e0 0x5566efa64ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76d97fa640 0 client.17007 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fe969740 1 --2- 172.21.15.84:0/2597950148 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5566efa55820 0x5566efa68560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fe969740 1 --2- 172.21.15.84:0/2597950148 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5566efa54f00 0x5566efa68020 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.790 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fe969740 1 -- 172.21.15.84:0/2597950148 >> 172.21.15.84:0/2597950148 conn(0x5566efa51410 msgr2=0x5566efa57140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fe969740 1 -- 172.21.15.84:0/2597950148 shutdown_connections 2024-04-26T18:41:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fe969740 1 -- 172.21.15.84:0/2597950148 wait complete. 2024-04-26T18:41:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fe969740 20 client.17007 trim_cache size 0 max 16384 2024-04-26T18:41:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:41:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:41:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:41:21.791 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:41:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fbc52640 1 -- 172.21.15.84:0/1174743765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f76dc0a3dc0 msgr2=0x7f76dc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fbc52640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f76dc0a3dc0 0x7f76dc0a6280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f76ec0107a0 tx=0x7f76ec01d000 comp rx=0 tx=0).stop 2024-04-26T18:41:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fbc52640 1 -- 172.21.15.84:0/1174743765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76f4103bc0 msgr2=0x7f76f4101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fbc52640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76f4103bc0 0x7f76f4101810 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f76e400eab0 tx=0x7f76e400ef70 comp rx=0 tx=0).stop 2024-04-26T18:41:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fbc52640 1 -- 172.21.15.84:0/1174743765 shutdown_connections 2024-04-26T18:41:21.792 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fbc52640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f76dc0a3dc0 0x7f76dc0a6280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fbc52640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f76f410cf00 0x7f76f4102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fbc52640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f76f41044e0 0x7f76f4101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fbc52640 1 --2- 172.21.15.84:0/1174743765 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f76f4103bc0 0x7f76f4101810 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.789+0000 7f76fbc52640 1 -- 172.21.15.84:0/1174743765 >> 172.21.15.84:0/1174743765 conn(0x7f76f40789b0 msgr2=0x7f76f40fe920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.790+0000 7f76fbc52640 1 -- 172.21.15.84:0/1174743765 shutdown_connections 2024-04-26T18:41:21.793 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:21.790+0000 7f76fbc52640 1 -- 172.21.15.84:0/1174743765 wait complete. 2024-04-26T18:41:21.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:21 smithi084 ceph-mon[29141]: pgmap v420: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:41:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:21 smithi103 ceph-mon[32727]: pgmap v420: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:41:23.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:23 smithi084 ceph-mon[29141]: pgmap v421: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:41:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:23 smithi103 ceph-mon[32727]: pgmap v421: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:41:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:23 smithi174 ceph-mon[32705]: pgmap v421: 97 pgs: 97 active+clean; 1.8 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:41:25.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:25 smithi084 ceph-mon[29141]: pgmap v422: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 69 KiB/s wr, 26 op/s 2024-04-26T18:41:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:25 smithi103 ceph-mon[32727]: pgmap v422: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 69 KiB/s wr, 26 op/s 2024-04-26T18:41:25.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:25 smithi174 ceph-mon[32705]: pgmap v422: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 69 KiB/s wr, 26 op/s 2024-04-26T18:41:27.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:27 smithi084 ceph-mon[29141]: pgmap v423: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:41:27.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:27 smithi103 ceph-mon[32727]: pgmap v423: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:41:27.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:27 smithi174 ceph-mon[32705]: pgmap v423: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:41:29.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:29 smithi084 ceph-mon[29141]: pgmap v424: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:41:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:29 smithi103 ceph-mon[32727]: pgmap v424: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:41:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:29 smithi174 ceph-mon[32705]: pgmap v424: 97 pgs: 97 active+clean; 2.0 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:41:31.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:31 smithi084 ceph-mon[29141]: pgmap v425: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-26T18:41:31.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:31 smithi103 ceph-mon[32727]: pgmap v425: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-26T18:41:31.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:31 smithi174 ceph-mon[32705]: pgmap v425: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-26T18:41:33.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:33 smithi084 ceph-mon[29141]: pgmap v426: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:41:33.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:33 smithi103 ceph-mon[32727]: pgmap v426: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:41:33.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:33 smithi174 ceph-mon[32705]: pgmap v426: 97 pgs: 97 active+clean; 2.2 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:41:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.885+0000 7f355adbf640 1 -- 172.21.15.84:0/765504711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35541044e0 msgr2=0x7f355410c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.885+0000 7f355adbf640 1 --2- 172.21.15.84:0/765504711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35541044e0 0x7f355410c9c0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3540009900 tx=0x7f354002a510 comp rx=0 tx=0).stop 2024-04-26T18:41:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.885+0000 7f355adbf640 1 -- 172.21.15.84:0/765504711 shutdown_connections 2024-04-26T18:41:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.885+0000 7f355adbf640 1 --2- 172.21.15.84:0/765504711 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f355410cf00 0x7f355410f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.885+0000 7f355adbf640 1 --2- 172.21.15.84:0/765504711 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f35541044e0 0x7f355410c9c0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.885+0000 7f355adbf640 1 --2- 172.21.15.84:0/765504711 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3554103bc0 0x7f3554103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.885+0000 7f355adbf640 1 -- 172.21.15.84:0/765504711 >> 172.21.15.84:0/765504711 conn(0x7f35540789b0 msgr2=0x7f3554078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.885+0000 7f355adbf640 1 -- 172.21.15.84:0/765504711 shutdown_connections 2024-04-26T18:41:34.887 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.885+0000 7f355adbf640 1 -- 172.21.15.84:0/765504711 wait complete. 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.885+0000 7f355adbf640 1 Processor -- start 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.885+0000 7f355adbf640 1 -- start start 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f355adbf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3554103bc0 0x7f3554199050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f355adbf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35541044e0 0x7f3554199590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f355adbf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355410cf00 0x7f355419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f355adbf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3554112bb0 con 0x7f3554103bc0 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f355adbf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3554112a30 con 0x7f35541044e0 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f355adbf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3554112d30 con 0x7f355410cf00 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f3558b5b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3554103bc0 0x7f3554199050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f3558b5b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3554103bc0 0x7f3554199050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56434/0 (socket says 172.21.15.84:56434) 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f3558b5b640 1 -- 172.21.15.84:0/2761148910 learned_addr learned my addr 172.21.15.84:0/2761148910 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f355935c640 1 --2- 172.21.15.84:0/2761148910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355410cf00 0x7f355419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f3558b5b640 1 -- 172.21.15.84:0/2761148910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355410cf00 msgr2=0x7f355419d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f3558b5b640 1 --2- 172.21.15.84:0/2761148910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355410cf00 0x7f355419d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.888 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f3558b5b640 1 -- 172.21.15.84:0/2761148910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35541044e0 msgr2=0x7f3554199590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f3558b5b640 1 --2- 172.21.15.84:0/2761148910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35541044e0 0x7f3554199590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f3558b5b640 1 -- 172.21.15.84:0/2761148910 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f355419e040 con 0x7f3554103bc0 2024-04-26T18:41:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f3558b5b640 1 --2- 172.21.15.84:0/2761148910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3554103bc0 0x7f3554199050 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f354800c920 tx=0x7f354800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f3551ffb640 1 -- 172.21.15.84:0/2761148910 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548007c70 con 0x7f3554103bc0 2024-04-26T18:41:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f3551ffb640 1 -- 172.21.15.84:0/2761148910 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3548007e10 con 0x7f3554103bc0 2024-04-26T18:41:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f3551ffb640 1 -- 172.21.15.84:0/2761148910 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3548018820 con 0x7f3554103bc0 2024-04-26T18:41:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.886+0000 7f355adbf640 1 -- 172.21.15.84:0/2761148910 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f355419e330 con 0x7f3554103bc0 2024-04-26T18:41:34.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.887+0000 7f355adbf640 1 -- 172.21.15.84:0/2761148910 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35541a5bc0 con 0x7f3554103bc0 2024-04-26T18:41:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.889+0000 7f3551ffb640 1 -- 172.21.15.84:0/2761148910 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f354800f450 con 0x7f3554103bc0 2024-04-26T18:41:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.889+0000 7f3551ffb640 1 --2- 172.21.15.84:0/2761148910 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f35440a3dc0 0x7f35440a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.889+0000 7f355dbbb740 1 Processor -- start 2024-04-26T18:41:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.889+0000 7f355dbbb740 1 -- start start 2024-04-26T18:41:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.889+0000 7f3553fff640 1 --2- 172.21.15.84:0/2761148910 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f35440a3dc0 0x7f35440a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.889+0000 7f3551ffb640 1 -- 172.21.15.84:0/2761148910 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f35480d0a10 con 0x7f3554103bc0 2024-04-26T18:41:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.889+0000 7f355dbbb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 0x55862ec061e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.889+0000 7f355dbbb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec06720 0x55862ec14fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.889+0000 7f355dbbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec154f0 0x55862ec178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.889+0000 7f355dbbb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55862eb6c460 con 0x55862ec05e00 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.889+0000 7f355dbbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55862eb26590 con 0x55862ec154f0 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.889+0000 7f355dbbb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55862eb68420 con 0x55862ec06720 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f3558b5b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 0x55862ec061e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f3558b5b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 0x55862ec061e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56438/0 (socket says 172.21.15.84:56438) 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f3558b5b640 1 -- 172.21.15.84:0/1320340006 learned_addr learned my addr 172.21.15.84:0/1320340006 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f3553fff640 1 --2- 172.21.15.84:0/1320340006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec06720 0x55862ec14fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f3558b5b640 1 -- 172.21.15.84:0/1320340006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec06720 msgr2=0x55862ec14fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f3558b5b640 1 --2- 172.21.15.84:0/1320340006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec06720 0x55862ec14fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f3558b5b640 1 -- 172.21.15.84:0/1320340006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec154f0 msgr2=0x55862ec178e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f3558b5b640 1 --2- 172.21.15.84:0/1320340006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec154f0 0x55862ec178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f3558b5b640 1 -- 172.21.15.84:0/1320340006 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55862eb330d0 con 0x55862ec05e00 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f3553fff640 1 --2- 172.21.15.84:0/2761148910 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f35440a3dc0 0x7f35440a6280 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f35400096c0 tx=0x7f3540000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f3558b5b640 1 --2- 172.21.15.84:0/1320340006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 0x55862ec061e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3548098dd0 tx=0x7f354800dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3021e577c239715c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:34.897 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f353effd640 1 -- 172.21.15.84:0/1320340006 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35480a0d20 con 0x55862ec05e00 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f353effd640 1 -- 172.21.15.84:0/1320340006 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35480a55e0 con 0x55862ec05e00 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.890+0000 7f353effd640 1 -- 172.21.15.84:0/1320340006 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35480a5150 con 0x55862ec05e00 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.891+0000 7f355dbbb740 1 -- 172.21.15.84:0/1320340006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 msgr2=0x55862ec061e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.891+0000 7f355dbbb740 1 --2- 172.21.15.84:0/1320340006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 0x55862ec061e0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f3548098dd0 tx=0x7f354800dee0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.891+0000 7f355dbbb740 1 -- 172.21.15.84:0/1320340006 shutdown_connections 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.891+0000 7f355dbbb740 1 --2- 172.21.15.84:0/1320340006 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec154f0 0x55862ec178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.891+0000 7f355dbbb740 1 --2- 172.21.15.84:0/1320340006 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec06720 0x55862ec14fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.891+0000 7f355dbbb740 1 --2- 172.21.15.84:0/1320340006 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 0x55862ec061e0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.891+0000 7f355dbbb740 1 -- 172.21.15.84:0/1320340006 >> 172.21.15.84:0/1320340006 conn(0x55862ec02410 msgr2=0x55862ec027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.891+0000 7f355dbbb740 1 -- 172.21.15.84:0/1320340006 shutdown_connections 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.891+0000 7f355dbbb740 1 -- 172.21.15.84:0/1320340006 wait complete. 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.891+0000 7f355dbbb740 1 -- start start 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.891+0000 7f355dbbb740 1 Processor -- start 2024-04-26T18:41:34.898 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355dbbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec05e00 0x55862ec12470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355dbbb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec06720 0x55862ec129b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355dbbb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec154f0 0x55862ec12ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355dbbb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55862ec18120 con 0x55862ec154f0 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355dbbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55862eaf3b60 con 0x55862ec05e00 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355dbbb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55862ec182a0 con 0x55862ec06720 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355dbbb740 10 client.0 _ll_register_callbacks cb 0x55862eb182b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355935c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec154f0 0x55862ec12ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355935c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec154f0 0x55862ec12ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56446/0 (socket says 172.21.15.84:56446) 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355935c640 1 -- 172.21.15.84:0/277909004 learned_addr learned my addr 172.21.15.84:0/277909004 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f3553fff640 1 --2- 172.21.15.84:0/277909004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec06720 0x55862ec129b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f3558b5b640 1 --2- 172.21.15.84:0/277909004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec05e00 0x55862ec12470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355935c640 1 -- 172.21.15.84:0/277909004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec06720 msgr2=0x55862ec129b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355935c640 1 --2- 172.21.15.84:0/277909004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec06720 0x55862ec129b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.899 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355935c640 1 -- 172.21.15.84:0/277909004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec05e00 msgr2=0x55862ec12470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355935c640 1 --2- 172.21.15.84:0/277909004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec05e00 0x55862ec12470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355935c640 1 -- 172.21.15.84:0/277909004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55862e87e210 con 0x55862ec154f0 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f355935c640 1 --2- 172.21.15.84:0/277909004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec154f0 0x55862ec12ef0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f354c00e710 tx=0x7f354c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bf27b3fc934d62 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.892+0000 7f3533fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f3533fff640 1 -- 172.21.15.84:0/277909004 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354c003b70 con 0x55862ec154f0 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f3533fff640 1 -- 172.21.15.84:0/277909004 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f354c003d10 con 0x55862ec154f0 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f355dbbb740 1 -- 172.21.15.84:0/277909004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f35480a55e0 con 0x55862ec154f0 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f3533fff640 1 -- 172.21.15.84:0/277909004 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f354c00c8b0 con 0x55862ec154f0 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f355dbbb740 10 client.17019 fetch_fsmap learned FSMap version 29 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f355dbbb740 1 -- 172.21.15.84:0/277909004 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55862e94a620 con 0x55862ec154f0 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f3533fff640 1 -- 172.21.15.84:0/277909004 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f35480a55e0 con 0x55862ec154f0 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f3533fff640 1 -- 172.21.15.84:0/277909004 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f354c0104c0 con 0x55862ec154f0 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f355dbbb740 10 client.17019 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f355dbbb740 10 client.17019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:41:34.900 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f355dbbb740 20 client.17019 populate_metadata read hostname 'smithi084' 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f355dbbb740 1 --2- 172.21.15.84:0/277909004 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55862ec25780 0x55862ec27b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f355dbbb740 4 client.17019 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f355dbbb740 1 -- 172.21.15.84:0/277909004 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55862eb7a070 con 0x55862ec25780 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.893+0000 7f3558b5b640 1 --2- 172.21.15.84:0/277909004 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55862ec25780 0x55862ec27b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.894+0000 7f3558b5b640 1 --2- 172.21.15.84:0/277909004 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55862ec25780 0x55862ec27b70 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c64d1a72ca5036 server_cookie=b5aaff670422504f in_seq=0 out_seq=0 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.894+0000 7f3533fff640 10 client.17019 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f3533fff640 1 -- 172.21.15.84:0/277909004 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55862eb7a070 con 0x55862ec25780 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f3533fff640 10 client.17019 handle_command_reply: tid=0 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f355dbbb740 1 client.17019 shutdown 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f355dbbb740 1 -- 172.21.15.84:0/277909004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec154f0 msgr2=0x55862ec12ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f355dbbb740 1 --2- 172.21.15.84:0/277909004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec154f0 0x55862ec12ef0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f354c00e710 tx=0x7f354c00ebd0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f355dbbb740 1 -- 172.21.15.84:0/277909004 shutdown_connections 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f355dbbb740 1 --2- 172.21.15.84:0/277909004 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55862ec25780 0x55862ec27b70 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f355dbbb740 1 --2- 172.21.15.84:0/277909004 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec154f0 0x55862ec12ef0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f355dbbb740 1 --2- 172.21.15.84:0/277909004 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec06720 0x55862ec129b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.901 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f3533fff640 0 client.17019 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f355dbbb740 1 --2- 172.21.15.84:0/277909004 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec05e00 0x55862ec12470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f355dbbb740 1 -- 172.21.15.84:0/277909004 >> 172.21.15.84:0/277909004 conn(0x55862ec02410 msgr2=0x55862ec08280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f355dbbb740 1 -- 172.21.15.84:0/277909004 shutdown_connections 2024-04-26T18:41:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f355dbbb740 1 -- 172.21.15.84:0/277909004 wait complete. 2024-04-26T18:41:34.902 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.895+0000 7f355dbbb740 20 client.17019 trim_cache size 0 max 16384 2024-04-26T18:41:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f355dbbb740 1 Processor -- start 2024-04-26T18:41:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f355dbbb740 1 -- start start 2024-04-26T18:41:34.914 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f355dbbb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 0x55862ec20fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f355dbbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec06720 0x55862ec143b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f355dbbb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec154f0 0x55862ec2d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f355dbbb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f35541ac3e0 con 0x55862ec05e00 2024-04-26T18:41:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f355dbbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f35541adf20 con 0x55862ec06720 2024-04-26T18:41:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f355dbbb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35541acab0 con 0x55862ec154f0 2024-04-26T18:41:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f3558b5b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 0x55862ec20fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f3558b5b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 0x55862ec20fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56450/0 (socket says 172.21.15.84:56450) 2024-04-26T18:41:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f3558b5b640 1 -- 172.21.15.84:0/2044979015 learned_addr learned my addr 172.21.15.84:0/2044979015 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f3558b5b640 1 -- 172.21.15.84:0/2044979015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec154f0 msgr2=0x55862ec2d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f3558b5b640 1 --2- 172.21.15.84:0/2044979015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec154f0 0x55862ec2d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f3558b5b640 1 -- 172.21.15.84:0/2044979015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec06720 msgr2=0x55862ec143b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f3558b5b640 1 --2- 172.21.15.84:0/2044979015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec06720 0x55862ec143b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.915 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f3558b5b640 1 -- 172.21.15.84:0/2044979015 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55862ebfdd60 con 0x55862ec05e00 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f3558b5b640 1 --2- 172.21.15.84:0/2044979015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 0x55862ec20fe0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f354800cfd0 tx=0x7f3548020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83b90de1c5d3a85 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.910+0000 7f353e7fc640 1 -- 172.21.15.84:0/2044979015 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35480a55e0 con 0x55862ec05e00 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.911+0000 7f353e7fc640 1 -- 172.21.15.84:0/2044979015 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f354800fa90 con 0x55862ec05e00 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.911+0000 7f353e7fc640 1 -- 172.21.15.84:0/2044979015 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f35480a5150 con 0x55862ec05e00 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.911+0000 7f355dbbb740 1 -- 172.21.15.84:0/2044979015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 msgr2=0x55862ec20fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.911+0000 7f355dbbb740 1 --2- 172.21.15.84:0/2044979015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 0x55862ec20fe0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f354800cfd0 tx=0x7f3548020000 comp rx=0 tx=0).stop 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.911+0000 7f355dbbb740 1 -- 172.21.15.84:0/2044979015 shutdown_connections 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.911+0000 7f355dbbb740 1 --2- 172.21.15.84:0/2044979015 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec154f0 0x55862ec2d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.911+0000 7f355dbbb740 1 --2- 172.21.15.84:0/2044979015 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec06720 0x55862ec143b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.911+0000 7f355dbbb740 1 --2- 172.21.15.84:0/2044979015 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec05e00 0x55862ec20fe0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.911+0000 7f355dbbb740 1 -- 172.21.15.84:0/2044979015 >> 172.21.15.84:0/2044979015 conn(0x55862ec02410 msgr2=0x55862ec13ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.911+0000 7f355dbbb740 1 -- 172.21.15.84:0/2044979015 shutdown_connections 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.911+0000 7f355dbbb740 1 -- 172.21.15.84:0/2044979015 wait complete. 2024-04-26T18:41:34.916 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.911+0000 7f355dbbb740 1 -- start start 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f355dbbb740 1 Processor -- start 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f355dbbb740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec05e00 0x55862ec0f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f355dbbb740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec06720 0x55862ec0f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f355dbbb740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec154f0 0x55862ec0fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f355dbbb740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f35541ad850 con 0x55862ec06720 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f355dbbb740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f35541ad180 con 0x55862ec154f0 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f355dbbb740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f35541aa400 con 0x55862ec05e00 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f355dbbb740 10 client.0 _ll_register_callbacks cb 0x7f354c007440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f3553fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec06720 0x55862ec0f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f3553fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec06720 0x55862ec0f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:56460/0 (socket says 172.21.15.84:56460) 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f3553fff640 1 -- 172.21.15.84:0/3027655783 learned_addr learned my addr 172.21.15.84:0/3027655783 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f355935c640 1 --2- 172.21.15.84:0/3027655783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec154f0 0x55862ec0fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f3558b5b640 1 --2- 172.21.15.84:0/3027655783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec05e00 0x55862ec0f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f3553fff640 1 -- 172.21.15.84:0/3027655783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec05e00 msgr2=0x55862ec0f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:34.917 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f3553fff640 1 --2- 172.21.15.84:0/3027655783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec05e00 0x55862ec0f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f3553fff640 1 -- 172.21.15.84:0/3027655783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec154f0 msgr2=0x55862ec0fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f3553fff640 1 --2- 172.21.15.84:0/3027655783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec154f0 0x55862ec0fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.912+0000 7f3553fff640 1 -- 172.21.15.84:0/3027655783 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55862ec8af70 con 0x55862ec06720 2024-04-26T18:41:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f3553fff640 1 --2- 172.21.15.84:0/3027655783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec06720 0x55862ec0f8e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3540009900 tx=0x7f3540009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613d5b26016276f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f3533fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:41:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f3533fff640 1 -- 172.21.15.84:0/3027655783 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f354004c030 con 0x55862ec06720 2024-04-26T18:41:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f3533fff640 1 -- 172.21.15.84:0/3027655783 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f35400038c0 con 0x55862ec06720 2024-04-26T18:41:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f355dbbb740 1 -- 172.21.15.84:0/3027655783 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f354800fa90 con 0x55862ec06720 2024-04-26T18:41:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f3533fff640 1 -- 172.21.15.84:0/3027655783 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3540043070 con 0x55862ec06720 2024-04-26T18:41:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f355dbbb740 10 client.17025 fetch_fsmap learned FSMap version 29 2024-04-26T18:41:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f355dbbb740 1 -- 172.21.15.84:0/3027655783 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55862ec8c260 con 0x55862ec06720 2024-04-26T18:41:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f3533fff640 1 -- 172.21.15.84:0/3027655783 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f354800fa90 con 0x55862ec06720 2024-04-26T18:41:34.918 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f3533fff640 1 -- 172.21.15.84:0/3027655783 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f354003a040 con 0x55862ec06720 2024-04-26T18:41:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f355dbbb740 10 client.17025 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:41:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f355dbbb740 10 client.17025 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:41:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f355dbbb740 20 client.17025 populate_metadata read hostname 'smithi084' 2024-04-26T18:41:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f355dbbb740 1 --2- 172.21.15.84:0/3027655783 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55862ec8ca30 0x55862ec8ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f355dbbb740 4 client.17025 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:41:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.913+0000 7f355dbbb740 1 -- 172.21.15.84:0/3027655783 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55862eb73ee0 con 0x55862ec8ca30 2024-04-26T18:41:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.914+0000 7f3558b5b640 1 --2- 172.21.15.84:0/3027655783 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55862ec8ca30 0x55862ec8ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.914+0000 7f3558b5b640 1 --2- 172.21.15.84:0/3027655783 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55862ec8ca30 0x55862ec8ee50 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71f11642ce16757d server_cookie=6050550c72bf6481 in_seq=0 out_seq=0 2024-04-26T18:41:34.919 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.914+0000 7f3533fff640 10 client.17025 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f3533fff640 1 -- 172.21.15.84:0/3027655783 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55862eb73ee0 con 0x55862ec8ca30 2024-04-26T18:41:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f3533fff640 10 client.17025 handle_command_reply: tid=0 2024-04-26T18:41:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f355dbbb740 1 client.17025 shutdown 2024-04-26T18:41:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f355dbbb740 1 -- 172.21.15.84:0/3027655783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec06720 msgr2=0x55862ec0f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f355dbbb740 1 --2- 172.21.15.84:0/3027655783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec06720 0x55862ec0f8e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f3540009900 tx=0x7f3540009260 comp rx=0 tx=0).stop 2024-04-26T18:41:34.929 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f355dbbb740 1 -- 172.21.15.84:0/3027655783 shutdown_connections 2024-04-26T18:41:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f355dbbb740 1 --2- 172.21.15.84:0/3027655783 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55862ec8ca30 0x55862ec8ee50 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f355dbbb740 1 --2- 172.21.15.84:0/3027655783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55862ec154f0 0x55862ec0fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f355dbbb740 1 --2- 172.21.15.84:0/3027655783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55862ec06720 0x55862ec0f8e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f3533fff640 0 client.17025 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f355dbbb740 1 --2- 172.21.15.84:0/3027655783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55862ec05e00 0x55862ec0f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f355dbbb740 1 -- 172.21.15.84:0/3027655783 >> 172.21.15.84:0/3027655783 conn(0x55862ec02410 msgr2=0x55862ec12ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f355dbbb740 1 -- 172.21.15.84:0/3027655783 shutdown_connections 2024-04-26T18:41:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f355dbbb740 1 -- 172.21.15.84:0/3027655783 wait complete. 2024-04-26T18:41:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.929+0000 7f355dbbb740 20 client.17025 trim_cache size 0 max 16384 2024-04-26T18:41:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:41:34.930 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:41:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:41:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:41:34.931 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.930+0000 7f355adbf640 1 -- 172.21.15.84:0/2761148910 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f35440a3dc0 msgr2=0x7f35440a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.930+0000 7f355adbf640 1 --2- 172.21.15.84:0/2761148910 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f35440a3dc0 0x7f35440a6280 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f35400096c0 tx=0x7f3540000bc0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.930+0000 7f355adbf640 1 -- 172.21.15.84:0/2761148910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3554103bc0 msgr2=0x7f3554199050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.930+0000 7f355adbf640 1 --2- 172.21.15.84:0/2761148910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3554103bc0 0x7f3554199050 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f354800c920 tx=0x7f354800cde0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.930+0000 7f355adbf640 1 -- 172.21.15.84:0/2761148910 shutdown_connections 2024-04-26T18:41:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.930+0000 7f355adbf640 1 --2- 172.21.15.84:0/2761148910 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f35440a3dc0 0x7f35440a6280 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.930+0000 7f355adbf640 1 --2- 172.21.15.84:0/2761148910 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f355410cf00 0x7f355419d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.930+0000 7f355adbf640 1 --2- 172.21.15.84:0/2761148910 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f35541044e0 0x7f3554199590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.930+0000 7f355adbf640 1 --2- 172.21.15.84:0/2761148910 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3554103bc0 0x7f3554199050 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.930+0000 7f355adbf640 1 -- 172.21.15.84:0/2761148910 >> 172.21.15.84:0/2761148910 conn(0x7f35540789b0 msgr2=0x7f35540ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.930+0000 7f355adbf640 1 -- 172.21.15.84:0/2761148910 shutdown_connections 2024-04-26T18:41:34.932 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:34.930+0000 7f355adbf640 1 -- 172.21.15.84:0/2761148910 wait complete. 2024-04-26T18:41:35.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:35 smithi084 ceph-mon[29141]: pgmap v427: 97 pgs: 97 active+clean; 2.4 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-26T18:41:35.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:35 smithi103 ceph-mon[32727]: pgmap v427: 97 pgs: 97 active+clean; 2.4 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-26T18:41:35.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:35 smithi174 ceph-mon[32705]: pgmap v427: 97 pgs: 97 active+clean; 2.4 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-26T18:41:37.383 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 34.4 2024-04-26T18:41:37.384 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:41:37.638 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:41:37.762 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:37 smithi084 ceph-mon[29141]: pgmap v428: 97 pgs: 97 active+clean; 2.4 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:41:37.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:37 smithi103 ceph-mon[32727]: pgmap v428: 97 pgs: 97 active+clean; 2.4 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:41:37.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:37 smithi174 ceph-mon[32705]: pgmap v428: 97 pgs: 97 active+clean; 2.4 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:41:39.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.492+0000 7f9a90a06700 1 -- 172.21.15.84:0/1131521626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a8c102ce0 msgr2=0x7f9a8c10f1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:39.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.492+0000 7f9a90a06700 1 --2- 172.21.15.84:0/1131521626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a8c102ce0 0x7f9a8c10f1d0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400b0a0 tx=0x7f9a8400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:41:39.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.492+0000 7f9a90a06700 1 -- 172.21.15.84:0/1131521626 shutdown_connections 2024-04-26T18:41:39.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.492+0000 7f9a90a06700 1 --2- 172.21.15.84:0/1131521626 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a8c102ce0 0x7f9a8c10f1d0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:39.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.492+0000 7f9a90a06700 1 --2- 172.21.15.84:0/1131521626 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a8c108390 0x7f9a8c108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:39.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.492+0000 7f9a90a06700 1 --2- 172.21.15.84:0/1131521626 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a8c102320 0x7f9a8c1027a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:39.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.492+0000 7f9a90a06700 1 -- 172.21.15.84:0/1131521626 >> 172.21.15.84:0/1131521626 conn(0x7f9a8c0792d0 msgr2=0x7f9a8c0796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:39.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.492+0000 7f9a90a06700 1 -- 172.21.15.84:0/1131521626 shutdown_connections 2024-04-26T18:41:39.494 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.492+0000 7f9a90a06700 1 -- 172.21.15.84:0/1131521626 wait complete. 2024-04-26T18:41:39.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.493+0000 7f9a90a06700 1 Processor -- start 2024-04-26T18:41:39.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.493+0000 7f9a90a06700 1 -- start start 2024-04-26T18:41:39.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a90a06700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a8c102320 0x7f9a8c19a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:39.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a90a06700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a8c102ce0 0x7f9a8c19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:39.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a90a06700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a8c108390 0x7f9a8c1a5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:39.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a90a06700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c112530 con 0x7f9a8c108390 2024-04-26T18:41:39.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a90a06700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c1123b0 con 0x7f9a8c102320 2024-04-26T18:41:39.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a90a06700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9a8c1126b0 con 0x7f9a8c102ce0 2024-04-26T18:41:39.496 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a8ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a8c108390 0x7f9a8c1a5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a8ad9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a8c108390 0x7f9a8c1a5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51188/0 (socket says 172.21.15.84:51188) 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a8ad9d700 1 -- 172.21.15.84:0/2637807141 learned_addr learned my addr 172.21.15.84:0/2637807141 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a89d9b700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a8c102ce0 0x7f9a8c19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a8a59c700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a8c102320 0x7f9a8c19a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a8ad9d700 1 -- 172.21.15.84:0/2637807141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a8c102ce0 msgr2=0x7f9a8c19aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a8ad9d700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a8c102ce0 0x7f9a8c19aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a8ad9d700 1 -- 172.21.15.84:0/2637807141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a8c102320 msgr2=0x7f9a8c19a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a8ad9d700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a8c102320 0x7f9a8c19a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a8ad9d700 1 -- 172.21.15.84:0/2637807141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a8c1a5f50 con 0x7f9a8c108390 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a8ad9d700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a8c108390 0x7f9a8c1a5a10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400b830 tx=0x7f9a84004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a7b7fe700 1 -- 172.21.15.84:0/2637807141 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a84021070 con 0x7f9a8c108390 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a90a06700 1 -- 172.21.15.84:0/2637807141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a8c1a6120 con 0x7f9a8c108390 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a7b7fe700 1 -- 172.21.15.84:0/2637807141 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a84007e30 con 0x7f9a8c108390 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a7b7fe700 1 -- 172.21.15.84:0/2637807141 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a84004700 con 0x7f9a8c108390 2024-04-26T18:41:39.497 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.494+0000 7f9a90a06700 1 -- 172.21.15.84:0/2637807141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a8c1a6620 con 0x7f9a8c108390 2024-04-26T18:41:39.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.495+0000 7f9a90a06700 1 -- 172.21.15.84:0/2637807141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a8c10a900 con 0x7f9a8c108390 2024-04-26T18:41:39.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.496+0000 7f9a7b7fe700 1 -- 172.21.15.84:0/2637807141 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9a84007980 con 0x7f9a8c108390 2024-04-26T18:41:39.498 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.497+0000 7f9a7b7fe700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9a740a3e30 0x7f9a740a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:39.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.497+0000 7f9a8a59c700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9a740a3e30 0x7f9a740a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:39.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.497+0000 7f9a7b7fe700 1 -- 172.21.15.84:0/2637807141 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f9a8400f020 con 0x7f9a8c108390 2024-04-26T18:41:39.499 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.498+0000 7f9a8a59c700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9a740a3e30 0x7f9a740a62f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009a10 tx=0x7f9a7c009190 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:39.503 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.502+0000 7f9a7b7fe700 1 -- 172.21.15.84:0/2637807141 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a8402a020 con 0x7f9a8c108390 2024-04-26T18:41:39.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.752+0000 7f9a90a06700 1 -- 172.21.15.84:0/2637807141 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a8c06b710 con 0x7f9a8c108390 2024-04-26T18:41:39.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.753+0000 7f9a7b7fe700 1 -- 172.21.15.84:0/2637807141 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f9a8409e380 con 0x7f9a8c108390 2024-04-26T18:41:39.754 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:41:39.754 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:41:39.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.755+0000 7f9a90a06700 1 -- 172.21.15.84:0/2637807141 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9a740a3e30 msgr2=0x7f9a740a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:39.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.755+0000 7f9a90a06700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9a740a3e30 0x7f9a740a62f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9a7c009a10 tx=0x7f9a7c009190 comp rx=0 tx=0).stop 2024-04-26T18:41:39.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.755+0000 7f9a90a06700 1 -- 172.21.15.84:0/2637807141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a8c108390 msgr2=0x7f9a8c1a5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:39.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.755+0000 7f9a90a06700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a8c108390 0x7f9a8c1a5a10 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f9a8400b830 tx=0x7f9a84004ce0 comp rx=0 tx=0).stop 2024-04-26T18:41:39.756 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.755+0000 7f9a90a06700 1 -- 172.21.15.84:0/2637807141 shutdown_connections 2024-04-26T18:41:39.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.755+0000 7f9a90a06700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f9a740a3e30 0x7f9a740a62f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:39.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.755+0000 7f9a90a06700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9a8c108390 0x7f9a8c1a5a10 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:39.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.755+0000 7f9a90a06700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9a8c102320 0x7f9a8c19a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:39.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.755+0000 7f9a90a06700 1 --2- 172.21.15.84:0/2637807141 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f9a8c102ce0 0x7f9a8c19aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:39.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.755+0000 7f9a90a06700 1 -- 172.21.15.84:0/2637807141 >> 172.21.15.84:0/2637807141 conn(0x7f9a8c0792d0 msgr2=0x7f9a8c100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:39.757 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:39 smithi084 ceph-mon[29141]: pgmap v429: 97 pgs: 97 active+clean; 2.4 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:41:39.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.755+0000 7f9a90a06700 1 -- 172.21.15.84:0/2637807141 shutdown_connections 2024-04-26T18:41:39.757 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:39.755+0000 7f9a90a06700 1 -- 172.21.15.84:0/2637807141 wait complete. 2024-04-26T18:41:39.758 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:41:39.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:39 smithi103 ceph-mon[32727]: pgmap v429: 97 pgs: 97 active+clean; 2.4 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:41:39.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:39 smithi174 ceph-mon[32705]: pgmap v429: 97 pgs: 97 active+clean; 2.4 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:41:40.436 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 60 --expiration 94.4 --await-for 42.0' 2024-04-26T18:41:40.436 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db / --timeout 60 --expiration 94.4 --await-for 42.0 2024-04-26T18:41:40.686 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:41:40.808 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:40 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2637807141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:41:40.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:40 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2637807141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:41:40.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:40 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2637807141' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:41:41.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:41 smithi084 ceph-mon[29141]: pgmap v430: 97 pgs: 97 active+clean; 2.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-26T18:41:41.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:41 smithi103 ceph-mon[32727]: pgmap v430: 97 pgs: 97 active+clean; 2.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-26T18:41:41.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:41 smithi174 ceph-mon[32705]: pgmap v430: 97 pgs: 97 active+clean; 2.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 60 KiB/s wr, 22 op/s 2024-04-26T18:41:42.405 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.405+0000 7f65b2a77700 1 -- 172.21.15.84:0/3399837856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ac102d00 msgr2=0x7f65ac10f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:42.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.405+0000 7f65b2a77700 1 --2- 172.21.15.84:0/3399837856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ac102d00 0x7f65ac10f1f0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f65a400b0a0 tx=0x7f65a400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.405+0000 7f65b2a77700 1 -- 172.21.15.84:0/3399837856 shutdown_connections 2024-04-26T18:41:42.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.405+0000 7f65b2a77700 1 --2- 172.21.15.84:0/3399837856 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ac102d00 0x7f65ac10f1f0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.405+0000 7f65b2a77700 1 --2- 172.21.15.84:0/3399837856 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ac1083f0 0x7f65ac1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.405+0000 7f65b2a77700 1 --2- 172.21.15.84:0/3399837856 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ac102340 0x7f65ac1027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.405+0000 7f65b2a77700 1 -- 172.21.15.84:0/3399837856 >> 172.21.15.84:0/3399837856 conn(0x7f65ac079340 msgr2=0x7f65ac079750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:42.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.405+0000 7f65b2a77700 1 -- 172.21.15.84:0/3399837856 shutdown_connections 2024-04-26T18:41:42.406 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.405+0000 7f65b2a77700 1 -- 172.21.15.84:0/3399837856 wait complete. 2024-04-26T18:41:42.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.406+0000 7f65b2a77700 1 Processor -- start 2024-04-26T18:41:42.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.406+0000 7f65b2a77700 1 -- start start 2024-04-26T18:41:42.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.406+0000 7f65b2a77700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ac102340 0x7f65ac19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.406+0000 7f65b2a77700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ac102d00 0x7f65ac19fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.406+0000 7f65b2a77700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ac1083f0 0x7f65ac1997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.406+0000 7f65b2a77700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac112560 con 0x7f65ac1083f0 2024-04-26T18:41:42.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.406+0000 7f65b2a77700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1123e0 con 0x7f65ac102340 2024-04-26T18:41:42.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.406+0000 7f65b2a77700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1126e0 con 0x7f65ac102d00 2024-04-26T18:41:42.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.406+0000 7f65b1014700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ac1083f0 0x7f65ac1997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.408 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.406+0000 7f65b1014700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ac1083f0 0x7f65ac1997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51206/0 (socket says 172.21.15.84:51206) 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.406+0000 7f65b1014700 1 -- 172.21.15.84:0/3419074599 learned_addr learned my addr 172.21.15.84:0/3419074599 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.406+0000 7f65b0813700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ac102340 0x7f65ac19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.406+0000 7f65abfff700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ac102d00 0x7f65ac19fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.407+0000 7f65b1014700 1 -- 172.21.15.84:0/3419074599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ac102d00 msgr2=0x7f65ac19fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.407+0000 7f65b1014700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ac102d00 0x7f65ac19fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.407+0000 7f65b1014700 1 -- 172.21.15.84:0/3419074599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ac102340 msgr2=0x7f65ac19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.407+0000 7f65b1014700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ac102340 0x7f65ac19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.407+0000 7f65b1014700 1 -- 172.21.15.84:0/3419074599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac199fd0 con 0x7f65ac1083f0 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.407+0000 7f65abfff700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ac102d00 0x7f65ac19fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.407+0000 7f65b1014700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ac1083f0 0x7f65ac1997f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f65a400b830 tx=0x7f65a4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.407+0000 7f65a9ffb700 1 -- 172.21.15.84:0/3419074599 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a4021070 con 0x7f65ac1083f0 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.407+0000 7f65a9ffb700 1 -- 172.21.15.84:0/3419074599 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a4007b80 con 0x7f65ac1083f0 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.407+0000 7f65b2a77700 1 -- 172.21.15.84:0/3419074599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ac19a260 con 0x7f65ac1083f0 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.407+0000 7f65a9ffb700 1 -- 172.21.15.84:0/3419074599 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a40052a0 con 0x7f65ac1083f0 2024-04-26T18:41:42.409 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.407+0000 7f65b2a77700 1 -- 172.21.15.84:0/3419074599 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ac1a64d0 con 0x7f65ac1083f0 2024-04-26T18:41:42.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.410+0000 7f65a9ffb700 1 -- 172.21.15.84:0/3419074599 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f65a4024070 con 0x7f65ac1083f0 2024-04-26T18:41:42.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.410+0000 7f65a9ffb700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f659c0a3e50 0x7f659c0a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.410+0000 7f65b0813700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f659c0a3e50 0x7f659c0a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.410 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.410+0000 7f65a9ffb700 1 -- 172.21.15.84:0/3419074599 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f65a400f020 con 0x7f65ac1083f0 2024-04-26T18:41:42.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65bd309b80 1 Processor -- start 2024-04-26T18:41:42.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65b0813700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f659c0a3e50 0x7f659c0a6310 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f65a0005950 tx=0x7f65a000b410 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:42.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65bd309b80 1 -- start start 2024-04-26T18:41:42.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65bd309b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 0x56541614f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65bd309b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541614fb50 0x56541615e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65bd309b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541615ee10 0x565416161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.412 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65bd309b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5654160a28d0 con 0x56541614f230 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65bd309b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565416041770 con 0x56541614fb50 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65bd309b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56541609e890 con 0x56541615ee10 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65b0813700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 0x56541614f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65b0813700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 0x56541614f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51220/0 (socket says 172.21.15.84:51220) 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65b0813700 1 -- 172.21.15.84:0/3512901901 learned_addr learned my addr 172.21.15.84:0/3512901901 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65abfff700 1 --2- 172.21.15.84:0/3512901901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541614fb50 0x56541615e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65b1014700 1 --2- 172.21.15.84:0/3512901901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541615ee10 0x565416161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65b0813700 1 -- 172.21.15.84:0/3512901901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541615ee10 msgr2=0x565416161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65b0813700 1 --2- 172.21.15.84:0/3512901901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541615ee10 0x565416161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65b0813700 1 -- 172.21.15.84:0/3512901901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541614fb50 msgr2=0x56541615e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.411+0000 7f65b0813700 1 --2- 172.21.15.84:0/3512901901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541614fb50 0x56541615e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.412+0000 7f65b0813700 1 -- 172.21.15.84:0/3512901901 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565415fefa60 con 0x56541614f230 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.412+0000 7f65b1014700 1 --2- 172.21.15.84:0/3512901901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541615ee10 0x565416161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:41:42.413 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.412+0000 7f65abfff700 1 --2- 172.21.15.84:0/3512901901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541614fb50 0x56541615e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:41:42.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.412+0000 7f65b0813700 1 --2- 172.21.15.84:0/3512901901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 0x56541614f610 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f65a0010fa0 tx=0x7f65a0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8599a4d4bdc4b782 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:42.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.412+0000 7f6596ffd700 1 -- 172.21.15.84:0/3512901901 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a000be60 con 0x56541614f230 2024-04-26T18:41:42.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.412+0000 7f6596ffd700 1 -- 172.21.15.84:0/3512901901 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a001ead0 con 0x56541614f230 2024-04-26T18:41:42.414 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.412+0000 7f6596ffd700 1 -- 172.21.15.84:0/3512901901 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a001cc10 con 0x56541614f230 2024-04-26T18:41:42.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.413+0000 7f65bd309b80 1 -- 172.21.15.84:0/3512901901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 msgr2=0x56541614f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:42.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.413+0000 7f65bd309b80 1 --2- 172.21.15.84:0/3512901901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 0x56541614f610 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f65a0010fa0 tx=0x7f65a0013000 comp rx=0 tx=0).stop 2024-04-26T18:41:42.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.413+0000 7f65bd309b80 1 -- 172.21.15.84:0/3512901901 shutdown_connections 2024-04-26T18:41:42.417 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.413+0000 7f65bd309b80 1 --2- 172.21.15.84:0/3512901901 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 0x56541614f610 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.413+0000 7f65bd309b80 1 --2- 172.21.15.84:0/3512901901 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541615ee10 0x565416161200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.413+0000 7f65bd309b80 1 --2- 172.21.15.84:0/3512901901 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541614fb50 0x56541615e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.413+0000 7f65bd309b80 1 -- 172.21.15.84:0/3512901901 >> 172.21.15.84:0/3512901901 conn(0x56541607ced0 msgr2=0x56541614b720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.413+0000 7f65bd309b80 1 -- 172.21.15.84:0/3512901901 shutdown_connections 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.413+0000 7f65bd309b80 1 -- 172.21.15.84:0/3512901901 wait complete. 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.413+0000 7f65bd309b80 1 -- start start 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.413+0000 7f65bd309b80 1 Processor -- start 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65bd309b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541614f230 0x56541615e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65bd309b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 0x56541615b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65bd309b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541615ee10 0x56541615bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65bd309b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5654161619e0 con 0x56541614fb50 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65bd309b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x565416008ae0 con 0x56541615ee10 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65bd309b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x565416161b60 con 0x56541614f230 2024-04-26T18:41:42.418 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65bd309b80 10 client.0 _ll_register_callbacks cb 0x565416036710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65abfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 0x56541615b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65abfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 0x56541615b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51236/0 (socket says 172.21.15.84:51236) 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65abfff700 1 -- 172.21.15.84:0/929841464 learned_addr learned my addr 172.21.15.84:0/929841464 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65b0813700 1 --2- 172.21.15.84:0/929841464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541614f230 0x56541615e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65b1014700 1 --2- 172.21.15.84:0/929841464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541615ee10 0x56541615bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65abfff700 1 -- 172.21.15.84:0/929841464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541614f230 msgr2=0x56541615e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65abfff700 1 --2- 172.21.15.84:0/929841464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541614f230 0x56541615e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65abfff700 1 -- 172.21.15.84:0/929841464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541615ee10 msgr2=0x56541615bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65abfff700 1 --2- 172.21.15.84:0/929841464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541615ee10 0x56541615bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65abfff700 1 -- 172.21.15.84:0/929841464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56541615c470 con 0x56541614fb50 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65abfff700 1 --2- 172.21.15.84:0/929841464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 0x56541615b880 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f659800f290 tx=0x7f659800f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a95a86a24723fbe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f658bfff700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f658bfff700 1 -- 172.21.15.84:0/929841464 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598018020 con 0x56541614fb50 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f658bfff700 1 -- 172.21.15.84:0/929841464 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f6598014660 con 0x56541614fb50 2024-04-26T18:41:42.419 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.414+0000 7f65bd309b80 1 -- 172.21.15.84:0/929841464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a001ead0 con 0x56541614fb50 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.415+0000 7f658bfff700 1 -- 172.21.15.84:0/929841464 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6598023050 con 0x56541614fb50 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.415+0000 7f65bd309b80 10 client.17043 fetch_fsmap learned FSMap version 29 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.415+0000 7f65bd309b80 1 -- 172.21.15.84:0/929841464 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56541616c4c0 con 0x56541614fb50 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.415+0000 7f658bfff700 1 -- 172.21.15.84:0/929841464 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a001ead0 con 0x56541614fb50 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.415+0000 7f658bfff700 1 -- 172.21.15.84:0/929841464 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f65980138f0 con 0x56541614fb50 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.415+0000 7f65bd309b80 10 client.17043 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.415+0000 7f65bd309b80 10 client.17043 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.415+0000 7f65bd309b80 20 client.17043 populate_metadata read hostname 'smithi084' 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.415+0000 7f65bd309b80 1 --2- 172.21.15.84:0/929841464 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56541616eb50 0x565416170f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.415+0000 7f65bd309b80 4 client.17043 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.415+0000 7f65bd309b80 1 -- 172.21.15.84:0/929841464 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654160b02b0 con 0x56541616eb50 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.415+0000 7f65b0813700 1 --2- 172.21.15.84:0/929841464 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56541616eb50 0x565416170f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.416+0000 7f658bfff700 10 client.17043 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.416+0000 7f65b0813700 1 --2- 172.21.15.84:0/929841464 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56541616eb50 0x565416170f40 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b97c58c041dcb9 server_cookie=e361cbf7c5a8576a in_seq=0 out_seq=0 2024-04-26T18:41:42.420 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.416+0000 7f658bfff700 1 -- 172.21.15.84:0/929841464 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654160b02b0 con 0x56541616eb50 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.416+0000 7f658bfff700 10 client.17043 handle_command_reply: tid=0 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.417+0000 7f65bd309b80 1 client.17043 shutdown 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.417+0000 7f65bd309b80 1 -- 172.21.15.84:0/929841464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 msgr2=0x56541615b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.417+0000 7f65bd309b80 1 --2- 172.21.15.84:0/929841464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 0x56541615b880 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f659800f290 tx=0x7f659800f5a0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.417+0000 7f65bd309b80 1 -- 172.21.15.84:0/929841464 shutdown_connections 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.417+0000 7f65bd309b80 1 --2- 172.21.15.84:0/929841464 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56541616eb50 0x565416170f40 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.417+0000 7f65bd309b80 1 --2- 172.21.15.84:0/929841464 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541615ee10 0x56541615bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.417+0000 7f65bd309b80 1 --2- 172.21.15.84:0/929841464 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541614f230 0x56541615e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.417+0000 7f65bd309b80 1 --2- 172.21.15.84:0/929841464 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 0x56541615b880 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.417+0000 7f658bfff700 0 client.17043 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.417+0000 7f65bd309b80 1 -- 172.21.15.84:0/929841464 >> 172.21.15.84:0/929841464 conn(0x56541607ced0 msgr2=0x56541614b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.417+0000 7f65bd309b80 1 -- 172.21.15.84:0/929841464 shutdown_connections 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.417+0000 7f65bd309b80 1 -- 172.21.15.84:0/929841464 wait complete. 2024-04-26T18:41:42.421 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.417+0000 7f65bd309b80 20 client.17043 trim_cache size 0 max 16384 2024-04-26T18:41:42.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65bd309b80 1 Processor -- start 2024-04-26T18:41:42.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65bd309b80 1 -- start start 2024-04-26T18:41:42.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65bd309b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 0x56541614b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65bd309b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541614fb50 0x565416157100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65bd309b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541615ee10 0x56541615d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65bd309b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1aba50 con 0x56541614f230 2024-04-26T18:41:42.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65bd309b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1ad590 con 0x56541614fb50 2024-04-26T18:41:42.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65bd309b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1ac120 con 0x56541615ee10 2024-04-26T18:41:42.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65b0813700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 0x56541614b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65b0813700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 0x56541614b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51242/0 (socket says 172.21.15.84:51242) 2024-04-26T18:41:42.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65b0813700 1 -- 172.21.15.84:0/1041123154 learned_addr learned my addr 172.21.15.84:0/1041123154 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:42.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65b1014700 1 --2- 172.21.15.84:0/1041123154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541615ee10 0x56541615d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65b0813700 1 -- 172.21.15.84:0/1041123154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541615ee10 msgr2=0x56541615d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:42.438 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65b0813700 1 --2- 172.21.15.84:0/1041123154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541615ee10 0x56541615d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65b0813700 1 -- 172.21.15.84:0/1041123154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541614fb50 msgr2=0x565416157100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:42.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65b0813700 1 --2- 172.21.15.84:0/1041123154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541614fb50 0x565416157100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65b0813700 1 -- 172.21.15.84:0/1041123154 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565416150cc0 con 0x56541614f230 2024-04-26T18:41:42.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65b1014700 1 --2- 172.21.15.84:0/1041123154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541615ee10 0x56541615d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:41:42.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65b0813700 1 --2- 172.21.15.84:0/1041123154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 0x56541614b8d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f65a0010f70 tx=0x7f65a001c6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cab40870b866d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:42.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65967fc700 1 -- 172.21.15.84:0/1041123154 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a0005300 con 0x56541614f230 2024-04-26T18:41:42.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65967fc700 1 -- 172.21.15.84:0/1041123154 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a003b070 con 0x56541614f230 2024-04-26T18:41:42.439 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.437+0000 7f65967fc700 1 -- 172.21.15.84:0/1041123154 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a001cb00 con 0x56541614f230 2024-04-26T18:41:42.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.438+0000 7f65bd309b80 1 -- 172.21.15.84:0/1041123154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 msgr2=0x56541614b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:42.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.438+0000 7f65bd309b80 1 --2- 172.21.15.84:0/1041123154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 0x56541614b8d0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f65a0010f70 tx=0x7f65a001c6d0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.438+0000 7f65bd309b80 1 -- 172.21.15.84:0/1041123154 shutdown_connections 2024-04-26T18:41:42.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.438+0000 7f65bd309b80 1 --2- 172.21.15.84:0/1041123154 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614f230 0x56541614b8d0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.438+0000 7f65bd309b80 1 --2- 172.21.15.84:0/1041123154 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541615ee10 0x56541615d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.442 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.438+0000 7f65bd309b80 1 --2- 172.21.15.84:0/1041123154 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541614fb50 0x565416157100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.438+0000 7f65bd309b80 1 -- 172.21.15.84:0/1041123154 >> 172.21.15.84:0/1041123154 conn(0x56541607ced0 msgr2=0x56541615d0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.438+0000 7f65bd309b80 1 -- 172.21.15.84:0/1041123154 shutdown_connections 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.438+0000 7f65bd309b80 1 -- 172.21.15.84:0/1041123154 wait complete. 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.438+0000 7f65bd309b80 1 -- start start 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65bd309b80 1 Processor -- start 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65bd309b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541614f230 0x565416166df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65bd309b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 0x565416167330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65bd309b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541615ee10 0x565416163f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65bd309b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1acec0 con 0x56541614fb50 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65bd309b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1ac7f0 con 0x56541615ee10 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65bd309b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac1a9a10 con 0x56541614f230 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65bd309b80 10 client.0 _ll_register_callbacks cb 0x565416036710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65abfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 0x565416167330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.443 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65abfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 0x565416167330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51244/0 (socket says 172.21.15.84:51244) 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65b0813700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541614f230 0x565416166df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65abfff700 1 -- 172.21.15.84:0/2004980808 learned_addr learned my addr 172.21.15.84:0/2004980808 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65b1014700 1 --2- 172.21.15.84:0/2004980808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541615ee10 0x565416163f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65abfff700 1 -- 172.21.15.84:0/2004980808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541614f230 msgr2=0x565416166df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65abfff700 1 --2- 172.21.15.84:0/2004980808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541614f230 0x565416166df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65abfff700 1 -- 172.21.15.84:0/2004980808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541615ee10 msgr2=0x565416163f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65abfff700 1 --2- 172.21.15.84:0/2004980808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541615ee10 0x565416163f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65abfff700 1 -- 172.21.15.84:0/2004980808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565416164780 con 0x56541614fb50 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.439+0000 7f65b1014700 1 --2- 172.21.15.84:0/2004980808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541615ee10 0x565416163f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f65abfff700 1 --2- 172.21.15.84:0/2004980808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 0x565416167330 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6598013d10 tx=0x7f659800fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c93447b069cf6965 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f658bfff700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f658bfff700 1 -- 172.21.15.84:0/2004980808 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6598018070 con 0x56541614fb50 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f658bfff700 1 -- 172.21.15.84:0/2004980808 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f65980133c0 con 0x56541614fb50 2024-04-26T18:41:42.444 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f65bd309b80 1 -- 172.21.15.84:0/2004980808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a003b070 con 0x56541614fb50 2024-04-26T18:41:42.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f658bfff700 1 -- 172.21.15.84:0/2004980808 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6598035050 con 0x56541614fb50 2024-04-26T18:41:42.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f65bd309b80 10 client.17049 fetch_fsmap learned FSMap version 29 2024-04-26T18:41:42.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f65bd309b80 1 -- 172.21.15.84:0/2004980808 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654161d5c80 con 0x56541614fb50 2024-04-26T18:41:42.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f658bfff700 1 -- 172.21.15.84:0/2004980808 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65a003b070 con 0x56541614fb50 2024-04-26T18:41:42.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f658bfff700 1 -- 172.21.15.84:0/2004980808 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f659801ca50 con 0x56541614fb50 2024-04-26T18:41:42.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f65bd309b80 10 client.17049 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:41:42.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f65bd309b80 10 client.17049 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:41:42.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f65bd309b80 20 client.17049 populate_metadata read hostname 'smithi084' 2024-04-26T18:41:42.445 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f65bd309b80 1 --2- 172.21.15.84:0/2004980808 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5654161d6420 0x5654161d8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:42.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f65bd309b80 4 client.17049 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 94.4, "await_for": 42.0}] 2024-04-26T18:41:42.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.440+0000 7f65bd309b80 1 -- 172.21.15.84:0/2004980808 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 94.4, "await_for": 42.0}) -- 0x5654160aa350 con 0x5654161d6420 2024-04-26T18:41:42.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.441+0000 7f65b0813700 1 --2- 172.21.15.84:0/2004980808 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5654161d6420 0x5654161d8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:42.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.441+0000 7f658bfff700 10 client.17049 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:42.446 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.441+0000 7f65b0813700 1 --2- 172.21.15.84:0/2004980808 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5654161d6420 0x5654161d8840 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7b614b7273f61b6 server_cookie=d09e874bed126673 in_seq=0 out_seq=0 2024-04-26T18:41:42.457 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f658bfff700 1 -- 172.21.15.84:0/2004980808 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5654160aa350 con 0x5654161d6420 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f658bfff700 10 client.17049 handle_command_reply: tid=0 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f65bd309b80 1 client.17049 shutdown 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f65bd309b80 1 -- 172.21.15.84:0/2004980808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 msgr2=0x565416167330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f65bd309b80 1 --2- 172.21.15.84:0/2004980808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 0x565416167330 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f6598013d10 tx=0x7f659800fee0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f65bd309b80 1 -- 172.21.15.84:0/2004980808 shutdown_connections 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f65bd309b80 1 --2- 172.21.15.84:0/2004980808 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5654161d6420 0x5654161d8840 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f65bd309b80 1 --2- 172.21.15.84:0/2004980808 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56541615ee10 0x565416163f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f65bd309b80 1 --2- 172.21.15.84:0/2004980808 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56541614f230 0x565416166df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f65bd309b80 1 --2- 172.21.15.84:0/2004980808 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56541614fb50 0x565416167330 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f65bd309b80 1 -- 172.21.15.84:0/2004980808 >> 172.21.15.84:0/2004980808 conn(0x56541607ced0 msgr2=0x565416151a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f658bfff700 0 client.17049 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f65bd309b80 1 -- 172.21.15.84:0/2004980808 shutdown_connections 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f65bd309b80 1 -- 172.21.15.84:0/2004980808 wait complete. 2024-04-26T18:41:42.458 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.457+0000 7f65bd309b80 20 client.17049 trim_cache size 0 max 16384 2024-04-26T18:41:42.459 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:41:42.459 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:41:42.459 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:41:42.459 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 38, 2024-04-26T18:41:42.459 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:41:42.459 INFO:teuthology.orchestra.run.smithi084.stdout: "fa6dff99": { 2024-04-26T18:41:42.459 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 38, 2024-04-26T18:41:42.459 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:41:42.459 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:41:42.459 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:41:42.459 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 94.4, 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:41:42.460 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:41:42.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.458+0000 7f65b2a77700 1 -- 172.21.15.84:0/3419074599 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f659c0a3e50 msgr2=0x7f659c0a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:42.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.458+0000 7f65b2a77700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f659c0a3e50 0x7f659c0a6310 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f65a0005950 tx=0x7f65a000b410 comp rx=0 tx=0).stop 2024-04-26T18:41:42.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.458+0000 7f65b2a77700 1 -- 172.21.15.84:0/3419074599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ac1083f0 msgr2=0x7f65ac1997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:42.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.458+0000 7f65b2a77700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ac1083f0 0x7f65ac1997f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f65a400b830 tx=0x7f65a4004d00 comp rx=0 tx=0).stop 2024-04-26T18:41:42.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.458+0000 7f65b2a77700 1 -- 172.21.15.84:0/3419074599 shutdown_connections 2024-04-26T18:41:42.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.458+0000 7f65b2a77700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f659c0a3e50 0x7f659c0a6310 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.458+0000 7f65b2a77700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f65ac1083f0 0x7f65ac1997f0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.458+0000 7f65b2a77700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f65ac102340 0x7f65ac19f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.458+0000 7f65b2a77700 1 --2- 172.21.15.84:0/3419074599 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f65ac102d00 0x7f65ac19fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:42.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.458+0000 7f65b2a77700 1 -- 172.21.15.84:0/3419074599 >> 172.21.15.84:0/3419074599 conn(0x7f65ac079340 msgr2=0x7f65ac1015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:42.461 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.458+0000 7f65b2a77700 1 -- 172.21.15.84:0/3419074599 shutdown_connections 2024-04-26T18:41:42.462 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:41:42.458+0000 7f65b2a77700 1 -- 172.21.15.84:0/3419074599 wait complete. 2024-04-26T18:41:42.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:42 smithi084 ceph-mon[29141]: pgmap v431: 97 pgs: 97 active+clean; 2.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:41:42.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:42 smithi103 ceph-mon[32727]: pgmap v431: 97 pgs: 97 active+clean; 2.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:41:42.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:42 smithi174 ceph-mon[32705]: pgmap v431: 97 pgs: 97 active+clean; 2.7 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:41:43.387 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'fa6dff99', quiesce took 6.0 seconds. Will release after: 34.4 2024-04-26T18:41:45.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:44 smithi084 ceph-mon[29141]: pgmap v432: 97 pgs: 97 active+clean; 2.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-26T18:41:45.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:44 smithi103 ceph-mon[32727]: pgmap v432: 97 pgs: 97 active+clean; 2.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-26T18:41:45.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:44 smithi174 ceph-mon[32705]: pgmap v432: 97 pgs: 97 active+clean; 2.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 57 KiB/s wr, 21 op/s 2024-04-26T18:41:47.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:46 smithi084 ceph-mon[29141]: pgmap v433: 97 pgs: 97 active+clean; 2.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:41:47.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:46 smithi103 ceph-mon[32727]: pgmap v433: 97 pgs: 97 active+clean; 2.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:41:47.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:46 smithi174 ceph-mon[32705]: pgmap v433: 97 pgs: 97 active+clean; 2.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:41:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.026+0000 7fe2003a4640 1 -- 172.21.15.84:0/100561321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1f8107820 msgr2=0x7fe1f8107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.026+0000 7fe2003a4640 1 --2- 172.21.15.84:0/100561321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1f8107820 0x7fe1f8107c00 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009930 tx=0x7fe1f402a420 comp rx=0 tx=0).stop 2024-04-26T18:41:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.026+0000 7fe2003a4640 1 -- 172.21.15.84:0/100561321 shutdown_connections 2024-04-26T18:41:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.026+0000 7fe2003a4640 1 --2- 172.21.15.84:0/100561321 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1f8102220 0x7fe1f810e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.026+0000 7fe2003a4640 1 --2- 172.21.15.84:0/100561321 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1f8101860 0x7fe1f8101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.026+0000 7fe2003a4640 1 --2- 172.21.15.84:0/100561321 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1f8107820 0x7fe1f8107c00 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.029 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.026+0000 7fe2003a4640 1 -- 172.21.15.84:0/100561321 >> 172.21.15.84:0/100561321 conn(0x7fe1f80789b0 msgr2=0x7fe1f8078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.026+0000 7fe2003a4640 1 -- 172.21.15.84:0/100561321 shutdown_connections 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.026+0000 7fe2003a4640 1 -- 172.21.15.84:0/100561321 wait complete. 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.027+0000 7fe2003a4640 1 Processor -- start 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.027+0000 7fe2003a4640 1 -- start start 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.027+0000 7fe2003a4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1f8101860 0x7fe1f80fe7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.027+0000 7fe2003a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1f8102220 0x7fe1f80fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.027+0000 7fe2003a4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1f8107820 0x7fe1f80ff270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.027+0000 7fe2003a4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8111aa0 con 0x7fe1f8102220 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.027+0000 7fe2003a4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8111920 con 0x7fe1f8107820 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.027+0000 7fe2003a4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f8111c20 con 0x7fe1f8101860 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.027+0000 7fe1fd93f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1f8102220 0x7fe1f80fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.027+0000 7fe1fd93f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1f8102220 0x7fe1f80fed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34180/0 (socket says 172.21.15.84:34180) 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.027+0000 7fe1fd93f640 1 -- 172.21.15.84:0/2692330907 learned_addr learned my addr 172.21.15.84:0/2692330907 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:48.030 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.027+0000 7fe1fe140640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1f8101860 0x7fe1f80fe7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.028+0000 7fe1fe941640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1f8107820 0x7fe1f80ff270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.028+0000 7fe1fd93f640 1 -- 172.21.15.84:0/2692330907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1f8101860 msgr2=0x7fe1f80fe7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.028+0000 7fe1fd93f640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1f8101860 0x7fe1f80fe7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.028+0000 7fe1fd93f640 1 -- 172.21.15.84:0/2692330907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1f8107820 msgr2=0x7fe1f80ff270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.028+0000 7fe1fd93f640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1f8107820 0x7fe1f80ff270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.028+0000 7fe1fd93f640 1 -- 172.21.15.84:0/2692330907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1f81a75d0 con 0x7fe1f8102220 2024-04-26T18:41:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.028+0000 7fe1fe941640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1f8107820 0x7fe1f80ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:41:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.028+0000 7fe1fd93f640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1f8102220 0x7fe1f80fed30 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800d950 tx=0x7fe1e800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.028+0000 7fe1ef7fe640 1 -- 172.21.15.84:0/2692330907 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8017070 con 0x7fe1f8102220 2024-04-26T18:41:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.028+0000 7fe2003a4640 1 -- 172.21.15.84:0/2692330907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1f81a78c0 con 0x7fe1f8102220 2024-04-26T18:41:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.028+0000 7fe1ef7fe640 1 -- 172.21.15.84:0/2692330907 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1e8009c30 con 0x7fe1f8102220 2024-04-26T18:41:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.028+0000 7fe1ef7fe640 1 -- 172.21.15.84:0/2692330907 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1e800f620 con 0x7fe1f8102220 2024-04-26T18:41:48.031 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.028+0000 7fe2003a4640 1 -- 172.21.15.84:0/2692330907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1f81a7e60 con 0x7fe1f8102220 2024-04-26T18:41:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.030+0000 7fe1ef7fe640 1 -- 172.21.15.84:0/2692330907 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fe1e8010460 con 0x7fe1f8102220 2024-04-26T18:41:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe203198740 1 Processor -- start 2024-04-26T18:41:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe203198740 1 -- start start 2024-04-26T18:41:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe203198740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 0x55ef734722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe1ef7fe640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe1e40a3dc0 0x7fe1e40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe203198740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73472820 0x55ef7347cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe203198740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef7347d0e0 0x55ef7347f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.038 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe203198740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ef733d8460 con 0x55ef73471f00 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe203198740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef73392590 con 0x55ef7347d0e0 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe203198740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef733d4420 con 0x55ef73472820 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe1fe140640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe1e40a3dc0 0x7fe1e40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe1fe941640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 0x55ef734722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe1ef7fe640 1 -- 172.21.15.84:0/2692330907 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7fe1e80d0480 con 0x7fe1f8102220 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe1fe941640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 0x55ef734722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34182/0 (socket says 172.21.15.84:34182) 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.031+0000 7fe1fe941640 1 -- 172.21.15.84:0/4263497965 learned_addr learned my addr 172.21.15.84:0/4263497965 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.032+0000 7fe1fe941640 1 --2- 172.21.15.84:0/4263497965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef7347d0e0 0x55ef7347f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.032+0000 7fe1fd93f640 1 --2- 172.21.15.84:0/4263497965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73472820 0x55ef7347cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.032+0000 7fe1fe941640 1 -- 172.21.15.84:0/4263497965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73472820 msgr2=0x55ef7347cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.032+0000 7fe1fe941640 1 --2- 172.21.15.84:0/4263497965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73472820 0x55ef7347cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.032+0000 7fe1fe941640 1 -- 172.21.15.84:0/4263497965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef7347d0e0 msgr2=0x55ef7347f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.032+0000 7fe1fe941640 1 --2- 172.21.15.84:0/4263497965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef7347d0e0 0x55ef7347f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.032+0000 7fe1fe941640 1 -- 172.21.15.84:0/4263497965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef7339f0d0 con 0x55ef73471f00 2024-04-26T18:41:48.039 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.032+0000 7fe1fe140640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe1e40a3dc0 0x7fe1e40a6280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009900 tx=0x7fe1f4000bc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.032+0000 7fe1fe941640 1 --2- 172.21.15.84:0/4263497965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 0x55ef734722e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0012760 tx=0x7fe1f000a880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a6b410c857d565 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.032+0000 7fe1ecbfe640 1 -- 172.21.15.84:0/4263497965 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0018070 con 0x55ef73471f00 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.032+0000 7fe1ecbfe640 1 -- 172.21.15.84:0/4263497965 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f0010d20 con 0x55ef73471f00 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.032+0000 7fe1ecbfe640 1 -- 172.21.15.84:0/4263497965 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f00043e0 con 0x55ef73471f00 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.033+0000 7fe203198740 1 -- 172.21.15.84:0/4263497965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 msgr2=0x55ef734722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.033+0000 7fe203198740 1 --2- 172.21.15.84:0/4263497965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 0x55ef734722e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0012760 tx=0x7fe1f000a880 comp rx=0 tx=0).stop 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.033+0000 7fe203198740 1 -- 172.21.15.84:0/4263497965 shutdown_connections 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.033+0000 7fe203198740 1 --2- 172.21.15.84:0/4263497965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef7347d0e0 0x55ef7347f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.033+0000 7fe203198740 1 --2- 172.21.15.84:0/4263497965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73472820 0x55ef7347cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.033+0000 7fe203198740 1 --2- 172.21.15.84:0/4263497965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 0x55ef734722e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.033+0000 7fe203198740 1 -- 172.21.15.84:0/4263497965 >> 172.21.15.84:0/4263497965 conn(0x55ef7346e410 msgr2=0x55ef7346e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.033+0000 7fe203198740 1 -- 172.21.15.84:0/4263497965 shutdown_connections 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.033+0000 7fe203198740 1 -- 172.21.15.84:0/4263497965 wait complete. 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.033+0000 7fe203198740 1 -- start start 2024-04-26T18:41:48.040 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.033+0000 7fe203198740 1 Processor -- start 2024-04-26T18:41:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe203198740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 0x55ef73479ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe203198740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 0x55ef7347a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe203198740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7347d0e0 0x55ef7347aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe203198740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55ef7347fd10 con 0x55ef73471f00 2024-04-26T18:41:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe203198740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55ef7335fb60 con 0x55ef73472820 2024-04-26T18:41:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe203198740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55ef7347fe90 con 0x55ef7347d0e0 2024-04-26T18:41:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe203198740 10 client.0 _ll_register_callbacks cb 0x55ef733842b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:41:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1fe140640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 0x55ef73479ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1fe140640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 0x55ef73479ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34184/0 (socket says 172.21.15.84:34184) 2024-04-26T18:41:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1fe140640 1 -- 172.21.15.84:0/2646679635 learned_addr learned my addr 172.21.15.84:0/2646679635 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1fe941640 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7347d0e0 0x55ef7347aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.041 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1fd93f640 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 0x55ef7347a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1fe140640 1 -- 172.21.15.84:0/2646679635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7347d0e0 msgr2=0x55ef7347aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1fe140640 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7347d0e0 0x55ef7347aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1fe140640 1 -- 172.21.15.84:0/2646679635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 msgr2=0x55ef7347a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1fe140640 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 0x55ef7347a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1fe140640 1 -- 172.21.15.84:0/2646679635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef730ea210 con 0x55ef73471f00 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1fe941640 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7347d0e0 0x55ef7347aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1fe140640 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 0x55ef73479ff0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4033000 tx=0x7fe1f4009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dca46044e4ca95c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1e1ffb640 1 -- 172.21.15.84:0/2646679635 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f404c070 con 0x55ef73471f00 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe1e1ffb640 1 -- 172.21.15.84:0/2646679635 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7fe1f40035f0 con 0x55ef73471f00 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.034+0000 7fe203198740 1 -- 172.21.15.84:0/2646679635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1f0010d20 con 0x55ef73471f00 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.035+0000 7fe1e1ffb640 1 -- 172.21.15.84:0/2646679635 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe1f4058050 con 0x55ef73471f00 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.035+0000 7fe203198740 10 client.17061 fetch_fsmap learned FSMap version 29 2024-04-26T18:41:48.042 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.035+0000 7fe203198740 1 -- 172.21.15.84:0/2646679635 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef731b6620 con 0x55ef73471f00 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.035+0000 7fe1e1ffb640 1 -- 172.21.15.84:0/2646679635 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f0010d20 con 0x55ef73471f00 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.035+0000 7fe1e1ffb640 1 -- 172.21.15.84:0/2646679635 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe1f403c940 con 0x55ef73471f00 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.035+0000 7fe203198740 10 client.17061 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.035+0000 7fe203198740 10 client.17061 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.035+0000 7fe203198740 20 client.17061 populate_metadata read hostname 'smithi084' 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.035+0000 7fe203198740 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ef7348d1a0 0x55ef7348f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.035+0000 7fe203198740 4 client.17061 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.035+0000 7fe203198740 1 -- 172.21.15.84:0/2646679635 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef733e6070 con 0x55ef7348d1a0 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.035+0000 7fe1fd93f640 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ef7348d1a0 0x55ef7348f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.036+0000 7fe1fd93f640 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ef7348d1a0 0x55ef7348f590 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f79b14b9ee8eef9 server_cookie=51d46180554a4b2e in_seq=0 out_seq=0 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.036+0000 7fe1e1ffb640 10 client.17061 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.036+0000 7fe1e1ffb640 1 -- 172.21.15.84:0/2646679635 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef733e6070 con 0x55ef7348d1a0 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe1e1ffb640 10 client.17061 handle_command_reply: tid=0 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe203198740 1 client.17061 shutdown 2024-04-26T18:41:48.043 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe203198740 1 -- 172.21.15.84:0/2646679635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 msgr2=0x55ef73479ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe203198740 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 0x55ef73479ff0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4033000 tx=0x7fe1f4009260 comp rx=0 tx=0).stop 2024-04-26T18:41:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe203198740 1 -- 172.21.15.84:0/2646679635 shutdown_connections 2024-04-26T18:41:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe203198740 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ef7348d1a0 0x55ef7348f590 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe203198740 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef7347d0e0 0x55ef7347aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe1e1ffb640 0 client.17061 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe203198740 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 0x55ef7347a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe203198740 1 --2- 172.21.15.84:0/2646679635 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef73471f00 0x55ef73479ff0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe203198740 1 -- 172.21.15.84:0/2646679635 >> 172.21.15.84:0/2646679635 conn(0x55ef7346e410 msgr2=0x55ef73474190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe203198740 1 -- 172.21.15.84:0/2646679635 shutdown_connections 2024-04-26T18:41:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe203198740 1 -- 172.21.15.84:0/2646679635 wait complete. 2024-04-26T18:41:48.044 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.037+0000 7fe203198740 20 client.17061 trim_cache size 0 max 16384 2024-04-26T18:41:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.051+0000 7fe203198740 1 Processor -- start 2024-04-26T18:41:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.051+0000 7fe203198740 1 -- start start 2024-04-26T18:41:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe203198740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73471f00 0x55ef73488bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe203198740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 0x55ef73494e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe203198740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 0x55ef734953c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe203198740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b08c0 con 0x55ef7347d0e0 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe203198740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b2400 con 0x55ef73472820 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe203198740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b0f90 con 0x55ef73471f00 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1fe941640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 0x55ef734953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1fe941640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 0x55ef734953c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34194/0 (socket says 172.21.15.84:34194) 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1fe941640 1 -- 172.21.15.84:0/3855065409 learned_addr learned my addr 172.21.15.84:0/3855065409 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1fe140640 1 --2- 172.21.15.84:0/3855065409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73471f00 0x55ef73488bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1fe941640 1 -- 172.21.15.84:0/3855065409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73471f00 msgr2=0x55ef73488bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1fe941640 1 --2- 172.21.15.84:0/3855065409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73471f00 0x55ef73488bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1fd93f640 1 --2- 172.21.15.84:0/3855065409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 0x55ef73494e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1fe941640 1 -- 172.21.15.84:0/3855065409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 msgr2=0x55ef73494e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1fe941640 1 --2- 172.21.15.84:0/3855065409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 0x55ef73494e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1fe941640 1 -- 172.21.15.84:0/3855065409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef73469d60 con 0x55ef7347d0e0 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1fd93f640 1 --2- 172.21.15.84:0/3855065409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 0x55ef73494e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:41:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1fe941640 1 --2- 172.21.15.84:0/3855065409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 0x55ef734953c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0003f90 tx=0x7fe1f000ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7ed5fc642bd86f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1e3fff640 1 -- 172.21.15.84:0/3855065409 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0018040 con 0x55ef7347d0e0 2024-04-26T18:41:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1e3fff640 1 -- 172.21.15.84:0/3855065409 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f002a070 con 0x55ef7347d0e0 2024-04-26T18:41:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.052+0000 7fe1e3fff640 1 -- 172.21.15.84:0/3855065409 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f00042f0 con 0x55ef7347d0e0 2024-04-26T18:41:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.053+0000 7fe203198740 1 -- 172.21.15.84:0/3855065409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 msgr2=0x55ef734953c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.053+0000 7fe203198740 1 --2- 172.21.15.84:0/3855065409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 0x55ef734953c0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0003f90 tx=0x7fe1f000ab70 comp rx=0 tx=0).stop 2024-04-26T18:41:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.053+0000 7fe203198740 1 -- 172.21.15.84:0/3855065409 shutdown_connections 2024-04-26T18:41:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.053+0000 7fe203198740 1 --2- 172.21.15.84:0/3855065409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 0x55ef734953c0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.053+0000 7fe203198740 1 --2- 172.21.15.84:0/3855065409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 0x55ef73494e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.053+0000 7fe203198740 1 --2- 172.21.15.84:0/3855065409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73471f00 0x55ef73488bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.053+0000 7fe203198740 1 -- 172.21.15.84:0/3855065409 >> 172.21.15.84:0/3855065409 conn(0x55ef7346e410 msgr2=0x55ef7346e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.053+0000 7fe203198740 1 -- 172.21.15.84:0/3855065409 shutdown_connections 2024-04-26T18:41:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.053+0000 7fe203198740 1 -- 172.21.15.84:0/3855065409 wait complete. 2024-04-26T18:41:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.053+0000 7fe203198740 1 -- start start 2024-04-26T18:41:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.053+0000 7fe203198740 1 Processor -- start 2024-04-26T18:41:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe203198740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73471f00 0x55ef73476eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe203198740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 0x55ef734773f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe203198740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 0x55ef73477930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe203198740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b1d30 con 0x55ef7347d0e0 2024-04-26T18:41:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe203198740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81b1660 con 0x55ef73472820 2024-04-26T18:41:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe203198740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe1f81ae8c0 con 0x55ef73471f00 2024-04-26T18:41:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe203198740 10 client.0 _ll_register_callbacks cb 0x7fe1f4040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:41:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1fe941640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 0x55ef73477930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1fe941640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 0x55ef73477930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34206/0 (socket says 172.21.15.84:34206) 2024-04-26T18:41:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1fe941640 1 -- 172.21.15.84:0/2493519050 learned_addr learned my addr 172.21.15.84:0/2493519050 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:41:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1fe140640 1 --2- 172.21.15.84:0/2493519050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73471f00 0x55ef73476eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1fe941640 1 -- 172.21.15.84:0/2493519050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73471f00 msgr2=0x55ef73476eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1fe941640 1 --2- 172.21.15.84:0/2493519050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73471f00 0x55ef73476eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1fe941640 1 -- 172.21.15.84:0/2493519050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 msgr2=0x55ef734773f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1fe941640 1 --2- 172.21.15.84:0/2493519050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 0x55ef734773f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1fe941640 1 -- 172.21.15.84:0/2493519050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef734f28f0 con 0x55ef7347d0e0 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1fe941640 1 --2- 172.21.15.84:0/2493519050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 0x55ef73477930 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0003e30 tx=0x7fe1f000a7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=488ebec204e1bb1a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1e1ffb640 1 -- 172.21.15.84:0/2493519050 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1f0018050 con 0x55ef7347d0e0 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1e1ffb640 1 -- 172.21.15.84:0/2493519050 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7fe1f0010870 con 0x55ef7347d0e0 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe203198740 1 -- 172.21.15.84:0/2493519050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1f002a070 con 0x55ef7347d0e0 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe1e1ffb640 1 -- 172.21.15.84:0/2493519050 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe1f0010a70 con 0x55ef7347d0e0 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.054+0000 7fe203198740 10 client.17067 fetch_fsmap learned FSMap version 29 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.055+0000 7fe203198740 1 -- 172.21.15.84:0/2493519050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef734f3b50 con 0x55ef7347d0e0 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.055+0000 7fe1e1ffb640 1 -- 172.21.15.84:0/2493519050 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1f002a070 con 0x55ef7347d0e0 2024-04-26T18:41:48.060 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.055+0000 7fe1e1ffb640 1 -- 172.21.15.84:0/2493519050 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe1f0034020 con 0x55ef7347d0e0 2024-04-26T18:41:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.055+0000 7fe203198740 10 client.17067 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:41:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.055+0000 7fe203198740 10 client.17067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:41:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.055+0000 7fe203198740 20 client.17067 populate_metadata read hostname 'smithi084' 2024-04-26T18:41:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.055+0000 7fe203198740 1 --2- 172.21.15.84:0/2493519050 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ef734f3fc0 0x55ef734f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:41:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.055+0000 7fe203198740 4 client.17067 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:41:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.055+0000 7fe203198740 1 -- 172.21.15.84:0/2493519050 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ef733dfee0 con 0x55ef734f3fc0 2024-04-26T18:41:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.055+0000 7fe1fe140640 1 --2- 172.21.15.84:0/2493519050 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ef734f3fc0 0x55ef734f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:41:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.056+0000 7fe1e1ffb640 10 client.17067 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:48.061 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.056+0000 7fe1fe140640 1 --2- 172.21.15.84:0/2493519050 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ef734f3fc0 0x55ef734f63b0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d26b9711e1a53f server_cookie=b0a88adecfcb03dc in_seq=0 out_seq=0 2024-04-26T18:41:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe1e1ffb640 1 -- 172.21.15.84:0/2493519050 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ef733dfee0 con 0x55ef734f3fc0 2024-04-26T18:41:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe1e1ffb640 10 client.17067 handle_command_reply: tid=0 2024-04-26T18:41:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe203198740 1 client.17067 shutdown 2024-04-26T18:41:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe203198740 1 -- 172.21.15.84:0/2493519050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 msgr2=0x55ef73477930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe203198740 1 --2- 172.21.15.84:0/2493519050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 0x55ef73477930 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fe1f0003e30 tx=0x7fe1f000a7e0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe203198740 1 -- 172.21.15.84:0/2493519050 shutdown_connections 2024-04-26T18:41:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe203198740 1 --2- 172.21.15.84:0/2493519050 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55ef734f3fc0 0x55ef734f63b0 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe203198740 1 --2- 172.21.15.84:0/2493519050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55ef7347d0e0 0x55ef73477930 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe1e1ffb640 0 client.17067 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:41:48.071 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe203198740 1 --2- 172.21.15.84:0/2493519050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55ef73472820 0x55ef734773f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe203198740 1 --2- 172.21.15.84:0/2493519050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55ef73471f00 0x55ef73476eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe203198740 1 -- 172.21.15.84:0/2493519050 >> 172.21.15.84:0/2493519050 conn(0x55ef7346e410 msgr2=0x55ef7347b820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe203198740 1 -- 172.21.15.84:0/2493519050 shutdown_connections 2024-04-26T18:41:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.070+0000 7fe203198740 1 -- 172.21.15.84:0/2493519050 wait complete. 2024-04-26T18:41:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.071+0000 7fe203198740 20 client.17067 trim_cache size 0 max 16384 2024-04-26T18:41:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:41:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:41:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:41:48.072 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:41:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.071+0000 7fe2003a4640 1 -- 172.21.15.84:0/2692330907 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe1e40a3dc0 msgr2=0x7fe1e40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.071+0000 7fe2003a4640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe1e40a3dc0 0x7fe1e40a6280 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009900 tx=0x7fe1f4000bc0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.071+0000 7fe2003a4640 1 -- 172.21.15.84:0/2692330907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1f8102220 msgr2=0x7fe1f80fed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:41:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.071+0000 7fe2003a4640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1f8102220 0x7fe1f80fed30 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800d950 tx=0x7fe1e800de10 comp rx=0 tx=0).stop 2024-04-26T18:41:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.071+0000 7fe2003a4640 1 -- 172.21.15.84:0/2692330907 shutdown_connections 2024-04-26T18:41:48.073 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.071+0000 7fe2003a4640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe1e40a3dc0 0x7fe1e40a6280 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.071+0000 7fe2003a4640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe1f8107820 0x7fe1f80ff270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.071+0000 7fe2003a4640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe1f8102220 0x7fe1f80fed30 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.071+0000 7fe2003a4640 1 --2- 172.21.15.84:0/2692330907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe1f8101860 0x7fe1f80fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:41:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.071+0000 7fe2003a4640 1 -- 172.21.15.84:0/2692330907 >> 172.21.15.84:0/2692330907 conn(0x7fe1f80789b0 msgr2=0x7fe1f8100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:41:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.071+0000 7fe2003a4640 1 -- 172.21.15.84:0/2692330907 shutdown_connections 2024-04-26T18:41:48.074 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:41:48.071+0000 7fe2003a4640 1 -- 172.21.15.84:0/2692330907 wait complete. 2024-04-26T18:41:49.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:48 smithi084 ceph-mon[29141]: pgmap v434: 97 pgs: 97 active+clean; 2.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:41:49.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:48 smithi103 ceph-mon[32727]: pgmap v434: 97 pgs: 97 active+clean; 2.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:41:49.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:48 smithi174 ceph-mon[32705]: pgmap v434: 97 pgs: 97 active+clean; 2.8 MiB data, 435 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:41:51.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:50 smithi084 ceph-mon[29141]: pgmap v435: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-26T18:41:51.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:50 smithi103 ceph-mon[32727]: pgmap v435: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-26T18:41:51.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:50 smithi174 ceph-mon[32705]: pgmap v435: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 48 KiB/s wr, 22 op/s 2024-04-26T18:41:53.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:52 smithi084 ceph-mon[29141]: pgmap v436: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-26T18:41:53.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:52 smithi103 ceph-mon[32727]: pgmap v436: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-26T18:41:53.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:52 smithi174 ceph-mon[32705]: pgmap v436: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 KiB/s wr, 13 op/s 2024-04-26T18:41:55.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:54 smithi084 ceph-mon[29141]: pgmap v437: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-04-26T18:41:55.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:54 smithi103 ceph-mon[32727]: pgmap v437: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-04-26T18:41:55.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:54 smithi174 ceph-mon[32705]: pgmap v437: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-04-26T18:41:57.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:56 smithi084 ceph-mon[29141]: pgmap v438: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-26T18:41:57.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:56 smithi103 ceph-mon[32727]: pgmap v438: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-26T18:41:57.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:56 smithi174 ceph-mon[32705]: pgmap v438: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-26T18:41:59.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:41:58 smithi084 ceph-mon[29141]: pgmap v439: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-26T18:41:59.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:41:58 smithi103 ceph-mon[32727]: pgmap v439: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-26T18:41:59.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:41:58 smithi174 ceph-mon[32705]: pgmap v439: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.2 KiB/s wr, 11 op/s 2024-04-26T18:42:01.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:00 smithi084 ceph-mon[29141]: pgmap v440: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-26T18:42:01.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:00 smithi103 ceph-mon[32727]: pgmap v440: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-26T18:42:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.164+0000 7f38e0628640 1 -- 172.21.15.84:0/59118785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d8102910 msgr2=0x7f38d8102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.164+0000 7f38e0628640 1 --2- 172.21.15.84:0/59118785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d8102910 0x7f38d8102d90 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f38c8009930 tx=0x7f38c802a420 comp rx=0 tx=0).stop 2024-04-26T18:42:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.164+0000 7f38e0628640 1 -- 172.21.15.84:0/59118785 shutdown_connections 2024-04-26T18:42:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.164+0000 7f38e0628640 1 --2- 172.21.15.84:0/59118785 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38d81032d0 0x7f38d810f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.164+0000 7f38e0628640 1 --2- 172.21.15.84:0/59118785 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d8102910 0x7f38d8102d90 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.164+0000 7f38e0628640 1 --2- 172.21.15.84:0/59118785 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d8108920 0x7f38d8108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.164+0000 7f38e0628640 1 -- 172.21.15.84:0/59118785 >> 172.21.15.84:0/59118785 conn(0x7f38d80fe3e0 msgr2=0x7f38d8100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.164+0000 7f38e0628640 1 -- 172.21.15.84:0/59118785 shutdown_connections 2024-04-26T18:42:01.167 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.164+0000 7f38e0628640 1 -- 172.21.15.84:0/59118785 wait complete. 2024-04-26T18:42:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38e0628640 1 Processor -- start 2024-04-26T18:42:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38e0628640 1 -- start start 2024-04-26T18:42:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38e0628640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38d8102910 0x7f38d8199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38e0628640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d81032d0 0x7f38d8199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38e0628640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d8108920 0x7f38d819d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38e0628640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f38d8112bb0 con 0x7f38d8108920 2024-04-26T18:42:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38e0628640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f38d8112a30 con 0x7f38d8102910 2024-04-26T18:42:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38e0628640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38d8112d30 con 0x7f38d81032d0 2024-04-26T18:42:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38debc5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d8108920 0x7f38d819d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38debc5640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d8108920 0x7f38d819d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33248/0 (socket says 172.21.15.84:33248) 2024-04-26T18:42:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38debc5640 1 -- 172.21.15.84:0/3886252395 learned_addr learned my addr 172.21.15.84:0/3886252395 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38de3c4640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38d8102910 0x7f38d8199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d81032d0 0x7f38d8199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38debc5640 1 -- 172.21.15.84:0/3886252395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d81032d0 msgr2=0x7f38d8199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38debc5640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d81032d0 0x7f38d8199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38debc5640 1 -- 172.21.15.84:0/3886252395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38d8102910 msgr2=0x7f38d8199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38debc5640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38d8102910 0x7f38d8199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.165+0000 7f38debc5640 1 -- 172.21.15.84:0/3886252395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38d819e100 con 0x7f38d8108920 2024-04-26T18:42:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.166+0000 7f38debc5640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d8108920 0x7f38d819d9d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f38d000eaa0 tx=0x7f38d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.166+0000 7f38cf7fe640 1 -- 172.21.15.84:0/3886252395 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d000cdb0 con 0x7f38d8108920 2024-04-26T18:42:01.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.166+0000 7f38cf7fe640 1 -- 172.21.15.84:0/3886252395 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38d0005020 con 0x7f38d8108920 2024-04-26T18:42:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.166+0000 7f38e0628640 1 -- 172.21.15.84:0/3886252395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38d819e3f0 con 0x7f38d8108920 2024-04-26T18:42:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.166+0000 7f38cf7fe640 1 -- 172.21.15.84:0/3886252395 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d00116d0 con 0x7f38d8108920 2024-04-26T18:42:01.170 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.166+0000 7f38e0628640 1 -- 172.21.15.84:0/3886252395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38d81a5c80 con 0x7f38d8108920 2024-04-26T18:42:01.176 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.168+0000 7f38cf7fe640 1 -- 172.21.15.84:0/3886252395 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f38d00051d0 con 0x7f38d8108920 2024-04-26T18:42:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.168+0000 7f38e3416740 1 Processor -- start 2024-04-26T18:42:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38e3416740 1 -- start start 2024-04-26T18:42:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38e3416740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c6df00 0x55f430c6e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38e3416740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 0x55f430c78ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38e3416740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c790e0 0x55f430c7b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38e3416740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f430bd4460 con 0x55f430c6e820 2024-04-26T18:42:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38e3416740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f430b8e590 con 0x55f430c6df00 2024-04-26T18:42:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38e3416740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f430bd0420 con 0x55f430c790e0 2024-04-26T18:42:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38cf7fe640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f38c40a3e40 0x7f38c40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38debc5640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f38c40a3e40 0x7f38c40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38de3c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 0x55f430c78ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.177 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38de3c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 0x55f430c78ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33264/0 (socket says 172.21.15.84:33264) 2024-04-26T18:42:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38de3c4640 1 -- 172.21.15.84:0/2747881659 learned_addr learned my addr 172.21.15.84:0/2747881659 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/2747881659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c790e0 0x55f430c7b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38cf7fe640 1 -- 172.21.15.84:0/3886252395 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f38d00d0910 con 0x7f38d8108920 2024-04-26T18:42:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/2747881659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c6df00 0x55f430c6e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38de3c4640 1 -- 172.21.15.84:0/2747881659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c790e0 msgr2=0x55f430c7b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38de3c4640 1 --2- 172.21.15.84:0/2747881659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c790e0 0x55f430c7b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38de3c4640 1 -- 172.21.15.84:0/2747881659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c6df00 msgr2=0x55f430c6e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38de3c4640 1 --2- 172.21.15.84:0/2747881659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c6df00 0x55f430c6e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38de3c4640 1 -- 172.21.15.84:0/2747881659 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f430b9b0d0 con 0x55f430c6e820 2024-04-26T18:42:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.169+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/2747881659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c790e0 0x55f430c7b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:42:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.170+0000 7f38debc5640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f38c40a3e40 0x7f38c40a6300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38d0098d80 tx=0x7f38d0087040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.170+0000 7f38de3c4640 1 --2- 172.21.15.84:0/2747881659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 0x55f430c78ba0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f38d400bc50 tx=0x7f38d40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fb7a611652b1d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:01.178 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.170+0000 7f38ccbfe640 1 -- 172.21.15.84:0/2747881659 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d401c480 con 0x55f430c6e820 2024-04-26T18:42:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.170+0000 7f38ccbfe640 1 -- 172.21.15.84:0/2747881659 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38d401cad0 con 0x55f430c6e820 2024-04-26T18:42:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.170+0000 7f38ccbfe640 1 -- 172.21.15.84:0/2747881659 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4011ac0 con 0x55f430c6e820 2024-04-26T18:42:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.170+0000 7f38e3416740 1 -- 172.21.15.84:0/2747881659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 msgr2=0x55f430c78ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.170+0000 7f38e3416740 1 --2- 172.21.15.84:0/2747881659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 0x55f430c78ba0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f38d400bc50 tx=0x7f38d40107a0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 -- 172.21.15.84:0/2747881659 shutdown_connections 2024-04-26T18:42:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 --2- 172.21.15.84:0/2747881659 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c790e0 0x55f430c7b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 --2- 172.21.15.84:0/2747881659 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 0x55f430c78ba0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.179 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 --2- 172.21.15.84:0/2747881659 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c6df00 0x55f430c6e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 -- 172.21.15.84:0/2747881659 >> 172.21.15.84:0/2747881659 conn(0x55f430c6a410 msgr2=0x55f430c6a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 -- 172.21.15.84:0/2747881659 shutdown_connections 2024-04-26T18:42:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 -- 172.21.15.84:0/2747881659 wait complete. 2024-04-26T18:42:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 -- start start 2024-04-26T18:42:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 Processor -- start 2024-04-26T18:42:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6df00 0x55f430c760b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6e820 0x55f430c765f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.180 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c790e0 0x55f430c76b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f430c7bd10 con 0x55f430c6df00 2024-04-26T18:42:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f430b5bb60 con 0x55f430c790e0 2024-04-26T18:42:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f430c7be90 con 0x55f430c6e820 2024-04-26T18:42:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38e3416740 10 client.0 _ll_register_callbacks cb 0x55f430b802b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:42:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38ddbc3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6df00 0x55f430c760b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38ddbc3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6df00 0x55f430c760b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33278/0 (socket says 172.21.15.84:33278) 2024-04-26T18:42:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.171+0000 7f38ddbc3640 1 -- 172.21.15.84:0/2176002607 learned_addr learned my addr 172.21.15.84:0/2176002607 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38de3c4640 1 --2- 172.21.15.84:0/2176002607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6e820 0x55f430c765f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.181 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/2176002607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c790e0 0x55f430c76b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38ddbc3640 1 -- 172.21.15.84:0/2176002607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6e820 msgr2=0x55f430c765f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/2176002607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6e820 0x55f430c765f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38ddbc3640 1 -- 172.21.15.84:0/2176002607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c790e0 msgr2=0x55f430c76b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/2176002607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c790e0 0x55f430c76b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38ddbc3640 1 -- 172.21.15.84:0/2176002607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4308e6210 con 0x55f430c6df00 2024-04-26T18:42:01.182 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/2176002607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6df00 0x55f430c760b0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f38c8005c40 tx=0x7f38c8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef16c33bc2bbaad server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:42:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38c1ffb640 1 -- 172.21.15.84:0/2176002607 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8044070 con 0x55f430c6df00 2024-04-26T18:42:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38c1ffb640 1 -- 172.21.15.84:0/2176002607 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f38c8004240 con 0x55f430c6df00 2024-04-26T18:42:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38e3416740 1 -- 172.21.15.84:0/2176002607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38d401cad0 con 0x55f430c6df00 2024-04-26T18:42:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38c1ffb640 1 -- 172.21.15.84:0/2176002607 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38c8035070 con 0x55f430c6df00 2024-04-26T18:42:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38e3416740 10 client.17079 fetch_fsmap learned FSMap version 29 2024-04-26T18:42:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38e3416740 1 -- 172.21.15.84:0/2176002607 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f4309b2620 con 0x55f430c6df00 2024-04-26T18:42:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38c1ffb640 1 -- 172.21.15.84:0/2176002607 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38d401cad0 con 0x55f430c6df00 2024-04-26T18:42:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.172+0000 7f38c1ffb640 1 -- 172.21.15.84:0/2176002607 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f38c803d4c0 con 0x55f430c6df00 2024-04-26T18:42:01.183 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.173+0000 7f38e3416740 10 client.17079 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:42:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.173+0000 7f38e3416740 10 client.17079 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:42:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.173+0000 7f38e3416740 20 client.17079 populate_metadata read hostname 'smithi084' 2024-04-26T18:42:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.173+0000 7f38e3416740 1 --2- 172.21.15.84:0/2176002607 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f430c89270 0x55f430c8b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.173+0000 7f38e3416740 4 client.17079 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:42:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.173+0000 7f38e3416740 1 -- 172.21.15.84:0/2176002607 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f430be2070 con 0x55f430c89270 2024-04-26T18:42:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.173+0000 7f38de3c4640 1 --2- 172.21.15.84:0/2176002607 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f430c89270 0x55f430c8b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.173+0000 7f38c1ffb640 10 client.17079 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.173+0000 7f38de3c4640 1 --2- 172.21.15.84:0/2176002607 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f430c89270 0x55f430c8b680 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf122ca8a81e8719 server_cookie=a9ef26157fa0be37 in_seq=0 out_seq=0 2024-04-26T18:42:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38c1ffb640 1 -- 172.21.15.84:0/2176002607 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f430be2070 con 0x55f430c89270 2024-04-26T18:42:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38c1ffb640 10 client.17079 handle_command_reply: tid=0 2024-04-26T18:42:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38e3416740 1 client.17079 shutdown 2024-04-26T18:42:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38e3416740 1 -- 172.21.15.84:0/2176002607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6df00 msgr2=0x55f430c760b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:01.184 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38e3416740 1 --2- 172.21.15.84:0/2176002607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6df00 0x55f430c760b0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f38c8005c40 tx=0x7f38c8009260 comp rx=0 tx=0).stop 2024-04-26T18:42:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38e3416740 1 -- 172.21.15.84:0/2176002607 shutdown_connections 2024-04-26T18:42:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38e3416740 1 --2- 172.21.15.84:0/2176002607 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f430c89270 0x55f430c8b680 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38e3416740 1 --2- 172.21.15.84:0/2176002607 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c790e0 0x55f430c76b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38e3416740 1 --2- 172.21.15.84:0/2176002607 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6e820 0x55f430c765f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38c1ffb640 0 client.17079 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:01.185 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38e3416740 1 --2- 172.21.15.84:0/2176002607 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6df00 0x55f430c760b0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38e3416740 1 -- 172.21.15.84:0/2176002607 >> 172.21.15.84:0/2176002607 conn(0x55f430c6a410 msgr2=0x55f430c70190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38e3416740 1 -- 172.21.15.84:0/2176002607 shutdown_connections 2024-04-26T18:42:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38e3416740 1 -- 172.21.15.84:0/2176002607 wait complete. 2024-04-26T18:42:01.186 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.174+0000 7f38e3416740 20 client.17079 trim_cache size 0 max 16384 2024-04-26T18:42:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38e3416740 1 Processor -- start 2024-04-26T18:42:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38e3416740 1 -- start start 2024-04-26T18:42:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38e3416740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c6df00 0x55f430c84bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.196 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38e3416740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6e820 0x55f430c90f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38e3416740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c790e0 0x55f430c91490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38e3416740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81ae700 con 0x55f430c790e0 2024-04-26T18:42:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38e3416740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81b0240 con 0x55f430c6df00 2024-04-26T18:42:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38e3416740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81aedd0 con 0x55f430c6e820 2024-04-26T18:42:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38ddbc3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c790e0 0x55f430c91490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38ddbc3640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c790e0 0x55f430c91490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33288/0 (socket says 172.21.15.84:33288) 2024-04-26T18:42:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38ddbc3640 1 -- 172.21.15.84:0/1444458809 learned_addr learned my addr 172.21.15.84:0/1444458809 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38de3c4640 1 --2- 172.21.15.84:0/1444458809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6e820 0x55f430c90f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/1444458809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c6df00 0x55f430c84bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38ddbc3640 1 -- 172.21.15.84:0/1444458809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6e820 msgr2=0x55f430c90f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/1444458809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6e820 0x55f430c90f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.197 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38ddbc3640 1 -- 172.21.15.84:0/1444458809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c6df00 msgr2=0x55f430c84bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/1444458809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c6df00 0x55f430c84bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.189+0000 7f38ddbc3640 1 -- 172.21.15.84:0/1444458809 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f430c74640 con 0x55f430c790e0 2024-04-26T18:42:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.190+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/1444458809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c790e0 0x55f430c91490 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f38c80057c0 tx=0x7f38c8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4df0faa8d7ee4666 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.190+0000 7f38c3fff640 1 -- 172.21.15.84:0/1444458809 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8044070 con 0x55f430c790e0 2024-04-26T18:42:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.190+0000 7f38c3fff640 1 -- 172.21.15.84:0/1444458809 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38c803db80 con 0x55f430c790e0 2024-04-26T18:42:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.190+0000 7f38c3fff640 1 -- 172.21.15.84:0/1444458809 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38c8054040 con 0x55f430c790e0 2024-04-26T18:42:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.190+0000 7f38e3416740 1 -- 172.21.15.84:0/1444458809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c790e0 msgr2=0x55f430c91490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.190+0000 7f38e3416740 1 --2- 172.21.15.84:0/1444458809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c790e0 0x55f430c91490 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f38c80057c0 tx=0x7f38c8000bc0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.190+0000 7f38e3416740 1 -- 172.21.15.84:0/1444458809 shutdown_connections 2024-04-26T18:42:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.190+0000 7f38e3416740 1 --2- 172.21.15.84:0/1444458809 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c790e0 0x55f430c91490 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.190+0000 7f38e3416740 1 --2- 172.21.15.84:0/1444458809 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6e820 0x55f430c90f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.190+0000 7f38e3416740 1 --2- 172.21.15.84:0/1444458809 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c6df00 0x55f430c84bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.198 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.190+0000 7f38e3416740 1 -- 172.21.15.84:0/1444458809 >> 172.21.15.84:0/1444458809 conn(0x55f430c6a410 msgr2=0x55f430c71460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.190+0000 7f38e3416740 1 -- 172.21.15.84:0/1444458809 shutdown_connections 2024-04-26T18:42:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.190+0000 7f38e3416740 1 -- 172.21.15.84:0/1444458809 wait complete. 2024-04-26T18:42:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38e3416740 1 -- start start 2024-04-26T18:42:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38e3416740 1 Processor -- start 2024-04-26T18:42:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38e3416740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6df00 0x55f430c810e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38e3416740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 0x55f430c81620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38e3416740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c790e0 0x55f430c7e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38e3416740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81afb70 con 0x55f430c6e820 2024-04-26T18:42:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38e3416740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81af4a0 con 0x55f430c790e0 2024-04-26T18:42:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38e3416740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f38d81ac700 con 0x55f430c6df00 2024-04-26T18:42:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38e3416740 10 client.0 _ll_register_callbacks cb 0x7f38c8032e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:42:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38de3c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 0x55f430c81620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38de3c4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 0x55f430c81620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:33298/0 (socket says 172.21.15.84:33298) 2024-04-26T18:42:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38de3c4640 1 -- 172.21.15.84:0/2659158200 learned_addr learned my addr 172.21.15.84:0/2659158200 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6df00 0x55f430c810e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c790e0 0x55f430c7e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38de3c4640 1 -- 172.21.15.84:0/2659158200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6df00 msgr2=0x55f430c810e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38de3c4640 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6df00 0x55f430c810e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38de3c4640 1 -- 172.21.15.84:0/2659158200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c790e0 msgr2=0x55f430c7e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38de3c4640 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c790e0 0x55f430c7e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38de3c4640 1 -- 172.21.15.84:0/2659158200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f430c7e6f0 con 0x55f430c6e820 2024-04-26T18:42:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.191+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6df00 0x55f430c810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:42:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38de3c4640 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 0x55f430c81620 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005940 tx=0x7f38d4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59240e1737a37b54 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38c1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:42:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38c1ffb640 1 -- 172.21.15.84:0/2659158200 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4030060 con 0x55f430c6e820 2024-04-26T18:42:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38e3416740 1 -- 172.21.15.84:0/2659158200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38c803db80 con 0x55f430c6e820 2024-04-26T18:42:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38c1ffb640 1 -- 172.21.15.84:0/2659158200 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(110..110 src has 1..110) ==== 8579+0+0 (secure 0 0 0) 0x7f38d4005450 con 0x55f430c6e820 2024-04-26T18:42:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38c1ffb640 1 -- 172.21.15.84:0/2659158200 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38d4020070 con 0x55f430c6e820 2024-04-26T18:42:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38e3416740 10 client.17085 fetch_fsmap learned FSMap version 29 2024-04-26T18:42:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38e3416740 1 -- 172.21.15.84:0/2659158200 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f430cf0120 con 0x55f430c6e820 2024-04-26T18:42:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38c1ffb640 1 -- 172.21.15.84:0/2659158200 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38c803db80 con 0x55f430c6e820 2024-04-26T18:42:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38c1ffb640 1 -- 172.21.15.84:0/2659158200 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f38d402a190 con 0x55f430c6e820 2024-04-26T18:42:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38e3416740 10 client.17085 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:42:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38e3416740 10 client.17085 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:42:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38e3416740 20 client.17085 populate_metadata read hostname 'smithi084' 2024-04-26T18:42:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38e3416740 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f430cf0570 0x55f430cf2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38e3416740 4 client.17085 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:42:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38e3416740 1 -- 172.21.15.84:0/2659158200 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f430bdbee0 con 0x55f430cf0570 2024-04-26T18:42:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.192+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f430cf0570 0x55f430cf2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.193+0000 7f38ddbc3640 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f430cf0570 0x55f430cf2990 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d0d2f7de5561f6 server_cookie=c5f2238d2032c94a in_seq=0 out_seq=0 2024-04-26T18:42:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.193+0000 7f38c1ffb640 10 client.17085 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:01.203 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:00 smithi174 ceph-mon[32705]: pgmap v440: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-26T18:42:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38c1ffb640 1 -- 172.21.15.84:0/2659158200 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f430bdbee0 con 0x55f430cf0570 2024-04-26T18:42:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38c1ffb640 10 client.17085 handle_command_reply: tid=0 2024-04-26T18:42:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e3416740 1 client.17085 shutdown 2024-04-26T18:42:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e3416740 1 -- 172.21.15.84:0/2659158200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 msgr2=0x55f430c81620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e3416740 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 0x55f430c81620 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f38d4005940 tx=0x7f38d4005850 comp rx=0 tx=0).stop 2024-04-26T18:42:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e3416740 1 -- 172.21.15.84:0/2659158200 shutdown_connections 2024-04-26T18:42:01.205 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e3416740 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f430cf0570 0x55f430cf2990 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e3416740 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f430c790e0 0x55f430c7e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e3416740 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f430c6e820 0x55f430c81620 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e3416740 1 --2- 172.21.15.84:0/2659158200 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f430c6df00 0x55f430c810e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38c1ffb640 0 client.17085 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e3416740 1 -- 172.21.15.84:0/2659158200 >> 172.21.15.84:0/2659158200 conn(0x55f430c6a410 msgr2=0x55f430c6f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e3416740 1 -- 172.21.15.84:0/2659158200 shutdown_connections 2024-04-26T18:42:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e3416740 1 -- 172.21.15.84:0/2659158200 wait complete. 2024-04-26T18:42:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e3416740 20 client.17085 trim_cache size 0 max 16384 2024-04-26T18:42:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:42:01.206 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:42:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:42:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:42:01.207 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e0628640 1 -- 172.21.15.84:0/3886252395 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f38c40a3e40 msgr2=0x7f38c40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e0628640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f38c40a3e40 0x7f38c40a6300 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f38d0098d80 tx=0x7f38d0087040 comp rx=0 tx=0).stop 2024-04-26T18:42:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e0628640 1 -- 172.21.15.84:0/3886252395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d8108920 msgr2=0x7f38d819d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e0628640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d8108920 0x7f38d819d9d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f38d000eaa0 tx=0x7f38d000ef60 comp rx=0 tx=0).stop 2024-04-26T18:42:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e0628640 1 -- 172.21.15.84:0/3886252395 shutdown_connections 2024-04-26T18:42:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e0628640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f38c40a3e40 0x7f38c40a6300 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e0628640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f38d8108920 0x7f38d819d9d0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e0628640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f38d81032d0 0x7f38d8199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e0628640 1 --2- 172.21.15.84:0/3886252395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f38d8102910 0x7f38d8199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.205+0000 7f38e0628640 1 -- 172.21.15.84:0/3886252395 >> 172.21.15.84:0/3886252395 conn(0x7f38d80fe3e0 msgr2=0x7f38d80feb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.206+0000 7f38e0628640 1 -- 172.21.15.84:0/3886252395 shutdown_connections 2024-04-26T18:42:01.208 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:01.206+0000 7f38e0628640 1 -- 172.21.15.84:0/3886252395 wait complete. 2024-04-26T18:42:03.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:02 smithi084 ceph-mon[29141]: pgmap v441: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-26T18:42:03.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:02 smithi103 ceph-mon[32727]: pgmap v441: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-26T18:42:03.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:02 smithi174 ceph-mon[32705]: pgmap v441: 97 pgs: 97 active+clean; 2.9 MiB data, 436 MiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-26T18:42:05.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:04 smithi084 ceph-mon[29141]: pgmap v442: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 50 KiB/s wr, 31 op/s 2024-04-26T18:42:05.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:04 smithi084 ceph-mon[29141]: osdmap e111: 12 total, 12 up, 12 in 2024-04-26T18:42:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:04 smithi103 ceph-mon[32727]: pgmap v442: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 50 KiB/s wr, 31 op/s 2024-04-26T18:42:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:04 smithi103 ceph-mon[32727]: osdmap e111: 12 total, 12 up, 12 in 2024-04-26T18:42:05.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:04 smithi174 ceph-mon[32705]: pgmap v442: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 50 KiB/s wr, 31 op/s 2024-04-26T18:42:05.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:04 smithi174 ceph-mon[32705]: osdmap e111: 12 total, 12 up, 12 in 2024-04-26T18:42:07.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:06 smithi084 ceph-mon[29141]: pgmap v444: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-04-26T18:42:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:06 smithi103 ceph-mon[32727]: pgmap v444: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-04-26T18:42:07.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:06 smithi174 ceph-mon[32705]: pgmap v444: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-04-26T18:42:09.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:08 smithi084 ceph-mon[29141]: pgmap v445: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-04-26T18:42:09.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:08 smithi103 ceph-mon[32727]: pgmap v445: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-04-26T18:42:09.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:08 smithi174 ceph-mon[32705]: pgmap v445: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 60 KiB/s wr, 33 op/s 2024-04-26T18:42:11.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:10 smithi084 ceph-mon[29141]: pgmap v446: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-26T18:42:11.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:10 smithi103 ceph-mon[32727]: pgmap v446: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-26T18:42:11.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:10 smithi174 ceph-mon[32705]: pgmap v446: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-26T18:42:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:13 smithi084 ceph-mon[29141]: pgmap v447: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-26T18:42:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:13 smithi084 ceph-mon[29141]: osdmap e112: 12 total, 12 up, 12 in 2024-04-26T18:42:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:13 smithi103 ceph-mon[32727]: pgmap v447: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-26T18:42:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:13 smithi103 ceph-mon[32727]: osdmap e112: 12 total, 12 up, 12 in 2024-04-26T18:42:13.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:13 smithi174 ceph-mon[32705]: pgmap v447: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 58 KiB/s wr, 32 op/s 2024-04-26T18:42:13.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:13 smithi174 ceph-mon[32705]: osdmap e112: 12 total, 12 up, 12 in 2024-04-26T18:42:14.303 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.301+0000 7f47ae3af640 1 -- 172.21.15.84:0/3871896210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a8102080 msgr2=0x7f47a810edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.301+0000 7f47ae3af640 1 --2- 172.21.15.84:0/3871896210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a8102080 0x7f47a810edd0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f479c009900 tx=0x7f479c02a410 comp rx=0 tx=0).stop 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.301+0000 7f47ae3af640 1 -- 172.21.15.84:0/3871896210 shutdown_connections 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.301+0000 7f47ae3af640 1 --2- 172.21.15.84:0/3871896210 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a810f310 0x7f47a8111700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.301+0000 7f47ae3af640 1 --2- 172.21.15.84:0/3871896210 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a8102080 0x7f47a810edd0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.301+0000 7f47ae3af640 1 --2- 172.21.15.84:0/3871896210 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47a8101760 0x7f47a8101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.301+0000 7f47ae3af640 1 -- 172.21.15.84:0/3871896210 >> 172.21.15.84:0/3871896210 conn(0x7f47a80789b0 msgr2=0x7f47a8078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.301+0000 7f47ae3af640 1 -- 172.21.15.84:0/3871896210 shutdown_connections 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.301+0000 7f47ae3af640 1 -- 172.21.15.84:0/3871896210 wait complete. 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.301+0000 7f47ae3af640 1 Processor -- start 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.301+0000 7f47ae3af640 1 -- start start 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47ae3af640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a8101760 0x7f47a819b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47ae3af640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47a8102080 0x7f47a819b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47ae3af640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a810f310 0x7f47a819fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.304 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47ae3af640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8114dd0 con 0x7f47a8101760 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47ae3af640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8114c50 con 0x7f47a8102080 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47ae3af640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47a8114f50 con 0x7f47a810f310 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a8101760 0x7f47a819b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a8101760 0x7f47a819b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48778/0 (socket says 172.21.15.84:48778) 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47a7fff640 1 -- 172.21.15.84:0/4286295909 learned_addr learned my addr 172.21.15.84:0/4286295909 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47ac94c640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a810f310 0x7f47a819fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47a77fe640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47a8102080 0x7f47a819b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47a7fff640 1 -- 172.21.15.84:0/4286295909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a810f310 msgr2=0x7f47a819fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47a7fff640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a810f310 0x7f47a819fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47a7fff640 1 -- 172.21.15.84:0/4286295909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47a8102080 msgr2=0x7f47a819b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47a7fff640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47a8102080 0x7f47a819b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.302+0000 7f47a7fff640 1 -- 172.21.15.84:0/4286295909 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47a81a02a0 con 0x7f47a8101760 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.303+0000 7f47a7fff640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a8101760 0x7f47a819b2b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f479800c890 tx=0x7f479800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:14.305 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.303+0000 7f47a57fa640 1 -- 172.21.15.84:0/4286295909 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798004220 con 0x7f47a8101760 2024-04-26T18:42:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.303+0000 7f47a57fa640 1 -- 172.21.15.84:0/4286295909 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47980043c0 con 0x7f47a8101760 2024-04-26T18:42:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.303+0000 7f47ae3af640 1 -- 172.21.15.84:0/4286295909 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47a81a0590 con 0x7f47a8101760 2024-04-26T18:42:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.303+0000 7f47a57fa640 1 -- 172.21.15.84:0/4286295909 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798004a00 con 0x7f47a8101760 2024-04-26T18:42:14.306 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.303+0000 7f47ae3af640 1 -- 172.21.15.84:0/4286295909 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47a81a7e20 con 0x7f47a8101760 2024-04-26T18:42:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47a57fa640 1 -- 172.21.15.84:0/4286295909 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f47980079b0 con 0x7f47a8101760 2024-04-26T18:42:14.312 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47b11a4740 1 Processor -- start 2024-04-26T18:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47b11a4740 1 -- start start 2024-04-26T18:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47b11a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95aef00 0x5646c95af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47b11a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95af820 0x5646c95b9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47b11a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95ba0e0 0x5646c95bc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47b11a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5646c9515460 con 0x5646c95af820 2024-04-26T18:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47b11a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5646c94cf590 con 0x5646c95ba0e0 2024-04-26T18:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47b11a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646c9511420 con 0x5646c95aef00 2024-04-26T18:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95af820 0x5646c95b9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47a57fa640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f47900a3dc0 0x7f47900a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95af820 0x5646c95b9ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48790/0 (socket says 172.21.15.84:48790) 2024-04-26T18:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47a7fff640 1 -- 172.21.15.84:0/1336062581 learned_addr learned my addr 172.21.15.84:0/1336062581 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47ac94c640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f47900a3dc0 0x7f47900a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.313 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47a77fe640 1 --2- 172.21.15.84:0/1336062581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95ba0e0 0x5646c95bc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47ac94c640 1 --2- 172.21.15.84:0/1336062581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95aef00 0x5646c95af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47a57fa640 1 -- 172.21.15.84:0/4286295909 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7f47980d0590 con 0x7f47a8101760 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.306+0000 7f47a7fff640 1 -- 172.21.15.84:0/1336062581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95aef00 msgr2=0x5646c95af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.307+0000 7f47a7fff640 1 --2- 172.21.15.84:0/1336062581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95aef00 0x5646c95af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.307+0000 7f47a7fff640 1 -- 172.21.15.84:0/1336062581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95ba0e0 msgr2=0x5646c95bc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.307+0000 7f47a7fff640 1 --2- 172.21.15.84:0/1336062581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95ba0e0 0x5646c95bc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.307+0000 7f47a7fff640 1 -- 172.21.15.84:0/1336062581 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c94dc0d0 con 0x5646c95af820 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.307+0000 7f47a7fff640 1 --2- 172.21.15.84:0/1336062581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95af820 0x5646c95b9ba0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f479809a160 tx=0x7f4798005760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258c820c412a6548 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.307+0000 7f47ac94c640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f47900a3dc0 0x7f47900a6280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f47a8103940 tx=0x7f47a0009b70 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.307+0000 7f478e7fc640 1 -- 172.21.15.84:0/1336062581 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479801d070 con 0x5646c95af820 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.307+0000 7f478e7fc640 1 -- 172.21.15.84:0/1336062581 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47980a5160 con 0x5646c95af820 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.307+0000 7f478e7fc640 1 -- 172.21.15.84:0/1336062581 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4798098b60 con 0x5646c95af820 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 -- 172.21.15.84:0/1336062581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95af820 msgr2=0x5646c95b9ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 --2- 172.21.15.84:0/1336062581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95af820 0x5646c95b9ba0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f479809a160 tx=0x7f4798005760 comp rx=0 tx=0).stop 2024-04-26T18:42:14.314 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 -- 172.21.15.84:0/1336062581 shutdown_connections 2024-04-26T18:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 --2- 172.21.15.84:0/1336062581 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95ba0e0 0x5646c95bc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 --2- 172.21.15.84:0/1336062581 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95af820 0x5646c95b9ba0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 --2- 172.21.15.84:0/1336062581 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95aef00 0x5646c95af2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 -- 172.21.15.84:0/1336062581 >> 172.21.15.84:0/1336062581 conn(0x5646c95ab410 msgr2=0x5646c95ab7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 -- 172.21.15.84:0/1336062581 shutdown_connections 2024-04-26T18:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 -- 172.21.15.84:0/1336062581 wait complete. 2024-04-26T18:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 -- start start 2024-04-26T18:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 Processor -- start 2024-04-26T18:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 0x5646c95b7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95af820 0x5646c95b7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.308+0000 7f47b11a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95ba0e0 0x5646c95b7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47b11a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5646c95bcd10 con 0x5646c95aef00 2024-04-26T18:42:14.315 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47b11a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5646c949cb60 con 0x5646c95af820 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47b11a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5646c95bce90 con 0x5646c95ba0e0 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47b11a4740 10 client.0 _ll_register_callbacks cb 0x5646c94c12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 0x5646c95b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 0x5646c95b7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48792/0 (socket says 172.21.15.84:48792) 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47a7fff640 1 -- 172.21.15.84:0/930245815 learned_addr learned my addr 172.21.15.84:0/930245815 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47a7fff640 1 -- 172.21.15.84:0/930245815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95ba0e0 msgr2=0x5646c95b7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47a7fff640 1 --2- 172.21.15.84:0/930245815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95ba0e0 0x5646c95b7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47a7fff640 1 -- 172.21.15.84:0/930245815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95af820 msgr2=0x5646c95b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47a7fff640 1 --2- 172.21.15.84:0/930245815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95af820 0x5646c95b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47a7fff640 1 -- 172.21.15.84:0/930245815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c9227210 con 0x5646c95aef00 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47a7fff640 1 --2- 172.21.15.84:0/930245815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 0x5646c95b7050 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f479809c480 tx=0x7f479809cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9359be7f849943b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47877fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47877fe640 1 -- 172.21.15.84:0/930245815 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479801d030 con 0x5646c95aef00 2024-04-26T18:42:14.316 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47877fe640 1 -- 172.21.15.84:0/930245815 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7f479809bb50 con 0x5646c95aef00 2024-04-26T18:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47b11a4740 1 -- 172.21.15.84:0/930245815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47980a5160 con 0x5646c95aef00 2024-04-26T18:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47877fe640 1 -- 172.21.15.84:0/930245815 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f479809c4f0 con 0x5646c95aef00 2024-04-26T18:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47b11a4740 10 client.17097 fetch_fsmap learned FSMap version 29 2024-04-26T18:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.309+0000 7f47b11a4740 1 -- 172.21.15.84:0/930245815 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646c92f3620 con 0x5646c95aef00 2024-04-26T18:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.310+0000 7f47877fe640 1 -- 172.21.15.84:0/930245815 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47980a5160 con 0x5646c95aef00 2024-04-26T18:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.310+0000 7f47877fe640 1 -- 172.21.15.84:0/930245815 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f479809b440 con 0x5646c95aef00 2024-04-26T18:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.310+0000 7f47b11a4740 10 client.17097 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.310+0000 7f47b11a4740 10 client.17097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.310+0000 7f47b11a4740 20 client.17097 populate_metadata read hostname 'smithi084' 2024-04-26T18:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.310+0000 7f47b11a4740 1 --2- 172.21.15.84:0/930245815 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5646c95ca090 0x5646c95cc480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.310+0000 7f47b11a4740 4 client.17097 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:42:14.317 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.310+0000 7f47b11a4740 1 -- 172.21.15.84:0/930245815 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646c9523070 con 0x5646c95ca090 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.310+0000 7f47a77fe640 1 --2- 172.21.15.84:0/930245815 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5646c95ca090 0x5646c95cc480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.311+0000 7f47877fe640 10 client.17097 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.311+0000 7f47a77fe640 1 --2- 172.21.15.84:0/930245815 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5646c95ca090 0x5646c95cc480 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e1cc40e0ca50c0 server_cookie=fd15e4e4dc792eef in_seq=0 out_seq=0 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.311+0000 7f47877fe640 1 -- 172.21.15.84:0/930245815 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646c9523070 con 0x5646c95ca090 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.311+0000 7f47877fe640 10 client.17097 handle_command_reply: tid=0 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.311+0000 7f47b11a4740 1 client.17097 shutdown 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.312+0000 7f47b11a4740 1 -- 172.21.15.84:0/930245815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 msgr2=0x5646c95b7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.312+0000 7f47b11a4740 1 --2- 172.21.15.84:0/930245815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 0x5646c95b7050 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f479809c480 tx=0x7f479809cd40 comp rx=0 tx=0).stop 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.312+0000 7f47b11a4740 1 -- 172.21.15.84:0/930245815 shutdown_connections 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.312+0000 7f47b11a4740 1 --2- 172.21.15.84:0/930245815 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5646c95ca090 0x5646c95cc480 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.312+0000 7f47b11a4740 1 --2- 172.21.15.84:0/930245815 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95ba0e0 0x5646c95b7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.312+0000 7f47b11a4740 1 --2- 172.21.15.84:0/930245815 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95af820 0x5646c95b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.312+0000 7f47877fe640 0 client.17097 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:14.318 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.312+0000 7f47b11a4740 1 --2- 172.21.15.84:0/930245815 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 0x5646c95b7050 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.312+0000 7f47b11a4740 1 -- 172.21.15.84:0/930245815 >> 172.21.15.84:0/930245815 conn(0x5646c95ab410 msgr2=0x5646c95aa5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.312+0000 7f47b11a4740 1 -- 172.21.15.84:0/930245815 shutdown_connections 2024-04-26T18:42:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.312+0000 7f47b11a4740 1 -- 172.21.15.84:0/930245815 wait complete. 2024-04-26T18:42:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.312+0000 7f47b11a4740 20 client.17097 trim_cache size 0 max 16384 2024-04-26T18:42:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.327+0000 7f47b11a4740 1 Processor -- start 2024-04-26T18:42:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.327+0000 7f47b11a4740 1 -- start start 2024-04-26T18:42:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.327+0000 7f47b11a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 0x5646c95b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.327+0000 7f47b11a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95af820 0x5646c95b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.327+0000 7f47b11a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95ba0e0 0x5646c95b4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.327+0000 7f47b11a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81b0870 con 0x5646c95aef00 2024-04-26T18:42:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.327+0000 7f47b11a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81b23b0 con 0x5646c95ba0e0 2024-04-26T18:42:14.332 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.327+0000 7f47b11a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81b0f40 con 0x5646c95af820 2024-04-26T18:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.327+0000 7f47a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 0x5646c95b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.328+0000 7f47a7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 0x5646c95b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48798/0 (socket says 172.21.15.84:48798) 2024-04-26T18:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.328+0000 7f47a7fff640 1 -- 172.21.15.84:0/922508395 learned_addr learned my addr 172.21.15.84:0/922508395 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.328+0000 7f47a7fff640 1 -- 172.21.15.84:0/922508395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95af820 msgr2=0x5646c95b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.328+0000 7f47ac94c640 1 --2- 172.21.15.84:0/922508395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95ba0e0 0x5646c95b4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.328+0000 7f47a7fff640 1 --2- 172.21.15.84:0/922508395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95af820 0x5646c95b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.328+0000 7f47a7fff640 1 -- 172.21.15.84:0/922508395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95ba0e0 msgr2=0x5646c95b4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.328+0000 7f47a7fff640 1 --2- 172.21.15.84:0/922508395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95ba0e0 0x5646c95b4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.328+0000 7f47a7fff640 1 -- 172.21.15.84:0/922508395 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646c95af650 con 0x5646c95aef00 2024-04-26T18:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.328+0000 7f47ac94c640 1 --2- 172.21.15.84:0/922508395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95ba0e0 0x5646c95b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.328+0000 7f47a7fff640 1 --2- 172.21.15.84:0/922508395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 0x5646c95b0fc0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4798098950 tx=0x7f47980d0520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38be6d8e591c5432 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.328+0000 7f478dffb640 1 -- 172.21.15.84:0/922508395 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479801d070 con 0x5646c95aef00 2024-04-26T18:42:14.333 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.328+0000 7f478dffb640 1 -- 172.21.15.84:0/922508395 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47980a5160 con 0x5646c95aef00 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.328+0000 7f478dffb640 1 -- 172.21.15.84:0/922508395 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f479809b3b0 con 0x5646c95aef00 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.329+0000 7f47b11a4740 1 -- 172.21.15.84:0/922508395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 msgr2=0x5646c95b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.329+0000 7f47b11a4740 1 --2- 172.21.15.84:0/922508395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 0x5646c95b0fc0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f4798098950 tx=0x7f47980d0520 comp rx=0 tx=0).stop 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.329+0000 7f47b11a4740 1 -- 172.21.15.84:0/922508395 shutdown_connections 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.329+0000 7f47b11a4740 1 --2- 172.21.15.84:0/922508395 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95ba0e0 0x5646c95b4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.329+0000 7f47b11a4740 1 --2- 172.21.15.84:0/922508395 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95af820 0x5646c95b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.329+0000 7f47b11a4740 1 --2- 172.21.15.84:0/922508395 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95aef00 0x5646c95b0fc0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.329+0000 7f47b11a4740 1 -- 172.21.15.84:0/922508395 >> 172.21.15.84:0/922508395 conn(0x5646c95ab410 msgr2=0x5646c95b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.329+0000 7f47b11a4740 1 -- 172.21.15.84:0/922508395 shutdown_connections 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.329+0000 7f47b11a4740 1 -- 172.21.15.84:0/922508395 wait complete. 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.329+0000 7f47b11a4740 1 -- start start 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.329+0000 7f47b11a4740 1 Processor -- start 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47b11a4740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95aef00 0x5646c95cf1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47b11a4740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95af820 0x5646c95cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.334 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47b11a4740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95ba0e0 0x5646c95cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47b11a4740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81b1ce0 con 0x5646c95ba0e0 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47b11a4740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81b1610 con 0x5646c95aef00 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47b11a4740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f47a81ae870 con 0x5646c95af820 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47b11a4740 10 client.0 _ll_register_callbacks cb 0x7f4798004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47ac94c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95ba0e0 0x5646c95cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47ac94c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95ba0e0 0x5646c95cd770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:48804/0 (socket says 172.21.15.84:48804) 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47ac94c640 1 -- 172.21.15.84:0/973075240 learned_addr learned my addr 172.21.15.84:0/973075240 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47a7fff640 1 --2- 172.21.15.84:0/973075240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95aef00 0x5646c95cf1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47a77fe640 1 --2- 172.21.15.84:0/973075240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95af820 0x5646c95cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47ac94c640 1 -- 172.21.15.84:0/973075240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95af820 msgr2=0x5646c95cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47ac94c640 1 --2- 172.21.15.84:0/973075240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95af820 0x5646c95cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47ac94c640 1 -- 172.21.15.84:0/973075240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95aef00 msgr2=0x5646c95cf1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47ac94c640 1 --2- 172.21.15.84:0/973075240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95aef00 0x5646c95cf1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.335 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47ac94c640 1 -- 172.21.15.84:0/973075240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646c95cde10 con 0x5646c95ba0e0 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47ac94c640 1 --2- 172.21.15.84:0/973075240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95ba0e0 0x5646c95cd770 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f47a0013010 tx=0x7f47a0009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df443285ef94a23d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47877fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47877fe640 1 -- 172.21.15.84:0/973075240 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47a000fcc0 con 0x5646c95ba0e0 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47877fe640 1 -- 172.21.15.84:0/973075240 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7f47a0004550 con 0x5646c95ba0e0 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47b11a4740 1 -- 172.21.15.84:0/973075240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47980a5160 con 0x5646c95ba0e0 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.330+0000 7f47877fe640 1 -- 172.21.15.84:0/973075240 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f47a000fe60 con 0x5646c95ba0e0 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.331+0000 7f47b11a4740 10 client.17103 fetch_fsmap learned FSMap version 29 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.331+0000 7f47b11a4740 1 -- 172.21.15.84:0/973075240 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646c9630d50 con 0x5646c95ba0e0 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.331+0000 7f47877fe640 1 -- 172.21.15.84:0/973075240 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47980a5160 con 0x5646c95ba0e0 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.331+0000 7f47877fe640 1 -- 172.21.15.84:0/973075240 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f47a001f440 con 0x5646c95ba0e0 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.331+0000 7f47b11a4740 10 client.17103 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.331+0000 7f47b11a4740 10 client.17103 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.331+0000 7f47b11a4740 20 client.17103 populate_metadata read hostname 'smithi084' 2024-04-26T18:42:14.336 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.331+0000 7f47b11a4740 1 --2- 172.21.15.84:0/973075240 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5646c9631530 0x5646c9633950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.331+0000 7f47b11a4740 4 client.17103 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:42:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.331+0000 7f47b11a4740 1 -- 172.21.15.84:0/973075240 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5646c951cee0 con 0x5646c9631530 2024-04-26T18:42:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.331+0000 7f47a7fff640 1 --2- 172.21.15.84:0/973075240 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5646c9631530 0x5646c9633950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.331+0000 7f47a7fff640 1 --2- 172.21.15.84:0/973075240 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5646c9631530 0x5646c9633950 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ddb7cf8d267e808 server_cookie=9039af83357f2459 in_seq=0 out_seq=0 2024-04-26T18:42:14.337 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.331+0000 7f47877fe640 10 client.17103 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.342+0000 7f47877fe640 1 -- 172.21.15.84:0/973075240 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5646c951cee0 con 0x5646c9631530 2024-04-26T18:42:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.342+0000 7f47877fe640 10 client.17103 handle_command_reply: tid=0 2024-04-26T18:42:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.342+0000 7f47b11a4740 1 client.17103 shutdown 2024-04-26T18:42:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.342+0000 7f47b11a4740 1 -- 172.21.15.84:0/973075240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95ba0e0 msgr2=0x5646c95cd770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.342+0000 7f47b11a4740 1 --2- 172.21.15.84:0/973075240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95ba0e0 0x5646c95cd770 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f47a0013010 tx=0x7f47a0009e70 comp rx=0 tx=0).stop 2024-04-26T18:42:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.342+0000 7f47b11a4740 1 -- 172.21.15.84:0/973075240 shutdown_connections 2024-04-26T18:42:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47b11a4740 1 --2- 172.21.15.84:0/973075240 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5646c9631530 0x5646c9633950 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.343 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47b11a4740 1 --2- 172.21.15.84:0/973075240 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5646c95ba0e0 0x5646c95cd770 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47b11a4740 1 --2- 172.21.15.84:0/973075240 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5646c95af820 0x5646c95cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47b11a4740 1 --2- 172.21.15.84:0/973075240 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5646c95aef00 0x5646c95cf1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47877fe640 0 client.17103 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47b11a4740 1 -- 172.21.15.84:0/973075240 >> 172.21.15.84:0/973075240 conn(0x5646c95ab410 msgr2=0x5646c95b08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47b11a4740 1 -- 172.21.15.84:0/973075240 shutdown_connections 2024-04-26T18:42:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47b11a4740 1 -- 172.21.15.84:0/973075240 wait complete. 2024-04-26T18:42:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47b11a4740 20 client.17103 trim_cache size 0 max 16384 2024-04-26T18:42:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:42:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:42:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:42:14.344 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:42:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47ae3af640 1 -- 172.21.15.84:0/4286295909 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f47900a3dc0 msgr2=0x7f47900a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47ae3af640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f47900a3dc0 0x7f47900a6280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f47a8103940 tx=0x7f47a0009b70 comp rx=0 tx=0).stop 2024-04-26T18:42:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47ae3af640 1 -- 172.21.15.84:0/4286295909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a8101760 msgr2=0x7f47a819b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47ae3af640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a8101760 0x7f47a819b2b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f479800c890 tx=0x7f479800cd50 comp rx=0 tx=0).stop 2024-04-26T18:42:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47ae3af640 1 -- 172.21.15.84:0/4286295909 shutdown_connections 2024-04-26T18:42:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47ae3af640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f47900a3dc0 0x7f47900a6280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.345 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47ae3af640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f47a810f310 0x7f47a819fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47ae3af640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f47a8102080 0x7f47a819b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47ae3af640 1 --2- 172.21.15.84:0/4286295909 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f47a8101760 0x7f47a819b2b0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47ae3af640 1 -- 172.21.15.84:0/4286295909 >> 172.21.15.84:0/4286295909 conn(0x7f47a80789b0 msgr2=0x7f47a8110800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47ae3af640 1 -- 172.21.15.84:0/4286295909 shutdown_connections 2024-04-26T18:42:14.346 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:14.343+0000 7f47ae3af640 1 -- 172.21.15.84:0/4286295909 wait complete. 2024-04-26T18:42:15.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:15 smithi084 ceph-mon[29141]: pgmap v449: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 103 B/s wr, 10 op/s 2024-04-26T18:42:15.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:15 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:42:15.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:15 smithi103 ceph-mon[32727]: pgmap v449: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 103 B/s wr, 10 op/s 2024-04-26T18:42:15.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:15 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:42:15.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:15 smithi174 ceph-mon[32705]: pgmap v449: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 103 B/s wr, 10 op/s 2024-04-26T18:42:15.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:15 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:42:16.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:16 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:42:16.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:16 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:42:16.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:16 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:42:16.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:16 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:42:16.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:16 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:42:16.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:16 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:42:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:16 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:42:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:16 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:42:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:16 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:42:17.788 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:42:17.816 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:17 smithi084 ceph-mon[29141]: pgmap v450: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T18:42:17.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:17 smithi103 ceph-mon[32727]: pgmap v450: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T18:42:17.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:17 smithi174 ceph-mon[32705]: pgmap v450: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T18:42:18.039 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:42:19.780 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:19 smithi084 ceph-mon[29141]: pgmap v451: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T18:42:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:19 smithi103 ceph-mon[32727]: pgmap v451: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T18:42:19.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:19 smithi174 ceph-mon[32705]: pgmap v451: 97 pgs: 97 active+clean; 2.9 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-04-26T18:42:19.997 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.994+0000 7fbb449e3700 1 -- 172.21.15.84:0/3361061696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4006bad0 msgr2=0x7fbb401102e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.994+0000 7fbb449e3700 1 --2- 172.21.15.84:0/3361061696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4006bad0 0x7fbb401102e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fbb3800b0a0 tx=0x7fbb3800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:42:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.994+0000 7fbb449e3700 1 -- 172.21.15.84:0/3361061696 shutdown_connections 2024-04-26T18:42:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.994+0000 7fbb449e3700 1 --2- 172.21.15.84:0/3361061696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4006bad0 0x7fbb401102e0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.994+0000 7fbb449e3700 1 --2- 172.21.15.84:0/3361061696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb401094a0 0x7fbb40109880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.994+0000 7fbb449e3700 1 --2- 172.21.15.84:0/3361061696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb4006b130 0x7fbb4006b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.994+0000 7fbb449e3700 1 -- 172.21.15.84:0/3361061696 >> 172.21.15.84:0/3361061696 conn(0x7fbb400792d0 msgr2=0x7fbb400796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:19.998 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.994+0000 7fbb449e3700 1 -- 172.21.15.84:0/3361061696 shutdown_connections 2024-04-26T18:42:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.994+0000 7fbb449e3700 1 -- 172.21.15.84:0/3361061696 wait complete. 2024-04-26T18:42:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.995+0000 7fbb449e3700 1 Processor -- start 2024-04-26T18:42:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.995+0000 7fbb449e3700 1 -- start start 2024-04-26T18:42:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.995+0000 7fbb449e3700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4006b130 0x7fbb4019f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.995+0000 7fbb449e3700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb4006bad0 0x7fbb4019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.995+0000 7fbb449e3700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb401094a0 0x7fbb40199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.995+0000 7fbb449e3700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fbb401135a0 con 0x7fbb4006b130 2024-04-26T18:42:19.999 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.995+0000 7fbb449e3700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40113420 con 0x7fbb4006bad0 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.995+0000 7fbb449e3700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fbb40113720 con 0x7fbb401094a0 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb3e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4006b130 0x7fbb4019f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb3e59c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4006b130 0x7fbb4019f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51006/0 (socket says 172.21.15.84:51006) 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb3e59c700 1 -- 172.21.15.84:0/3652963173 learned_addr learned my addr 172.21.15.84:0/3652963173 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb3ed9d700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb401094a0 0x7fbb40199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb3dd9b700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb4006bad0 0x7fbb4019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb3e59c700 1 -- 172.21.15.84:0/3652963173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb401094a0 msgr2=0x7fbb40199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb3e59c700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb401094a0 0x7fbb40199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb3e59c700 1 -- 172.21.15.84:0/3652963173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb4006bad0 msgr2=0x7fbb4019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb3e59c700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb4006bad0 0x7fbb4019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb3e59c700 1 -- 172.21.15.84:0/3652963173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb4019a080 con 0x7fbb4006b130 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb3e59c700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4006b130 0x7fbb4019f6f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800d950 tx=0x7fbb2800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb377fe700 1 -- 172.21.15.84:0/3652963173 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb28016070 con 0x7fbb4006b130 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb449e3700 1 -- 172.21.15.84:0/3652963173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb4019a370 con 0x7fbb4006b130 2024-04-26T18:42:20.000 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb377fe700 1 -- 172.21.15.84:0/3652963173 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb28009df0 con 0x7fbb4006b130 2024-04-26T18:42:20.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb377fe700 1 -- 172.21.15.84:0/3652963173 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb2800f3c0 con 0x7fbb4006b130 2024-04-26T18:42:20.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.996+0000 7fbb449e3700 1 -- 172.21.15.84:0/3652963173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb401a65c0 con 0x7fbb4006b130 2024-04-26T18:42:20.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.997+0000 7fbb449e3700 1 -- 172.21.15.84:0/3652963173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb4006a110 con 0x7fbb4006b130 2024-04-26T18:42:20.001 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.998+0000 7fbb377fe700 1 -- 172.21.15.84:0/3652963173 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fbb28019070 con 0x7fbb4006b130 2024-04-26T18:42:20.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.999+0000 7fbb377fe700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbb2c0a3db0 0x7fbb2c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:20.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.999+0000 7fbb3dd9b700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbb2c0a3db0 0x7fbb2c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:20.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.999+0000 7fbb377fe700 1 -- 172.21.15.84:0/3652963173 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7fbb280d03e0 con 0x7fbb4006b130 2024-04-26T18:42:20.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:19.999+0000 7fbb3dd9b700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbb2c0a3db0 0x7fbb2c0a6270 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbb30005fd0 tx=0x7fbb3000a560 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:20.002 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.003+0000 7fbb377fe700 1 -- 172.21.15.84:0/3652963173 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbb2809ca40 con 0x7fbb4006b130 2024-04-26T18:42:20.253 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.253+0000 7fbb449e3700 1 -- 172.21.15.84:0/3652963173 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb4002e490 con 0x7fbb4006b130 2024-04-26T18:42:20.253 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.253+0000 7fbb377fe700 1 -- 172.21.15.84:0/3652963173 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7fbb2809c340 con 0x7fbb4006b130 2024-04-26T18:42:20.254 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:42:20.254 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:42:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.256+0000 7fbb449e3700 1 -- 172.21.15.84:0/3652963173 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbb2c0a3db0 msgr2=0x7fbb2c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.256+0000 7fbb449e3700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbb2c0a3db0 0x7fbb2c0a6270 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fbb30005fd0 tx=0x7fbb3000a560 comp rx=0 tx=0).stop 2024-04-26T18:42:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.256+0000 7fbb449e3700 1 -- 172.21.15.84:0/3652963173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4006b130 msgr2=0x7fbb4019f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.256+0000 7fbb449e3700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4006b130 0x7fbb4019f6f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800d950 tx=0x7fbb2800dc60 comp rx=0 tx=0).stop 2024-04-26T18:42:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.256+0000 7fbb449e3700 1 -- 172.21.15.84:0/3652963173 shutdown_connections 2024-04-26T18:42:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.256+0000 7fbb449e3700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fbb2c0a3db0 0x7fbb2c0a6270 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.256+0000 7fbb449e3700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fbb4006b130 0x7fbb4019f6f0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.256+0000 7fbb449e3700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fbb401094a0 0x7fbb40199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.256+0000 7fbb449e3700 1 --2- 172.21.15.84:0/3652963173 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fbb4006bad0 0x7fbb4019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.256+0000 7fbb449e3700 1 -- 172.21.15.84:0/3652963173 >> 172.21.15.84:0/3652963173 conn(0x7fbb400792d0 msgr2=0x7fbb40100370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:20.256 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.256+0000 7fbb449e3700 1 -- 172.21.15.84:0/3652963173 shutdown_connections 2024-04-26T18:42:20.257 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:20.256+0000 7fbb449e3700 1 -- 172.21.15.84:0/3652963173 wait complete. 2024-04-26T18:42:20.257 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:42:20.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:20 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3652963173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:42:20.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:20 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3652963173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:42:20.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:20 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3652963173' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:42:20.950 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id fa6dff99 --release --await' 2024-04-26T18:42:20.950 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db --set-id fa6dff99 --release --await 2024-04-26T18:42:21.229 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:42:21.634 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:21 smithi084 ceph-mon[29141]: pgmap v452: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-26T18:42:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:21 smithi103 ceph-mon[32727]: pgmap v452: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-26T18:42:21.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:21 smithi174 ceph-mon[32705]: pgmap v452: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-26T18:42:22.778 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:22 smithi103 ceph-mon[32727]: pgmap v453: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-26T18:42:22.801 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:22 smithi084 ceph-mon[29141]: pgmap v453: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-26T18:42:22.840 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:22 smithi174 ceph-mon[32705]: pgmap v453: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 12 op/s 2024-04-26T18:42:23.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.034+0000 7f860fd9d700 1 -- 172.21.15.84:0/523434810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86081094f0 msgr2=0x7f86081098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:23.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.034+0000 7f860fd9d700 1 --2- 172.21.15.84:0/523434810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86081094f0 0x7f86081098d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8604009990 tx=0x7f8604009ca0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.034+0000 7f860fd9d700 1 -- 172.21.15.84:0/523434810 shutdown_connections 2024-04-26T18:42:23.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.034+0000 7f860fd9d700 1 --2- 172.21.15.84:0/523434810 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f86081094f0 0x7f86081098d0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.034+0000 7f860fd9d700 1 --2- 172.21.15.84:0/523434810 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8608103e90 0x7f86081102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.034+0000 7f860fd9d700 1 --2- 172.21.15.84:0/523434810 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86081034d0 0x7f8608103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.034+0000 7f860fd9d700 1 -- 172.21.15.84:0/523434810 >> 172.21.15.84:0/523434810 conn(0x7f860807a4b0 msgr2=0x7f860807a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:23.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.034+0000 7f860fd9d700 1 -- 172.21.15.84:0/523434810 shutdown_connections 2024-04-26T18:42:23.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.034+0000 7f860fd9d700 1 -- 172.21.15.84:0/523434810 wait complete. 2024-04-26T18:42:23.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.035+0000 7f860fd9d700 1 Processor -- start 2024-04-26T18:42:23.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.035+0000 7f860fd9d700 1 -- start start 2024-04-26T18:42:23.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.035+0000 7f860fd9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86081034d0 0x7f860819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.035+0000 7f860fd9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8608103e90 0x7f860819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.035+0000 7f860fd9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86081094f0 0x7f8608199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.035+0000 7f860fd9d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8608113670 con 0x7f8608103e90 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.035+0000 7f860fd9d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86081134f0 con 0x7f86081094f0 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.035+0000 7f860fd9d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86081137f0 con 0x7f86081034d0 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.035+0000 7f860d338700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8608103e90 0x7f860819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.035+0000 7f860d338700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8608103e90 0x7f860819fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51038/0 (socket says 172.21.15.84:51038) 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.035+0000 7f860d338700 1 -- 172.21.15.84:0/982899197 learned_addr learned my addr 172.21.15.84:0/982899197 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.035+0000 7f860e33a700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86081094f0 0x7f8608199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.036+0000 7f860db39700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86081034d0 0x7f860819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.036+0000 7f860d338700 1 -- 172.21.15.84:0/982899197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86081034d0 msgr2=0x7f860819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.036+0000 7f860d338700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86081034d0 0x7f860819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.036+0000 7f860d338700 1 -- 172.21.15.84:0/982899197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86081094f0 msgr2=0x7f8608199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.036+0000 7f860d338700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86081094f0 0x7f8608199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.037 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.036+0000 7f860d338700 1 -- 172.21.15.84:0/982899197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f860819a0f0 con 0x7f8608103e90 2024-04-26T18:42:23.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.036+0000 7f860db39700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86081034d0 0x7f860819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:42:23.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.036+0000 7f860d338700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8608103e90 0x7f860819fd10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b530 tx=0x7f85f800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:23.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.036+0000 7f85feffd700 1 -- 172.21.15.84:0/982899197 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8015020 con 0x7f8608103e90 2024-04-26T18:42:23.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.036+0000 7f860fd9d700 1 -- 172.21.15.84:0/982899197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f860819a3e0 con 0x7f8608103e90 2024-04-26T18:42:23.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.036+0000 7f85feffd700 1 -- 172.21.15.84:0/982899197 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f8011ce0 con 0x7f8608103e90 2024-04-26T18:42:23.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.036+0000 7f85feffd700 1 -- 172.21.15.84:0/982899197 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f8010e40 con 0x7f8608103e90 2024-04-26T18:42:23.038 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.036+0000 7f860fd9d700 1 -- 172.21.15.84:0/982899197 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8608075650 con 0x7f8608103e90 2024-04-26T18:42:23.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.038+0000 7f85feffd700 1 -- 172.21.15.84:0/982899197 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f85f8018020 con 0x7f8608103e90 2024-04-26T18:42:23.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.039+0000 7f85feffd700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f85f40a81c0 0x7f85f40aa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.039+0000 7f860db39700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f85f40a81c0 0x7f85f40aa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.039+0000 7f85feffd700 1 -- 172.21.15.84:0/982899197 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7f85f80cffd0 con 0x7f8608103e90 2024-04-26T18:42:23.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.039+0000 7f860db39700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f85f40a81c0 0x7f85f40aa680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8604000c00 tx=0x7f8604005fb0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:23.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.039+0000 7f861a62fb80 1 Processor -- start 2024-04-26T18:42:23.039 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f861a62fb80 1 -- start start 2024-04-26T18:42:23.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f861a62fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f861a62fb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 0x558d774a8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f861a62fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774a8870 0x558d774a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f861a62fb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558d773fa930 con 0x558d774ae7c0 2024-04-26T18:42:23.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f861a62fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558d77360a30 con 0x558d774adea0 2024-04-26T18:42:23.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f861a62fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558d773f68f0 con 0x558d774a8870 2024-04-26T18:42:23.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f860d338700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 0x558d774a8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f860d338700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 0x558d774a8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51042/0 (socket says 172.21.15.84:51042) 2024-04-26T18:42:23.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f860d338700 1 -- 172.21.15.84:0/2743671571 learned_addr learned my addr 172.21.15.84:0/2743671571 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:23.041 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f860db39700 1 --2- 172.21.15.84:0/2743671571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f860e33a700 1 --2- 172.21.15.84:0/2743671571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774a8870 0x558d774a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f860d338700 1 -- 172.21.15.84:0/2743671571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774a8870 msgr2=0x558d774a8cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:23.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f860d338700 1 --2- 172.21.15.84:0/2743671571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774a8870 0x558d774a8cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f860d338700 1 -- 172.21.15.84:0/2743671571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 msgr2=0x558d774ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:23.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f860d338700 1 --2- 172.21.15.84:0/2743671571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f860d338700 1 -- 172.21.15.84:0/2743671571 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d773479d0 con 0x558d774ae7c0 2024-04-26T18:42:23.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f860e33a700 1 --2- 172.21.15.84:0/2743671571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774a8870 0x558d774a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:42:23.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.040+0000 7f860db39700 1 --2- 172.21.15.84:0/2743671571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:42:23.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.041+0000 7f860d338700 1 --2- 172.21.15.84:0/2743671571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 0x558d774a8250 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f85f80984a0 tx=0x7f85f809ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=556c7fdf36cdb354 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:23.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.041+0000 7f85f3fff700 1 -- 172.21.15.84:0/2743671571 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f80a03f0 con 0x558d774ae7c0 2024-04-26T18:42:23.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.041+0000 7f85f3fff700 1 -- 172.21.15.84:0/2743671571 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f80a4cb0 con 0x558d774ae7c0 2024-04-26T18:42:23.042 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.041+0000 7f85f3fff700 1 -- 172.21.15.84:0/2743671571 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f80a4820 con 0x558d774ae7c0 2024-04-26T18:42:23.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.041+0000 7f861a62fb80 1 -- 172.21.15.84:0/2743671571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 msgr2=0x558d774a8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:23.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.041+0000 7f861a62fb80 1 --2- 172.21.15.84:0/2743671571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 0x558d774a8250 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f85f80984a0 tx=0x7f85f809ba20 comp rx=0 tx=0).stop 2024-04-26T18:42:23.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.041+0000 7f861a62fb80 1 -- 172.21.15.84:0/2743671571 shutdown_connections 2024-04-26T18:42:23.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.041+0000 7f861a62fb80 1 --2- 172.21.15.84:0/2743671571 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774a8870 0x558d774a8cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.046 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.041+0000 7f861a62fb80 1 --2- 172.21.15.84:0/2743671571 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.041+0000 7f861a62fb80 1 --2- 172.21.15.84:0/2743671571 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 0x558d774a8250 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.041+0000 7f861a62fb80 1 -- 172.21.15.84:0/2743671571 >> 172.21.15.84:0/2743671571 conn(0x558d773d66c0 msgr2=0x558d774a3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.042+0000 7f861a62fb80 1 -- 172.21.15.84:0/2743671571 shutdown_connections 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.042+0000 7f861a62fb80 1 -- 172.21.15.84:0/2743671571 wait complete. 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.042+0000 7f861a62fb80 1 -- start start 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.042+0000 7f861a62fb80 1 Processor -- start 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.042+0000 7f861a62fb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 0x558d774b4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.042+0000 7f861a62fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774b1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.042+0000 7f861a62fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774ae7c0 0x558d774b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.042+0000 7f861a62fb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558d774b51d0 con 0x558d774a8870 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.042+0000 7f861a62fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558d773b4b80 con 0x558d774adea0 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.042+0000 7f861a62fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558d774b5350 con 0x558d774ae7c0 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f861a62fb80 10 client.0 _ll_register_callbacks cb 0x558d77392a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:42:23.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f860db39700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 0x558d774b4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f860db39700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 0x558d774b4570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51050/0 (socket says 172.21.15.84:51050) 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f860db39700 1 -- 172.21.15.84:0/195741938 learned_addr learned my addr 172.21.15.84:0/195741938 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f860e33a700 1 --2- 172.21.15.84:0/195741938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774ae7c0 0x558d774b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f860db39700 1 -- 172.21.15.84:0/195741938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774ae7c0 msgr2=0x558d774b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f860d338700 1 --2- 172.21.15.84:0/195741938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774b1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f860db39700 1 --2- 172.21.15.84:0/195741938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774ae7c0 0x558d774b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f860db39700 1 -- 172.21.15.84:0/195741938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 msgr2=0x558d774b1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f860db39700 1 --2- 172.21.15.84:0/195741938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774b1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f860db39700 1 -- 172.21.15.84:0/195741938 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d773b8580 con 0x558d774a8870 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f860db39700 1 --2- 172.21.15.84:0/195741938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 0x558d774b4570 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8604015000 tx=0x7f8604009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7cbb19dbcdb685 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f85f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f85f17fa700 1 -- 172.21.15.84:0/195741938 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860400bd90 con 0x558d774a8870 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f85f17fa700 1 -- 172.21.15.84:0/195741938 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7f860400f460 con 0x558d774a8870 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f861a62fb80 1 -- 172.21.15.84:0/195741938 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f80a4cb0 con 0x558d774a8870 2024-04-26T18:42:23.048 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f85f17fa700 1 -- 172.21.15.84:0/195741938 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f860400f660 con 0x558d774a8870 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f861a62fb80 10 client.17121 fetch_fsmap learned FSMap version 29 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.043+0000 7f861a62fb80 1 -- 172.21.15.84:0/195741938 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d774bfa10 con 0x558d774a8870 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.044+0000 7f85f17fa700 1 -- 172.21.15.84:0/195741938 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f80a4cb0 con 0x558d774a8870 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.044+0000 7f85f17fa700 1 -- 172.21.15.84:0/195741938 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f860402d020 con 0x558d774a8870 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.044+0000 7f861a62fb80 10 client.17121 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.044+0000 7f861a62fb80 10 client.17121 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.044+0000 7f861a62fb80 20 client.17121 populate_metadata read hostname 'smithi084' 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.044+0000 7f861a62fb80 1 --2- 172.21.15.84:0/195741938 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558d774c2050 0x558d774c4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.044+0000 7f861a62fb80 4 client.17121 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.044+0000 7f861a62fb80 1 -- 172.21.15.84:0/195741938 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d77408310 con 0x558d774c2050 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.044+0000 7f860d338700 1 --2- 172.21.15.84:0/195741938 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558d774c2050 0x558d774c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.045+0000 7f85f17fa700 10 client.17121 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.045+0000 7f860d338700 1 --2- 172.21.15.84:0/195741938 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558d774c2050 0x558d774c4440 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a25c87de48a2d9 server_cookie=58f3dfe0820ffa3c in_seq=0 out_seq=0 2024-04-26T18:42:23.049 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.045+0000 7f85f17fa700 1 -- 172.21.15.84:0/195741938 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d77408310 con 0x558d774c2050 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.045+0000 7f85f17fa700 10 client.17121 handle_command_reply: tid=0 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.045+0000 7f861a62fb80 1 client.17121 shutdown 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.045+0000 7f861a62fb80 1 -- 172.21.15.84:0/195741938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 msgr2=0x558d774b4570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.045+0000 7f861a62fb80 1 --2- 172.21.15.84:0/195741938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 0x558d774b4570 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f8604015000 tx=0x7f8604009ca0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.046+0000 7f861a62fb80 1 -- 172.21.15.84:0/195741938 shutdown_connections 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.046+0000 7f861a62fb80 1 --2- 172.21.15.84:0/195741938 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558d774c2050 0x558d774c4440 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.046+0000 7f861a62fb80 1 --2- 172.21.15.84:0/195741938 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 0x558d774b4570 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.046+0000 7f861a62fb80 1 --2- 172.21.15.84:0/195741938 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774ae7c0 0x558d774b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.046+0000 7f85f17fa700 0 client.17121 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.046+0000 7f861a62fb80 1 --2- 172.21.15.84:0/195741938 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774b1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.046+0000 7f861a62fb80 1 -- 172.21.15.84:0/195741938 >> 172.21.15.84:0/195741938 conn(0x558d773d66c0 msgr2=0x558d774a7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.046+0000 7f861a62fb80 1 -- 172.21.15.84:0/195741938 shutdown_connections 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.046+0000 7f861a62fb80 1 -- 172.21.15.84:0/195741938 wait complete. 2024-04-26T18:42:23.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.046+0000 7f861a62fb80 20 client.17121 trim_cache size 0 max 16384 2024-04-26T18:42:23.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f861a62fb80 1 Processor -- start 2024-04-26T18:42:23.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f861a62fb80 1 -- start start 2024-04-26T18:42:23.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f861a62fb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 0x558d774a8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f861a62fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774adea0 0x558d774be0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f861a62fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774ae7c0 0x558d774c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f861a62fb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f86081abbd0 con 0x558d774a8870 2024-04-26T18:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f861a62fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86081ad710 con 0x558d774ae7c0 2024-04-26T18:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f861a62fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86081ac2a0 con 0x558d774adea0 2024-04-26T18:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f860db39700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 0x558d774a8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f860db39700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 0x558d774a8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51058/0 (socket says 172.21.15.84:51058) 2024-04-26T18:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f860db39700 1 -- 172.21.15.84:0/2854268446 learned_addr learned my addr 172.21.15.84:0/2854268446 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f860db39700 1 -- 172.21.15.84:0/2854268446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774adea0 msgr2=0x558d774be0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f860db39700 1 --2- 172.21.15.84:0/2854268446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774adea0 0x558d774be0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f860db39700 1 -- 172.21.15.84:0/2854268446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774ae7c0 msgr2=0x558d774c9cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f860db39700 1 --2- 172.21.15.84:0/2854268446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774ae7c0 0x558d774c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f860db39700 1 -- 172.21.15.84:0/2854268446 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d774c4aa0 con 0x558d774a8870 2024-04-26T18:42:23.066 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.065+0000 7f860db39700 1 --2- 172.21.15.84:0/2854268446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 0x558d774a8250 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f86040050a0 tx=0x7f860400f5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbf4bf2892bc8f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:23.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.066+0000 7f85f37fe700 1 -- 172.21.15.84:0/2854268446 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860403b030 con 0x558d774a8870 2024-04-26T18:42:23.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.066+0000 7f85f37fe700 1 -- 172.21.15.84:0/2854268446 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8604026040 con 0x558d774a8870 2024-04-26T18:42:23.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.066+0000 7f85f37fe700 1 -- 172.21.15.84:0/2854268446 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f860400faf0 con 0x558d774a8870 2024-04-26T18:42:23.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.066+0000 7f861a62fb80 1 -- 172.21.15.84:0/2854268446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 msgr2=0x558d774a8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:23.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.066+0000 7f861a62fb80 1 --2- 172.21.15.84:0/2854268446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 0x558d774a8250 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f86040050a0 tx=0x7f860400f5f0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.066+0000 7f861a62fb80 1 -- 172.21.15.84:0/2854268446 shutdown_connections 2024-04-26T18:42:23.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.066+0000 7f861a62fb80 1 --2- 172.21.15.84:0/2854268446 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774a8870 0x558d774a8250 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.066+0000 7f861a62fb80 1 --2- 172.21.15.84:0/2854268446 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774ae7c0 0x558d774c9cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.066+0000 7f861a62fb80 1 --2- 172.21.15.84:0/2854268446 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774adea0 0x558d774be0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.066+0000 7f861a62fb80 1 -- 172.21.15.84:0/2854268446 >> 172.21.15.84:0/2854268446 conn(0x558d773d66c0 msgr2=0x558d774acb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.067+0000 7f861a62fb80 1 -- 172.21.15.84:0/2854268446 shutdown_connections 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.067+0000 7f861a62fb80 1 -- 172.21.15.84:0/2854268446 wait complete. 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.067+0000 7f861a62fb80 1 -- start start 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.067+0000 7f861a62fb80 1 Processor -- start 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.067+0000 7f861a62fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774a8870 0x558d774ba940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.067+0000 7f861a62fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.067+0000 7f861a62fb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 0x558d774b8220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.067+0000 7f861a62fb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f86081ad040 con 0x558d774ae7c0 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.067+0000 7f861a62fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f86081ac970 con 0x558d774adea0 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.067+0000 7f861a62fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f86081a9b90 con 0x558d774a8870 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.067+0000 7f861a62fb80 10 client.0 _ll_register_callbacks cb 0x558d77392a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.067+0000 7f860e33a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 0x558d774b8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f860e33a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 0x558d774b8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51074/0 (socket says 172.21.15.84:51074) 2024-04-26T18:42:23.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f860e33a700 1 -- 172.21.15.84:0/3230096368 learned_addr learned my addr 172.21.15.84:0/3230096368 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f860d338700 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f860db39700 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774a8870 0x558d774ba940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f860e33a700 1 -- 172.21.15.84:0/3230096368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774a8870 msgr2=0x558d774ba940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f860e33a700 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774a8870 0x558d774ba940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f860e33a700 1 -- 172.21.15.84:0/3230096368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 msgr2=0x558d774b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f860e33a700 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774b7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f860e33a700 1 -- 172.21.15.84:0/3230096368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d774b8a30 con 0x558d774ae7c0 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f860d338700 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774b7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f860e33a700 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 0x558d774b8220 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8600004a60 tx=0x7f8600011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c159eaa0e712930 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f85f17fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f85f17fa700 1 -- 172.21.15.84:0/3230096368 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8600010040 con 0x558d774ae7c0 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f85f17fa700 1 -- 172.21.15.84:0/3230096368 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7f8600004db0 con 0x558d774ae7c0 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f861a62fb80 1 -- 172.21.15.84:0/3230096368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8604026040 con 0x558d774ae7c0 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f85f17fa700 1 -- 172.21.15.84:0/3230096368 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8600015600 con 0x558d774ae7c0 2024-04-26T18:42:23.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f861a62fb80 10 client.17127 fetch_fsmap learned FSMap version 29 2024-04-26T18:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.068+0000 7f861a62fb80 1 -- 172.21.15.84:0/3230096368 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d7752a5d0 con 0x558d774ae7c0 2024-04-26T18:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.069+0000 7f85f17fa700 1 -- 172.21.15.84:0/3230096368 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8604026040 con 0x558d774ae7c0 2024-04-26T18:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.069+0000 7f85f17fa700 1 -- 172.21.15.84:0/3230096368 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f86000055c0 con 0x558d774ae7c0 2024-04-26T18:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.069+0000 7f861a62fb80 10 client.17127 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.069+0000 7f861a62fb80 10 client.17127 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.069+0000 7f861a62fb80 20 client.17127 populate_metadata read hostname 'smithi084' 2024-04-26T18:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.069+0000 7f861a62fb80 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558d7752ad30 0x558d7752d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.069+0000 7f861a62fb80 4 client.17127 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "fa6dff99", "release": true, "await": true}] 2024-04-26T18:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.069+0000 7f861a62fb80 1 -- 172.21.15.84:0/3230096368 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "set_id": "fa6dff99", "release": true, "await": true}) -- 0x558d774023b0 con 0x558d7752ad30 2024-04-26T18:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.069+0000 7f860db39700 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558d7752ad30 0x558d7752d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:23.072 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.069+0000 7f860db39700 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558d7752ad30 0x558d7752d150 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77854cb442ca4c85 server_cookie=7f4501ad99d05117 in_seq=0 out_seq=0 2024-04-26T18:42:23.073 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.069+0000 7f85f17fa700 10 client.17127 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.074+0000 7f85f17fa700 1 -- 172.21.15.84:0/3230096368 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x558d774023b0 con 0x558d7752ad30 2024-04-26T18:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.074+0000 7f85f17fa700 10 client.17127 handle_command_reply: tid=0 2024-04-26T18:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.074+0000 7f861a62fb80 1 client.17127 shutdown 2024-04-26T18:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.074+0000 7f861a62fb80 1 -- 172.21.15.84:0/3230096368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 msgr2=0x558d774b8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.074+0000 7f861a62fb80 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 0x558d774b8220 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f8600004a60 tx=0x7f8600011c90 comp rx=0 tx=0).stop 2024-04-26T18:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.074+0000 7f861a62fb80 1 -- 172.21.15.84:0/3230096368 shutdown_connections 2024-04-26T18:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.074+0000 7f861a62fb80 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558d7752ad30 0x558d7752d150 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.074+0000 7f861a62fb80 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558d774ae7c0 0x558d774b8220 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.074+0000 7f861a62fb80 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558d774a8870 0x558d774ba940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.074+0000 7f861a62fb80 1 --2- 172.21.15.84:0/3230096368 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558d774adea0 0x558d774b7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.074+0000 7f85f17fa700 0 client.17127 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.074+0000 7f861a62fb80 1 -- 172.21.15.84:0/3230096368 >> 172.21.15.84:0/3230096368 conn(0x558d773d66c0 msgr2=0x558d774ad390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:23.075 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.074+0000 7f861a62fb80 1 -- 172.21.15.84:0/3230096368 shutdown_connections 2024-04-26T18:42:23.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f861a62fb80 1 -- 172.21.15.84:0/3230096368 wait complete. 2024-04-26T18:42:23.076 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f861a62fb80 20 client.17127 trim_cache size 0 max 16384 2024-04-26T18:42:23.076 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:42:23.076 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:42:23.076 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:42:23.076 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 40, 2024-04-26T18:42:23.076 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:42:23.076 INFO:teuthology.orchestra.run.smithi084.stdout: "fa6dff99": { 2024-04-26T18:42:23.076 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 40, 2024-04-26T18:42:23.076 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:42:23.076 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 94.4, 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:42:23.077 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:42:23.078 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:42:23.078 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:42:23.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f860fd9d700 1 -- 172.21.15.84:0/982899197 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f85f40a81c0 msgr2=0x7f85f40aa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:23.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f860fd9d700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f85f40a81c0 0x7f85f40aa680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f8604000c00 tx=0x7f8604005fb0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f860fd9d700 1 -- 172.21.15.84:0/982899197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8608103e90 msgr2=0x7f860819fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:23.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f860fd9d700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8608103e90 0x7f860819fd10 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b530 tx=0x7f85f800b840 comp rx=0 tx=0).stop 2024-04-26T18:42:23.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f860fd9d700 1 -- 172.21.15.84:0/982899197 shutdown_connections 2024-04-26T18:42:23.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f860fd9d700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f85f40a81c0 0x7f85f40aa680 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.078 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f860fd9d700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f86081094f0 0x7f8608199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f860fd9d700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f86081034d0 0x7f860819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f860fd9d700 1 --2- 172.21.15.84:0/982899197 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8608103e90 0x7f860819fd10 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:23.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f860fd9d700 1 -- 172.21.15.84:0/982899197 >> 172.21.15.84:0/982899197 conn(0x7f860807a4b0 msgr2=0x7f8608102120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:23.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f860fd9d700 1 -- 172.21.15.84:0/982899197 shutdown_connections 2024-04-26T18:42:23.079 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:42:23.075+0000 7f860fd9d700 1 -- 172.21.15.84:0/982899197 wait complete. 2024-04-26T18:42:23.787 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'fa6dff99', seconds elapsed: 46.4 2024-04-26T18:42:25.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:24 smithi084 ceph-mon[29141]: pgmap v454: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-04-26T18:42:25.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:24 smithi103 ceph-mon[32727]: pgmap v454: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-04-26T18:42:25.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:24 smithi174 ceph-mon[32705]: pgmap v454: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 7.3 KiB/s wr, 15 op/s 2024-04-26T18:42:27.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:26 smithi084 ceph-mon[29141]: pgmap v455: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-04-26T18:42:27.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:26 smithi103 ceph-mon[32727]: pgmap v455: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-04-26T18:42:27.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:26 smithi174 ceph-mon[32705]: pgmap v455: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-04-26T18:42:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.440+0000 7f974bb6a640 1 -- 172.21.15.84:0/1405688800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744102080 msgr2=0x7f974410edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.440+0000 7f974bb6a640 1 --2- 172.21.15.84:0/1405688800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744102080 0x7f974410edd0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f9734009900 tx=0x7f973402a410 comp rx=0 tx=0).stop 2024-04-26T18:42:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.440+0000 7f974bb6a640 1 -- 172.21.15.84:0/1405688800 shutdown_connections 2024-04-26T18:42:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.440+0000 7f974bb6a640 1 --2- 172.21.15.84:0/1405688800 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f974410f310 0x7f9744111700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.440+0000 7f974bb6a640 1 --2- 172.21.15.84:0/1405688800 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744102080 0x7f974410edd0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.440+0000 7f974bb6a640 1 --2- 172.21.15.84:0/1405688800 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744101760 0x7f9744101b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.440+0000 7f974bb6a640 1 -- 172.21.15.84:0/1405688800 >> 172.21.15.84:0/1405688800 conn(0x7f9744077bb0 msgr2=0x7f9744077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.440+0000 7f974bb6a640 1 -- 172.21.15.84:0/1405688800 shutdown_connections 2024-04-26T18:42:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.440+0000 7f974bb6a640 1 -- 172.21.15.84:0/1405688800 wait complete. 2024-04-26T18:42:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.440+0000 7f974bb6a640 1 Processor -- start 2024-04-26T18:42:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.440+0000 7f974bb6a640 1 -- start start 2024-04-26T18:42:27.442 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f974bb6a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744101760 0x7f974419b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f974bb6a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744102080 0x7f974419b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f974bb6a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f974410f310 0x7f974419fb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f974bb6a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f9744114e20 con 0x7f9744102080 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f974bb6a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f9744114ca0 con 0x7f9744101760 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f974bb6a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f9744114fa0 con 0x7f974410f310 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f9749105640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744102080 0x7f974419b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f9749105640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744102080 0x7f974419b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51408/0 (socket says 172.21.15.84:51408) 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f9749105640 1 -- 172.21.15.84:0/1527404556 learned_addr learned my addr 172.21.15.84:0/1527404556 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f9749906640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744101760 0x7f974419b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f974a107640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f974410f310 0x7f974419fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f9749105640 1 -- 172.21.15.84:0/1527404556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f974410f310 msgr2=0x7f974419fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f9749105640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f974410f310 0x7f974419fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f9749105640 1 -- 172.21.15.84:0/1527404556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744101760 msgr2=0x7f974419b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f9749105640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744101760 0x7f974419b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.443 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f9749105640 1 -- 172.21.15.84:0/1527404556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97441a0290 con 0x7f9744102080 2024-04-26T18:42:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f974a107640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f974410f310 0x7f974419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:42:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f9749105640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744102080 0x7f974419b7e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f9734005940 tx=0x7f97340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f973affd640 1 -- 172.21.15.84:0/1527404556 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973403f030 con 0x7f9744102080 2024-04-26T18:42:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f973affd640 1 -- 172.21.15.84:0/1527404556 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9734003cb0 con 0x7f9744102080 2024-04-26T18:42:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f974bb6a640 1 -- 172.21.15.84:0/1527404556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97441a0520 con 0x7f9744102080 2024-04-26T18:42:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f973affd640 1 -- 172.21.15.84:0/1527404556 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9734035070 con 0x7f9744102080 2024-04-26T18:42:27.444 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.441+0000 7f974bb6a640 1 -- 172.21.15.84:0/1527404556 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97441a7e10 con 0x7f9744102080 2024-04-26T18:42:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.444+0000 7f973affd640 1 -- 172.21.15.84:0/1527404556 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f9734042030 con 0x7f9744102080 2024-04-26T18:42:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.444+0000 7f974e882740 1 Processor -- start 2024-04-26T18:42:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f974e882740 1 -- start start 2024-04-26T18:42:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f973affd640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97300a3dc0 0x7f97300a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f974e882740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 0x557e2a77e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f974e882740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a788ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f974e882740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a7890e0 0x557e2a78b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f974e882740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557e2a6e4460 con 0x557e2a77df00 2024-04-26T18:42:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f974e882740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557e2a69e590 con 0x557e2a7890e0 2024-04-26T18:42:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f974e882740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557e2a6e0420 con 0x557e2a77e820 2024-04-26T18:42:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f974a107640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97300a3dc0 0x7f97300a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f9749906640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 0x557e2a77e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.451 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f973affd640 1 -- 172.21.15.84:0/1527404556 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7f97340f1090 con 0x7f9744102080 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f9749906640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 0x557e2a77e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51412/0 (socket says 172.21.15.84:51412) 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f9749906640 1 -- 172.21.15.84:0/1143885692 learned_addr learned my addr 172.21.15.84:0/1143885692 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f9749105640 1 --2- 172.21.15.84:0/1143885692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a788ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f974a107640 1 --2- 172.21.15.84:0/1143885692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a7890e0 0x557e2a78b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f9749906640 1 -- 172.21.15.84:0/1143885692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 msgr2=0x557e2a788ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f9749906640 1 --2- 172.21.15.84:0/1143885692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a788ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f9749906640 1 -- 172.21.15.84:0/1143885692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a7890e0 msgr2=0x557e2a78b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f9749906640 1 --2- 172.21.15.84:0/1143885692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a7890e0 0x557e2a78b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f9749906640 1 -- 172.21.15.84:0/1143885692 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e2a6ab0d0 con 0x557e2a77df00 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f974a107640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97300a3dc0 0x7f97300a6280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f974406af50 tx=0x7f973c0127c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.445+0000 7f9749906640 1 --2- 172.21.15.84:0/1143885692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 0x557e2a77e2e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f974000bd10 tx=0x7f97400107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ad4cbab45c4447 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.446+0000 7f972ffff640 1 -- 172.21.15.84:0/1143885692 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740004030 con 0x557e2a77df00 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.446+0000 7f972ffff640 1 -- 172.21.15.84:0/1143885692 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97400035f0 con 0x557e2a77df00 2024-04-26T18:42:27.452 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.446+0000 7f972ffff640 1 -- 172.21.15.84:0/1143885692 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740011680 con 0x557e2a77df00 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.446+0000 7f974e882740 1 -- 172.21.15.84:0/1143885692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 msgr2=0x557e2a77e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.446+0000 7f974e882740 1 --2- 172.21.15.84:0/1143885692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 0x557e2a77e2e0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f974000bd10 tx=0x7f97400107a0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.446+0000 7f974e882740 1 -- 172.21.15.84:0/1143885692 shutdown_connections 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.446+0000 7f974e882740 1 --2- 172.21.15.84:0/1143885692 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a7890e0 0x557e2a78b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.446+0000 7f974e882740 1 --2- 172.21.15.84:0/1143885692 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a788ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.446+0000 7f974e882740 1 --2- 172.21.15.84:0/1143885692 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 0x557e2a77e2e0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.446+0000 7f974e882740 1 -- 172.21.15.84:0/1143885692 >> 172.21.15.84:0/1143885692 conn(0x557e2a77a410 msgr2=0x557e2a77a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.447+0000 7f974e882740 1 -- 172.21.15.84:0/1143885692 shutdown_connections 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.447+0000 7f974e882740 1 -- 172.21.15.84:0/1143885692 wait complete. 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.447+0000 7f974e882740 1 -- start start 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.447+0000 7f974e882740 1 Processor -- start 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.447+0000 7f974e882740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a77df00 0x557e2a786010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.447+0000 7f974e882740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a786550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.447+0000 7f974e882740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 0x557e2a786a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.453 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.447+0000 7f974e882740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557e2a78bd10 con 0x557e2a7890e0 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.447+0000 7f974e882740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557e2a66bb60 con 0x557e2a77df00 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.447+0000 7f974e882740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557e2a78be90 con 0x557e2a77e820 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f974e882740 10 client.0 _ll_register_callbacks cb 0x557e2a6902b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f974a107640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 0x557e2a786a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f974a107640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 0x557e2a786a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51418/0 (socket says 172.21.15.84:51418) 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f974a107640 1 -- 172.21.15.84:0/1815792029 learned_addr learned my addr 172.21.15.84:0/1815792029 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f9749105640 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a786550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f9749906640 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a77df00 0x557e2a786010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f974a107640 1 -- 172.21.15.84:0/1815792029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 msgr2=0x557e2a786550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f974a107640 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a786550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f974a107640 1 -- 172.21.15.84:0/1815792029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a77df00 msgr2=0x557e2a786010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f974a107640 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a77df00 0x557e2a786010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f974a107640 1 -- 172.21.15.84:0/1815792029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e2a3f6210 con 0x557e2a7890e0 2024-04-26T18:42:27.454 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f9749906640 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a77df00 0x557e2a786010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f974a107640 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 0x557e2a786a90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f973c010e10 tx=0x7f973c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2407f22f7b3af87d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f972d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f972d7fa640 1 -- 172.21.15.84:0/1815792029 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c024070 con 0x557e2a7890e0 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f972d7fa640 1 -- 172.21.15.84:0/1815792029 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7f973c01e670 con 0x557e2a7890e0 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f974e882740 1 -- 172.21.15.84:0/1815792029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97400035f0 con 0x557e2a7890e0 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f972d7fa640 1 -- 172.21.15.84:0/1815792029 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f973c030050 con 0x557e2a7890e0 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f974e882740 10 client.17139 fetch_fsmap learned FSMap version 29 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.448+0000 7f974e882740 1 -- 172.21.15.84:0/1815792029 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e2a4c2620 con 0x557e2a7890e0 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.449+0000 7f972d7fa640 1 -- 172.21.15.84:0/1815792029 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97400035f0 con 0x557e2a7890e0 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.449+0000 7f972d7fa640 1 -- 172.21.15.84:0/1815792029 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f973c01c040 con 0x557e2a7890e0 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.449+0000 7f974e882740 10 client.17139 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.449+0000 7f974e882740 10 client.17139 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.449+0000 7f974e882740 20 client.17139 populate_metadata read hostname 'smithi084' 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.449+0000 7f974e882740 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557e2a7991c0 0x557e2a79b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.455 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.449+0000 7f974e882740 4 client.17139 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.449+0000 7f974e882740 1 -- 172.21.15.84:0/1815792029 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e2a6f2070 con 0x557e2a7991c0 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.449+0000 7f9749906640 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557e2a7991c0 0x557e2a79b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.450+0000 7f972d7fa640 10 client.17139 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.450+0000 7f9749906640 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557e2a7991c0 0x557e2a79b5f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b04112d43fe378 server_cookie=2dd69ee6935a9472 in_seq=0 out_seq=0 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.450+0000 7f972d7fa640 1 -- 172.21.15.84:0/1815792029 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557e2a6f2070 con 0x557e2a7991c0 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.450+0000 7f972d7fa640 10 client.17139 handle_command_reply: tid=0 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.450+0000 7f974e882740 1 client.17139 shutdown 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.450+0000 7f974e882740 1 -- 172.21.15.84:0/1815792029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 msgr2=0x557e2a786a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.450+0000 7f974e882740 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 0x557e2a786a90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f973c010e10 tx=0x7f973c004ab0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.450+0000 7f974e882740 1 -- 172.21.15.84:0/1815792029 shutdown_connections 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.450+0000 7f974e882740 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557e2a7991c0 0x557e2a79b5f0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.451+0000 7f974e882740 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 0x557e2a786a90 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.451+0000 7f974e882740 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a786550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.451+0000 7f972d7fa640 0 client.17139 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.451+0000 7f974e882740 1 --2- 172.21.15.84:0/1815792029 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a77df00 0x557e2a786010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.456 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.451+0000 7f974e882740 1 -- 172.21.15.84:0/1815792029 >> 172.21.15.84:0/1815792029 conn(0x557e2a77a410 msgr2=0x557e2a780140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.451+0000 7f974e882740 1 -- 172.21.15.84:0/1815792029 shutdown_connections 2024-04-26T18:42:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.451+0000 7f974e882740 1 -- 172.21.15.84:0/1815792029 wait complete. 2024-04-26T18:42:27.457 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.451+0000 7f974e882740 20 client.17139 trim_cache size 0 max 16384 2024-04-26T18:42:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.465+0000 7f974e882740 1 Processor -- start 2024-04-26T18:42:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.465+0000 7f974e882740 1 -- start start 2024-04-26T18:42:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f974e882740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 0x557e2a794bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f974e882740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a7754f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f974e882740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a7890e0 0x557e2a7808c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.467 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f974e882740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f97441b07a0 con 0x557e2a77df00 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f974e882740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97441b22e0 con 0x557e2a7890e0 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f974e882740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97441b0e70 con 0x557e2a77e820 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f9749906640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 0x557e2a794bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f9749906640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 0x557e2a794bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51434/0 (socket says 172.21.15.84:51434) 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f9749906640 1 -- 172.21.15.84:0/3162384896 learned_addr learned my addr 172.21.15.84:0/3162384896 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f9749105640 1 --2- 172.21.15.84:0/3162384896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a7754f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f9749906640 1 -- 172.21.15.84:0/3162384896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 msgr2=0x557e2a7754f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f9749906640 1 --2- 172.21.15.84:0/3162384896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a7754f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f9749906640 1 -- 172.21.15.84:0/3162384896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a7890e0 msgr2=0x557e2a7808c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f9749906640 1 --2- 172.21.15.84:0/3162384896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a7890e0 0x557e2a7808c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f9749906640 1 -- 172.21.15.84:0/3162384896 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e2a784fe0 con 0x557e2a77df00 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f9749906640 1 --2- 172.21.15.84:0/3162384896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 0x557e2a794bd0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f974000bd10 tx=0x7f9740003f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455e59d1d53d78cf server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f972f7fe640 1 -- 172.21.15.84:0/3162384896 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740028420 con 0x557e2a77df00 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f972f7fe640 1 -- 172.21.15.84:0/3162384896 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9740005040 con 0x557e2a77df00 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.466+0000 7f972f7fe640 1 -- 172.21.15.84:0/3162384896 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97400038e0 con 0x557e2a77df00 2024-04-26T18:42:27.468 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.467+0000 7f974e882740 1 -- 172.21.15.84:0/3162384896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 msgr2=0x557e2a794bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.467+0000 7f974e882740 1 --2- 172.21.15.84:0/3162384896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 0x557e2a794bd0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f974000bd10 tx=0x7f9740003f40 comp rx=0 tx=0).stop 2024-04-26T18:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.467+0000 7f974e882740 1 -- 172.21.15.84:0/3162384896 shutdown_connections 2024-04-26T18:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.467+0000 7f974e882740 1 --2- 172.21.15.84:0/3162384896 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a7890e0 0x557e2a7808c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.467+0000 7f974e882740 1 --2- 172.21.15.84:0/3162384896 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a7754f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.467+0000 7f974e882740 1 --2- 172.21.15.84:0/3162384896 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a77df00 0x557e2a794bd0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.467+0000 7f974e882740 1 -- 172.21.15.84:0/3162384896 >> 172.21.15.84:0/3162384896 conn(0x557e2a77a410 msgr2=0x557e2a7795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.467+0000 7f974e882740 1 -- 172.21.15.84:0/3162384896 shutdown_connections 2024-04-26T18:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.467+0000 7f974e882740 1 -- 172.21.15.84:0/3162384896 wait complete. 2024-04-26T18:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.467+0000 7f974e882740 1 -- start start 2024-04-26T18:42:27.469 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.467+0000 7f974e882740 1 Processor -- start 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974e882740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a77df00 0x557e2a7910e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974e882740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a791620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974e882740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 0x557e2a78e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974e882740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f97441b1c10 con 0x557e2a7890e0 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974e882740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f97441b1540 con 0x557e2a77df00 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974e882740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f97441ae7a0 con 0x557e2a77e820 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974e882740 10 client.0 _ll_register_callbacks cb 0x7f973c010a90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974a107640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 0x557e2a78e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974a107640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 0x557e2a78e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51448/0 (socket says 172.21.15.84:51448) 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974a107640 1 -- 172.21.15.84:0/2104471011 learned_addr learned my addr 172.21.15.84:0/2104471011 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f9749105640 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a791620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f9749906640 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a77df00 0x557e2a7910e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974a107640 1 -- 172.21.15.84:0/2104471011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 msgr2=0x557e2a791620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974a107640 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a791620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.472 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974a107640 1 -- 172.21.15.84:0/2104471011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a77df00 msgr2=0x557e2a7910e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974a107640 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a77df00 0x557e2a7910e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974a107640 1 -- 172.21.15.84:0/2104471011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e2a78e6f0 con 0x557e2a7890e0 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f9749906640 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a77df00 0x557e2a7910e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974a107640 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 0x557e2a78e0b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f973c010de0 tx=0x7f973c0073c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03e50589523e92d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f972d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f972d7fa640 1 -- 172.21.15.84:0/2104471011 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c02d020 con 0x557e2a7890e0 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f972d7fa640 1 -- 172.21.15.84:0/2104471011 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7f973c01d3d0 con 0x557e2a7890e0 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.468+0000 7f974e882740 1 -- 172.21.15.84:0/2104471011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9740005040 con 0x557e2a7890e0 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.469+0000 7f972d7fa640 1 -- 172.21.15.84:0/2104471011 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f973c024070 con 0x557e2a7890e0 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.469+0000 7f974e882740 10 client.17145 fetch_fsmap learned FSMap version 29 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.469+0000 7f974e882740 1 -- 172.21.15.84:0/2104471011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557e2a800090 con 0x557e2a7890e0 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.469+0000 7f972d7fa640 1 -- 172.21.15.84:0/2104471011 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9740005040 con 0x557e2a7890e0 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.469+0000 7f972d7fa640 1 -- 172.21.15.84:0/2104471011 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f973c03a020 con 0x557e2a7890e0 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.469+0000 7f974e882740 10 client.17145 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.469+0000 7f974e882740 10 client.17145 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.469+0000 7f974e882740 20 client.17145 populate_metadata read hostname 'smithi084' 2024-04-26T18:42:27.473 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.469+0000 7f974e882740 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557e2a800760 0x557e2a802b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.469+0000 7f974e882740 4 client.17145 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:42:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.469+0000 7f974e882740 1 -- 172.21.15.84:0/2104471011 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e2a6ebee0 con 0x557e2a800760 2024-04-26T18:42:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.469+0000 7f9749906640 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557e2a800760 0x557e2a802b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.470+0000 7f972d7fa640 10 client.17145 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:27.474 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.470+0000 7f9749906640 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557e2a800760 0x557e2a802b80 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bef0c360659d9f9 server_cookie=97cf7a220d7b1b17 in_seq=0 out_seq=0 2024-04-26T18:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.479+0000 7f972d7fa640 1 -- 172.21.15.84:0/2104471011 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557e2a6ebee0 con 0x557e2a800760 2024-04-26T18:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.479+0000 7f972d7fa640 10 client.17145 handle_command_reply: tid=0 2024-04-26T18:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.479+0000 7f974e882740 1 client.17145 shutdown 2024-04-26T18:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974e882740 1 -- 172.21.15.84:0/2104471011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 msgr2=0x557e2a78e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974e882740 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 0x557e2a78e0b0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f973c010de0 tx=0x7f973c0073c0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974e882740 1 -- 172.21.15.84:0/2104471011 shutdown_connections 2024-04-26T18:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974e882740 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557e2a800760 0x557e2a802b80 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974e882740 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557e2a7890e0 0x557e2a78e0b0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974e882740 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557e2a77e820 0x557e2a791620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.480 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974e882740 1 --2- 172.21.15.84:0/2104471011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557e2a77df00 0x557e2a7910e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f972d7fa640 0 client.17145 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974e882740 1 -- 172.21.15.84:0/2104471011 >> 172.21.15.84:0/2104471011 conn(0x557e2a77a410 msgr2=0x557e2a7795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974e882740 1 -- 172.21.15.84:0/2104471011 shutdown_connections 2024-04-26T18:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974e882740 1 -- 172.21.15.84:0/2104471011 wait complete. 2024-04-26T18:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974e882740 20 client.17145 trim_cache size 0 max 16384 2024-04-26T18:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:42:27.481 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:42:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974bb6a640 1 -- 172.21.15.84:0/1527404556 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97300a3dc0 msgr2=0x7f97300a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974bb6a640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97300a3dc0 0x7f97300a6280 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f974406af50 tx=0x7f973c0127c0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974bb6a640 1 -- 172.21.15.84:0/1527404556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744102080 msgr2=0x7f974419b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.480+0000 7f974bb6a640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744102080 0x7f974419b7e0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f9734005940 tx=0x7f97340042f0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.481+0000 7f974bb6a640 1 -- 172.21.15.84:0/1527404556 shutdown_connections 2024-04-26T18:42:27.482 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.481+0000 7f974bb6a640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f97300a3dc0 0x7f97300a6280 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.481+0000 7f974bb6a640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f974410f310 0x7f974419fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.481+0000 7f974bb6a640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f9744102080 0x7f974419b7e0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.481+0000 7f974bb6a640 1 --2- 172.21.15.84:0/1527404556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f9744101760 0x7f974419b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.481+0000 7f974bb6a640 1 -- 172.21.15.84:0/1527404556 >> 172.21.15.84:0/1527404556 conn(0x7f9744077bb0 msgr2=0x7f9744110680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.481+0000 7f974bb6a640 1 -- 172.21.15.84:0/1527404556 shutdown_connections 2024-04-26T18:42:27.483 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:27.481+0000 7f974bb6a640 1 -- 172.21.15.84:0/1527404556 wait complete. 2024-04-26T18:42:29.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:28 smithi084 ceph-mon[29141]: pgmap v456: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-04-26T18:42:29.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:28 smithi103 ceph-mon[32727]: pgmap v456: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-04-26T18:42:29.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:28 smithi174 ceph-mon[32705]: pgmap v456: 97 pgs: 97 active+clean; 3.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 6.7 KiB/s wr, 10 op/s 2024-04-26T18:42:31.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:30 smithi084 ceph-mon[29141]: pgmap v457: 97 pgs: 97 active+clean; 3.2 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 19 op/s 2024-04-26T18:42:31.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:30 smithi103 ceph-mon[32727]: pgmap v457: 97 pgs: 97 active+clean; 3.2 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 19 op/s 2024-04-26T18:42:31.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:30 smithi174 ceph-mon[32705]: pgmap v457: 97 pgs: 97 active+clean; 3.2 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 28 KiB/s wr, 19 op/s 2024-04-26T18:42:33.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:32 smithi084 ceph-mon[29141]: pgmap v458: 97 pgs: 97 active+clean; 3.2 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-26T18:42:33.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:32 smithi103 ceph-mon[32727]: pgmap v458: 97 pgs: 97 active+clean; 3.2 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-26T18:42:33.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:32 smithi174 ceph-mon[32705]: pgmap v458: 97 pgs: 97 active+clean; 3.2 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 13 op/s 2024-04-26T18:42:35.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:34 smithi084 ceph-mon[29141]: pgmap v459: 97 pgs: 97 active+clean; 3.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-26T18:42:35.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:34 smithi103 ceph-mon[32727]: pgmap v459: 97 pgs: 97 active+clean; 3.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-26T18:42:35.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:34 smithi174 ceph-mon[32705]: pgmap v459: 97 pgs: 97 active+clean; 3.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 19 op/s 2024-04-26T18:42:37.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:36 smithi084 ceph-mon[29141]: pgmap v460: 97 pgs: 97 active+clean; 3.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-26T18:42:37.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:36 smithi103 ceph-mon[32727]: pgmap v460: 97 pgs: 97 active+clean; 3.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-26T18:42:37.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:36 smithi174 ceph-mon[32705]: pgmap v460: 97 pgs: 97 active+clean; 3.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-26T18:42:39.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:38 smithi084 ceph-mon[29141]: pgmap v461: 97 pgs: 97 active+clean; 3.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-26T18:42:39.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:38 smithi103 ceph-mon[32727]: pgmap v461: 97 pgs: 97 active+clean; 3.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-26T18:42:39.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:38 smithi174 ceph-mon[32705]: pgmap v461: 97 pgs: 97 active+clean; 3.4 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 15 op/s 2024-04-26T18:42:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.568+0000 7fd2d8cf1640 1 -- 172.21.15.84:0/3377644546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2d4102220 msgr2=0x7fd2d410e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.568+0000 7fd2d8cf1640 1 --2- 172.21.15.84:0/3377644546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2d4102220 0x7fd2d410e6e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc009cd0 tx=0x7fd2cc02a8e0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.568+0000 7fd2d8cf1640 1 -- 172.21.15.84:0/3377644546 shutdown_connections 2024-04-26T18:42:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.568+0000 7fd2d8cf1640 1 --2- 172.21.15.84:0/3377644546 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2d4102220 0x7fd2d410e6e0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.568+0000 7fd2d8cf1640 1 --2- 172.21.15.84:0/3377644546 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2d4101860 0x7fd2d4101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.568+0000 7fd2d8cf1640 1 --2- 172.21.15.84:0/3377644546 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2d4107820 0x7fd2d4107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.568+0000 7fd2d8cf1640 1 -- 172.21.15.84:0/3377644546 >> 172.21.15.84:0/3377644546 conn(0x7fd2d40789b0 msgr2=0x7fd2d4078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.568+0000 7fd2d8cf1640 1 -- 172.21.15.84:0/3377644546 shutdown_connections 2024-04-26T18:42:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.568+0000 7fd2d8cf1640 1 -- 172.21.15.84:0/3377644546 wait complete. 2024-04-26T18:42:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.569+0000 7fd2d8cf1640 1 Processor -- start 2024-04-26T18:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.569+0000 7fd2d8cf1640 1 -- start start 2024-04-26T18:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d8cf1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2d4101860 0x7fd2d4199f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d8cf1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2d4102220 0x7fd2d419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d8cf1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2d4107820 0x7fd2d41a4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d8cf1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d4111ae0 con 0x7fd2d4107820 2024-04-26T18:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d8cf1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d4111960 con 0x7fd2d4102220 2024-04-26T18:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d8cf1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d4111c60 con 0x7fd2d4101860 2024-04-26T18:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2d4107820 0x7fd2d41a4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d2d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2d4107820 0x7fd2d41a4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42388/0 (socket says 172.21.15.84:42388) 2024-04-26T18:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d2d9d640 1 -- 172.21.15.84:0/2520739425 learned_addr learned my addr 172.21.15.84:0/2520739425 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d259c640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2d4101860 0x7fd2d4199f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2d4102220 0x7fd2d419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d2d9d640 1 -- 172.21.15.84:0/2520739425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2d4101860 msgr2=0x7fd2d4199f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d2d9d640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2d4101860 0x7fd2d4199f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d2d9d640 1 -- 172.21.15.84:0/2520739425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2d4102220 msgr2=0x7fd2d419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d2d9d640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2d4102220 0x7fd2d419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d2d9d640 1 -- 172.21.15.84:0/2520739425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2d41a54e0 con 0x7fd2d4107820 2024-04-26T18:42:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2d4102220 0x7fd2d419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:42:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d2d9d640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2d4107820 0x7fd2d41a4fa0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc005cc0 tx=0x7fd2cc003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2cbfff640 1 -- 172.21.15.84:0/2520739425 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc004450 con 0x7fd2d4107820 2024-04-26T18:42:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d8cf1640 1 -- 172.21.15.84:0/2520739425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2d41a56b0 con 0x7fd2d4107820 2024-04-26T18:42:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2cbfff640 1 -- 172.21.15.84:0/2520739425 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2cc031c90 con 0x7fd2d4107820 2024-04-26T18:42:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2cbfff640 1 -- 172.21.15.84:0/2520739425 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2cc030e70 con 0x7fd2d4107820 2024-04-26T18:42:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.570+0000 7fd2d8cf1640 1 -- 172.21.15.84:0/2520739425 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2d41a5b10 con 0x7fd2d4107820 2024-04-26T18:42:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2cbfff640 1 -- 172.21.15.84:0/2520739425 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fd2cc0045f0 con 0x7fd2d4107820 2024-04-26T18:42:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2dba07740 1 Processor -- start 2024-04-26T18:42:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2dba07740 1 -- start start 2024-04-26T18:42:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2dba07740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895b372e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2dba07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b37820 0x563895b41ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2dba07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b420e0 0x563895b444d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2dba07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563895a9d460 con 0x563895b420e0 2024-04-26T18:42:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2dba07740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563895a57590 con 0x563895b36f00 2024-04-26T18:42:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2dba07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563895a99420 con 0x563895b37820 2024-04-26T18:42:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2c9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b420e0 0x563895b444d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.580 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2c9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b420e0 0x563895b444d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42400/0 (socket says 172.21.15.84:42400) 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2c9d9b640 1 -- 172.21.15.84:0/2381946893 learned_addr learned my addr 172.21.15.84:0/2381946893 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2cbfff640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd2c00a3dc0 0x7fd2c00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.573+0000 7fd2d259c640 1 --2- 172.21.15.84:0/2381946893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b37820 0x563895b41ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.574+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/2381946893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895b372e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.574+0000 7fd2c9d9b640 1 -- 172.21.15.84:0/2381946893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b37820 msgr2=0x563895b41ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.574+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/2381946893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b37820 0x563895b41ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.574+0000 7fd2c9d9b640 1 -- 172.21.15.84:0/2381946893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 msgr2=0x563895b372e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.574+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/2381946893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895b372e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.574+0000 7fd2c9d9b640 1 -- 172.21.15.84:0/2381946893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563895a640d0 con 0x563895b420e0 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.574+0000 7fd2d2d9d640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd2c00a3dc0 0x7fd2c00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.574+0000 7fd2cbfff640 1 -- 172.21.15.84:0/2520739425 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7fd2cc02e030 con 0x7fd2d4107820 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.574+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/2381946893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b420e0 0x563895b444d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc013690 tx=0x7fd2bc00c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7202d0c6e5a7ebb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.574+0000 7fd2c8d99640 1 -- 172.21.15.84:0/2381946893 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc019030 con 0x563895b420e0 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.574+0000 7fd2c8d99640 1 -- 172.21.15.84:0/2381946893 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2bc011a80 con 0x563895b420e0 2024-04-26T18:42:40.581 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.574+0000 7fd2c8d99640 1 -- 172.21.15.84:0/2381946893 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc00fb00 con 0x563895b420e0 2024-04-26T18:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 -- 172.21.15.84:0/2381946893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b420e0 msgr2=0x563895b444d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2d2d9d640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd2c00a3dc0 0x7fd2c00a6280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0ef4c0 tx=0x7fd2cc088260 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 --2- 172.21.15.84:0/2381946893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b420e0 0x563895b444d0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc013690 tx=0x7fd2bc00c750 comp rx=0 tx=0).stop 2024-04-26T18:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 -- 172.21.15.84:0/2381946893 shutdown_connections 2024-04-26T18:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 --2- 172.21.15.84:0/2381946893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b420e0 0x563895b444d0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 --2- 172.21.15.84:0/2381946893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b37820 0x563895b41ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 --2- 172.21.15.84:0/2381946893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895b372e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 -- 172.21.15.84:0/2381946893 >> 172.21.15.84:0/2381946893 conn(0x563895b33410 msgr2=0x563895b337f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 -- 172.21.15.84:0/2381946893 shutdown_connections 2024-04-26T18:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 -- 172.21.15.84:0/2381946893 wait complete. 2024-04-26T18:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 -- start start 2024-04-26T18:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 Processor -- start 2024-04-26T18:42:40.582 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b36f00 0x563895b3f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b37820 0x563895b3f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b420e0 0x563895b3faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x563895b44d10 con 0x563895b36f00 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x563895a24b60 con 0x563895b420e0 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.575+0000 7fd2dba07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x563895b44e90 con 0x563895b37820 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2dba07740 10 client.0 _ll_register_callbacks cb 0x563895a492b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2c9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b36f00 0x563895b3f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2c9d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b36f00 0x563895b3f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42412/0 (socket says 172.21.15.84:42412) 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2c9d9b640 1 -- 172.21.15.84:0/2630806621 learned_addr learned my addr 172.21.15.84:0/2630806621 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2d259c640 1 --2- 172.21.15.84:0/2630806621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b37820 0x563895b3f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2c9d9b640 1 -- 172.21.15.84:0/2630806621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b37820 msgr2=0x563895b3f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/2630806621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b37820 0x563895b3f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2c9d9b640 1 -- 172.21.15.84:0/2630806621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b420e0 msgr2=0x563895b3faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:40.583 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/2630806621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b420e0 0x563895b3faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2c9d9b640 1 -- 172.21.15.84:0/2630806621 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638957af210 con 0x563895b36f00 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/2630806621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b36f00 0x563895b3f070 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc016040 tx=0x7fd2bc00c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfae6d132a3f53f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2b9ffb640 1 -- 172.21.15.84:0/2630806621 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2bc022070 con 0x563895b36f00 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2b9ffb640 1 -- 172.21.15.84:0/2630806621 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7fd2bc004030 con 0x563895b36f00 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2dba07740 1 -- 172.21.15.84:0/2630806621 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2bc011a80 con 0x563895b36f00 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2b9ffb640 1 -- 172.21.15.84:0/2630806621 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2bc02e050 con 0x563895b36f00 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2dba07740 10 client.17157 fetch_fsmap learned FSMap version 29 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2dba07740 1 -- 172.21.15.84:0/2630806621 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56389587b620 con 0x563895b36f00 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.576+0000 7fd2b9ffb640 1 -- 172.21.15.84:0/2630806621 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2bc011a80 con 0x563895b36f00 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.577+0000 7fd2b9ffb640 1 -- 172.21.15.84:0/2630806621 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd2bc00f880 con 0x563895b36f00 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.577+0000 7fd2dba07740 10 client.17157 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.577+0000 7fd2dba07740 10 client.17157 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:42:40.584 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.577+0000 7fd2dba07740 20 client.17157 populate_metadata read hostname 'smithi084' 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.577+0000 7fd2dba07740 1 --2- 172.21.15.84:0/2630806621 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563895b52170 0x563895b54560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.577+0000 7fd2dba07740 4 client.17157 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.577+0000 7fd2dba07740 1 -- 172.21.15.84:0/2630806621 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563895aab070 con 0x563895b52170 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.577+0000 7fd2d259c640 1 --2- 172.21.15.84:0/2630806621 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563895b52170 0x563895b54560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.577+0000 7fd2d259c640 1 --2- 172.21.15.84:0/2630806621 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563895b52170 0x563895b54560 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee26cfabb380907e server_cookie=fbb85808185aa6f in_seq=0 out_seq=0 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.577+0000 7fd2b9ffb640 10 client.17157 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2b9ffb640 1 -- 172.21.15.84:0/2630806621 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563895aab070 con 0x563895b52170 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2b9ffb640 10 client.17157 handle_command_reply: tid=0 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2dba07740 1 client.17157 shutdown 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2dba07740 1 -- 172.21.15.84:0/2630806621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b36f00 msgr2=0x563895b3f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2dba07740 1 --2- 172.21.15.84:0/2630806621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b36f00 0x563895b3f070 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd2bc016040 tx=0x7fd2bc00c4a0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2dba07740 1 -- 172.21.15.84:0/2630806621 shutdown_connections 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2dba07740 1 --2- 172.21.15.84:0/2630806621 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563895b52170 0x563895b54560 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.585 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2dba07740 1 --2- 172.21.15.84:0/2630806621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b420e0 0x563895b3faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2dba07740 1 --2- 172.21.15.84:0/2630806621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b37820 0x563895b3f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2dba07740 1 --2- 172.21.15.84:0/2630806621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b36f00 0x563895b3f070 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2b9ffb640 0 client.17157 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2dba07740 1 -- 172.21.15.84:0/2630806621 >> 172.21.15.84:0/2630806621 conn(0x563895b33410 msgr2=0x563895b39140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2dba07740 1 -- 172.21.15.84:0/2630806621 shutdown_connections 2024-04-26T18:42:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2dba07740 1 -- 172.21.15.84:0/2630806621 wait complete. 2024-04-26T18:42:40.586 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.578+0000 7fd2dba07740 20 client.17157 trim_cache size 0 max 16384 2024-04-26T18:42:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2dba07740 1 Processor -- start 2024-04-26T18:42:40.597 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2dba07740 1 -- start start 2024-04-26T18:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2dba07740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895b4dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2dba07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 0x563895b4a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2dba07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b420e0 0x563895b398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2dba07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d41ae500 con 0x563895b37820 2024-04-26T18:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2dba07740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d41b0040 con 0x563895b36f00 2024-04-26T18:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2dba07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d41aebd0 con 0x563895b420e0 2024-04-26T18:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2d259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 0x563895b4a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2d259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 0x563895b4a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42418/0 (socket says 172.21.15.84:42418) 2024-04-26T18:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2d259c640 1 -- 172.21.15.84:0/342420213 learned_addr learned my addr 172.21.15.84:0/342420213 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/342420213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b420e0 0x563895b398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2d259c640 1 -- 172.21.15.84:0/342420213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b420e0 msgr2=0x563895b398c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2d259c640 1 --2- 172.21.15.84:0/342420213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b420e0 0x563895b398c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.598 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2d259c640 1 -- 172.21.15.84:0/342420213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 msgr2=0x563895b4dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/342420213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895b4dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2d259c640 1 --2- 172.21.15.84:0/342420213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895b4dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2d259c640 1 -- 172.21.15.84:0/342420213 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563895b3d5c0 con 0x563895b37820 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/342420213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895b4dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2d259c640 1 --2- 172.21.15.84:0/342420213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 0x563895b4a4c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x563895b54b00 tx=0x7fd2c4009110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c97a3a90c9503e6c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.593+0000 7fd2bbfff640 1 -- 172.21.15.84:0/342420213 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4025070 con 0x563895b37820 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.594+0000 7fd2bbfff640 1 -- 172.21.15.84:0/342420213 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2c401f890 con 0x563895b37820 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.594+0000 7fd2bbfff640 1 -- 172.21.15.84:0/342420213 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4014670 con 0x563895b37820 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.594+0000 7fd2dba07740 1 -- 172.21.15.84:0/342420213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 msgr2=0x563895b4a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.594+0000 7fd2dba07740 1 --2- 172.21.15.84:0/342420213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 0x563895b4a4c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x563895b54b00 tx=0x7fd2c4009110 comp rx=0 tx=0).stop 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.594+0000 7fd2dba07740 1 -- 172.21.15.84:0/342420213 shutdown_connections 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.594+0000 7fd2dba07740 1 --2- 172.21.15.84:0/342420213 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b420e0 0x563895b398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.594+0000 7fd2dba07740 1 --2- 172.21.15.84:0/342420213 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 0x563895b4a4c0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.599 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.594+0000 7fd2dba07740 1 --2- 172.21.15.84:0/342420213 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895b4dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.594+0000 7fd2dba07740 1 -- 172.21.15.84:0/342420213 >> 172.21.15.84:0/342420213 conn(0x563895b33410 msgr2=0x563895b325b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.594+0000 7fd2dba07740 1 -- 172.21.15.84:0/342420213 shutdown_connections 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.594+0000 7fd2dba07740 1 -- 172.21.15.84:0/342420213 wait complete. 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.594+0000 7fd2dba07740 1 -- start start 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.594+0000 7fd2dba07740 1 Processor -- start 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2dba07740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895bb0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2dba07740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 0x563895bb08d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2dba07740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b420e0 0x563895bbb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2dba07740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d41af970 con 0x563895b37820 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2dba07740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d41af2a0 con 0x563895b36f00 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2dba07740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fd2d41ac500 con 0x563895b420e0 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2dba07740 10 client.0 _ll_register_callbacks cb 0x7fd2c40093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2d259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 0x563895bb08d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2d259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 0x563895bb08d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42432/0 (socket says 172.21.15.84:42432) 2024-04-26T18:42:40.600 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2d259c640 1 -- 172.21.15.84:0/3943477860 learned_addr learned my addr 172.21.15.84:0/3943477860 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895bb0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b420e0 0x563895bbb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2d259c640 1 -- 172.21.15.84:0/3943477860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b420e0 msgr2=0x563895bbb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2d259c640 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b420e0 0x563895bbb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2d259c640 1 -- 172.21.15.84:0/3943477860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 msgr2=0x563895bb0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2d259c640 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895bb0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2d259c640 1 -- 172.21.15.84:0/3943477860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563895bb1190 con 0x563895b37820 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895bb0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2d259c640 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 0x563895bb08d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4023040 tx=0x7fd2c401f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22ff9ab5ea0e4029 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2b9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2b9ffb640 1 -- 172.21.15.84:0/3943477860 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2c4025070 con 0x563895b37820 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2b9ffb640 1 -- 172.21.15.84:0/3943477860 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7fd2c4015ab0 con 0x563895b37820 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2dba07740 1 -- 172.21.15.84:0/3943477860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2c401f890 con 0x563895b37820 2024-04-26T18:42:40.601 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2b9ffb640 1 -- 172.21.15.84:0/3943477860 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2c4029400 con 0x563895b37820 2024-04-26T18:42:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2dba07740 10 client.17163 fetch_fsmap learned FSMap version 29 2024-04-26T18:42:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.595+0000 7fd2dba07740 1 -- 172.21.15.84:0/3943477860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563895bbc9a0 con 0x563895b37820 2024-04-26T18:42:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.596+0000 7fd2b9ffb640 1 -- 172.21.15.84:0/3943477860 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2c401f890 con 0x563895b37820 2024-04-26T18:42:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.596+0000 7fd2b9ffb640 1 -- 172.21.15.84:0/3943477860 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fd2c4014710 con 0x563895b37820 2024-04-26T18:42:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.596+0000 7fd2dba07740 10 client.17163 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:42:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.596+0000 7fd2dba07740 10 client.17163 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:42:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.596+0000 7fd2dba07740 20 client.17163 populate_metadata read hostname 'smithi084' 2024-04-26T18:42:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.596+0000 7fd2dba07740 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563895bbd060 0x563895bbf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.596+0000 7fd2dba07740 4 client.17163 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:42:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.596+0000 7fd2dba07740 1 -- 172.21.15.84:0/3943477860 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563895aa4ee0 con 0x563895bbd060 2024-04-26T18:42:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.596+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563895bbd060 0x563895bbf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.596+0000 7fd2c9d9b640 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563895bbd060 0x563895bbf480 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc2dcff176aa123c server_cookie=23bbbdc4e6611040 in_seq=0 out_seq=0 2024-04-26T18:42:40.602 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.596+0000 7fd2b9ffb640 10 client.17163 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2b9ffb640 1 -- 172.21.15.84:0/3943477860 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563895aa4ee0 con 0x563895bbd060 2024-04-26T18:42:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2b9ffb640 10 client.17163 handle_command_reply: tid=0 2024-04-26T18:42:40.611 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2dba07740 1 client.17163 shutdown 2024-04-26T18:42:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2dba07740 1 -- 172.21.15.84:0/3943477860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 msgr2=0x563895bb08d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2dba07740 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 0x563895bb08d0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7fd2c4023040 tx=0x7fd2c401f610 comp rx=0 tx=0).stop 2024-04-26T18:42:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2dba07740 1 -- 172.21.15.84:0/3943477860 shutdown_connections 2024-04-26T18:42:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2dba07740 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x563895bbd060 0x563895bbf480 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2dba07740 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x563895b420e0 0x563895bbb410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2b9ffb640 0 client.17163 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2dba07740 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x563895b37820 0x563895bb08d0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2dba07740 1 --2- 172.21.15.84:0/3943477860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x563895b36f00 0x563895bb0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2dba07740 1 -- 172.21.15.84:0/3943477860 >> 172.21.15.84:0/3943477860 conn(0x563895b33410 msgr2=0x563895b3efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2dba07740 1 -- 172.21.15.84:0/3943477860 shutdown_connections 2024-04-26T18:42:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2dba07740 1 -- 172.21.15.84:0/3943477860 wait complete. 2024-04-26T18:42:40.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2dba07740 20 client.17163 trim_cache size 0 max 16384 2024-04-26T18:42:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:42:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:42:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:42:40.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:42:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2d8cf1640 1 -- 172.21.15.84:0/2520739425 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd2c00a3dc0 msgr2=0x7fd2c00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2d8cf1640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd2c00a3dc0 0x7fd2c00a6280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc0ef4c0 tx=0x7fd2cc088260 comp rx=0 tx=0).stop 2024-04-26T18:42:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2d8cf1640 1 -- 172.21.15.84:0/2520739425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2d4107820 msgr2=0x7fd2d41a4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.611+0000 7fd2d8cf1640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2d4107820 0x7fd2d41a4fa0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc005cc0 tx=0x7fd2cc003c20 comp rx=0 tx=0).stop 2024-04-26T18:42:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.612+0000 7fd2d8cf1640 1 -- 172.21.15.84:0/2520739425 shutdown_connections 2024-04-26T18:42:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.612+0000 7fd2d8cf1640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fd2c00a3dc0 0x7fd2c00a6280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.612+0000 7fd2d8cf1640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fd2d4107820 0x7fd2d41a4fa0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.612+0000 7fd2d8cf1640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fd2d4102220 0x7fd2d419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.612+0000 7fd2d8cf1640 1 --2- 172.21.15.84:0/2520739425 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fd2d4101860 0x7fd2d4199f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.612+0000 7fd2d8cf1640 1 -- 172.21.15.84:0/2520739425 >> 172.21.15.84:0/2520739425 conn(0x7fd2d40789b0 msgr2=0x7fd2d41042e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:40.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.612+0000 7fd2d8cf1640 1 -- 172.21.15.84:0/2520739425 shutdown_connections 2024-04-26T18:42:40.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:40.612+0000 7fd2d8cf1640 1 -- 172.21.15.84:0/2520739425 wait complete. 2024-04-26T18:42:41.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:40 smithi084 ceph-mon[29141]: pgmap v462: 97 pgs: 97 active+clean; 3.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-26T18:42:41.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:40 smithi103 ceph-mon[32727]: pgmap v462: 97 pgs: 97 active+clean; 3.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-26T18:42:41.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:40 smithi174 ceph-mon[32705]: pgmap v462: 97 pgs: 97 active+clean; 3.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-26T18:42:43.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:42 smithi084 ceph-mon[29141]: pgmap v463: 97 pgs: 97 active+clean; 3.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-26T18:42:43.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:42 smithi103 ceph-mon[32727]: pgmap v463: 97 pgs: 97 active+clean; 3.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-26T18:42:43.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:42 smithi174 ceph-mon[32705]: pgmap v463: 97 pgs: 97 active+clean; 3.6 MiB data, 441 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-26T18:42:45.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:44 smithi084 ceph-mon[29141]: pgmap v464: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-26T18:42:45.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:44 smithi103 ceph-mon[32727]: pgmap v464: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-26T18:42:45.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:44 smithi174 ceph-mon[32705]: pgmap v464: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-26T18:42:47.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:46 smithi084 ceph-mon[29141]: pgmap v465: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:42:47.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:46 smithi103 ceph-mon[32727]: pgmap v465: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:42:47.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:46 smithi174 ceph-mon[32705]: pgmap v465: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:42:49.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:48 smithi084 ceph-mon[29141]: pgmap v466: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:42:49.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:48 smithi103 ceph-mon[32727]: pgmap v466: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:42:49.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:48 smithi174 ceph-mon[32705]: pgmap v466: 97 pgs: 97 active+clean; 3.8 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:42:49.303 INFO:tasks.workunit.client.0.smithi084.stdout:437/600: open fd = 3 2024-04-26T18:42:49.303 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.303 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.303 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.303 INFO:tasks.workunit.client.0.smithi084.stdout:438/600: open fd = 3 2024-04-26T18:42:49.303 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.303 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.303 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:439/600: open fd = 3 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:440/600: open fd = 3 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:441/600: open fd = 3 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:442/600: open fd = 3 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.304 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:443/600: open fd = 3 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:444/600: open fd = 3 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:445/600: open fd = 3 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:446/600: open fd = 3 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.305 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.306 INFO:tasks.workunit.client.0.smithi084.stdout:447/600: open fd = 3 2024-04-26T18:42:49.306 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.306 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.306 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.306 INFO:tasks.workunit.client.0.smithi084.stdout:448/600: open fd = 3 2024-04-26T18:42:49.306 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.306 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.306 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.306 INFO:tasks.workunit.client.0.smithi084.stdout:449/600: open fd = 3 2024-04-26T18:42:49.306 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.306 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.306 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:450/600: open fd = 3 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:451/600: open fd = 3 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:452/600: open fd = 3 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:453/600: open fd = 3 2024-04-26T18:42:49.307 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.308 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.308 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.308 INFO:tasks.workunit.client.0.smithi084.stdout:454/600: open fd = 3 2024-04-26T18:42:49.308 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.308 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.308 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.308 INFO:tasks.workunit.client.0.smithi084.stdout:455/600: open fd = 3 2024-04-26T18:42:49.308 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.308 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.308 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.308 INFO:tasks.workunit.client.0.smithi084.stdout:456/600: open fd = 3 2024-04-26T18:42:49.308 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.308 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:457/600: open fd = 3 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:458/600: open fd = 3 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:459/600: open fd = 3 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:460/600: open fd = 3 2024-04-26T18:42:49.309 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:461/600: open fd = 3 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:462/600: open fd = 3 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:463/600: open fd = 3 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.310 INFO:tasks.workunit.client.0.smithi084.stdout:464/600: open fd = 3 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:465/600: open fd = 3 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:466/600: open fd = 3 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:467/600: open fd = 3 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:468/600: open fd = 3 2024-04-26T18:42:49.311 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:469/600: open fd = 3 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:470/600: open fd = 3 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:471/600: open fd = 3 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.312 INFO:tasks.workunit.client.0.smithi084.stdout:472/600: open fd = 3 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:473/600: open fd = 3 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:474/600: open fd = 3 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:475/600: open fd = 3 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.313 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:476/600: open fd = 3 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:477/600: open fd = 3 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:478/600: open fd = 3 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:479/600: open fd = 3 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.314 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:480/600: open fd = 3 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:481/600: open fd = 3 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:482/600: open fd = 3 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:483/600: open fd = 3 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.315 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:484/600: open fd = 3 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:485/600: open fd = 3 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:486/600: open fd = 3 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:487/600: open fd = 3 2024-04-26T18:42:49.316 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:488/600: open fd = 3 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:489/600: open fd = 3 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:490/600: open fd = 3 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:491/600: open fd = 3 2024-04-26T18:42:49.317 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:492/600: open fd = 3 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:493/600: open fd = 3 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:494/600: open fd = 3 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.318 INFO:tasks.workunit.client.0.smithi084.stdout:495/600: open fd = 3 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:496/600: open fd = 3 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:497/600: open fd = 3 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:498/600: open fd = 3 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:42:49.319 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:42:51.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:50 smithi084 ceph-mon[29141]: pgmap v467: 97 pgs: 97 active+clean; 4.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:42:51.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:50 smithi103 ceph-mon[32727]: pgmap v467: 97 pgs: 97 active+clean; 4.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:42:51.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:50 smithi174 ceph-mon[32705]: pgmap v467: 97 pgs: 97 active+clean; 4.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:42:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.707+0000 7fc9c592b640 1 -- 172.21.15.84:0/3570246353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9c0108930 msgr2=0x7fc9c0108d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.707+0000 7fc9c592b640 1 --2- 172.21.15.84:0/3570246353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9c0108930 0x7fc9c0108d10 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009930 tx=0x7fc9b002a420 comp rx=0 tx=0).stop 2024-04-26T18:42:53.709 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.707+0000 7fc9c592b640 1 -- 172.21.15.84:0/3570246353 shutdown_connections 2024-04-26T18:42:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.707+0000 7fc9c592b640 1 --2- 172.21.15.84:0/3570246353 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9c01032e0 0x7fc9c010f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.707+0000 7fc9c592b640 1 --2- 172.21.15.84:0/3570246353 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9c0102920 0x7fc9c0102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.707+0000 7fc9c592b640 1 --2- 172.21.15.84:0/3570246353 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9c0108930 0x7fc9c0108d10 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.707+0000 7fc9c592b640 1 -- 172.21.15.84:0/3570246353 >> 172.21.15.84:0/3570246353 conn(0x7fc9c00fe3f0 msgr2=0x7fc9c0100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.707+0000 7fc9c592b640 1 -- 172.21.15.84:0/3570246353 shutdown_connections 2024-04-26T18:42:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.707+0000 7fc9c592b640 1 -- 172.21.15.84:0/3570246353 wait complete. 2024-04-26T18:42:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.707+0000 7fc9c592b640 1 Processor -- start 2024-04-26T18:42:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9c592b640 1 -- start start 2024-04-26T18:42:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9c592b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9c0102920 0x7fc9c0199130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9c592b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9c01032e0 0x7fc9c0199670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9c592b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9c0108930 0x7fc9c019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9c592b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c0112bf0 con 0x7fc9c0102920 2024-04-26T18:42:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9c592b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c0112a70 con 0x7fc9c0108930 2024-04-26T18:42:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9c592b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c0112d70 con 0x7fc9c01032e0 2024-04-26T18:42:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9beffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9c0102920 0x7fc9c0199130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9beffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9c0102920 0x7fc9c0199130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60836/0 (socket says 172.21.15.84:60836) 2024-04-26T18:42:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9beffd640 1 -- 172.21.15.84:0/1829196604 learned_addr learned my addr 172.21.15.84:0/1829196604 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9bf7fe640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9c0108930 0x7fc9c019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9be7fc640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9c01032e0 0x7fc9c0199670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9beffd640 1 -- 172.21.15.84:0/1829196604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9c01032e0 msgr2=0x7fc9c0199670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9beffd640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9c01032e0 0x7fc9c0199670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9beffd640 1 -- 172.21.15.84:0/1829196604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9c0108930 msgr2=0x7fc9c019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9beffd640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9c0108930 0x7fc9c019d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9beffd640 1 -- 172.21.15.84:0/1829196604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c019e0d0 con 0x7fc9c0102920 2024-04-26T18:42:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9be7fc640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9c01032e0 0x7fc9c0199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:42:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.708+0000 7fc9beffd640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9c0102920 0x7fc9c0199130 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0004770 tx=0x7fc9b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.709+0000 7fc9c4929640 1 -- 172.21.15.84:0/1829196604 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b003f070 con 0x7fc9c0102920 2024-04-26T18:42:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.709+0000 7fc9c592b640 1 -- 172.21.15.84:0/1829196604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c019e360 con 0x7fc9c0102920 2024-04-26T18:42:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.709+0000 7fc9c4929640 1 -- 172.21.15.84:0/1829196604 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9b003b460 con 0x7fc9c0102920 2024-04-26T18:42:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.709+0000 7fc9c4929640 1 -- 172.21.15.84:0/1829196604 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0035070 con 0x7fc9c0102920 2024-04-26T18:42:53.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.709+0000 7fc9c592b640 1 -- 172.21.15.84:0/1829196604 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c01a5c50 con 0x7fc9c0102920 2024-04-26T18:42:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9c4929640 1 -- 172.21.15.84:0/1829196604 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fc9b003b8d0 con 0x7fc9c0102920 2024-04-26T18:42:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9c8648740 1 Processor -- start 2024-04-26T18:42:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9c8648740 1 -- start start 2024-04-26T18:42:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9c8648740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 0x55d672fd71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9c8648740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fd7720 0x55d672fe5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9c8648740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fe64f0 0x55d672fe88e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9c8648740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d672f3d460 con 0x55d672fd6e00 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9c8648740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d672ef7590 con 0x55d672fe64f0 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9c8648740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d672f39420 con 0x55d672fd7720 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9bf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 0x55d672fd71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9c4929640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc9a80a3dc0 0x7fc9a80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9bf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 0x55d672fd71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60842/0 (socket says 172.21.15.84:60842) 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9bf7fe640 1 -- 172.21.15.84:0/545212959 learned_addr learned my addr 172.21.15.84:0/545212959 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9be7fc640 1 --2- 172.21.15.84:0/545212959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fe64f0 0x55d672fe88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.712+0000 7fc9bf7fe640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc9a80a3dc0 0x7fc9a80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.713+0000 7fc9c4929640 1 -- 172.21.15.84:0/1829196604 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7fc9b00f0b80 con 0x7fc9c0102920 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.713+0000 7fc9beffd640 1 --2- 172.21.15.84:0/545212959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fd7720 0x55d672fe5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.713+0000 7fc9bf7fe640 1 -- 172.21.15.84:0/545212959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fd7720 msgr2=0x55d672fe5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.713+0000 7fc9bf7fe640 1 --2- 172.21.15.84:0/545212959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fd7720 0x55d672fe5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.713+0000 7fc9bf7fe640 1 -- 172.21.15.84:0/545212959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fe64f0 msgr2=0x55d672fe88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.713+0000 7fc9bf7fe640 1 --2- 172.21.15.84:0/545212959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fe64f0 0x55d672fe88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.713+0000 7fc9bf7fe640 1 -- 172.21.15.84:0/545212959 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d672f040d0 con 0x55d672fd6e00 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.713+0000 7fc9beffd640 1 --2- 172.21.15.84:0/545212959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fd7720 0x55d672fe5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.713+0000 7fc9be7fc640 1 --2- 172.21.15.84:0/545212959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fe64f0 0x55d672fe88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.713+0000 7fc9bf7fe640 1 --2- 172.21.15.84:0/545212959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 0x55d672fd71e0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8014a60 tx=0x7fc9b8014f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a601a522d3fd1af1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.713+0000 7fc9a5ffb640 1 -- 172.21.15.84:0/545212959 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b801ec50 con 0x55d672fd6e00 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.713+0000 7fc9a5ffb640 1 -- 172.21.15.84:0/545212959 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9b8023070 con 0x55d672fd6e00 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.713+0000 7fc9a5ffb640 1 -- 172.21.15.84:0/545212959 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8005520 con 0x55d672fd6e00 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.714+0000 7fc9c8648740 1 -- 172.21.15.84:0/545212959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 msgr2=0x55d672fd71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.714+0000 7fc9bf7fe640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc9a80a3dc0 0x7fc9a80a6280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8022000 tx=0x7fc9b801e000 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.714+0000 7fc9c8648740 1 --2- 172.21.15.84:0/545212959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 0x55d672fd71e0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8014a60 tx=0x7fc9b8014f20 comp rx=0 tx=0).stop 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.714+0000 7fc9c8648740 1 -- 172.21.15.84:0/545212959 shutdown_connections 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.714+0000 7fc9c8648740 1 --2- 172.21.15.84:0/545212959 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fe64f0 0x55d672fe88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.714+0000 7fc9c8648740 1 --2- 172.21.15.84:0/545212959 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fd7720 0x55d672fe5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.714+0000 7fc9c8648740 1 --2- 172.21.15.84:0/545212959 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 0x55d672fd71e0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.714+0000 7fc9c8648740 1 -- 172.21.15.84:0/545212959 >> 172.21.15.84:0/545212959 conn(0x55d672fd3410 msgr2=0x55d672fd37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.714+0000 7fc9c8648740 1 -- 172.21.15.84:0/545212959 shutdown_connections 2024-04-26T18:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.714+0000 7fc9c8648740 1 -- 172.21.15.84:0/545212959 wait complete. 2024-04-26T18:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.714+0000 7fc9c8648740 1 -- start start 2024-04-26T18:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.714+0000 7fc9c8648740 1 Processor -- start 2024-04-26T18:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9c8648740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fd6e00 0x55d672fe3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9c8648740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd7720 0x55d672fe38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9c8648740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fe64f0 0x55d672fe3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9c8648740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55d672fe9120 con 0x55d672fd7720 2024-04-26T18:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9c8648740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55d672ec4b60 con 0x55d672fd6e00 2024-04-26T18:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9c8648740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55d672fe92a0 con 0x55d672fe64f0 2024-04-26T18:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9c8648740 10 client.0 _ll_register_callbacks cb 0x55d672ee92b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:42:53.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9be7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd7720 0x55d672fe38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9be7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd7720 0x55d672fe38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60852/0 (socket says 172.21.15.84:60852) 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9be7fc640 1 -- 172.21.15.84:0/3853653221 learned_addr learned my addr 172.21.15.84:0/3853653221 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9beffd640 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fd6e00 0x55d672fe3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9bf7fe640 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fe64f0 0x55d672fe3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9be7fc640 1 -- 172.21.15.84:0/3853653221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fe64f0 msgr2=0x55d672fe3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9be7fc640 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fe64f0 0x55d672fe3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9be7fc640 1 -- 172.21.15.84:0/3853653221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fd6e00 msgr2=0x55d672fe3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9be7fc640 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fd6e00 0x55d672fe3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9be7fc640 1 -- 172.21.15.84:0/3853653221 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d672c4f210 con 0x55d672fd7720 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9bf7fe640 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fe64f0 0x55d672fe3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9be7fc640 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd7720 0x55d672fe38d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400be80 tx=0x7fc9b40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bea1c1f7e106ca6b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc99effd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc99effd640 1 -- 172.21.15.84:0/3853653221 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4012650 con 0x55d672fd7720 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc99effd640 1 -- 172.21.15.84:0/3853653221 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7fc9b40127f0 con 0x55d672fd7720 2024-04-26T18:42:53.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.715+0000 7fc9c8648740 1 -- 172.21.15.84:0/3853653221 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9b8023070 con 0x55d672fd7720 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.716+0000 7fc99effd640 1 -- 172.21.15.84:0/3853653221 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc9b40129f0 con 0x55d672fd7720 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.716+0000 7fc9c8648740 10 client.17175 fetch_fsmap learned FSMap version 29 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.716+0000 7fc9c8648740 1 -- 172.21.15.84:0/3853653221 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d672d1b620 con 0x55d672fd7720 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.716+0000 7fc99effd640 1 -- 172.21.15.84:0/3853653221 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9b8023070 con 0x55d672fd7720 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.716+0000 7fc99effd640 1 -- 172.21.15.84:0/3853653221 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc9b4021030 con 0x55d672fd7720 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.716+0000 7fc9c8648740 10 client.17175 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.716+0000 7fc9c8648740 10 client.17175 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.716+0000 7fc9c8648740 20 client.17175 populate_metadata read hostname 'smithi084' 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.716+0000 7fc9c8648740 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d672ff66f0 0x55d672ff8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.716+0000 7fc9c8648740 4 client.17175 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.716+0000 7fc9c8648740 1 -- 172.21.15.84:0/3853653221 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d672f4b070 con 0x55d672ff66f0 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.716+0000 7fc9beffd640 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d672ff66f0 0x55d672ff8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.717+0000 7fc9beffd640 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d672ff66f0 0x55d672ff8ae0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=117dcb076d41da43 server_cookie=c0d32361c82674f6 in_seq=0 out_seq=0 2024-04-26T18:42:53.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.717+0000 7fc99effd640 10 client.17175 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.717+0000 7fc99effd640 1 -- 172.21.15.84:0/3853653221 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d672f4b070 con 0x55d672ff66f0 2024-04-26T18:42:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.717+0000 7fc99effd640 10 client.17175 handle_command_reply: tid=0 2024-04-26T18:42:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.717+0000 7fc9c8648740 1 client.17175 shutdown 2024-04-26T18:42:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.717+0000 7fc9c8648740 1 -- 172.21.15.84:0/3853653221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd7720 msgr2=0x55d672fe38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.717+0000 7fc9c8648740 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd7720 0x55d672fe38d0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400be80 tx=0x7fc9b40117b0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.717+0000 7fc9c8648740 1 -- 172.21.15.84:0/3853653221 shutdown_connections 2024-04-26T18:42:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.717+0000 7fc9c8648740 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d672ff66f0 0x55d672ff8ae0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.718+0000 7fc9c8648740 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fe64f0 0x55d672fe3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.718+0000 7fc9c8648740 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd7720 0x55d672fe38d0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.718+0000 7fc99effd640 0 client.17175 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:53.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.718+0000 7fc9c8648740 1 --2- 172.21.15.84:0/3853653221 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fd6e00 0x55d672fe3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.718+0000 7fc9c8648740 1 -- 172.21.15.84:0/3853653221 >> 172.21.15.84:0/3853653221 conn(0x55d672fd3410 msgr2=0x55d672fd9280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.718+0000 7fc9c8648740 1 -- 172.21.15.84:0/3853653221 shutdown_connections 2024-04-26T18:42:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.718+0000 7fc9c8648740 1 -- 172.21.15.84:0/3853653221 wait complete. 2024-04-26T18:42:53.726 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.718+0000 7fc9c8648740 20 client.17175 trim_cache size 0 max 16384 2024-04-26T18:42:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.732+0000 7fc9c8648740 1 Processor -- start 2024-04-26T18:42:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.732+0000 7fc9c8648740 1 -- start start 2024-04-26T18:42:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9c8648740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 0x55d672ff2020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9c8648740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fd7720 0x55d672fe2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9c8648740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fe64f0 0x55d672fe3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9c8648740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01ac3d0 con 0x55d672fd6e00 2024-04-26T18:42:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9c8648740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01adf10 con 0x55d672fd7720 2024-04-26T18:42:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9c8648740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01acaa0 con 0x55d672fe64f0 2024-04-26T18:42:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9beffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 0x55d672ff2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9beffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 0x55d672ff2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60860/0 (socket says 172.21.15.84:60860) 2024-04-26T18:42:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9beffd640 1 -- 172.21.15.84:0/1415645488 learned_addr learned my addr 172.21.15.84:0/1415645488 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9beffd640 1 -- 172.21.15.84:0/1415645488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fe64f0 msgr2=0x55d672fe3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9beffd640 1 --2- 172.21.15.84:0/1415645488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fe64f0 0x55d672fe3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9beffd640 1 -- 172.21.15.84:0/1415645488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fd7720 msgr2=0x55d672fe2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:53.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9beffd640 1 --2- 172.21.15.84:0/1415645488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fd7720 0x55d672fe2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9beffd640 1 -- 172.21.15.84:0/1415645488 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d672fd7550 con 0x55d672fd6e00 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9beffd640 1 --2- 172.21.15.84:0/1415645488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 0x55d672ff2020 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00b8f40 tx=0x7fc9b003bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f2fe271a4b9439a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9a57fa640 1 -- 172.21.15.84:0/1415645488 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b0048040 con 0x55d672fd6e00 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9a57fa640 1 -- 172.21.15.84:0/1415645488 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9b00c5750 con 0x55d672fd6e00 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.733+0000 7fc9a57fa640 1 -- 172.21.15.84:0/1415645488 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b00b9150 con 0x55d672fd6e00 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.734+0000 7fc9c8648740 1 -- 172.21.15.84:0/1415645488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 msgr2=0x55d672ff2020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.734+0000 7fc9c8648740 1 --2- 172.21.15.84:0/1415645488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 0x55d672ff2020 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc9b00b8f40 tx=0x7fc9b003bc70 comp rx=0 tx=0).stop 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.734+0000 7fc9c8648740 1 -- 172.21.15.84:0/1415645488 shutdown_connections 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.734+0000 7fc9c8648740 1 --2- 172.21.15.84:0/1415645488 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fe64f0 0x55d672fe3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.734+0000 7fc9c8648740 1 --2- 172.21.15.84:0/1415645488 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fd7720 0x55d672fe2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.734+0000 7fc9c8648740 1 --2- 172.21.15.84:0/1415645488 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fd6e00 0x55d672ff2020 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.734+0000 7fc9c8648740 1 -- 172.21.15.84:0/1415645488 >> 172.21.15.84:0/1415645488 conn(0x55d672fd3410 msgr2=0x55d672fd1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.734+0000 7fc9c8648740 1 -- 172.21.15.84:0/1415645488 shutdown_connections 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.734+0000 7fc9c8648740 1 -- 172.21.15.84:0/1415645488 wait complete. 2024-04-26T18:42:53.739 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.734+0000 7fc9c8648740 1 -- start start 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.734+0000 7fc9c8648740 1 Processor -- start 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9c8648740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fd6e00 0x55d672fe0000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9c8648740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fd7720 0x55d672fe0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9c8648740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fe64f0 0x55d672fe0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9c8648740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01ad840 con 0x55d672fe64f0 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9c8648740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01ad170 con 0x55d672fd6e00 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9c8648740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c01aa3d0 con 0x55d672fd7720 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9c8648740 10 client.0 _ll_register_callbacks cb 0x7fc9b400b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9bf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fe64f0 0x55d672fe0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9bf7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fe64f0 0x55d672fe0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60870/0 (socket says 172.21.15.84:60870) 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9bf7fe640 1 -- 172.21.15.84:0/1685662545 learned_addr learned my addr 172.21.15.84:0/1685662545 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9be7fc640 1 --2- 172.21.15.84:0/1685662545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fd7720 0x55d672fe0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9bf7fe640 1 -- 172.21.15.84:0/1685662545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fd7720 msgr2=0x55d672fe0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9bf7fe640 1 --2- 172.21.15.84:0/1685662545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fd7720 0x55d672fe0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9bf7fe640 1 -- 172.21.15.84:0/1685662545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fd6e00 msgr2=0x55d672fe0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9bf7fe640 1 --2- 172.21.15.84:0/1685662545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fd6e00 0x55d672fe0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9bf7fe640 1 -- 172.21.15.84:0/1685662545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d67305bc40 con 0x55d672fe64f0 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9be7fc640 1 --2- 172.21.15.84:0/1685662545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fd7720 0x55d672fe0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9bf7fe640 1 --2- 172.21.15.84:0/1685662545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fe64f0 0x55d672fe0a80 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8014990 tx=0x7fc9b8012960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12c0f00026cbd0c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc99effd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc99effd640 1 -- 172.21.15.84:0/1685662545 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9b8012df0 con 0x55d672fe64f0 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc99effd640 1 -- 172.21.15.84:0/1685662545 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(112..112 src has 1..112) ==== 8579+0+0 (secure 0 0 0) 0x7fc9b80057c0 con 0x55d672fe64f0 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc9c8648740 1 -- 172.21.15.84:0/1685662545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9b00c5750 con 0x55d672fe64f0 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.735+0000 7fc99effd640 1 -- 172.21.15.84:0/1685662545 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc9b803f050 con 0x55d672fe64f0 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.736+0000 7fc9c8648740 10 client.17181 fetch_fsmap learned FSMap version 29 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.736+0000 7fc9c8648740 1 -- 172.21.15.84:0/1685662545 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d67305ce90 con 0x55d672fe64f0 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.736+0000 7fc99effd640 1 -- 172.21.15.84:0/1685662545 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9b00c5750 con 0x55d672fe64f0 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.736+0000 7fc99effd640 1 -- 172.21.15.84:0/1685662545 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fc9b803a030 con 0x55d672fe64f0 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.736+0000 7fc9c8648740 10 client.17181 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:42:53.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.736+0000 7fc9c8648740 10 client.17181 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:42:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.736+0000 7fc9c8648740 20 client.17181 populate_metadata read hostname 'smithi084' 2024-04-26T18:42:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.736+0000 7fc9c8648740 1 --2- 172.21.15.84:0/1685662545 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d67305d410 0x55d67305f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:42:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.736+0000 7fc9c8648740 4 client.17181 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:42:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.736+0000 7fc9c8648740 1 -- 172.21.15.84:0/1685662545 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d672f44ee0 con 0x55d67305d410 2024-04-26T18:42:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.736+0000 7fc9beffd640 1 --2- 172.21.15.84:0/1685662545 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d67305d410 0x55d67305f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:42:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.736+0000 7fc99effd640 10 client.17181 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:53.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.736+0000 7fc9beffd640 1 --2- 172.21.15.84:0/1685662545 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d67305d410 0x55d67305f830 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0ac5ac12445b0ca server_cookie=80dfcf9d47f1aa1e in_seq=0 out_seq=0 2024-04-26T18:42:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.750+0000 7fc99effd640 1 -- 172.21.15.84:0/1685662545 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d672f44ee0 con 0x55d67305d410 2024-04-26T18:42:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.750+0000 7fc99effd640 10 client.17181 handle_command_reply: tid=0 2024-04-26T18:42:53.751 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.750+0000 7fc9c8648740 1 client.17181 shutdown 2024-04-26T18:42:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.750+0000 7fc9c8648740 1 -- 172.21.15.84:0/1685662545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fe64f0 msgr2=0x55d672fe0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.750+0000 7fc9c8648740 1 --2- 172.21.15.84:0/1685662545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fe64f0 0x55d672fe0a80 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8014990 tx=0x7fc9b8012960 comp rx=0 tx=0).stop 2024-04-26T18:42:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c8648740 1 -- 172.21.15.84:0/1685662545 shutdown_connections 2024-04-26T18:42:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c8648740 1 --2- 172.21.15.84:0/1685662545 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55d67305d410 0x55d67305f830 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c8648740 1 --2- 172.21.15.84:0/1685662545 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55d672fe64f0 0x55d672fe0a80 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc99effd640 0 client.17181 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:42:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c8648740 1 --2- 172.21.15.84:0/1685662545 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55d672fd7720 0x55d672fe0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c8648740 1 --2- 172.21.15.84:0/1685662545 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55d672fd6e00 0x55d672fe0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c8648740 1 -- 172.21.15.84:0/1685662545 >> 172.21.15.84:0/1685662545 conn(0x55d672fd3410 msgr2=0x55d672fd25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c8648740 1 -- 172.21.15.84:0/1685662545 shutdown_connections 2024-04-26T18:42:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c8648740 1 -- 172.21.15.84:0/1685662545 wait complete. 2024-04-26T18:42:53.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c8648740 20 client.17181 trim_cache size 0 max 16384 2024-04-26T18:42:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:42:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:42:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:42:53.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:42:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c592b640 1 -- 172.21.15.84:0/1829196604 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc9a80a3dc0 msgr2=0x7fc9a80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c592b640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc9a80a3dc0 0x7fc9a80a6280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8022000 tx=0x7fc9b801e000 comp rx=0 tx=0).stop 2024-04-26T18:42:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c592b640 1 -- 172.21.15.84:0/1829196604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9c0102920 msgr2=0x7fc9c0199130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:42:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c592b640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9c0102920 0x7fc9c0199130 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0004770 tx=0x7fc9b00042f0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c592b640 1 -- 172.21.15.84:0/1829196604 shutdown_connections 2024-04-26T18:42:53.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c592b640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fc9a80a3dc0 0x7fc9a80a6280 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c592b640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fc9c0108930 0x7fc9c019d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c592b640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fc9c01032e0 0x7fc9c0199670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c592b640 1 --2- 172.21.15.84:0/1829196604 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fc9c0102920 0x7fc9c0199130 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:42:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c592b640 1 -- 172.21.15.84:0/1829196604 >> 172.21.15.84:0/1829196604 conn(0x7fc9c00fe3f0 msgr2=0x7fc9c00fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:42:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c592b640 1 -- 172.21.15.84:0/1829196604 shutdown_connections 2024-04-26T18:42:53.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:42:53.751+0000 7fc9c592b640 1 -- 172.21.15.84:0/1829196604 wait complete. 2024-04-26T18:42:53.830 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:53 smithi103 ceph-mon[32727]: pgmap v468: 97 pgs: 97 active+clean; 4.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:42:53.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:53 smithi084 ceph-mon[29141]: pgmap v468: 97 pgs: 97 active+clean; 4.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:42:53.892 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:53 smithi174 ceph-mon[32705]: pgmap v468: 97 pgs: 97 active+clean; 4.0 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:42:55.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:55 smithi084 ceph-mon[29141]: pgmap v469: 97 pgs: 97 active+clean; 4.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:42:55.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:55 smithi103 ceph-mon[32727]: pgmap v469: 97 pgs: 97 active+clean; 4.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:42:55.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:55 smithi174 ceph-mon[32705]: pgmap v469: 97 pgs: 97 active+clean; 4.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-26T18:42:57.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:57 smithi084 ceph-mon[29141]: pgmap v470: 97 pgs: 97 active+clean; 4.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:42:57.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:57 smithi103 ceph-mon[32727]: pgmap v470: 97 pgs: 97 active+clean; 4.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:42:57.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:57 smithi174 ceph-mon[32705]: pgmap v470: 97 pgs: 97 active+clean; 4.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:42:59.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:42:59 smithi084 ceph-mon[29141]: pgmap v471: 97 pgs: 97 active+clean; 4.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:42:59.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:42:59 smithi103 ceph-mon[32727]: pgmap v471: 97 pgs: 97 active+clean; 4.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:42:59.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:42:59 smithi174 ceph-mon[32705]: pgmap v471: 97 pgs: 97 active+clean; 4.2 MiB data, 446 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 13 op/s 2024-04-26T18:43:01.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:01 smithi084 ceph-mon[29141]: pgmap v472: 97 pgs: 97 active+clean; 4.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-26T18:43:01.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:01 smithi103 ceph-mon[32727]: pgmap v472: 97 pgs: 97 active+clean; 4.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-26T18:43:01.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:01 smithi174 ceph-mon[32705]: pgmap v472: 97 pgs: 97 active+clean; 4.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-26T18:43:03.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:03 smithi084 ceph-mon[29141]: pgmap v473: 97 pgs: 97 active+clean; 4.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:43:03.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:03 smithi103 ceph-mon[32727]: pgmap v473: 97 pgs: 97 active+clean; 4.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:43:03.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:03 smithi174 ceph-mon[32705]: pgmap v473: 97 pgs: 97 active+clean; 4.4 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-26T18:43:05.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:05 smithi084 ceph-mon[29141]: pgmap v474: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-26T18:43:05.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:05 smithi084 ceph-mon[29141]: osdmap e113: 12 total, 12 up, 12 in 2024-04-26T18:43:05.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:05 smithi103 ceph-mon[32727]: pgmap v474: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-26T18:43:05.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:05 smithi103 ceph-mon[32727]: osdmap e113: 12 total, 12 up, 12 in 2024-04-26T18:43:05.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:05 smithi174 ceph-mon[32705]: pgmap v474: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 104 KiB/s wr, 39 op/s 2024-04-26T18:43:05.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:05 smithi174 ceph-mon[32705]: osdmap e113: 12 total, 12 up, 12 in 2024-04-26T18:43:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.847+0000 7fed4b083640 1 -- 172.21.15.84:0/1247985196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed44107820 msgr2=0x7fed44107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.847+0000 7fed4b083640 1 --2- 172.21.15.84:0/1247985196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed44107820 0x7fed44107c00 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fed38009930 tx=0x7fed3802a420 comp rx=0 tx=0).stop 2024-04-26T18:43:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.847+0000 7fed4b083640 1 -- 172.21.15.84:0/1247985196 shutdown_connections 2024-04-26T18:43:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.847+0000 7fed4b083640 1 --2- 172.21.15.84:0/1247985196 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed44102220 0x7fed4410e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.847+0000 7fed4b083640 1 --2- 172.21.15.84:0/1247985196 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed44101860 0x7fed44101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.847+0000 7fed4b083640 1 --2- 172.21.15.84:0/1247985196 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed44107820 0x7fed44107c00 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.847+0000 7fed4b083640 1 -- 172.21.15.84:0/1247985196 >> 172.21.15.84:0/1247985196 conn(0x7fed440789b0 msgr2=0x7fed44078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:06.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.847+0000 7fed4b083640 1 -- 172.21.15.84:0/1247985196 shutdown_connections 2024-04-26T18:43:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.847+0000 7fed4b083640 1 -- 172.21.15.84:0/1247985196 wait complete. 2024-04-26T18:43:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed4b083640 1 Processor -- start 2024-04-26T18:43:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed4b083640 1 -- start start 2024-04-26T18:43:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed4b083640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed44101860 0x7fed4419a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed4b083640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed44102220 0x7fed4419a540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed4b083640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed44107820 0x7fed441a5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed4b083640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fed44111b00 con 0x7fed44102220 2024-04-26T18:43:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed4b083640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed44111980 con 0x7fed44107820 2024-04-26T18:43:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed4b083640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed44111c80 con 0x7fed44101860 2024-04-26T18:43:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed44102220 0x7fed4419a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed44102220 0x7fed4419a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47168/0 (socket says 172.21.15.84:47168) 2024-04-26T18:43:06.852 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed43fff640 1 -- 172.21.15.84:0/16323272 learned_addr learned my addr 172.21.15.84:0/16323272 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed48e1f640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed44101860 0x7fed4419a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed49620640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed44107820 0x7fed441a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed43fff640 1 -- 172.21.15.84:0/16323272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed44101860 msgr2=0x7fed4419a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.848+0000 7fed43fff640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed44101860 0x7fed4419a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.849+0000 7fed43fff640 1 -- 172.21.15.84:0/16323272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed44107820 msgr2=0x7fed441a5080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.849+0000 7fed43fff640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed44107820 0x7fed441a5080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.849+0000 7fed43fff640 1 -- 172.21.15.84:0/16323272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed441a55c0 con 0x7fed44102220 2024-04-26T18:43:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.849+0000 7fed49620640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed44107820 0x7fed441a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.849+0000 7fed43fff640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed44102220 0x7fed4419a540 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fed3000d770 tx=0x7fed3000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.849+0000 7fed41ffb640 1 -- 172.21.15.84:0/16323272 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30017070 con 0x7fed44102220 2024-04-26T18:43:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.849+0000 7fed41ffb640 1 -- 172.21.15.84:0/16323272 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed300054d0 con 0x7fed44102220 2024-04-26T18:43:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.849+0000 7fed4b083640 1 -- 172.21.15.84:0/16323272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed441a5790 con 0x7fed44102220 2024-04-26T18:43:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.849+0000 7fed41ffb640 1 -- 172.21.15.84:0/16323272 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30004c30 con 0x7fed44102220 2024-04-26T18:43:06.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.849+0000 7fed4b083640 1 -- 172.21.15.84:0/16323272 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed441a5c60 con 0x7fed44102220 2024-04-26T18:43:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.851+0000 7fed41ffb640 1 -- 172.21.15.84:0/16323272 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fed30004e10 con 0x7fed44102220 2024-04-26T18:43:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed41ffb640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fed340a3dc0 0x7fed340a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed4dda5740 1 Processor -- start 2024-04-26T18:43:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed4dda5740 1 -- start start 2024-04-26T18:43:06.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed48e1f640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fed340a3dc0 0x7fed340a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed41ffb640 1 -- 172.21.15.84:0/16323272 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(113..113 src has 1..113) ==== 8623+0+0 (secure 0 0 0) 0x7fed300d0330 con 0x7fed44102220 2024-04-26T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed4dda5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839bef00 0x556c839bf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed4dda5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bf820 0x556c839c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed4dda5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839ca0e0 0x556c839cc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed4dda5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556c83925460 con 0x556c839bef00 2024-04-26T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed4dda5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c838df590 con 0x556c839ca0e0 2024-04-26T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed4dda5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c83921420 con 0x556c839bf820 2024-04-26T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed49620640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839ca0e0 0x556c839cc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed48e1f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839bef00 0x556c839bf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed49620640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839ca0e0 0x556c839cc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.103:3300/0 says I am v2:172.21.15.84:37550/0 (socket says 172.21.15.84:37550) 2024-04-26T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed49620640 1 -- 172.21.15.84:0/3626156071 learned_addr learned my addr 172.21.15.84:0/3626156071 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed43fff640 1 --2- 172.21.15.84:0/3626156071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bf820 0x556c839c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.852+0000 7fed48e1f640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fed340a3dc0 0x7fed340a6280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fed38009900 tx=0x7fed38005fd0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.853+0000 7fed48e1f640 1 -- 172.21.15.84:0/3626156071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bf820 msgr2=0x556c839c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.853+0000 7fed48e1f640 1 --2- 172.21.15.84:0/3626156071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bf820 0x556c839c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.853+0000 7fed48e1f640 1 -- 172.21.15.84:0/3626156071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839ca0e0 msgr2=0x556c839cc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.853+0000 7fed48e1f640 1 --2- 172.21.15.84:0/3626156071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839ca0e0 0x556c839cc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.853+0000 7fed48e1f640 1 -- 172.21.15.84:0/3626156071 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c838ec0d0 con 0x556c839bef00 2024-04-26T18:43:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.853+0000 7fed48e1f640 1 --2- 172.21.15.84:0/3626156071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839bef00 0x556c839bf2e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fed38031440 tx=0x7fed38045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281b86d32df39e6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.853+0000 7fed2effd640 1 -- 172.21.15.84:0/3626156071 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38035040 con 0x556c839bef00 2024-04-26T18:43:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.853+0000 7fed2effd640 1 -- 172.21.15.84:0/3626156071 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed38003aa0 con 0x556c839bef00 2024-04-26T18:43:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.853+0000 7fed2effd640 1 -- 172.21.15.84:0/3626156071 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3803c9c0 con 0x556c839bef00 2024-04-26T18:43:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.854+0000 7fed4dda5740 1 -- 172.21.15.84:0/3626156071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839bef00 msgr2=0x556c839bf2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:06.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.854+0000 7fed4dda5740 1 --2- 172.21.15.84:0/3626156071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839bef00 0x556c839bf2e0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fed38031440 tx=0x7fed38045000 comp rx=0 tx=0).stop 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.854+0000 7fed4dda5740 1 -- 172.21.15.84:0/3626156071 shutdown_connections 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.854+0000 7fed4dda5740 1 --2- 172.21.15.84:0/3626156071 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839ca0e0 0x556c839cc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.854+0000 7fed4dda5740 1 --2- 172.21.15.84:0/3626156071 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bf820 0x556c839c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.854+0000 7fed4dda5740 1 --2- 172.21.15.84:0/3626156071 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839bef00 0x556c839bf2e0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.854+0000 7fed4dda5740 1 -- 172.21.15.84:0/3626156071 >> 172.21.15.84:0/3626156071 conn(0x556c839bb410 msgr2=0x556c839bb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.854+0000 7fed4dda5740 1 -- 172.21.15.84:0/3626156071 shutdown_connections 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.854+0000 7fed4dda5740 1 -- 172.21.15.84:0/3626156071 wait complete. 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.854+0000 7fed4dda5740 1 -- start start 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.854+0000 7fed4dda5740 1 Processor -- start 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed4dda5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bef00 0x556c839c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed4dda5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bf820 0x556c839c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed4dda5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 0x556c839c7a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed4dda5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556c839cccb0 con 0x556c839ca0e0 2024-04-26T18:43:06.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed4dda5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556c838acb60 con 0x556c839bf820 2024-04-26T18:43:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed4dda5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556c839cce30 con 0x556c839bef00 2024-04-26T18:43:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed4dda5740 10 client.0 _ll_register_callbacks cb 0x556c838d12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed49620640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 0x556c839c7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed49620640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 0x556c839c7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47192/0 (socket says 172.21.15.84:47192) 2024-04-26T18:43:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed49620640 1 -- 172.21.15.84:0/1672795858 learned_addr learned my addr 172.21.15.84:0/1672795858 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed48e1f640 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bef00 0x556c839c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed43fff640 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bf820 0x556c839c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed49620640 1 -- 172.21.15.84:0/1672795858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bef00 msgr2=0x556c839c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed49620640 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bef00 0x556c839c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed49620640 1 -- 172.21.15.84:0/1672795858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bf820 msgr2=0x556c839c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed49620640 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bf820 0x556c839c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.870 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed49620640 1 -- 172.21.15.84:0/1672795858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c83637210 con 0x556c839ca0e0 2024-04-26T18:43:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed43fff640 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bf820 0x556c839c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed48e1f640 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bef00 0x556c839c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:43:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed49620640 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 0x556c839c7a60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00ce90 tx=0x7fed3c007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1944f94599b13d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed23fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed23fff640 1 -- 172.21.15.84:0/1672795858 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c016400 con 0x556c839ca0e0 2024-04-26T18:43:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed23fff640 1 -- 172.21.15.84:0/1672795858 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8623+0+0 (secure 0 0 0) 0x7fed3c0165a0 con 0x556c839ca0e0 2024-04-26T18:43:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.855+0000 7fed4dda5740 1 -- 172.21.15.84:0/1672795858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed38003aa0 con 0x556c839ca0e0 2024-04-26T18:43:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.856+0000 7fed23fff640 1 -- 172.21.15.84:0/1672795858 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fed3c024050 con 0x556c839ca0e0 2024-04-26T18:43:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.856+0000 7fed4dda5740 10 client.17193 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.856+0000 7fed4dda5740 1 -- 172.21.15.84:0/1672795858 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c83703620 con 0x556c839ca0e0 2024-04-26T18:43:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.856+0000 7fed23fff640 1 -- 172.21.15.84:0/1672795858 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed38003aa0 con 0x556c839ca0e0 2024-04-26T18:43:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.856+0000 7fed23fff640 1 -- 172.21.15.84:0/1672795858 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fed3c00a590 con 0x556c839ca0e0 2024-04-26T18:43:06.871 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.856+0000 7fed4dda5740 10 client.17193 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.856+0000 7fed4dda5740 10 client.17193 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.856+0000 7fed4dda5740 20 client.17193 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.856+0000 7fed4dda5740 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556c839da170 0x556c839dc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.856+0000 7fed4dda5740 4 client.17193 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.856+0000 7fed4dda5740 1 -- 172.21.15.84:0/1672795858 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c83933070 con 0x556c839da170 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.856+0000 7fed48e1f640 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556c839da170 0x556c839dc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.857+0000 7fed23fff640 10 client.17193 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.857+0000 7fed48e1f640 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556c839da170 0x556c839dc560 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fafd9b6853fb12f server_cookie=da531ec0a4c7173a in_seq=0 out_seq=0 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.857+0000 7fed23fff640 1 -- 172.21.15.84:0/1672795858 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c83933070 con 0x556c839da170 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.857+0000 7fed23fff640 10 client.17193 handle_command_reply: tid=0 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.857+0000 7fed4dda5740 1 client.17193 shutdown 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.858+0000 7fed4dda5740 1 -- 172.21.15.84:0/1672795858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 msgr2=0x556c839c7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.858+0000 7fed4dda5740 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 0x556c839c7a60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00ce90 tx=0x7fed3c007680 comp rx=0 tx=0).stop 2024-04-26T18:43:06.872 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.858+0000 7fed4dda5740 1 -- 172.21.15.84:0/1672795858 shutdown_connections 2024-04-26T18:43:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.858+0000 7fed4dda5740 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556c839da170 0x556c839dc560 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.858+0000 7fed4dda5740 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 0x556c839c7a60 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.858+0000 7fed4dda5740 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bf820 0x556c839c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.858+0000 7fed4dda5740 1 --2- 172.21.15.84:0/1672795858 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bef00 0x556c839c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.858+0000 7fed23fff640 0 client.17193 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.858+0000 7fed4dda5740 1 -- 172.21.15.84:0/1672795858 >> 172.21.15.84:0/1672795858 conn(0x556c839bb410 msgr2=0x556c839c1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.858+0000 7fed4dda5740 1 -- 172.21.15.84:0/1672795858 shutdown_connections 2024-04-26T18:43:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.858+0000 7fed4dda5740 1 -- 172.21.15.84:0/1672795858 wait complete. 2024-04-26T18:43:06.873 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.858+0000 7fed4dda5740 20 client.17193 trim_cache size 0 max 16384 2024-04-26T18:43:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed4dda5740 1 Processor -- start 2024-04-26T18:43:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed4dda5740 1 -- start start 2024-04-26T18:43:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed4dda5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bef00 0x556c839d5b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed4dda5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839bf820 0x556c839c18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed4dda5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839ca0e0 0x556c839e1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.878 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed4dda5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ae720 con 0x556c839bf820 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed4dda5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed441b0260 con 0x556c839bef00 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed4dda5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed441aedf0 con 0x556c839ca0e0 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839bf820 0x556c839c18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed43fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839bf820 0x556c839c18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47198/0 (socket says 172.21.15.84:47198) 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed43fff640 1 -- 172.21.15.84:0/1587841705 learned_addr learned my addr 172.21.15.84:0/1587841705 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed48e1f640 1 --2- 172.21.15.84:0/1587841705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bef00 0x556c839d5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed43fff640 1 -- 172.21.15.84:0/1587841705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839ca0e0 msgr2=0x556c839e1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed43fff640 1 --2- 172.21.15.84:0/1587841705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839ca0e0 0x556c839e1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed43fff640 1 -- 172.21.15.84:0/1587841705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bef00 msgr2=0x556c839d5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed43fff640 1 --2- 172.21.15.84:0/1587841705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bef00 0x556c839d5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.873+0000 7fed43fff640 1 -- 172.21.15.84:0/1587841705 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c839dcbc0 con 0x556c839bf820 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.874+0000 7fed43fff640 1 --2- 172.21.15.84:0/1587841705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839bf820 0x556c839c18c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fed3009bf70 tx=0x7fed3009ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b4e292583a78a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.874+0000 7fed2e7fc640 1 -- 172.21.15.84:0/1587841705 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed300cf040 con 0x556c839bf820 2024-04-26T18:43:06.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.874+0000 7fed2e7fc640 1 -- 172.21.15.84:0/1587841705 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed300a4ee0 con 0x556c839bf820 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.874+0000 7fed2e7fc640 1 -- 172.21.15.84:0/1587841705 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed30020040 con 0x556c839bf820 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.874+0000 7fed4dda5740 1 -- 172.21.15.84:0/1587841705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839bf820 msgr2=0x556c839c18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.874+0000 7fed4dda5740 1 --2- 172.21.15.84:0/1587841705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839bf820 0x556c839c18c0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fed3009bf70 tx=0x7fed3009ca80 comp rx=0 tx=0).stop 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.874+0000 7fed4dda5740 1 -- 172.21.15.84:0/1587841705 shutdown_connections 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.874+0000 7fed4dda5740 1 --2- 172.21.15.84:0/1587841705 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839ca0e0 0x556c839e1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.874+0000 7fed4dda5740 1 --2- 172.21.15.84:0/1587841705 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839bf820 0x556c839c18c0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.874+0000 7fed4dda5740 1 --2- 172.21.15.84:0/1587841705 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bef00 0x556c839d5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.874+0000 7fed4dda5740 1 -- 172.21.15.84:0/1587841705 >> 172.21.15.84:0/1587841705 conn(0x556c839bb410 msgr2=0x556c839c7830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.874+0000 7fed4dda5740 1 -- 172.21.15.84:0/1587841705 shutdown_connections 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.874+0000 7fed4dda5740 1 -- 172.21.15.84:0/1587841705 wait complete. 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed4dda5740 1 -- start start 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed4dda5740 1 Processor -- start 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed4dda5740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bef00 0x556c839d2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed4dda5740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bf820 0x556c839cd790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed4dda5740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 0x556c839cdcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed4dda5740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fed441afb90 con 0x556c839ca0e0 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed4dda5740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fed441af4c0 con 0x556c839bef00 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed4dda5740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fed441ac720 con 0x556c839bf820 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed4dda5740 10 client.0 _ll_register_callbacks cb 0x7fed3c00caa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed49620640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 0x556c839cdcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed48e1f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bef00 0x556c839d2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed49620640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 0x556c839cdcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47208/0 (socket says 172.21.15.84:47208) 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed49620640 1 -- 172.21.15.84:0/3270105193 learned_addr learned my addr 172.21.15.84:0/3270105193 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.875+0000 7fed43fff640 1 --2- 172.21.15.84:0/3270105193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bf820 0x556c839cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed49620640 1 -- 172.21.15.84:0/3270105193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bf820 msgr2=0x556c839cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed49620640 1 --2- 172.21.15.84:0/3270105193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bf820 0x556c839cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed49620640 1 -- 172.21.15.84:0/3270105193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bef00 msgr2=0x556c839d2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:06.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed49620640 1 --2- 172.21.15.84:0/3270105193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bef00 0x556c839d2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed49620640 1 -- 172.21.15.84:0/3270105193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c839ce500 con 0x556c839ca0e0 2024-04-26T18:43:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed48e1f640 1 --2- 172.21.15.84:0/3270105193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bef00 0x556c839d2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:43:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed49620640 1 --2- 172.21.15.84:0/3270105193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 0x556c839cdcd0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00a9b0 tx=0x7fed3c007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b9f48bfde47a243 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed23fff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed23fff640 1 -- 172.21.15.84:0/3270105193 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3c009e90 con 0x556c839ca0e0 2024-04-26T18:43:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed23fff640 1 -- 172.21.15.84:0/3270105193 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(113..113 src has 1..113) ==== 8623+0+0 (secure 0 0 0) 0x7fed3c00a610 con 0x556c839ca0e0 2024-04-26T18:43:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed4dda5740 1 -- 172.21.15.84:0/3270105193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed300a4ee0 con 0x556c839ca0e0 2024-04-26T18:43:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed23fff640 1 -- 172.21.15.84:0/3270105193 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fed3c009640 con 0x556c839ca0e0 2024-04-26T18:43:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed4dda5740 10 client.17199 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed4dda5740 1 -- 172.21.15.84:0/3270105193 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c83a406a0 con 0x556c839ca0e0 2024-04-26T18:43:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed23fff640 1 -- 172.21.15.84:0/3270105193 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed300a4ee0 con 0x556c839ca0e0 2024-04-26T18:43:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed23fff640 1 -- 172.21.15.84:0/3270105193 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fed3c016400 con 0x556c839ca0e0 2024-04-26T18:43:06.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed4dda5740 10 client.17199 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed4dda5740 10 client.17199 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:43:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed4dda5740 20 client.17199 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed4dda5740 1 --2- 172.21.15.84:0/3270105193 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556c83a40df0 0x556c83a43210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed4dda5740 4 client.17199 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:43:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.876+0000 7fed4dda5740 1 -- 172.21.15.84:0/3270105193 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c8392cee0 con 0x556c83a40df0 2024-04-26T18:43:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.877+0000 7fed48e1f640 1 --2- 172.21.15.84:0/3270105193 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556c83a40df0 0x556c83a43210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.877+0000 7fed48e1f640 1 --2- 172.21.15.84:0/3270105193 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556c83a40df0 0x556c83a43210 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=529ca6c48aa01da3 server_cookie=4da3be913bda9f26 in_seq=0 out_seq=0 2024-04-26T18:43:06.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.877+0000 7fed23fff640 10 client.17199 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed23fff640 1 -- 172.21.15.84:0/3270105193 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c8392cee0 con 0x556c83a40df0 2024-04-26T18:43:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed23fff640 10 client.17199 handle_command_reply: tid=0 2024-04-26T18:43:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed4dda5740 1 client.17199 shutdown 2024-04-26T18:43:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed4dda5740 1 -- 172.21.15.84:0/3270105193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 msgr2=0x556c839cdcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed4dda5740 1 --2- 172.21.15.84:0/3270105193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 0x556c839cdcd0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fed3c00a9b0 tx=0x7fed3c007ec0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed4dda5740 1 -- 172.21.15.84:0/3270105193 shutdown_connections 2024-04-26T18:43:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed4dda5740 1 --2- 172.21.15.84:0/3270105193 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556c83a40df0 0x556c83a43210 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed4dda5740 1 --2- 172.21.15.84:0/3270105193 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556c839ca0e0 0x556c839cdcd0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed4dda5740 1 --2- 172.21.15.84:0/3270105193 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556c839bf820 0x556c839cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed4dda5740 1 --2- 172.21.15.84:0/3270105193 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556c839bef00 0x556c839d2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed4dda5740 1 -- 172.21.15.84:0/3270105193 >> 172.21.15.84:0/3270105193 conn(0x556c839bb410 msgr2=0x556c839c7660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed23fff640 0 client.17199 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:06.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed4dda5740 1 -- 172.21.15.84:0/3270105193 shutdown_connections 2024-04-26T18:43:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed4dda5740 1 -- 172.21.15.84:0/3270105193 wait complete. 2024-04-26T18:43:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.892+0000 7fed4dda5740 20 client.17199 trim_cache size 0 max 16384 2024-04-26T18:43:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:43:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:43:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:43:06.894 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:43:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.893+0000 7fed4b083640 1 -- 172.21.15.84:0/16323272 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fed340a3dc0 msgr2=0x7fed340a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.893+0000 7fed4b083640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fed340a3dc0 0x7fed340a6280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fed38009900 tx=0x7fed38005fd0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.893+0000 7fed4b083640 1 -- 172.21.15.84:0/16323272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed44102220 msgr2=0x7fed4419a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.893+0000 7fed4b083640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed44102220 0x7fed4419a540 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fed3000d770 tx=0x7fed3000dc30 comp rx=0 tx=0).stop 2024-04-26T18:43:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.893+0000 7fed4b083640 1 -- 172.21.15.84:0/16323272 shutdown_connections 2024-04-26T18:43:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.893+0000 7fed4b083640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fed340a3dc0 0x7fed340a6280 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.895 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.893+0000 7fed4b083640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fed44107820 0x7fed441a5080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.893+0000 7fed4b083640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fed44102220 0x7fed4419a540 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.893+0000 7fed4b083640 1 --2- 172.21.15.84:0/16323272 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fed44101860 0x7fed4419a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.893+0000 7fed4b083640 1 -- 172.21.15.84:0/16323272 >> 172.21.15.84:0/16323272 conn(0x7fed440789b0 msgr2=0x7fed44104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.893+0000 7fed4b083640 1 -- 172.21.15.84:0/16323272 shutdown_connections 2024-04-26T18:43:06.896 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:06.893+0000 7fed4b083640 1 -- 172.21.15.84:0/16323272 wait complete. 2024-04-26T18:43:07.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:07 smithi084 ceph-mon[29141]: pgmap v476: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-26T18:43:07.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:07 smithi103 ceph-mon[32727]: pgmap v476: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-26T18:43:07.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:07 smithi174 ceph-mon[32705]: pgmap v476: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-26T18:43:09.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:09 smithi084 ceph-mon[29141]: pgmap v477: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-26T18:43:09.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:09 smithi103 ceph-mon[32727]: pgmap v477: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-26T18:43:09.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:09 smithi174 ceph-mon[32705]: pgmap v477: 97 pgs: 97 active+clean; 4.7 MiB data, 447 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 104 KiB/s wr, 38 op/s 2024-04-26T18:43:11.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:11 smithi084 ceph-mon[29141]: pgmap v478: 97 pgs: 97 active+clean; 4.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-26T18:43:11.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:11 smithi103 ceph-mon[32727]: pgmap v478: 97 pgs: 97 active+clean; 4.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-26T18:43:11.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:11 smithi174 ceph-mon[32705]: pgmap v478: 97 pgs: 97 active+clean; 4.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-26T18:43:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:13 smithi084 ceph-mon[29141]: pgmap v479: 97 pgs: 97 active+clean; 4.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-26T18:43:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:13 smithi084 ceph-mon[29141]: osdmap e114: 12 total, 12 up, 12 in 2024-04-26T18:43:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:13 smithi103 ceph-mon[32727]: pgmap v479: 97 pgs: 97 active+clean; 4.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-26T18:43:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:13 smithi103 ceph-mon[32727]: osdmap e114: 12 total, 12 up, 12 in 2024-04-26T18:43:13.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:13 smithi174 ceph-mon[32705]: pgmap v479: 97 pgs: 97 active+clean; 4.9 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 108 KiB/s wr, 39 op/s 2024-04-26T18:43:13.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:13 smithi174 ceph-mon[32705]: osdmap e114: 12 total, 12 up, 12 in 2024-04-26T18:43:15.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:15 smithi084 ceph-mon[29141]: pgmap v481: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 20 op/s 2024-04-26T18:43:15.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:15 smithi103 ceph-mon[32727]: pgmap v481: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 20 op/s 2024-04-26T18:43:15.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:15 smithi174 ceph-mon[32705]: pgmap v481: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 20 op/s 2024-04-26T18:43:16.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:16 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:43:16.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:16 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:43:16.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:16 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:43:16.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:16 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:43:16.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:16 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:43:16.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:16 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:43:16.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:16 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:43:16.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:16 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:43:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:16 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:43:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:16 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:43:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:16 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:43:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:16 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:43:17.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:17 smithi084 ceph-mon[29141]: pgmap v482: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-26T18:43:17.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:17 smithi103 ceph-mon[32727]: pgmap v482: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-26T18:43:17.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:17 smithi174 ceph-mon[32705]: pgmap v482: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-26T18:43:19.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:19 smithi084 ceph-mon[29141]: pgmap v483: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-26T18:43:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:19 smithi103 ceph-mon[32727]: pgmap v483: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-26T18:43:19.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:19 smithi174 ceph-mon[32705]: pgmap v483: 97 pgs: 97 active+clean; 5.1 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 53 KiB/s wr, 19 op/s 2024-04-26T18:43:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.988+0000 7f72ee1c6640 1 -- 172.21.15.84:0/1611936691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72e81032f0 msgr2=0x7f72e810f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.988+0000 7f72ee1c6640 1 --2- 172.21.15.84:0/1611936691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72e81032f0 0x7f72e810f800 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009cd0 tx=0x7f72e002a8e0 comp rx=0 tx=0).stop 2024-04-26T18:43:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.988+0000 7f72ee1c6640 1 -- 172.21.15.84:0/1611936691 shutdown_connections 2024-04-26T18:43:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.988+0000 7f72ee1c6640 1 --2- 172.21.15.84:0/1611936691 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72e81032f0 0x7f72e810f800 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.988+0000 7f72ee1c6640 1 --2- 172.21.15.84:0/1611936691 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72e8102930 0x7f72e8102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.988+0000 7f72ee1c6640 1 --2- 172.21.15.84:0/1611936691 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72e8108940 0x7f72e8108d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.988+0000 7f72ee1c6640 1 -- 172.21.15.84:0/1611936691 >> 172.21.15.84:0/1611936691 conn(0x7f72e80fe400 msgr2=0x7f72e8100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.988+0000 7f72ee1c6640 1 -- 172.21.15.84:0/1611936691 shutdown_connections 2024-04-26T18:43:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.988+0000 7f72ee1c6640 1 -- 172.21.15.84:0/1611936691 wait complete. 2024-04-26T18:43:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.989+0000 7f72ee1c6640 1 Processor -- start 2024-04-26T18:43:19.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.989+0000 7f72ee1c6640 1 -- start start 2024-04-26T18:43:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.989+0000 7f72ee1c6640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72e8102930 0x7f72e81990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.989+0000 7f72ee1c6640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72e81032f0 0x7f72e81995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.989+0000 7f72ee1c6640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72e8108940 0x7f72e819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.989+0000 7f72ee1c6640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8112c00 con 0x7f72e8102930 2024-04-26T18:43:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.989+0000 7f72ee1c6640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8112a80 con 0x7f72e81032f0 2024-04-26T18:43:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.989+0000 7f72ee1c6640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72e8112d80 con 0x7f72e8108940 2024-04-26T18:43:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.989+0000 7f72e77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72e8102930 0x7f72e81990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.989+0000 7f72e77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72e8102930 0x7f72e81990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38630/0 (socket says 172.21.15.84:38630) 2024-04-26T18:43:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.989+0000 7f72e77fe640 1 -- 172.21.15.84:0/2587712429 learned_addr learned my addr 172.21.15.84:0/2587712429 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.989+0000 7f72e7fff640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72e8108940 0x7f72e819d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.989+0000 7f72e6ffd640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72e81032f0 0x7f72e81995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:19.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.990+0000 7f72e77fe640 1 -- 172.21.15.84:0/2587712429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72e8108940 msgr2=0x7f72e819d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.990+0000 7f72e77fe640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72e8108940 0x7f72e819d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.990+0000 7f72e77fe640 1 -- 172.21.15.84:0/2587712429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72e81032f0 msgr2=0x7f72e81995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.990+0000 7f72e77fe640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72e81032f0 0x7f72e81995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.990+0000 7f72e77fe640 1 -- 172.21.15.84:0/2587712429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72e819e040 con 0x7f72e8102930 2024-04-26T18:43:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.990+0000 7f72e6ffd640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72e81032f0 0x7f72e81995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:43:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.990+0000 7f72e77fe640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72e8102930 0x7f72e81990a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f72d800d770 tx=0x7f72d800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.990+0000 7f72e4ff9640 1 -- 172.21.15.84:0/2587712429 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8017070 con 0x7f72e8102930 2024-04-26T18:43:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.990+0000 7f72ee1c6640 1 -- 172.21.15.84:0/2587712429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72e819e330 con 0x7f72e8102930 2024-04-26T18:43:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.990+0000 7f72e4ff9640 1 -- 172.21.15.84:0/2587712429 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d80054d0 con 0x7f72e8102930 2024-04-26T18:43:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.990+0000 7f72e4ff9640 1 -- 172.21.15.84:0/2587712429 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8004c30 con 0x7f72e8102930 2024-04-26T18:43:19.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.990+0000 7f72ee1c6640 1 -- 172.21.15.84:0/2587712429 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72e81a5bc0 con 0x7f72e8102930 2024-04-26T18:43:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.992+0000 7f72e4ff9640 1 -- 172.21.15.84:0/2587712429 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f72d8004e10 con 0x7f72e8102930 2024-04-26T18:43:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72f0fc1740 1 Processor -- start 2024-04-26T18:43:20.001 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72f0fc1740 1 -- start start 2024-04-26T18:43:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e4ff9640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f72d00a3dc0 0x7f72d00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72f0fc1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcb9f00 0x55abddcba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72f0fc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcba820 0x55abddcc4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72f0fc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcc50e0 0x55abddcc74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72f0fc1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55abddc20460 con 0x55abddcb9f00 2024-04-26T18:43:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e7fff640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f72d00a3dc0 0x7f72d00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72f0fc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55abddbda590 con 0x55abddcc50e0 2024-04-26T18:43:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72f0fc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abddc1c420 con 0x55abddcba820 2024-04-26T18:43:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e4ff9640 1 -- 172.21.15.84:0/2587712429 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7f72d80d02a0 con 0x7f72e8102930 2024-04-26T18:43:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcb9f00 0x55abddcba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e77fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcb9f00 0x55abddcba2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38642/0 (socket says 172.21.15.84:38642) 2024-04-26T18:43:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e77fe640 1 -- 172.21.15.84:0/2397245016 learned_addr learned my addr 172.21.15.84:0/2397245016 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:20.002 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e6ffd640 1 --2- 172.21.15.84:0/2397245016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcba820 0x55abddcc4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e7fff640 1 --2- 172.21.15.84:0/2397245016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcc50e0 0x55abddcc74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e77fe640 1 -- 172.21.15.84:0/2397245016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcba820 msgr2=0x55abddcc4ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e77fe640 1 --2- 172.21.15.84:0/2397245016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcba820 0x55abddcc4ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e77fe640 1 -- 172.21.15.84:0/2397245016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcc50e0 msgr2=0x55abddcc74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e77fe640 1 --2- 172.21.15.84:0/2397245016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcc50e0 0x55abddcc74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e77fe640 1 -- 172.21.15.84:0/2397245016 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abddbe70d0 con 0x55abddcb9f00 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.993+0000 7f72e7fff640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f72d00a3dc0 0x7f72d00a6280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f72e0005e10 tx=0x7f72e0009960 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.994+0000 7f72e77fe640 1 --2- 172.21.15.84:0/2397245016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcb9f00 0x55abddcba2e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f72d8098660 tx=0x7f72d809c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381bc983f0278902 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.994+0000 7f72cdffb640 1 -- 172.21.15.84:0/2397245016 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d8020040 con 0x55abddcb9f00 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.994+0000 7f72cdffb640 1 -- 172.21.15.84:0/2397245016 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d80228d0 con 0x55abddcb9f00 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.994+0000 7f72cdffb640 1 -- 172.21.15.84:0/2397245016 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72d80a05f0 con 0x55abddcb9f00 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.994+0000 7f72f0fc1740 1 -- 172.21.15.84:0/2397245016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcb9f00 msgr2=0x55abddcba2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.994+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/2397245016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcb9f00 0x55abddcba2e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f72d8098660 tx=0x7f72d809c100 comp rx=0 tx=0).stop 2024-04-26T18:43:20.003 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.994+0000 7f72f0fc1740 1 -- 172.21.15.84:0/2397245016 shutdown_connections 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.994+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/2397245016 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcc50e0 0x55abddcc74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.994+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/2397245016 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcba820 0x55abddcc4ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.994+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/2397245016 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcb9f00 0x55abddcba2e0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.994+0000 7f72f0fc1740 1 -- 172.21.15.84:0/2397245016 >> 172.21.15.84:0/2397245016 conn(0x55abddcb6410 msgr2=0x55abddcb67f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.995+0000 7f72f0fc1740 1 -- 172.21.15.84:0/2397245016 shutdown_connections 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.995+0000 7f72f0fc1740 1 -- 172.21.15.84:0/2397245016 wait complete. 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.995+0000 7f72f0fc1740 1 -- start start 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.995+0000 7f72f0fc1740 1 Processor -- start 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.995+0000 7f72f0fc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcb9f00 0x55abddcc20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.995+0000 7f72f0fc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcba820 0x55abddcc2600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.995+0000 7f72f0fc1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 0x55abddcc2b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.995+0000 7f72f0fc1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55abddcc7c50 con 0x55abddcc50e0 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.995+0000 7f72f0fc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55abddba7b60 con 0x55abddcba820 2024-04-26T18:43:20.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.995+0000 7f72f0fc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55abddcc7dd0 con 0x55abddcb9f00 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.995+0000 7f72f0fc1740 10 client.0 _ll_register_callbacks cb 0x55abddbcc2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72e7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 0x55abddcc2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72e7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 0x55abddcc2b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38658/0 (socket says 172.21.15.84:38658) 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72e7fff640 1 -- 172.21.15.84:0/775322675 learned_addr learned my addr 172.21.15.84:0/775322675 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72e77fe640 1 --2- 172.21.15.84:0/775322675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcb9f00 0x55abddcc20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72e6ffd640 1 --2- 172.21.15.84:0/775322675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcba820 0x55abddcc2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72e7fff640 1 -- 172.21.15.84:0/775322675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcb9f00 msgr2=0x55abddcc20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72e7fff640 1 --2- 172.21.15.84:0/775322675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcb9f00 0x55abddcc20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72e7fff640 1 -- 172.21.15.84:0/775322675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcba820 msgr2=0x55abddcc2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72e7fff640 1 --2- 172.21.15.84:0/775322675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcba820 0x55abddcc2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72e7fff640 1 -- 172.21.15.84:0/775322675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abdd932210 con 0x55abddcc50e0 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72e7fff640 1 --2- 172.21.15.84:0/775322675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 0x55abddcc2b40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009cd0 tx=0x7f72e002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2d1d877b29ade1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:20.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72c6ffd640 1 -- 172.21.15.84:0/775322675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0042040 con 0x55abddcc50e0 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72c6ffd640 1 -- 172.21.15.84:0/775322675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7f72e0004510 con 0x55abddcc50e0 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72f0fc1740 1 -- 172.21.15.84:0/775322675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72d80228d0 con 0x55abddcc50e0 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72c6ffd640 1 -- 172.21.15.84:0/775322675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72e0004710 con 0x55abddcc50e0 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72f0fc1740 10 client.17211 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.996+0000 7f72f0fc1740 1 -- 172.21.15.84:0/775322675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55abdd9fe620 con 0x55abddcc50e0 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.997+0000 7f72c6ffd640 1 -- 172.21.15.84:0/775322675 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72d80228d0 con 0x55abddcc50e0 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.997+0000 7f72c6ffd640 1 -- 172.21.15.84:0/775322675 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f72e0057030 con 0x55abddcc50e0 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.997+0000 7f72f0fc1740 10 client.17211 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.997+0000 7f72f0fc1740 10 client.17211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.997+0000 7f72f0fc1740 20 client.17211 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.997+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/775322675 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55abddcd52c0 0x55abddcd76b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.997+0000 7f72f0fc1740 4 client.17211 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.997+0000 7f72f0fc1740 1 -- 172.21.15.84:0/775322675 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abddc2e070 con 0x55abddcd52c0 2024-04-26T18:43:20.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.997+0000 7f72e77fe640 1 --2- 172.21.15.84:0/775322675 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55abddcd52c0 0x55abddcd76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.997+0000 7f72e77fe640 1 --2- 172.21.15.84:0/775322675 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55abddcd52c0 0x55abddcd76b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90b7c72ce00df9ff server_cookie=69e200df41c81250 in_seq=0 out_seq=0 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.997+0000 7f72c6ffd640 10 client.17211 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72c6ffd640 1 -- 172.21.15.84:0/775322675 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55abddc2e070 con 0x55abddcd52c0 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72c6ffd640 10 client.17211 handle_command_reply: tid=0 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72f0fc1740 1 client.17211 shutdown 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72f0fc1740 1 -- 172.21.15.84:0/775322675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 msgr2=0x55abddcc2b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/775322675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 0x55abddcc2b40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009cd0 tx=0x7f72e002a8e0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72f0fc1740 1 -- 172.21.15.84:0/775322675 shutdown_connections 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/775322675 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55abddcd52c0 0x55abddcd76b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/775322675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 0x55abddcc2b40 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/775322675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcba820 0x55abddcc2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72c6ffd640 0 client.17211 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/775322675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcb9f00 0x55abddcc20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72f0fc1740 1 -- 172.21.15.84:0/775322675 >> 172.21.15.84:0/775322675 conn(0x55abddcb6410 msgr2=0x55abddcbc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72f0fc1740 1 -- 172.21.15.84:0/775322675 shutdown_connections 2024-04-26T18:43:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.998+0000 7f72f0fc1740 1 -- 172.21.15.84:0/775322675 wait complete. 2024-04-26T18:43:20.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:19.999+0000 7f72f0fc1740 20 client.17211 trim_cache size 0 max 16384 2024-04-26T18:43:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.013+0000 7f72f0fc1740 1 Processor -- start 2024-04-26T18:43:20.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.013+0000 7f72f0fc1740 1 -- start start 2024-04-26T18:43:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.013+0000 7f72f0fc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcb9f00 0x55abddcd0b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.013+0000 7f72f0fc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcba820 0x55abddcdcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.013+0000 7f72f0fc1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 0x55abddcdd500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72f0fc1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81ae570 con 0x55abddcc50e0 2024-04-26T18:43:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72f0fc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81b00b0 con 0x55abddcb9f00 2024-04-26T18:43:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72f0fc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81aec40 con 0x55abddcba820 2024-04-26T18:43:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72e7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 0x55abddcdd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72e7fff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 0x55abddcdd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38674/0 (socket says 172.21.15.84:38674) 2024-04-26T18:43:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72e7fff640 1 -- 172.21.15.84:0/3968640265 learned_addr learned my addr 172.21.15.84:0/3968640265 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72e77fe640 1 --2- 172.21.15.84:0/3968640265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcb9f00 0x55abddcd0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72e7fff640 1 -- 172.21.15.84:0/3968640265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcba820 msgr2=0x55abddcdcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72e7fff640 1 --2- 172.21.15.84:0/3968640265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcba820 0x55abddcdcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72e7fff640 1 -- 172.21.15.84:0/3968640265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcb9f00 msgr2=0x55abddcd0b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72e7fff640 1 --2- 172.21.15.84:0/3968640265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcb9f00 0x55abddcd0b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72e7fff640 1 -- 172.21.15.84:0/3968640265 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abddcd7a20 con 0x55abddcc50e0 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72e7fff640 1 --2- 172.21.15.84:0/3968640265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 0x55abddcdd500 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009930 tx=0x7f72e0004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cae0d22cb08ce6f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72cd7fa640 1 -- 172.21.15.84:0/3968640265 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0042050 con 0x55abddcc50e0 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72cd7fa640 1 -- 172.21.15.84:0/3968640265 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e0044050 con 0x55abddcc50e0 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.014+0000 7f72cd7fa640 1 -- 172.21.15.84:0/3968640265 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72e0030800 con 0x55abddcc50e0 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.015+0000 7f72f0fc1740 1 -- 172.21.15.84:0/3968640265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 msgr2=0x55abddcdd500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.015+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/3968640265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 0x55abddcdd500 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009930 tx=0x7f72e0004970 comp rx=0 tx=0).stop 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.015+0000 7f72f0fc1740 1 -- 172.21.15.84:0/3968640265 shutdown_connections 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.015+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/3968640265 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcc50e0 0x55abddcdd500 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.015+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/3968640265 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcba820 0x55abddcdcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.015+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/3968640265 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcb9f00 0x55abddcd0b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.015+0000 7f72f0fc1740 1 -- 172.21.15.84:0/3968640265 >> 172.21.15.84:0/3968640265 conn(0x55abddcb6410 msgr2=0x55abddcb55b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:20.020 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.015+0000 7f72f0fc1740 1 -- 172.21.15.84:0/3968640265 shutdown_connections 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.015+0000 7f72f0fc1740 1 -- 172.21.15.84:0/3968640265 wait complete. 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.015+0000 7f72f0fc1740 1 -- start start 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.015+0000 7f72f0fc1740 1 Processor -- start 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72f0fc1740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcb9f00 0x55abddcbf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72f0fc1740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcba820 0x55abddcbf9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72f0fc1740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcc50e0 0x55abddd3a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72f0fc1740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81af9e0 con 0x55abddcba820 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72f0fc1740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81af310 con 0x55abddcb9f00 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72f0fc1740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f72e81ac570 con 0x55abddcc50e0 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72f0fc1740 10 client.0 _ll_register_callbacks cb 0x7f72e0009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72e6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcba820 0x55abddcbf9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72e6ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcba820 0x55abddcbf9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:38688/0 (socket says 172.21.15.84:38688) 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72e6ffd640 1 -- 172.21.15.84:0/2338793866 learned_addr learned my addr 172.21.15.84:0/2338793866 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:20.021 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72e77fe640 1 --2- 172.21.15.84:0/2338793866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcb9f00 0x55abddcbf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72e7fff640 1 --2- 172.21.15.84:0/2338793866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcc50e0 0x55abddd3a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72e6ffd640 1 -- 172.21.15.84:0/2338793866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcc50e0 msgr2=0x55abddd3a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72e6ffd640 1 --2- 172.21.15.84:0/2338793866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcc50e0 0x55abddd3a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72e6ffd640 1 -- 172.21.15.84:0/2338793866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcb9f00 msgr2=0x55abddcbf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72e6ffd640 1 --2- 172.21.15.84:0/2338793866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcb9f00 0x55abddcbf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72e6ffd640 1 -- 172.21.15.84:0/2338793866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abddd3ad70 con 0x55abddcba820 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72e6ffd640 1 --2- 172.21.15.84:0/2338793866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcba820 0x55abddcbf9a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f72e81009f0 tx=0x7f72dc0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b42c9ef0eaa72f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72c6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72c6ffd640 1 -- 172.21.15.84:0/2338793866 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72dc004020 con 0x55abddcba820 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72c6ffd640 1 -- 172.21.15.84:0/2338793866 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7f72dc0041c0 con 0x55abddcba820 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.016+0000 7f72f0fc1740 1 -- 172.21.15.84:0/2338793866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f72e0044050 con 0x55abddcba820 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.017+0000 7f72c6ffd640 1 -- 172.21.15.84:0/2338793866 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f72dc0043c0 con 0x55abddcba820 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.017+0000 7f72f0fc1740 10 client.17217 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:20.022 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.017+0000 7f72f0fc1740 1 -- 172.21.15.84:0/2338793866 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55abddd3c070 con 0x55abddcba820 2024-04-26T18:43:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.017+0000 7f72c6ffd640 1 -- 172.21.15.84:0/2338793866 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f72e0044050 con 0x55abddcba820 2024-04-26T18:43:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.017+0000 7f72c6ffd640 1 -- 172.21.15.84:0/2338793866 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f72dc00f040 con 0x55abddcba820 2024-04-26T18:43:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.017+0000 7f72f0fc1740 10 client.17217 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.017+0000 7f72f0fc1740 10 client.17217 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:43:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.017+0000 7f72f0fc1740 20 client.17217 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.017+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/2338793866 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55abddd3c850 0x55abddd3ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.017+0000 7f72f0fc1740 4 client.17217 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:43:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.017+0000 7f72f0fc1740 1 -- 172.21.15.84:0/2338793866 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55abddc27ee0 con 0x55abddd3c850 2024-04-26T18:43:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.017+0000 7f72e77fe640 1 --2- 172.21.15.84:0/2338793866 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55abddd3c850 0x55abddd3ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.018+0000 7f72e77fe640 1 --2- 172.21.15.84:0/2338793866 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55abddd3c850 0x55abddd3ec70 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67f944879c20d890 server_cookie=1a5e31e6d2c3fe79 in_seq=0 out_seq=0 2024-04-26T18:43:20.023 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.018+0000 7f72c6ffd640 10 client.17217 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:20.033 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72c6ffd640 1 -- 172.21.15.84:0/2338793866 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55abddc27ee0 con 0x55abddd3c850 2024-04-26T18:43:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72c6ffd640 10 client.17217 handle_command_reply: tid=0 2024-04-26T18:43:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72f0fc1740 1 client.17217 shutdown 2024-04-26T18:43:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72f0fc1740 1 -- 172.21.15.84:0/2338793866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcba820 msgr2=0x55abddcbf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/2338793866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcba820 0x55abddcbf9a0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f72e81009f0 tx=0x7f72dc0128e0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72f0fc1740 1 -- 172.21.15.84:0/2338793866 shutdown_connections 2024-04-26T18:43:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/2338793866 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55abddd3c850 0x55abddd3ec70 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/2338793866 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55abddcc50e0 0x55abddd3a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/2338793866 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55abddcba820 0x55abddcbf9a0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72f0fc1740 1 --2- 172.21.15.84:0/2338793866 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55abddcb9f00 0x55abddcbf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72c6ffd640 0 client.17217 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72f0fc1740 1 -- 172.21.15.84:0/2338793866 >> 172.21.15.84:0/2338793866 conn(0x55abddcb6410 msgr2=0x55abddcb4d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72f0fc1740 1 -- 172.21.15.84:0/2338793866 shutdown_connections 2024-04-26T18:43:20.034 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72f0fc1740 1 -- 172.21.15.84:0/2338793866 wait complete. 2024-04-26T18:43:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.031+0000 7f72f0fc1740 20 client.17217 trim_cache size 0 max 16384 2024-04-26T18:43:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:43:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:43:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:43:20.035 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:43:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.032+0000 7f72ee1c6640 1 -- 172.21.15.84:0/2587712429 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f72d00a3dc0 msgr2=0x7f72d00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.032+0000 7f72ee1c6640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f72d00a3dc0 0x7f72d00a6280 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f72e0005e10 tx=0x7f72e0009960 comp rx=0 tx=0).stop 2024-04-26T18:43:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.032+0000 7f72ee1c6640 1 -- 172.21.15.84:0/2587712429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72e8102930 msgr2=0x7f72e81990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.032+0000 7f72ee1c6640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72e8102930 0x7f72e81990a0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f72d800d770 tx=0x7f72d800dc30 comp rx=0 tx=0).stop 2024-04-26T18:43:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.032+0000 7f72ee1c6640 1 -- 172.21.15.84:0/2587712429 shutdown_connections 2024-04-26T18:43:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.032+0000 7f72ee1c6640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f72d00a3dc0 0x7f72d00a6280 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.032+0000 7f72ee1c6640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f72e8108940 0x7f72e819d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.032+0000 7f72ee1c6640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f72e81032f0 0x7f72e81995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.032+0000 7f72ee1c6640 1 --2- 172.21.15.84:0/2587712429 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f72e8102930 0x7f72e81990a0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.032+0000 7f72ee1c6640 1 -- 172.21.15.84:0/2587712429 >> 172.21.15.84:0/2587712429 conn(0x7f72e80fe400 msgr2=0x7f72e80fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:20.036 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.032+0000 7f72ee1c6640 1 -- 172.21.15.84:0/2587712429 shutdown_connections 2024-04-26T18:43:20.037 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:20.032+0000 7f72ee1c6640 1 -- 172.21.15.84:0/2587712429 wait complete. 2024-04-26T18:43:21.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:21 smithi084 ceph-mon[29141]: pgmap v484: 97 pgs: 97 active+clean; 5.3 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:43:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:21 smithi103 ceph-mon[32727]: pgmap v484: 97 pgs: 97 active+clean; 5.3 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:43:21.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:21 smithi174 ceph-mon[32705]: pgmap v484: 97 pgs: 97 active+clean; 5.3 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:43:23.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:23 smithi084 ceph-mon[29141]: pgmap v485: 97 pgs: 97 active+clean; 5.3 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:43:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:23 smithi103 ceph-mon[32727]: pgmap v485: 97 pgs: 97 active+clean; 5.3 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:43:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:23 smithi174 ceph-mon[32705]: pgmap v485: 97 pgs: 97 active+clean; 5.3 MiB data, 448 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-26T18:43:25.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:25 smithi084 ceph-mon[29141]: pgmap v486: 97 pgs: 97 active+clean; 1.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 24 op/s 2024-04-26T18:43:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:25 smithi103 ceph-mon[32727]: pgmap v486: 97 pgs: 97 active+clean; 1.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 24 op/s 2024-04-26T18:43:25.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:25 smithi174 ceph-mon[32705]: pgmap v486: 97 pgs: 97 active+clean; 1.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 61 KiB/s wr, 24 op/s 2024-04-26T18:43:27.674 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 10 2024-04-26T18:43:27.675 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:43:27.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:27 smithi084 ceph-mon[29141]: pgmap v487: 97 pgs: 97 active+clean; 1.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-26T18:43:27.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:27 smithi103 ceph-mon[32727]: pgmap v487: 97 pgs: 97 active+clean; 1.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-26T18:43:27.929 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:43:27.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:27 smithi174 ceph-mon[32705]: pgmap v487: 97 pgs: 97 active+clean; 1.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-26T18:43:29.779 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:29 smithi084 ceph-mon[29141]: pgmap v488: 97 pgs: 97 active+clean; 1.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-26T18:43:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:29 smithi103 ceph-mon[32727]: pgmap v488: 97 pgs: 97 active+clean; 1.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-26T18:43:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.878+0000 7fb752d9d700 1 -- 172.21.15.84:0/3219865373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74c101e50 msgr2=0x7fb74c1022d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.878+0000 7fb752d9d700 1 --2- 172.21.15.84:0/3219865373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74c101e50 0x7fb74c1022d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fb738009990 tx=0x7fb738009ca0 comp rx=0 tx=0).stop 2024-04-26T18:43:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.878+0000 7fb752d9d700 1 -- 172.21.15.84:0/3219865373 shutdown_connections 2024-04-26T18:43:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.878+0000 7fb752d9d700 1 --2- 172.21.15.84:0/3219865373 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb74c102810 0x7fb74c110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.878+0000 7fb752d9d700 1 --2- 172.21.15.84:0/3219865373 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb74c109600 0x7fb74c1099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:29.904 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.878+0000 7fb752d9d700 1 --2- 172.21.15.84:0/3219865373 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74c101e50 0x7fb74c1022d0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.878+0000 7fb752d9d700 1 -- 172.21.15.84:0/3219865373 >> 172.21.15.84:0/3219865373 conn(0x7fb74c077590 msgr2=0x7fb74c0779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.878+0000 7fb752d9d700 1 -- 172.21.15.84:0/3219865373 shutdown_connections 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.878+0000 7fb752d9d700 1 -- 172.21.15.84:0/3219865373 wait complete. 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.879+0000 7fb752d9d700 1 Processor -- start 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.879+0000 7fb752d9d700 1 -- start start 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.879+0000 7fb752d9d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74c101e50 0x7fb74c19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.879+0000 7fb752d9d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb74c102810 0x7fb74c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.879+0000 7fb752d9d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb74c109600 0x7fb74c1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.879+0000 7fb752d9d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c1135b0 con 0x7fb74c101e50 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.879+0000 7fb752d9d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c113430 con 0x7fb74c102810 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.879+0000 7fb752d9d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fb74c113730 con 0x7fb74c109600 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.879+0000 7fb750b39700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74c101e50 0x7fb74c19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.879+0000 7fb750b39700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74c101e50 0x7fb74c19f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:40998/0 (socket says 172.21.15.84:40998) 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.879+0000 7fb750b39700 1 -- 172.21.15.84:0/169834176 learned_addr learned my addr 172.21.15.84:0/169834176 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:29.905 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.879+0000 7fb75133a700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb74c109600 0x7fb74c1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb74bfff700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb74c102810 0x7fb74c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb750b39700 1 -- 172.21.15.84:0/169834176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb74c109600 msgr2=0x7fb74c1998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb750b39700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb74c109600 0x7fb74c1998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb750b39700 1 -- 172.21.15.84:0/169834176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb74c102810 msgr2=0x7fb74c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb750b39700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb74c102810 0x7fb74c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb750b39700 1 -- 172.21.15.84:0/169834176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb74c19a030 con 0x7fb74c101e50 2024-04-26T18:43:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb75133a700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb74c109600 0x7fb74c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb74bfff700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb74c102810 0x7fb74c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:43:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb750b39700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74c101e50 0x7fb74c19f710 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb74000d980 tx=0x7fb74000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb749ffb700 1 -- 172.21.15.84:0/169834176 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740016070 con 0x7fb74c101e50 2024-04-26T18:43:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb752d9d700 1 -- 172.21.15.84:0/169834176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb74c19a320 con 0x7fb74c101e50 2024-04-26T18:43:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb749ffb700 1 -- 172.21.15.84:0/169834176 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7400044d0 con 0x7fb74c101e50 2024-04-26T18:43:29.906 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb749ffb700 1 -- 172.21.15.84:0/169834176 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb740004d00 con 0x7fb74c101e50 2024-04-26T18:43:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.880+0000 7fb752d9d700 1 -- 172.21.15.84:0/169834176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb74c1a65e0 con 0x7fb74c101e50 2024-04-26T18:43:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.881+0000 7fb752d9d700 1 -- 172.21.15.84:0/169834176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb74c06a110 con 0x7fb74c101e50 2024-04-26T18:43:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.882+0000 7fb749ffb700 1 -- 172.21.15.84:0/169834176 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fb740004020 con 0x7fb74c101e50 2024-04-26T18:43:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.883+0000 7fb749ffb700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fb73c0a3db0 0x7fb73c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.883+0000 7fb74bfff700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fb73c0a3db0 0x7fb73c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.883+0000 7fb749ffb700 1 -- 172.21.15.84:0/169834176 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7fb7400d0680 con 0x7fb74c101e50 2024-04-26T18:43:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.883+0000 7fb74bfff700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fb73c0a3db0 0x7fb73c0a6270 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb73800b5c0 tx=0x7fb738005fb0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:29.907 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:29.887+0000 7fb749ffb700 1 -- 172.21.15.84:0/169834176 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb74001f020 con 0x7fb74c101e50 2024-04-26T18:43:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:29 smithi174 ceph-mon[32705]: pgmap v488: 97 pgs: 97 active+clean; 1.5 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-26T18:43:30.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.133+0000 7fb752d9d700 1 -- 172.21.15.84:0/169834176 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb74c02e490 con 0x7fb74c101e50 2024-04-26T18:43:30.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.134+0000 7fb749ffb700 1 -- 172.21.15.84:0/169834176 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7fb74009cce0 con 0x7fb74c101e50 2024-04-26T18:43:30.135 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:43:30.136 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:43:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.136+0000 7fb752d9d700 1 -- 172.21.15.84:0/169834176 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fb73c0a3db0 msgr2=0x7fb73c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.136+0000 7fb752d9d700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fb73c0a3db0 0x7fb73c0a6270 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb73800b5c0 tx=0x7fb738005fb0 comp rx=0 tx=0).stop 2024-04-26T18:43:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.136+0000 7fb752d9d700 1 -- 172.21.15.84:0/169834176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74c101e50 msgr2=0x7fb74c19f710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.136+0000 7fb752d9d700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74c101e50 0x7fb74c19f710 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb74000d980 tx=0x7fb74000dd40 comp rx=0 tx=0).stop 2024-04-26T18:43:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.137+0000 7fb752d9d700 1 -- 172.21.15.84:0/169834176 shutdown_connections 2024-04-26T18:43:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.137+0000 7fb752d9d700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fb73c0a3db0 0x7fb73c0a6270 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.137+0000 7fb752d9d700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fb74c101e50 0x7fb74c19f710 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.137+0000 7fb752d9d700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fb74c109600 0x7fb74c1998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.137+0000 7fb752d9d700 1 --2- 172.21.15.84:0/169834176 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fb74c102810 0x7fb74c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.137+0000 7fb752d9d700 1 -- 172.21.15.84:0/169834176 >> 172.21.15.84:0/169834176 conn(0x7fb74c077590 msgr2=0x7fb74c100a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.137+0000 7fb752d9d700 1 -- 172.21.15.84:0/169834176 shutdown_connections 2024-04-26T18:43:30.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:30.137+0000 7fb752d9d700 1 -- 172.21.15.84:0/169834176 wait complete. 2024-04-26T18:43:30.139 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:43:30.809 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:30 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/169834176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:43:30.838 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 60 --expiration 70 --await-for 42.0' 2024-04-26T18:43:30.838 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db / --timeout 60 --expiration 70 --await-for 42.0 2024-04-26T18:43:30.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:30 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/169834176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:43:30.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:30 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/169834176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:43:31.085 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:43:31.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:31 smithi084 ceph-mon[29141]: pgmap v489: 97 pgs: 97 active+clean; 1.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-26T18:43:31.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:31 smithi103 ceph-mon[32727]: pgmap v489: 97 pgs: 97 active+clean; 1.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-26T18:43:31.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:31 smithi174 ceph-mon[32705]: pgmap v489: 97 pgs: 97 active+clean; 1.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-26T18:43:32.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.814+0000 7f465004f700 1 -- 172.21.15.84:0/2606145361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46481094a0 msgr2=0x7f4648109880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:32.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.814+0000 7f465004f700 1 --2- 172.21.15.84:0/2606145361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46481094a0 0x7f4648109880 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f46440099c0 tx=0x7f4644009cd0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.814+0000 7f465004f700 1 -- 172.21.15.84:0/2606145361 shutdown_connections 2024-04-26T18:43:32.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.814+0000 7f465004f700 1 --2- 172.21.15.84:0/2606145361 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f46481094a0 0x7f4648109880 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.814+0000 7f465004f700 1 --2- 172.21.15.84:0/2606145361 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4648103e90 0x7f46481102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.814+0000 7f465004f700 1 --2- 172.21.15.84:0/2606145361 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46481034d0 0x7f4648103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.814+0000 7f465004f700 1 -- 172.21.15.84:0/2606145361 >> 172.21.15.84:0/2606145361 conn(0x7f464807a4f0 msgr2=0x7f464807a900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:32.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.814+0000 7f465004f700 1 -- 172.21.15.84:0/2606145361 shutdown_connections 2024-04-26T18:43:32.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.814+0000 7f465004f700 1 -- 172.21.15.84:0/2606145361 wait complete. 2024-04-26T18:43:32.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.815+0000 7f465004f700 1 Processor -- start 2024-04-26T18:43:32.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.815+0000 7f465004f700 1 -- start start 2024-04-26T18:43:32.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.815+0000 7f465004f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46481034d0 0x7f464819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.815+0000 7f465004f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4648103e90 0x7f464819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.815+0000 7f465004f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46481094a0 0x7f4648199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.815+0000 7f465004f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f46481136d0 con 0x7f4648103e90 2024-04-26T18:43:32.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.815+0000 7f465004f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4648113550 con 0x7f46481034d0 2024-04-26T18:43:32.819 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.815+0000 7f465004f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4648113850 con 0x7f46481094a0 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.815+0000 7f464d5ea700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4648103e90 0x7f464819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f464d5ea700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4648103e90 0x7f464819fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41012/0 (socket says 172.21.15.84:41012) 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f464d5ea700 1 -- 172.21.15.84:0/1972850742 learned_addr learned my addr 172.21.15.84:0/1972850742 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f464ddeb700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46481034d0 0x7f464819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f464e5ec700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46481094a0 0x7f4648199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f464d5ea700 1 -- 172.21.15.84:0/1972850742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46481094a0 msgr2=0x7f4648199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f464d5ea700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46481094a0 0x7f4648199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f464d5ea700 1 -- 172.21.15.84:0/1972850742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46481034d0 msgr2=0x7f464819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f464d5ea700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46481034d0 0x7f464819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f464d5ea700 1 -- 172.21.15.84:0/1972850742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464819a070 con 0x7f4648103e90 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f464e5ec700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46481094a0 0x7f4648199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f464ddeb700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46481034d0 0x7f464819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f464d5ea700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4648103e90 0x7f464819fc90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f463800b530 tx=0x7f463800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f463effd700 1 -- 172.21.15.84:0/1972850742 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638015020 con 0x7f4648103e90 2024-04-26T18:43:32.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f463effd700 1 -- 172.21.15.84:0/1972850742 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4638011ce0 con 0x7f4648103e90 2024-04-26T18:43:32.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f465004f700 1 -- 172.21.15.84:0/1972850742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464819a360 con 0x7f4648103e90 2024-04-26T18:43:32.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f463effd700 1 -- 172.21.15.84:0/1972850742 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638010d20 con 0x7f4648103e90 2024-04-26T18:43:32.821 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.816+0000 7f465004f700 1 -- 172.21.15.84:0/1972850742 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4648075650 con 0x7f4648103e90 2024-04-26T18:43:32.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.818+0000 7f463effd700 1 -- 172.21.15.84:0/1972850742 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4638018020 con 0x7f4648103e90 2024-04-26T18:43:32.823 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.819+0000 7f463effd700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46340a3e50 0x7f46340a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.819+0000 7f464ddeb700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46340a3e50 0x7f46340a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.819+0000 7f463effd700 1 -- 172.21.15.84:0/1972850742 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7f46380cffe0 con 0x7f4648103e90 2024-04-26T18:43:32.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.820+0000 7f464ddeb700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46340a3e50 0x7f46340a6310 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4644000c00 tx=0x7f464401a040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:32.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.820+0000 7f465a8e1b80 1 Processor -- start 2024-04-26T18:43:32.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.820+0000 7f465a8e1b80 1 -- start start 2024-04-26T18:43:32.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.820+0000 7f465a8e1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fff30 0x557eb8200310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.820+0000 7f465a8e1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb8200850 0x557eb81fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.820+0000 7f465a8e1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb81fa900 0x557eb81fad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.820+0000 7f465a8e1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557eb814c9c0 con 0x557eb8200850 2024-04-26T18:43:32.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.820+0000 7f465a8e1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557eb80b2ac0 con 0x557eb81fff30 2024-04-26T18:43:32.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.820+0000 7f465a8e1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557eb8148980 con 0x557eb81fa900 2024-04-26T18:43:32.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f464d5ea700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb8200850 0x557eb81fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.824 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f464d5ea700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb8200850 0x557eb81fa2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41014/0 (socket says 172.21.15.84:41014) 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f464d5ea700 1 -- 172.21.15.84:0/2007357621 learned_addr learned my addr 172.21.15.84:0/2007357621 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f464ddeb700 1 --2- 172.21.15.84:0/2007357621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fff30 0x557eb8200310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f464e5ec700 1 --2- 172.21.15.84:0/2007357621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb81fa900 0x557eb81fad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f464d5ea700 1 -- 172.21.15.84:0/2007357621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb81fa900 msgr2=0x557eb81fad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f464d5ea700 1 --2- 172.21.15.84:0/2007357621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb81fa900 0x557eb81fad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f464d5ea700 1 -- 172.21.15.84:0/2007357621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fff30 msgr2=0x557eb8200310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f464d5ea700 1 --2- 172.21.15.84:0/2007357621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fff30 0x557eb8200310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f464d5ea700 1 -- 172.21.15.84:0/2007357621 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eb8099a60 con 0x557eb8200850 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f464d5ea700 1 --2- 172.21.15.84:0/2007357621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb8200850 0x557eb81fa2e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f46380a4730 tx=0x7f463809bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d9f6b89c4575a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f4633fff700 1 -- 172.21.15.84:0/2007357621 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463809bff0 con 0x557eb8200850 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f4633fff700 1 -- 172.21.15.84:0/2007357621 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46380a03a0 con 0x557eb8200850 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.821+0000 7f4633fff700 1 -- 172.21.15.84:0/2007357621 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46380a47d0 con 0x557eb8200850 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.822+0000 7f465a8e1b80 1 -- 172.21.15.84:0/2007357621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb8200850 msgr2=0x557eb81fa2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:32.825 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.822+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/2007357621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb8200850 0x557eb81fa2e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f46380a4730 tx=0x7f463809bc60 comp rx=0 tx=0).stop 2024-04-26T18:43:32.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.822+0000 7f465a8e1b80 1 -- 172.21.15.84:0/2007357621 shutdown_connections 2024-04-26T18:43:32.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.822+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/2007357621 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb81fa900 0x557eb81fad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.822+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/2007357621 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fff30 0x557eb8200310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.822+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/2007357621 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb8200850 0x557eb81fa2e0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.822+0000 7f465a8e1b80 1 -- 172.21.15.84:0/2007357621 >> 172.21.15.84:0/2007357621 conn(0x557eb8128750 msgr2=0x557eb81f5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:32.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.822+0000 7f465a8e1b80 1 -- 172.21.15.84:0/2007357621 shutdown_connections 2024-04-26T18:43:32.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.822+0000 7f465a8e1b80 1 -- 172.21.15.84:0/2007357621 wait complete. 2024-04-26T18:43:32.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f465a8e1b80 1 -- start start 2024-04-26T18:43:32.826 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f465a8e1b80 1 Processor -- start 2024-04-26T18:43:32.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f465a8e1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fa900 0x557eb8206550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f465a8e1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fff30 0x557eb8203940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f465a8e1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 0x557eb8203e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f465a8e1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557eb82072c0 con 0x557eb81fa900 2024-04-26T18:43:32.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f465a8e1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557eb8106c10 con 0x557eb81fff30 2024-04-26T18:43:32.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f465a8e1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557eb8207440 con 0x557eb8200850 2024-04-26T18:43:32.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f465a8e1b80 10 client.0 _ll_register_callbacks cb 0x557eb80e4ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:32.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f464ddeb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fa900 0x557eb8206550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.830 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f464ddeb700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fa900 0x557eb8206550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41024/0 (socket says 172.21.15.84:41024) 2024-04-26T18:43:32.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f464ddeb700 1 -- 172.21.15.84:0/854453439 learned_addr learned my addr 172.21.15.84:0/854453439 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:32.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f464d5ea700 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fff30 0x557eb8203940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.823+0000 7f464e5ec700 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 0x557eb8203e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f464ddeb700 1 -- 172.21.15.84:0/854453439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 msgr2=0x557eb8203e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:32.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f464ddeb700 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 0x557eb8203e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f464ddeb700 1 -- 172.21.15.84:0/854453439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fff30 msgr2=0x557eb8203940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:32.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f464ddeb700 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fff30 0x557eb8203940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f464ddeb700 1 -- 172.21.15.84:0/854453439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eb810a610 con 0x557eb81fa900 2024-04-26T18:43:32.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f464d5ea700 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fff30 0x557eb8203940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:43:32.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f464ddeb700 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fa900 0x557eb8206550 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f46440099c0 tx=0x7f4644009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195e1a457ca58443 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:32.831 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f46317fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:32.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f46317fa700 1 -- 172.21.15.84:0/854453439 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464402f850 con 0x557eb81fa900 2024-04-26T18:43:32.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f46317fa700 1 -- 172.21.15.84:0/854453439 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7f464402f9f0 con 0x557eb81fa900 2024-04-26T18:43:32.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f465a8e1b80 1 -- 172.21.15.84:0/854453439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46380a03a0 con 0x557eb81fa900 2024-04-26T18:43:32.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f46317fa700 1 -- 172.21.15.84:0/854453439 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f464403d050 con 0x557eb81fa900 2024-04-26T18:43:32.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f465a8e1b80 10 client.17235 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:32.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f465a8e1b80 1 -- 172.21.15.84:0/854453439 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557eb8211ae0 con 0x557eb81fa900 2024-04-26T18:43:32.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.824+0000 7f46317fa700 1 -- 172.21.15.84:0/854453439 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46380a03a0 con 0x557eb81fa900 2024-04-26T18:43:32.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.825+0000 7f46317fa700 1 -- 172.21.15.84:0/854453439 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4644023b20 con 0x557eb81fa900 2024-04-26T18:43:32.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.825+0000 7f465a8e1b80 10 client.17235 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.825+0000 7f465a8e1b80 10 client.17235 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.825+0000 7f465a8e1b80 20 client.17235 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.825+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557eb8214120 0x557eb8216560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.825+0000 7f465a8e1b80 4 client.17235 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.825+0000 7f465a8e1b80 1 -- 172.21.15.84:0/854453439 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557eb815a3a0 con 0x557eb8214120 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.825+0000 7f464d5ea700 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557eb8214120 0x557eb8216560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.825+0000 7f46317fa700 10 client.17235 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.825+0000 7f464d5ea700 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557eb8214120 0x557eb8216560 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d88664585c35b9ac server_cookie=b3c8d2ea3bfdbce4 in_seq=0 out_seq=0 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f46317fa700 1 -- 172.21.15.84:0/854453439 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557eb815a3a0 con 0x557eb8214120 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f46317fa700 10 client.17235 handle_command_reply: tid=0 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f465a8e1b80 1 client.17235 shutdown 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f465a8e1b80 1 -- 172.21.15.84:0/854453439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fa900 msgr2=0x557eb8206550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fa900 0x557eb8206550 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f46440099c0 tx=0x7f4644009310 comp rx=0 tx=0).stop 2024-04-26T18:43:32.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f465a8e1b80 1 -- 172.21.15.84:0/854453439 shutdown_connections 2024-04-26T18:43:32.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557eb8214120 0x557eb8216560 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fa900 0x557eb8206550 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 0x557eb8203e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f46317fa700 0 client.17235 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:32.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/854453439 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fff30 0x557eb8203940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f465a8e1b80 1 -- 172.21.15.84:0/854453439 >> 172.21.15.84:0/854453439 conn(0x557eb8128750 msgr2=0x557eb81f92d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:32.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f465a8e1b80 1 -- 172.21.15.84:0/854453439 shutdown_connections 2024-04-26T18:43:32.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.826+0000 7f465a8e1b80 1 -- 172.21.15.84:0/854453439 wait complete. 2024-04-26T18:43:32.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.827+0000 7f465a8e1b80 20 client.17235 trim_cache size 0 max 16384 2024-04-26T18:43:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f465a8e1b80 1 Processor -- start 2024-04-26T18:43:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f465a8e1b80 1 -- start start 2024-04-26T18:43:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f465a8e1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fa900 0x557eb81fa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f465a8e1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 0x557eb82101c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f465a8e1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 0x557eb82012e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f465a8e1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f46481aff90 con 0x557eb81fff30 2024-04-26T18:43:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f465a8e1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46481b1ad0 con 0x557eb81fa900 2024-04-26T18:43:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f465a8e1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46481b0660 con 0x557eb8200850 2024-04-26T18:43:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f464d5ea700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 0x557eb82101c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.849 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f464d5ea700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 0x557eb82101c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41028/0 (socket says 172.21.15.84:41028) 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f464d5ea700 1 -- 172.21.15.84:0/640428079 learned_addr learned my addr 172.21.15.84:0/640428079 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f464ddeb700 1 --2- 172.21.15.84:0/640428079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fa900 0x557eb81fa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f464d5ea700 1 -- 172.21.15.84:0/640428079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 msgr2=0x557eb82012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f464e5ec700 1 --2- 172.21.15.84:0/640428079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 0x557eb82012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f464d5ea700 1 --2- 172.21.15.84:0/640428079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 0x557eb82012e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.846+0000 7f464d5ea700 1 -- 172.21.15.84:0/640428079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fa900 msgr2=0x557eb81fa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.847+0000 7f464d5ea700 1 --2- 172.21.15.84:0/640428079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fa900 0x557eb81fa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.847+0000 7f464d5ea700 1 -- 172.21.15.84:0/640428079 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557eb8201ee0 con 0x557eb81fff30 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.847+0000 7f464ddeb700 1 --2- 172.21.15.84:0/640428079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fa900 0x557eb81fa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.847+0000 7f464d5ea700 1 --2- 172.21.15.84:0/640428079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 0x557eb82101c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4638098450 tx=0x7f4638004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e100663e3e76565 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.847+0000 7f46337fe700 1 -- 172.21.15.84:0/640428079 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4638064030 con 0x557eb81fff30 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.847+0000 7f46337fe700 1 -- 172.21.15.84:0/640428079 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f463809a2f0 con 0x557eb81fff30 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.847+0000 7f46337fe700 1 -- 172.21.15.84:0/640428079 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463809c4e0 con 0x557eb81fff30 2024-04-26T18:43:32.850 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 -- 172.21.15.84:0/640428079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 msgr2=0x557eb82101c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/640428079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 0x557eb82101c0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f4638098450 tx=0x7f4638004860 comp rx=0 tx=0).stop 2024-04-26T18:43:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 -- 172.21.15.84:0/640428079 shutdown_connections 2024-04-26T18:43:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/640428079 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 0x557eb82012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/640428079 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fa900 0x557eb81fa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/640428079 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 0x557eb82101c0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 -- 172.21.15.84:0/640428079 >> 172.21.15.84:0/640428079 conn(0x557eb8128750 msgr2=0x557eb81eef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 -- 172.21.15.84:0/640428079 shutdown_connections 2024-04-26T18:43:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 -- 172.21.15.84:0/640428079 wait complete. 2024-04-26T18:43:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 -- start start 2024-04-26T18:43:32.851 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 Processor -- start 2024-04-26T18:43:32.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fa900 0x557eb820cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 0x557eb82096a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 0x557eb8209de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f46481b1400 con 0x557eb81fff30 2024-04-26T18:43:32.853 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f46481b0d30 con 0x557eb81fa900 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f46481adf70 con 0x557eb8200850 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.848+0000 7f465a8e1b80 10 client.0 _ll_register_callbacks cb 0x557eb80e4ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f464d5ea700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 0x557eb82096a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f464d5ea700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 0x557eb82096a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41036/0 (socket says 172.21.15.84:41036) 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f464d5ea700 1 -- 172.21.15.84:0/3573160795 learned_addr learned my addr 172.21.15.84:0/3573160795 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f464e5ec700 1 --2- 172.21.15.84:0/3573160795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 0x557eb8209de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f464ddeb700 1 --2- 172.21.15.84:0/3573160795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fa900 0x557eb820cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f464d5ea700 1 -- 172.21.15.84:0/3573160795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 msgr2=0x557eb8209de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f464d5ea700 1 --2- 172.21.15.84:0/3573160795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 0x557eb8209de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f464d5ea700 1 -- 172.21.15.84:0/3573160795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fa900 msgr2=0x557eb820cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f464d5ea700 1 --2- 172.21.15.84:0/3573160795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fa900 0x557eb820cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f464d5ea700 1 -- 172.21.15.84:0/3573160795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557eb827a6b0 con 0x557eb81fff30 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f464d5ea700 1 --2- 172.21.15.84:0/3573160795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 0x557eb82096a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f463809bca0 tx=0x7f46380cdc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce1d1e557af3e920 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:32.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f46317fa700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f46317fa700 1 -- 172.21.15.84:0/3573160795 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46380a47d0 con 0x557eb81fff30 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f46317fa700 1 -- 172.21.15.84:0/3573160795 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7f46380a4970 con 0x557eb81fff30 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f465a8e1b80 1 -- 172.21.15.84:0/3573160795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f463809a2f0 con 0x557eb81fff30 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f46317fa700 1 -- 172.21.15.84:0/3573160795 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4638085100 con 0x557eb81fff30 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f465a8e1b80 10 client.17241 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f465a8e1b80 1 -- 172.21.15.84:0/3573160795 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557eb827b8a0 con 0x557eb81fff30 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.849+0000 7f46317fa700 1 -- 172.21.15.84:0/3573160795 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f463809a2f0 con 0x557eb81fff30 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.850+0000 7f46317fa700 1 -- 172.21.15.84:0/3573160795 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f463809b440 con 0x557eb81fff30 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.850+0000 7f465a8e1b80 10 client.17241 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.850+0000 7f465a8e1b80 10 client.17241 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.850+0000 7f465a8e1b80 20 client.17241 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.850+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/3573160795 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557eb827bac0 0x557eb827deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.850+0000 7f465a8e1b80 4 client.17241 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}] 2024-04-26T18:43:32.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.850+0000 7f465a8e1b80 1 -- 172.21.15.84:0/3573160795 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 70.0, "await_for": 42.0}) -- 0x557eb8154440 con 0x557eb827bac0 2024-04-26T18:43:32.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.850+0000 7f464ddeb700 1 --2- 172.21.15.84:0/3573160795 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557eb827bac0 0x557eb827deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:32.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.850+0000 7f464ddeb700 1 --2- 172.21.15.84:0/3573160795 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557eb827bac0 0x557eb827deb0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55a047d2c5171a6c server_cookie=5349244000b29687 in_seq=0 out_seq=0 2024-04-26T18:43:32.856 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.850+0000 7f46317fa700 10 client.17241 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:32.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.866+0000 7f46317fa700 1 -- 172.21.15.84:0/3573160795 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x557eb8154440 con 0x557eb827bac0 2024-04-26T18:43:32.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.866+0000 7f46317fa700 10 client.17241 handle_command_reply: tid=0 2024-04-26T18:43:32.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465a8e1b80 1 client.17241 shutdown 2024-04-26T18:43:32.868 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465a8e1b80 1 -- 172.21.15.84:0/3573160795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 msgr2=0x557eb82096a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/3573160795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 0x557eb82096a0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f463809bca0 tx=0x7f46380cdc50 comp rx=0 tx=0).stop 2024-04-26T18:43:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465a8e1b80 1 -- 172.21.15.84:0/3573160795 shutdown_connections 2024-04-26T18:43:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/3573160795 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557eb827bac0 0x557eb827deb0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/3573160795 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557eb8200850 0x557eb8209de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/3573160795 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557eb81fa900 0x557eb820cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f46317fa700 0 client.17241 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465a8e1b80 1 --2- 172.21.15.84:0/3573160795 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557eb81fff30 0x557eb82096a0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465a8e1b80 1 -- 172.21.15.84:0/3573160795 >> 172.21.15.84:0/3573160795 conn(0x557eb8128750 msgr2=0x557eb81eef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465a8e1b80 1 -- 172.21.15.84:0/3573160795 shutdown_connections 2024-04-26T18:43:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465a8e1b80 1 -- 172.21.15.84:0/3573160795 wait complete. 2024-04-26T18:43:32.869 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465a8e1b80 20 client.17241 trim_cache size 0 max 16384 2024-04-26T18:43:32.870 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:43:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:43:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:43:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 42, 2024-04-26T18:43:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:43:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: "835e6b7c": { 2024-04-26T18:43:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 42, 2024-04-26T18:43:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:43:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:43:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:43:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:43:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:43:32.870 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:43:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 70.0, 2024-04-26T18:43:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:43:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:43:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:43:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:43:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:43:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:43:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:43:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:43:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:43:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:43:32.871 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:43:32.871 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:43:32.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465004f700 1 -- 172.21.15.84:0/1972850742 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46340a3e50 msgr2=0x7f46340a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:32.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465004f700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46340a3e50 0x7f46340a6310 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4644000c00 tx=0x7f464401a040 comp rx=0 tx=0).stop 2024-04-26T18:43:32.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465004f700 1 -- 172.21.15.84:0/1972850742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4648103e90 msgr2=0x7f464819fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:32.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.867+0000 7f465004f700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4648103e90 0x7f464819fc90 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f463800b530 tx=0x7f463800b8f0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.868+0000 7f465004f700 1 -- 172.21.15.84:0/1972850742 shutdown_connections 2024-04-26T18:43:32.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.868+0000 7f465004f700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f46340a3e50 0x7f46340a6310 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.868+0000 7f465004f700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f46481094a0 0x7f4648199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.872 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.868+0000 7f465004f700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f46481034d0 0x7f464819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.868+0000 7f465004f700 1 --2- 172.21.15.84:0/1972850742 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4648103e90 0x7f464819fc90 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:32.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.868+0000 7f465004f700 1 -- 172.21.15.84:0/1972850742 >> 172.21.15.84:0/1972850742 conn(0x7f464807a4f0 msgr2=0x7f4648101fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:32.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.868+0000 7f465004f700 1 -- 172.21.15.84:0/1972850742 shutdown_connections 2024-04-26T18:43:32.873 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:32.868+0000 7f465004f700 1 -- 172.21.15.84:0/1972850742 wait complete. 2024-04-26T18:43:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.118+0000 7f09e7b3f640 1 -- 172.21.15.84:0/2360706186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09e01062e0 msgr2=0x7f09e010deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.118+0000 7f09e7b3f640 1 --2- 172.21.15.84:0/2360706186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09e01062e0 0x7f09e010deb0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f09d8009ca0 tx=0x7f09d802a8c0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.122 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.118+0000 7f09e7b3f640 1 -- 172.21.15.84:0/2360706186 shutdown_connections 2024-04-26T18:43:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.118+0000 7f09e7b3f640 1 --2- 172.21.15.84:0/2360706186 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09e01062e0 0x7f09e010deb0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.118+0000 7f09e7b3f640 1 --2- 172.21.15.84:0/2360706186 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0101bc0 0x7f09e0105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.118+0000 7f09e7b3f640 1 --2- 172.21.15.84:0/2360706186 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09e0101210 0x7f09e01015f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.118+0000 7f09e7b3f640 1 -- 172.21.15.84:0/2360706186 >> 172.21.15.84:0/2360706186 conn(0x7f09e0077d50 msgr2=0x7f09e00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.119+0000 7f09e7b3f640 1 -- 172.21.15.84:0/2360706186 shutdown_connections 2024-04-26T18:43:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.119+0000 7f09e7b3f640 1 -- 172.21.15.84:0/2360706186 wait complete. 2024-04-26T18:43:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.119+0000 7f09e7b3f640 1 Processor -- start 2024-04-26T18:43:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.119+0000 7f09e7b3f640 1 -- start start 2024-04-26T18:43:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.119+0000 7f09e7b3f640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0101210 0x7f09e0197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.119+0000 7f09e7b3f640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09e01062e0 0x7f09e0198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.119+0000 7f09e7b3f640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09e019c810 0x7f09e019ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.119+0000 7f09e7b3f640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01112a0 con 0x7f09e019c810 2024-04-26T18:43:33.123 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.119+0000 7f09e7b3f640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09e0111120 con 0x7f09e0101210 2024-04-26T18:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.119+0000 7f09e7b3f640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09e0111420 con 0x7f09e01062e0 2024-04-26T18:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.119+0000 7f09e60dc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09e019c810 0x7f09e019ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09e60dc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09e019c810 0x7f09e019ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41056/0 (socket says 172.21.15.84:41056) 2024-04-26T18:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09e60dc640 1 -- 172.21.15.84:0/4227707034 learned_addr learned my addr 172.21.15.84:0/4227707034 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09e58db640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0101210 0x7f09e0197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09e50da640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09e01062e0 0x7f09e0198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09e60dc640 1 -- 172.21.15.84:0/4227707034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09e01062e0 msgr2=0x7f09e0198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09e60dc640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09e01062e0 0x7f09e0198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09e60dc640 1 -- 172.21.15.84:0/4227707034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0101210 msgr2=0x7f09e0197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:33.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09e60dc640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0101210 0x7f09e0197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09e60dc640 1 -- 172.21.15.84:0/4227707034 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09e019d300 con 0x7f09e019c810 2024-04-26T18:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09e50da640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09e01062e0 0x7f09e0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09e60dc640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09e019c810 0x7f09e019ccc0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f09d8005cc0 tx=0x7f09d802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09d6ffd640 1 -- 172.21.15.84:0/4227707034 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d8003cf0 con 0x7f09e019c810 2024-04-26T18:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09e7b3f640 1 -- 172.21.15.84:0/4227707034 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09e019d590 con 0x7f09e019c810 2024-04-26T18:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09d6ffd640 1 -- 172.21.15.84:0/4227707034 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09d80044f0 con 0x7f09e019c810 2024-04-26T18:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09d6ffd640 1 -- 172.21.15.84:0/4227707034 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d802fc30 con 0x7f09e019c810 2024-04-26T18:43:33.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.120+0000 7f09e7b3f640 1 -- 172.21.15.84:0/4227707034 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09e01a4d90 con 0x7f09e019c810 2024-04-26T18:43:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.122+0000 7f09d6ffd640 1 -- 172.21.15.84:0/4227707034 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f09d80046a0 con 0x7f09e019c810 2024-04-26T18:43:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.123+0000 7f09d6ffd640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f09cc0a3dc0 0x7f09cc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.123+0000 7f09ea858740 1 Processor -- start 2024-04-26T18:43:33.132 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.123+0000 7f09e58db640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f09cc0a3dc0 0x7f09cc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.123+0000 7f09d6ffd640 1 -- 172.21.15.84:0/4227707034 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7f09d80f1e20 con 0x7f09e019c810 2024-04-26T18:43:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.123+0000 7f09ea858740 1 -- start start 2024-04-26T18:43:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09ea858740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae541a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09ea858740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae546e0 0x560a9ae60db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09ea858740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae612f0 0x560a9ae636e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09ea858740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560a9adba460 con 0x560a9ae53dc0 2024-04-26T18:43:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09ea858740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560a9ad74590 con 0x560a9ae612f0 2024-04-26T18:43:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09ea858740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560a9adb6420 con 0x560a9ae546e0 2024-04-26T18:43:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09e58db640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f09cc0a3dc0 0x7f09cc0a6280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009990 tx=0x7f09d0009110 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:33.133 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09e58db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09e58db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41058/0 (socket says 172.21.15.84:41058) 2024-04-26T18:43:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09e58db640 1 -- 172.21.15.84:0/1769207764 learned_addr learned my addr 172.21.15.84:0/1769207764 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09e50da640 1 --2- 172.21.15.84:0/1769207764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae546e0 0x560a9ae60db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09e60dc640 1 --2- 172.21.15.84:0/1769207764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae612f0 0x560a9ae636e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09e58db640 1 -- 172.21.15.84:0/1769207764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae546e0 msgr2=0x560a9ae60db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09e58db640 1 --2- 172.21.15.84:0/1769207764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae546e0 0x560a9ae60db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09e58db640 1 -- 172.21.15.84:0/1769207764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae612f0 msgr2=0x560a9ae636e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09e58db640 1 --2- 172.21.15.84:0/1769207764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae612f0 0x560a9ae636e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09e58db640 1 -- 172.21.15.84:0/1769207764 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a9ad810d0 con 0x560a9ae53dc0 2024-04-26T18:43:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09e58db640 1 --2- 172.21.15.84:0/1769207764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae541a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f09d000ebd0 tx=0x7f09d0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a95936579d1043 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:33.134 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09cbfff640 1 -- 172.21.15.84:0/1769207764 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d001b040 con 0x560a9ae53dc0 2024-04-26T18:43:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09cbfff640 1 -- 172.21.15.84:0/1769207764 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09d0024070 con 0x560a9ae53dc0 2024-04-26T18:43:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.124+0000 7f09cbfff640 1 -- 172.21.15.84:0/1769207764 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d001f3f0 con 0x560a9ae53dc0 2024-04-26T18:43:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.125+0000 7f09ea858740 1 -- 172.21.15.84:0/1769207764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 msgr2=0x560a9ae541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.125+0000 7f09ea858740 1 --2- 172.21.15.84:0/1769207764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae541a0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f09d000ebd0 tx=0x7f09d0011000 comp rx=0 tx=0).stop 2024-04-26T18:43:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.125+0000 7f09ea858740 1 -- 172.21.15.84:0/1769207764 shutdown_connections 2024-04-26T18:43:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.125+0000 7f09ea858740 1 --2- 172.21.15.84:0/1769207764 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae612f0 0x560a9ae636e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.125+0000 7f09ea858740 1 --2- 172.21.15.84:0/1769207764 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae546e0 0x560a9ae60db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.125+0000 7f09ea858740 1 --2- 172.21.15.84:0/1769207764 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae541a0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.125+0000 7f09ea858740 1 -- 172.21.15.84:0/1769207764 >> 172.21.15.84:0/1769207764 conn(0x560a9ae50410 msgr2=0x560a9ae507f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.125+0000 7f09ea858740 1 -- 172.21.15.84:0/1769207764 shutdown_connections 2024-04-26T18:43:33.135 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.125+0000 7f09ea858740 1 -- 172.21.15.84:0/1769207764 wait complete. 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09ea858740 1 -- start start 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09ea858740 1 Processor -- start 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09ea858740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae53dc0 0x560a9ae5e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09ea858740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae546e0 0x560a9ae5e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09ea858740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae612f0 0x560a9ae5ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09ea858740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x560a9ae63ec0 con 0x560a9ae546e0 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09ea858740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x560a9ad41b60 con 0x560a9ae53dc0 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09ea858740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x560a9ae64040 con 0x560a9ae612f0 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09ea858740 10 client.0 _ll_register_callbacks cb 0x560a9ad662b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09e50da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae546e0 0x560a9ae5e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09e50da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae546e0 0x560a9ae5e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41068/0 (socket says 172.21.15.84:41068) 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09e50da640 1 -- 172.21.15.84:0/2128416338 learned_addr learned my addr 172.21.15.84:0/2128416338 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09e60dc640 1 --2- 172.21.15.84:0/2128416338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae612f0 0x560a9ae5ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.136 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09e58db640 1 --2- 172.21.15.84:0/2128416338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae53dc0 0x560a9ae5e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09e50da640 1 -- 172.21.15.84:0/2128416338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae612f0 msgr2=0x560a9ae5ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09e50da640 1 --2- 172.21.15.84:0/2128416338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae612f0 0x560a9ae5ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09e50da640 1 -- 172.21.15.84:0/2128416338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae53dc0 msgr2=0x560a9ae5e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09e50da640 1 --2- 172.21.15.84:0/2128416338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae53dc0 0x560a9ae5e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.126+0000 7f09e50da640 1 -- 172.21.15.84:0/2128416338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a9aacc210 con 0x560a9ae546e0 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09e50da640 1 --2- 172.21.15.84:0/2128416338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae546e0 0x560a9ae5e790 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f09dc011850 tx=0x7f09dc011d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=badea95ef50b5f55 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09c97fa640 1 -- 172.21.15.84:0/2128416338 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09dc00f9a0 con 0x560a9ae546e0 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09c97fa640 1 -- 172.21.15.84:0/2128416338 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7f09dc00fb40 con 0x560a9ae546e0 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09ea858740 1 -- 172.21.15.84:0/2128416338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09d0024070 con 0x560a9ae546e0 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09c97fa640 1 -- 172.21.15.84:0/2128416338 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09dc00fd40 con 0x560a9ae546e0 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09ea858740 10 client.17253 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09ea858740 1 -- 172.21.15.84:0/2128416338 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a9ab98620 con 0x560a9ae546e0 2024-04-26T18:43:33.137 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09c97fa640 1 -- 172.21.15.84:0/2128416338 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09d0024070 con 0x560a9ae546e0 2024-04-26T18:43:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09c97fa640 1 -- 172.21.15.84:0/2128416338 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f09dc018030 con 0x560a9ae546e0 2024-04-26T18:43:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09ea858740 10 client.17253 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09ea858740 10 client.17253 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:43:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09ea858740 20 client.17253 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09ea858740 1 --2- 172.21.15.84:0/2128416338 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560a9ae71420 0x560a9ae73810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09ea858740 4 client.17253 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:43:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.127+0000 7f09ea858740 1 -- 172.21.15.84:0/2128416338 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a9adc8070 con 0x560a9ae71420 2024-04-26T18:43:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.128+0000 7f09e58db640 1 --2- 172.21.15.84:0/2128416338 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560a9ae71420 0x560a9ae73810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.128+0000 7f09e58db640 1 --2- 172.21.15.84:0/2128416338 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560a9ae71420 0x560a9ae73810 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea888092a543e48e server_cookie=7910cc2b1756c9db in_seq=0 out_seq=0 2024-04-26T18:43:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.128+0000 7f09c97fa640 10 client.17253 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09c97fa640 1 -- 172.21.15.84:0/2128416338 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a9adc8070 con 0x560a9ae71420 2024-04-26T18:43:33.138 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09c97fa640 10 client.17253 handle_command_reply: tid=0 2024-04-26T18:43:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09ea858740 1 client.17253 shutdown 2024-04-26T18:43:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09ea858740 1 -- 172.21.15.84:0/2128416338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae546e0 msgr2=0x560a9ae5e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09ea858740 1 --2- 172.21.15.84:0/2128416338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae546e0 0x560a9ae5e790 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f09dc011850 tx=0x7f09dc011d10 comp rx=0 tx=0).stop 2024-04-26T18:43:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09ea858740 1 -- 172.21.15.84:0/2128416338 shutdown_connections 2024-04-26T18:43:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09ea858740 1 --2- 172.21.15.84:0/2128416338 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560a9ae71420 0x560a9ae73810 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09c97fa640 0 client.17253 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09ea858740 1 --2- 172.21.15.84:0/2128416338 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae612f0 0x560a9ae5ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09ea858740 1 --2- 172.21.15.84:0/2128416338 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae546e0 0x560a9ae5e790 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09ea858740 1 --2- 172.21.15.84:0/2128416338 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae53dc0 0x560a9ae5e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09ea858740 1 -- 172.21.15.84:0/2128416338 >> 172.21.15.84:0/2128416338 conn(0x560a9ae50410 msgr2=0x560a9ae561b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09ea858740 1 -- 172.21.15.84:0/2128416338 shutdown_connections 2024-04-26T18:43:33.139 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09ea858740 1 -- 172.21.15.84:0/2128416338 wait complete. 2024-04-26T18:43:33.140 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.129+0000 7f09ea858740 20 client.17253 trim_cache size 0 max 16384 2024-04-26T18:43:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09ea858740 1 Processor -- start 2024-04-26T18:43:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09ea858740 1 -- start start 2024-04-26T18:43:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09ea858740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae6cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09ea858740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae546e0 0x560a9ae56030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09ea858740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae612f0 0x560a9ae79120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09ea858740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01ac430 con 0x560a9ae53dc0 2024-04-26T18:43:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09ea858740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01adf70 con 0x560a9ae612f0 2024-04-26T18:43:33.154 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09ea858740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01acb00 con 0x560a9ae546e0 2024-04-26T18:43:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09e58db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae6cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09e58db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae6cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41084/0 (socket says 172.21.15.84:41084) 2024-04-26T18:43:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09e58db640 1 -- 172.21.15.84:0/2708381121 learned_addr learned my addr 172.21.15.84:0/2708381121 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09e50da640 1 --2- 172.21.15.84:0/2708381121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae546e0 0x560a9ae56030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09e60dc640 1 --2- 172.21.15.84:0/2708381121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae612f0 0x560a9ae79120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09e58db640 1 -- 172.21.15.84:0/2708381121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae546e0 msgr2=0x560a9ae56030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09e58db640 1 --2- 172.21.15.84:0/2708381121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae546e0 0x560a9ae56030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09e58db640 1 -- 172.21.15.84:0/2708381121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae612f0 msgr2=0x560a9ae79120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09e58db640 1 --2- 172.21.15.84:0/2708381121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae612f0 0x560a9ae79120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.148+0000 7f09e58db640 1 -- 172.21.15.84:0/2708381121 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a9ae4bd60 con 0x560a9ae53dc0 2024-04-26T18:43:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.149+0000 7f09e58db640 1 --2- 172.21.15.84:0/2708381121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae6cdc0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009770 tx=0x7f09d0006d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf10522034c18222 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:33.155 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.149+0000 7f09cb7fe640 1 -- 172.21.15.84:0/2708381121 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0036070 con 0x560a9ae53dc0 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.149+0000 7f09cb7fe640 1 -- 172.21.15.84:0/2708381121 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09d0024070 con 0x560a9ae53dc0 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.149+0000 7f09cb7fe640 1 -- 172.21.15.84:0/2708381121 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0029660 con 0x560a9ae53dc0 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.149+0000 7f09ea858740 1 -- 172.21.15.84:0/2708381121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 msgr2=0x560a9ae6cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 --2- 172.21.15.84:0/2708381121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae6cdc0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009770 tx=0x7f09d0006d00 comp rx=0 tx=0).stop 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 -- 172.21.15.84:0/2708381121 shutdown_connections 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 --2- 172.21.15.84:0/2708381121 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae612f0 0x560a9ae79120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 --2- 172.21.15.84:0/2708381121 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae546e0 0x560a9ae56030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 --2- 172.21.15.84:0/2708381121 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae6cdc0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 -- 172.21.15.84:0/2708381121 >> 172.21.15.84:0/2708381121 conn(0x560a9ae50410 msgr2=0x560a9ae55920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 -- 172.21.15.84:0/2708381121 shutdown_connections 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 -- 172.21.15.84:0/2708381121 wait complete. 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 -- start start 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 Processor -- start 2024-04-26T18:43:33.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae5b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae546e0 0x560a9ae5bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae612f0 0x560a9aed6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01ad8a0 con 0x560a9ae53dc0 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01ad1d0 con 0x560a9ae546e0 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f09e01aa430 con 0x560a9ae612f0 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09ea858740 10 client.0 _ll_register_callbacks cb 0x7f09dc011460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09e58db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae5b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.150+0000 7f09e58db640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae5b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41096/0 (socket says 172.21.15.84:41096) 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09e58db640 1 -- 172.21.15.84:0/3157799796 learned_addr learned my addr 172.21.15.84:0/3157799796 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09e50da640 1 --2- 172.21.15.84:0/3157799796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae546e0 0x560a9ae5bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09e60dc640 1 --2- 172.21.15.84:0/3157799796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae612f0 0x560a9aed6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09e58db640 1 -- 172.21.15.84:0/3157799796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae612f0 msgr2=0x560a9aed6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09e58db640 1 --2- 172.21.15.84:0/3157799796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae612f0 0x560a9aed6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09e58db640 1 -- 172.21.15.84:0/3157799796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae546e0 msgr2=0x560a9ae5bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09e58db640 1 --2- 172.21.15.84:0/3157799796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae546e0 0x560a9ae5bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09e58db640 1 -- 172.21.15.84:0/3157799796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a9aed6ed0 con 0x560a9ae53dc0 2024-04-26T18:43:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09e58db640 1 --2- 172.21.15.84:0/3157799796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae5b5c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f09d000eba0 tx=0x7f09d0013ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abf48435d462d92 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09c97fa640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09c97fa640 1 -- 172.21.15.84:0/3157799796 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09d0041060 con 0x560a9ae53dc0 2024-04-26T18:43:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09c97fa640 1 -- 172.21.15.84:0/3157799796 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7f09d0015d90 con 0x560a9ae53dc0 2024-04-26T18:43:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09ea858740 1 -- 172.21.15.84:0/3157799796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09d0024070 con 0x560a9ae53dc0 2024-04-26T18:43:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09c97fa640 1 -- 172.21.15.84:0/3157799796 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09d0036030 con 0x560a9ae53dc0 2024-04-26T18:43:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09ea858740 10 client.17259 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09ea858740 1 -- 172.21.15.84:0/3157799796 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a9aed8140 con 0x560a9ae53dc0 2024-04-26T18:43:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.151+0000 7f09c97fa640 1 -- 172.21.15.84:0/3157799796 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09d0024070 con 0x560a9ae53dc0 2024-04-26T18:43:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.152+0000 7f09c97fa640 1 -- 172.21.15.84:0/3157799796 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f09d0030030 con 0x560a9ae53dc0 2024-04-26T18:43:33.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.152+0000 7f09ea858740 10 client.17259 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.152+0000 7f09ea858740 10 client.17259 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:43:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.152+0000 7f09ea858740 20 client.17259 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.152+0000 7f09ea858740 1 --2- 172.21.15.84:0/3157799796 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560a9aed8520 0x560a9aeda910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.152+0000 7f09ea858740 4 client.17259 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:43:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.152+0000 7f09ea858740 1 -- 172.21.15.84:0/3157799796 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560a9adc1ee0 con 0x560a9aed8520 2024-04-26T18:43:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.152+0000 7f09e50da640 1 --2- 172.21.15.84:0/3157799796 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560a9aed8520 0x560a9aeda910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.152+0000 7f09e50da640 1 --2- 172.21.15.84:0/3157799796 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560a9aed8520 0x560a9aeda910 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1f394a4957cef5e server_cookie=886c772f705a19b6 in_seq=0 out_seq=0 2024-04-26T18:43:33.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.152+0000 7f09c97fa640 10 client.17259 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09c97fa640 1 -- 172.21.15.84:0/3157799796 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560a9adc1ee0 con 0x560a9aed8520 2024-04-26T18:43:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09c97fa640 10 client.17259 handle_command_reply: tid=0 2024-04-26T18:43:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09ea858740 1 client.17259 shutdown 2024-04-26T18:43:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09ea858740 1 -- 172.21.15.84:0/3157799796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 msgr2=0x560a9ae5b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09ea858740 1 --2- 172.21.15.84:0/3157799796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae5b5c0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f09d000eba0 tx=0x7f09d0013ed0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09ea858740 1 -- 172.21.15.84:0/3157799796 shutdown_connections 2024-04-26T18:43:33.168 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09ea858740 1 --2- 172.21.15.84:0/3157799796 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x560a9aed8520 0x560a9aeda910 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09ea858740 1 --2- 172.21.15.84:0/3157799796 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x560a9ae612f0 0x560a9aed6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09c97fa640 0 client.17259 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09ea858740 1 --2- 172.21.15.84:0/3157799796 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x560a9ae546e0 0x560a9ae5bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09ea858740 1 --2- 172.21.15.84:0/3157799796 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x560a9ae53dc0 0x560a9ae5b5c0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09ea858740 1 -- 172.21.15.84:0/3157799796 >> 172.21.15.84:0/3157799796 conn(0x560a9ae50410 msgr2=0x560a9ae5c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09ea858740 1 -- 172.21.15.84:0/3157799796 shutdown_connections 2024-04-26T18:43:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09ea858740 1 -- 172.21.15.84:0/3157799796 wait complete. 2024-04-26T18:43:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09ea858740 20 client.17259 trim_cache size 0 max 16384 2024-04-26T18:43:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:43:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:43:33.169 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:43:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:43:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09e7b3f640 1 -- 172.21.15.84:0/4227707034 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f09cc0a3dc0 msgr2=0x7f09cc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09e7b3f640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f09cc0a3dc0 0x7f09cc0a6280 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f09d0009990 tx=0x7f09d0009110 comp rx=0 tx=0).stop 2024-04-26T18:43:33.170 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09e7b3f640 1 -- 172.21.15.84:0/4227707034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09e019c810 msgr2=0x7f09e019ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09e7b3f640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09e019c810 0x7f09e019ccc0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f09d8005cc0 tx=0x7f09d802f470 comp rx=0 tx=0).stop 2024-04-26T18:43:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09e7b3f640 1 -- 172.21.15.84:0/4227707034 shutdown_connections 2024-04-26T18:43:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09e7b3f640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f09cc0a3dc0 0x7f09cc0a6280 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09e7b3f640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f09e019c810 0x7f09e019ccc0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09e7b3f640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f09e01062e0 0x7f09e0198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09e7b3f640 1 --2- 172.21.15.84:0/4227707034 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f09e0101210 0x7f09e0197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09e7b3f640 1 -- 172.21.15.84:0/4227707034 >> 172.21.15.84:0/4227707034 conn(0x7f09e0077d50 msgr2=0x7f09e00ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09e7b3f640 1 -- 172.21.15.84:0/4227707034 shutdown_connections 2024-04-26T18:43:33.171 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:33.167+0000 7f09e7b3f640 1 -- 172.21.15.84:0/4227707034 wait complete. 2024-04-26T18:43:33.571 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '835e6b7c', quiesce took 5.9 seconds. Will release after: 10 2024-04-26T18:43:33.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:33 smithi084 ceph-mon[29141]: pgmap v490: 97 pgs: 97 active+clean; 1.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:43:33.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:33 smithi103 ceph-mon[32727]: pgmap v490: 97 pgs: 97 active+clean; 1.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:43:33.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:33 smithi174 ceph-mon[32705]: pgmap v490: 97 pgs: 97 active+clean; 1.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-26T18:43:34.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:34 smithi084 ceph-mon[29141]: pgmap v491: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-04-26T18:43:34.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:34 smithi103 ceph-mon[32727]: pgmap v491: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-04-26T18:43:34.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:34 smithi174 ceph-mon[32705]: pgmap v491: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 65 KiB/s wr, 25 op/s 2024-04-26T18:43:37.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:36 smithi084 ceph-mon[29141]: pgmap v492: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:43:37.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:36 smithi103 ceph-mon[32727]: pgmap v492: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:43:37.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:36 smithi174 ceph-mon[32705]: pgmap v492: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:43:39.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:38 smithi084 ceph-mon[29141]: pgmap v493: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:43:39.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:38 smithi103 ceph-mon[32727]: pgmap v493: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:43:39.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:38 smithi174 ceph-mon[32705]: pgmap v493: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-26T18:43:41.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:40 smithi084 ceph-mon[29141]: pgmap v494: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 20 op/s 2024-04-26T18:43:41.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:40 smithi103 ceph-mon[32727]: pgmap v494: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 20 op/s 2024-04-26T18:43:41.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:40 smithi174 ceph-mon[32705]: pgmap v494: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 20 op/s 2024-04-26T18:43:43.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:42 smithi084 ceph-mon[29141]: pgmap v495: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-26T18:43:43.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:42 smithi103 ceph-mon[32727]: pgmap v495: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-26T18:43:43.190 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:42 smithi174 ceph-mon[32705]: pgmap v495: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 13 op/s 2024-04-26T18:43:43.572 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:43:43.826 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:43:45.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:44 smithi084 ceph-mon[29141]: pgmap v496: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-04-26T18:43:45.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:44 smithi103 ceph-mon[32727]: pgmap v496: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-04-26T18:43:45.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:44 smithi174 ceph-mon[32705]: pgmap v496: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 28 KiB/s wr, 18 op/s 2024-04-26T18:43:45.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.479+0000 7fcfd572c700 1 -- 172.21.15.84:0/4181026320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd0103e30 msgr2=0x7fcfd0110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:45.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.479+0000 7fcfd572c700 1 --2- 172.21.15.84:0/4181026320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd0103e30 0x7fcfd0110290 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fcfc800b0a0 tx=0x7fcfc800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:43:45.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.479+0000 7fcfd572c700 1 -- 172.21.15.84:0/4181026320 shutdown_connections 2024-04-26T18:43:45.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.479+0000 7fcfd572c700 1 --2- 172.21.15.84:0/4181026320 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd0103e30 0x7fcfd0110290 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:45.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.479+0000 7fcfd572c700 1 --2- 172.21.15.84:0/4181026320 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfd01094c0 0x7fcfd01098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:45.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.479+0000 7fcfd572c700 1 --2- 172.21.15.84:0/4181026320 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfd0103470 0x7fcfd01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:45.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.479+0000 7fcfd572c700 1 -- 172.21.15.84:0/4181026320 >> 172.21.15.84:0/4181026320 conn(0x7fcfd007a580 msgr2=0x7fcfd007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.479+0000 7fcfd572c700 1 -- 172.21.15.84:0/4181026320 shutdown_connections 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.479+0000 7fcfd572c700 1 -- 172.21.15.84:0/4181026320 wait complete. 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.479+0000 7fcfd572c700 1 Processor -- start 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfd572c700 1 -- start start 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfd572c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd0103470 0x7fcfd0075720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfd572c700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfd0103e30 0x7fcfd0075c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfd572c700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfd01094c0 0x7fcfd00761a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfd572c700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd01135c0 con 0x7fcfd0103470 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfd572c700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd0113440 con 0x7fcfd0103e30 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfd572c700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcfd0113740 con 0x7fcfd01094c0 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfceffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd0103470 0x7fcfd0075720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfceffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd0103470 0x7fcfd0075720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51210/0 (socket says 172.21.15.84:51210) 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfceffd700 1 -- 172.21.15.84:0/3306757485 learned_addr learned my addr 172.21.15.84:0/3306757485 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:45.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfcf7fe700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfd01094c0 0x7fcfd00761a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfce7fc700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfd0103e30 0x7fcfd0075c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfceffd700 1 -- 172.21.15.84:0/3306757485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfd01094c0 msgr2=0x7fcfd00761a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfceffd700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfd01094c0 0x7fcfd00761a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfceffd700 1 -- 172.21.15.84:0/3306757485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfd0103e30 msgr2=0x7fcfd0075c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfceffd700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfd0103e30 0x7fcfd0075c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfceffd700 1 -- 172.21.15.84:0/3306757485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfd01a7f20 con 0x7fcfd0103470 2024-04-26T18:43:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfce7fc700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfd0103e30 0x7fcfd0075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:43:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.480+0000 7fcfceffd700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd0103470 0x7fcfd0075720 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000c920 tx=0x7fcfc000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.481+0000 7fcfc7fff700 1 -- 172.21.15.84:0/3306757485 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0007b00 con 0x7fcfd0103470 2024-04-26T18:43:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.481+0000 7fcfd572c700 1 -- 172.21.15.84:0/3306757485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfd01a8210 con 0x7fcfd0103470 2024-04-26T18:43:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.481+0000 7fcfc7fff700 1 -- 172.21.15.84:0/3306757485 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfc0004ce0 con 0x7fcfd0103470 2024-04-26T18:43:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.481+0000 7fcfc7fff700 1 -- 172.21.15.84:0/3306757485 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfc0005500 con 0x7fcfd0103470 2024-04-26T18:43:45.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.481+0000 7fcfd572c700 1 -- 172.21.15.84:0/3306757485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfd01a8750 con 0x7fcfd0103470 2024-04-26T18:43:45.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.482+0000 7fcfd572c700 1 -- 172.21.15.84:0/3306757485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcfd006a110 con 0x7fcfd0103470 2024-04-26T18:43:45.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.483+0000 7fcfc7fff700 1 -- 172.21.15.84:0/3306757485 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fcfc0007ca0 con 0x7fcfd0103470 2024-04-26T18:43:45.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.483+0000 7fcfc7fff700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcfbc0a3db0 0x7fcfbc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:45.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.483+0000 7fcfce7fc700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcfbc0a3db0 0x7fcfbc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:45.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.483+0000 7fcfc7fff700 1 -- 172.21.15.84:0/3306757485 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7fcfc00d0360 con 0x7fcfd0103470 2024-04-26T18:43:45.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.484+0000 7fcfce7fc700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcfbc0a3db0 0x7fcfbc0a6270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8005fd0 tx=0x7fcfb8005e20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:45.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.488+0000 7fcfc7fff700 1 -- 172.21.15.84:0/3306757485 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcfc0014030 con 0x7fcfd0103470 2024-04-26T18:43:45.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.737+0000 7fcfd572c700 1 -- 172.21.15.84:0/3306757485 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcfd002e490 con 0x7fcfd0103470 2024-04-26T18:43:45.737 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.738+0000 7fcfc7fff700 1 -- 172.21.15.84:0/3306757485 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7fcfc009c9c0 con 0x7fcfd0103470 2024-04-26T18:43:45.738 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:43:45.738 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:43:45.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.740+0000 7fcfd572c700 1 -- 172.21.15.84:0/3306757485 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcfbc0a3db0 msgr2=0x7fcfbc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:45.740 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.740+0000 7fcfd572c700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcfbc0a3db0 0x7fcfbc0a6270 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcfb8005fd0 tx=0x7fcfb8005e20 comp rx=0 tx=0).stop 2024-04-26T18:43:45.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.740+0000 7fcfd572c700 1 -- 172.21.15.84:0/3306757485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd0103470 msgr2=0x7fcfd0075720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:45.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.740+0000 7fcfd572c700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd0103470 0x7fcfd0075720 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fcfc000c920 tx=0x7fcfc000cc30 comp rx=0 tx=0).stop 2024-04-26T18:43:45.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.740+0000 7fcfd572c700 1 -- 172.21.15.84:0/3306757485 shutdown_connections 2024-04-26T18:43:45.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.740+0000 7fcfd572c700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcfbc0a3db0 0x7fcfbc0a6270 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:45.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.740+0000 7fcfd572c700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcfd0103470 0x7fcfd0075720 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:45.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.740+0000 7fcfd572c700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcfd01094c0 0x7fcfd00761a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:45.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.740+0000 7fcfd572c700 1 --2- 172.21.15.84:0/3306757485 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcfd0103e30 0x7fcfd0075c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:45.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.740+0000 7fcfd572c700 1 -- 172.21.15.84:0/3306757485 >> 172.21.15.84:0/3306757485 conn(0x7fcfd007a580 msgr2=0x7fcfd01020a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:45.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.740+0000 7fcfd572c700 1 -- 172.21.15.84:0/3306757485 shutdown_connections 2024-04-26T18:43:45.741 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:45.740+0000 7fcfd572c700 1 -- 172.21.15.84:0/3306757485 wait complete. 2024-04-26T18:43:45.741 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:43:46.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:45 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3306757485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:43:46.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:45 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3306757485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:43:46.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:45 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3306757485' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:43:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.255+0000 7fdd68f46640 1 -- 172.21.15.84:0/2213376457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd64108950 msgr2=0x7fdd64108d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.255+0000 7fdd68f46640 1 --2- 172.21.15.84:0/2213376457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd64108950 0x7fdd64108d30 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009930 tx=0x7fdd5402a420 comp rx=0 tx=0).stop 2024-04-26T18:43:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.255+0000 7fdd68f46640 1 -- 172.21.15.84:0/2213376457 shutdown_connections 2024-04-26T18:43:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.255+0000 7fdd68f46640 1 --2- 172.21.15.84:0/2213376457 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd64103300 0x7fdd6410f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.255+0000 7fdd68f46640 1 --2- 172.21.15.84:0/2213376457 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd64102940 0x7fdd64102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.255+0000 7fdd68f46640 1 --2- 172.21.15.84:0/2213376457 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd64108950 0x7fdd64108d30 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.255+0000 7fdd68f46640 1 -- 172.21.15.84:0/2213376457 >> 172.21.15.84:0/2213376457 conn(0x7fdd64079dc0 msgr2=0x7fdd6407a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.255+0000 7fdd68f46640 1 -- 172.21.15.84:0/2213376457 shutdown_connections 2024-04-26T18:43:46.267 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.255+0000 7fdd68f46640 1 -- 172.21.15.84:0/2213376457 wait complete. 2024-04-26T18:43:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.256+0000 7fdd68f46640 1 Processor -- start 2024-04-26T18:43:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.256+0000 7fdd68f46640 1 -- start start 2024-04-26T18:43:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.256+0000 7fdd68f46640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd64102940 0x7fdd64199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.256+0000 7fdd68f46640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd64103300 0x7fdd64199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.256+0000 7fdd68f46640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd64108950 0x7fdd6419d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.256+0000 7fdd68f46640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64112c30 con 0x7fdd64108950 2024-04-26T18:43:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.256+0000 7fdd68f46640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64112ab0 con 0x7fdd64103300 2024-04-26T18:43:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.256+0000 7fdd68f46640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64112db0 con 0x7fdd64102940 2024-04-26T18:43:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.256+0000 7fdd62d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd64108950 0x7fdd6419d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd62d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd64108950 0x7fdd6419d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51224/0 (socket says 172.21.15.84:51224) 2024-04-26T18:43:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd62d9d640 1 -- 172.21.15.84:0/2903208050 learned_addr learned my addr 172.21.15.84:0/2903208050 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd6259c640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd64102940 0x7fdd64199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.268 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd64103300 0x7fdd64199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd62d9d640 1 -- 172.21.15.84:0/2903208050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd64102940 msgr2=0x7fdd64199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd62d9d640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd64102940 0x7fdd64199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd62d9d640 1 -- 172.21.15.84:0/2903208050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd64103300 msgr2=0x7fdd64199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd62d9d640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd64103300 0x7fdd64199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd62d9d640 1 -- 172.21.15.84:0/2903208050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6419e100 con 0x7fdd64108950 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd6259c640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd64102940 0x7fdd64199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd64103300 0x7fdd64199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd62d9d640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd64108950 0x7fdd6419d9d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00b9e0 tx=0x7fdd5c00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd537fe640 1 -- 172.21.15.84:0/2903208050 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c019070 con 0x7fdd64108950 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd537fe640 1 -- 172.21.15.84:0/2903208050 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd5c014440 con 0x7fdd64108950 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd68f46640 1 -- 172.21.15.84:0/2903208050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6419e3f0 con 0x7fdd64108950 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd537fe640 1 -- 172.21.15.84:0/2903208050 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c0075d0 con 0x7fdd64108950 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.257+0000 7fdd68f46640 1 -- 172.21.15.84:0/2903208050 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd641a5c80 con 0x7fdd64108950 2024-04-26T18:43:46.269 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.259+0000 7fdd537fe640 1 -- 172.21.15.84:0/2903208050 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fdd5c01c070 con 0x7fdd64108950 2024-04-26T18:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd6bc64740 1 Processor -- start 2024-04-26T18:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd6bc64740 1 -- start start 2024-04-26T18:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd6bc64740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dcef00 0x5573f6dcf2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd6bc64740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcf820 0x5573f6dd9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd6bc64740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dda0e0 0x5573f6ddc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd6bc64740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5573f6d35460 con 0x5573f6dda0e0 2024-04-26T18:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd6bc64740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5573f6cef590 con 0x5573f6dcf820 2024-04-26T18:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd6bc64740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5573f6d31420 con 0x5573f6dcef00 2024-04-26T18:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd537fe640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fdd4c0a3e40 0x7fdd4c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd61d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dda0e0 0x5573f6ddc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd61d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dda0e0 0x5573f6ddc4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51232/0 (socket says 172.21.15.84:51232) 2024-04-26T18:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd62d9d640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fdd4c0a3e40 0x7fdd4c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.270 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd61d9b640 1 -- 172.21.15.84:0/3314591102 learned_addr learned my addr 172.21.15.84:0/3314591102 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd6259c640 1 --2- 172.21.15.84:0/3314591102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcf820 0x5573f6dd9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/3314591102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dcef00 0x5573f6dcf2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.260+0000 7fdd537fe640 1 -- 172.21.15.84:0/2903208050 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7fdd5c0cfd00 con 0x7fdd64108950 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.261+0000 7fdd61d9b640 1 -- 172.21.15.84:0/3314591102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dcef00 msgr2=0x5573f6dcf2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.261+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/3314591102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dcef00 0x5573f6dcf2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.261+0000 7fdd61d9b640 1 -- 172.21.15.84:0/3314591102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcf820 msgr2=0x5573f6dd9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.261+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/3314591102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcf820 0x5573f6dd9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.261+0000 7fdd61d9b640 1 -- 172.21.15.84:0/3314591102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573f6cfc0d0 con 0x5573f6dda0e0 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.261+0000 7fdd62d9d640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fdd4c0a3e40 0x7fdd4c0a6300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c09bc60 tx=0x7fdd5c0c6de0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.261+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/3314591102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dda0e0 0x5573f6ddc4d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fdd58012470 tx=0x7fdd5800b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf01b48f35e6c329 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.261+0000 7fdd50ff9640 1 -- 172.21.15.84:0/3314591102 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58010490 con 0x5573f6dda0e0 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.261+0000 7fdd50ff9640 1 -- 172.21.15.84:0/3314591102 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd58010ae0 con 0x5573f6dda0e0 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.261+0000 7fdd50ff9640 1 -- 172.21.15.84:0/3314591102 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800ea20 con 0x5573f6dda0e0 2024-04-26T18:43:46.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 -- 172.21.15.84:0/3314591102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dda0e0 msgr2=0x5573f6ddc4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/3314591102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dda0e0 0x5573f6ddc4d0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fdd58012470 tx=0x7fdd5800b420 comp rx=0 tx=0).stop 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 -- 172.21.15.84:0/3314591102 shutdown_connections 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/3314591102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dda0e0 0x5573f6ddc4d0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/3314591102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcf820 0x5573f6dd9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/3314591102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dcef00 0x5573f6dcf2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 -- 172.21.15.84:0/3314591102 >> 172.21.15.84:0/3314591102 conn(0x5573f6dcb410 msgr2=0x5573f6dcb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 -- 172.21.15.84:0/3314591102 shutdown_connections 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 -- 172.21.15.84:0/3314591102 wait complete. 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 -- start start 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 Processor -- start 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dcef00 0x5573f6dd7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 0x5573f6dd7570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dda0e0 0x5573f6dd7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x5573f6ddcd10 con 0x5573f6dcf820 2024-04-26T18:43:46.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x5573f6cbcb60 con 0x5573f6dda0e0 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x5573f6ddce90 con 0x5573f6dcef00 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6bc64740 10 client.0 _ll_register_callbacks cb 0x5573f6ce12b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 0x5573f6dd7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 0x5573f6dd7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51246/0 (socket says 172.21.15.84:51246) 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.262+0000 7fdd6259c640 1 -- 172.21.15.84:0/1179138082 learned_addr learned my addr 172.21.15.84:0/1179138082 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dda0e0 0x5573f6dd7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dcef00 0x5573f6dd7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd6259c640 1 -- 172.21.15.84:0/1179138082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dcef00 msgr2=0x5573f6dd7030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd6259c640 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dcef00 0x5573f6dd7030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd6259c640 1 -- 172.21.15.84:0/1179138082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dda0e0 msgr2=0x5573f6dd7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd6259c640 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dda0e0 0x5573f6dd7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd6259c640 1 -- 172.21.15.84:0/1179138082 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573f6a47210 con 0x5573f6dcf820 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dda0e0 0x5573f6dd7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:46.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd6259c640 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 0x5573f6dd7570 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009900 tx=0x7fdd540316d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31199b69cbeee04e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd49ffb640 1 -- 172.21.15.84:0/1179138082 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54040070 con 0x5573f6dcf820 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd49ffb640 1 -- 172.21.15.84:0/1179138082 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7fdd54004240 con 0x5573f6dcf820 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1179138082 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd58010ae0 con 0x5573f6dcf820 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd49ffb640 1 -- 172.21.15.84:0/1179138082 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd54035040 con 0x5573f6dcf820 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd6bc64740 10 client.17277 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1179138082 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573f6b13620 con 0x5573f6dcf820 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.263+0000 7fdd49ffb640 1 -- 172.21.15.84:0/1179138082 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd58010ae0 con 0x5573f6dcf820 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.264+0000 7fdd49ffb640 1 -- 172.21.15.84:0/1179138082 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdd540048f0 con 0x5573f6dcf820 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.264+0000 7fdd6bc64740 10 client.17277 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.264+0000 7fdd6bc64740 10 client.17277 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.264+0000 7fdd6bc64740 20 client.17277 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.264+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5573f6dea1d0 0x5573f6dec5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.264+0000 7fdd6bc64740 4 client.17277 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.264+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1179138082 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573f6d43070 con 0x5573f6dea1d0 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.264+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5573f6dea1d0 0x5573f6dec5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.264+0000 7fdd49ffb640 10 client.17277 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.264+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5573f6dea1d0 0x5573f6dec5c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ffbd4dafcb8f0f2 server_cookie=c0916ae713c071ca in_seq=0 out_seq=0 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd49ffb640 1 -- 172.21.15.84:0/1179138082 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5573f6d43070 con 0x5573f6dea1d0 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd49ffb640 10 client.17277 handle_command_reply: tid=0 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd6bc64740 1 client.17277 shutdown 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1179138082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 msgr2=0x5573f6dd7570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 0x5573f6dd7570 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009900 tx=0x7fdd540316d0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1179138082 shutdown_connections 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5573f6dea1d0 0x5573f6dec5c0 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dda0e0 0x5573f6dd7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd49ffb640 0 client.17277 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:46.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 0x5573f6dd7570 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/1179138082 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dcef00 0x5573f6dd7030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1179138082 >> 172.21.15.84:0/1179138082 conn(0x5573f6dcb410 msgr2=0x5573f6dd1140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1179138082 shutdown_connections 2024-04-26T18:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.265+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1179138082 wait complete. 2024-04-26T18:43:46.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.266+0000 7fdd6bc64740 20 client.17277 trim_cache size 0 max 16384 2024-04-26T18:43:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.280+0000 7fdd6bc64740 1 Processor -- start 2024-04-26T18:43:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.280+0000 7fdd6bc64740 1 -- start start 2024-04-26T18:43:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.280+0000 7fdd6bc64740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcef00 0x5573f6dcad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.280+0000 7fdd6bc64740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 0x5573f6dd18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.280+0000 7fdd6bc64740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dda0e0 0x5573f6df1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.280+0000 7fdd6bc64740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641ae6d0 con 0x5573f6dcf820 2024-04-26T18:43:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.280+0000 7fdd6bc64740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641b0210 con 0x5573f6dcef00 2024-04-26T18:43:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.280+0000 7fdd6bc64740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641aeda0 con 0x5573f6dda0e0 2024-04-26T18:43:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.280+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 0x5573f6dd18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.280+0000 7fdd6259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 0x5573f6dd18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51250/0 (socket says 172.21.15.84:51250) 2024-04-26T18:43:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.280+0000 7fdd6259c640 1 -- 172.21.15.84:0/3926483971 learned_addr learned my addr 172.21.15.84:0/3926483971 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6259c640 1 -- 172.21.15.84:0/3926483971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dda0e0 msgr2=0x5573f6df1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6259c640 1 --2- 172.21.15.84:0/3926483971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dda0e0 0x5573f6df1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6259c640 1 -- 172.21.15.84:0/3926483971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcef00 msgr2=0x5573f6dcad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6259c640 1 --2- 172.21.15.84:0/3926483971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcef00 0x5573f6dcad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6259c640 1 -- 172.21.15.84:0/3926483971 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573f6dd1e00 con 0x5573f6dcf820 2024-04-26T18:43:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6259c640 1 --2- 172.21.15.84:0/3926483971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 0x5573f6dd18c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fdd54004eb0 tx=0x7fdd5402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cc4b6a7fa8fde server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd4bfff640 1 -- 172.21.15.84:0/3926483971 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54050040 con 0x5573f6dcf820 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd4bfff640 1 -- 172.21.15.84:0/3926483971 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd54031c50 con 0x5573f6dcf820 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd4bfff640 1 -- 172.21.15.84:0/3926483971 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd54040040 con 0x5573f6dcf820 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6bc64740 1 -- 172.21.15.84:0/3926483971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 msgr2=0x5573f6dd18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/3926483971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 0x5573f6dd18c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fdd54004eb0 tx=0x7fdd5402a420 comp rx=0 tx=0).stop 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6bc64740 1 -- 172.21.15.84:0/3926483971 shutdown_connections 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/3926483971 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dda0e0 0x5573f6df1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/3926483971 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcf820 0x5573f6dd18c0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/3926483971 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcef00 0x5573f6dcad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6bc64740 1 -- 172.21.15.84:0/3926483971 >> 172.21.15.84:0/3926483971 conn(0x5573f6dcb410 msgr2=0x5573f6dd7880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6bc64740 1 -- 172.21.15.84:0/3926483971 shutdown_connections 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.281+0000 7fdd6bc64740 1 -- 172.21.15.84:0/3926483971 wait complete. 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd6bc64740 1 -- start start 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd6bc64740 1 Processor -- start 2024-04-26T18:43:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd6bc64740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcef00 0x5573f6dde310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd6bc64740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcf820 0x5573f6dde850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd6bc64740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dda0e0 0x5573f6e496d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd6bc64740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641afb40 con 0x5573f6dcef00 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd6bc64740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641af470 con 0x5573f6dcf820 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd6bc64740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fdd641ac6d0 con 0x5573f6dda0e0 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd6bc64740 10 client.0 _ll_register_callbacks cb 0x7fdd54032da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd61d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcef00 0x5573f6dde310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd61d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcef00 0x5573f6dde310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:51260/0 (socket says 172.21.15.84:51260) 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd61d9b640 1 -- 172.21.15.84:0/1480539575 learned_addr learned my addr 172.21.15.84:0/1480539575 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd6259c640 1 --2- 172.21.15.84:0/1480539575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcf820 0x5573f6dde850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd61d9b640 1 -- 172.21.15.84:0/1480539575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dda0e0 msgr2=0x5573f6e496d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/1480539575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dda0e0 0x5573f6e496d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd61d9b640 1 -- 172.21.15.84:0/1480539575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcf820 msgr2=0x5573f6dde850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/1480539575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcf820 0x5573f6dde850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd61d9b640 1 -- 172.21.15.84:0/1480539575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573f6e49c90 con 0x5573f6dcef00 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.282+0000 7fdd6259c640 1 --2- 172.21.15.84:0/1480539575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcf820 0x5573f6dde850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd61d9b640 1 --2- 172.21.15.84:0/1480539575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcef00 0x5573f6dde310 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x5573f6dcc5a0 tx=0x7fdd5800b5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14564a1388d9baa6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd49ffb640 1 -- 172.21.15.84:0/1480539575 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58005030 con 0x5573f6dcef00 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd49ffb640 1 -- 172.21.15.84:0/1480539575 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7fdd580051d0 con 0x5573f6dcef00 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1480539575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd54031c50 con 0x5573f6dcef00 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd49ffb640 1 -- 172.21.15.84:0/1480539575 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdd580053d0 con 0x5573f6dcef00 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd6bc64740 10 client.17283 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1480539575 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5573f6e51030 con 0x5573f6dcef00 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd49ffb640 1 -- 172.21.15.84:0/1480539575 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd54031c50 con 0x5573f6dcef00 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd49ffb640 1 -- 172.21.15.84:0/1480539575 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fdd5800f040 con 0x5573f6dcef00 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd6bc64740 10 client.17283 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd6bc64740 10 client.17283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:43:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd6bc64740 20 client.17283 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/1480539575 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5573f6e515f0 0x5573f6e53a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd6bc64740 4 client.17283 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:43:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.283+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1480539575 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5573f6d3cee0 con 0x5573f6e515f0 2024-04-26T18:43:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.284+0000 7fdd6259c640 1 --2- 172.21.15.84:0/1480539575 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5573f6e515f0 0x5573f6e53a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.284+0000 7fdd49ffb640 10 client.17283 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.284+0000 7fdd6259c640 1 --2- 172.21.15.84:0/1480539575 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5573f6e515f0 0x5573f6e53a10 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c48247eff87bf56f server_cookie=4b0b0c61d8890de7 in_seq=0 out_seq=0 2024-04-26T18:43:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.295+0000 7fdd49ffb640 1 -- 172.21.15.84:0/1480539575 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5573f6d3cee0 con 0x5573f6e515f0 2024-04-26T18:43:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.295+0000 7fdd49ffb640 10 client.17283 handle_command_reply: tid=0 2024-04-26T18:43:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.295+0000 7fdd6bc64740 1 client.17283 shutdown 2024-04-26T18:43:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.295+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1480539575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcef00 msgr2=0x5573f6dde310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.295+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/1480539575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcef00 0x5573f6dde310 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x5573f6dcc5a0 tx=0x7fdd5800b5a0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.295+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1480539575 shutdown_connections 2024-04-26T18:43:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.295+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/1480539575 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x5573f6e515f0 0x5573f6e53a10 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.295+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/1480539575 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x5573f6dda0e0 0x5573f6e496d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.295+0000 7fdd49ffb640 0 client.17283 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/1480539575 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x5573f6dcf820 0x5573f6dde850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd6bc64740 1 --2- 172.21.15.84:0/1480539575 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x5573f6dcef00 0x5573f6dde310 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1480539575 >> 172.21.15.84:0/1480539575 conn(0x5573f6dcb410 msgr2=0x5573f6dd8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1480539575 shutdown_connections 2024-04-26T18:43:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd6bc64740 1 -- 172.21.15.84:0/1480539575 wait complete. 2024-04-26T18:43:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd6bc64740 20 client.17283 trim_cache size 0 max 16384 2024-04-26T18:43:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:43:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:43:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:43:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:43:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd68f46640 1 -- 172.21.15.84:0/2903208050 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fdd4c0a3e40 msgr2=0x7fdd4c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd68f46640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fdd4c0a3e40 0x7fdd4c0a6300 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c09bc60 tx=0x7fdd5c0c6de0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd68f46640 1 -- 172.21.15.84:0/2903208050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd64108950 msgr2=0x7fdd6419d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd68f46640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd64108950 0x7fdd6419d9d0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00b9e0 tx=0x7fdd5c00bea0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd68f46640 1 -- 172.21.15.84:0/2903208050 shutdown_connections 2024-04-26T18:43:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd68f46640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fdd4c0a3e40 0x7fdd4c0a6300 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd68f46640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fdd64108950 0x7fdd6419d9d0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd68f46640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fdd64103300 0x7fdd64199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd68f46640 1 --2- 172.21.15.84:0/2903208050 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fdd64102940 0x7fdd64199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd68f46640 1 -- 172.21.15.84:0/2903208050 >> 172.21.15.84:0/2903208050 conn(0x7fdd64079dc0 msgr2=0x7fdd64100280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd68f46640 1 -- 172.21.15.84:0/2903208050 shutdown_connections 2024-04-26T18:43:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:46.296+0000 7fdd68f46640 1 -- 172.21.15.84:0/2903208050 wait complete. 2024-04-26T18:43:46.519 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id 835e6b7c --release --await' 2024-04-26T18:43:46.519 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db --set-id 835e6b7c --release --await 2024-04-26T18:43:46.755 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:43:47.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:46 smithi084 ceph-mon[29141]: pgmap v497: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-26T18:43:47.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:46 smithi103 ceph-mon[32727]: pgmap v497: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-26T18:43:47.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:46 smithi174 ceph-mon[32705]: pgmap v497: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-26T18:43:48.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.471+0000 7ff2f9ead700 1 -- 172.21.15.84:0/1925987030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f406b9c0 msgr2=0x7ff2f406be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:48.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.471+0000 7ff2f9ead700 1 --2- 172.21.15.84:0/1925987030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f406b9c0 0x7ff2f406be20 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009990 tx=0x7ff2e8009ca0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.473 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.471+0000 7ff2f9ead700 1 -- 172.21.15.84:0/1925987030 shutdown_connections 2024-04-26T18:43:48.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.471+0000 7ff2f9ead700 1 --2- 172.21.15.84:0/1925987030 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f406c360 0x7ff2f41102d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.471+0000 7ff2f9ead700 1 --2- 172.21.15.84:0/1925987030 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f4109490 0x7ff2f4109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.471+0000 7ff2f9ead700 1 --2- 172.21.15.84:0/1925987030 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f406b9c0 0x7ff2f406be20 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.471+0000 7ff2f9ead700 1 -- 172.21.15.84:0/1925987030 >> 172.21.15.84:0/1925987030 conn(0x7ff2f4079290 msgr2=0x7ff2f40796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:48.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.471+0000 7ff2f9ead700 1 -- 172.21.15.84:0/1925987030 shutdown_connections 2024-04-26T18:43:48.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.471+0000 7ff2f9ead700 1 -- 172.21.15.84:0/1925987030 wait complete. 2024-04-26T18:43:48.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.472+0000 7ff2f9ead700 1 Processor -- start 2024-04-26T18:43:48.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.472+0000 7ff2f9ead700 1 -- start start 2024-04-26T18:43:48.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.472+0000 7ff2f9ead700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f406b9c0 0x7ff2f419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.472+0000 7ff2f9ead700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f406c360 0x7ff2f419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.472+0000 7ff2f9ead700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4109490 0x7ff2f4199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.472+0000 7ff2f9ead700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41135a0 con 0x7ff2f406c360 2024-04-26T18:43:48.474 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.472+0000 7ff2f9ead700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4113420 con 0x7ff2f4109490 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.472+0000 7ff2f9ead700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f4113720 con 0x7ff2f406b9c0 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.472+0000 7ff2f2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f406c360 0x7ff2f419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.472+0000 7ff2f2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f406c360 0x7ff2f419fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34636/0 (socket says 172.21.15.84:34636) 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.472+0000 7ff2f2ffd700 1 -- 172.21.15.84:0/2174381523 learned_addr learned my addr 172.21.15.84:0/2174381523 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.472+0000 7ff2f3fff700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4109490 0x7ff2f4199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.472+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f406b9c0 0x7ff2f419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.473+0000 7ff2f2ffd700 1 -- 172.21.15.84:0/2174381523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f406b9c0 msgr2=0x7ff2f419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.473+0000 7ff2f2ffd700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f406b9c0 0x7ff2f419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.473+0000 7ff2f2ffd700 1 -- 172.21.15.84:0/2174381523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4109490 msgr2=0x7ff2f4199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.473+0000 7ff2f2ffd700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4109490 0x7ff2f4199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.473+0000 7ff2f2ffd700 1 -- 172.21.15.84:0/2174381523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f419a060 con 0x7ff2f406c360 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.473+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f406b9c0 0x7ff2f419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.473+0000 7ff2f3fff700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4109490 0x7ff2f4199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.473+0000 7ff2f2ffd700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f406c360 0x7ff2f419fc80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8000c00 tx=0x7ff2e80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:48.475 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.473+0000 7ff2f0ff9700 1 -- 172.21.15.84:0/2174381523 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8027070 con 0x7ff2f406c360 2024-04-26T18:43:48.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.473+0000 7ff2f9ead700 1 -- 172.21.15.84:0/2174381523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f419a2f0 con 0x7ff2f406c360 2024-04-26T18:43:48.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.473+0000 7ff2f0ff9700 1 -- 172.21.15.84:0/2174381523 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e8003fa0 con 0x7ff2f406c360 2024-04-26T18:43:48.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.473+0000 7ff2f0ff9700 1 -- 172.21.15.84:0/2174381523 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8022db0 con 0x7ff2f406c360 2024-04-26T18:43:48.476 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.473+0000 7ff2f9ead700 1 -- 172.21.15.84:0/2174381523 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f4102310 con 0x7ff2f406c360 2024-04-26T18:43:48.481 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.475+0000 7ff2f0ff9700 1 -- 172.21.15.84:0/2174381523 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff2e802a070 con 0x7ff2f406c360 2024-04-26T18:43:48.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.476+0000 7ff2f0ff9700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff2dc0a3db0 0x7ff2dc0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.476+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff2dc0a3db0 0x7ff2dc0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.476+0000 7ff2f0ff9700 1 -- 172.21.15.84:0/2174381523 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7ff2e80d1580 con 0x7ff2f406c360 2024-04-26T18:43:48.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.476+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff2dc0a3db0 0x7ff2dc0a6270 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005e20 tx=0x7ff2e4005db0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:48.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff30473fb80 1 Processor -- start 2024-04-26T18:43:48.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff30473fb80 1 -- start start 2024-04-26T18:43:48.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff30473fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d90ea0 0x562429d91280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff30473fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d917c0 0x562429d8b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff30473fb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 0x562429d8bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff30473fb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562429cdd930 con 0x562429d8b870 2024-04-26T18:43:48.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff30473fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562429c43a30 con 0x562429d90ea0 2024-04-26T18:43:48.482 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff30473fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562429cd98f0 con 0x562429d917c0 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 0x562429d8bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff2f3fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 0x562429d8bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34646/0 (socket says 172.21.15.84:34646) 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff2f3fff700 1 -- 172.21.15.84:0/2643739907 learned_addr learned my addr 172.21.15.84:0/2643739907 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff2f2ffd700 1 --2- 172.21.15.84:0/2643739907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d917c0 0x562429d8b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff2f3fff700 1 -- 172.21.15.84:0/2643739907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d917c0 msgr2=0x562429d8b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/2643739907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d90ea0 0x562429d91280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff2f3fff700 1 --2- 172.21.15.84:0/2643739907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d917c0 0x562429d8b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff2f3fff700 1 -- 172.21.15.84:0/2643739907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d90ea0 msgr2=0x562429d91280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff2f3fff700 1 --2- 172.21.15.84:0/2643739907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d90ea0 0x562429d91280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff2f3fff700 1 -- 172.21.15.84:0/2643739907 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562429c2a9d0 con 0x562429d8b870 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff2f2ffd700 1 --2- 172.21.15.84:0/2643739907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d917c0 0x562429d8b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.477+0000 7ff2f3fff700 1 --2- 172.21.15.84:0/2643739907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 0x562429d8bcd0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009e60 tx=0x7ff2ec0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7838520b949748 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.478+0000 7ff2d9ffb700 1 -- 172.21.15.84:0/2643739907 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec014610 con 0x562429d8b870 2024-04-26T18:43:48.483 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.478+0000 7ff2d9ffb700 1 -- 172.21.15.84:0/2643739907 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2ec019070 con 0x562429d8b870 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.478+0000 7ff2d9ffb700 1 -- 172.21.15.84:0/2643739907 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec013a90 con 0x562429d8b870 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.478+0000 7ff30473fb80 1 -- 172.21.15.84:0/2643739907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 msgr2=0x562429d8bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.478+0000 7ff30473fb80 1 --2- 172.21.15.84:0/2643739907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 0x562429d8bcd0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009e60 tx=0x7ff2ec0077e0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.478+0000 7ff30473fb80 1 -- 172.21.15.84:0/2643739907 shutdown_connections 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.478+0000 7ff30473fb80 1 --2- 172.21.15.84:0/2643739907 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 0x562429d8bcd0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.478+0000 7ff30473fb80 1 --2- 172.21.15.84:0/2643739907 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d90ea0 0x562429d91280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.478+0000 7ff30473fb80 1 --2- 172.21.15.84:0/2643739907 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d917c0 0x562429d8b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.478+0000 7ff30473fb80 1 -- 172.21.15.84:0/2643739907 >> 172.21.15.84:0/2643739907 conn(0x562429cb96c0 msgr2=0x562429d86000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.478+0000 7ff30473fb80 1 -- 172.21.15.84:0/2643739907 shutdown_connections 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.478+0000 7ff30473fb80 1 -- 172.21.15.84:0/2643739907 wait complete. 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.479+0000 7ff30473fb80 1 -- start start 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.479+0000 7ff30473fb80 1 Processor -- start 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.479+0000 7ff30473fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d8b870 0x562429d97460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.484 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.479+0000 7ff30473fb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 0x562429d94800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.479+0000 7ff30473fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d94d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.479+0000 7ff30473fb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562429d98230 con 0x562429d90ea0 2024-04-26T18:43:48.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.479+0000 7ff30473fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562429c97b80 con 0x562429d917c0 2024-04-26T18:43:48.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.479+0000 7ff30473fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562429d983b0 con 0x562429d8b870 2024-04-26T18:43:48.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.479+0000 7ff30473fb80 10 client.0 _ll_register_callbacks cb 0x562429c75a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:48.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.479+0000 7ff2f2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 0x562429d94800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.479+0000 7ff2f2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 0x562429d94800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34656/0 (socket says 172.21.15.84:34656) 2024-04-26T18:43:48.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.479+0000 7ff2f2ffd700 1 -- 172.21.15.84:0/1396138539 learned_addr learned my addr 172.21.15.84:0/1396138539 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:48.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d8b870 0x562429d97460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2f3fff700 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d94d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2f2ffd700 1 -- 172.21.15.84:0/1396138539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d8b870 msgr2=0x562429d97460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:48.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2f2ffd700 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d8b870 0x562429d97460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.485 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2f2ffd700 1 -- 172.21.15.84:0/1396138539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 msgr2=0x562429d94d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:48.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2f2ffd700 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d94d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2f2ffd700 1 -- 172.21.15.84:0/1396138539 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562429c9b580 con 0x562429d90ea0 2024-04-26T18:43:48.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2f3fff700 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d94d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:43:48.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2f2ffd700 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 0x562429d94800 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff2e809d270 tx=0x7ff2e809d570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1aabcfafa3973b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:48.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:48.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2d2ffd700 1 -- 172.21.15.84:0/1396138539 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80a6150 con 0x562429d90ea0 2024-04-26T18:43:48.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2d2ffd700 1 -- 172.21.15.84:0/1396138539 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7ff2e809d9e0 con 0x562429d90ea0 2024-04-26T18:43:48.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff30473fb80 1 -- 172.21.15.84:0/1396138539 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2ec019070 con 0x562429d90ea0 2024-04-26T18:43:48.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2d2ffd700 1 -- 172.21.15.84:0/1396138539 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2e80a5cc0 con 0x562429d90ea0 2024-04-26T18:43:48.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff30473fb80 10 client.17295 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:48.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff30473fb80 1 -- 172.21.15.84:0/1396138539 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562429da2960 con 0x562429d90ea0 2024-04-26T18:43:48.486 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.480+0000 7ff2d2ffd700 1 -- 172.21.15.84:0/1396138539 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2ec019070 con 0x562429d90ea0 2024-04-26T18:43:48.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.481+0000 7ff2d2ffd700 1 -- 172.21.15.84:0/1396138539 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff2e80bc2b0 con 0x562429d90ea0 2024-04-26T18:43:48.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.481+0000 7ff30473fb80 10 client.17295 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:48.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.481+0000 7ff30473fb80 10 client.17295 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:43:48.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.481+0000 7ff30473fb80 20 client.17295 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:48.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.481+0000 7ff30473fb80 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562429da4f60 0x562429da7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.481+0000 7ff30473fb80 4 client.17295 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:43:48.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.481+0000 7ff30473fb80 1 -- 172.21.15.84:0/1396138539 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562429ceb310 con 0x562429da4f60 2024-04-26T18:43:48.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.481+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562429da4f60 0x562429da7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.481+0000 7ff2d2ffd700 10 client.17295 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:48.487 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.481+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562429da4f60 0x562429da7380 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41fc3c1f25b21f6 server_cookie=1ce0a9304f8518cb in_seq=0 out_seq=0 2024-04-26T18:43:48.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff2d2ffd700 1 -- 172.21.15.84:0/1396138539 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562429ceb310 con 0x562429da4f60 2024-04-26T18:43:48.488 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff2d2ffd700 10 client.17295 handle_command_reply: tid=0 2024-04-26T18:43:48.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff30473fb80 1 client.17295 shutdown 2024-04-26T18:43:48.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff30473fb80 1 -- 172.21.15.84:0/1396138539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 msgr2=0x562429d94800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:48.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff30473fb80 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 0x562429d94800 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7ff2e809d270 tx=0x7ff2e809d570 comp rx=0 tx=0).stop 2024-04-26T18:43:48.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff30473fb80 1 -- 172.21.15.84:0/1396138539 shutdown_connections 2024-04-26T18:43:48.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff30473fb80 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562429da4f60 0x562429da7380 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff30473fb80 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d94d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff2d2ffd700 0 client.17295 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:48.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff30473fb80 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d8b870 0x562429d97460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff30473fb80 1 --2- 172.21.15.84:0/1396138539 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 0x562429d94800 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff30473fb80 1 -- 172.21.15.84:0/1396138539 >> 172.21.15.84:0/1396138539 conn(0x562429cb96c0 msgr2=0x562429d8a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:48.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff30473fb80 1 -- 172.21.15.84:0/1396138539 shutdown_connections 2024-04-26T18:43:48.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.482+0000 7ff30473fb80 1 -- 172.21.15.84:0/1396138539 wait complete. 2024-04-26T18:43:48.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.483+0000 7ff30473fb80 20 client.17295 trim_cache size 0 max 16384 2024-04-26T18:43:48.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.501+0000 7ff30473fb80 1 Processor -- start 2024-04-26T18:43:48.501 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.501+0000 7ff30473fb80 1 -- start start 2024-04-26T18:43:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff30473fb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 0x562429d8b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff30473fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d90ea0 0x562429da1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff30473fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d7fd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff30473fb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41afe40 con 0x562429d8b870 2024-04-26T18:43:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff30473fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b1980 con 0x562429d917c0 2024-04-26T18:43:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff30473fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b0510 con 0x562429d90ea0 2024-04-26T18:43:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff2f37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 0x562429d8b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff2f37fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 0x562429d8b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34672/0 (socket says 172.21.15.84:34672) 2024-04-26T18:43:48.506 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff2f37fe700 1 -- 172.21.15.84:0/1791844565 learned_addr learned my addr 172.21.15.84:0/1791844565 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff2f3fff700 1 --2- 172.21.15.84:0/1791844565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d7fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff2f37fe700 1 -- 172.21.15.84:0/1791844565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d90ea0 msgr2=0x562429da1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff2f2ffd700 1 --2- 172.21.15.84:0/1791844565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d90ea0 0x562429da1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/1791844565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d90ea0 0x562429da1130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff2f37fe700 1 -- 172.21.15.84:0/1791844565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 msgr2=0x562429d7fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/1791844565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d7fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff2f37fe700 1 -- 172.21.15.84:0/1791844565 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562429d827f0 con 0x562429d8b870 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff2f3fff700 1 --2- 172.21.15.84:0/1791844565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d7fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.502+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/1791844565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 0x562429d8b250 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005950 tx=0x7ff2e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2fabc496ad7830f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.503+0000 7ff2d97fa700 1 -- 172.21.15.84:0/1791844565 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e402c070 con 0x562429d8b870 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.503+0000 7ff2d97fa700 1 -- 172.21.15.84:0/1791844565 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e4027850 con 0x562429d8b870 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.503+0000 7ff2d97fa700 1 -- 172.21.15.84:0/1791844565 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e4010ad0 con 0x562429d8b870 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.503+0000 7ff30473fb80 1 -- 172.21.15.84:0/1791844565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 msgr2=0x562429d8b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:48.507 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.503+0000 7ff30473fb80 1 --2- 172.21.15.84:0/1791844565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 0x562429d8b250 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005950 tx=0x7ff2e4014000 comp rx=0 tx=0).stop 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.503+0000 7ff30473fb80 1 -- 172.21.15.84:0/1791844565 shutdown_connections 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.503+0000 7ff30473fb80 1 --2- 172.21.15.84:0/1791844565 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d8b870 0x562429d8b250 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.503+0000 7ff30473fb80 1 --2- 172.21.15.84:0/1791844565 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d7fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.503+0000 7ff30473fb80 1 --2- 172.21.15.84:0/1791844565 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d90ea0 0x562429da1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.503+0000 7ff30473fb80 1 -- 172.21.15.84:0/1791844565 >> 172.21.15.84:0/1791844565 conn(0x562429cb96c0 msgr2=0x562429d7f670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.503+0000 7ff30473fb80 1 -- 172.21.15.84:0/1791844565 shutdown_connections 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.503+0000 7ff30473fb80 1 -- 172.21.15.84:0/1791844565 wait complete. 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.504+0000 7ff30473fb80 1 -- start start 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.504+0000 7ff30473fb80 1 Processor -- start 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.504+0000 7ff30473fb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d8b870 0x562429d9a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.504+0000 7ff30473fb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 0x562429d9ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.504+0000 7ff30473fb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d9b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.504+0000 7ff30473fb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b12b0 con 0x562429d90ea0 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.504+0000 7ff30473fb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41b0be0 con 0x562429d917c0 2024-04-26T18:43:48.508 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.504+0000 7ff30473fb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f41ade00 con 0x562429d8b870 2024-04-26T18:43:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.504+0000 7ff30473fb80 10 client.0 _ll_register_callbacks cb 0x562429c75a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.504+0000 7ff2f2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 0x562429d9ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2f2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 0x562429d9ab50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:34682/0 (socket says 172.21.15.84:34682) 2024-04-26T18:43:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2f2ffd700 1 -- 172.21.15.84:0/605095723 learned_addr learned my addr 172.21.15.84:0/605095723 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d8b870 0x562429d9a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2f3fff700 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d9b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2f2ffd700 1 -- 172.21.15.84:0/605095723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d8b870 msgr2=0x562429d9a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2f2ffd700 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d8b870 0x562429d9a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2f2ffd700 1 -- 172.21.15.84:0/605095723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 msgr2=0x562429d9b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2f2ffd700 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d9b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2f2ffd700 1 -- 172.21.15.84:0/605095723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562429d7e230 con 0x562429d90ea0 2024-04-26T18:43:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2f3fff700 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d9b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:48.509 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2f2ffd700 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 0x562429d9ab50 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff2e809b820 tx=0x7ff2e8022880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820472ee5c1b53b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2d2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2d2ffd700 1 -- 172.21.15.84:0/605095723 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e80a6150 con 0x562429d90ea0 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2d2ffd700 1 -- 172.21.15.84:0/605095723 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7ff2e80a5e60 con 0x562429d90ea0 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff30473fb80 1 -- 172.21.15.84:0/605095723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2e4027850 con 0x562429d90ea0 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff2d2ffd700 1 -- 172.21.15.84:0/605095723 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff2e80bc2d0 con 0x562429d90ea0 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff30473fb80 10 client.17301 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.505+0000 7ff30473fb80 1 -- 172.21.15.84:0/605095723 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562429e0c700 con 0x562429d90ea0 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.506+0000 7ff2d2ffd700 1 -- 172.21.15.84:0/605095723 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e4027850 con 0x562429d90ea0 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.506+0000 7ff2d2ffd700 1 -- 172.21.15.84:0/605095723 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff2e80d1040 con 0x562429d90ea0 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.506+0000 7ff30473fb80 10 client.17301 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.506+0000 7ff30473fb80 10 client.17301 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.506+0000 7ff30473fb80 20 client.17301 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.506+0000 7ff30473fb80 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562429e0ccd0 0x562429e0f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:48.510 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.506+0000 7ff30473fb80 4 client.17301 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "835e6b7c", "release": true, "await": true}] 2024-04-26T18:43:48.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.506+0000 7ff30473fb80 1 -- 172.21.15.84:0/605095723 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "set_id": "835e6b7c", "release": true, "await": true}) -- 0x562429ce53b0 con 0x562429e0ccd0 2024-04-26T18:43:48.511 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.506+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562429e0ccd0 0x562429e0f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:48.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.507+0000 7ff2f37fe700 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562429e0ccd0 0x562429e0f0f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89c7f737a751356a server_cookie=53f5459bc7864fd0 in_seq=0 out_seq=0 2024-04-26T18:43:48.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.507+0000 7ff2d2ffd700 10 client.17301 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:48.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.510+0000 7ff2d2ffd700 1 -- 172.21.15.84:0/605095723 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x562429ce53b0 con 0x562429e0ccd0 2024-04-26T18:43:48.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.510+0000 7ff2d2ffd700 10 client.17301 handle_command_reply: tid=0 2024-04-26T18:43:48.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.510+0000 7ff30473fb80 1 client.17301 shutdown 2024-04-26T18:43:48.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff30473fb80 1 -- 172.21.15.84:0/605095723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 msgr2=0x562429d9ab50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:48.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff30473fb80 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 0x562429d9ab50 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7ff2e809b820 tx=0x7ff2e8022880 comp rx=0 tx=0).stop 2024-04-26T18:43:48.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff30473fb80 1 -- 172.21.15.84:0/605095723 shutdown_connections 2024-04-26T18:43:48.512 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff30473fb80 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562429e0ccd0 0x562429e0f0f0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff30473fb80 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562429d917c0 0x562429d9b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff30473fb80 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562429d8b870 0x562429d9a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff30473fb80 1 --2- 172.21.15.84:0/605095723 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562429d90ea0 0x562429d9ab50 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff30473fb80 1 -- 172.21.15.84:0/605095723 >> 172.21.15.84:0/605095723 conn(0x562429cb96c0 msgr2=0x562429d95e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:48.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff2d2ffd700 0 client.17301 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:48.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff30473fb80 1 -- 172.21.15.84:0/605095723 shutdown_connections 2024-04-26T18:43:48.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff30473fb80 1 -- 172.21.15.84:0/605095723 wait complete. 2024-04-26T18:43:48.513 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff30473fb80 20 client.17301 trim_cache size 0 max 16384 2024-04-26T18:43:48.513 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:43:48.513 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:43:48.513 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 44, 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: "835e6b7c": { 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 44, 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 70.0, 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:43:48.514 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:43:48.515 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:43:48.515 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:43:48.515 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:43:48.515 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:43:48.515 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:43:48.515 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:43:48.515 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:43:48.515 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:43:48.515 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:43:48.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff2f9ead700 1 -- 172.21.15.84:0/2174381523 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff2dc0a3db0 msgr2=0x7ff2dc0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:48.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff2f9ead700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff2dc0a3db0 0x7ff2dc0a6270 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff2e4005e20 tx=0x7ff2e4005db0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff2f9ead700 1 -- 172.21.15.84:0/2174381523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f406c360 msgr2=0x7ff2f419fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:48.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff2f9ead700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f406c360 0x7ff2f419fc80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8000c00 tx=0x7ff2e80047b0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.511+0000 7ff2f9ead700 1 -- 172.21.15.84:0/2174381523 shutdown_connections 2024-04-26T18:43:48.516 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.512+0000 7ff2f9ead700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff2dc0a3db0 0x7ff2dc0a6270 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.512+0000 7ff2f9ead700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff2f4109490 0x7ff2f4199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.512+0000 7ff2f9ead700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff2f406b9c0 0x7ff2f419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.512+0000 7ff2f9ead700 1 --2- 172.21.15.84:0/2174381523 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff2f406c360 0x7ff2f419fc80 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:48.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.512+0000 7ff2f9ead700 1 -- 172.21.15.84:0/2174381523 >> 172.21.15.84:0/2174381523 conn(0x7ff2f4079290 msgr2=0x7ff2f4100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:48.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.512+0000 7ff2f9ead700 1 -- 172.21.15.84:0/2174381523 shutdown_connections 2024-04-26T18:43:48.517 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:43:48.512+0000 7ff2f9ead700 1 -- 172.21.15.84:0/2174381523 wait complete. 2024-04-26T18:43:49.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:48 smithi084 ceph-mon[29141]: pgmap v498: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-26T18:43:49.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:48 smithi103 ceph-mon[32727]: pgmap v498: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-26T18:43:49.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:48 smithi174 ceph-mon[32705]: pgmap v498: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 2.1 KiB/s wr, 8 op/s 2024-04-26T18:43:49.271 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '835e6b7c', seconds elapsed: 21.6 2024-04-26T18:43:51.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:50 smithi084 ceph-mon[29141]: pgmap v499: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-26T18:43:51.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:50 smithi103 ceph-mon[32727]: pgmap v499: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-26T18:43:51.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:50 smithi174 ceph-mon[32705]: pgmap v499: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-04-26T18:43:53.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:52 smithi084 ceph-mon[29141]: pgmap v500: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-26T18:43:53.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:52 smithi103 ceph-mon[32727]: pgmap v500: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-26T18:43:53.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:52 smithi174 ceph-mon[32705]: pgmap v500: 97 pgs: 97 active+clean; 2.0 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-04-26T18:43:55.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:54 smithi084 ceph-mon[29141]: pgmap v501: 97 pgs: 97 active+clean; 2.2 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-04-26T18:43:55.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:54 smithi103 ceph-mon[32727]: pgmap v501: 97 pgs: 97 active+clean; 2.2 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-04-26T18:43:55.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:54 smithi174 ceph-mon[32705]: pgmap v501: 97 pgs: 97 active+clean; 2.2 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 KiB/s wr, 17 op/s 2024-04-26T18:43:57.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:56 smithi084 ceph-mon[29141]: pgmap v502: 97 pgs: 97 active+clean; 2.2 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-26T18:43:57.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:56 smithi103 ceph-mon[32727]: pgmap v502: 97 pgs: 97 active+clean; 2.2 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-26T18:43:57.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:56 smithi174 ceph-mon[32705]: pgmap v502: 97 pgs: 97 active+clean; 2.2 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-26T18:43:59.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:43:58 smithi084 ceph-mon[29141]: pgmap v503: 97 pgs: 97 active+clean; 2.2 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-26T18:43:59.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:43:58 smithi103 ceph-mon[32727]: pgmap v503: 97 pgs: 97 active+clean; 2.2 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-26T18:43:59.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:43:58 smithi174 ceph-mon[32705]: pgmap v503: 97 pgs: 97 active+clean; 2.2 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-26T18:43:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.391+0000 7ff9a6ff9640 1 -- 172.21.15.84:0/2435703019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a01044e0 msgr2=0x7ff9a010c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.391+0000 7ff9a6ff9640 1 --2- 172.21.15.84:0/2435703019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a01044e0 0x7ff9a010c9c0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7ff990009ca0 tx=0x7ff99002a9e0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 -- 172.21.15.84:0/2435703019 shutdown_connections 2024-04-26T18:43:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 --2- 172.21.15.84:0/2435703019 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a010cf00 0x7ff9a010f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 --2- 172.21.15.84:0/2435703019 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a01044e0 0x7ff9a010c9c0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 --2- 172.21.15.84:0/2435703019 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9a0103bc0 0x7ff9a0103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 -- 172.21.15.84:0/2435703019 >> 172.21.15.84:0/2435703019 conn(0x7ff9a00789b0 msgr2=0x7ff9a0078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:59.394 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 -- 172.21.15.84:0/2435703019 shutdown_connections 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 -- 172.21.15.84:0/2435703019 wait complete. 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 Processor -- start 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 -- start start 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9a0103bc0 0x7ff9a0198fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a01044e0 0x7ff9a0199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a010cf00 0x7ff9a019d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0112bb0 con 0x7ff9a010cf00 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0112a30 con 0x7ff9a0103bc0 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.392+0000 7ff9a6ff9640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a0112d30 con 0x7ff9a01044e0 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff9a5596640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a010cf00 0x7ff9a019d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff9a5596640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a010cf00 0x7ff9a019d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42748/0 (socket says 172.21.15.84:42748) 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff9a5596640 1 -- 172.21.15.84:0/378159355 learned_addr learned my addr 172.21.15.84:0/378159355 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff99ffff640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a01044e0 0x7ff9a0199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff9a4d95640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9a0103bc0 0x7ff9a0198fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.395 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff9a5596640 1 -- 172.21.15.84:0/378159355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a01044e0 msgr2=0x7ff9a0199510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff9a5596640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a01044e0 0x7ff9a0199510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff9a5596640 1 -- 172.21.15.84:0/378159355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9a0103bc0 msgr2=0x7ff9a0198fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff9a5596640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9a0103bc0 0x7ff9a0198fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff9a5596640 1 -- 172.21.15.84:0/378159355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9a019dfc0 con 0x7ff9a010cf00 2024-04-26T18:43:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff99ffff640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a01044e0 0x7ff9a0199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff9a5596640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a010cf00 0x7ff9a019d890 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c970 tx=0x7ff98800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff99dffb640 1 -- 172.21.15.84:0/378159355 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988007bb0 con 0x7ff9a010cf00 2024-04-26T18:43:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff99dffb640 1 -- 172.21.15.84:0/378159355 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff988007d50 con 0x7ff9a010cf00 2024-04-26T18:43:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff9a6ff9640 1 -- 172.21.15.84:0/378159355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9a019e2b0 con 0x7ff9a010cf00 2024-04-26T18:43:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.393+0000 7ff99dffb640 1 -- 172.21.15.84:0/378159355 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff988018770 con 0x7ff9a010cf00 2024-04-26T18:43:59.396 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.394+0000 7ff9a6ff9640 1 -- 172.21.15.84:0/378159355 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9a01a5b40 con 0x7ff9a010cf00 2024-04-26T18:43:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.396+0000 7ff99dffb640 1 -- 172.21.15.84:0/378159355 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff98800f990 con 0x7ff9a010cf00 2024-04-26T18:43:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.396+0000 7ff9a9dea740 1 Processor -- start 2024-04-26T18:43:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.396+0000 7ff9a9dea740 1 -- start start 2024-04-26T18:43:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.396+0000 7ff99dffb640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff98c0a3dc0 0x7ff98c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.396+0000 7ff9a9dea740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 0x55f4ae0051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.396+0000 7ff9a9dea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 0x55f4ae013fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.403 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.396+0000 7ff9a9dea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae0144f0 0x55f4ae0168e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.396+0000 7ff9a9dea740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f4adf6b460 con 0x55f4ae004e00 2024-04-26T18:43:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.396+0000 7ff9a9dea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f4adf25590 con 0x55f4ae005720 2024-04-26T18:43:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.396+0000 7ff9a9dea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4adf67420 con 0x55f4ae0144f0 2024-04-26T18:43:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.396+0000 7ff99ffff640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff98c0a3dc0 0x7ff98c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.396+0000 7ff9a4d95640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 0x55f4ae0051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.396+0000 7ff9a4d95640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 0x55f4ae0051e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42754/0 (socket says 172.21.15.84:42754) 2024-04-26T18:43:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff9a4d95640 1 -- 172.21.15.84:0/3139562456 learned_addr learned my addr 172.21.15.84:0/3139562456 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff99dffb640 1 -- 172.21.15.84:0/378159355 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7ff9880d0980 con 0x7ff9a010cf00 2024-04-26T18:43:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff99ffff640 1 --2- 172.21.15.84:0/3139562456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 0x55f4ae013fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff9a5596640 1 --2- 172.21.15.84:0/3139562456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae0144f0 0x55f4ae0168e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff9a4d95640 1 -- 172.21.15.84:0/3139562456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae0144f0 msgr2=0x55f4ae0168e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff9a4d95640 1 --2- 172.21.15.84:0/3139562456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae0144f0 0x55f4ae0168e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.404 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff9a4d95640 1 -- 172.21.15.84:0/3139562456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 msgr2=0x55f4ae013fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff9a4d95640 1 --2- 172.21.15.84:0/3139562456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 0x55f4ae013fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff9a4d95640 1 -- 172.21.15.84:0/3139562456 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4adf320d0 con 0x55f4ae004e00 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff9a5596640 1 --2- 172.21.15.84:0/3139562456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae0144f0 0x55f4ae0168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff99ffff640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff98c0a3dc0 0x7ff98c0a6280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff990009880 tx=0x7ff99002edc0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff9a4d95640 1 --2- 172.21.15.84:0/3139562456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 0x55f4ae0051e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff99800cc60 tx=0x7ff99800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c916f288dec1955c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff996ffd640 1 -- 172.21.15.84:0/3139562456 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998014070 con 0x55f4ae004e00 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff996ffd640 1 -- 172.21.15.84:0/3139562456 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9980047a0 con 0x55f4ae004e00 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.397+0000 7ff996ffd640 1 -- 172.21.15.84:0/3139562456 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998010630 con 0x55f4ae004e00 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.398+0000 7ff9a9dea740 1 -- 172.21.15.84:0/3139562456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 msgr2=0x55f4ae0051e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.398+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/3139562456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 0x55f4ae0051e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7ff99800cc60 tx=0x7ff99800e720 comp rx=0 tx=0).stop 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.398+0000 7ff9a9dea740 1 -- 172.21.15.84:0/3139562456 shutdown_connections 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.398+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/3139562456 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae0144f0 0x55f4ae0168e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.398+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/3139562456 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 0x55f4ae013fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.398+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/3139562456 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 0x55f4ae0051e0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.405 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.398+0000 7ff9a9dea740 1 -- 172.21.15.84:0/3139562456 >> 172.21.15.84:0/3139562456 conn(0x55f4ae001410 msgr2=0x55f4ae0017f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.398+0000 7ff9a9dea740 1 -- 172.21.15.84:0/3139562456 shutdown_connections 2024-04-26T18:43:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.398+0000 7ff9a9dea740 1 -- 172.21.15.84:0/3139562456 wait complete. 2024-04-26T18:43:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.398+0000 7ff9a9dea740 1 -- start start 2024-04-26T18:43:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.398+0000 7ff9a9dea740 1 Processor -- start 2024-04-26T18:43:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff9a9dea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae004e00 0x55f4ae0114b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff9a9dea740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae005720 0x55f4ae0119f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff9a9dea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae0144f0 0x55f4ae011f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff9a9dea740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f4ae017120 con 0x55f4ae005720 2024-04-26T18:43:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff9a9dea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f4adef2b60 con 0x55f4ae0144f0 2024-04-26T18:43:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff9a9dea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f4ae0172a0 con 0x55f4ae004e00 2024-04-26T18:43:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff9a9dea740 10 client.0 _ll_register_callbacks cb 0x55f4adf172b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff99ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae005720 0x55f4ae0119f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.406 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff99ffff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae005720 0x55f4ae0119f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42764/0 (socket says 172.21.15.84:42764) 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff99ffff640 1 -- 172.21.15.84:0/59323551 learned_addr learned my addr 172.21.15.84:0/59323551 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff9a4d95640 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae004e00 0x55f4ae0114b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff99ffff640 1 -- 172.21.15.84:0/59323551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae004e00 msgr2=0x55f4ae0114b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff99ffff640 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae004e00 0x55f4ae0114b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff9a5596640 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae0144f0 0x55f4ae011f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff99ffff640 1 -- 172.21.15.84:0/59323551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae0144f0 msgr2=0x55f4ae011f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff99ffff640 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae0144f0 0x55f4ae011f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff99ffff640 1 -- 172.21.15.84:0/59323551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4adc7d210 con 0x55f4ae005720 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff9a4d95640 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae004e00 0x55f4ae0114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff99ffff640 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae005720 0x55f4ae0119f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff990035000 tx=0x7ff99002a9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69c22668ebf1d38 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.399+0000 7ff97ffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff97ffff640 1 -- 172.21.15.84:0/59323551 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff990033040 con 0x55f4ae005720 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff97ffff640 1 -- 172.21.15.84:0/59323551 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7ff99002fb40 con 0x55f4ae005720 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff9a9dea740 1 -- 172.21.15.84:0/59323551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9980047a0 con 0x55f4ae005720 2024-04-26T18:43:59.407 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff97ffff640 1 -- 172.21.15.84:0/59323551 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff990004040 con 0x55f4ae005720 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff9a9dea740 10 client.17313 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff9a9dea740 1 -- 172.21.15.84:0/59323551 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f4add49620 con 0x55f4ae005720 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff97ffff640 1 -- 172.21.15.84:0/59323551 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9980047a0 con 0x55f4ae005720 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff97ffff640 1 -- 172.21.15.84:0/59323551 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff990058030 con 0x55f4ae005720 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff9a9dea740 10 client.17313 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff9a9dea740 10 client.17313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff9a9dea740 20 client.17313 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f4ae024690 0x55f4ae026b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff9a9dea740 4 client.17313 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.400+0000 7ff9a9dea740 1 -- 172.21.15.84:0/59323551 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4adf79070 con 0x55f4ae024690 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.401+0000 7ff9a4d95640 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f4ae024690 0x55f4ae026b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.401+0000 7ff97ffff640 10 client.17313 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.401+0000 7ff9a4d95640 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f4ae024690 0x55f4ae026b20 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=555afa093a408166 server_cookie=ef4a3573a4d15579 in_seq=0 out_seq=0 2024-04-26T18:43:59.408 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff97ffff640 1 -- 172.21.15.84:0/59323551 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f4adf79070 con 0x55f4ae024690 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff97ffff640 10 client.17313 handle_command_reply: tid=0 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff9a9dea740 1 client.17313 shutdown 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff9a9dea740 1 -- 172.21.15.84:0/59323551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae005720 msgr2=0x55f4ae0119f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae005720 0x55f4ae0119f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7ff990035000 tx=0x7ff99002a9e0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff9a9dea740 1 -- 172.21.15.84:0/59323551 shutdown_connections 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f4ae024690 0x55f4ae026b20 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae0144f0 0x55f4ae011f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae005720 0x55f4ae0119f0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff97ffff640 0 client.17313 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/59323551 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae004e00 0x55f4ae0114b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff9a9dea740 1 -- 172.21.15.84:0/59323551 >> 172.21.15.84:0/59323551 conn(0x55f4ae001410 msgr2=0x55f4ae007280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff9a9dea740 1 -- 172.21.15.84:0/59323551 shutdown_connections 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff9a9dea740 1 -- 172.21.15.84:0/59323551 wait complete. 2024-04-26T18:43:59.409 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.402+0000 7ff9a9dea740 20 client.17313 trim_cache size 0 max 16384 2024-04-26T18:43:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a9dea740 1 Processor -- start 2024-04-26T18:43:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a9dea740 1 -- start start 2024-04-26T18:43:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a9dea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae004e00 0x55f4ae01ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a9dea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 0x55f4ae000430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a9dea740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae0144f0 0x55f4ae02c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.421 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a9dea740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01ac3a0 con 0x55f4ae0144f0 2024-04-26T18:43:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a9dea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01adee0 con 0x55f4ae005720 2024-04-26T18:43:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a9dea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01aca70 con 0x55f4ae004e00 2024-04-26T18:43:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a5596640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae0144f0 0x55f4ae02c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a5596640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae0144f0 0x55f4ae02c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42778/0 (socket says 172.21.15.84:42778) 2024-04-26T18:43:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a5596640 1 -- 172.21.15.84:0/619836751 learned_addr learned my addr 172.21.15.84:0/619836751 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a5596640 1 -- 172.21.15.84:0/619836751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae004e00 msgr2=0x55f4ae01ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a5596640 1 --2- 172.21.15.84:0/619836751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae004e00 0x55f4ae01ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a5596640 1 -- 172.21.15.84:0/619836751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 msgr2=0x55f4ae000430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a5596640 1 --2- 172.21.15.84:0/619836751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 0x55f4ae000430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a5596640 1 -- 172.21.15.84:0/619836751 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4ae005c60 con 0x55f4ae0144f0 2024-04-26T18:43:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.417+0000 7ff9a5596640 1 --2- 172.21.15.84:0/619836751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae0144f0 0x55f4ae02c430 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff9880a5000 tx=0x7ff98809c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a80ff7f6408348b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.418+0000 7ff9967fc640 1 -- 172.21.15.84:0/619836751 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98809cfc0 con 0x55f4ae0144f0 2024-04-26T18:43:59.422 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.418+0000 7ff9967fc640 1 -- 172.21.15.84:0/619836751 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff988016300 con 0x55f4ae0144f0 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.418+0000 7ff9967fc640 1 -- 172.21.15.84:0/619836751 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9880870f0 con 0x55f4ae0144f0 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.418+0000 7ff9a9dea740 1 -- 172.21.15.84:0/619836751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae0144f0 msgr2=0x55f4ae02c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.418+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/619836751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae0144f0 0x55f4ae02c430 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7ff9880a5000 tx=0x7ff98809c500 comp rx=0 tx=0).stop 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.418+0000 7ff9a9dea740 1 -- 172.21.15.84:0/619836751 shutdown_connections 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.418+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/619836751 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae0144f0 0x55f4ae02c430 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.418+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/619836751 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 0x55f4ae000430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.418+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/619836751 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae004e00 0x55f4ae01ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.418+0000 7ff9a9dea740 1 -- 172.21.15.84:0/619836751 >> 172.21.15.84:0/619836751 conn(0x55f4ae001410 msgr2=0x55f4adfffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.418+0000 7ff9a9dea740 1 -- 172.21.15.84:0/619836751 shutdown_connections 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.418+0000 7ff9a9dea740 1 -- 172.21.15.84:0/619836751 wait complete. 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a9dea740 1 -- start start 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a9dea740 1 Processor -- start 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a9dea740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 0x55f4ae018110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a9dea740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 0x55f4ae018650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.423 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a9dea740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae0144f0 0x55f4ae018b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a9dea740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01ad810 con 0x55f4ae004e00 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a9dea740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01ad140 con 0x55f4ae005720 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a9dea740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff9a01aa3c0 con 0x55f4ae0144f0 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a9dea740 10 client.0 _ll_register_callbacks cb 0x7ff990009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a4d95640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 0x55f4ae018110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a4d95640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 0x55f4ae018110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:42788/0 (socket says 172.21.15.84:42788) 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a4d95640 1 -- 172.21.15.84:0/2988490564 learned_addr learned my addr 172.21.15.84:0/2988490564 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff99ffff640 1 --2- 172.21.15.84:0/2988490564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 0x55f4ae018650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a4d95640 1 -- 172.21.15.84:0/2988490564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae0144f0 msgr2=0x55f4ae018b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a4d95640 1 --2- 172.21.15.84:0/2988490564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae0144f0 0x55f4ae018b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a4d95640 1 -- 172.21.15.84:0/2988490564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 msgr2=0x55f4ae018650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a4d95640 1 --2- 172.21.15.84:0/2988490564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 0x55f4ae018650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff9a4d95640 1 -- 172.21.15.84:0/2988490564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4ae001240 con 0x55f4ae004e00 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.419+0000 7ff99ffff640 1 --2- 172.21.15.84:0/2988490564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 0x55f4ae018650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:43:59.424 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff9a4d95640 1 --2- 172.21.15.84:0/2988490564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 0x55f4ae018110 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff99800f010 tx=0x7ff998010550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e649da4f5c134630 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff97ffff640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff97ffff640 1 -- 172.21.15.84:0/2988490564 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998014030 con 0x55f4ae004e00 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff97ffff640 1 -- 172.21.15.84:0/2988490564 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(114..114 src has 1..114) ==== 8579+0+0 (secure 0 0 0) 0x7ff998005520 con 0x55f4ae004e00 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff9a9dea740 1 -- 172.21.15.84:0/2988490564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff988016300 con 0x55f4ae004e00 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff97ffff640 1 -- 172.21.15.84:0/2988490564 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9980107f0 con 0x55f4ae004e00 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff9a9dea740 10 client.17319 fetch_fsmap learned FSMap version 29 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff9a9dea740 1 -- 172.21.15.84:0/2988490564 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f4ae08b030 con 0x55f4ae004e00 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff97ffff640 1 -- 172.21.15.84:0/2988490564 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff988016300 con 0x55f4ae004e00 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff97ffff640 1 -- 172.21.15.84:0/2988490564 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7ff99802b020 con 0x55f4ae004e00 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff9a9dea740 10 client.17319 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff9a9dea740 10 client.17319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff9a9dea740 20 client.17319 populate_metadata read hostname 'smithi084' 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/2988490564 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f4ae024690 0x55f4ae08d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff9a9dea740 4 client.17319 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.420+0000 7ff9a9dea740 1 -- 172.21.15.84:0/2988490564 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4adf72ee0 con 0x55f4ae024690 2024-04-26T18:43:59.425 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.421+0000 7ff99ffff640 1 --2- 172.21.15.84:0/2988490564 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f4ae024690 0x55f4ae08d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:43:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.421+0000 7ff97ffff640 10 client.17319 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:59.426 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.421+0000 7ff99ffff640 1 --2- 172.21.15.84:0/2988490564 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f4ae024690 0x55f4ae08d660 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f109c478ac603db6 server_cookie=af8e044d6b43a82b in_seq=0 out_seq=0 2024-04-26T18:43:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.433+0000 7ff97ffff640 1 -- 172.21.15.84:0/2988490564 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4adf72ee0 con 0x55f4ae024690 2024-04-26T18:43:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.433+0000 7ff97ffff640 10 client.17319 handle_command_reply: tid=0 2024-04-26T18:43:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff9a9dea740 1 client.17319 shutdown 2024-04-26T18:43:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff9a9dea740 1 -- 172.21.15.84:0/2988490564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 msgr2=0x55f4ae018110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:59.434 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/2988490564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 0x55f4ae018110 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7ff99800f010 tx=0x7ff998010550 comp rx=0 tx=0).stop 2024-04-26T18:43:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff9a9dea740 1 -- 172.21.15.84:0/2988490564 shutdown_connections 2024-04-26T18:43:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/2988490564 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f4ae024690 0x55f4ae08d660 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/2988490564 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f4ae0144f0 0x55f4ae018b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/2988490564 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f4ae005720 0x55f4ae018650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff9a9dea740 1 --2- 172.21.15.84:0/2988490564 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f4ae004e00 0x55f4ae018110 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff97ffff640 0 client.17319 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:43:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff9a9dea740 1 -- 172.21.15.84:0/2988490564 >> 172.21.15.84:0/2988490564 conn(0x55f4ae001410 msgr2=0x55f4ae007230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff9a9dea740 1 -- 172.21.15.84:0/2988490564 shutdown_connections 2024-04-26T18:43:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff9a9dea740 1 -- 172.21.15.84:0/2988490564 wait complete. 2024-04-26T18:43:59.435 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff9a9dea740 20 client.17319 trim_cache size 0 max 16384 2024-04-26T18:43:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:43:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:43:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:43:59.436 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:43:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.434+0000 7ff9a6ff9640 1 -- 172.21.15.84:0/378159355 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff98c0a3dc0 msgr2=0x7ff98c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.435+0000 7ff9a6ff9640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff98c0a3dc0 0x7ff98c0a6280 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff990009880 tx=0x7ff99002edc0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.435+0000 7ff9a6ff9640 1 -- 172.21.15.84:0/378159355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a010cf00 msgr2=0x7ff9a019d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:43:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.435+0000 7ff9a6ff9640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a010cf00 0x7ff9a019d890 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7ff98800c970 tx=0x7ff98800ce30 comp rx=0 tx=0).stop 2024-04-26T18:43:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.435+0000 7ff9a6ff9640 1 -- 172.21.15.84:0/378159355 shutdown_connections 2024-04-26T18:43:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.435+0000 7ff9a6ff9640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff98c0a3dc0 0x7ff98c0a6280 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.435+0000 7ff9a6ff9640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff9a010cf00 0x7ff9a019d890 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.435+0000 7ff9a6ff9640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff9a01044e0 0x7ff9a0199510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.435+0000 7ff9a6ff9640 1 --2- 172.21.15.84:0/378159355 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff9a0103bc0 0x7ff9a0198fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:43:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.435+0000 7ff9a6ff9640 1 -- 172.21.15.84:0/378159355 >> 172.21.15.84:0/378159355 conn(0x7ff9a00789b0 msgr2=0x7ff9a00fe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:43:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.435+0000 7ff9a6ff9640 1 -- 172.21.15.84:0/378159355 shutdown_connections 2024-04-26T18:43:59.437 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:43:59.435+0000 7ff9a6ff9640 1 -- 172.21.15.84:0/378159355 wait complete. 2024-04-26T18:44:01.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:00 smithi084 ceph-mon[29141]: pgmap v504: 97 pgs: 97 active+clean; 2.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-04-26T18:44:01.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:00 smithi103 ceph-mon[32727]: pgmap v504: 97 pgs: 97 active+clean; 2.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-04-26T18:44:01.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:00 smithi174 ceph-mon[32705]: pgmap v504: 97 pgs: 97 active+clean; 2.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 19 op/s 2024-04-26T18:44:03.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:02 smithi084 ceph-mon[29141]: pgmap v505: 97 pgs: 97 active+clean; 2.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-04-26T18:44:03.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:02 smithi103 ceph-mon[32727]: pgmap v505: 97 pgs: 97 active+clean; 2.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-04-26T18:44:03.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:02 smithi174 ceph-mon[32705]: pgmap v505: 97 pgs: 97 active+clean; 2.4 MiB data, 443 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-04-26T18:44:05.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:04 smithi084 ceph-mon[29141]: pgmap v506: 97 pgs: 97 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 101 KiB/s wr, 38 op/s 2024-04-26T18:44:05.096 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:04 smithi084 ceph-mon[29141]: osdmap e115: 12 total, 12 up, 12 in 2024-04-26T18:44:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:04 smithi103 ceph-mon[32727]: pgmap v506: 97 pgs: 97 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 101 KiB/s wr, 38 op/s 2024-04-26T18:44:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:04 smithi103 ceph-mon[32727]: osdmap e115: 12 total, 12 up, 12 in 2024-04-26T18:44:05.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:04 smithi174 ceph-mon[32705]: pgmap v506: 97 pgs: 97 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 101 KiB/s wr, 38 op/s 2024-04-26T18:44:05.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:04 smithi174 ceph-mon[32705]: osdmap e115: 12 total, 12 up, 12 in 2024-04-26T18:44:06.669 INFO:tasks.workunit.client.0.smithi084.stdout:499/600: open fd = 3 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:500/600: open fd = 3 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:501/600: open fd = 3 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:502/600: open fd = 3 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.670 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:503/600: open fd = 3 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:504/600: open fd = 3 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:505/600: open fd = 3 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:506/600: open fd = 3 2024-04-26T18:44:06.671 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:507/600: open fd = 3 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:508/600: open fd = 3 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:509/600: open fd = 3 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.672 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:510/600: open fd = 3 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:511/600: open fd = 3 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:512/600: open fd = 3 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:513/600: open fd = 3 2024-04-26T18:44:06.673 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:514/600: open fd = 3 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:515/600: open fd = 3 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:516/600: open fd = 3 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.674 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:517/600: open fd = 3 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:518/600: open fd = 3 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:519/600: open fd = 3 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:520/600: open fd = 3 2024-04-26T18:44:06.675 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:521/600: open fd = 3 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:522/600: open fd = 3 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:523/600: open fd = 3 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.676 INFO:tasks.workunit.client.0.smithi084.stdout:524/600: open fd = 3 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:525/600: open fd = 3 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:526/600: open fd = 3 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:527/600: open fd = 3 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.677 INFO:tasks.workunit.client.0.smithi084.stdout:528/600: open fd = 3 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:529/600: open fd = 3 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:530/600: open fd = 3 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:531/600: open fd = 3 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.678 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:532/600: open fd = 3 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:533/600: open fd = 3 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:534/600: open fd = 3 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:535/600: open fd = 3 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.679 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:536/600: open fd = 3 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:537/600: open fd = 3 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:538/600: open fd = 3 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:539/600: open fd = 3 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.680 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:540/600: open fd = 3 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:541/600: open fd = 3 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:542/600: open fd = 3 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:543/600: open fd = 3 2024-04-26T18:44:06.681 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.682 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.682 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.682 INFO:tasks.workunit.client.0.smithi084.stdout:544/600: open fd = 3 2024-04-26T18:44:06.682 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.682 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.682 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.682 INFO:tasks.workunit.client.0.smithi084.stdout:545/600: open fd = 3 2024-04-26T18:44:06.682 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.682 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.682 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.682 INFO:tasks.workunit.client.0.smithi084.stdout:546/600: open fd = 3 2024-04-26T18:44:06.682 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.682 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:547/600: open fd = 3 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:548/600: open fd = 3 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:549/600: open fd = 3 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.683 INFO:tasks.workunit.client.0.smithi084.stdout:550/600: open fd = 3 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:551/600: open fd = 3 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:552/600: open fd = 3 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:553/600: open fd = 3 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.684 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.685 INFO:tasks.workunit.client.0.smithi084.stdout:554/600: open fd = 3 2024-04-26T18:44:06.685 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.685 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.685 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.685 INFO:tasks.workunit.client.0.smithi084.stdout:555/600: open fd = 3 2024-04-26T18:44:06.685 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.685 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.685 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.685 INFO:tasks.workunit.client.0.smithi084.stdout:556/600: open fd = 3 2024-04-26T18:44:06.685 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.685 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:557/600: open fd = 3 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:558/600: open fd = 3 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:559/600: open fd = 3 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:560/600: open fd = 3 2024-04-26T18:44:06.686 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.687 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:44:06.687 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:44:07.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:06 smithi084 ceph-mon[29141]: pgmap v508: 97 pgs: 97 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-26T18:44:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:06 smithi103 ceph-mon[32727]: pgmap v508: 97 pgs: 97 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-26T18:44:07.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:06 smithi174 ceph-mon[32705]: pgmap v508: 97 pgs: 97 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-26T18:44:07.843 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 32.6 2024-04-26T18:44:07.844 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:44:08.094 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:44:09.804 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:09 smithi084 ceph-mon[29141]: pgmap v509: 97 pgs: 97 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-26T18:44:09.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.812+0000 7f31e438a700 1 -- 172.21.15.84:0/3671149146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31dc1083a0 msgr2=0x7f31dc108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:09.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.812+0000 7f31e438a700 1 --2- 172.21.15.84:0/3671149146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31dc1083a0 0x7f31dc108780 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f31d80099c0 tx=0x7f31d8009cd0 comp rx=0 tx=0).stop 2024-04-26T18:44:09.812 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.812+0000 7f31e438a700 1 -- 172.21.15.84:0/3671149146 shutdown_connections 2024-04-26T18:44:09.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.812+0000 7f31e438a700 1 --2- 172.21.15.84:0/3671149146 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31dc1083a0 0x7f31dc108780 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:09.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.812+0000 7f31e438a700 1 --2- 172.21.15.84:0/3671149146 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31dc06c520 0x7f31dc1102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:09.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.812+0000 7f31e438a700 1 --2- 172.21.15.84:0/3671149146 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31dc06bb60 0x7f31dc06bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:09.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.812+0000 7f31e438a700 1 -- 172.21.15.84:0/3671149146 >> 172.21.15.84:0/3671149146 conn(0x7f31dc0783c0 msgr2=0x7f31dc0787d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:09.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.812+0000 7f31e438a700 1 -- 172.21.15.84:0/3671149146 shutdown_connections 2024-04-26T18:44:09.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.812+0000 7f31e438a700 1 -- 172.21.15.84:0/3671149146 wait complete. 2024-04-26T18:44:09.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.812+0000 7f31e438a700 1 Processor -- start 2024-04-26T18:44:09.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.812+0000 7f31e438a700 1 -- start start 2024-04-26T18:44:09.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e438a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31dc06bb60 0x7f31dc19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:09.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e438a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31dc06c520 0x7f31dc19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:09.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e438a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31dc1083a0 0x7f31dc1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:09.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e438a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc113700 con 0x7f31dc06c520 2024-04-26T18:44:09.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e438a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc113580 con 0x7f31dc1083a0 2024-04-26T18:44:09.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e438a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31dc113880 con 0x7f31dc06bb60 2024-04-26T18:44:09.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e1925700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31dc06c520 0x7f31dc19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:09.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e1925700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31dc06c520 0x7f31dc19fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39662/0 (socket says 172.21.15.84:39662) 2024-04-26T18:44:09.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e1925700 1 -- 172.21.15.84:0/1671043860 learned_addr learned my addr 172.21.15.84:0/1671043860 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:09.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e2126700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31dc06bb60 0x7f31dc19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:09.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e2927700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31dc1083a0 0x7f31dc1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:09.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e1925700 1 -- 172.21.15.84:0/1671043860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31dc06bb60 msgr2=0x7f31dc19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:09.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e1925700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31dc06bb60 0x7f31dc19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:09.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e1925700 1 -- 172.21.15.84:0/1671043860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31dc1083a0 msgr2=0x7f31dc1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:09.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e1925700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31dc1083a0 0x7f31dc1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:09.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e1925700 1 -- 172.21.15.84:0/1671043860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31dc19a140 con 0x7f31dc06c520 2024-04-26T18:44:09.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e2927700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31dc1083a0 0x7f31dc1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:44:09.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.813+0000 7f31e1925700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31dc06c520 0x7f31dc19fd60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00b530 tx=0x7f31cc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:09.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.814+0000 7f31d37fe700 1 -- 172.21.15.84:0/1671043860 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc015020 con 0x7f31dc06c520 2024-04-26T18:44:09.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.814+0000 7f31d37fe700 1 -- 172.21.15.84:0/1671043860 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31cc011ce0 con 0x7f31dc06c520 2024-04-26T18:44:09.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.814+0000 7f31e438a700 1 -- 172.21.15.84:0/1671043860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31dc19a430 con 0x7f31dc06c520 2024-04-26T18:44:09.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.814+0000 7f31e438a700 1 -- 172.21.15.84:0/1671043860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31dc1a66f0 con 0x7f31dc06c520 2024-04-26T18:44:09.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.814+0000 7f31d37fe700 1 -- 172.21.15.84:0/1671043860 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31cc010e40 con 0x7f31dc06c520 2024-04-26T18:44:09.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.815+0000 7f31e438a700 1 -- 172.21.15.84:0/1671043860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31dc104b80 con 0x7f31dc06c520 2024-04-26T18:44:09.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.816+0000 7f31d37fe700 1 -- 172.21.15.84:0/1671043860 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f31cc018020 con 0x7f31dc06c520 2024-04-26T18:44:09.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.817+0000 7f31d37fe700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f31c80a3d10 0x7f31c80a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:09.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.817+0000 7f31e2126700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f31c80a3d10 0x7f31c80a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:09.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.817+0000 7f31d37fe700 1 -- 172.21.15.84:0/1671043860 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(115..115 src has 1..115) ==== 8623+0+0 (secure 0 0 0) 0x7f31cc0d0460 con 0x7f31dc06c520 2024-04-26T18:44:09.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.817+0000 7f31e2126700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f31c80a3d10 0x7f31c80a61d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f31dc19ab20 tx=0x7f31d800b870 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:09.820 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:09.820+0000 7f31d37fe700 1 -- 172.21.15.84:0/1671043860 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f31cc01f020 con 0x7f31dc06c520 2024-04-26T18:44:09.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:09 smithi103 ceph-mon[32727]: pgmap v509: 97 pgs: 97 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-26T18:44:09.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:09 smithi174 ceph-mon[32705]: pgmap v509: 97 pgs: 97 active+clean; 2.6 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-26T18:44:10.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.064+0000 7f31e438a700 1 -- 172.21.15.84:0/1671043860 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31dc06a110 con 0x7f31dc06c520 2024-04-26T18:44:10.065 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.065+0000 7f31d37fe700 1 -- 172.21.15.84:0/1671043860 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f31cc09cbb0 con 0x7f31dc06c520 2024-04-26T18:44:10.065 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:44:10.065 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:44:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.067+0000 7f31e438a700 1 -- 172.21.15.84:0/1671043860 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f31c80a3d10 msgr2=0x7f31c80a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.067+0000 7f31e438a700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f31c80a3d10 0x7f31c80a61d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f31dc19ab20 tx=0x7f31d800b870 comp rx=0 tx=0).stop 2024-04-26T18:44:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.067+0000 7f31e438a700 1 -- 172.21.15.84:0/1671043860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31dc06c520 msgr2=0x7f31dc19fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.067+0000 7f31e438a700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31dc06c520 0x7f31dc19fd60 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f31cc00b530 tx=0x7f31cc00b840 comp rx=0 tx=0).stop 2024-04-26T18:44:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.067+0000 7f31e438a700 1 -- 172.21.15.84:0/1671043860 shutdown_connections 2024-04-26T18:44:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.067+0000 7f31e438a700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f31c80a3d10 0x7f31c80a61d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.067+0000 7f31e438a700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f31dc1083a0 0x7f31dc1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.067+0000 7f31e438a700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f31dc06bb60 0x7f31dc19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.067+0000 7f31e438a700 1 --2- 172.21.15.84:0/1671043860 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f31dc06c520 0x7f31dc19fd60 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.067+0000 7f31e438a700 1 -- 172.21.15.84:0/1671043860 >> 172.21.15.84:0/1671043860 conn(0x7f31dc0783c0 msgr2=0x7f31dc100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.068+0000 7f31e438a700 1 -- 172.21.15.84:0/1671043860 shutdown_connections 2024-04-26T18:44:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:10.068+0000 7f31e438a700 1 -- 172.21.15.84:0/1671043860 wait complete. 2024-04-26T18:44:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:44:10.701 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:10 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1671043860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:44:10.730 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db / --timeout 60 --expiration 92.6 --await-for 42.0' 2024-04-26T18:44:10.730 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db / --timeout 60 --expiration 92.6 --await-for 42.0 2024-04-26T18:44:10.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:10 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1671043860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:44:10.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:10 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1671043860' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:44:11.004 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:44:11.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:11 smithi084 ceph-mon[29141]: pgmap v510: 97 pgs: 97 active+clean; 2.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-04-26T18:44:11.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:11 smithi103 ceph-mon[32727]: pgmap v510: 97 pgs: 97 active+clean; 2.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-04-26T18:44:11.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:11 smithi174 ceph-mon[32705]: pgmap v510: 97 pgs: 97 active+clean; 2.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-04-26T18:44:12.525 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.525+0000 7f6e7a12b640 1 -- 172.21.15.84:0/865820011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e741021d0 msgr2=0x7f6e7410e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.525+0000 7f6e7a12b640 1 --2- 172.21.15.84:0/865820011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e741021d0 0x7f6e7410e6e0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009cd0 tx=0x7f6e6c02a8e0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.525+0000 7f6e7a12b640 1 -- 172.21.15.84:0/865820011 shutdown_connections 2024-04-26T18:44:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.525+0000 7f6e7a12b640 1 --2- 172.21.15.84:0/865820011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e741021d0 0x7f6e7410e6e0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.525+0000 7f6e7a12b640 1 --2- 172.21.15.84:0/865820011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e74101810 0x7f6e74101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.525+0000 7f6e7a12b640 1 --2- 172.21.15.84:0/865820011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e74107820 0x7f6e74107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.525+0000 7f6e7a12b640 1 -- 172.21.15.84:0/865820011 >> 172.21.15.84:0/865820011 conn(0x7f6e740fd300 msgr2=0x7f6e740ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.525+0000 7f6e7a12b640 1 -- 172.21.15.84:0/865820011 shutdown_connections 2024-04-26T18:44:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.525+0000 7f6e7a12b640 1 -- 172.21.15.84:0/865820011 wait complete. 2024-04-26T18:44:12.526 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.526+0000 7f6e7a12b640 1 Processor -- start 2024-04-26T18:44:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.526+0000 7f6e7a12b640 1 -- start start 2024-04-26T18:44:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.526+0000 7f6e7a12b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e74101810 0x7f6e741990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.526+0000 7f6e7a12b640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e741021d0 0x7f6e741995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.526+0000 7f6e7a12b640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e74107820 0x7f6e7419d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.526+0000 7f6e7a12b640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e74111ae0 con 0x7f6e74101810 2024-04-26T18:44:12.528 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.526+0000 7f6e7a12b640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e74111960 con 0x7f6e741021d0 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.526+0000 7f6e7a12b640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e74111c60 con 0x7f6e74107820 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.526+0000 7f6e737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e74101810 0x7f6e741990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.526+0000 7f6e737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e74101810 0x7f6e741990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39690/0 (socket says 172.21.15.84:39690) 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.526+0000 7f6e737fe640 1 -- 172.21.15.84:0/1854033378 learned_addr learned my addr 172.21.15.84:0/1854033378 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.526+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e741021d0 0x7f6e741995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.526+0000 7f6e73fff640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e74107820 0x7f6e7419d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.527+0000 7f6e737fe640 1 -- 172.21.15.84:0/1854033378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e74107820 msgr2=0x7f6e7419d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.527+0000 7f6e737fe640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e74107820 0x7f6e7419d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.527+0000 7f6e737fe640 1 -- 172.21.15.84:0/1854033378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e741021d0 msgr2=0x7f6e741995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.527+0000 7f6e737fe640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e741021d0 0x7f6e741995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.527+0000 7f6e737fe640 1 -- 172.21.15.84:0/1854033378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e7419e040 con 0x7f6e74101810 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.527+0000 7f6e737fe640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e74101810 0x7f6e741990a0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400d950 tx=0x7f6e6400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.527+0000 7f6e70ff9640 1 -- 172.21.15.84:0/1854033378 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64017030 con 0x7f6e74101810 2024-04-26T18:44:12.529 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.527+0000 7f6e7a12b640 1 -- 172.21.15.84:0/1854033378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e7419e330 con 0x7f6e74101810 2024-04-26T18:44:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.527+0000 7f6e70ff9640 1 -- 172.21.15.84:0/1854033378 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e64004020 con 0x7f6e74101810 2024-04-26T18:44:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.527+0000 7f6e70ff9640 1 -- 172.21.15.84:0/1854033378 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e640047a0 con 0x7f6e74101810 2024-04-26T18:44:12.530 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.527+0000 7f6e7a12b640 1 -- 172.21.15.84:0/1854033378 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e741086b0 con 0x7f6e74101810 2024-04-26T18:44:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.530+0000 7f6e70ff9640 1 -- 172.21.15.84:0/1854033378 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f6e64004940 con 0x7f6e74101810 2024-04-26T18:44:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.530+0000 7f6e7ce47740 1 Processor -- start 2024-04-26T18:44:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.530+0000 7f6e7ce47740 1 -- start start 2024-04-26T18:44:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.530+0000 7f6e7ce47740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a71f00 0x555e25a722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.530+0000 7f6e7ce47740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a72820 0x555e25a7cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.530+0000 7f6e7ce47740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a7d0e0 0x555e25a7f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e7ce47740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555e259d8460 con 0x555e25a7d0e0 2024-04-26T18:44:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e7ce47740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555e25992590 con 0x555e25a72820 2024-04-26T18:44:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e7ce47740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555e259d4420 con 0x555e25a71f00 2024-04-26T18:44:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e72ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a7d0e0 0x555e25a7f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e72ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a7d0e0 0x555e25a7f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39694/0 (socket says 172.21.15.84:39694) 2024-04-26T18:44:12.533 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e72ffd640 1 -- 172.21.15.84:0/3441176011 learned_addr learned my addr 172.21.15.84:0/3441176011 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e73fff640 1 --2- 172.21.15.84:0/3441176011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a71f00 0x555e25a722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e70ff9640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e5c0a3dc0 0x7f6e5c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e737fe640 1 --2- 172.21.15.84:0/3441176011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a72820 0x555e25a7cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e73fff640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e5c0a3dc0 0x7f6e5c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e72ffd640 1 -- 172.21.15.84:0/3441176011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a71f00 msgr2=0x555e25a722e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/3441176011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a71f00 0x555e25a722e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e72ffd640 1 -- 172.21.15.84:0/3441176011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a72820 msgr2=0x555e25a7cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/3441176011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a72820 0x555e25a7cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e72ffd640 1 -- 172.21.15.84:0/3441176011 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e2599f0d0 con 0x555e25a7d0e0 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e70ff9640 1 -- 172.21.15.84:0/1854033378 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f6e640d0230 con 0x7f6e74101810 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/3441176011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a7d0e0 0x555e25a7f4d0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f6e68010de0 tx=0x7f6e6800a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9a028dc7d72dee server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e59ffb640 1 -- 172.21.15.84:0/3441176011 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68019760 con 0x555e25a7d0e0 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.531+0000 7f6e73fff640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e5c0a3dc0 0x7f6e5c0a6280 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009430 tx=0x7f6e6c02a8e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:12.534 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.532+0000 7f6e59ffb640 1 -- 172.21.15.84:0/3441176011 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e6801f070 con 0x555e25a7d0e0 2024-04-26T18:44:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.532+0000 7f6e59ffb640 1 -- 172.21.15.84:0/3441176011 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6801aba0 con 0x555e25a7d0e0 2024-04-26T18:44:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.532+0000 7f6e7ce47740 1 -- 172.21.15.84:0/3441176011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a7d0e0 msgr2=0x555e25a7f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.532+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/3441176011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a7d0e0 0x555e25a7f4d0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f6e68010de0 tx=0x7f6e6800a820 comp rx=0 tx=0).stop 2024-04-26T18:44:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.532+0000 7f6e7ce47740 1 -- 172.21.15.84:0/3441176011 shutdown_connections 2024-04-26T18:44:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.532+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/3441176011 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a7d0e0 0x555e25a7f4d0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.532+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/3441176011 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a72820 0x555e25a7cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.532+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/3441176011 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a71f00 0x555e25a722e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.532+0000 7f6e7ce47740 1 -- 172.21.15.84:0/3441176011 >> 172.21.15.84:0/3441176011 conn(0x555e25a6e410 msgr2=0x555e25a6e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.532+0000 7f6e7ce47740 1 -- 172.21.15.84:0/3441176011 shutdown_connections 2024-04-26T18:44:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.532+0000 7f6e7ce47740 1 -- 172.21.15.84:0/3441176011 wait complete. 2024-04-26T18:44:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.533+0000 7f6e7ce47740 1 -- start start 2024-04-26T18:44:12.535 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.533+0000 7f6e7ce47740 1 Processor -- start 2024-04-26T18:44:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.533+0000 7f6e7ce47740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 0x555e25a7a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.533+0000 7f6e7ce47740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a72820 0x555e25a7a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.533+0000 7f6e7ce47740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a7d0e0 0x555e25a7ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.533+0000 7f6e7ce47740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x555e25a7fd10 con 0x555e25a71f00 2024-04-26T18:44:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.533+0000 7f6e7ce47740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x555e2595fb60 con 0x555e25a7d0e0 2024-04-26T18:44:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.533+0000 7f6e7ce47740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x555e25a7fe90 con 0x555e25a72820 2024-04-26T18:44:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.533+0000 7f6e7ce47740 10 client.0 _ll_register_callbacks cb 0x555e259842b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:44:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.533+0000 7f6e737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 0x555e25a7a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.533+0000 7f6e737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 0x555e25a7a0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39696/0 (socket says 172.21.15.84:39696) 2024-04-26T18:44:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.533+0000 7f6e737fe640 1 -- 172.21.15.84:0/450575706 learned_addr learned my addr 172.21.15.84:0/450575706 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:12.539 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e73fff640 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a7d0e0 0x555e25a7ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a72820 0x555e25a7a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e737fe640 1 -- 172.21.15.84:0/450575706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a72820 msgr2=0x555e25a7a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e737fe640 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a72820 0x555e25a7a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e737fe640 1 -- 172.21.15.84:0/450575706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a7d0e0 msgr2=0x555e25a7ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e737fe640 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a7d0e0 0x555e25a7ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e737fe640 1 -- 172.21.15.84:0/450575706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e256ea210 con 0x555e25a71f00 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a72820 0x555e25a7a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e737fe640 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 0x555e25a7a0c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f6e6409be90 tx=0x7f6e64066790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27b57d3559bb3f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e52ffd640 1 -- 172.21.15.84:0/450575706 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e64066ca0 con 0x555e25a71f00 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e52ffd640 1 -- 172.21.15.84:0/450575706 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f6e64098800 con 0x555e25a71f00 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e7ce47740 1 -- 172.21.15.84:0/450575706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e6801f070 con 0x555e25a71f00 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e52ffd640 1 -- 172.21.15.84:0/450575706 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6e6409c890 con 0x555e25a71f00 2024-04-26T18:44:12.540 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e7ce47740 10 client.17337 fetch_fsmap learned FSMap version 29 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e7ce47740 1 -- 172.21.15.84:0/450575706 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555e257b6620 con 0x555e25a71f00 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.534+0000 7f6e52ffd640 1 -- 172.21.15.84:0/450575706 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e6801f070 con 0x555e25a71f00 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.535+0000 7f6e52ffd640 1 -- 172.21.15.84:0/450575706 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6e640a0120 con 0x555e25a71f00 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.535+0000 7f6e7ce47740 10 client.17337 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.535+0000 7f6e7ce47740 10 client.17337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.535+0000 7f6e7ce47740 20 client.17337 populate_metadata read hostname 'smithi084' 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.535+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555e25a8d4a0 0x555e25a8f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.535+0000 7f6e7ce47740 4 client.17337 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.535+0000 7f6e7ce47740 1 -- 172.21.15.84:0/450575706 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555e259e6070 con 0x555e25a8d4a0 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.535+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555e25a8d4a0 0x555e25a8f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.535+0000 7f6e52ffd640 10 client.17337 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.535+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555e25a8d4a0 0x555e25a8f890 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1601ad59b831823e server_cookie=255963b789e23bf in_seq=0 out_seq=0 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.536+0000 7f6e52ffd640 1 -- 172.21.15.84:0/450575706 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555e259e6070 con 0x555e25a8d4a0 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.536+0000 7f6e52ffd640 10 client.17337 handle_command_reply: tid=0 2024-04-26T18:44:12.541 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.536+0000 7f6e7ce47740 1 client.17337 shutdown 2024-04-26T18:44:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.536+0000 7f6e7ce47740 1 -- 172.21.15.84:0/450575706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 msgr2=0x555e25a7a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.542 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.536+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 0x555e25a7a0c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f6e6409be90 tx=0x7f6e64066790 comp rx=0 tx=0).stop 2024-04-26T18:44:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.536+0000 7f6e7ce47740 1 -- 172.21.15.84:0/450575706 shutdown_connections 2024-04-26T18:44:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.536+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555e25a8d4a0 0x555e25a8f890 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.536+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a7d0e0 0x555e25a7ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.536+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a72820 0x555e25a7a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.536+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/450575706 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 0x555e25a7a0c0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.536+0000 7f6e7ce47740 1 -- 172.21.15.84:0/450575706 >> 172.21.15.84:0/450575706 conn(0x555e25a6e410 msgr2=0x555e25a6d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.536+0000 7f6e52ffd640 0 client.17337 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.537+0000 7f6e7ce47740 1 -- 172.21.15.84:0/450575706 shutdown_connections 2024-04-26T18:44:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.537+0000 7f6e7ce47740 1 -- 172.21.15.84:0/450575706 wait complete. 2024-04-26T18:44:12.543 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.537+0000 7f6e7ce47740 20 client.17337 trim_cache size 0 max 16384 2024-04-26T18:44:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.551+0000 7f6e7ce47740 1 Processor -- start 2024-04-26T18:44:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.551+0000 7f6e7ce47740 1 -- start start 2024-04-26T18:44:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.551+0000 7f6e7ce47740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a71f00 0x555e25a88bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.551+0000 7f6e7ce47740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a72820 0x555e25a94f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.551+0000 7f6e7ce47740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a7d0e0 0x555e25a95480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.551+0000 7f6e7ce47740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e741ac420 con 0x555e25a72820 2024-04-26T18:44:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.551+0000 7f6e7ce47740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e741adf60 con 0x555e25a71f00 2024-04-26T18:44:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.551+0000 7f6e7ce47740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e741acaf0 con 0x555e25a7d0e0 2024-04-26T18:44:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.551+0000 7f6e72ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a72820 0x555e25a94f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.554 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.551+0000 7f6e72ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a72820 0x555e25a94f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39698/0 (socket says 172.21.15.84:39698) 2024-04-26T18:44:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.551+0000 7f6e72ffd640 1 -- 172.21.15.84:0/2086822898 learned_addr learned my addr 172.21.15.84:0/2086822898 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.552+0000 7f6e737fe640 1 --2- 172.21.15.84:0/2086822898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a71f00 0x555e25a88bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.552+0000 7f6e72ffd640 1 -- 172.21.15.84:0/2086822898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a7d0e0 msgr2=0x555e25a95480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.552+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/2086822898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a7d0e0 0x555e25a95480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.552+0000 7f6e72ffd640 1 -- 172.21.15.84:0/2086822898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a71f00 msgr2=0x555e25a88bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.552+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/2086822898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a71f00 0x555e25a88bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.552+0000 7f6e72ffd640 1 -- 172.21.15.84:0/2086822898 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555e25a78610 con 0x555e25a72820 2024-04-26T18:44:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.552+0000 7f6e737fe640 1 --2- 172.21.15.84:0/2086822898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a71f00 0x555e25a88bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:44:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.552+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/2086822898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a72820 0x555e25a94f40 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6e68010d80 tx=0x7f6e68013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99df0b4c1d43d2fe server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.552+0000 7f6e597fa640 1 -- 172.21.15.84:0/2086822898 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e68030070 con 0x555e25a72820 2024-04-26T18:44:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.552+0000 7f6e597fa640 1 -- 172.21.15.84:0/2086822898 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e6801f030 con 0x555e25a72820 2024-04-26T18:44:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.552+0000 7f6e597fa640 1 -- 172.21.15.84:0/2086822898 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e6802b850 con 0x555e25a72820 2024-04-26T18:44:12.555 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 -- 172.21.15.84:0/2086822898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a72820 msgr2=0x555e25a94f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.556 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/2086822898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a72820 0x555e25a94f40 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f6e68010d80 tx=0x7f6e68013040 comp rx=0 tx=0).stop 2024-04-26T18:44:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 -- 172.21.15.84:0/2086822898 shutdown_connections 2024-04-26T18:44:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/2086822898 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a7d0e0 0x555e25a95480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/2086822898 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a72820 0x555e25a94f40 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/2086822898 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a71f00 0x555e25a88bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 -- 172.21.15.84:0/2086822898 >> 172.21.15.84:0/2086822898 conn(0x555e25a6e410 msgr2=0x555e25a6e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 -- 172.21.15.84:0/2086822898 shutdown_connections 2024-04-26T18:44:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 -- 172.21.15.84:0/2086822898 wait complete. 2024-04-26T18:44:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 -- start start 2024-04-26T18:44:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 Processor -- start 2024-04-26T18:44:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 0x555e25a854d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.559 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a72820 0x555e25a76ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a7d0e0 0x555e25a77000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f6e741ad890 con 0x555e25a71f00 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f6e741ad1c0 con 0x555e25a72820 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f6e741aa420 con 0x555e25a7d0e0 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.553+0000 7f6e7ce47740 10 client.0 _ll_register_callbacks cb 0x7f6e640046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 0x555e25a854d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e737fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 0x555e25a854d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39710/0 (socket says 172.21.15.84:39710) 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e737fe640 1 -- 172.21.15.84:0/1398462275 learned_addr learned my addr 172.21.15.84:0/1398462275 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/1398462275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a72820 0x555e25a76ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e737fe640 1 -- 172.21.15.84:0/1398462275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a7d0e0 msgr2=0x555e25a77000 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e737fe640 1 --2- 172.21.15.84:0/1398462275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a7d0e0 0x555e25a77000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e737fe640 1 -- 172.21.15.84:0/1398462275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a72820 msgr2=0x555e25a76ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e737fe640 1 --2- 172.21.15.84:0/1398462275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a72820 0x555e25a76ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e737fe640 1 -- 172.21.15.84:0/1398462275 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555e25a77830 con 0x555e25a71f00 2024-04-26T18:44:12.560 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/1398462275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a72820 0x555e25a76ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e737fe640 1 --2- 172.21.15.84:0/1398462275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 0x555e25a854d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6e640985f0 tx=0x7f6e640044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cff628db1477cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e52ffd640 1 -- 172.21.15.84:0/1398462275 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e640a4e00 con 0x555e25a71f00 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e52ffd640 1 -- 172.21.15.84:0/1398462275 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f6e640cdbc0 con 0x555e25a71f00 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e7ce47740 1 -- 172.21.15.84:0/1398462275 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6e6801f030 con 0x555e25a71f00 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e52ffd640 1 -- 172.21.15.84:0/1398462275 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6e6409c190 con 0x555e25a71f00 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e7ce47740 10 client.17343 fetch_fsmap learned FSMap version 29 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e7ce47740 1 -- 172.21.15.84:0/1398462275 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555e25af3760 con 0x555e25a71f00 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e52ffd640 1 -- 172.21.15.84:0/1398462275 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e6801f030 con 0x555e25a71f00 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e52ffd640 1 -- 172.21.15.84:0/1398462275 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f6e6406bae0 con 0x555e25a71f00 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e7ce47740 10 client.17343 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.554+0000 7f6e7ce47740 10 client.17343 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.555+0000 7f6e7ce47740 20 client.17343 populate_metadata read hostname 'smithi084' 2024-04-26T18:44:12.561 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.555+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/1398462275 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555e25af3eb0 0x555e25af62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.555+0000 7f6e7ce47740 4 client.17343 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:44:12.562 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.555+0000 7f6e7ce47740 1 -- 172.21.15.84:0/1398462275 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555e259dfee0 con 0x555e25af3eb0 2024-04-26T18:44:12.574 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.574+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/1398462275 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555e25af3eb0 0x555e25af62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.594+0000 7f6e72ffd640 1 --2- 172.21.15.84:0/1398462275 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555e25af3eb0 0x555e25af62f0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41e6bf0e20af0bc server_cookie=e2243305e4964a30 in_seq=0 out_seq=0 2024-04-26T18:44:12.594 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.594+0000 7f6e52ffd640 10 client.17343 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:12.611 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e52ffd640 1 -- 172.21.15.84:0/1398462275 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555e259dfee0 con 0x555e25af3eb0 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e52ffd640 10 client.17343 handle_command_reply: tid=0 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e7ce47740 1 client.17343 shutdown 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e7ce47740 1 -- 172.21.15.84:0/1398462275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 msgr2=0x555e25a854d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/1398462275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 0x555e25a854d0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f6e640985f0 tx=0x7f6e640044d0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e7ce47740 1 -- 172.21.15.84:0/1398462275 shutdown_connections 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/1398462275 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x555e25af3eb0 0x555e25af62f0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/1398462275 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x555e25a7d0e0 0x555e25a77000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/1398462275 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x555e25a72820 0x555e25a76ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e7ce47740 1 --2- 172.21.15.84:0/1398462275 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x555e25a71f00 0x555e25a854d0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e52ffd640 0 client.17343 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e7ce47740 1 -- 172.21.15.84:0/1398462275 >> 172.21.15.84:0/1398462275 conn(0x555e25a6e410 msgr2=0x555e25a8f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e7ce47740 1 -- 172.21.15.84:0/1398462275 shutdown_connections 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e7ce47740 1 -- 172.21.15.84:0/1398462275 wait complete. 2024-04-26T18:44:12.612 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.611+0000 7f6e7ce47740 20 client.17343 trim_cache size 0 max 16384 2024-04-26T18:44:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:44:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:44:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:44:12.613 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:44:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.612+0000 7f6e7a12b640 1 -- 172.21.15.84:0/1854033378 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e5c0a3dc0 msgr2=0x7f6e5c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.612+0000 7f6e7a12b640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e5c0a3dc0 0x7f6e5c0a6280 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c009430 tx=0x7f6e6c02a8e0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.612+0000 7f6e7a12b640 1 -- 172.21.15.84:0/1854033378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e74101810 msgr2=0x7f6e741990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.612+0000 7f6e7a12b640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e74101810 0x7f6e741990a0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f6e6400d950 tx=0x7f6e6400de10 comp rx=0 tx=0).stop 2024-04-26T18:44:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.612+0000 7f6e7a12b640 1 -- 172.21.15.84:0/1854033378 shutdown_connections 2024-04-26T18:44:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.612+0000 7f6e7a12b640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f6e5c0a3dc0 0x7f6e5c0a6280 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.612+0000 7f6e7a12b640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f6e74107820 0x7f6e7419d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.612+0000 7f6e7a12b640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f6e741021d0 0x7f6e741995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.612+0000 7f6e7a12b640 1 --2- 172.21.15.84:0/1854033378 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f6e74101810 0x7f6e741990a0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.614 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.612+0000 7f6e7a12b640 1 -- 172.21.15.84:0/1854033378 >> 172.21.15.84:0/1854033378 conn(0x7f6e740fd300 msgr2=0x7f6e7406b1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.612+0000 7f6e7a12b640 1 -- 172.21.15.84:0/1854033378 shutdown_connections 2024-04-26T18:44:12.615 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:12.612+0000 7f6e7a12b640 1 -- 172.21.15.84:0/1854033378 wait complete. 2024-04-26T18:44:12.801 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.801+0000 7f4ad699a700 1 -- 172.21.15.84:0/1740274805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad0108460 msgr2=0x7f4ad0108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.801+0000 7f4ad699a700 1 --2- 172.21.15.84:0/1740274805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad0108460 0x7f4ad0108840 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009990 tx=0x7f4ac0009ca0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.801+0000 7f4ad699a700 1 -- 172.21.15.84:0/1740274805 shutdown_connections 2024-04-26T18:44:12.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.801+0000 7f4ad699a700 1 --2- 172.21.15.84:0/1740274805 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad0108460 0x7f4ad0108840 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.801+0000 7f4ad699a700 1 --2- 172.21.15.84:0/1740274805 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad0102d90 0x7f4ad010f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.801+0000 7f4ad699a700 1 --2- 172.21.15.84:0/1740274805 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad01023d0 0x7f4ad0102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.801+0000 7f4ad699a700 1 -- 172.21.15.84:0/1740274805 >> 172.21.15.84:0/1740274805 conn(0x7f4ad0079370 msgr2=0x7f4ad0079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:12.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.801+0000 7f4ad699a700 1 -- 172.21.15.84:0/1740274805 shutdown_connections 2024-04-26T18:44:12.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.801+0000 7f4ad699a700 1 -- 172.21.15.84:0/1740274805 wait complete. 2024-04-26T18:44:12.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.802+0000 7f4ad699a700 1 Processor -- start 2024-04-26T18:44:12.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.802+0000 7f4ad699a700 1 -- start start 2024-04-26T18:44:12.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad699a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad01023d0 0x7f4ad019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad699a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad0102d90 0x7f4ad019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad699a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad0108460 0x7f4ad01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad699a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad0113770 con 0x7f4ad0108460 2024-04-26T18:44:12.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad699a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01135f0 con 0x7f4ad01023d0 2024-04-26T18:44:12.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad699a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01138f0 con 0x7f4ad0102d90 2024-04-26T18:44:12.804 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad4f37700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad0108460 0x7f4ad01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad4f37700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad0108460 0x7f4ad01999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39732/0 (socket says 172.21.15.84:39732) 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad4f37700 1 -- 172.21.15.84:0/2590491688 learned_addr learned my addr 172.21.15.84:0/2590491688 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4acffff700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad01023d0 0x7f4ad019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4acf7fe700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad0102d90 0x7f4ad019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad4f37700 1 -- 172.21.15.84:0/2590491688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad0102d90 msgr2=0x7f4ad019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad4f37700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad0102d90 0x7f4ad019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad4f37700 1 -- 172.21.15.84:0/2590491688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad01023d0 msgr2=0x7f4ad019f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad4f37700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad01023d0 0x7f4ad019f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad4f37700 1 -- 172.21.15.84:0/2590491688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ad019a130 con 0x7f4ad0108460 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4ad4f37700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad0108460 0x7f4ad01999d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800b970 tx=0x7f4ac800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4acd7fa700 1 -- 172.21.15.84:0/2590491688 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac800d860 con 0x7f4ad0108460 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4acd7fa700 1 -- 172.21.15.84:0/2590491688 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ac800e430 con 0x7f4ad0108460 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.803+0000 7f4acd7fa700 1 -- 172.21.15.84:0/2590491688 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac8017660 con 0x7f4ad0108460 2024-04-26T18:44:12.805 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.804+0000 7f4ad699a700 1 -- 172.21.15.84:0/2590491688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ad019a420 con 0x7f4ad0108460 2024-04-26T18:44:12.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.804+0000 7f4ad699a700 1 -- 172.21.15.84:0/2590491688 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ad01a6690 con 0x7f4ad0108460 2024-04-26T18:44:12.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.806+0000 7f4acd7fa700 1 -- 172.21.15.84:0/2590491688 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4ac800da00 con 0x7f4ad0108460 2024-04-26T18:44:12.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.806+0000 7f4acd7fa700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4ab80a3e50 0x7f4ab80a6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.806+0000 7f4acffff700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4ab80a3e50 0x7f4ab80a6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.806 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.806+0000 7f4acd7fa700 1 -- 172.21.15.84:0/2590491688 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f4ac80d0430 con 0x7f4ad0108460 2024-04-26T18:44:12.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.807+0000 7f4acffff700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4ab80a3e50 0x7f4ab80a6310 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ad01022b0 tx=0x7f4ac0005960 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:12.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.807+0000 7f4ae122cb80 1 Processor -- start 2024-04-26T18:44:12.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.807+0000 7f4ae122cb80 1 -- start start 2024-04-26T18:44:12.808 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.807+0000 7f4ae122cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c9a4f30 0x56234c9a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.807+0000 7f4ae122cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a5850 0x56234c99f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.807+0000 7f4ae122cb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 0x56234c99fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.807+0000 7f4ae122cb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56234c8f19c0 con 0x56234c99f900 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.807+0000 7f4ae122cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56234c857ac0 con 0x56234c9a5850 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.807+0000 7f4ae122cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56234c8ed980 con 0x56234c9a4f30 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.807+0000 7f4ad4f37700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 0x56234c99fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.807+0000 7f4ad4f37700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 0x56234c99fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39742/0 (socket says 172.21.15.84:39742) 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.807+0000 7f4ad4f37700 1 -- 172.21.15.84:0/2834583475 learned_addr learned my addr 172.21.15.84:0/2834583475 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.807+0000 7f4acf7fe700 1 --2- 172.21.15.84:0/2834583475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a5850 0x56234c99f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.808+0000 7f4acffff700 1 --2- 172.21.15.84:0/2834583475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c9a4f30 0x56234c9a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.808+0000 7f4ad4f37700 1 -- 172.21.15.84:0/2834583475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c9a4f30 msgr2=0x56234c9a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.808+0000 7f4ad4f37700 1 --2- 172.21.15.84:0/2834583475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c9a4f30 0x56234c9a5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.808+0000 7f4ad4f37700 1 -- 172.21.15.84:0/2834583475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a5850 msgr2=0x56234c99f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.809 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.808+0000 7f4ad4f37700 1 --2- 172.21.15.84:0/2834583475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a5850 0x56234c99f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.808+0000 7f4ad4f37700 1 -- 172.21.15.84:0/2834583475 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56234c83ea60 con 0x56234c99f900 2024-04-26T18:44:12.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.808+0000 7f4acf7fe700 1 --2- 172.21.15.84:0/2834583475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a5850 0x56234c99f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:44:12.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.808+0000 7f4ad4f37700 1 --2- 172.21.15.84:0/2834583475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 0x56234c99fd60 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80987f0 tx=0x7f4ac809ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39e8e47c6a3e8030 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:12.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.808+0000 7f4ab67fc700 1 -- 172.21.15.84:0/2834583475 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80a5000 con 0x56234c99f900 2024-04-26T18:44:12.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.808+0000 7f4ab67fc700 1 -- 172.21.15.84:0/2834583475 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ac800e8e0 con 0x56234c99f900 2024-04-26T18:44:12.810 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.808+0000 7f4ab67fc700 1 -- 172.21.15.84:0/2834583475 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac809c5f0 con 0x56234c99f900 2024-04-26T18:44:12.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.809+0000 7f4ae122cb80 1 -- 172.21.15.84:0/2834583475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 msgr2=0x56234c99fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.809+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/2834583475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 0x56234c99fd60 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f4ac80987f0 tx=0x7f4ac809ca90 comp rx=0 tx=0).stop 2024-04-26T18:44:12.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.809+0000 7f4ae122cb80 1 -- 172.21.15.84:0/2834583475 shutdown_connections 2024-04-26T18:44:12.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.809+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/2834583475 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 0x56234c99fd60 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.809+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/2834583475 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c9a4f30 0x56234c9a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.809+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/2834583475 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a5850 0x56234c99f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.809+0000 7f4ae122cb80 1 -- 172.21.15.84:0/2834583475 >> 172.21.15.84:0/2834583475 conn(0x56234c8cd750 msgr2=0x56234c99a090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.809+0000 7f4ae122cb80 1 -- 172.21.15.84:0/2834583475 shutdown_connections 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.809+0000 7f4ae122cb80 1 -- 172.21.15.84:0/2834583475 wait complete. 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.809+0000 7f4ae122cb80 1 -- start start 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4ae122cb80 1 Processor -- start 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4ae122cb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 0x56234c9ab580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4ae122cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a4f30 0x56234c9a8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4ae122cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c9a5850 0x56234c9a8e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4ae122cb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x56234c9ac2c0 con 0x56234c99f900 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4ae122cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x56234c8abc10 con 0x56234c9a4f30 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4ae122cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x56234c9ac440 con 0x56234c9a5850 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4ae122cb80 10 client.0 _ll_register_callbacks cb 0x56234c889ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4acffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 0x56234c9ab580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4acffff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 0x56234c9ab580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39746/0 (socket says 172.21.15.84:39746) 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4acffff700 1 -- 172.21.15.84:0/1919143061 learned_addr learned my addr 172.21.15.84:0/1919143061 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:12.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4ad4f37700 1 --2- 172.21.15.84:0/1919143061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c9a5850 0x56234c9a8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4acffff700 1 -- 172.21.15.84:0/1919143061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c9a5850 msgr2=0x56234c9a8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4acffff700 1 --2- 172.21.15.84:0/1919143061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c9a5850 0x56234c9a8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4acffff700 1 -- 172.21.15.84:0/1919143061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a4f30 msgr2=0x56234c9a8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4acffff700 1 --2- 172.21.15.84:0/1919143061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a4f30 0x56234c9a8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4acffff700 1 -- 172.21.15.84:0/1919143061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56234c8af610 con 0x56234c99f900 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4ad4f37700 1 --2- 172.21.15.84:0/1919143061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c9a5850 0x56234c9a8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4acffff700 1 --2- 172.21.15.84:0/1919143061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 0x56234c9ab580 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009990 tx=0x7f4ac0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5893f499d0f0e3f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4aaf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4aaf7fe700 1 -- 172.21.15.84:0/1919143061 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac0034060 con 0x56234c99f900 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.810+0000 7f4aaf7fe700 1 -- 172.21.15.84:0/1919143061 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f4ac000f660 con 0x56234c99f900 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.811+0000 7f4ae122cb80 1 -- 172.21.15.84:0/1919143061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac800e8e0 con 0x56234c99f900 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.811+0000 7f4aaf7fe700 1 -- 172.21.15.84:0/1919143061 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ac000bea0 con 0x56234c99f900 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.811+0000 7f4ae122cb80 10 client.17355 fetch_fsmap learned FSMap version 29 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.811+0000 7f4ae122cb80 1 -- 172.21.15.84:0/1919143061 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56234c9b6b20 con 0x56234c99f900 2024-04-26T18:44:12.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.811+0000 7f4aaf7fe700 1 -- 172.21.15.84:0/1919143061 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ac800e8e0 con 0x56234c99f900 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.811+0000 7f4aaf7fe700 1 -- 172.21.15.84:0/1919143061 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4ac002f910 con 0x56234c99f900 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.811+0000 7f4ae122cb80 10 client.17355 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.811+0000 7f4ae122cb80 10 client.17355 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.811+0000 7f4ae122cb80 20 client.17355 populate_metadata read hostname 'smithi084' 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.811+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/1919143061 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56234c9b90b0 0x56234c9bb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.811+0000 7f4ae122cb80 4 client.17355 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.811+0000 7f4ae122cb80 1 -- 172.21.15.84:0/1919143061 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56234c8ff3a0 con 0x56234c9b90b0 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.811+0000 7f4acf7fe700 1 --2- 172.21.15.84:0/1919143061 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56234c9b90b0 0x56234c9bb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.812+0000 7f4acf7fe700 1 --2- 172.21.15.84:0/1919143061 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56234c9b90b0 0x56234c9bb500 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12f9f42b64332c31 server_cookie=4ab74fbae6cb104b in_seq=0 out_seq=0 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.812+0000 7f4aaf7fe700 10 client.17355 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.812+0000 7f4aaf7fe700 1 -- 172.21.15.84:0/1919143061 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56234c8ff3a0 con 0x56234c9b90b0 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.812+0000 7f4aaf7fe700 10 client.17355 handle_command_reply: tid=0 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.813+0000 7f4ae122cb80 1 client.17355 shutdown 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.813+0000 7f4ae122cb80 1 -- 172.21.15.84:0/1919143061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 msgr2=0x56234c9ab580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.813+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/1919143061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 0x56234c9ab580 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f4ac0009990 tx=0x7f4ac0009310 comp rx=0 tx=0).stop 2024-04-26T18:44:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.813+0000 7f4ae122cb80 1 -- 172.21.15.84:0/1919143061 shutdown_connections 2024-04-26T18:44:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.813+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/1919143061 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56234c9b90b0 0x56234c9bb500 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.813+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/1919143061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c99f900 0x56234c9ab580 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.813+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/1919143061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c9a5850 0x56234c9a8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.813+0000 7f4aaf7fe700 0 client.17355 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.813+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/1919143061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a4f30 0x56234c9a8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.813+0000 7f4ae122cb80 1 -- 172.21.15.84:0/1919143061 >> 172.21.15.84:0/1919143061 conn(0x56234c8cd750 msgr2=0x56234c99e2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.813+0000 7f4ae122cb80 1 -- 172.21.15.84:0/1919143061 shutdown_connections 2024-04-26T18:44:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.813+0000 7f4ae122cb80 1 -- 172.21.15.84:0/1919143061 wait complete. 2024-04-26T18:44:12.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.813+0000 7f4ae122cb80 20 client.17355 trim_cache size 0 max 16384 2024-04-26T18:44:12.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4ae122cb80 1 Processor -- start 2024-04-26T18:44:12.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4ae122cb80 1 -- start start 2024-04-26T18:44:12.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4ae122cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c99f900 0x56234c99f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4ae122cb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a4f30 0x56234c9a78e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4ae122cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a5850 0x56234c9b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4ae122cb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01addb0 con 0x56234c9a4f30 2024-04-26T18:44:12.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4ae122cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01af8f0 con 0x56234c9a5850 2024-04-26T18:44:12.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4ae122cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01ae480 con 0x56234c99f900 2024-04-26T18:44:12.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4acf7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a4f30 0x56234c9a78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4acf7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a4f30 0x56234c9a78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39756/0 (socket says 172.21.15.84:39756) 2024-04-26T18:44:12.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4acf7fe700 1 -- 172.21.15.84:0/835830893 learned_addr learned my addr 172.21.15.84:0/835830893 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:12.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4acffff700 1 --2- 172.21.15.84:0/835830893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c99f900 0x56234c99f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4ad4f37700 1 --2- 172.21.15.84:0/835830893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a5850 0x56234c9b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4acf7fe700 1 -- 172.21.15.84:0/835830893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c99f900 msgr2=0x56234c99f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4acf7fe700 1 --2- 172.21.15.84:0/835830893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c99f900 0x56234c99f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4acf7fe700 1 -- 172.21.15.84:0/835830893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a5850 msgr2=0x56234c9b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4acf7fe700 1 --2- 172.21.15.84:0/835830893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a5850 0x56234c9b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4acf7fe700 1 -- 172.21.15.84:0/835830893 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56234c996880 con 0x56234c9a4f30 2024-04-26T18:44:12.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.833+0000 7f4acf7fe700 1 --2- 172.21.15.84:0/835830893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a4f30 0x56234c9a78e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x56234c9a9ef0 tx=0x7f4ac4006ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a63ebef029e3f47 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:12.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.834+0000 7f4ab5ffb700 1 -- 172.21.15.84:0/835830893 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac4022070 con 0x56234c9a4f30 2024-04-26T18:44:12.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.834+0000 7f4ab5ffb700 1 -- 172.21.15.84:0/835830893 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ac40148c0 con 0x56234c9a4f30 2024-04-26T18:44:12.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.834+0000 7f4ab5ffb700 1 -- 172.21.15.84:0/835830893 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac40139a0 con 0x56234c9a4f30 2024-04-26T18:44:12.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.834+0000 7f4ae122cb80 1 -- 172.21.15.84:0/835830893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a4f30 msgr2=0x56234c9a78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.834+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/835830893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a4f30 0x56234c9a78e0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x56234c9a9ef0 tx=0x7f4ac4006ac0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.834+0000 7f4ae122cb80 1 -- 172.21.15.84:0/835830893 shutdown_connections 2024-04-26T18:44:12.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.834+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/835830893 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a5850 0x56234c9b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.834+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/835830893 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c99f900 0x56234c99f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.834+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/835830893 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a4f30 0x56234c9a78e0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.834+0000 7f4ae122cb80 1 -- 172.21.15.84:0/835830893 >> 172.21.15.84:0/835830893 conn(0x56234c8cd750 msgr2=0x56234c9a81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.834+0000 7f4ae122cb80 1 -- 172.21.15.84:0/835830893 shutdown_connections 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.834+0000 7f4ae122cb80 1 -- 172.21.15.84:0/835830893 wait complete. 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.835+0000 7f4ae122cb80 1 -- start start 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.835+0000 7f4ae122cb80 1 Processor -- start 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.835+0000 7f4ae122cb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c99f900 0x56234ca1dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.835+0000 7f4ae122cb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a4f30 0x56234ca1e020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.835+0000 7f4ae122cb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a5850 0x56234c9c01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.835+0000 7f4ae122cb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01af220 con 0x56234c9a5850 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.835+0000 7f4ae122cb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01aeb50 con 0x56234c9a4f30 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.835+0000 7f4ae122cb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4ad01abd70 con 0x56234c99f900 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.835+0000 7f4ad4f37700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a5850 0x56234c9c01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.835+0000 7f4ad4f37700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a5850 0x56234c9c01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39770/0 (socket says 172.21.15.84:39770) 2024-04-26T18:44:12.839 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.835+0000 7f4ad4f37700 1 -- 172.21.15.84:0/4180971991 learned_addr learned my addr 172.21.15.84:0/4180971991 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.835+0000 7f4acffff700 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c99f900 0x56234ca1dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.835+0000 7f4ae122cb80 10 client.0 _ll_register_callbacks cb 0x56234c889ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4acf7fe700 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a4f30 0x56234ca1e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4ad4f37700 1 -- 172.21.15.84:0/4180971991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c99f900 msgr2=0x56234ca1dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4ad4f37700 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c99f900 0x56234ca1dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4ad4f37700 1 -- 172.21.15.84:0/4180971991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a4f30 msgr2=0x56234ca1e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4ad4f37700 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a4f30 0x56234ca1e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4ad4f37700 1 -- 172.21.15.84:0/4180971991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56234c9922c0 con 0x56234c9a5850 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4acf7fe700 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a4f30 0x56234ca1e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4ad4f37700 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a5850 0x56234c9c01c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800ecf0 tx=0x7f4ac80098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72246cea2244f6f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4aaf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4aaf7fe700 1 -- 172.21.15.84:0/4180971991 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ac80bb020 con 0x56234c9a5850 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4aaf7fe700 1 -- 172.21.15.84:0/4180971991 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f4ac8098a00 con 0x56234c9a5850 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4ae122cb80 1 -- 172.21.15.84:0/4180971991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ac40148c0 con 0x56234c9a5850 2024-04-26T18:44:12.840 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4aaf7fe700 1 -- 172.21.15.84:0/4180971991 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ac800dcf0 con 0x56234c9a5850 2024-04-26T18:44:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4ae122cb80 10 client.17361 fetch_fsmap learned FSMap version 29 2024-04-26T18:44:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4ae122cb80 1 -- 172.21.15.84:0/4180971991 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56234ca25150 con 0x56234c9a5850 2024-04-26T18:44:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4aaf7fe700 1 -- 172.21.15.84:0/4180971991 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ac40148c0 con 0x56234c9a5850 2024-04-26T18:44:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4aaf7fe700 1 -- 172.21.15.84:0/4180971991 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f4ac801f030 con 0x56234c9a5850 2024-04-26T18:44:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.836+0000 7f4ae122cb80 10 client.17361 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:44:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.837+0000 7f4ae122cb80 10 client.17361 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:44:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.837+0000 7f4ae122cb80 20 client.17361 populate_metadata read hostname 'smithi084' 2024-04-26T18:44:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.837+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56234ca25480 0x56234ca27870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.837+0000 7f4ae122cb80 4 client.17361 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 92.6, "await_for": 42.0}] 2024-04-26T18:44:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.837+0000 7f4ae122cb80 1 -- 172.21.15.84:0/4180971991 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 92.6, "await_for": 42.0}) -- 0x56234c8f9440 con 0x56234ca25480 2024-04-26T18:44:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.837+0000 7f4acffff700 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56234ca25480 0x56234ca27870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.837+0000 7f4aaf7fe700 10 client.17361 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:12.841 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.837+0000 7f4acffff700 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56234ca25480 0x56234ca27870 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0f10db13fb3309f server_cookie=978545f77f564702 in_seq=0 out_seq=0 2024-04-26T18:44:12.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.853+0000 7f4aaf7fe700 1 -- 172.21.15.84:0/4180971991 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x56234c8f9440 con 0x56234ca25480 2024-04-26T18:44:12.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.853+0000 7f4aaf7fe700 10 client.17361 handle_command_reply: tid=0 2024-04-26T18:44:12.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.853+0000 7f4ae122cb80 1 client.17361 shutdown 2024-04-26T18:44:12.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.853+0000 7f4ae122cb80 1 -- 172.21.15.84:0/4180971991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a5850 msgr2=0x56234c9c01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.853+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a5850 0x56234c9c01c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800ecf0 tx=0x7f4ac80098b0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.853+0000 7f4ae122cb80 1 -- 172.21.15.84:0/4180971991 shutdown_connections 2024-04-26T18:44:12.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.853+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x56234ca25480 0x56234ca27870 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.854 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.853+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x56234c9a5850 0x56234c9c01c0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.853+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x56234c99f900 0x56234ca1dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.853+0000 7f4ae122cb80 1 --2- 172.21.15.84:0/4180971991 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x56234c9a4f30 0x56234ca1e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ae122cb80 1 -- 172.21.15.84:0/4180971991 >> 172.21.15.84:0/4180971991 conn(0x56234c8cd750 msgr2=0x56234c9a4350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:12.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4aaf7fe700 0 client.17361 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:12.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ae122cb80 1 -- 172.21.15.84:0/4180971991 shutdown_connections 2024-04-26T18:44:12.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ae122cb80 1 -- 172.21.15.84:0/4180971991 wait complete. 2024-04-26T18:44:12.855 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ae122cb80 20 client.17361 trim_cache size 0 max 16384 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 46, 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout: "8da9d5cd": { 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 46, 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:44:12.856 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 92.6, 2024-04-26T18:44:12.857 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:44:12.857 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:44:12.857 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:44:12.857 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:44:12.857 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "QUIESCED", 2024-04-26T18:44:12.857 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:44:12.857 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:44:12.857 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:44:12.857 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:44:12.857 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:44:12.857 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:44:12.857 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:44:12.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ad699a700 1 -- 172.21.15.84:0/2590491688 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4ab80a3e50 msgr2=0x7f4ab80a6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ad699a700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4ab80a3e50 0x7f4ab80a6310 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ad01022b0 tx=0x7f4ac0005960 comp rx=0 tx=0).stop 2024-04-26T18:44:12.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ad699a700 1 -- 172.21.15.84:0/2590491688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad0108460 msgr2=0x7f4ad01999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:12.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ad699a700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad0108460 0x7f4ad01999d0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f4ac800b970 tx=0x7f4ac800bc80 comp rx=0 tx=0).stop 2024-04-26T18:44:12.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ad699a700 1 -- 172.21.15.84:0/2590491688 shutdown_connections 2024-04-26T18:44:12.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ad699a700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4ab80a3e50 0x7f4ab80a6310 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ad699a700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4ad0108460 0x7f4ad01999d0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ad699a700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4ad01023d0 0x7f4ad019f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ad699a700 1 --2- 172.21.15.84:0/2590491688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4ad0102d90 0x7f4ad019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:12.858 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ad699a700 1 -- 172.21.15.84:0/2590491688 >> 172.21.15.84:0/2590491688 conn(0x7f4ad0079370 msgr2=0x7f4ad0100320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:12.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ad699a700 1 -- 172.21.15.84:0/2590491688 shutdown_connections 2024-04-26T18:44:12.859 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:12.854+0000 7f4ad699a700 1 -- 172.21.15.84:0/2590491688 wait complete. 2024-04-26T18:44:13.565 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '8da9d5cd', quiesce took 5.7 seconds. Will release after: 32.6 2024-04-26T18:44:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:13 smithi084 ceph-mon[29141]: pgmap v511: 97 pgs: 97 active+clean; 2.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-04-26T18:44:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:13 smithi084 ceph-mon[29141]: osdmap e116: 12 total, 12 up, 12 in 2024-04-26T18:44:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:13 smithi103 ceph-mon[32727]: pgmap v511: 97 pgs: 97 active+clean; 2.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-04-26T18:44:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:13 smithi103 ceph-mon[32727]: osdmap e116: 12 total, 12 up, 12 in 2024-04-26T18:44:13.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:13 smithi174 ceph-mon[32705]: pgmap v511: 97 pgs: 97 active+clean; 2.9 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 110 KiB/s wr, 39 op/s 2024-04-26T18:44:13.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:13 smithi174 ceph-mon[32705]: osdmap e116: 12 total, 12 up, 12 in 2024-04-26T18:44:15.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:15 smithi084 ceph-mon[29141]: pgmap v513: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 19 op/s 2024-04-26T18:44:15.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:15 smithi103 ceph-mon[32727]: pgmap v513: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 19 op/s 2024-04-26T18:44:15.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:15 smithi174 ceph-mon[32705]: pgmap v513: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 61 KiB/s wr, 19 op/s 2024-04-26T18:44:16.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:16 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:44:16.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:16 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:44:16.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:16 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:44:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:17 smithi084 ceph-mon[29141]: pgmap v514: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 19 op/s 2024-04-26T18:44:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:17 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:44:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:17 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:44:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:17 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:44:17.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:17 smithi103 ceph-mon[32727]: pgmap v514: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 19 op/s 2024-04-26T18:44:17.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:17 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:44:17.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:17 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:44:17.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:17 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:44:17.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:17 smithi174 ceph-mon[32705]: pgmap v514: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 19 op/s 2024-04-26T18:44:17.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:17 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:44:17.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:17 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:44:17.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:17 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:44:19.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:19 smithi084 ceph-mon[29141]: pgmap v515: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 19 op/s 2024-04-26T18:44:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:19 smithi103 ceph-mon[32727]: pgmap v515: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 19 op/s 2024-04-26T18:44:19.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:19 smithi174 ceph-mon[32705]: pgmap v515: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 60 KiB/s wr, 19 op/s 2024-04-26T18:44:21.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:21 smithi084 ceph-mon[29141]: pgmap v516: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-26T18:44:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:21 smithi103 ceph-mon[32727]: pgmap v516: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-26T18:44:21.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:21 smithi174 ceph-mon[32705]: pgmap v516: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-26T18:44:23.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:23 smithi084 ceph-mon[29141]: pgmap v517: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-26T18:44:23.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:23 smithi103 ceph-mon[32727]: pgmap v517: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-26T18:44:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:23 smithi174 ceph-mon[32705]: pgmap v517: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-26T18:44:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.707+0000 7f01b2938640 1 -- 172.21.15.84:0/485972906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01ac103be0 msgr2=0x7f01ac103fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.707+0000 7f01b2938640 1 --2- 172.21.15.84:0/485972906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01ac103be0 0x7f01ac103fc0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f019c009900 tx=0x7f019c02a480 comp rx=0 tx=0).stop 2024-04-26T18:44:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.707+0000 7f01b2938640 1 -- 172.21.15.84:0/485972906 shutdown_connections 2024-04-26T18:44:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.707+0000 7f01b2938640 1 --2- 172.21.15.84:0/485972906 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ac10cfb0 0x7f01ac10f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.707+0000 7f01b2938640 1 --2- 172.21.15.84:0/485972906 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ac104500 0x7f01ac10c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.710 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.707+0000 7f01b2938640 1 --2- 172.21.15.84:0/485972906 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01ac103be0 0x7f01ac103fc0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.707+0000 7f01b2938640 1 -- 172.21.15.84:0/485972906 >> 172.21.15.84:0/485972906 conn(0x7f01ac077bb0 msgr2=0x7f01ac077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.707+0000 7f01b2938640 1 -- 172.21.15.84:0/485972906 shutdown_connections 2024-04-26T18:44:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.707+0000 7f01b2938640 1 -- 172.21.15.84:0/485972906 wait complete. 2024-04-26T18:44:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.707+0000 7f01b2938640 1 Processor -- start 2024-04-26T18:44:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.707+0000 7f01b2938640 1 -- start start 2024-04-26T18:44:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01b2938640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01ac103be0 0x7f01ac1990b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01b2938640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ac104500 0x7f01ac1995f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01b2938640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ac10cfb0 0x7f01ac19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01b2938640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac112bf0 con 0x7f01ac103be0 2024-04-26T18:44:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01b2938640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac112a70 con 0x7f01ac104500 2024-04-26T18:44:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01b2938640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac112d70 con 0x7f01ac10cfb0 2024-04-26T18:44:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01ac103be0 0x7f01ac1990b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.711 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01ac103be0 0x7f01ac1990b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39682/0 (socket says 172.21.15.84:39682) 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01abfff640 1 -- 172.21.15.84:0/3516959925 learned_addr learned my addr 172.21.15.84:0/3516959925 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01b0ed5640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ac10cfb0 0x7f01ac19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01abfff640 1 -- 172.21.15.84:0/3516959925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ac10cfb0 msgr2=0x7f01ac19d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01abfff640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ac10cfb0 0x7f01ac19d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01ab7fe640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ac104500 0x7f01ac1995f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01abfff640 1 -- 172.21.15.84:0/3516959925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ac104500 msgr2=0x7f01ac1995f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01abfff640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ac104500 0x7f01ac1995f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01abfff640 1 -- 172.21.15.84:0/3516959925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01ac19e050 con 0x7f01ac103be0 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01abfff640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01ac103be0 0x7f01ac1990b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f019c005940 tx=0x7f019c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.708+0000 7f01a97fa640 1 -- 172.21.15.84:0/3516959925 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c03f030 con 0x7f01ac103be0 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.709+0000 7f01a97fa640 1 -- 172.21.15.84:0/3516959925 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f019c003aa0 con 0x7f01ac103be0 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.709+0000 7f01b2938640 1 -- 172.21.15.84:0/3516959925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01ac19e2e0 con 0x7f01ac103be0 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.709+0000 7f01a97fa640 1 -- 172.21.15.84:0/3516959925 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c035070 con 0x7f01ac103be0 2024-04-26T18:44:25.712 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.709+0000 7f01b2938640 1 -- 172.21.15.84:0/3516959925 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01ac1a5bd0 con 0x7f01ac103be0 2024-04-26T18:44:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.711+0000 7f01a97fa640 1 -- 172.21.15.84:0/3516959925 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f019c042030 con 0x7f01ac103be0 2024-04-26T18:44:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.711+0000 7f01b5655740 1 Processor -- start 2024-04-26T18:44:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.711+0000 7f01b5655740 1 -- start start 2024-04-26T18:44:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01b5655740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed7806f00 0x557ed78072e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.719 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01b5655740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7807820 0x557ed7811ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01b5655740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed78120e0 0x557ed78144d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01b5655740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557ed776d460 con 0x557ed7807820 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01b5655740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557ed7727590 con 0x557ed7806f00 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01b5655740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ed7769420 con 0x557ed78120e0 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01a97fa640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f01940a3dc0 0x7f01940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01b0ed5640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f01940a3dc0 0x7f01940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7807820 0x557ed7811ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7807820 0x557ed7811ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39692/0 (socket says 172.21.15.84:39692) 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01ab7fe640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed78120e0 0x557ed78144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01abfff640 1 -- 172.21.15.84:0/550899257 learned_addr learned my addr 172.21.15.84:0/550899257 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01b0ed5640 1 --2- 172.21.15.84:0/550899257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed7806f00 0x557ed78072e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01a97fa640 1 -- 172.21.15.84:0/3516959925 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f019c0f0fe0 con 0x7f01ac103be0 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01abfff640 1 -- 172.21.15.84:0/550899257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed78120e0 msgr2=0x557ed78144d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:25.720 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01abfff640 1 --2- 172.21.15.84:0/550899257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed78120e0 0x557ed78144d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01abfff640 1 -- 172.21.15.84:0/550899257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed7806f00 msgr2=0x557ed78072e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01abfff640 1 --2- 172.21.15.84:0/550899257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed7806f00 0x557ed78072e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01abfff640 1 -- 172.21.15.84:0/550899257 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed77340d0 con 0x557ed7807820 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01abfff640 1 --2- 172.21.15.84:0/550899257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7807820 0x557ed7811ba0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f019c0bced0 tx=0x7f019c08fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8232eea232d831ec server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01b0ed5640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f01940a3dc0 0x7f01940a6280 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009270 tx=0x7f01a4009db0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.712+0000 7f01927fc640 1 -- 172.21.15.84:0/550899257 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c049070 con 0x557ed7807820 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.713+0000 7f01927fc640 1 -- 172.21.15.84:0/550899257 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f019c0bd640 con 0x557ed7807820 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.713+0000 7f01927fc640 1 -- 172.21.15.84:0/550899257 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c0ee970 con 0x557ed7807820 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.713+0000 7f01b5655740 1 -- 172.21.15.84:0/550899257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7807820 msgr2=0x557ed7811ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.713+0000 7f01b5655740 1 --2- 172.21.15.84:0/550899257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7807820 0x557ed7811ba0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f019c0bced0 tx=0x7f019c08fa30 comp rx=0 tx=0).stop 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.713+0000 7f01b5655740 1 -- 172.21.15.84:0/550899257 shutdown_connections 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.713+0000 7f01b5655740 1 --2- 172.21.15.84:0/550899257 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed78120e0 0x557ed78144d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.713+0000 7f01b5655740 1 --2- 172.21.15.84:0/550899257 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7807820 0x557ed7811ba0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.721 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.713+0000 7f01b5655740 1 --2- 172.21.15.84:0/550899257 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed7806f00 0x557ed78072e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.713+0000 7f01b5655740 1 -- 172.21.15.84:0/550899257 >> 172.21.15.84:0/550899257 conn(0x557ed7803410 msgr2=0x557ed78037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.713+0000 7f01b5655740 1 -- 172.21.15.84:0/550899257 shutdown_connections 2024-04-26T18:44:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.713+0000 7f01b5655740 1 -- 172.21.15.84:0/550899257 wait complete. 2024-04-26T18:44:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01b5655740 1 -- start start 2024-04-26T18:44:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01b5655740 1 Processor -- start 2024-04-26T18:44:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01b5655740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed781beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01b5655740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed7807820 0x557ed781c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01b5655740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 0x557ed7816220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01b5655740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x557ed7814ad0 con 0x557ed7806f00 2024-04-26T18:44:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01b5655740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x557ed76f4b60 con 0x557ed78120e0 2024-04-26T18:44:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01b5655740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x557ed7814c50 con 0x557ed7807820 2024-04-26T18:44:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01b5655740 10 client.0 _ll_register_callbacks cb 0x557ed77192b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:44:25.722 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed781beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed781beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39702/0 (socket says 172.21.15.84:39702) 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01abfff640 1 -- 172.21.15.84:0/2204254944 learned_addr learned my addr 172.21.15.84:0/2204254944 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01b0ed5640 1 --2- 172.21.15.84:0/2204254944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 0x557ed7816220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.714+0000 7f01ab7fe640 1 --2- 172.21.15.84:0/2204254944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed7807820 0x557ed781c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f01abfff640 1 -- 172.21.15.84:0/2204254944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed7807820 msgr2=0x557ed781c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f01abfff640 1 --2- 172.21.15.84:0/2204254944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed7807820 0x557ed781c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f01abfff640 1 -- 172.21.15.84:0/2204254944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 msgr2=0x557ed7816220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f01abfff640 1 --2- 172.21.15.84:0/2204254944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 0x557ed7816220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f01abfff640 1 -- 172.21.15.84:0/2204254944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed747f210 con 0x557ed7806f00 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f01abfff640 1 --2- 172.21.15.84:0/2204254944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed781beb0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f019c0ee7a0 tx=0x7f019c003e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5667d66c1cf7092 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f018b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f018b7fe640 1 -- 172.21.15.84:0/2204254944 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c049040 con 0x557ed7806f00 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f01b5655740 1 -- 172.21.15.84:0/2204254944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f019c0bd640 con 0x557ed7806f00 2024-04-26T18:44:25.723 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f018b7fe640 1 -- 172.21.15.84:0/2204254944 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f019c0b95b0 con 0x557ed7806f00 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f018b7fe640 1 -- 172.21.15.84:0/2204254944 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f019c0b91e0 con 0x557ed7806f00 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f01b5655740 10 client.17373 fetch_fsmap learned FSMap version 29 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f01b5655740 1 -- 172.21.15.84:0/2204254944 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557ed754b620 con 0x557ed7806f00 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f018b7fe640 1 -- 172.21.15.84:0/2204254944 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f019c0bd640 con 0x557ed7806f00 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f018b7fe640 1 -- 172.21.15.84:0/2204254944 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f019c0c5720 con 0x557ed7806f00 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f01b5655740 10 client.17373 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.715+0000 7f01b5655740 10 client.17373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.716+0000 7f01b5655740 20 client.17373 populate_metadata read hostname 'smithi084' 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.716+0000 7f01b5655740 1 --2- 172.21.15.84:0/2204254944 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557ed78262b0 0x557ed78286a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.716+0000 7f01b5655740 4 client.17373 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.716+0000 7f01b5655740 1 -- 172.21.15.84:0/2204254944 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ed777b070 con 0x557ed78262b0 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.716+0000 7f01ab7fe640 1 --2- 172.21.15.84:0/2204254944 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557ed78262b0 0x557ed78286a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.716+0000 7f01ab7fe640 1 --2- 172.21.15.84:0/2204254944 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557ed78262b0 0x557ed78286a0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef1f850640324d26 server_cookie=93e75761a29889f0 in_seq=0 out_seq=0 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.716+0000 7f018b7fe640 10 client.17373 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:25.724 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f018b7fe640 1 -- 172.21.15.84:0/2204254944 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557ed777b070 con 0x557ed78262b0 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f018b7fe640 10 client.17373 handle_command_reply: tid=0 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f01b5655740 1 client.17373 shutdown 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f01b5655740 1 -- 172.21.15.84:0/2204254944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 msgr2=0x557ed781beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f01b5655740 1 --2- 172.21.15.84:0/2204254944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed781beb0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f019c0ee7a0 tx=0x7f019c003e80 comp rx=0 tx=0).stop 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f01b5655740 1 -- 172.21.15.84:0/2204254944 shutdown_connections 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f01b5655740 1 --2- 172.21.15.84:0/2204254944 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557ed78262b0 0x557ed78286a0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f01b5655740 1 --2- 172.21.15.84:0/2204254944 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 0x557ed7816220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f01b5655740 1 --2- 172.21.15.84:0/2204254944 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed7807820 0x557ed781c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f018b7fe640 0 client.17373 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f01b5655740 1 --2- 172.21.15.84:0/2204254944 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed781beb0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f01b5655740 1 -- 172.21.15.84:0/2204254944 >> 172.21.15.84:0/2204254944 conn(0x557ed7803410 msgr2=0x557ed7809190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f01b5655740 1 -- 172.21.15.84:0/2204254944 shutdown_connections 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f01b5655740 1 -- 172.21.15.84:0/2204254944 wait complete. 2024-04-26T18:44:25.725 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.717+0000 7f01b5655740 20 client.17373 trim_cache size 0 max 16384 2024-04-26T18:44:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.732+0000 7f01b5655740 1 Processor -- start 2024-04-26T18:44:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01b5655740 1 -- start start 2024-04-26T18:44:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01b5655740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed77fe7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01b5655740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed7807820 0x557ed78161c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01b5655740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 0x557ed782dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01b5655740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1ac3e0 con 0x557ed7806f00 2024-04-26T18:44:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01b5655740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1adf20 con 0x557ed78120e0 2024-04-26T18:44:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01b5655740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1acab0 con 0x557ed7807820 2024-04-26T18:44:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed77fe7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed77fe7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39706/0 (socket says 172.21.15.84:39706) 2024-04-26T18:44:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01abfff640 1 -- 172.21.15.84:0/4019826696 learned_addr learned my addr 172.21.15.84:0/4019826696 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01abfff640 1 -- 172.21.15.84:0/4019826696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed7807820 msgr2=0x557ed78161c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01abfff640 1 --2- 172.21.15.84:0/4019826696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed7807820 0x557ed78161c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01abfff640 1 -- 172.21.15.84:0/4019826696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 msgr2=0x557ed782dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01abfff640 1 --2- 172.21.15.84:0/4019826696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 0x557ed782dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01abfff640 1 -- 172.21.15.84:0/4019826696 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ed7811a40 con 0x557ed7806f00 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f01abfff640 1 --2- 172.21.15.84:0/4019826696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed77fe7b0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f019c0eecf0 tx=0x7f019c0ee710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a2a8af955a5b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f0191ffb640 1 -- 172.21.15.84:0/4019826696 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c0da070 con 0x557ed7806f00 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f0191ffb640 1 -- 172.21.15.84:0/4019826696 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f019c0bcf40 con 0x557ed7806f00 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.733+0000 7f0191ffb640 1 -- 172.21.15.84:0/4019826696 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c049050 con 0x557ed7806f00 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.734+0000 7f01b5655740 1 -- 172.21.15.84:0/4019826696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 msgr2=0x557ed77fe7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.734+0000 7f01b5655740 1 --2- 172.21.15.84:0/4019826696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed77fe7b0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f019c0eecf0 tx=0x7f019c0ee710 comp rx=0 tx=0).stop 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.734+0000 7f01b5655740 1 -- 172.21.15.84:0/4019826696 shutdown_connections 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.734+0000 7f01b5655740 1 --2- 172.21.15.84:0/4019826696 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 0x557ed782dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.734+0000 7f01b5655740 1 --2- 172.21.15.84:0/4019826696 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed7807820 0x557ed78161c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.734+0000 7f01b5655740 1 --2- 172.21.15.84:0/4019826696 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed77fe7b0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.734+0000 7f01b5655740 1 -- 172.21.15.84:0/4019826696 >> 172.21.15.84:0/4019826696 conn(0x557ed7803410 msgr2=0x557ed7815ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.734+0000 7f01b5655740 1 -- 172.21.15.84:0/4019826696 shutdown_connections 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.734+0000 7f01b5655740 1 -- 172.21.15.84:0/4019826696 wait complete. 2024-04-26T18:44:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.734+0000 7f01b5655740 1 -- start start 2024-04-26T18:44:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01b5655740 1 Processor -- start 2024-04-26T18:44:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01b5655740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed7822560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01b5655740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed7807820 0x557ed781f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01b5655740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 0x557ed781fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01b5655740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1ad850 con 0x557ed7806f00 2024-04-26T18:44:25.740 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01b5655740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1ad180 con 0x557ed78120e0 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01b5655740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f01ac1aa400 con 0x557ed7807820 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01b5655740 10 client.0 _ll_register_callbacks cb 0x7f01a40126d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed7822560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01abfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed7822560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39716/0 (socket says 172.21.15.84:39716) 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01abfff640 1 -- 172.21.15.84:0/3146484568 learned_addr learned my addr 172.21.15.84:0/3146484568 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01b0ed5640 1 --2- 172.21.15.84:0/3146484568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 0x557ed781fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01abfff640 1 -- 172.21.15.84:0/3146484568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed7807820 msgr2=0x557ed781f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01abfff640 1 --2- 172.21.15.84:0/3146484568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed7807820 0x557ed781f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01abfff640 1 -- 172.21.15.84:0/3146484568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 msgr2=0x557ed781fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01abfff640 1 --2- 172.21.15.84:0/3146484568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 0x557ed781fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01abfff640 1 -- 172.21.15.84:0/3146484568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ed78206d0 con 0x557ed7806f00 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f01abfff640 1 --2- 172.21.15.84:0/3146484568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed7822560 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f019c0bb1f0 tx=0x7f019c02a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe474ecdfe1e588 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f018b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.735+0000 7f018b7fe640 1 -- 172.21.15.84:0/3146484568 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c049070 con 0x557ed7806f00 2024-04-26T18:44:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f018b7fe640 1 -- 172.21.15.84:0/3146484568 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f019c0bb240 con 0x557ed7806f00 2024-04-26T18:44:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f01b5655740 1 -- 172.21.15.84:0/3146484568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f019c0bcf40 con 0x557ed7806f00 2024-04-26T18:44:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f018b7fe640 1 -- 172.21.15.84:0/3146484568 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f019c0da050 con 0x557ed7806f00 2024-04-26T18:44:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f01b5655740 10 client.17379 fetch_fsmap learned FSMap version 29 2024-04-26T18:44:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f01b5655740 1 -- 172.21.15.84:0/3146484568 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557ed78206d0 con 0x557ed7806f00 2024-04-26T18:44:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f018b7fe640 1 -- 172.21.15.84:0/3146484568 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f019c0bcf40 con 0x557ed7806f00 2024-04-26T18:44:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f018b7fe640 1 -- 172.21.15.84:0/3146484568 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f019c0b8c90 con 0x557ed7806f00 2024-04-26T18:44:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f01b5655740 10 client.17379 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:44:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f01b5655740 10 client.17379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:44:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f01b5655740 20 client.17379 populate_metadata read hostname 'smithi084' 2024-04-26T18:44:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f01b5655740 1 --2- 172.21.15.84:0/3146484568 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557ed788c830 0x557ed788ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f01b5655740 4 client.17379 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:44:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f01b5655740 1 -- 172.21.15.84:0/3146484568 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557ed7774ee0 con 0x557ed788c830 2024-04-26T18:44:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.736+0000 7f01ab7fe640 1 --2- 172.21.15.84:0/3146484568 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557ed788c830 0x557ed788ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.737+0000 7f018b7fe640 10 client.17379 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.737+0000 7f01ab7fe640 1 --2- 172.21.15.84:0/3146484568 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557ed788c830 0x557ed788ec20 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f02c67da261a4f3 server_cookie=56c5ae9f3d817836 in_seq=0 out_seq=0 2024-04-26T18:44:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.750+0000 7f018b7fe640 1 -- 172.21.15.84:0/3146484568 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x557ed7774ee0 con 0x557ed788c830 2024-04-26T18:44:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.750+0000 7f018b7fe640 10 client.17379 handle_command_reply: tid=0 2024-04-26T18:44:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.750+0000 7f01b5655740 1 client.17379 shutdown 2024-04-26T18:44:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.750+0000 7f01b5655740 1 -- 172.21.15.84:0/3146484568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 msgr2=0x557ed7822560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.750+0000 7f01b5655740 1 --2- 172.21.15.84:0/3146484568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed7822560 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f019c0bb1f0 tx=0x7f019c02a480 comp rx=0 tx=0).stop 2024-04-26T18:44:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b5655740 1 -- 172.21.15.84:0/3146484568 shutdown_connections 2024-04-26T18:44:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b5655740 1 --2- 172.21.15.84:0/3146484568 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x557ed788c830 0x557ed788ec20 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b5655740 1 --2- 172.21.15.84:0/3146484568 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x557ed78120e0 0x557ed781fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f018b7fe640 0 client.17379 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b5655740 1 --2- 172.21.15.84:0/3146484568 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x557ed7807820 0x557ed781f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.752 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b5655740 1 --2- 172.21.15.84:0/3146484568 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x557ed7806f00 0x557ed7822560 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b5655740 1 -- 172.21.15.84:0/3146484568 >> 172.21.15.84:0/3146484568 conn(0x557ed7803410 msgr2=0x557ed781ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b5655740 1 -- 172.21.15.84:0/3146484568 shutdown_connections 2024-04-26T18:44:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b5655740 1 -- 172.21.15.84:0/3146484568 wait complete. 2024-04-26T18:44:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b5655740 20 client.17379 trim_cache size 0 max 16384 2024-04-26T18:44:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:44:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:44:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:44:25.753 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:44:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b2938640 1 -- 172.21.15.84:0/3516959925 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f01940a3dc0 msgr2=0x7f01940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b2938640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f01940a3dc0 0x7f01940a6280 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009270 tx=0x7f01a4009db0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.754 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b2938640 1 -- 172.21.15.84:0/3516959925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01ac103be0 msgr2=0x7f01ac1990b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b2938640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01ac103be0 0x7f01ac1990b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f019c005940 tx=0x7f019c004270 comp rx=0 tx=0).stop 2024-04-26T18:44:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b2938640 1 -- 172.21.15.84:0/3516959925 shutdown_connections 2024-04-26T18:44:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b2938640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f01940a3dc0 0x7f01940a6280 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b2938640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f01ac10cfb0 0x7f01ac19d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b2938640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f01ac104500 0x7f01ac1995f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b2938640 1 --2- 172.21.15.84:0/3516959925 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f01ac103be0 0x7f01ac1990b0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b2938640 1 -- 172.21.15.84:0/3516959925 >> 172.21.15.84:0/3516959925 conn(0x7f01ac077bb0 msgr2=0x7f01ac0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b2938640 1 -- 172.21.15.84:0/3516959925 shutdown_connections 2024-04-26T18:44:25.755 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:25.751+0000 7f01b2938640 1 -- 172.21.15.84:0/3516959925 wait complete. 2024-04-26T18:44:25.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:25 smithi084 ceph-mon[29141]: pgmap v518: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-04-26T18:44:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:25 smithi103 ceph-mon[32727]: pgmap v518: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-04-26T18:44:25.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:25 smithi174 ceph-mon[32705]: pgmap v518: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-04-26T18:44:27.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:27 smithi084 ceph-mon[29141]: pgmap v519: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-26T18:44:27.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:27 smithi103 ceph-mon[32727]: pgmap v519: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-26T18:44:27.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:27 smithi174 ceph-mon[32705]: pgmap v519: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-26T18:44:29.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:29 smithi084 ceph-mon[29141]: pgmap v520: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-26T18:44:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:29 smithi103 ceph-mon[32727]: pgmap v520: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-26T18:44:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:29 smithi174 ceph-mon[32705]: pgmap v520: 97 pgs: 97 active+clean; 3.2 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-04-26T18:44:31.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:31 smithi084 ceph-mon[29141]: pgmap v521: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-26T18:44:31.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:31 smithi103 ceph-mon[32727]: pgmap v521: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-26T18:44:31.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:31 smithi174 ceph-mon[32705]: pgmap v521: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-26T18:44:32.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:32 smithi084 ceph-mon[29141]: pgmap v522: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-26T18:44:32.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:32 smithi103 ceph-mon[32727]: pgmap v522: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-26T18:44:32.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:32 smithi174 ceph-mon[32705]: pgmap v522: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 9.8 KiB/s rd, 8.7 KiB/s wr, 11 op/s 2024-04-26T18:44:35.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:34 smithi084 ceph-mon[29141]: pgmap v523: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-26T18:44:35.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:34 smithi103 ceph-mon[32727]: pgmap v523: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-26T18:44:35.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:34 smithi174 ceph-mon[32705]: pgmap v523: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 15 op/s 2024-04-26T18:44:37.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:36 smithi084 ceph-mon[29141]: pgmap v524: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-26T18:44:37.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:36 smithi103 ceph-mon[32727]: pgmap v524: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-26T18:44:37.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:36 smithi174 ceph-mon[32705]: pgmap v524: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-26T18:44:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.849+0000 7f404d5bf640 1 -- 172.21.15.84:0/141844932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40481032d0 msgr2=0x7f404810f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.849+0000 7f404d5bf640 1 --2- 172.21.15.84:0/141844932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40481032d0 0x7f404810f7e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4040009cd0 tx=0x7f404002a8e0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.849+0000 7f404d5bf640 1 -- 172.21.15.84:0/141844932 shutdown_connections 2024-04-26T18:44:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.849+0000 7f404d5bf640 1 --2- 172.21.15.84:0/141844932 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40481032d0 0x7f404810f7e0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.849+0000 7f404d5bf640 1 --2- 172.21.15.84:0/141844932 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4048102910 0x7f4048102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.849+0000 7f404d5bf640 1 --2- 172.21.15.84:0/141844932 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4048108920 0x7f4048108d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.849+0000 7f404d5bf640 1 -- 172.21.15.84:0/141844932 >> 172.21.15.84:0/141844932 conn(0x7f40480fe400 msgr2=0x7f4048100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.849+0000 7f404d5bf640 1 -- 172.21.15.84:0/141844932 shutdown_connections 2024-04-26T18:44:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.849+0000 7f404d5bf640 1 -- 172.21.15.84:0/141844932 wait complete. 2024-04-26T18:44:38.850 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.850+0000 7f404d5bf640 1 Processor -- start 2024-04-26T18:44:38.851 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.850+0000 7f404d5bf640 1 -- start start 2024-04-26T18:44:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.850+0000 7f404d5bf640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4048102910 0x7f40481990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.850+0000 7f404d5bf640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40481032d0 0x7f40481995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.850+0000 7f404d5bf640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4048108920 0x7f404819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.850+0000 7f404d5bf640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4048112be0 con 0x7f40481032d0 2024-04-26T18:44:38.853 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.850+0000 7f404d5bf640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4048112a60 con 0x7f4048108920 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.850+0000 7f404d5bf640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4048112d60 con 0x7f4048102910 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.850+0000 7f40467fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40481032d0 0x7f40481995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.850+0000 7f40467fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40481032d0 0x7f40481995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41300/0 (socket says 172.21.15.84:41300) 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.850+0000 7f40467fc640 1 -- 172.21.15.84:0/3747669664 learned_addr learned my addr 172.21.15.84:0/3747669664 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f40477fe640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4048108920 0x7f404819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f4046ffd640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4048102910 0x7f40481990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f40467fc640 1 -- 172.21.15.84:0/3747669664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4048102910 msgr2=0x7f40481990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f40467fc640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4048102910 0x7f40481990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f40467fc640 1 -- 172.21.15.84:0/3747669664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4048108920 msgr2=0x7f404819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f40467fc640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4048108920 0x7f404819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f40467fc640 1 -- 172.21.15.84:0/3747669664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f404819e090 con 0x7f40481032d0 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f40477fe640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4048108920 0x7f404819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f40467fc640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40481032d0 0x7f40481995e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f403800b4a0 tx=0x7f403800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:38.854 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f402ffff640 1 -- 172.21.15.84:0/3747669664 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038014070 con 0x7f40481032d0 2024-04-26T18:44:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f402ffff640 1 -- 172.21.15.84:0/3747669664 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4038010ac0 con 0x7f40481032d0 2024-04-26T18:44:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f404d5bf640 1 -- 172.21.15.84:0/3747669664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f404819e380 con 0x7f40481032d0 2024-04-26T18:44:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f402ffff640 1 -- 172.21.15.84:0/3747669664 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038017c30 con 0x7f40481032d0 2024-04-26T18:44:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.851+0000 7f404d5bf640 1 -- 172.21.15.84:0/3747669664 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40481a5c10 con 0x7f40481032d0 2024-04-26T18:44:38.855 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.853+0000 7f402ffff640 1 -- 172.21.15.84:0/3747669664 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4038017070 con 0x7f40481032d0 2024-04-26T18:44:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40503bc740 1 Processor -- start 2024-04-26T18:44:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40503bc740 1 -- start start 2024-04-26T18:44:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f402ffff640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f40300a3dc0 0x7f40300a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40503bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309eff00 0x556d309f02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40503bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 0x556d309faba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40503bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309fb0e0 0x556d309fd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40503bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556d30956460 con 0x556d309f0820 2024-04-26T18:44:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40503bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556d30910590 con 0x556d309eff00 2024-04-26T18:44:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40503bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556d30952420 con 0x556d309fb0e0 2024-04-26T18:44:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40477fe640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f40300a3dc0 0x7f40300a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f402ffff640 1 -- 172.21.15.84:0/3747669664 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f40380cfec0 con 0x7f40481032d0 2024-04-26T18:44:38.862 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40467fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 0x556d309faba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40467fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 0x556d309faba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41308/0 (socket says 172.21.15.84:41308) 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40467fc640 1 -- 172.21.15.84:0/800883442 learned_addr learned my addr 172.21.15.84:0/800883442 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f4046ffd640 1 --2- 172.21.15.84:0/800883442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309eff00 0x556d309f02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40477fe640 1 --2- 172.21.15.84:0/800883442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309fb0e0 0x556d309fd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40467fc640 1 -- 172.21.15.84:0/800883442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309fb0e0 msgr2=0x556d309fd4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40467fc640 1 --2- 172.21.15.84:0/800883442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309fb0e0 0x556d309fd4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40467fc640 1 -- 172.21.15.84:0/800883442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309eff00 msgr2=0x556d309f02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40467fc640 1 --2- 172.21.15.84:0/800883442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309eff00 0x556d309f02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.854+0000 7f40467fc640 1 -- 172.21.15.84:0/800883442 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d3091d0d0 con 0x556d309f0820 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.855+0000 7f40467fc640 1 --2- 172.21.15.84:0/800883442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 0x556d309faba0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f403809bec0 tx=0x7f40380047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62b19b55e9d554b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.855+0000 7f40477fe640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f40300a3dc0 0x7f40300a6280 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4040006010 tx=0x7f4040031720 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.855+0000 7f402d7fa640 1 -- 172.21.15.84:0/800883442 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403801e070 con 0x556d309f0820 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.855+0000 7f402d7fa640 1 -- 172.21.15.84:0/800883442 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f403809c630 con 0x556d309f0820 2024-04-26T18:44:38.863 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.855+0000 7f402d7fa640 1 -- 172.21.15.84:0/800883442 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4038010c70 con 0x556d309f0820 2024-04-26T18:44:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.855+0000 7f40503bc740 1 -- 172.21.15.84:0/800883442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 msgr2=0x556d309faba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.855+0000 7f40503bc740 1 --2- 172.21.15.84:0/800883442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 0x556d309faba0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f403809bec0 tx=0x7f40380047d0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.855+0000 7f40503bc740 1 -- 172.21.15.84:0/800883442 shutdown_connections 2024-04-26T18:44:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.855+0000 7f40503bc740 1 --2- 172.21.15.84:0/800883442 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309fb0e0 0x556d309fd4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.855+0000 7f40503bc740 1 --2- 172.21.15.84:0/800883442 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 0x556d309faba0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.855+0000 7f40503bc740 1 --2- 172.21.15.84:0/800883442 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309eff00 0x556d309f02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.864 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.855+0000 7f40503bc740 1 -- 172.21.15.84:0/800883442 >> 172.21.15.84:0/800883442 conn(0x556d309ec410 msgr2=0x556d309ec7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f40503bc740 1 -- 172.21.15.84:0/800883442 shutdown_connections 2024-04-26T18:44:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f40503bc740 1 -- 172.21.15.84:0/800883442 wait complete. 2024-04-26T18:44:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f40503bc740 1 -- start start 2024-04-26T18:44:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f40503bc740 1 Processor -- start 2024-04-26T18:44:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f40503bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 0x556d309f8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f40503bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309f0820 0x556d309f8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f40503bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309fb0e0 0x556d309f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f40503bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x556d309fdd10 con 0x556d309eff00 2024-04-26T18:44:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f40503bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x556d308ddb60 con 0x556d309fb0e0 2024-04-26T18:44:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f40503bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x556d309fde90 con 0x556d309f0820 2024-04-26T18:44:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f40503bc740 10 client.0 _ll_register_callbacks cb 0x556d309022b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:44:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 0x556d309f8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.865 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 0x556d309f8030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41312/0 (socket says 172.21.15.84:41312) 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f4046ffd640 1 -- 172.21.15.84:0/929064366 learned_addr learned my addr 172.21.15.84:0/929064366 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f40477fe640 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309fb0e0 0x556d309f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.856+0000 7f40467fc640 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309f0820 0x556d309f8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f4046ffd640 1 -- 172.21.15.84:0/929064366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309f0820 msgr2=0x556d309f8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f4046ffd640 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309f0820 0x556d309f8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f4046ffd640 1 -- 172.21.15.84:0/929064366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309fb0e0 msgr2=0x556d309f8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f4046ffd640 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309fb0e0 0x556d309f8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f4046ffd640 1 -- 172.21.15.84:0/929064366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d30668210 con 0x556d309eff00 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f40467fc640 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309f0820 0x556d309f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f40477fe640 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309fb0e0 0x556d309f8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f4046ffd640 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 0x556d309f8030 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f403c0106a0 tx=0x7f403c010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d99232f4b459897c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f402aff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f402aff5640 1 -- 172.21.15.84:0/929064366 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c010ea0 con 0x556d309eff00 2024-04-26T18:44:38.866 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f402aff5640 1 -- 172.21.15.84:0/929064366 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f403c00cab0 con 0x556d309eff00 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f40503bc740 1 -- 172.21.15.84:0/929064366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f403809c630 con 0x556d309eff00 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f402aff5640 1 -- 172.21.15.84:0/929064366 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f403c00ccd0 con 0x556d309eff00 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f40503bc740 10 client.17391 fetch_fsmap learned FSMap version 29 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f40503bc740 1 -- 172.21.15.84:0/929064366 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d30734620 con 0x556d309eff00 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f402aff5640 1 -- 172.21.15.84:0/929064366 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f403809c630 con 0x556d309eff00 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f402aff5640 1 -- 172.21.15.84:0/929064366 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f403c00e050 con 0x556d309eff00 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.857+0000 7f40503bc740 10 client.17391 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.858+0000 7f40503bc740 10 client.17391 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.858+0000 7f40503bc740 20 client.17391 populate_metadata read hostname 'smithi084' 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.858+0000 7f40503bc740 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556d30a0b230 0x556d30a0d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.858+0000 7f40503bc740 4 client.17391 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.858+0000 7f40503bc740 1 -- 172.21.15.84:0/929064366 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d30964070 con 0x556d30a0b230 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.858+0000 7f40467fc640 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556d30a0b230 0x556d30a0d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.867 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.858+0000 7f402aff5640 10 client.17391 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.858+0000 7f40467fc640 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556d30a0b230 0x556d30a0d620 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5366a14a2034d989 server_cookie=45e8cfa95790d028 in_seq=0 out_seq=0 2024-04-26T18:44:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f402aff5640 1 -- 172.21.15.84:0/929064366 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556d30964070 con 0x556d30a0b230 2024-04-26T18:44:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f402aff5640 10 client.17391 handle_command_reply: tid=0 2024-04-26T18:44:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f40503bc740 1 client.17391 shutdown 2024-04-26T18:44:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f40503bc740 1 -- 172.21.15.84:0/929064366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 msgr2=0x556d309f8030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f40503bc740 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 0x556d309f8030 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f403c0106a0 tx=0x7f403c010b60 comp rx=0 tx=0).stop 2024-04-26T18:44:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f40503bc740 1 -- 172.21.15.84:0/929064366 shutdown_connections 2024-04-26T18:44:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f40503bc740 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556d30a0b230 0x556d30a0d620 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f40503bc740 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309fb0e0 0x556d309f8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f40503bc740 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309f0820 0x556d309f8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f402aff5640 0 client.17391 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:38.868 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f40503bc740 1 --2- 172.21.15.84:0/929064366 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 0x556d309f8030 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f40503bc740 1 -- 172.21.15.84:0/929064366 >> 172.21.15.84:0/929064366 conn(0x556d309ec410 msgr2=0x556d309f2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f40503bc740 1 -- 172.21.15.84:0/929064366 shutdown_connections 2024-04-26T18:44:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f40503bc740 1 -- 172.21.15.84:0/929064366 wait complete. 2024-04-26T18:44:38.869 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.859+0000 7f40503bc740 20 client.17391 trim_cache size 0 max 16384 2024-04-26T18:44:38.878 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.874+0000 7f40503bc740 1 Processor -- start 2024-04-26T18:44:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.874+0000 7f40503bc740 1 -- start start 2024-04-26T18:44:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.874+0000 7f40503bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309eff00 0x556d30a06bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.874+0000 7f40503bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 0x556d309f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.874+0000 7f40503bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309fb0e0 0x556d309f9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.874+0000 7f40503bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40481ae6e0 con 0x556d309f0820 2024-04-26T18:44:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.874+0000 7f40503bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f40481b0220 con 0x556d309fb0e0 2024-04-26T18:44:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.874+0000 7f40503bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40481aedb0 con 0x556d309eff00 2024-04-26T18:44:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.874+0000 7f40467fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 0x556d309f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.874+0000 7f40467fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 0x556d309f28c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41328/0 (socket says 172.21.15.84:41328) 2024-04-26T18:44:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.874+0000 7f40467fc640 1 -- 172.21.15.84:0/1789557537 learned_addr learned my addr 172.21.15.84:0/1789557537 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40477fe640 1 --2- 172.21.15.84:0/1789557537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309fb0e0 0x556d309f9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.879 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40467fc640 1 -- 172.21.15.84:0/1789557537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309eff00 msgr2=0x556d30a06bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40467fc640 1 --2- 172.21.15.84:0/1789557537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309eff00 0x556d30a06bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40467fc640 1 -- 172.21.15.84:0/1789557537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309fb0e0 msgr2=0x556d309f9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40467fc640 1 --2- 172.21.15.84:0/1789557537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309fb0e0 0x556d309f9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40467fc640 1 -- 172.21.15.84:0/1789557537 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d30a07ff0 con 0x556d309f0820 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40467fc640 1 --2- 172.21.15.84:0/1789557537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 0x556d309f28c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4038098340 tx=0x7f4038010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2707fc1c5e51df6d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f402cff9640 1 -- 172.21.15.84:0/1789557537 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403801e040 con 0x556d309f0820 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f402cff9640 1 -- 172.21.15.84:0/1789557537 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40380a02e0 con 0x556d309f0820 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f402cff9640 1 -- 172.21.15.84:0/1789557537 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40380985a0 con 0x556d309f0820 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40503bc740 1 -- 172.21.15.84:0/1789557537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 msgr2=0x556d309f28c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40503bc740 1 --2- 172.21.15.84:0/1789557537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 0x556d309f28c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f4038098340 tx=0x7f4038010f90 comp rx=0 tx=0).stop 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40503bc740 1 -- 172.21.15.84:0/1789557537 shutdown_connections 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40503bc740 1 --2- 172.21.15.84:0/1789557537 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309fb0e0 0x556d309f9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40503bc740 1 --2- 172.21.15.84:0/1789557537 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309f0820 0x556d309f28c0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.880 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40503bc740 1 --2- 172.21.15.84:0/1789557537 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309eff00 0x556d30a06bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40503bc740 1 -- 172.21.15.84:0/1789557537 >> 172.21.15.84:0/1789557537 conn(0x556d309ec410 msgr2=0x556d309f9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.875+0000 7f40503bc740 1 -- 172.21.15.84:0/1789557537 shutdown_connections 2024-04-26T18:44:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f40503bc740 1 -- 172.21.15.84:0/1789557537 wait complete. 2024-04-26T18:44:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f40503bc740 1 -- start start 2024-04-26T18:44:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f40503bc740 1 Processor -- start 2024-04-26T18:44:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f40503bc740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 0x556d309f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f40503bc740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309f0820 0x556d309f5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f40503bc740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309fb0e0 0x556d30a74b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f40503bc740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f40481afb50 con 0x556d309eff00 2024-04-26T18:44:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f40503bc740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f40481af480 con 0x556d309f0820 2024-04-26T18:44:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f40503bc740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f40481ac6e0 con 0x556d309fb0e0 2024-04-26T18:44:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f40503bc740 10 client.0 _ll_register_callbacks cb 0x7f403c00e370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:44:38.881 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 0x556d309f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f4046ffd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 0x556d309f50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:41344/0 (socket says 172.21.15.84:41344) 2024-04-26T18:44:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f4046ffd640 1 -- 172.21.15.84:0/1140286331 learned_addr learned my addr 172.21.15.84:0/1140286331 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f4046ffd640 1 -- 172.21.15.84:0/1140286331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309fb0e0 msgr2=0x556d30a74b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f4046ffd640 1 --2- 172.21.15.84:0/1140286331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309fb0e0 0x556d30a74b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f40467fc640 1 --2- 172.21.15.84:0/1140286331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309f0820 0x556d309f5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f4046ffd640 1 -- 172.21.15.84:0/1140286331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309f0820 msgr2=0x556d309f5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f4046ffd640 1 --2- 172.21.15.84:0/1140286331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309f0820 0x556d309f5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.876+0000 7f4046ffd640 1 -- 172.21.15.84:0/1140286331 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d30a750d0 con 0x556d309eff00 2024-04-26T18:44:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f4046ffd640 1 --2- 172.21.15.84:0/1140286331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 0x556d309f50e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f403c00c810 tx=0x7f403c010ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b320592cd4fe92f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f402aff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:44:38.882 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f402aff5640 1 -- 172.21.15.84:0/1140286331 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f403c01c990 con 0x556d309eff00 2024-04-26T18:44:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f402aff5640 1 -- 172.21.15.84:0/1140286331 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f403c00cab0 con 0x556d309eff00 2024-04-26T18:44:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f40503bc740 1 -- 172.21.15.84:0/1140286331 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556d3095dee0 con 0x556d309eff00 2024-04-26T18:44:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f402aff5640 1 -- 172.21.15.84:0/1140286331 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f403c005e50 con 0x556d309eff00 2024-04-26T18:44:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f40503bc740 10 client.17397 fetch_fsmap learned FSMap version 29 2024-04-26T18:44:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f40503bc740 1 -- 172.21.15.84:0/1140286331 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556d30a76280 con 0x556d309eff00 2024-04-26T18:44:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f402aff5640 1 -- 172.21.15.84:0/1140286331 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556d3095dee0 con 0x556d309eff00 2024-04-26T18:44:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f402aff5640 1 -- 172.21.15.84:0/1140286331 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f403c00f040 con 0x556d309eff00 2024-04-26T18:44:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f40503bc740 10 client.17397 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:44:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f40503bc740 10 client.17397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:44:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f40503bc740 20 client.17397 populate_metadata read hostname 'smithi084' 2024-04-26T18:44:38.883 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f40503bc740 1 --2- 172.21.15.84:0/1140286331 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556d30a76a50 0x556d30a78e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f40503bc740 4 client.17397 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:44:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f40503bc740 1 -- 172.21.15.84:0/1140286331 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556d308d95f0 con 0x556d30a76a50 2024-04-26T18:44:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.877+0000 7f40467fc640 1 --2- 172.21.15.84:0/1140286331 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556d30a76a50 0x556d30a78e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.878+0000 7f402aff5640 10 client.17397 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:38.884 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.878+0000 7f40467fc640 1 --2- 172.21.15.84:0/1140286331 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556d30a76a50 0x556d30a78e90 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9612fc5318c58e0 server_cookie=b5eed8ee500f8840 in_seq=0 out_seq=0 2024-04-26T18:44:38.889 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f402aff5640 1 -- 172.21.15.84:0/1140286331 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556d308d95f0 con 0x556d30a76a50 2024-04-26T18:44:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f402aff5640 10 client.17397 handle_command_reply: tid=0 2024-04-26T18:44:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f40503bc740 1 client.17397 shutdown 2024-04-26T18:44:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f40503bc740 1 -- 172.21.15.84:0/1140286331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 msgr2=0x556d309f50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f40503bc740 1 --2- 172.21.15.84:0/1140286331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 0x556d309f50e0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f403c00c810 tx=0x7f403c010ea0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f40503bc740 1 -- 172.21.15.84:0/1140286331 shutdown_connections 2024-04-26T18:44:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f40503bc740 1 --2- 172.21.15.84:0/1140286331 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x556d30a76a50 0x556d30a78e90 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f40503bc740 1 --2- 172.21.15.84:0/1140286331 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x556d309fb0e0 0x556d30a74b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f402aff5640 0 client.17397 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f40503bc740 1 --2- 172.21.15.84:0/1140286331 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x556d309f0820 0x556d309f5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.890 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f40503bc740 1 --2- 172.21.15.84:0/1140286331 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x556d309eff00 0x556d309f50e0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f40503bc740 1 -- 172.21.15.84:0/1140286331 >> 172.21.15.84:0/1140286331 conn(0x556d309ec410 msgr2=0x556d309fa0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f40503bc740 1 -- 172.21.15.84:0/1140286331 shutdown_connections 2024-04-26T18:44:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f40503bc740 1 -- 172.21.15.84:0/1140286331 wait complete. 2024-04-26T18:44:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.889+0000 7f40503bc740 20 client.17397 trim_cache size 0 max 16384 2024-04-26T18:44:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:44:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:44:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:44:38.891 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:44:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.890+0000 7f404d5bf640 1 -- 172.21.15.84:0/3747669664 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f40300a3dc0 msgr2=0x7f40300a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.890+0000 7f404d5bf640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f40300a3dc0 0x7f40300a6280 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f4040006010 tx=0x7f4040031720 comp rx=0 tx=0).stop 2024-04-26T18:44:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.890+0000 7f404d5bf640 1 -- 172.21.15.84:0/3747669664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40481032d0 msgr2=0x7f40481995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.890+0000 7f404d5bf640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40481032d0 0x7f40481995e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f403800b4a0 tx=0x7f403800b960 comp rx=0 tx=0).stop 2024-04-26T18:44:38.892 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.890+0000 7f404d5bf640 1 -- 172.21.15.84:0/3747669664 shutdown_connections 2024-04-26T18:44:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.890+0000 7f404d5bf640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f40300a3dc0 0x7f40300a6280 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.890+0000 7f404d5bf640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4048108920 0x7f404819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.890+0000 7f404d5bf640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f40481032d0 0x7f40481995e0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.890+0000 7f404d5bf640 1 --2- 172.21.15.84:0/3747669664 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4048102910 0x7f40481990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.890+0000 7f404d5bf640 1 -- 172.21.15.84:0/3747669664 >> 172.21.15.84:0/3747669664 conn(0x7f40480fe400 msgr2=0x7f40480fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.890+0000 7f404d5bf640 1 -- 172.21.15.84:0/3747669664 shutdown_connections 2024-04-26T18:44:38.893 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:38.890+0000 7f404d5bf640 1 -- 172.21.15.84:0/3747669664 wait complete. 2024-04-26T18:44:39.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:38 smithi084 ceph-mon[29141]: pgmap v525: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-26T18:44:39.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:38 smithi103 ceph-mon[32727]: pgmap v525: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-26T18:44:39.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:38 smithi174 ceph-mon[32705]: pgmap v525: 97 pgs: 97 active+clean; 3.2 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 6.2 KiB/s wr, 11 op/s 2024-04-26T18:44:41.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:40 smithi084 ceph-mon[29141]: pgmap v526: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-04-26T18:44:41.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:40 smithi103 ceph-mon[32727]: pgmap v526: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-04-26T18:44:41.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:40 smithi174 ceph-mon[32705]: pgmap v526: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.9 KiB/s wr, 15 op/s 2024-04-26T18:44:43.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:42 smithi084 ceph-mon[29141]: pgmap v527: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-26T18:44:43.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:42 smithi103 ceph-mon[32727]: pgmap v527: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-26T18:44:43.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:42 smithi174 ceph-mon[32705]: pgmap v527: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.7 KiB/s wr, 8 op/s 2024-04-26T18:44:45.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:44 smithi084 ceph-mon[29141]: pgmap v528: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-04-26T18:44:45.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:44 smithi103 ceph-mon[32727]: pgmap v528: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-04-26T18:44:45.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:44 smithi174 ceph-mon[32705]: pgmap v528: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-04-26T18:44:46.166 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:44:46.420 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:44:47.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:46 smithi084 ceph-mon[29141]: pgmap v529: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-26T18:44:47.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:46 smithi103 ceph-mon[32727]: pgmap v529: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-26T18:44:47.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:46 smithi174 ceph-mon[32705]: pgmap v529: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-26T18:44:48.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.119+0000 7f4c8ceac700 1 -- 172.21.15.84:0/2808765263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c881038b0 msgr2=0x7f4c8810f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:48.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.119+0000 7f4c8ceac700 1 --2- 172.21.15.84:0/2808765263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c881038b0 0x7f4c8810f2c0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000b0d0 tx=0x7f4c8000b3e0 comp rx=0 tx=0).stop 2024-04-26T18:44:48.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.119+0000 7f4c8ceac700 1 -- 172.21.15.84:0/2808765263 shutdown_connections 2024-04-26T18:44:48.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.119+0000 7f4c8ceac700 1 --2- 172.21.15.84:0/2808765263 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c881038b0 0x7f4c8810f2c0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:48.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.119+0000 7f4c8ceac700 1 --2- 172.21.15.84:0/2808765263 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c88108610 0x7f4c881089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:48.119 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.119+0000 7f4c8ceac700 1 --2- 172.21.15.84:0/2808765263 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c88102ef0 0x7f4c88103370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:48.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.119+0000 7f4c8ceac700 1 -- 172.21.15.84:0/2808765263 >> 172.21.15.84:0/2808765263 conn(0x7f4c88077640 msgr2=0x7f4c88077a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:48.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.119+0000 7f4c8ceac700 1 -- 172.21.15.84:0/2808765263 shutdown_connections 2024-04-26T18:44:48.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.119+0000 7f4c8ceac700 1 -- 172.21.15.84:0/2808765263 wait complete. 2024-04-26T18:44:48.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8ceac700 1 Processor -- start 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8ceac700 1 -- start start 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8ceac700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ef0 0x7f4c8819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8ceac700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c881038b0 0x7f4c8819af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8ceac700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c88108610 0x7f4c881a5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8ceac700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4c88112690 con 0x7f4c88102ef0 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8ceac700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c88112510 con 0x7f4c881038b0 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8ceac700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c88112810 con 0x7f4c88108610 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ef0 0x7f4c8819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8659c700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ef0 0x7f4c8819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47686/0 (socket says 172.21.15.84:47686) 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8659c700 1 -- 172.21.15.84:0/3118293102 learned_addr learned my addr 172.21.15.84:0/3118293102 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c86d9d700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c88108610 0x7f4c881a5ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c85d9b700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c881038b0 0x7f4c8819af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8659c700 1 -- 172.21.15.84:0/3118293102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c88108610 msgr2=0x7f4c881a5ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8659c700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c88108610 0x7f4c881a5ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:48.122 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8659c700 1 -- 172.21.15.84:0/3118293102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c881038b0 msgr2=0x7f4c8819af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:48.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8659c700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c881038b0 0x7f4c8819af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:48.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.120+0000 7f4c8659c700 1 -- 172.21.15.84:0/3118293102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c881a6010 con 0x7f4c88102ef0 2024-04-26T18:44:48.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.121+0000 7f4c8659c700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ef0 0x7f4c8819aa50 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d950 tx=0x7f4c7c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:48.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.121+0000 7f4c777fe700 1 -- 172.21.15.84:0/3118293102 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c016070 con 0x7f4c88102ef0 2024-04-26T18:44:48.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.121+0000 7f4c8ceac700 1 -- 172.21.15.84:0/3118293102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c881a6240 con 0x7f4c88102ef0 2024-04-26T18:44:48.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.121+0000 7f4c777fe700 1 -- 172.21.15.84:0/3118293102 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c7c00fd40 con 0x7f4c88102ef0 2024-04-26T18:44:48.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.121+0000 7f4c777fe700 1 -- 172.21.15.84:0/3118293102 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c00e3b0 con 0x7f4c88102ef0 2024-04-26T18:44:48.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.121+0000 7f4c8ceac700 1 -- 172.21.15.84:0/3118293102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c881a67a0 con 0x7f4c88102ef0 2024-04-26T18:44:48.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.122+0000 7f4c8ceac700 1 -- 172.21.15.84:0/3118293102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8806a110 con 0x7f4c88102ef0 2024-04-26T18:44:48.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.123+0000 7f4c777fe700 1 -- 172.21.15.84:0/3118293102 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4c7c019070 con 0x7f4c88102ef0 2024-04-26T18:44:48.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.123+0000 7f4c777fe700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c700a3db0 0x7f4c700a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:48.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.124+0000 7f4c85d9b700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c700a3db0 0x7f4c700a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:48.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.124+0000 7f4c777fe700 1 -- 172.21.15.84:0/3118293102 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f4c7c0d0340 con 0x7f4c88102ef0 2024-04-26T18:44:48.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.124+0000 7f4c85d9b700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c700a3db0 0x7f4c700a6270 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c88077b60 tx=0x7f4c78009190 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:48.128 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.128+0000 7f4c777fe700 1 -- 172.21.15.84:0/3118293102 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c7c09c940 con 0x7f4c88102ef0 2024-04-26T18:44:48.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.379+0000 7f4c8ceac700 1 -- 172.21.15.84:0/3118293102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c8802e490 con 0x7f4c88102ef0 2024-04-26T18:44:48.380 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.380+0000 7f4c777fe700 1 -- 172.21.15.84:0/3118293102 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f4c7c09c240 con 0x7f4c88102ef0 2024-04-26T18:44:48.380 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:44:48.380 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:44:48.382 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.382+0000 7f4c8ceac700 1 -- 172.21.15.84:0/3118293102 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c700a3db0 msgr2=0x7f4c700a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.382+0000 7f4c8ceac700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c700a3db0 0x7f4c700a6270 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4c88077b60 tx=0x7f4c78009190 comp rx=0 tx=0).stop 2024-04-26T18:44:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.382+0000 7f4c8ceac700 1 -- 172.21.15.84:0/3118293102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ef0 msgr2=0x7f4c8819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.382+0000 7f4c8ceac700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ef0 0x7f4c8819aa50 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c00d950 tx=0x7f4c7c00dc60 comp rx=0 tx=0).stop 2024-04-26T18:44:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.382+0000 7f4c8ceac700 1 -- 172.21.15.84:0/3118293102 shutdown_connections 2024-04-26T18:44:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.382+0000 7f4c8ceac700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c700a3db0 0x7f4c700a6270 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.382+0000 7f4c8ceac700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c88102ef0 0x7f4c8819aa50 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.382+0000 7f4c8ceac700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c88108610 0x7f4c881a5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.382+0000 7f4c8ceac700 1 --2- 172.21.15.84:0/3118293102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c881038b0 0x7f4c8819af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.382+0000 7f4c8ceac700 1 -- 172.21.15.84:0/3118293102 >> 172.21.15.84:0/3118293102 conn(0x7f4c88077640 msgr2=0x7f4c880ffd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.382+0000 7f4c8ceac700 1 -- 172.21.15.84:0/3118293102 shutdown_connections 2024-04-26T18:44:48.383 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:48.382+0000 7f4c8ceac700 1 -- 172.21.15.84:0/3118293102 wait complete. 2024-04-26T18:44:48.384 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:44:49.014 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:48 smithi084 ceph-mon[29141]: pgmap v530: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-26T18:44:49.015 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:48 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3118293102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:44:49.043 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --set-id 8da9d5cd --release --await' 2024-04-26T18:44:49.043 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db --set-id 8da9d5cd --release --await 2024-04-26T18:44:49.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:48 smithi103 ceph-mon[32727]: pgmap v530: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-26T18:44:49.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:48 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3118293102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:44:49.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:48 smithi174 ceph-mon[32705]: pgmap v530: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.9 KiB/s wr, 10 op/s 2024-04-26T18:44:49.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:48 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3118293102' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:44:49.285 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:44:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.017+0000 7f5965a0f700 1 -- 172.21.15.84:0/2349444482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5960102350 msgr2=0x7f59601027b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.017+0000 7f5965a0f700 1 --2- 172.21.15.84:0/2349444482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5960102350 0x7f59601027b0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f5948009990 tx=0x7f5948009ca0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.017+0000 7f5965a0f700 1 -- 172.21.15.84:0/2349444482 shutdown_connections 2024-04-26T18:44:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.017+0000 7f5965a0f700 1 --2- 172.21.15.84:0/2349444482 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5960102cf0 0x7f596010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.017+0000 7f5965a0f700 1 --2- 172.21.15.84:0/2349444482 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5960108390 0x7f5960108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.017+0000 7f5965a0f700 1 --2- 172.21.15.84:0/2349444482 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5960102350 0x7f59601027b0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.017+0000 7f5965a0f700 1 -- 172.21.15.84:0/2349444482 >> 172.21.15.84:0/2349444482 conn(0x7f59600792c0 msgr2=0x7f59600796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.018+0000 7f5965a0f700 1 -- 172.21.15.84:0/2349444482 shutdown_connections 2024-04-26T18:44:51.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.018+0000 7f5965a0f700 1 -- 172.21.15.84:0/2349444482 wait complete. 2024-04-26T18:44:51.019 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:50 smithi084 ceph-mon[29141]: pgmap v531: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-04-26T18:44:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.018+0000 7f5965a0f700 1 Processor -- start 2024-04-26T18:44:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f5965a0f700 1 -- start start 2024-04-26T18:44:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f5965a0f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5960102350 0x7f596019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f5965a0f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5960102cf0 0x7f596019eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f5965a0f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5960108390 0x7f59601987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f5965a0f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f59601124a0 con 0x7f5960108390 2024-04-26T18:44:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f5965a0f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5960112320 con 0x7f5960102350 2024-04-26T18:44:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f5965a0f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5960112620 con 0x7f5960102cf0 2024-04-26T18:44:51.021 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f595f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5960108390 0x7f59601987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f595f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5960108390 0x7f59601987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47696/0 (socket says 172.21.15.84:47696) 2024-04-26T18:44:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f595f7fe700 1 -- 172.21.15.84:0/574067511 learned_addr learned my addr 172.21.15.84:0/574067511 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f595effd700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5960102350 0x7f596019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f595e7fc700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5960102cf0 0x7f596019eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f595f7fe700 1 -- 172.21.15.84:0/574067511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5960102cf0 msgr2=0x7f596019eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f595f7fe700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5960102cf0 0x7f596019eba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f595f7fe700 1 -- 172.21.15.84:0/574067511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5960102350 msgr2=0x7f596019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.019+0000 7f595f7fe700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5960102350 0x7f596019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.020+0000 7f595f7fe700 1 -- 172.21.15.84:0/574067511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5960198fc0 con 0x7f5960108390 2024-04-26T18:44:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.020+0000 7f595f7fe700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5960108390 0x7f59601987e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f595800efb0 tx=0x7f595800c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.020+0000 7f5964a0d700 1 -- 172.21.15.84:0/574067511 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5958011670 con 0x7f5960108390 2024-04-26T18:44:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.020+0000 7f5965a0f700 1 -- 172.21.15.84:0/574067511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59601992b0 con 0x7f5960108390 2024-04-26T18:44:51.022 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.020+0000 7f5964a0d700 1 -- 172.21.15.84:0/574067511 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f595800ce20 con 0x7f5960108390 2024-04-26T18:44:51.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.020+0000 7f5964a0d700 1 -- 172.21.15.84:0/574067511 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595801abe0 con 0x7f5960108390 2024-04-26T18:44:51.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.020+0000 7f5965a0f700 1 -- 172.21.15.84:0/574067511 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596006bb70 con 0x7f5960108390 2024-04-26T18:44:51.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.022+0000 7f5964a0d700 1 -- 172.21.15.84:0/574067511 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5958011810 con 0x7f5960108390 2024-04-26T18:44:51.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.023+0000 7f5964a0d700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f594c0a3db0 0x7f594c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.023 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.023+0000 7f595effd700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f594c0a3db0 0x7f594c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.023+0000 7f5964a0d700 1 -- 172.21.15.84:0/574067511 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f59580d30b0 con 0x7f5960108390 2024-04-26T18:44:51.024 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.023+0000 7f595effd700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f594c0a3db0 0x7f594c0a6270 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f595000abd0 tx=0x7f595000a380 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.024+0000 7f59702a1b80 1 Processor -- start 2024-04-26T18:44:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.024+0000 7f59702a1b80 1 -- start start 2024-04-26T18:44:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.024+0000 7f59702a1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649229ea0 0x55f64922a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.024+0000 7f59702a1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64922a7c0 0x55f649224250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.024+0000 7f59702a1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f649224870 0x55f649224cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.024+0000 7f59702a1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f649176930 con 0x55f649229ea0 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.024+0000 7f59702a1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f6490dca30 con 0x55f64922a7c0 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.024+0000 7f59702a1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f6491728f0 con 0x55f649224870 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.024+0000 7f595effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649229ea0 0x55f64922a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.024+0000 7f595effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649229ea0 0x55f64922a280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47704/0 (socket says 172.21.15.84:47704) 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.024+0000 7f595effd700 1 -- 172.21.15.84:0/1665926675 learned_addr learned my addr 172.21.15.84:0/1665926675 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.025+0000 7f595f7fe700 1 --2- 172.21.15.84:0/1665926675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f649224870 0x55f649224cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.025+0000 7f595e7fc700 1 --2- 172.21.15.84:0/1665926675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64922a7c0 0x55f649224250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.025+0000 7f595effd700 1 -- 172.21.15.84:0/1665926675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f649224870 msgr2=0x55f649224cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.025+0000 7f595effd700 1 --2- 172.21.15.84:0/1665926675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f649224870 0x55f649224cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.025+0000 7f595effd700 1 -- 172.21.15.84:0/1665926675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64922a7c0 msgr2=0x55f649224250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.025+0000 7f595effd700 1 --2- 172.21.15.84:0/1665926675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64922a7c0 0x55f649224250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.025+0000 7f595effd700 1 -- 172.21.15.84:0/1665926675 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6490c39d0 con 0x55f649229ea0 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.025+0000 7f595f7fe700 1 --2- 172.21.15.84:0/1665926675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f649224870 0x55f649224cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:44:51.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.025+0000 7f595effd700 1 --2- 172.21.15.84:0/1665926675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649229ea0 0x55f64922a280 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5950004530 tx=0x7f5950012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07af1614d111ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.025+0000 7f5955ffb700 1 -- 172.21.15.84:0/1665926675 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950016070 con 0x55f649229ea0 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.025+0000 7f5955ffb700 1 -- 172.21.15.84:0/1665926675 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5950014070 con 0x55f649229ea0 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.025+0000 7f5955ffb700 1 -- 172.21.15.84:0/1665926675 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595001d400 con 0x55f649229ea0 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.026+0000 7f59702a1b80 1 -- 172.21.15.84:0/1665926675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649229ea0 msgr2=0x55f64922a280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.026+0000 7f59702a1b80 1 --2- 172.21.15.84:0/1665926675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649229ea0 0x55f64922a280 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5950004530 tx=0x7f5950012000 comp rx=0 tx=0).stop 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.026+0000 7f59702a1b80 1 -- 172.21.15.84:0/1665926675 shutdown_connections 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.026+0000 7f59702a1b80 1 --2- 172.21.15.84:0/1665926675 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649229ea0 0x55f64922a280 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.026+0000 7f59702a1b80 1 --2- 172.21.15.84:0/1665926675 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f649224870 0x55f649224cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.026+0000 7f59702a1b80 1 --2- 172.21.15.84:0/1665926675 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f64922a7c0 0x55f649224250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.026+0000 7f59702a1b80 1 -- 172.21.15.84:0/1665926675 >> 172.21.15.84:0/1665926675 conn(0x55f6491526c0 msgr2=0x55f64921f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.026+0000 7f59702a1b80 1 -- 172.21.15.84:0/1665926675 shutdown_connections 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.026+0000 7f59702a1b80 1 -- 172.21.15.84:0/1665926675 wait complete. 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.026+0000 7f59702a1b80 1 -- start start 2024-04-26T18:44:51.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.026+0000 7f59702a1b80 1 Processor -- start 2024-04-26T18:44:51.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f59702a1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f649224870 0x55f649230460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f59702a1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 0x55f64922d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f59702a1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f64922a7c0 0x55f64922dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f59702a1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55f649231230 con 0x55f64922a7c0 2024-04-26T18:44:51.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f59702a1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55f649130b80 con 0x55f649229ea0 2024-04-26T18:44:51.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f59702a1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55f6492313b0 con 0x55f649224870 2024-04-26T18:44:51.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f59702a1b80 10 client.0 _ll_register_callbacks cb 0x55f64910ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:44:51.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f595f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f64922a7c0 0x55f64922dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f595f7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f64922a7c0 0x55f64922dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47720/0 (socket says 172.21.15.84:47720) 2024-04-26T18:44:51.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f595f7fe700 1 -- 172.21.15.84:0/1610310889 learned_addr learned my addr 172.21.15.84:0/1610310889 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:51.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f595effd700 1 --2- 172.21.15.84:0/1610310889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f649224870 0x55f649230460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f595e7fc700 1 --2- 172.21.15.84:0/1610310889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 0x55f64922d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f595f7fe700 1 -- 172.21.15.84:0/1610310889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f649224870 msgr2=0x55f649230460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f595f7fe700 1 --2- 172.21.15.84:0/1610310889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f649224870 0x55f649230460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f595f7fe700 1 -- 172.21.15.84:0/1610310889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 msgr2=0x55f64922d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f595f7fe700 1 --2- 172.21.15.84:0/1610310889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 0x55f64922d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f595f7fe700 1 -- 172.21.15.84:0/1610310889 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f649134580 con 0x55f64922a7c0 2024-04-26T18:44:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f595f7fe700 1 --2- 172.21.15.84:0/1610310889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f64922a7c0 0x55f64922dd40 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5958007fd0 tx=0x7f595809ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c62d4bbc770600 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f593effd700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:44:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f593effd700 1 -- 172.21.15.84:0/1610310889 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59580b7020 con 0x55f64922a7c0 2024-04-26T18:44:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f593effd700 1 -- 172.21.15.84:0/1610310889 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f5958011b00 con 0x55f64922a7c0 2024-04-26T18:44:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.027+0000 7f59702a1b80 1 -- 172.21.15.84:0/1610310889 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5950014070 con 0x55f64922a7c0 2024-04-26T18:44:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.028+0000 7f593effd700 1 -- 172.21.15.84:0/1610310889 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f59580a33c0 con 0x55f64922a7c0 2024-04-26T18:44:51.033 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.028+0000 7f59702a1b80 10 client.17415 fetch_fsmap learned FSMap version 29 2024-04-26T18:44:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.028+0000 7f59702a1b80 1 -- 172.21.15.84:0/1610310889 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f64923b9c0 con 0x55f64922a7c0 2024-04-26T18:44:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.028+0000 7f593effd700 1 -- 172.21.15.84:0/1610310889 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5950014070 con 0x55f64922a7c0 2024-04-26T18:44:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.028+0000 7f593effd700 1 -- 172.21.15.84:0/1610310889 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f595800f040 con 0x55f64922a7c0 2024-04-26T18:44:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.028+0000 7f59702a1b80 10 client.17415 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:44:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.028+0000 7f59702a1b80 10 client.17415 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:44:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.028+0000 7f59702a1b80 20 client.17415 populate_metadata read hostname 'smithi084' 2024-04-26T18:44:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.028+0000 7f59702a1b80 1 --2- 172.21.15.84:0/1610310889 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f64923df50 0x55f649240340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.028+0000 7f59702a1b80 4 client.17415 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:44:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.028+0000 7f59702a1b80 1 -- 172.21.15.84:0/1610310889 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f649184310 con 0x55f64923df50 2024-04-26T18:44:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.028+0000 7f595effd700 1 --2- 172.21.15.84:0/1610310889 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f64923df50 0x55f649240340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.029+0000 7f593effd700 10 client.17415 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:51.034 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.029+0000 7f595effd700 1 --2- 172.21.15.84:0/1610310889 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f64923df50 0x55f649240340 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e741eb4b948e6ce server_cookie=322928228f6ff839 in_seq=0 out_seq=0 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.029+0000 7f593effd700 1 -- 172.21.15.84:0/1610310889 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f649184310 con 0x55f64923df50 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.029+0000 7f593effd700 10 client.17415 handle_command_reply: tid=0 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.029+0000 7f59702a1b80 1 client.17415 shutdown 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.030+0000 7f59702a1b80 1 -- 172.21.15.84:0/1610310889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f64922a7c0 msgr2=0x55f64922dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.030+0000 7f59702a1b80 1 --2- 172.21.15.84:0/1610310889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f64922a7c0 0x55f64922dd40 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5958007fd0 tx=0x7f595809ea60 comp rx=0 tx=0).stop 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.030+0000 7f59702a1b80 1 -- 172.21.15.84:0/1610310889 shutdown_connections 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.030+0000 7f59702a1b80 1 --2- 172.21.15.84:0/1610310889 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f64923df50 0x55f649240340 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.030+0000 7f59702a1b80 1 --2- 172.21.15.84:0/1610310889 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f64922a7c0 0x55f64922dd40 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.030+0000 7f59702a1b80 1 --2- 172.21.15.84:0/1610310889 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f649224870 0x55f649230460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.030+0000 7f59702a1b80 1 --2- 172.21.15.84:0/1610310889 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 0x55f64922d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.030+0000 7f59702a1b80 1 -- 172.21.15.84:0/1610310889 >> 172.21.15.84:0/1610310889 conn(0x55f6491526c0 msgr2=0x55f649223240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.030+0000 7f593effd700 0 client.17415 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.030+0000 7f59702a1b80 1 -- 172.21.15.84:0/1610310889 shutdown_connections 2024-04-26T18:44:51.035 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.030+0000 7f59702a1b80 1 -- 172.21.15.84:0/1610310889 wait complete. 2024-04-26T18:44:51.036 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.030+0000 7f59702a1b80 20 client.17415 trim_cache size 0 max 16384 2024-04-26T18:44:51.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.049+0000 7f59702a1b80 1 Processor -- start 2024-04-26T18:44:51.050 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.049+0000 7f59702a1b80 1 -- start start 2024-04-26T18:44:51.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.049+0000 7f59702a1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 0x55f649224250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.049+0000 7f59702a1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 0x55f64923a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.049+0000 7f59702a1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f64922a7c0 0x55f649236a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.049+0000 7f59702a1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f59601aba90 con 0x55f649224870 2024-04-26T18:44:51.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.049+0000 7f59702a1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59601ad5d0 con 0x55f649229ea0 2024-04-26T18:44:51.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.049+0000 7f59702a1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59601ac160 con 0x55f64922a7c0 2024-04-26T18:44:51.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f595effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 0x55f649224250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f595effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 0x55f649224250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47734/0 (socket says 172.21.15.84:47734) 2024-04-26T18:44:51.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f595effd700 1 -- 172.21.15.84:0/3400290086 learned_addr learned my addr 172.21.15.84:0/3400290086 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:51.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f595f7fe700 1 --2- 172.21.15.84:0/3400290086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f64922a7c0 0x55f649236a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f595effd700 1 -- 172.21.15.84:0/3400290086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f64922a7c0 msgr2=0x55f649236a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:51.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f595effd700 1 --2- 172.21.15.84:0/3400290086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f64922a7c0 0x55f649236a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f595effd700 1 -- 172.21.15.84:0/3400290086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 msgr2=0x55f64923a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:51.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f595effd700 1 --2- 172.21.15.84:0/3400290086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 0x55f64923a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f595effd700 1 -- 172.21.15.84:0/3400290086 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f64922e120 con 0x55f649224870 2024-04-26T18:44:51.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f595f7fe700 1 --2- 172.21.15.84:0/3400290086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f64922a7c0 0x55f649236a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:44:51.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f595effd700 1 --2- 172.21.15.84:0/3400290086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 0x55f649224250 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f59500044d0 tx=0x7f595000a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd84a8b0eaec558 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:51.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f59557fa700 1 -- 172.21.15.84:0/3400290086 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950038070 con 0x55f649224870 2024-04-26T18:44:51.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f59557fa700 1 -- 172.21.15.84:0/3400290086 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5950014040 con 0x55f649224870 2024-04-26T18:44:51.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.050+0000 7f59557fa700 1 -- 172.21.15.84:0/3400290086 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950016040 con 0x55f649224870 2024-04-26T18:44:51.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.051+0000 7f59702a1b80 1 -- 172.21.15.84:0/3400290086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 msgr2=0x55f649224250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.051+0000 7f59702a1b80 1 --2- 172.21.15.84:0/3400290086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 0x55f649224250 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f59500044d0 tx=0x7f595000a3f0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.051+0000 7f59702a1b80 1 -- 172.21.15.84:0/3400290086 shutdown_connections 2024-04-26T18:44:51.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.051+0000 7f59702a1b80 1 --2- 172.21.15.84:0/3400290086 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 0x55f649224250 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.055 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.051+0000 7f59702a1b80 1 --2- 172.21.15.84:0/3400290086 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f64922a7c0 0x55f649236a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.051+0000 7f59702a1b80 1 --2- 172.21.15.84:0/3400290086 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 0x55f64923a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.051+0000 7f59702a1b80 1 -- 172.21.15.84:0/3400290086 >> 172.21.15.84:0/3400290086 conn(0x55f6491526c0 msgr2=0x55f649218f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.051+0000 7f59702a1b80 1 -- 172.21.15.84:0/3400290086 shutdown_connections 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.051+0000 7f59702a1b80 1 -- 172.21.15.84:0/3400290086 wait complete. 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.051+0000 7f59702a1b80 1 -- start start 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.051+0000 7f59702a1b80 1 Processor -- start 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f59702a1b80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 0x55f6492a2230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f59702a1b80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 0x55f6492a2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f59702a1b80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f64922a7c0 0x55f64929c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f59702a1b80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f59601acf00 con 0x55f649224870 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f59702a1b80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f59601ac830 con 0x55f649229ea0 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f59702a1b80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f59601a9a50 con 0x55f64922a7c0 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f59702a1b80 10 client.0 _ll_register_callbacks cb 0x55f64910ea50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:44:51.056 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f595effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 0x55f6492a2230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f595effd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 0x55f6492a2230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47740/0 (socket says 172.21.15.84:47740) 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f595effd700 1 -- 172.21.15.84:0/3601219914 learned_addr learned my addr 172.21.15.84:0/3601219914 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f595effd700 1 -- 172.21.15.84:0/3601219914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f64922a7c0 msgr2=0x55f64929c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f595effd700 1 --2- 172.21.15.84:0/3601219914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f64922a7c0 0x55f64929c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f595effd700 1 -- 172.21.15.84:0/3601219914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 msgr2=0x55f6492a2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f595e7fc700 1 --2- 172.21.15.84:0/3601219914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 0x55f6492a2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f595effd700 1 --2- 172.21.15.84:0/3601219914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 0x55f6492a2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f595effd700 1 -- 172.21.15.84:0/3601219914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f64929cd50 con 0x55f649224870 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f595effd700 1 --2- 172.21.15.84:0/3601219914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 0x55f6492a2230 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f5950010be0 tx=0x7f595000a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5374a399cf7074b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f593effd700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f593effd700 1 -- 172.21.15.84:0/3601219914 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5950038030 con 0x55f649224870 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f593effd700 1 -- 172.21.15.84:0/3601219914 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f5950005040 con 0x55f649224870 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.052+0000 7f59702a1b80 1 -- 172.21.15.84:0/3601219914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5950014040 con 0x55f649224870 2024-04-26T18:44:51.057 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.053+0000 7f593effd700 1 -- 172.21.15.84:0/3601219914 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5950016040 con 0x55f649224870 2024-04-26T18:44:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.053+0000 7f59702a1b80 10 client.17421 fetch_fsmap learned FSMap version 29 2024-04-26T18:44:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.053+0000 7f59702a1b80 1 -- 172.21.15.84:0/3601219914 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6492a9a10 con 0x55f649224870 2024-04-26T18:44:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.053+0000 7f593effd700 1 -- 172.21.15.84:0/3601219914 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5950014040 con 0x55f649224870 2024-04-26T18:44:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.053+0000 7f593effd700 1 -- 172.21.15.84:0/3601219914 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f5950004660 con 0x55f649224870 2024-04-26T18:44:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.053+0000 7f59702a1b80 10 client.17421 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:44:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.053+0000 7f59702a1b80 10 client.17421 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:44:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.053+0000 7f59702a1b80 20 client.17421 populate_metadata read hostname 'smithi084' 2024-04-26T18:44:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.053+0000 7f59702a1b80 1 --2- 172.21.15.84:0/3601219914 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f6492a9e10 0x55f6492ac200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.053+0000 7f59702a1b80 4 client.17421 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "8da9d5cd", "release": true, "await": true}] 2024-04-26T18:44:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.053+0000 7f59702a1b80 1 -- 172.21.15.84:0/3601219914 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "set_id": "8da9d5cd", "release": true, "await": true}) -- 0x55f64917e3b0 con 0x55f6492a9e10 2024-04-26T18:44:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.053+0000 7f595e7fc700 1 --2- 172.21.15.84:0/3601219914 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f6492a9e10 0x55f6492ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.054+0000 7f595e7fc700 1 --2- 172.21.15.84:0/3601219914 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f6492a9e10 0x55f6492ac200 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef1be21ee2c9015d server_cookie=532238c38af1cee2 in_seq=0 out_seq=0 2024-04-26T18:44:51.058 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.054+0000 7f593effd700 10 client.17421 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:51.059 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.058+0000 7f593effd700 1 -- 172.21.15.84:0/3601219914 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f64917e3b0 con 0x55f6492a9e10 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.058+0000 7f593effd700 10 client.17421 handle_command_reply: tid=0 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.058+0000 7f59702a1b80 1 client.17421 shutdown 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.058+0000 7f59702a1b80 1 -- 172.21.15.84:0/3601219914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 msgr2=0x55f6492a2230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.058+0000 7f59702a1b80 1 --2- 172.21.15.84:0/3601219914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 0x55f6492a2230 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f5950010be0 tx=0x7f595000a950 comp rx=0 tx=0).stop 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.058+0000 7f59702a1b80 1 -- 172.21.15.84:0/3601219914 shutdown_connections 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.058+0000 7f59702a1b80 1 --2- 172.21.15.84:0/3601219914 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55f6492a9e10 0x55f6492ac200 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.058+0000 7f59702a1b80 1 --2- 172.21.15.84:0/3601219914 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55f649224870 0x55f6492a2230 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.058+0000 7f59702a1b80 1 --2- 172.21.15.84:0/3601219914 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55f64922a7c0 0x55f64929c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.058+0000 7f593effd700 0 client.17421 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.058+0000 7f59702a1b80 1 --2- 172.21.15.84:0/3601219914 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55f649229ea0 0x55f6492a2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.058+0000 7f59702a1b80 1 -- 172.21.15.84:0/3601219914 >> 172.21.15.84:0/3601219914 conn(0x55f6491526c0 msgr2=0x55f64922ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f59702a1b80 1 -- 172.21.15.84:0/3601219914 shutdown_connections 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f59702a1b80 1 -- 172.21.15.84:0/3601219914 wait complete. 2024-04-26T18:44:51.060 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f59702a1b80 20 client.17421 trim_cache size 0 max 16384 2024-04-26T18:44:51.061 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:44:51.061 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:44:51.061 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:44:51.061 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 48, 2024-04-26T18:44:51.061 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": { 2024-04-26T18:44:51.061 INFO:teuthology.orchestra.run.smithi084.stdout: "8da9d5cd": { 2024-04-26T18:44:51.061 INFO:teuthology.orchestra.run.smithi084.stdout: "version": 48, 2024-04-26T18:44:51.061 INFO:teuthology.orchestra.run.smithi084.stdout: "age_ref": 0.0, 2024-04-26T18:44:51.061 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:44:51.062 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:44:51.062 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:44:51.062 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:44:51.062 INFO:teuthology.orchestra.run.smithi084.stdout: "timeout": 60.0, 2024-04-26T18:44:51.062 INFO:teuthology.orchestra.run.smithi084.stdout: "expiration": 92.6, 2024-04-26T18:44:51.062 INFO:teuthology.orchestra.run.smithi084.stdout: "members": { 2024-04-26T18:44:51.062 INFO:teuthology.orchestra.run.smithi084.stdout: "file:/": { 2024-04-26T18:44:51.062 INFO:teuthology.orchestra.run.smithi084.stdout: "excluded": false, 2024-04-26T18:44:51.062 INFO:teuthology.orchestra.run.smithi084.stdout: "state": { 2024-04-26T18:44:51.062 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RELEASED", 2024-04-26T18:44:51.062 INFO:teuthology.orchestra.run.smithi084.stdout: "age": 0.0 2024-04-26T18:44:51.062 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:44:51.062 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:44:51.063 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:44:51.063 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:44:51.063 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:44:51.063 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:44:51.063 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f5965a0f700 1 -- 172.21.15.84:0/574067511 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f594c0a3db0 msgr2=0x7f594c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f5965a0f700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f594c0a3db0 0x7f594c0a6270 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f595000abd0 tx=0x7f595000a380 comp rx=0 tx=0).stop 2024-04-26T18:44:51.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f5965a0f700 1 -- 172.21.15.84:0/574067511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5960108390 msgr2=0x7f59601987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f5965a0f700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5960108390 0x7f59601987e0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f595800efb0 tx=0x7f595800c550 comp rx=0 tx=0).stop 2024-04-26T18:44:51.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f5965a0f700 1 -- 172.21.15.84:0/574067511 shutdown_connections 2024-04-26T18:44:51.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f5965a0f700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f594c0a3db0 0x7f594c0a6270 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f5965a0f700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5960108390 0x7f59601987e0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f5965a0f700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5960102350 0x7f596019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f5965a0f700 1 --2- 172.21.15.84:0/574067511 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5960102cf0 0x7f596019eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f5965a0f700 1 -- 172.21.15.84:0/574067511 >> 172.21.15.84:0/574067511 conn(0x7f59600792c0 msgr2=0x7f5960101030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:51.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f5965a0f700 1 -- 172.21.15.84:0/574067511 shutdown_connections 2024-04-26T18:44:51.064 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:44:51.059+0000 7f5965a0f700 1 -- 172.21.15.84:0/574067511 wait complete. 2024-04-26T18:44:51.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:50 smithi103 ceph-mon[32727]: pgmap v531: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-04-26T18:44:51.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:50 smithi174 ceph-mon[32705]: pgmap v531: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2024-04-26T18:44:51.816 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '8da9d5cd', seconds elapsed: 44.0 2024-04-26T18:44:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.976+0000 7f8878d71640 1 -- 172.21.15.84:0/1496527061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88741032e0 msgr2=0x7f887410f7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.978 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.976+0000 7f8878d71640 1 --2- 172.21.15.84:0/1496527061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88741032e0 0x7f887410f7f0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f8864009cd0 tx=0x7f886402a8e0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.976+0000 7f8878d71640 1 -- 172.21.15.84:0/1496527061 shutdown_connections 2024-04-26T18:44:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.976+0000 7f8878d71640 1 --2- 172.21.15.84:0/1496527061 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f88741032e0 0x7f887410f7f0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.976+0000 7f8878d71640 1 --2- 172.21.15.84:0/1496527061 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874102920 0x7f8874102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.976+0000 7f8878d71640 1 --2- 172.21.15.84:0/1496527061 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8874108930 0x7f8874108d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.976+0000 7f8878d71640 1 -- 172.21.15.84:0/1496527061 >> 172.21.15.84:0/1496527061 conn(0x7f88740fe3f0 msgr2=0x7f8874100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.976+0000 7f8878d71640 1 -- 172.21.15.84:0/1496527061 shutdown_connections 2024-04-26T18:44:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.976+0000 7f8878d71640 1 -- 172.21.15.84:0/1496527061 wait complete. 2024-04-26T18:44:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f8878d71640 1 Processor -- start 2024-04-26T18:44:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f8878d71640 1 -- start start 2024-04-26T18:44:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f8878d71640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8874102920 0x7f8874199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f8878d71640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88741032e0 0x7f88741995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f8878d71640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874108930 0x7f887419d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.979 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f8878d71640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f8874112bf0 con 0x7f8874102920 2024-04-26T18:44:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f8878d71640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8874112a70 con 0x7f8874108930 2024-04-26T18:44:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f8878d71640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8874112d70 con 0x7f88741032e0 2024-04-26T18:44:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f887259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8874102920 0x7f8874199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f887259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8874102920 0x7f8874199090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47750/0 (socket says 172.21.15.84:47750) 2024-04-26T18:44:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f887259c640 1 -- 172.21.15.84:0/1121022189 learned_addr learned my addr 172.21.15.84:0/1121022189 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f8872d9d640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874108930 0x7f887419d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f887259c640 1 -- 172.21.15.84:0/1121022189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88741032e0 msgr2=0x7f88741995d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f8871d9b640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88741032e0 0x7f88741995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.980 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f887259c640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88741032e0 0x7f88741995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f887259c640 1 -- 172.21.15.84:0/1121022189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874108930 msgr2=0x7f887419d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f887259c640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874108930 0x7f887419d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f887259c640 1 -- 172.21.15.84:0/1121022189 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f887419e080 con 0x7f8874102920 2024-04-26T18:44:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f8872d9d640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874108930 0x7f887419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:44:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.977+0000 7f887259c640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8874102920 0x7f8874199090 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f886c00d770 tx=0x7f886c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.978+0000 7f88637fe640 1 -- 172.21.15.84:0/1121022189 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c017070 con 0x7f8874102920 2024-04-26T18:44:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.978+0000 7f88637fe640 1 -- 172.21.15.84:0/1121022189 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f886c0054d0 con 0x7f8874102920 2024-04-26T18:44:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.978+0000 7f8878d71640 1 -- 172.21.15.84:0/1121022189 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f887419e370 con 0x7f8874102920 2024-04-26T18:44:51.981 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.978+0000 7f88637fe640 1 -- 172.21.15.84:0/1121022189 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c004c30 con 0x7f8874102920 2024-04-26T18:44:51.982 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.978+0000 7f8878d71640 1 -- 172.21.15.84:0/1121022189 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88741a5c00 con 0x7f8874102920 2024-04-26T18:44:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.980+0000 7f88637fe640 1 -- 172.21.15.84:0/1121022189 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f886c004e10 con 0x7f8874102920 2024-04-26T18:44:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.980+0000 7f887bb66740 1 Processor -- start 2024-04-26T18:44:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.980+0000 7f887bb66740 1 -- start start 2024-04-26T18:44:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.980+0000 7f887bb66740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec3a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.988 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.980+0000 7f88637fe640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f885c0a3dc0 0x7f885c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.980+0000 7f887bb66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec3a820 0x558a4ec44ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.980+0000 7f887bb66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec450e0 0x558a4ec474d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.980+0000 7f887bb66740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558a4eba0460 con 0x558a4ec39f00 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.980+0000 7f887bb66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a4eb5a590 con 0x558a4ec450e0 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.980+0000 7f887bb66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a4eb9c420 con 0x558a4ec3a820 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f887259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec3a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f8872d9d640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f885c0a3dc0 0x7f885c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f887259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec3a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47758/0 (socket says 172.21.15.84:47758) 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f887259c640 1 -- 172.21.15.84:0/4184814880 learned_addr learned my addr 172.21.15.84:0/4184814880 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f8871d9b640 1 --2- 172.21.15.84:0/4184814880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec3a820 0x558a4ec44ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f88637fe640 1 -- 172.21.15.84:0/1121022189 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f886c0d0350 con 0x7f8874102920 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f8872d9d640 1 --2- 172.21.15.84:0/4184814880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec450e0 0x558a4ec474d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f887259c640 1 -- 172.21.15.84:0/4184814880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec3a820 msgr2=0x558a4ec44ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:51.989 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f887259c640 1 --2- 172.21.15.84:0/4184814880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec3a820 0x558a4ec44ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f887259c640 1 -- 172.21.15.84:0/4184814880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec450e0 msgr2=0x558a4ec474d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f887259c640 1 --2- 172.21.15.84:0/4184814880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec450e0 0x558a4ec474d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f887259c640 1 -- 172.21.15.84:0/4184814880 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a4eb670d0 con 0x558a4ec39f00 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f8872d9d640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f885c0a3dc0 0x7f885c0a6280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8864006010 tx=0x7f8864009800 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f887259c640 1 --2- 172.21.15.84:0/4184814880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec3a2e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f886c09bfb0 tx=0x7f886c09b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f7ffa003827f33 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f8860ff9640 1 -- 172.21.15.84:0/4184814880 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c020040 con 0x558a4ec39f00 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f8860ff9640 1 -- 172.21.15.84:0/4184814880 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f886c09c2b0 con 0x558a4ec39f00 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.981+0000 7f8860ff9640 1 -- 172.21.15.84:0/4184814880 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c098920 con 0x558a4ec39f00 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.982+0000 7f887bb66740 1 -- 172.21.15.84:0/4184814880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 msgr2=0x558a4ec3a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.982+0000 7f887bb66740 1 --2- 172.21.15.84:0/4184814880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec3a2e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f886c09bfb0 tx=0x7f886c09b700 comp rx=0 tx=0).stop 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.982+0000 7f887bb66740 1 -- 172.21.15.84:0/4184814880 shutdown_connections 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.982+0000 7f887bb66740 1 --2- 172.21.15.84:0/4184814880 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec450e0 0x558a4ec474d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.982+0000 7f887bb66740 1 --2- 172.21.15.84:0/4184814880 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec3a820 0x558a4ec44ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.990 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.982+0000 7f887bb66740 1 --2- 172.21.15.84:0/4184814880 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec3a2e0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.982+0000 7f887bb66740 1 -- 172.21.15.84:0/4184814880 >> 172.21.15.84:0/4184814880 conn(0x558a4ec36410 msgr2=0x558a4ec367f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.982+0000 7f887bb66740 1 -- 172.21.15.84:0/4184814880 shutdown_connections 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.982+0000 7f887bb66740 1 -- 172.21.15.84:0/4184814880 wait complete. 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.982+0000 7f887bb66740 1 -- start start 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.982+0000 7f887bb66740 1 Processor -- start 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887bb66740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec42060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887bb66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec3a820 0x558a4ec425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887bb66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec450e0 0x558a4ec42ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887bb66740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x558a4ec47d10 con 0x558a4ec39f00 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887bb66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x558a4eb27b60 con 0x558a4ec3a820 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887bb66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x558a4ec47e90 con 0x558a4ec450e0 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887bb66740 10 client.0 _ll_register_callbacks cb 0x558a4eb4c2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec42060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.991 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec42060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47766/0 (socket says 172.21.15.84:47766) 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887259c640 1 -- 172.21.15.84:0/944317409 learned_addr learned my addr 172.21.15.84:0/944317409 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f8871d9b640 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec3a820 0x558a4ec425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f8872d9d640 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec450e0 0x558a4ec42ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887259c640 1 -- 172.21.15.84:0/944317409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec450e0 msgr2=0x558a4ec42ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887259c640 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec450e0 0x558a4ec42ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887259c640 1 -- 172.21.15.84:0/944317409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec3a820 msgr2=0x558a4ec425a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887259c640 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec3a820 0x558a4ec425a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887259c640 1 -- 172.21.15.84:0/944317409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a4e8b2210 con 0x558a4ec39f00 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f8872d9d640 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec450e0 0x558a4ec42ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887259c640 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec42060 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f886c098710 tx=0x7f886c09ba60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb060fc349d4b81b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f8859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f8859ffb640 1 -- 172.21.15.84:0/944317409 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c020070 con 0x558a4ec39f00 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f8859ffb640 1 -- 172.21.15.84:0/944317409 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f886c098920 con 0x558a4ec39f00 2024-04-26T18:44:51.992 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887bb66740 1 -- 172.21.15.84:0/944317409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f886c09c2b0 con 0x558a4ec39f00 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f8859ffb640 1 -- 172.21.15.84:0/944317409 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f886c0a4f20 con 0x558a4ec39f00 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.983+0000 7f887bb66740 10 client.17433 fetch_fsmap learned FSMap version 29 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.984+0000 7f887bb66740 1 -- 172.21.15.84:0/944317409 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a4e97e620 con 0x558a4ec39f00 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.984+0000 7f8859ffb640 1 -- 172.21.15.84:0/944317409 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f886c09c2b0 con 0x558a4ec39f00 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.984+0000 7f8859ffb640 1 -- 172.21.15.84:0/944317409 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f886c0bb020 con 0x558a4ec39f00 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.984+0000 7f887bb66740 10 client.17433 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.984+0000 7f887bb66740 10 client.17433 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.984+0000 7f887bb66740 20 client.17433 populate_metadata read hostname 'smithi084' 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.984+0000 7f887bb66740 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a4ec55090 0x558a4ec57480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.984+0000 7f887bb66740 4 client.17433 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.984+0000 7f887bb66740 1 -- 172.21.15.84:0/944317409 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a4ebae070 con 0x558a4ec55090 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.984+0000 7f8871d9b640 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a4ec55090 0x558a4ec57480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.985+0000 7f8859ffb640 10 client.17433 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.985+0000 7f8871d9b640 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a4ec55090 0x558a4ec57480 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c96d79903edd32ba server_cookie=9a264729257d3829 in_seq=0 out_seq=0 2024-04-26T18:44:51.993 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.985+0000 7f8859ffb640 1 -- 172.21.15.84:0/944317409 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558a4ebae070 con 0x558a4ec55090 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.985+0000 7f8859ffb640 10 client.17433 handle_command_reply: tid=0 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.985+0000 7f887bb66740 1 client.17433 shutdown 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.986+0000 7f887bb66740 1 -- 172.21.15.84:0/944317409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 msgr2=0x558a4ec42060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.986+0000 7f887bb66740 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec42060 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f886c098710 tx=0x7f886c09ba60 comp rx=0 tx=0).stop 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.986+0000 7f887bb66740 1 -- 172.21.15.84:0/944317409 shutdown_connections 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.986+0000 7f887bb66740 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a4ec55090 0x558a4ec57480 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.986+0000 7f887bb66740 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec450e0 0x558a4ec42ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.986+0000 7f887bb66740 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec3a820 0x558a4ec425a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.986+0000 7f8859ffb640 0 client.17433 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.986+0000 7f887bb66740 1 --2- 172.21.15.84:0/944317409 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec42060 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.986+0000 7f887bb66740 1 -- 172.21.15.84:0/944317409 >> 172.21.15.84:0/944317409 conn(0x558a4ec36410 msgr2=0x558a4ec3c190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.986+0000 7f887bb66740 1 -- 172.21.15.84:0/944317409 shutdown_connections 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.986+0000 7f887bb66740 1 -- 172.21.15.84:0/944317409 wait complete. 2024-04-26T18:44:51.994 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:51.986+0000 7f887bb66740 20 client.17433 trim_cache size 0 max 16384 2024-04-26T18:44:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.000+0000 7f887bb66740 1 Processor -- start 2024-04-26T18:44:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.000+0000 7f887bb66740 1 -- start start 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.000+0000 7f887bb66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec39f00 0x558a4ec35d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.000+0000 7f887bb66740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec3a820 0x558a4ec44120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.000+0000 7f887bb66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec450e0 0x558a4ec507f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.000+0000 7f887bb66740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f88741ae580 con 0x558a4ec3a820 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.000+0000 7f887bb66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88741b00c0 con 0x558a4ec39f00 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.000+0000 7f887bb66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f88741aec50 con 0x558a4ec450e0 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.000+0000 7f8871d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec3a820 0x558a4ec44120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.001+0000 7f8871d9b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec3a820 0x558a4ec44120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47772/0 (socket says 172.21.15.84:47772) 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.001+0000 7f8871d9b640 1 -- 172.21.15.84:0/3276567744 learned_addr learned my addr 172.21.15.84:0/3276567744 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.001+0000 7f8871d9b640 1 -- 172.21.15.84:0/3276567744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec450e0 msgr2=0x558a4ec507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.001+0000 7f8871d9b640 1 --2- 172.21.15.84:0/3276567744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec450e0 0x558a4ec507f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.001+0000 7f8871d9b640 1 -- 172.21.15.84:0/3276567744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec39f00 msgr2=0x558a4ec35d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.001+0000 7f8871d9b640 1 --2- 172.21.15.84:0/3276567744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec39f00 0x558a4ec35d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.005 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.001+0000 7f8871d9b640 1 -- 172.21.15.84:0/3276567744 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a4ec418f0 con 0x558a4ec3a820 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.001+0000 7f8871d9b640 1 --2- 172.21.15.84:0/3276567744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec3a820 0x558a4ec44120 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8868004700 tx=0x7f886800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cebda196cc96d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.001+0000 7f885bfff640 1 -- 172.21.15.84:0/3276567744 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886801b430 con 0x558a4ec3a820 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.001+0000 7f885bfff640 1 -- 172.21.15.84:0/3276567744 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f886801a390 con 0x558a4ec3a820 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.001+0000 7f885bfff640 1 -- 172.21.15.84:0/3276567744 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886800e610 con 0x558a4ec3a820 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 -- 172.21.15.84:0/3276567744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec3a820 msgr2=0x558a4ec44120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 --2- 172.21.15.84:0/3276567744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec3a820 0x558a4ec44120 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f8868004700 tx=0x7f886800f7f0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 -- 172.21.15.84:0/3276567744 shutdown_connections 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 --2- 172.21.15.84:0/3276567744 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec450e0 0x558a4ec507f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 --2- 172.21.15.84:0/3276567744 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec3a820 0x558a4ec44120 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 --2- 172.21.15.84:0/3276567744 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec39f00 0x558a4ec35d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 -- 172.21.15.84:0/3276567744 >> 172.21.15.84:0/3276567744 conn(0x558a4ec36410 msgr2=0x558a4ec500e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 -- 172.21.15.84:0/3276567744 shutdown_connections 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 -- 172.21.15.84:0/3276567744 wait complete. 2024-04-26T18:44:52.006 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 -- start start 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 Processor -- start 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec585e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec3a820 0x558a4ec58b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec450e0 0x558a4ecba270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f88741af9f0 con 0x558a4ec39f00 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f88741af320 con 0x558a4ec450e0 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f88741ac580 con 0x558a4ec3a820 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887bb66740 10 client.0 _ll_register_callbacks cb 0x7f8868005f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec585e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:47776/0 (socket says 172.21.15.84:47776) 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887259c640 1 -- 172.21.15.84:0/688265595 learned_addr learned my addr 172.21.15.84:0/688265595 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f8871d9b640 1 --2- 172.21.15.84:0/688265595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec3a820 0x558a4ec58b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887259c640 1 -- 172.21.15.84:0/688265595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec3a820 msgr2=0x558a4ec58b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887259c640 1 --2- 172.21.15.84:0/688265595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec3a820 0x558a4ec58b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887259c640 1 -- 172.21.15.84:0/688265595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec450e0 msgr2=0x558a4ecba270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887259c640 1 --2- 172.21.15.84:0/688265595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec450e0 0x558a4ecba270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.002+0000 7f887259c640 1 -- 172.21.15.84:0/688265595 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a4ecba7b0 con 0x558a4ec39f00 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f887259c640 1 --2- 172.21.15.84:0/688265595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec585e0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f886c0ce060 tx=0x7f886c005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4d0bb56f4719f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f8859ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f8859ffb640 1 -- 172.21.15.84:0/688265595 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886c020070 con 0x558a4ec39f00 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f8859ffb640 1 -- 172.21.15.84:0/688265595 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(116..116 src has 1..116) ==== 8579+0+0 (secure 0 0 0) 0x7f886c09b8a0 con 0x558a4ec39f00 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f887bb66740 1 -- 172.21.15.84:0/688265595 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f886801a390 con 0x558a4ec39f00 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f8859ffb640 1 -- 172.21.15.84:0/688265595 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f886c005680 con 0x558a4ec39f00 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f887bb66740 10 client.17439 fetch_fsmap learned FSMap version 29 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f887bb66740 1 -- 172.21.15.84:0/688265595 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558a4ecbba70 con 0x558a4ec39f00 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f8859ffb640 1 -- 172.21.15.84:0/688265595 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f886801a390 con 0x558a4ec39f00 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f8859ffb640 1 -- 172.21.15.84:0/688265595 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f886c09b240 con 0x558a4ec39f00 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f887bb66740 10 client.17439 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:44:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f887bb66740 10 client.17439 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:44:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f887bb66740 20 client.17439 populate_metadata read hostname 'smithi084' 2024-04-26T18:44:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f887bb66740 1 --2- 172.21.15.84:0/688265595 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a4ecbbfa0 0x558a4ecbe390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:44:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f887bb66740 4 client.17439 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:44:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.003+0000 7f887bb66740 1 -- 172.21.15.84:0/688265595 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558a4eba7ee0 con 0x558a4ecbbfa0 2024-04-26T18:44:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.004+0000 7f8871d9b640 1 --2- 172.21.15.84:0/688265595 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a4ecbbfa0 0x558a4ecbe390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:44:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.004+0000 7f8871d9b640 1 --2- 172.21.15.84:0/688265595 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a4ecbbfa0 0x558a4ecbe390 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de3e1fee884ffc8b server_cookie=d7e97963629e6f39 in_seq=0 out_seq=0 2024-04-26T18:44:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.004+0000 7f8859ffb640 10 client.17439 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.015+0000 7f8859ffb640 1 -- 172.21.15.84:0/688265595 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558a4eba7ee0 con 0x558a4ecbbfa0 2024-04-26T18:44:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.015+0000 7f8859ffb640 10 client.17439 handle_command_reply: tid=0 2024-04-26T18:44:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.015+0000 7f887bb66740 1 client.17439 shutdown 2024-04-26T18:44:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.015+0000 7f887bb66740 1 -- 172.21.15.84:0/688265595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 msgr2=0x558a4ec585e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.015+0000 7f887bb66740 1 --2- 172.21.15.84:0/688265595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec585e0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f886c0ce060 tx=0x7f886c005bb0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.015+0000 7f887bb66740 1 -- 172.21.15.84:0/688265595 shutdown_connections 2024-04-26T18:44:52.016 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.015+0000 7f887bb66740 1 --2- 172.21.15.84:0/688265595 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x558a4ecbbfa0 0x558a4ecbe390 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.015+0000 7f887bb66740 1 --2- 172.21.15.84:0/688265595 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x558a4ec450e0 0x558a4ecba270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.015+0000 7f8859ffb640 0 client.17439 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:44:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.015+0000 7f887bb66740 1 --2- 172.21.15.84:0/688265595 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x558a4ec3a820 0x558a4ec58b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.015+0000 7f887bb66740 1 --2- 172.21.15.84:0/688265595 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x558a4ec39f00 0x558a4ec585e0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.015+0000 7f887bb66740 1 -- 172.21.15.84:0/688265595 >> 172.21.15.84:0/688265595 conn(0x558a4ec36410 msgr2=0x558a4ec41090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f887bb66740 1 -- 172.21.15.84:0/688265595 shutdown_connections 2024-04-26T18:44:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f887bb66740 1 -- 172.21.15.84:0/688265595 wait complete. 2024-04-26T18:44:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f887bb66740 20 client.17439 trim_cache size 0 max 16384 2024-04-26T18:44:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:44:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:44:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:44:52.017 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:44:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f8878d71640 1 -- 172.21.15.84:0/1121022189 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f885c0a3dc0 msgr2=0x7f885c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f8878d71640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f885c0a3dc0 0x7f885c0a6280 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8864006010 tx=0x7f8864009800 comp rx=0 tx=0).stop 2024-04-26T18:44:52.018 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f8878d71640 1 -- 172.21.15.84:0/1121022189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8874102920 msgr2=0x7f8874199090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:44:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f8878d71640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8874102920 0x7f8874199090 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f886c00d770 tx=0x7f886c00dc30 comp rx=0 tx=0).stop 2024-04-26T18:44:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f8878d71640 1 -- 172.21.15.84:0/1121022189 shutdown_connections 2024-04-26T18:44:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f8878d71640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f885c0a3dc0 0x7f885c0a6280 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f8878d71640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f8874108930 0x7f887419d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f8878d71640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f88741032e0 0x7f88741995d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f8878d71640 1 --2- 172.21.15.84:0/1121022189 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8874102920 0x7f8874199090 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:44:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f8878d71640 1 -- 172.21.15.84:0/1121022189 >> 172.21.15.84:0/1121022189 conn(0x7f88740fe3f0 msgr2=0x7f88740feb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:44:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f8878d71640 1 -- 172.21.15.84:0/1121022189 shutdown_connections 2024-04-26T18:44:52.019 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:44:52.016+0000 7f8878d71640 1 -- 172.21.15.84:0/1121022189 wait complete. 2024-04-26T18:44:53.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:52 smithi084 ceph-mon[29141]: pgmap v532: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-26T18:44:53.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:52 smithi103 ceph-mon[32727]: pgmap v532: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-26T18:44:53.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:52 smithi174 ceph-mon[32705]: pgmap v532: 97 pgs: 97 active+clean; 3.3 MiB data, 449 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-26T18:44:55.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:54 smithi084 ceph-mon[29141]: pgmap v533: 97 pgs: 97 active+clean; 3.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-26T18:44:55.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:54 smithi103 ceph-mon[32727]: pgmap v533: 97 pgs: 97 active+clean; 3.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-26T18:44:55.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:54 smithi174 ceph-mon[32705]: pgmap v533: 97 pgs: 97 active+clean; 3.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 18 KiB/s wr, 18 op/s 2024-04-26T18:44:57.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:56 smithi084 ceph-mon[29141]: pgmap v534: 97 pgs: 97 active+clean; 3.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-26T18:44:57.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:56 smithi103 ceph-mon[32727]: pgmap v534: 97 pgs: 97 active+clean; 3.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-26T18:44:57.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:56 smithi174 ceph-mon[32705]: pgmap v534: 97 pgs: 97 active+clean; 3.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-26T18:44:59.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:44:58 smithi084 ceph-mon[29141]: pgmap v535: 97 pgs: 97 active+clean; 3.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-26T18:44:59.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:44:58 smithi103 ceph-mon[32727]: pgmap v535: 97 pgs: 97 active+clean; 3.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-26T18:44:59.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:44:58 smithi174 ceph-mon[32705]: pgmap v535: 97 pgs: 97 active+clean; 3.4 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 14 KiB/s wr, 12 op/s 2024-04-26T18:45:01.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:00 smithi084 ceph-mon[29141]: pgmap v536: 97 pgs: 97 active+clean; 3.6 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 18 op/s 2024-04-26T18:45:01.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:00 smithi103 ceph-mon[32727]: pgmap v536: 97 pgs: 97 active+clean; 3.6 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 18 op/s 2024-04-26T18:45:01.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:00 smithi174 ceph-mon[32705]: pgmap v536: 97 pgs: 97 active+clean; 3.6 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 18 op/s 2024-04-26T18:45:03.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:02 smithi084 ceph-mon[29141]: pgmap v537: 97 pgs: 97 active+clean; 3.6 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-26T18:45:03.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:02 smithi103 ceph-mon[32727]: pgmap v537: 97 pgs: 97 active+clean; 3.6 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-26T18:45:03.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:02 smithi174 ceph-mon[32705]: pgmap v537: 97 pgs: 97 active+clean; 3.6 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-26T18:45:05.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:04 smithi084 ceph-mon[29141]: pgmap v538: 97 pgs: 97 active+clean; 3.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-04-26T18:45:05.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:04 smithi084 ceph-mon[29141]: osdmap e117: 12 total, 12 up, 12 in 2024-04-26T18:45:05.112 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.112+0000 7fe358ef1640 1 -- 172.21.15.84:0/2596302303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe354101850 msgr2=0x7fe354101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.112+0000 7fe358ef1640 1 --2- 172.21.15.84:0/2596302303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe354101850 0x7fe354101cd0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009930 tx=0x7fe33c02a420 comp rx=0 tx=0).stop 2024-04-26T18:45:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.112+0000 7fe358ef1640 1 -- 172.21.15.84:0/2596302303 shutdown_connections 2024-04-26T18:45:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.112+0000 7fe358ef1640 1 --2- 172.21.15.84:0/2596302303 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe354102210 0x7fe35410e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.112+0000 7fe358ef1640 1 --2- 172.21.15.84:0/2596302303 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe354101850 0x7fe354101cd0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.112+0000 7fe358ef1640 1 --2- 172.21.15.84:0/2596302303 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe354107860 0x7fe354107c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.112+0000 7fe358ef1640 1 -- 172.21.15.84:0/2596302303 >> 172.21.15.84:0/2596302303 conn(0x7fe354077bb0 msgr2=0x7fe354077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.112+0000 7fe358ef1640 1 -- 172.21.15.84:0/2596302303 shutdown_connections 2024-04-26T18:45:05.113 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.112+0000 7fe358ef1640 1 -- 172.21.15.84:0/2596302303 wait complete. 2024-04-26T18:45:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.113+0000 7fe358ef1640 1 Processor -- start 2024-04-26T18:45:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.113+0000 7fe358ef1640 1 -- start start 2024-04-26T18:45:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.113+0000 7fe358ef1640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe354101850 0x7fe354197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.115 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.113+0000 7fe358ef1640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe354102210 0x7fe354198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.113+0000 7fe358ef1640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe354107860 0x7fe35419c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.113+0000 7fe358ef1640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe354111b20 con 0x7fe354107860 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.113+0000 7fe358ef1640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3541119a0 con 0x7fe354101850 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.113+0000 7fe358ef1640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe354111ca0 con 0x7fe354102210 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.113+0000 7fe352d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe354107860 0x7fe35419c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.113+0000 7fe352d9d640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe354107860 0x7fe35419c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54236/0 (socket says 172.21.15.84:54236) 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.113+0000 7fe352d9d640 1 -- 172.21.15.84:0/3951245260 learned_addr learned my addr 172.21.15.84:0/3951245260 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe35259c640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe354101850 0x7fe354197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe34bfff640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe354102210 0x7fe354198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe352d9d640 1 -- 172.21.15.84:0/3951245260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe354102210 msgr2=0x7fe354198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe352d9d640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe354102210 0x7fe354198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe352d9d640 1 -- 172.21.15.84:0/3951245260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe354101850 msgr2=0x7fe354197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe352d9d640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe354101850 0x7fe354197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe352d9d640 1 -- 172.21.15.84:0/3951245260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe35419cf30 con 0x7fe354107860 2024-04-26T18:45:05.116 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe35259c640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe354101850 0x7fe354197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:45:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe352d9d640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe354107860 0x7fe35419c800 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe34c007cb0 tx=0x7fe34c00e420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe34b7fe640 1 -- 172.21.15.84:0/3951245260 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c00eb30 con 0x7fe354107860 2024-04-26T18:45:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe358ef1640 1 -- 172.21.15.84:0/3951245260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe35419d220 con 0x7fe354107860 2024-04-26T18:45:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe34b7fe640 1 -- 172.21.15.84:0/3951245260 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe34c00ecd0 con 0x7fe354107860 2024-04-26T18:45:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe34b7fe640 1 -- 172.21.15.84:0/3951245260 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0153f0 con 0x7fe354107860 2024-04-26T18:45:05.117 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.114+0000 7fe358ef1640 1 -- 172.21.15.84:0/3951245260 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe35406b160 con 0x7fe354107860 2024-04-26T18:45:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.116+0000 7fe34b7fe640 1 -- 172.21.15.84:0/3951245260 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fe34c00a440 con 0x7fe354107860 2024-04-26T18:45:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.117+0000 7fe35bc09740 1 Processor -- start 2024-04-26T18:45:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.117+0000 7fe35bc09740 1 -- start start 2024-04-26T18:45:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.117+0000 7fe35bc09740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7ae00 0x562dbfd7b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.117+0000 7fe35bc09740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 0x562dbfd89fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.117+0000 7fe35bc09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd8a4f0 0x562dbfd8c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.117+0000 7fe35bc09740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562dbfce1460 con 0x562dbfd7b720 2024-04-26T18:45:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.117+0000 7fe35bc09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562dbfc9b590 con 0x562dbfd8a4f0 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.117+0000 7fe35bc09740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562dbfcdd420 con 0x562dbfd7ae00 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.117+0000 7fe35259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 0x562dbfd89fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.117+0000 7fe35259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 0x562dbfd89fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54242/0 (socket says 172.21.15.84:54242) 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.117+0000 7fe35259c640 1 -- 172.21.15.84:0/2230555334 learned_addr learned my addr 172.21.15.84:0/2230555334 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.117+0000 7fe34b7fe640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe3400a3dc0 0x7fe3400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.117+0000 7fe34bfff640 1 --2- 172.21.15.84:0/2230555334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd8a4f0 0x562dbfd8c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe352d9d640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe3400a3dc0 0x7fe3400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe35259c640 1 -- 172.21.15.84:0/2230555334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7ae00 msgr2=0x562dbfd7b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe34bfff640 1 --2- 172.21.15.84:0/2230555334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7ae00 0x562dbfd7b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe35259c640 1 --2- 172.21.15.84:0/2230555334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7ae00 0x562dbfd7b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe35259c640 1 -- 172.21.15.84:0/2230555334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd8a4f0 msgr2=0x562dbfd8c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe35259c640 1 --2- 172.21.15.84:0/2230555334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd8a4f0 0x562dbfd8c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe35259c640 1 -- 172.21.15.84:0/2230555334 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dbfca80d0 con 0x562dbfd7b720 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe34bfff640 1 --2- 172.21.15.84:0/2230555334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7ae00 0x562dbfd7b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:45:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe34b7fe640 1 -- 172.21.15.84:0/3951245260 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 8623+0+0 (secure 0 0 0) 0x7fe34c0d24f0 con 0x7fe354107860 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe35259c640 1 --2- 172.21.15.84:0/2230555334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 0x562dbfd89fb0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe34400cbf0 tx=0x7fe34400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d3fcea79b342168 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe352d9d640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe3400a3dc0 0x7fe3400a6280 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0cfc90 tx=0x7fe34c00aa60 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe348ff9640 1 -- 172.21.15.84:0/2230555334 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344005220 con 0x562dbfd7b720 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe348ff9640 1 -- 172.21.15.84:0/2230555334 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe344016070 con 0x562dbfd7b720 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.118+0000 7fe348ff9640 1 -- 172.21.15.84:0/2230555334 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe344010630 con 0x562dbfd7b720 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.119+0000 7fe35bc09740 1 -- 172.21.15.84:0/2230555334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 msgr2=0x562dbfd89fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.119+0000 7fe35bc09740 1 --2- 172.21.15.84:0/2230555334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 0x562dbfd89fb0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fe34400cbf0 tx=0x7fe34400e720 comp rx=0 tx=0).stop 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.119+0000 7fe35bc09740 1 -- 172.21.15.84:0/2230555334 shutdown_connections 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.119+0000 7fe35bc09740 1 --2- 172.21.15.84:0/2230555334 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd8a4f0 0x562dbfd8c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.119+0000 7fe35bc09740 1 --2- 172.21.15.84:0/2230555334 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 0x562dbfd89fb0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.119+0000 7fe35bc09740 1 --2- 172.21.15.84:0/2230555334 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7ae00 0x562dbfd7b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.119+0000 7fe35bc09740 1 -- 172.21.15.84:0/2230555334 >> 172.21.15.84:0/2230555334 conn(0x562dbfd77410 msgr2=0x562dbfd777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.119+0000 7fe35bc09740 1 -- 172.21.15.84:0/2230555334 shutdown_connections 2024-04-26T18:45:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.119+0000 7fe35bc09740 1 -- 172.21.15.84:0/2230555334 wait complete. 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.119+0000 7fe35bc09740 1 -- start start 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.119+0000 7fe35bc09740 1 Processor -- start 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35bc09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7ae00 0x562dbfd873d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35bc09740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 0x562dbfd87910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35bc09740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd8a4f0 0x562dbfd87e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35bc09740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x562dbfd8d120 con 0x562dbfd7b720 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35bc09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x562dbfc68b60 con 0x562dbfd7ae00 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35bc09740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x562dbfd8d2a0 con 0x562dbfd8a4f0 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35bc09740 10 client.0 _ll_register_callbacks cb 0x562dbfc8d2b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 0x562dbfd87910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 0x562dbfd87910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54258/0 (socket says 172.21.15.84:54258) 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35259c640 1 -- 172.21.15.84:0/568796987 learned_addr learned my addr 172.21.15.84:0/568796987 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe34bfff640 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd8a4f0 0x562dbfd87e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe34bfff640 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7ae00 0x562dbfd873d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35259c640 1 -- 172.21.15.84:0/568796987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd8a4f0 msgr2=0x562dbfd87e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35259c640 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd8a4f0 0x562dbfd87e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35259c640 1 -- 172.21.15.84:0/568796987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7ae00 msgr2=0x562dbfd873d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35259c640 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7ae00 0x562dbfd873d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35259c640 1 -- 172.21.15.84:0/568796987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dbf9f3210 con 0x562dbfd7b720 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe34bfff640 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7ae00 0x562dbfd873d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35259c640 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 0x562dbfd87910 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe344005d70 tx=0x7fe344010b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4ae1ee9f461ef4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe339ffb640 1 -- 172.21.15.84:0/568796987 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe34402a060 con 0x562dbfd7b720 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe339ffb640 1 -- 172.21.15.84:0/568796987 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8623+0+0 (secure 0 0 0) 0x7fe344010ca0 con 0x562dbfd7b720 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.120+0000 7fe35bc09740 1 -- 172.21.15.84:0/568796987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe344016070 con 0x562dbfd7b720 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.121+0000 7fe339ffb640 1 -- 172.21.15.84:0/568796987 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe344025470 con 0x562dbfd7b720 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.121+0000 7fe35bc09740 10 client.17451 fetch_fsmap learned FSMap version 29 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.121+0000 7fe35bc09740 1 -- 172.21.15.84:0/568796987 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562dbfabf620 con 0x562dbfd7b720 2024-04-26T18:45:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.121+0000 7fe339ffb640 1 -- 172.21.15.84:0/568796987 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe344016070 con 0x562dbfd7b720 2024-04-26T18:45:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.121+0000 7fe339ffb640 1 -- 172.21.15.84:0/568796987 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe34400f3c0 con 0x562dbfd7b720 2024-04-26T18:45:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.121+0000 7fe35bc09740 10 client.17451 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:45:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.121+0000 7fe35bc09740 10 client.17451 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:45:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.121+0000 7fe35bc09740 20 client.17451 populate_metadata read hostname 'smithi084' 2024-04-26T18:45:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.121+0000 7fe35bc09740 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562dbfd9a490 0x562dbfd9c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.121+0000 7fe35bc09740 4 client.17451 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:45:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.121+0000 7fe35bc09740 1 -- 172.21.15.84:0/568796987 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562dbfcef070 con 0x562dbfd9a490 2024-04-26T18:45:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.121+0000 7fe34bfff640 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562dbfd9a490 0x562dbfd9c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.122+0000 7fe339ffb640 10 client.17451 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:45:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.122+0000 7fe34bfff640 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562dbfd9a490 0x562dbfd9c880 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0988a38fad58fc0 server_cookie=3f528dfd8c86c9ad in_seq=0 out_seq=0 2024-04-26T18:45:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.122+0000 7fe339ffb640 1 -- 172.21.15.84:0/568796987 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562dbfcef070 con 0x562dbfd9a490 2024-04-26T18:45:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.122+0000 7fe339ffb640 10 client.17451 handle_command_reply: tid=0 2024-04-26T18:45:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.122+0000 7fe35bc09740 1 client.17451 shutdown 2024-04-26T18:45:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.123+0000 7fe35bc09740 1 -- 172.21.15.84:0/568796987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 msgr2=0x562dbfd87910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.123+0000 7fe35bc09740 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 0x562dbfd87910 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fe344005d70 tx=0x7fe344010b00 comp rx=0 tx=0).stop 2024-04-26T18:45:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.123+0000 7fe35bc09740 1 -- 172.21.15.84:0/568796987 shutdown_connections 2024-04-26T18:45:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.123+0000 7fe35bc09740 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562dbfd9a490 0x562dbfd9c880 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.123+0000 7fe35bc09740 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd8a4f0 0x562dbfd87e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.123+0000 7fe35bc09740 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd7b720 0x562dbfd87910 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.123+0000 7fe339ffb640 0 client.17451 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:45:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.123+0000 7fe35bc09740 1 --2- 172.21.15.84:0/568796987 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7ae00 0x562dbfd873d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.123+0000 7fe35bc09740 1 -- 172.21.15.84:0/568796987 >> 172.21.15.84:0/568796987 conn(0x562dbfd77410 msgr2=0x562dbfd7d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.123+0000 7fe35bc09740 1 -- 172.21.15.84:0/568796987 shutdown_connections 2024-04-26T18:45:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.123+0000 7fe35bc09740 1 -- 172.21.15.84:0/568796987 wait complete. 2024-04-26T18:45:05.130 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.123+0000 7fe35bc09740 20 client.17451 trim_cache size 0 max 16384 2024-04-26T18:45:05.131 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:04 smithi103 ceph-mon[32727]: pgmap v538: 97 pgs: 97 active+clean; 3.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-04-26T18:45:05.131 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:04 smithi103 ceph-mon[32727]: osdmap e117: 12 total, 12 up, 12 in 2024-04-26T18:45:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.137+0000 7fe35bc09740 1 Processor -- start 2024-04-26T18:45:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.137+0000 7fe35bc09740 1 -- start start 2024-04-26T18:45:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe35bc09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7ae00 0x562dbfd95fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe35bc09740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7b720 0x562dbfda1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe35bc09740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 0x562dbfda2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe35bc09740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe3541ac440 con 0x562dbfd8a4f0 2024-04-26T18:45:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe35bc09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3541adf80 con 0x562dbfd7ae00 2024-04-26T18:45:05.142 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe35bc09740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3541acb10 con 0x562dbfd7b720 2024-04-26T18:45:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe34bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 0x562dbfda2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe34bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 0x562dbfda2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54272/0 (socket says 172.21.15.84:54272) 2024-04-26T18:45:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe34bfff640 1 -- 172.21.15.84:0/1724064168 learned_addr learned my addr 172.21.15.84:0/1724064168 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe35259c640 1 --2- 172.21.15.84:0/1724064168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7b720 0x562dbfda1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe34bfff640 1 -- 172.21.15.84:0/1724064168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7b720 msgr2=0x562dbfda1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe34bfff640 1 --2- 172.21.15.84:0/1724064168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7b720 0x562dbfda1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe34bfff640 1 -- 172.21.15.84:0/1724064168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7ae00 msgr2=0x562dbfd95fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe34bfff640 1 --2- 172.21.15.84:0/1724064168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7ae00 0x562dbfd95fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe34bfff640 1 -- 172.21.15.84:0/1724064168 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dbfd7bc60 con 0x562dbfd8a4f0 2024-04-26T18:45:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe35259c640 1 --2- 172.21.15.84:0/1724064168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7b720 0x562dbfda1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:45:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe34bfff640 1 --2- 172.21.15.84:0/1724064168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 0x562dbfda2390 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009900 tx=0x7fe33c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c59a2d8103e3ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe33bfff640 1 -- 172.21.15.84:0/1724064168 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c035040 con 0x562dbfd8a4f0 2024-04-26T18:45:05.143 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe33bfff640 1 -- 172.21.15.84:0/1724064168 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33c04a410 con 0x562dbfd8a4f0 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.138+0000 7fe33bfff640 1 -- 172.21.15.84:0/1724064168 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c005280 con 0x562dbfd8a4f0 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.139+0000 7fe35bc09740 1 -- 172.21.15.84:0/1724064168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 msgr2=0x562dbfda2390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.139+0000 7fe35bc09740 1 --2- 172.21.15.84:0/1724064168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 0x562dbfda2390 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009900 tx=0x7fe33c000bc0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.139+0000 7fe35bc09740 1 -- 172.21.15.84:0/1724064168 shutdown_connections 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.139+0000 7fe35bc09740 1 --2- 172.21.15.84:0/1724064168 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 0x562dbfda2390 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.139+0000 7fe35bc09740 1 --2- 172.21.15.84:0/1724064168 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7b720 0x562dbfda1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.139+0000 7fe35bc09740 1 --2- 172.21.15.84:0/1724064168 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7ae00 0x562dbfd95fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.139+0000 7fe35bc09740 1 -- 172.21.15.84:0/1724064168 >> 172.21.15.84:0/1724064168 conn(0x562dbfd77410 msgr2=0x562dbfd777f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.139+0000 7fe35bc09740 1 -- 172.21.15.84:0/1724064168 shutdown_connections 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.139+0000 7fe35bc09740 1 -- 172.21.15.84:0/1724064168 wait complete. 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.139+0000 7fe35bc09740 1 -- start start 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.139+0000 7fe35bc09740 1 Processor -- start 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe35bc09740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7ae00 0x562dbfd83ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe35bc09740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7b720 0x562dbfd84530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe35bc09740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 0x562dbfd84a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe35bc09740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fe3541ad8b0 con 0x562dbfd8a4f0 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe35bc09740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fe3541ad1e0 con 0x562dbfd7b720 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe35bc09740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fe3541aa440 con 0x562dbfd7ae00 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe35bc09740 10 client.0 _ll_register_callbacks cb 0x7fe344014070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe34bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 0x562dbfd84a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe34bfff640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 0x562dbfd84a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:54274/0 (socket says 172.21.15.84:54274) 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe34bfff640 1 -- 172.21.15.84:0/3930839191 learned_addr learned my addr 172.21.15.84:0/3930839191 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe34bfff640 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7ae00 0x562dbfd83ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe35259c640 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7b720 0x562dbfd84530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe34bfff640 1 -- 172.21.15.84:0/3930839191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7ae00 msgr2=0x562dbfd83ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe34bfff640 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7ae00 0x562dbfd83ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe34bfff640 1 -- 172.21.15.84:0/3930839191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7b720 msgr2=0x562dbfd84530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe34bfff640 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7b720 0x562dbfd84530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe34bfff640 1 -- 172.21.15.84:0/3930839191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dbfd77240 con 0x562dbfd8a4f0 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe35259c640 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7b720 0x562dbfd84530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:45:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe34bfff640 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 0x562dbfd84a70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe33c001eb0 tx=0x7fe33c005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68dc28c0787ffcf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe339ffb640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe339ffb640 1 -- 172.21.15.84:0/3930839191 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe33c035070 con 0x562dbfd8a4f0 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe339ffb640 1 -- 172.21.15.84:0/3930839191 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(117..117 src has 1..117) ==== 8623+0+0 (secure 0 0 0) 0x7fe33c0046c0 con 0x562dbfd8a4f0 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.140+0000 7fe35bc09740 1 -- 172.21.15.84:0/3930839191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe33c04a410 con 0x562dbfd8a4f0 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe339ffb640 1 -- 172.21.15.84:0/3930839191 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe33c0048c0 con 0x562dbfd8a4f0 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe35bc09740 10 client.17457 fetch_fsmap learned FSMap version 29 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe35bc09740 1 -- 172.21.15.84:0/3930839191 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562dbfe00720 con 0x562dbfd8a4f0 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe339ffb640 1 -- 172.21.15.84:0/3930839191 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe33c04a410 con 0x562dbfd8a4f0 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe339ffb640 1 -- 172.21.15.84:0/3930839191 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7fe33c03a040 con 0x562dbfd8a4f0 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe35bc09740 10 client.17457 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe35bc09740 10 client.17457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe35bc09740 20 client.17457 populate_metadata read hostname 'smithi084' 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe35bc09740 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562dbfe00e40 0x562dbfe03260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe35bc09740 4 client.17457 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:45:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe35bc09740 1 -- 172.21.15.84:0/3930839191 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562dbfce8ee0 con 0x562dbfe00e40 2024-04-26T18:45:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe35259c640 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562dbfe00e40 0x562dbfe03260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe35259c640 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562dbfe00e40 0x562dbfe03260 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d340b36a2c8683d2 server_cookie=36f8d84eba9faf86 in_seq=0 out_seq=0 2024-04-26T18:45:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.141+0000 7fe339ffb640 10 client.17457 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:45:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.155+0000 7fe339ffb640 1 -- 172.21.15.84:0/3930839191 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562dbfce8ee0 con 0x562dbfe00e40 2024-04-26T18:45:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.155+0000 7fe339ffb640 10 client.17457 handle_command_reply: tid=0 2024-04-26T18:45:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.155+0000 7fe35bc09740 1 client.17457 shutdown 2024-04-26T18:45:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe35bc09740 1 -- 172.21.15.84:0/3930839191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 msgr2=0x562dbfd84a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe35bc09740 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 0x562dbfd84a70 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fe33c001eb0 tx=0x7fe33c005c00 comp rx=0 tx=0).stop 2024-04-26T18:45:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe35bc09740 1 -- 172.21.15.84:0/3930839191 shutdown_connections 2024-04-26T18:45:05.156 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe35bc09740 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x562dbfe00e40 0x562dbfe03260 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe35bc09740 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x562dbfd8a4f0 0x562dbfd84a70 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe35bc09740 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x562dbfd7b720 0x562dbfd84530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe35bc09740 1 --2- 172.21.15.84:0/3930839191 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x562dbfd7ae00 0x562dbfd83ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe339ffb640 0 client.17457 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:45:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe35bc09740 1 -- 172.21.15.84:0/3930839191 >> 172.21.15.84:0/3930839191 conn(0x562dbfd77410 msgr2=0x562dbfd87370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe35bc09740 1 -- 172.21.15.84:0/3930839191 shutdown_connections 2024-04-26T18:45:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe35bc09740 1 -- 172.21.15.84:0/3930839191 wait complete. 2024-04-26T18:45:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe35bc09740 20 client.17457 trim_cache size 0 max 16384 2024-04-26T18:45:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:45:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:45:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:45:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:45:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe358ef1640 1 -- 172.21.15.84:0/3951245260 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe3400a3dc0 msgr2=0x7fe3400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe358ef1640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe3400a3dc0 0x7fe3400a6280 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe34c0cfc90 tx=0x7fe34c00aa60 comp rx=0 tx=0).stop 2024-04-26T18:45:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe358ef1640 1 -- 172.21.15.84:0/3951245260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe354107860 msgr2=0x7fe35419c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe358ef1640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe354107860 0x7fe35419c800 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fe34c007cb0 tx=0x7fe34c00e420 comp rx=0 tx=0).stop 2024-04-26T18:45:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe358ef1640 1 -- 172.21.15.84:0/3951245260 shutdown_connections 2024-04-26T18:45:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe358ef1640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fe3400a3dc0 0x7fe3400a6280 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe358ef1640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fe354107860 0x7fe35419c800 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe358ef1640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fe354102210 0x7fe354198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe358ef1640 1 --2- 172.21.15.84:0/3951245260 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fe354101850 0x7fe354197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.156+0000 7fe358ef1640 1 -- 172.21.15.84:0/3951245260 >> 172.21.15.84:0/3951245260 conn(0x7fe354077bb0 msgr2=0x7fe3540ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.157+0000 7fe358ef1640 1 -- 172.21.15.84:0/3951245260 shutdown_connections 2024-04-26T18:45:05.159 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:05.157+0000 7fe358ef1640 1 -- 172.21.15.84:0/3951245260 wait complete. 2024-04-26T18:45:05.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:04 smithi174 ceph-mon[32705]: pgmap v538: 97 pgs: 97 active+clean; 3.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 96 KiB/s wr, 38 op/s 2024-04-26T18:45:05.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:04 smithi174 ceph-mon[32705]: osdmap e117: 12 total, 12 up, 12 in 2024-04-26T18:45:07.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:06 smithi084 ceph-mon[29141]: pgmap v540: 97 pgs: 97 active+clean; 3.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-04-26T18:45:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:06 smithi103 ceph-mon[32727]: pgmap v540: 97 pgs: 97 active+clean; 3.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-04-26T18:45:07.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:06 smithi174 ceph-mon[32705]: pgmap v540: 97 pgs: 97 active+clean; 3.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-04-26T18:45:09.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:08 smithi084 ceph-mon[29141]: pgmap v541: 97 pgs: 97 active+clean; 3.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-04-26T18:45:09.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:08 smithi103 ceph-mon[32727]: pgmap v541: 97 pgs: 97 active+clean; 3.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-04-26T18:45:09.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:08 smithi174 ceph-mon[32705]: pgmap v541: 97 pgs: 97 active+clean; 3.8 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-04-26T18:45:11.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:10 smithi084 ceph-mon[29141]: pgmap v542: 97 pgs: 97 active+clean; 4.1 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:45:11.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:10 smithi103 ceph-mon[32727]: pgmap v542: 97 pgs: 97 active+clean; 4.1 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:45:11.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:10 smithi174 ceph-mon[32705]: pgmap v542: 97 pgs: 97 active+clean; 4.1 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:45:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:13 smithi084 ceph-mon[29141]: pgmap v543: 97 pgs: 97 active+clean; 4.1 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:45:13.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:13 smithi084 ceph-mon[29141]: osdmap e118: 12 total, 12 up, 12 in 2024-04-26T18:45:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:13 smithi103 ceph-mon[32727]: pgmap v543: 97 pgs: 97 active+clean; 4.1 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:45:13.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:13 smithi103 ceph-mon[32727]: osdmap e118: 12 total, 12 up, 12 in 2024-04-26T18:45:13.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:13 smithi174 ceph-mon[32705]: pgmap v543: 97 pgs: 97 active+clean; 4.1 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-26T18:45:13.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:13 smithi174 ceph-mon[32705]: osdmap e118: 12 total, 12 up, 12 in 2024-04-26T18:45:15.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:15 smithi084 ceph-mon[29141]: pgmap v545: 97 pgs: 97 active+clean; 4.3 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:45:15.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:15 smithi103 ceph-mon[32727]: pgmap v545: 97 pgs: 97 active+clean; 4.3 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:45:15.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:15 smithi174 ceph-mon[32705]: pgmap v545: 97 pgs: 97 active+clean; 4.3 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:45:17.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:17 smithi084 ceph-mon[29141]: pgmap v546: 97 pgs: 97 active+clean; 4.3 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:45:17.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:17 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:45:17.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:17 smithi103 ceph-mon[32727]: pgmap v546: 97 pgs: 97 active+clean; 4.3 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:45:17.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:17 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:45:17.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:17 smithi174 ceph-mon[32705]: pgmap v546: 97 pgs: 97 active+clean; 4.3 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:45:17.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:17 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:45:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.243+0000 7f31452da640 1 -- 172.21.15.84:0/3664626831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f314010cf00 msgr2=0x7f314010f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.243+0000 7f31452da640 1 --2- 172.21.15.84:0/3664626831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f314010cf00 0x7f314010f3a0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f3138009ca0 tx=0x7f313802a950 comp rx=0 tx=0).stop 2024-04-26T18:45:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.243+0000 7f31452da640 1 -- 172.21.15.84:0/3664626831 shutdown_connections 2024-04-26T18:45:18.245 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.243+0000 7f31452da640 1 --2- 172.21.15.84:0/3664626831 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f314010cf00 0x7f314010f3a0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.243+0000 7f31452da640 1 --2- 172.21.15.84:0/3664626831 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3140104400 0x7f314010c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.243+0000 7f31452da640 1 --2- 172.21.15.84:0/3664626831 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3140103ae0 0x7f3140103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.243+0000 7f31452da640 1 -- 172.21.15.84:0/3664626831 >> 172.21.15.84:0/3664626831 conn(0x7f31400fd300 msgr2=0x7f31400ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.243+0000 7f31452da640 1 -- 172.21.15.84:0/3664626831 shutdown_connections 2024-04-26T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.243+0000 7f31452da640 1 -- 172.21.15.84:0/3664626831 wait complete. 2024-04-26T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.243+0000 7f31452da640 1 Processor -- start 2024-04-26T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f31452da640 1 -- start start 2024-04-26T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f31452da640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3140103ae0 0x7f31401990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f31452da640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3140104400 0x7f3140199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f31452da640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f314010cf00 0x7f314019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f31452da640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3140112c00 con 0x7f314010cf00 2024-04-26T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f31452da640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3140112a80 con 0x7f3140103ae0 2024-04-26T18:45:18.246 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f31452da640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3140112d80 con 0x7f3140104400 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f313f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f314010cf00 0x7f314019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f313f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f314010cf00 0x7f314019d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55148/0 (socket says 172.21.15.84:55148) 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f313f7fe640 1 -- 172.21.15.84:0/2304424606 learned_addr learned my addr 172.21.15.84:0/2304424606 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f313e7fc640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3140104400 0x7f3140199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f313f7fe640 1 -- 172.21.15.84:0/2304424606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3140104400 msgr2=0x7f3140199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f313f7fe640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3140104400 0x7f3140199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f313effd640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3140103ae0 0x7f31401990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f313f7fe640 1 -- 172.21.15.84:0/2304424606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3140103ae0 msgr2=0x7f31401990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f313f7fe640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3140103ae0 0x7f31401990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f313f7fe640 1 -- 172.21.15.84:0/2304424606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314019e0d0 con 0x7f314010cf00 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f313effd640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3140103ae0 0x7f31401990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f313f7fe640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f314010cf00 0x7f314019d9a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3138009dc0 tx=0x7f3138003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f3127fff640 1 -- 172.21.15.84:0/2304424606 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138034040 con 0x7f314010cf00 2024-04-26T18:45:18.247 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.244+0000 7f3127fff640 1 -- 172.21.15.84:0/2304424606 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3138030850 con 0x7f314010cf00 2024-04-26T18:45:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.245+0000 7f31452da640 1 -- 172.21.15.84:0/2304424606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314019e360 con 0x7f314010cf00 2024-04-26T18:45:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.245+0000 7f3127fff640 1 -- 172.21.15.84:0/2304424606 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138004680 con 0x7f314010cf00 2024-04-26T18:45:18.248 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.245+0000 7f31452da640 1 -- 172.21.15.84:0/2304424606 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31401a5c50 con 0x7f314010cf00 2024-04-26T18:45:18.254 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f3127fff640 1 -- 172.21.15.84:0/2304424606 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3138037020 con 0x7f314010cf00 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f3147ff7740 1 Processor -- start 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f3147ff7740 1 -- start start 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f3127fff640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f31280a3e40 0x7f31280a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f3147ff7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da36e20 0x55637da37200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f3147ff7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da37740 0x55637da45fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f3147ff7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da464f0 0x55637da488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f3147ff7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55637d99d460 con 0x55637da464f0 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f3147ff7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55637d957590 con 0x55637da37740 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f3147ff7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55637d999420 con 0x55637da36e20 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f313e7fc640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f31280a3e40 0x7f31280a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f313f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da464f0 0x55637da488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f313f7fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da464f0 0x55637da488e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55152/0 (socket says 172.21.15.84:55152) 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f313f7fe640 1 -- 172.21.15.84:0/2004929718 learned_addr learned my addr 172.21.15.84:0/2004929718 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.247+0000 7f3127fff640 1 -- 172.21.15.84:0/2304424606 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7f31380f1620 con 0x7f314010cf00 2024-04-26T18:45:18.255 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.248+0000 7f313e7fc640 1 --2- 172.21.15.84:0/2004929718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da37740 0x55637da45fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.248+0000 7f313effd640 1 --2- 172.21.15.84:0/2004929718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da36e20 0x55637da37200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.248+0000 7f313f7fe640 1 -- 172.21.15.84:0/2004929718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da36e20 msgr2=0x55637da37200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.248+0000 7f313f7fe640 1 --2- 172.21.15.84:0/2004929718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da36e20 0x55637da37200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.248+0000 7f313f7fe640 1 -- 172.21.15.84:0/2004929718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da37740 msgr2=0x55637da45fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.248+0000 7f313f7fe640 1 --2- 172.21.15.84:0/2004929718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da37740 0x55637da45fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.248+0000 7f313f7fe640 1 -- 172.21.15.84:0/2004929718 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637d9640d0 con 0x55637da464f0 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.248+0000 7f313e7fc640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f31280a3e40 0x7f31280a6300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3130005940 tx=0x7f31300127c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.248+0000 7f313f7fe640 1 --2- 172.21.15.84:0/2004929718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da464f0 0x55637da488e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f31380ef330 tx=0x7f31380b99b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69014338ffc54d6e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.248+0000 7f31257fa640 1 -- 172.21.15.84:0/2004929718 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31380bcff0 con 0x55637da464f0 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.248+0000 7f31257fa640 1 -- 172.21.15.84:0/2004929718 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31380098d0 con 0x55637da464f0 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.248+0000 7f31257fa640 1 -- 172.21.15.84:0/2004929718 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3138089000 con 0x55637da464f0 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 -- 172.21.15.84:0/2004929718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da464f0 msgr2=0x55637da488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 --2- 172.21.15.84:0/2004929718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da464f0 0x55637da488e0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f31380ef330 tx=0x7f31380b99b0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 -- 172.21.15.84:0/2004929718 shutdown_connections 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 --2- 172.21.15.84:0/2004929718 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da464f0 0x55637da488e0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.256 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 --2- 172.21.15.84:0/2004929718 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da37740 0x55637da45fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 --2- 172.21.15.84:0/2004929718 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da36e20 0x55637da37200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 -- 172.21.15.84:0/2004929718 >> 172.21.15.84:0/2004929718 conn(0x55637da33410 msgr2=0x55637da337f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 -- 172.21.15.84:0/2004929718 shutdown_connections 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 -- 172.21.15.84:0/2004929718 wait complete. 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 -- start start 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 Processor -- start 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da36e20 0x55637da43480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 0x55637da439c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da464f0 0x55637da43f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55637da48ee0 con 0x55637da37740 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55637d924b60 con 0x55637da464f0 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55637da49060 con 0x55637da36e20 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f3147ff7740 10 client.0 _ll_register_callbacks cb 0x55637d9492b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:45:18.257 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.249+0000 7f313e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 0x55637da439c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f313e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 0x55637da439c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55158/0 (socket says 172.21.15.84:55158) 2024-04-26T18:45:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f313e7fc640 1 -- 172.21.15.84:0/2081582965 learned_addr learned my addr 172.21.15.84:0/2081582965 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f313effd640 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da36e20 0x55637da43480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f313f7fe640 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da464f0 0x55637da43f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f313e7fc640 1 -- 172.21.15.84:0/2081582965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da36e20 msgr2=0x55637da43480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f313e7fc640 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da36e20 0x55637da43480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f313e7fc640 1 -- 172.21.15.84:0/2081582965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da464f0 msgr2=0x55637da43f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f313e7fc640 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da464f0 0x55637da43f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f313e7fc640 1 -- 172.21.15.84:0/2081582965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637d6af210 con 0x55637da37740 2024-04-26T18:45:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f313effd640 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da36e20 0x55637da43480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T18:45:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f313e7fc640 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 0x55637da439c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3130010b40 tx=0x7f3130004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87e08d07af15f196 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f3122ff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:45:18.258 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f3122ff5640 1 -- 172.21.15.84:0/2081582965 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130023030 con 0x55637da37740 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f3122ff5640 1 -- 172.21.15.84:0/2081582965 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7f313000d9d0 con 0x55637da37740 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f3147ff7740 1 -- 172.21.15.84:0/2081582965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55637d9ab070 con 0x55637da37740 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f3122ff5640 1 -- 172.21.15.84:0/2081582965 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f313002f050 con 0x55637da37740 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f3147ff7740 10 client.17469 fetch_fsmap learned FSMap version 29 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f3147ff7740 1 -- 172.21.15.84:0/2081582965 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55637d77b620 con 0x55637da37740 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.250+0000 7f3122ff5640 1 -- 172.21.15.84:0/2081582965 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55637d9ab070 con 0x55637da37740 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.251+0000 7f3122ff5640 1 -- 172.21.15.84:0/2081582965 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f31300047a0 con 0x55637da37740 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.251+0000 7f3147ff7740 10 client.17469 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.251+0000 7f3147ff7740 10 client.17469 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.251+0000 7f3147ff7740 20 client.17469 populate_metadata read hostname 'smithi084' 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.251+0000 7f3147ff7740 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637da56350 0x55637da587a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.251+0000 7f3147ff7740 4 client.17469 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.251+0000 7f3147ff7740 1 -- 172.21.15.84:0/2081582965 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55637d9a4ee0 con 0x55637da56350 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.251+0000 7f313effd640 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637da56350 0x55637da587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.251+0000 7f3122ff5640 10 client.17469 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:45:18.259 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.251+0000 7f313effd640 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637da56350 0x55637da587a0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad59c3fa0b3e5345 server_cookie=5198959aaaaaa874 in_seq=0 out_seq=0 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3122ff5640 1 -- 172.21.15.84:0/2081582965 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55637d9a4ee0 con 0x55637da56350 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3122ff5640 10 client.17469 handle_command_reply: tid=0 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3147ff7740 1 client.17469 shutdown 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3147ff7740 1 -- 172.21.15.84:0/2081582965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 msgr2=0x55637da439c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3147ff7740 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 0x55637da439c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f3130010b40 tx=0x7f3130004730 comp rx=0 tx=0).stop 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3147ff7740 1 -- 172.21.15.84:0/2081582965 shutdown_connections 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3147ff7740 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637da56350 0x55637da587a0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3147ff7740 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da464f0 0x55637da43f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3122ff5640 0 client.17469 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3147ff7740 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 0x55637da439c0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3147ff7740 1 --2- 172.21.15.84:0/2081582965 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da36e20 0x55637da43480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3147ff7740 1 -- 172.21.15.84:0/2081582965 >> 172.21.15.84:0/2081582965 conn(0x55637da33410 msgr2=0x55637da39280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3147ff7740 1 -- 172.21.15.84:0/2081582965 shutdown_connections 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3147ff7740 1 -- 172.21.15.84:0/2081582965 wait complete. 2024-04-26T18:45:18.260 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.252+0000 7f3147ff7740 20 client.17469 trim_cache size 0 max 16384 2024-04-26T18:45:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.266+0000 7f3147ff7740 1 Processor -- start 2024-04-26T18:45:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.266+0000 7f3147ff7740 1 -- start start 2024-04-26T18:45:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f3147ff7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da36e20 0x55637da51cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f3147ff7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da37740 0x55637da5e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.271 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f3147ff7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da464f0 0x55637da5e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f3147ff7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f31401ac2f0 con 0x55637da36e20 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f3147ff7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31401ade30 con 0x55637da37740 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f3147ff7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31401ac9c0 con 0x55637da464f0 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f313effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da36e20 0x55637da51cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f313effd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da36e20 0x55637da51cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55166/0 (socket says 172.21.15.84:55166) 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f313effd640 1 -- 172.21.15.84:0/1359054419 learned_addr learned my addr 172.21.15.84:0/1359054419 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f313e7fc640 1 --2- 172.21.15.84:0/1359054419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da37740 0x55637da5e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f313effd640 1 -- 172.21.15.84:0/1359054419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da464f0 msgr2=0x55637da5e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f313effd640 1 --2- 172.21.15.84:0/1359054419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da464f0 0x55637da5e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f313effd640 1 -- 172.21.15.84:0/1359054419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da37740 msgr2=0x55637da5e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f313effd640 1 --2- 172.21.15.84:0/1359054419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da37740 0x55637da5e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f313effd640 1 -- 172.21.15.84:0/1359054419 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637da58e20 con 0x55637da36e20 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f313effd640 1 --2- 172.21.15.84:0/1359054419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da36e20 0x55637da51cf0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x55637da59050 tx=0x7f3134004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f055cfa962a5ab server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f3124ff9640 1 -- 172.21.15.84:0/1359054419 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f313401f440 con 0x55637da36e20 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f3124ff9640 1 -- 172.21.15.84:0/1359054419 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31340124c0 con 0x55637da36e20 2024-04-26T18:45:18.272 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.267+0000 7f3124ff9640 1 -- 172.21.15.84:0/1359054419 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134010630 con 0x55637da36e20 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 -- 172.21.15.84:0/1359054419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da36e20 msgr2=0x55637da51cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 --2- 172.21.15.84:0/1359054419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da36e20 0x55637da51cf0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x55637da59050 tx=0x7f3134004610 comp rx=0 tx=0).stop 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 -- 172.21.15.84:0/1359054419 shutdown_connections 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 --2- 172.21.15.84:0/1359054419 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da464f0 0x55637da5e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 --2- 172.21.15.84:0/1359054419 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da37740 0x55637da5e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 --2- 172.21.15.84:0/1359054419 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da36e20 0x55637da51cf0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 -- 172.21.15.84:0/1359054419 >> 172.21.15.84:0/1359054419 conn(0x55637da33410 msgr2=0x55637da45540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 -- 172.21.15.84:0/1359054419 shutdown_connections 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 -- 172.21.15.84:0/1359054419 wait complete. 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 -- start start 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 Processor -- start 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da36e20 0x55637da4a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 0x55637da4a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da464f0 0x55637dabb920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.273 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f31401ad760 con 0x55637da37740 2024-04-26T18:45:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f31401ad090 con 0x55637da36e20 2024-04-26T18:45:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f31401aa2f0 con 0x55637da464f0 2024-04-26T18:45:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f3147ff7740 10 client.0 _ll_register_callbacks cb 0x7f314006c160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:45:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.268+0000 7f313e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 0x55637da4a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f313e7fc640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 0x55637da4a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55178/0 (socket says 172.21.15.84:55178) 2024-04-26T18:45:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f313e7fc640 1 -- 172.21.15.84:0/1514252985 learned_addr learned my addr 172.21.15.84:0/1514252985 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f313effd640 1 --2- 172.21.15.84:0/1514252985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da36e20 0x55637da4a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f313e7fc640 1 -- 172.21.15.84:0/1514252985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da464f0 msgr2=0x55637dabb920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f313e7fc640 1 --2- 172.21.15.84:0/1514252985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da464f0 0x55637dabb920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f313e7fc640 1 -- 172.21.15.84:0/1514252985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da36e20 msgr2=0x55637da4a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f313e7fc640 1 --2- 172.21.15.84:0/1514252985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da36e20 0x55637da4a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f313e7fc640 1 -- 172.21.15.84:0/1514252985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637da38400 con 0x55637da37740 2024-04-26T18:45:18.274 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f313e7fc640 1 --2- 172.21.15.84:0/1514252985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 0x55637da4a560 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f3130010b10 tx=0x7f3130010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec8512edf72b078b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f3122ff5640 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f3122ff5640 1 -- 172.21.15.84:0/1514252985 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3130038060 con 0x55637da37740 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f3122ff5640 1 -- 172.21.15.84:0/1514252985 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7f31300047a0 con 0x55637da37740 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f3147ff7740 1 -- 172.21.15.84:0/1514252985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31340124c0 con 0x55637da37740 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f3122ff5640 1 -- 172.21.15.84:0/1514252985 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3130023040 con 0x55637da37740 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f3147ff7740 10 client.17475 fetch_fsmap learned FSMap version 29 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f3147ff7740 1 -- 172.21.15.84:0/1514252985 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55637da33240 con 0x55637da37740 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f3122ff5640 1 -- 172.21.15.84:0/1514252985 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31340124c0 con 0x55637da37740 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f3122ff5640 1 -- 172.21.15.84:0/1514252985 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f313001fab0 con 0x55637da37740 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f3147ff7740 10 client.17475 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f3147ff7740 10 client.17475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f3147ff7740 20 client.17475 populate_metadata read hostname 'smithi084' 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.269+0000 7f3147ff7740 1 --2- 172.21.15.84:0/1514252985 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637dabd3c0 0x55637dabf7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:18.275 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.270+0000 7f3147ff7740 4 client.17475 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-26T18:45:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.270+0000 7f3147ff7740 1 -- 172.21.15.84:0/1514252985 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55637d9205f0 con 0x55637dabd3c0 2024-04-26T18:45:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.270+0000 7f313effd640 1 --2- 172.21.15.84:0/1514252985 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637dabd3c0 0x55637dabf7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.270+0000 7f3122ff5640 10 client.17475 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:45:18.276 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.270+0000 7f313effd640 1 --2- 172.21.15.84:0/1514252985 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637dabd3c0 0x55637dabf7e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d559593e648aa95 server_cookie=f62fe331ab1e52d in_seq=0 out_seq=0 2024-04-26T18:45:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.282+0000 7f3122ff5640 1 -- 172.21.15.84:0/1514252985 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55637d9205f0 con 0x55637dabd3c0 2024-04-26T18:45:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.282+0000 7f3122ff5640 10 client.17475 handle_command_reply: tid=0 2024-04-26T18:45:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.282+0000 7f3147ff7740 1 client.17475 shutdown 2024-04-26T18:45:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.282+0000 7f3147ff7740 1 -- 172.21.15.84:0/1514252985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 msgr2=0x55637da4a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.282+0000 7f3147ff7740 1 --2- 172.21.15.84:0/1514252985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 0x55637da4a560 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f3130010b10 tx=0x7f3130010740 comp rx=0 tx=0).stop 2024-04-26T18:45:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.282+0000 7f3147ff7740 1 -- 172.21.15.84:0/1514252985 shutdown_connections 2024-04-26T18:45:18.283 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.282+0000 7f3147ff7740 1 --2- 172.21.15.84:0/1514252985 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637dabd3c0 0x55637dabf7e0 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f3147ff7740 1 --2- 172.21.15.84:0/1514252985 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637da464f0 0x55637dabb920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f3147ff7740 1 --2- 172.21.15.84:0/1514252985 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637da37740 0x55637da4a560 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f3122ff5640 0 client.17475 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:45:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f3147ff7740 1 --2- 172.21.15.84:0/1514252985 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637da36e20 0x55637da4a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f3147ff7740 1 -- 172.21.15.84:0/1514252985 >> 172.21.15.84:0/1514252985 conn(0x55637da33410 msgr2=0x55637da44cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f3147ff7740 1 -- 172.21.15.84:0/1514252985 shutdown_connections 2024-04-26T18:45:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f3147ff7740 1 -- 172.21.15.84:0/1514252985 wait complete. 2024-04-26T18:45:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f3147ff7740 20 client.17475 trim_cache size 0 max 16384 2024-04-26T18:45:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:{ 2024-04-26T18:45:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "message": "", 2024-04-26T18:45:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout: "return_code": 0 2024-04-26T18:45:18.284 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stdout:} 2024-04-26T18:45:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f31452da640 1 -- 172.21.15.84:0/2304424606 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f31280a3e40 msgr2=0x7f31280a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f31452da640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f31280a3e40 0x7f31280a6300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f3130005940 tx=0x7f31300127c0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f31452da640 1 -- 172.21.15.84:0/2304424606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f314010cf00 msgr2=0x7f314019d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:18.285 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f31452da640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f314010cf00 0x7f314019d9a0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3138009dc0 tx=0x7f3138003f10 comp rx=0 tx=0).stop 2024-04-26T18:45:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f31452da640 1 -- 172.21.15.84:0/2304424606 shutdown_connections 2024-04-26T18:45:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f31452da640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f31280a3e40 0x7f31280a6300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f31452da640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f314010cf00 0x7f314019d9a0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f31452da640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3140104400 0x7f3140199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f31452da640 1 --2- 172.21.15.84:0/2304424606 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3140103ae0 0x7f31401990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f31452da640 1 -- 172.21.15.84:0/2304424606 >> 172.21.15.84:0/2304424606 conn(0x7f31400fd300 msgr2=0x7f31400fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f31452da640 1 -- 172.21.15.84:0/2304424606 shutdown_connections 2024-04-26T18:45:18.286 INFO:teuthology.task.background_exec.ubuntu@smithi084.front.sepia.ceph.com.smithi084.stderr:2024-04-26T18:45:18.283+0000 7f31452da640 1 -- 172.21.15.84:0/2304424606 wait complete. 2024-04-26T18:45:18.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:18 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:45:18.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:18 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:45:18.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:18 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:45:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:18 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:45:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:18 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:45:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:18 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:45:18.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:18 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:45:18.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:18 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:45:18.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:18 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:45:19.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:19 smithi084 ceph-mon[29141]: pgmap v547: 97 pgs: 97 active+clean; 4.3 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:45:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:19 smithi103 ceph-mon[32727]: pgmap v547: 97 pgs: 97 active+clean; 4.3 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:45:19.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:19 smithi174 ceph-mon[32705]: pgmap v547: 97 pgs: 97 active+clean; 4.3 MiB data, 451 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-26T18:45:21.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:21 smithi084 ceph-mon[29141]: pgmap v548: 97 pgs: 97 active+clean; 4.5 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-26T18:45:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:21 smithi103 ceph-mon[32727]: pgmap v548: 97 pgs: 97 active+clean; 4.5 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-26T18:45:21.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:21 smithi174 ceph-mon[32705]: pgmap v548: 97 pgs: 97 active+clean; 4.5 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-26T18:45:23.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:23 smithi084 ceph-mon[29141]: pgmap v549: 97 pgs: 97 active+clean; 4.5 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-26T18:45:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:23 smithi103 ceph-mon[32727]: pgmap v549: 97 pgs: 97 active+clean; 4.5 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-26T18:45:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:23 smithi174 ceph-mon[32705]: pgmap v549: 97 pgs: 97 active+clean; 4.5 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 49 KiB/s wr, 18 op/s 2024-04-26T18:45:24.334 INFO:tasks.workunit.client.0.smithi084.stdout:561/600: open fd = 3 2024-04-26T18:45:24.335 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.335 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.335 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.335 INFO:tasks.workunit.client.0.smithi084.stdout:562/600: open fd = 3 2024-04-26T18:45:24.335 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.335 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.335 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.335 INFO:tasks.workunit.client.0.smithi084.stdout:563/600: open fd = 3 2024-04-26T18:45:24.335 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.335 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.335 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.335 INFO:tasks.workunit.client.0.smithi084.stdout:564/600: open fd = 3 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:565/600: open fd = 3 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:566/600: open fd = 3 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:567/600: open fd = 3 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.336 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.337 INFO:tasks.workunit.client.0.smithi084.stdout:568/600: open fd = 3 2024-04-26T18:45:24.337 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.337 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.337 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.337 INFO:tasks.workunit.client.0.smithi084.stdout:569/600: open fd = 3 2024-04-26T18:45:24.337 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.337 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.337 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.337 INFO:tasks.workunit.client.0.smithi084.stdout:570/600: open fd = 3 2024-04-26T18:45:24.337 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.337 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.337 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.337 INFO:tasks.workunit.client.0.smithi084.stdout:571/600: open fd = 3 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:572/600: open fd = 3 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:573/600: open fd = 3 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:574/600: open fd = 3 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.338 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.339 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.339 INFO:tasks.workunit.client.0.smithi084.stdout:575/600: open fd = 3 2024-04-26T18:45:24.339 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.339 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.339 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.339 INFO:tasks.workunit.client.0.smithi084.stdout:576/600: open fd = 3 2024-04-26T18:45:24.339 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.339 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.339 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.339 INFO:tasks.workunit.client.0.smithi084.stdout:577/600: open fd = 3 2024-04-26T18:45:24.339 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.339 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.339 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.340 INFO:tasks.workunit.client.0.smithi084.stdout:578/600: open fd = 3 2024-04-26T18:45:24.340 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.340 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.340 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.340 INFO:tasks.workunit.client.0.smithi084.stdout:579/600: open fd = 3 2024-04-26T18:45:24.340 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.340 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.340 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.340 INFO:tasks.workunit.client.0.smithi084.stdout:580/600: open fd = 3 2024-04-26T18:45:24.340 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.340 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.340 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.341 INFO:tasks.workunit.client.0.smithi084.stdout:581/600: open fd = 3 2024-04-26T18:45:24.341 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.341 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.341 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.341 INFO:tasks.workunit.client.0.smithi084.stdout:582/600: open fd = 3 2024-04-26T18:45:24.341 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.341 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.341 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.341 INFO:tasks.workunit.client.0.smithi084.stdout:583/600: open fd = 3 2024-04-26T18:45:24.341 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.341 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.341 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.341 INFO:tasks.workunit.client.0.smithi084.stdout:584/600: open fd = 3 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:585/600: open fd = 3 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:586/600: open fd = 3 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:587/600: open fd = 3 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.342 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:588/600: open fd = 3 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:589/600: open fd = 3 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:590/600: open fd = 3 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:591/600: open fd = 3 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.343 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.344 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.344 INFO:tasks.workunit.client.0.smithi084.stdout:592/600: open fd = 3 2024-04-26T18:45:24.344 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.344 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.344 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.344 INFO:tasks.workunit.client.0.smithi084.stdout:593/600: open fd = 3 2024-04-26T18:45:24.344 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.344 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.345 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.345 INFO:tasks.workunit.client.0.smithi084.stdout:594/600: open fd = 3 2024-04-26T18:45:24.345 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.345 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.345 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.345 INFO:tasks.workunit.client.0.smithi084.stdout:595/600: open fd = 3 2024-04-26T18:45:24.345 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.345 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:596/600: open fd = 3 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:597/600: open fd = 3 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:598/600: open fd = 3 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.346 INFO:tasks.workunit.client.0.smithi084.stdout:599/600: open fd = 3 2024-04-26T18:45:24.347 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.347 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.347 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.347 INFO:tasks.workunit.client.0.smithi084.stdout:600/600: open fd = 3 2024-04-26T18:45:24.347 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.347 INFO:tasks.workunit.client.0.smithi084.stdout:write ret = 32 2024-04-26T18:45:24.347 INFO:tasks.workunit.client.0.smithi084.stdout:pread ret = 64 2024-04-26T18:45:24.347 INFO:tasks.workunit.client.0.smithi084.stderr:+ echo OK 2024-04-26T18:45:24.348 INFO:tasks.workunit.client.0.smithi084.stdout:OK 2024-04-26T18:45:24.348 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T18:45:24.349 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T18:45:24.396 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-26T18:45:24.396 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T18:45:24.674 DEBUG:teuthology.parallel:result is None 2024-04-26T18:45:24.675 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T18:45:24.712 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T18:45:24.712 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-26T18:45:24.722 INFO:tasks.quiescer:joining Quiescers 2024-04-26T18:45:24.722 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-04-26T18:45:24.723 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:45:24.989 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:45:25.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:25 smithi084 ceph-mon[29141]: pgmap v550: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-04-26T18:45:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:25 smithi103 ceph-mon[32727]: pgmap v550: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-04-26T18:45:25.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:25 smithi174 ceph-mon[32705]: pgmap v550: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 62 KiB/s wr, 23 op/s 2024-04-26T18:45:26.886 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.863+0000 7f3cc56f7700 1 -- 172.21.15.84:0/340356655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cc0102320 msgr2=0x7f3cc01027a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.863+0000 7f3cc56f7700 1 --2- 172.21.15.84:0/340356655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cc0102320 0x7f3cc01027a0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4009990 tx=0x7f3cb4009ca0 comp rx=0 tx=0).stop 2024-04-26T18:45:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.863+0000 7f3cc56f7700 1 -- 172.21.15.84:0/340356655 shutdown_connections 2024-04-26T18:45:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.863+0000 7f3cc56f7700 1 --2- 172.21.15.84:0/340356655 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc0102ce0 0x7f3cc010f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.863+0000 7f3cc56f7700 1 --2- 172.21.15.84:0/340356655 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc0108390 0x7f3cc0108770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.863+0000 7f3cc56f7700 1 --2- 172.21.15.84:0/340356655 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cc0102320 0x7f3cc01027a0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.863+0000 7f3cc56f7700 1 -- 172.21.15.84:0/340356655 >> 172.21.15.84:0/340356655 conn(0x7f3cc00792d0 msgr2=0x7f3cc00796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.863+0000 7f3cc56f7700 1 -- 172.21.15.84:0/340356655 shutdown_connections 2024-04-26T18:45:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.863+0000 7f3cc56f7700 1 -- 172.21.15.84:0/340356655 wait complete. 2024-04-26T18:45:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.864+0000 7f3cc56f7700 1 Processor -- start 2024-04-26T18:45:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.864+0000 7f3cc56f7700 1 -- start start 2024-04-26T18:45:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.864+0000 7f3cc56f7700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cc0102320 0x7f3cc019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:26.887 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.864+0000 7f3cc56f7700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc0102ce0 0x7f3cc019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.864+0000 7f3cc56f7700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc0108390 0x7f3cc01999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.864+0000 7f3cc56f7700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc0112530 con 0x7f3cc0102320 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.864+0000 7f3cc56f7700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc01123b0 con 0x7f3cc0108390 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.864+0000 7f3cc56f7700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3cc01126b0 con 0x7f3cc0102ce0 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.864+0000 7f3cbeffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cc0102320 0x7f3cc019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.864+0000 7f3cbeffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cc0102320 0x7f3cc019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45408/0 (socket says 172.21.15.84:45408) 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.864+0000 7f3cbeffd700 1 -- 172.21.15.84:0/3020318818 learned_addr learned my addr 172.21.15.84:0/3020318818 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.864+0000 7f3cbf7fe700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc0108390 0x7f3cc01999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.864+0000 7f3cbe7fc700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc0102ce0 0x7f3cc019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.865+0000 7f3cbeffd700 1 -- 172.21.15.84:0/3020318818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc0102ce0 msgr2=0x7f3cc019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.865+0000 7f3cbeffd700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc0102ce0 0x7f3cc019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.865+0000 7f3cbeffd700 1 -- 172.21.15.84:0/3020318818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc0108390 msgr2=0x7f3cc01999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.865+0000 7f3cbeffd700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc0108390 0x7f3cc01999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.865+0000 7f3cbeffd700 1 -- 172.21.15.84:0/3020318818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cc019a100 con 0x7f3cc0102320 2024-04-26T18:45:26.888 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.865+0000 7f3cbeffd700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cc0102320 0x7f3cc019f790 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000c920 tx=0x7f3cb000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.865+0000 7f3ca7fff700 1 -- 172.21.15.84:0/3020318818 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0007b00 con 0x7f3cc0102320 2024-04-26T18:45:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.865+0000 7f3cc56f7700 1 -- 172.21.15.84:0/3020318818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cc019a3f0 con 0x7f3cc0102320 2024-04-26T18:45:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.865+0000 7f3ca7fff700 1 -- 172.21.15.84:0/3020318818 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cb0004ce0 con 0x7f3cc0102320 2024-04-26T18:45:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.865+0000 7f3ca7fff700 1 -- 172.21.15.84:0/3020318818 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cb0005500 con 0x7f3cc0102320 2024-04-26T18:45:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.865+0000 7f3cc56f7700 1 -- 172.21.15.84:0/3020318818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cc01a6660 con 0x7f3cc0102320 2024-04-26T18:45:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.866+0000 7f3cc56f7700 1 -- 172.21.15.84:0/3020318818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cc006a110 con 0x7f3cc0102320 2024-04-26T18:45:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.867+0000 7f3ca7fff700 1 -- 172.21.15.84:0/3020318818 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3cb0007ca0 con 0x7f3cc0102320 2024-04-26T18:45:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.867+0000 7f3ca7fff700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3ca80a3db0 0x7f3ca80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.868+0000 7f3cbe7fc700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3ca80a3db0 0x7f3ca80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.868+0000 7f3ca7fff700 1 -- 172.21.15.84:0/3020318818 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7f3cb00d07a0 con 0x7f3cc0102320 2024-04-26T18:45:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.868+0000 7f3cbe7fc700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3ca80a3db0 0x7f3ca80a6270 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4006010 tx=0x7f3cb401a040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:26.889 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:26.871+0000 7f3ca7fff700 1 -- 172.21.15.84:0/3020318818 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3cb0014030 con 0x7f3cc0102320 2024-04-26T18:45:27.120 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.120+0000 7f3cc56f7700 1 -- 172.21.15.84:0/3020318818 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3cc019ae80 con 0x7f3cc0102320 2024-04-26T18:45:27.121 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.121+0000 7f3ca7fff700 1 -- 172.21.15.84:0/3020318818 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f3cb009ce00 con 0x7f3cc0102320 2024-04-26T18:45:27.121 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:45:27.121 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:45:27.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.123+0000 7f3cc56f7700 1 -- 172.21.15.84:0/3020318818 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3ca80a3db0 msgr2=0x7f3ca80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:27.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.123+0000 7f3cc56f7700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3ca80a3db0 0x7f3ca80a6270 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4006010 tx=0x7f3cb401a040 comp rx=0 tx=0).stop 2024-04-26T18:45:27.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.123+0000 7f3cc56f7700 1 -- 172.21.15.84:0/3020318818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cc0102320 msgr2=0x7f3cc019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:27.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.123+0000 7f3cc56f7700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cc0102320 0x7f3cc019f790 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f3cb000c920 tx=0x7f3cb000cc30 comp rx=0 tx=0).stop 2024-04-26T18:45:27.123 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.123+0000 7f3cc56f7700 1 -- 172.21.15.84:0/3020318818 shutdown_connections 2024-04-26T18:45:27.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.123+0000 7f3cc56f7700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f3ca80a3db0 0x7f3ca80a6270 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:27.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.123+0000 7f3cc56f7700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3cc0102320 0x7f3cc019f790 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:27.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.123+0000 7f3cc56f7700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3cc0108390 0x7f3cc01999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:27.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.123+0000 7f3cc56f7700 1 --2- 172.21.15.84:0/3020318818 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3cc0102ce0 0x7f3cc019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:27.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.123+0000 7f3cc56f7700 1 -- 172.21.15.84:0/3020318818 >> 172.21.15.84:0/3020318818 conn(0x7f3cc00792d0 msgr2=0x7f3cc0101650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:27.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.123+0000 7f3cc56f7700 1 -- 172.21.15.84:0/3020318818 shutdown_connections 2024-04-26T18:45:27.124 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:27.123+0000 7f3cc56f7700 1 -- 172.21.15.84:0/3020318818 wait complete. 2024-04-26T18:45:27.124 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:45:27.802 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:27 smithi084 ceph-mon[29141]: pgmap v551: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:45:27.803 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:27 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3020318818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:45:27.831 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24434 quiesce db --cancel --all' 2024-04-26T18:45:27.831 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- timeout 75 ceph tell mds.24434 quiesce db --cancel --all 2024-04-26T18:45:27.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:27 smithi103 ceph-mon[32727]: pgmap v551: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:45:27.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:27 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3020318818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:45:27.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:27 smithi174 ceph-mon[32705]: pgmap v551: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:45:27.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:27 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3020318818' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:45:28.086 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:45:29.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.720+0000 7f366e1dc700 1 -- 172.21.15.84:0/2685196279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3668107640 msgr2=0x7f366810f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:29.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.720+0000 7f366e1dc700 1 --2- 172.21.15.84:0/2685196279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3668107640 0x7f366810f210 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f366000b0d0 tx=0x7f366000b3e0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.720+0000 7f366e1dc700 1 -- 172.21.15.84:0/2685196279 shutdown_connections 2024-04-26T18:45:29.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.720+0000 7f366e1dc700 1 --2- 172.21.15.84:0/2685196279 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f3668107640 0x7f366810f210 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.720+0000 7f366e1dc700 1 --2- 172.21.15.84:0/2685196279 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f36681025b0 0x7f3668102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.720+0000 7f366e1dc700 1 --2- 172.21.15.84:0/2685196279 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3668102f60 0x7f3668106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.720+0000 7f366e1dc700 1 -- 172.21.15.84:0/2685196279 >> 172.21.15.84:0/2685196279 conn(0x7f36680792d0 msgr2=0x7f36680796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:29.722 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.720+0000 7f366e1dc700 1 -- 172.21.15.84:0/2685196279 shutdown_connections 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.720+0000 7f366e1dc700 1 -- 172.21.15.84:0/2685196279 wait complete. 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.720+0000 7f366e1dc700 1 Processor -- start 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f366e1dc700 1 -- start start 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f366e1dc700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36681025b0 0x7f366819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f366e1dc700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102f60 0x7f366819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f366e1dc700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3668107640 0x7f36681998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f366e1dc700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36681124e0 con 0x7f36681025b0 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f366e1dc700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f3668112360 con 0x7f3668102f60 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f366e1dc700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f3668112660 con 0x7f3668107640 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f36677fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36681025b0 0x7f366819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f36677fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36681025b0 0x7f366819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45422/0 (socket says 172.21.15.84:45422) 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f36677fe700 1 -- 172.21.15.84:0/2612670490 learned_addr learned my addr 172.21.15.84:0/2612670490 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f3666ffd700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102f60 0x7f366819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f3667fff700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3668107640 0x7f36681998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.723 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f36677fe700 1 -- 172.21.15.84:0/2612670490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3668107640 msgr2=0x7f36681998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:29.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f36677fe700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3668107640 0x7f36681998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f36677fe700 1 -- 172.21.15.84:0/2612670490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102f60 msgr2=0x7f366819fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:29.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f36677fe700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102f60 0x7f366819fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f36677fe700 1 -- 172.21.15.84:0/2612670490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f366819a040 con 0x7f36681025b0 2024-04-26T18:45:29.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f3667fff700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3668107640 0x7f36681998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:45:29.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f36677fe700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36681025b0 0x7f366819f720 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f365800d950 tx=0x7f365800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:29.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.721+0000 7f3664ff9700 1 -- 172.21.15.84:0/2612670490 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658016070 con 0x7f36681025b0 2024-04-26T18:45:29.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.722+0000 7f366e1dc700 1 -- 172.21.15.84:0/2612670490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f366819a330 con 0x7f36681025b0 2024-04-26T18:45:29.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.722+0000 7f3664ff9700 1 -- 172.21.15.84:0/2612670490 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3658010460 con 0x7f36681025b0 2024-04-26T18:45:29.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.722+0000 7f3664ff9700 1 -- 172.21.15.84:0/2612670490 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365800f620 con 0x7f36681025b0 2024-04-26T18:45:29.724 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.722+0000 7f366e1dc700 1 -- 172.21.15.84:0/2612670490 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36681a65f0 con 0x7f36681025b0 2024-04-26T18:45:29.724 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:29 smithi084 ceph-mon[29141]: pgmap v552: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:45:29.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.724+0000 7f3664ff9700 1 -- 172.21.15.84:0/2612670490 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f3658019070 con 0x7f36681025b0 2024-04-26T18:45:29.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.724+0000 7f3664ff9700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36500a3db0 0x7f36500a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.725+0000 7f3666ffd700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36500a3db0 0x7f36500a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.725+0000 7f3664ff9700 1 -- 172.21.15.84:0/2612670490 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7f36580d0520 con 0x7f36681025b0 2024-04-26T18:45:29.725 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.725+0000 7f3666ffd700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36500a3db0 0x7f36500a6270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f365c005950 tx=0x7f365c0058e0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:29.726 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f3678a6eb80 1 Processor -- start 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f3678a6eb80 1 -- start start 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f3678a6eb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c279650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f3678a6eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c279b90 0x55637c2735d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f3678a6eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c273bf0 0x55637c274050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f3678a6eb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55637c1c18a0 con 0x55637c279270 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f3678a6eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55637c1279a0 con 0x55637c273bf0 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f3678a6eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55637c1bd860 con 0x55637c279b90 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f36677fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c279650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f36677fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c279650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45424/0 (socket says 172.21.15.84:45424) 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f36677fe700 1 -- 172.21.15.84:0/4217465043 learned_addr learned my addr 172.21.15.84:0/4217465043 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f36677fe700 1 -- 172.21.15.84:0/4217465043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c279b90 msgr2=0x55637c2735d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f36677fe700 1 --2- 172.21.15.84:0/4217465043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c279b90 0x55637c2735d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f36677fe700 1 -- 172.21.15.84:0/4217465043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c273bf0 msgr2=0x55637c274050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f36677fe700 1 --2- 172.21.15.84:0/4217465043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c273bf0 0x55637c274050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.727 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.726+0000 7f36677fe700 1 -- 172.21.15.84:0/4217465043 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637c10e940 con 0x55637c279270 2024-04-26T18:45:29.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.727+0000 7f36677fe700 1 --2- 172.21.15.84:0/4217465043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c279650 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f3658098880 tx=0x7f365809cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd4b3ed7791ff2f server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:29.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.727+0000 7f364dffb700 1 -- 172.21.15.84:0/4217465043 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3658020070 con 0x55637c279270 2024-04-26T18:45:29.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.727+0000 7f364dffb700 1 -- 172.21.15.84:0/4217465043 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36580a5090 con 0x55637c279270 2024-04-26T18:45:29.728 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.727+0000 7f364dffb700 1 -- 172.21.15.84:0/4217465043 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36580a4c00 con 0x55637c279270 2024-04-26T18:45:29.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.727+0000 7f3678a6eb80 1 -- 172.21.15.84:0/4217465043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 msgr2=0x55637c279650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:29.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.727+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/4217465043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c279650 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f3658098880 tx=0x7f365809cba0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.730 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.727+0000 7f3678a6eb80 1 -- 172.21.15.84:0/4217465043 shutdown_connections 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.727+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/4217465043 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c279650 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.727+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/4217465043 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c273bf0 0x55637c274050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.727+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/4217465043 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c279b90 0x55637c2735d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.727+0000 7f3678a6eb80 1 -- 172.21.15.84:0/4217465043 >> 172.21.15.84:0/4217465043 conn(0x55637c19d630 msgr2=0x55637c269f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.727+0000 7f3678a6eb80 1 -- 172.21.15.84:0/4217465043 shutdown_connections 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.727+0000 7f3678a6eb80 1 -- 172.21.15.84:0/4217465043 wait complete. 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.728+0000 7f3678a6eb80 1 -- start start 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.728+0000 7f3678a6eb80 1 Processor -- start 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.728+0000 7f3678a6eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c273bf0 0x55637c27f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.728+0000 7f3678a6eb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c27ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.728+0000 7f3678a6eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c279b90 0x55637c27d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.728+0000 7f3678a6eb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x55637c2805c0 con 0x55637c279270 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.728+0000 7f3678a6eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x55637c17baf0 con 0x55637c279b90 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.728+0000 7f3678a6eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x55637c280740 con 0x55637c273bf0 2024-04-26T18:45:29.731 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.728+0000 7f3678a6eb80 10 client.0 _ll_register_callbacks cb 0x55637c1599c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.728+0000 7f3666ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c27ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.728+0000 7f3666ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c27ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45436/0 (socket says 172.21.15.84:45436) 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.728+0000 7f3666ffd700 1 -- 172.21.15.84:0/801156161 learned_addr learned my addr 172.21.15.84:0/801156161 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f36677fe700 1 --2- 172.21.15.84:0/801156161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c273bf0 0x55637c27f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3667fff700 1 --2- 172.21.15.84:0/801156161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c279b90 0x55637c27d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3666ffd700 1 -- 172.21.15.84:0/801156161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c273bf0 msgr2=0x55637c27f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3666ffd700 1 --2- 172.21.15.84:0/801156161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c273bf0 0x55637c27f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3666ffd700 1 -- 172.21.15.84:0/801156161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c279b90 msgr2=0x55637c27d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3666ffd700 1 --2- 172.21.15.84:0/801156161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c279b90 0x55637c27d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3666ffd700 1 -- 172.21.15.84:0/801156161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637c17f4f0 con 0x55637c279270 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3666ffd700 1 --2- 172.21.15.84:0/801156161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c27ccd0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f366806b9f0 tx=0x7f365c00ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30a2b569521230e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3646ffd700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3646ffd700 1 -- 172.21.15.84:0/801156161 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c022070 con 0x55637c279270 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3678a6eb80 1 -- 172.21.15.84:0/801156161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36580a5090 con 0x55637c279270 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3646ffd700 1 -- 172.21.15.84:0/801156161 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7f365c011da0 con 0x55637c279270 2024-04-26T18:45:29.732 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3646ffd700 1 -- 172.21.15.84:0/801156161 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f365c02e050 con 0x55637c279270 2024-04-26T18:45:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3678a6eb80 10 client.17493 fetch_fsmap learned FSMap version 29 2024-04-26T18:45:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.729+0000 7f3678a6eb80 1 -- 172.21.15.84:0/801156161 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55637c28af50 con 0x55637c279270 2024-04-26T18:45:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.730+0000 7f3646ffd700 1 -- 172.21.15.84:0/801156161 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36580a5090 con 0x55637c279270 2024-04-26T18:45:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.730+0000 7f3646ffd700 1 -- 172.21.15.84:0/801156161 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f365c010510 con 0x55637c279270 2024-04-26T18:45:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.730+0000 7f3678a6eb80 10 client.17493 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:45:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.730+0000 7f3678a6eb80 10 client.17493 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:45:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.730+0000 7f3678a6eb80 20 client.17493 populate_metadata read hostname 'smithi084' 2024-04-26T18:45:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.730+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/801156161 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637c28d530 0x55637c28f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.730+0000 7f3678a6eb80 4 client.17493 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-26T18:45:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.730+0000 7f3678a6eb80 1 -- 172.21.15.84:0/801156161 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55637c1cf280 con 0x55637c28d530 2024-04-26T18:45:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.730+0000 7f36677fe700 1 --2- 172.21.15.84:0/801156161 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637c28d530 0x55637c28f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.730+0000 7f3646ffd700 10 client.17493 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:45:29.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.730+0000 7f36677fe700 1 --2- 172.21.15.84:0/801156161 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637c28d530 0x55637c28f920 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e269d98d9f4dbc server_cookie=a024e697acacdfc0 in_seq=0 out_seq=0 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.731+0000 7f3646ffd700 1 -- 172.21.15.84:0/801156161 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55637c1cf280 con 0x55637c28d530 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.731+0000 7f3646ffd700 10 client.17493 handle_command_reply: tid=0 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.731+0000 7f3678a6eb80 1 client.17493 shutdown 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.731+0000 7f3678a6eb80 1 -- 172.21.15.84:0/801156161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 msgr2=0x55637c27ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.731+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/801156161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c27ccd0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f366806b9f0 tx=0x7f365c00ffd0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.731+0000 7f3678a6eb80 1 -- 172.21.15.84:0/801156161 shutdown_connections 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.731+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/801156161 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637c28d530 0x55637c28f920 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.732+0000 7f3646ffd700 0 client.17493 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.732+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/801156161 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c279b90 0x55637c27d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.732+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/801156161 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c273bf0 0x55637c27f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.732+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/801156161 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c27ccd0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.732+0000 7f3678a6eb80 1 -- 172.21.15.84:0/801156161 >> 172.21.15.84:0/801156161 conn(0x55637c19d630 msgr2=0x55637c26e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.732+0000 7f3678a6eb80 1 -- 172.21.15.84:0/801156161 shutdown_connections 2024-04-26T18:45:29.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.732+0000 7f3678a6eb80 1 -- 172.21.15.84:0/801156161 wait complete. 2024-04-26T18:45:29.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.732+0000 7f3678a6eb80 20 client.17493 trim_cache size 0 max 16384 2024-04-26T18:45:29.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.751+0000 7f3678a6eb80 1 Processor -- start 2024-04-26T18:45:29.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.751+0000 7f3678a6eb80 1 -- start start 2024-04-26T18:45:29.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.751+0000 7f3678a6eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c273bf0 0x55637c2894c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.751+0000 7f3678a6eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c279270 0x55637c278520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.752 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.751+0000 7f3678a6eb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279b90 0x55637c2951f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.751+0000 7f3678a6eb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36681abc90 con 0x55637c279b90 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.751+0000 7f3678a6eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36681ad7d0 con 0x55637c279270 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.751+0000 7f3678a6eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36681ac360 con 0x55637c273bf0 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.751+0000 7f3667fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279b90 0x55637c2951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.751+0000 7f3667fff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279b90 0x55637c2951f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45442/0 (socket says 172.21.15.84:45442) 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.751+0000 7f3667fff700 1 -- 172.21.15.84:0/3542768783 learned_addr learned my addr 172.21.15.84:0/3542768783 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.751+0000 7f36677fe700 1 --2- 172.21.15.84:0/3542768783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c273bf0 0x55637c2894c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.752+0000 7f3666ffd700 1 --2- 172.21.15.84:0/3542768783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c279270 0x55637c278520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.752+0000 7f3667fff700 1 -- 172.21.15.84:0/3542768783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c273bf0 msgr2=0x55637c2894c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.752+0000 7f3667fff700 1 --2- 172.21.15.84:0/3542768783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c273bf0 0x55637c2894c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.752+0000 7f3667fff700 1 -- 172.21.15.84:0/3542768783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c279270 msgr2=0x55637c278520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.752+0000 7f3667fff700 1 --2- 172.21.15.84:0/3542768783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c279270 0x55637c278520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.752+0000 7f3667fff700 1 -- 172.21.15.84:0/3542768783 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55637c27f570 con 0x55637c279b90 2024-04-26T18:45:29.753 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.752+0000 7f36677fe700 1 --2- 172.21.15.84:0/3542768783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c273bf0 0x55637c2894c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:45:29.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.752+0000 7f3666ffd700 1 --2- 172.21.15.84:0/3542768783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c279270 0x55637c278520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:45:29.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.752+0000 7f3667fff700 1 --2- 172.21.15.84:0/3542768783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279b90 0x55637c2951f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f3660009dd0 tx=0x7f3660009e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d43c8ff1d72ddc25 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:29.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.752+0000 7f364d7fa700 1 -- 172.21.15.84:0/3542768783 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660009430 con 0x55637c279b90 2024-04-26T18:45:29.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.752+0000 7f364d7fa700 1 -- 172.21.15.84:0/3542768783 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36600074e0 con 0x55637c279b90 2024-04-26T18:45:29.754 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.752+0000 7f364d7fa700 1 -- 172.21.15.84:0/3542768783 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3660005260 con 0x55637c279b90 2024-04-26T18:45:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 -- 172.21.15.84:0/3542768783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279b90 msgr2=0x55637c2951f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/3542768783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279b90 0x55637c2951f0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7f3660009dd0 tx=0x7f3660009e00 comp rx=0 tx=0).stop 2024-04-26T18:45:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 -- 172.21.15.84:0/3542768783 shutdown_connections 2024-04-26T18:45:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/3542768783 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279b90 0x55637c2951f0 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/3542768783 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c273bf0 0x55637c2894c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/3542768783 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c279270 0x55637c278520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 -- 172.21.15.84:0/3542768783 >> 172.21.15.84:0/3542768783 conn(0x55637c19d630 msgr2=0x55637c277e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 -- 172.21.15.84:0/3542768783 shutdown_connections 2024-04-26T18:45:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 -- 172.21.15.84:0/3542768783 wait complete. 2024-04-26T18:45:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 -- start start 2024-04-26T18:45:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 Processor -- start 2024-04-26T18:45:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c273bf0 0x55637c282c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.758 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c2831c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.753+0000 7f3678a6eb80 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c279b90 0x55637c2edab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3678a6eb80 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f36681ad100 con 0x55637c279270 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3678a6eb80 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f36681aca30 con 0x55637c273bf0 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3678a6eb80 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f36681a9c50 con 0x55637c279b90 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3678a6eb80 10 client.0 _ll_register_callbacks cb 0x55637c1599c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3666ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c2831c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f36677fe700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c273bf0 0x55637c282c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3666ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c2831c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45446/0 (socket says 172.21.15.84:45446) 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3666ffd700 1 -- 172.21.15.84:0/2248082352 learned_addr learned my addr 172.21.15.84:0/2248082352 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3667fff700 1 --2- 172.21.15.84:0/2248082352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c279b90 0x55637c2edab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3666ffd700 1 -- 172.21.15.84:0/2248082352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c279b90 msgr2=0x55637c2edab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3666ffd700 1 --2- 172.21.15.84:0/2248082352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c279b90 0x55637c2edab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3666ffd700 1 -- 172.21.15.84:0/2248082352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c273bf0 msgr2=0x55637c282c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3666ffd700 1 --2- 172.21.15.84:0/2248082352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c273bf0 0x55637c282c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3666ffd700 1 -- 172.21.15.84:0/2248082352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55637c2ee070 con 0x55637c279270 2024-04-26T18:45:29.759 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3666ffd700 1 --2- 172.21.15.84:0/2248082352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c2831c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f365c000c00 tx=0x7f365c010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12a6380c72e3726 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3646ffd700 10 client.0 ms_handle_connect on v2:172.21.15.84:3300/0 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3646ffd700 1 -- 172.21.15.84:0/2248082352 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f365c038070 con 0x55637c279270 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3646ffd700 1 -- 172.21.15.84:0/2248082352 <== mon.0 v2:172.21.15.84:3300/0 2 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7f365c0105d0 con 0x55637c279270 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3678a6eb80 1 -- 172.21.15.84:0/2248082352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36600074e0 con 0x55637c279270 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.754+0000 7f3646ffd700 1 -- 172.21.15.84:0/2248082352 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f365c044050 con 0x55637c279270 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.755+0000 7f3678a6eb80 10 client.17499 fetch_fsmap learned FSMap version 29 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.755+0000 7f3678a6eb80 1 -- 172.21.15.84:0/2248082352 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55637c2f53b0 con 0x55637c279270 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.755+0000 7f3646ffd700 1 -- 172.21.15.84:0/2248082352 <== mon.0 v2:172.21.15.84:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36600074e0 con 0x55637c279270 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.755+0000 7f3646ffd700 1 -- 172.21.15.84:0/2248082352 <== mon.0 v2:172.21.15.84:3300/0 5 ==== fsmap(e 29) ==== 8354+0+0 (secure 0 0 0) 0x7f365c03d440 con 0x55637c279270 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.755+0000 7f3678a6eb80 10 client.17499 fetch_fsmap finished waiting for FSMap version 29 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.755+0000 7f3678a6eb80 10 client.17499 resolve_mds: validated gid 24434 aka daemon mds.b 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.755+0000 7f3678a6eb80 20 client.17499 populate_metadata read hostname 'smithi084' 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.755+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/2248082352 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637c28d530 0x55637c2f7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:29.760 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.755+0000 7f3678a6eb80 4 client.17499 mds_command: new command op to 24434 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.755+0000 7f3678a6eb80 1 -- 172.21.15.84:0/2248082352 --> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55637c1c9320 con 0x55637c28d530 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.755+0000 7f36677fe700 1 --2- 172.21.15.84:0/2248082352 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637c28d530 0x55637c2f7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.755+0000 7f3646ffd700 10 client.17499 ms_handle_connect on v2:172.21.15.103:6838/2465442393 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.755+0000 7f36677fe700 1 --2- 172.21.15.84:0/2248082352 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637c28d530 0x55637c2f7ad0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b09dbd99c5dad1 server_cookie=ec61edc070fbffc0 in_seq=0 out_seq=0 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3646ffd700 1 -- 172.21.15.84:0/2248082352 <== mds.0 v2:172.21.15.103:6838/2465442393 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55637c1c9320 con 0x55637c28d530 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3646ffd700 10 client.17499 handle_command_reply: tid=0 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3678a6eb80 1 client.17499 shutdown 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3678a6eb80 1 -- 172.21.15.84:0/2248082352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 msgr2=0x55637c2831c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/2248082352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c2831c0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f365c000c00 tx=0x7f365c010d80 comp rx=0 tx=0).stop 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3678a6eb80 1 -- 172.21.15.84:0/2248082352 shutdown_connections 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/2248082352 >> [v2:172.21.15.103:6838/2465442393,v1:172.21.15.103:6839/2465442393] conn(0x55637c28d530 0x55637c2f7ad0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/2248082352 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x55637c279b90 0x55637c2edab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/2248082352 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x55637c273bf0 0x55637c282c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3678a6eb80 1 --2- 172.21.15.84:0/2248082352 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x55637c279270 0x55637c2831c0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.761 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3678a6eb80 1 -- 172.21.15.84:0/2248082352 >> 172.21.15.84:0/2248082352 conn(0x55637c19d630 msgr2=0x55637c263e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:29.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3646ffd700 0 client.17499 ms_handle_reset on v2:172.21.15.103:6838/2465442393 2024-04-26T18:45:29.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3678a6eb80 1 -- 172.21.15.84:0/2248082352 shutdown_connections 2024-04-26T18:45:29.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3678a6eb80 1 -- 172.21.15.84:0/2248082352 wait complete. 2024-04-26T18:45:29.762 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.756+0000 7f3678a6eb80 20 client.17499 trim_cache size 0 max 16384 2024-04-26T18:45:29.762 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:45:29.762 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 29, 2024-04-26T18:45:29.762 INFO:teuthology.orchestra.run.smithi084.stdout: "leader": 24434, 2024-04-26T18:45:29.762 INFO:teuthology.orchestra.run.smithi084.stdout: "set_version": 48, 2024-04-26T18:45:29.762 INFO:teuthology.orchestra.run.smithi084.stdout: "sets": {} 2024-04-26T18:45:29.762 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:45:29.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.757+0000 7f366e1dc700 1 -- 172.21.15.84:0/2612670490 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36500a3db0 msgr2=0x7f36500a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:29.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.757+0000 7f366e1dc700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36500a3db0 0x7f36500a6270 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f365c005950 tx=0x7f365c0058e0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.757+0000 7f366e1dc700 1 -- 172.21.15.84:0/2612670490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36681025b0 msgr2=0x7f366819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:29.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.757+0000 7f366e1dc700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36681025b0 0x7f366819f720 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f365800d950 tx=0x7f365800dd10 comp rx=0 tx=0).stop 2024-04-26T18:45:29.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.757+0000 7f366e1dc700 1 -- 172.21.15.84:0/2612670490 shutdown_connections 2024-04-26T18:45:29.763 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.757+0000 7f366e1dc700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f36500a3db0 0x7f36500a6270 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.757+0000 7f366e1dc700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f36681025b0 0x7f366819f720 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.757+0000 7f366e1dc700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f3668107640 0x7f36681998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.757+0000 7f366e1dc700 1 --2- 172.21.15.84:0/2612670490 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f3668102f60 0x7f366819fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:29.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.757+0000 7f366e1dc700 1 -- 172.21.15.84:0/2612670490 >> 172.21.15.84:0/2612670490 conn(0x7f36680792d0 msgr2=0x7f366806be60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:29.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.757+0000 7f366e1dc700 1 -- 172.21.15.84:0/2612670490 shutdown_connections 2024-04-26T18:45:29.764 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:29.757+0000 7f366e1dc700 1 -- 172.21.15.84:0/2612670490 wait complete. 2024-04-26T18:45:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:29 smithi103 ceph-mon[32727]: pgmap v552: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:45:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:29 smithi174 ceph-mon[32705]: pgmap v552: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-26T18:45:30.500 INFO:tasks.quiescer:done joining Quiescers 2024-04-26T18:45:30.501 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-26T18:45:30.515 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi084.front.sepia.ceph.com 2024-04-26T18:45:30.523 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-26T18:45:30.569 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-26T18:45:30.569 INFO:teuthology.task.exec:Executing custom commands... 2024-04-26T18:45:30.569 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi084.front.sepia.ceph.com 2024-04-26T18:45:30.569 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-26T18:45:30.599 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-26T18:45:30.671 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-26T18:45:30.731 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:30 smithi084 ceph-mon[29141]: pgmap v553: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 20 op/s 2024-04-26T18:45:30.813 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.810+0000 7faabb007640 1 -- 172.21.15.84:0/3990779290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab4103820 msgr2=0x7faab406b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:30.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.810+0000 7faabb007640 1 --2- 172.21.15.84:0/3990779290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab4103820 0x7faab406b1a0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009900 tx=0x7faaa802a410 comp rx=0 tx=0).stop 2024-04-26T18:45:30.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.810+0000 7faabb007640 1 -- 172.21.15.84:0/3990779290 shutdown_connections 2024-04-26T18:45:30.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.810+0000 7faabb007640 1 --2- 172.21.15.84:0/3990779290 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab410f180 0x7faab4111590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:30.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.810+0000 7faabb007640 1 --2- 172.21.15.84:0/3990779290 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab406b6e0 0x7faab406bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:30.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.810+0000 7faabb007640 1 --2- 172.21.15.84:0/3990779290 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab4103820 0x7faab406b1a0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:30.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.810+0000 7faabb007640 1 -- 172.21.15.84:0/3990779290 >> 172.21.15.84:0/3990779290 conn(0x7faab4078580 msgr2=0x7faab4078990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:30.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.810+0000 7faabb007640 1 -- 172.21.15.84:0/3990779290 shutdown_connections 2024-04-26T18:45:30.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.810+0000 7faabb007640 1 -- 172.21.15.84:0/3990779290 wait complete. 2024-04-26T18:45:30.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faabb007640 1 Processor -- start 2024-04-26T18:45:30.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faabb007640 1 -- start start 2024-04-26T18:45:30.814 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faabb007640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab406b6e0 0x7faab419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faabb007640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab4103820 0x7faab419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faabb007640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab410f180 0x7faab419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faabb007640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7faab4114e60 con 0x7faab410f180 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faabb007640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7faab4114ce0 con 0x7faab4103820 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faabb007640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7faab4114fe0 con 0x7faab406b6e0 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faab95a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab410f180 0x7faab419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faab95a4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab410f180 0x7faab419fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45468/0 (socket says 172.21.15.84:45468) 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faab95a4640 1 -- 172.21.15.84:0/2592645540 learned_addr learned my addr 172.21.15.84:0/2592645540 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faab3fff640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab4103820 0x7faab419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faab8da3640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab406b6e0 0x7faab419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faab95a4640 1 -- 172.21.15.84:0/2592645540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab406b6e0 msgr2=0x7faab419b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faab95a4640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab406b6e0 0x7faab419b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faab95a4640 1 -- 172.21.15.84:0/2592645540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab4103820 msgr2=0x7faab419b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:30.815 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faab95a4640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab4103820 0x7faab419b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faab95a4640 1 -- 172.21.15.84:0/2592645540 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab41a0430 con 0x7faab410f180 2024-04-26T18:45:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faab3fff640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab4103820 0x7faab419b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:45:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.811+0000 7faab8da3640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab406b6e0 0x7faab419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:45:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.812+0000 7faab95a4640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab410f180 0x7faab419fd00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7faaac00b970 tx=0x7faaac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.812+0000 7faab1ffb640 1 -- 172.21.15.84:0/2592645540 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac019070 con 0x7faab410f180 2024-04-26T18:45:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.812+0000 7faab1ffb640 1 -- 172.21.15.84:0/2592645540 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaac014440 con 0x7faab410f180 2024-04-26T18:45:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.812+0000 7faabb007640 1 -- 172.21.15.84:0/2592645540 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab41a0720 con 0x7faab410f180 2024-04-26T18:45:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.812+0000 7faab1ffb640 1 -- 172.21.15.84:0/2592645540 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac0075d0 con 0x7faab410f180 2024-04-26T18:45:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.812+0000 7faabb007640 1 -- 172.21.15.84:0/2592645540 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab41a7fb0 con 0x7faab410f180 2024-04-26T18:45:30.816 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.813+0000 7faabb007640 1 -- 172.21.15.84:0/2592645540 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faab4101e90 con 0x7faab410f180 2024-04-26T18:45:30.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.814+0000 7faab1ffb640 1 -- 172.21.15.84:0/2592645540 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7faaac01c070 con 0x7faab410f180 2024-04-26T18:45:30.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.814+0000 7faab1ffb640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7faaa40a3dc0 0x7faaa40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:30.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.815+0000 7faab8da3640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7faaa40a3dc0 0x7faaa40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:30.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.815+0000 7faab1ffb640 1 -- 172.21.15.84:0/2592645540 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7faaac0d0260 con 0x7faab410f180 2024-04-26T18:45:30.817 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.815+0000 7faab8da3640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7faaa40a3dc0 0x7faaa40a6280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faaa8000c00 tx=0x7faaa8031400 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:30.818 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:30.818+0000 7faab1ffb640 1 -- 172.21.15.84:0/2592645540 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faaac09c9d0 con 0x7faab410f180 2024-04-26T18:45:30.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:30 smithi103 ceph-mon[32727]: pgmap v553: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 20 op/s 2024-04-26T18:45:30.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:30 smithi174 ceph-mon[32705]: pgmap v553: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 46 KiB/s wr, 20 op/s 2024-04-26T18:45:31.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.019+0000 7faabb007640 1 -- 172.21.15.84:0/2592645540 --> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7faab4110830 con 0x7faaa40a3dc0 2024-04-26T18:45:31.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.027+0000 7faab1ffb640 1 -- 172.21.15.84:0/2592645540 <== mgr.15240 v2:172.21.15.84:6800/3453983695 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+351 (secure 0 0 0) 0x7faab4110830 con 0x7faaa40a3dc0 2024-04-26T18:45:31.027 INFO:teuthology.orchestra.run.smithi084.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-26T00:00:00", "created": "2024-04-26T18:27:58", "first": "2024-04-26T18:28:00", "last": "2024-04-26T18:45:00", "last_pruned": "2024-04-26T18:45:00", "created_count": 18, "pruned_count": 12, "active": true} 2024-04-26T18:45:31.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.029+0000 7faabb007640 1 -- 172.21.15.84:0/2592645540 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7faaa40a3dc0 msgr2=0x7faaa40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.029+0000 7faabb007640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7faaa40a3dc0 0x7faaa40a6280 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faaa8000c00 tx=0x7faaa8031400 comp rx=0 tx=0).stop 2024-04-26T18:45:31.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.029+0000 7faabb007640 1 -- 172.21.15.84:0/2592645540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab410f180 msgr2=0x7faab419fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.029+0000 7faabb007640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab410f180 0x7faab419fd00 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7faaac00b970 tx=0x7faaac00be30 comp rx=0 tx=0).stop 2024-04-26T18:45:31.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.029+0000 7faabb007640 1 -- 172.21.15.84:0/2592645540 shutdown_connections 2024-04-26T18:45:31.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.029+0000 7faabb007640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7faaa40a3dc0 0x7faaa40a6280 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.029+0000 7faabb007640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7faab410f180 0x7faab419fd00 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.029+0000 7faabb007640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7faab4103820 0x7faab419b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.029+0000 7faabb007640 1 --2- 172.21.15.84:0/2592645540 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7faab406b6e0 0x7faab419b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.029+0000 7faabb007640 1 -- 172.21.15.84:0/2592645540 >> 172.21.15.84:0/2592645540 conn(0x7faab4078580 msgr2=0x7faab4110120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:31.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.029+0000 7faabb007640 1 -- 172.21.15.84:0/2592645540 shutdown_connections 2024-04-26T18:45:31.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.029+0000 7faabb007640 1 -- 172.21.15.84:0/2592645540 wait complete. 2024-04-26T18:45:31.040 DEBUG:teuthology.orchestra.run.smithi084:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-26T18:45:31.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.181+0000 7f4ca37f0640 1 -- 172.21.15.84:0/2992314688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c9c106800 msgr2=0x7f4c9c106be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.181+0000 7f4ca37f0640 1 --2- 172.21.15.84:0/2992314688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c9c106800 0x7f4c9c106be0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c009900 tx=0x7f4c8c02a410 comp rx=0 tx=0).stop 2024-04-26T18:45:31.183 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.181+0000 7f4ca37f0640 1 -- 172.21.15.84:0/2992314688 shutdown_connections 2024-04-26T18:45:31.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.181+0000 7f4ca37f0640 1 --2- 172.21.15.84:0/2992314688 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c9c1024d0 0x7f4c9c10e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.181+0000 7f4ca37f0640 1 --2- 172.21.15.84:0/2992314688 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c9c101b10 0x7f4c9c101f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.181+0000 7f4ca37f0640 1 --2- 172.21.15.84:0/2992314688 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c9c106800 0x7f4c9c106be0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.181+0000 7f4ca37f0640 1 -- 172.21.15.84:0/2992314688 >> 172.21.15.84:0/2992314688 conn(0x7f4c9c078610 msgr2=0x7f4c9c078a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:31.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.181+0000 7f4ca37f0640 1 -- 172.21.15.84:0/2992314688 shutdown_connections 2024-04-26T18:45:31.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.181+0000 7f4ca37f0640 1 -- 172.21.15.84:0/2992314688 wait complete. 2024-04-26T18:45:31.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.181+0000 7f4ca37f0640 1 Processor -- start 2024-04-26T18:45:31.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.181+0000 7f4ca37f0640 1 -- start start 2024-04-26T18:45:31.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca37f0640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c9c101b10 0x7f4c9c199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:31.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca37f0640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c9c1024d0 0x7f4c9c1995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:31.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca37f0640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c9c106800 0x7f4c9c19d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:31.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca37f0640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c111ae0 con 0x7f4c9c1024d0 2024-04-26T18:45:31.184 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca37f0640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c111960 con 0x7f4c9c101b10 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca37f0640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c9c111c60 con 0x7f4c9c106800 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca0d8b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c9c1024d0 0x7f4c9c1995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca0d8b640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c9c1024d0 0x7f4c9c1995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45498/0 (socket says 172.21.15.84:45498) 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca0d8b640 1 -- 172.21.15.84:0/4107597282 learned_addr learned my addr 172.21.15.84:0/4107597282 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca1d8d640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c9c106800 0x7f4c9c19d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca158c640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c9c101b10 0x7f4c9c199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca0d8b640 1 -- 172.21.15.84:0/4107597282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c9c106800 msgr2=0x7f4c9c19d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca0d8b640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c9c106800 0x7f4c9c19d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca0d8b640 1 -- 172.21.15.84:0/4107597282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c9c101b10 msgr2=0x7f4c9c199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca0d8b640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c9c101b10 0x7f4c9c199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca0d8b640 1 -- 172.21.15.84:0/4107597282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c9c19e050 con 0x7f4c9c1024d0 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca158c640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c9c101b10 0x7f4c9c199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca0d8b640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c9c1024d0 0x7f4c9c1995a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800b4a0 tx=0x7f4c9800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:31.185 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4c927fc640 1 -- 172.21.15.84:0/4107597282 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98014070 con 0x7f4c9c1024d0 2024-04-26T18:45:31.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4c927fc640 1 -- 172.21.15.84:0/4107597282 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c98010ac0 con 0x7f4c9c1024d0 2024-04-26T18:45:31.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4c927fc640 1 -- 172.21.15.84:0/4107597282 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c98017c30 con 0x7f4c9c1024d0 2024-04-26T18:45:31.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca37f0640 1 -- 172.21.15.84:0/4107597282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c9c19e340 con 0x7f4c9c1024d0 2024-04-26T18:45:31.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.182+0000 7f4ca37f0640 1 -- 172.21.15.84:0/4107597282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c9c1a5c50 con 0x7f4c9c1024d0 2024-04-26T18:45:31.186 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.183+0000 7f4ca37f0640 1 -- 172.21.15.84:0/4107597282 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c9c102f80 con 0x7f4c9c1024d0 2024-04-26T18:45:31.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.184+0000 7f4c927fc640 1 -- 172.21.15.84:0/4107597282 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4c98017070 con 0x7f4c9c1024d0 2024-04-26T18:45:31.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.185+0000 7f4c927fc640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c880a3e40 0x7f4c880a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:31.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.185+0000 7f4ca158c640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c880a3e40 0x7f4c880a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:31.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.185+0000 7f4c927fc640 1 -- 172.21.15.84:0/4107597282 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7f4c980cfe60 con 0x7f4c9c1024d0 2024-04-26T18:45:31.187 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.185+0000 7f4ca158c640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c880a3e40 0x7f4c880a6300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c000c00 tx=0x7f4c8c031400 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:31.189 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.189+0000 7f4c927fc640 1 -- 172.21.15.84:0/4107597282 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c9809c5d0 con 0x7f4c9c1024d0 2024-04-26T18:45:31.388 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.388+0000 7f4ca37f0640 1 -- 172.21.15.84:0/4107597282 --> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f4c9c062d70 con 0x7f4c880a3e40 2024-04-26T18:45:31.395 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.396+0000 7f4c927fc640 1 -- 172.21.15.84:0/4107597282 <== mgr.15240 v2:172.21.15.84:6800/3453983695 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f4c9c062d70 con 0x7f4c880a3e40 2024-04-26T18:45:31.396 INFO:teuthology.orchestra.run.smithi084.stdout:/ 1m 6m3h 2024-04-26T18:45:31.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.398+0000 7f4ca37f0640 1 -- 172.21.15.84:0/4107597282 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c880a3e40 msgr2=0x7f4c880a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.398+0000 7f4ca37f0640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c880a3e40 0x7f4c880a6300 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c000c00 tx=0x7f4c8c031400 comp rx=0 tx=0).stop 2024-04-26T18:45:31.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.398+0000 7f4ca37f0640 1 -- 172.21.15.84:0/4107597282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c9c1024d0 msgr2=0x7f4c9c1995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.398+0000 7f4ca37f0640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c9c1024d0 0x7f4c9c1995a0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f4c9800b4a0 tx=0x7f4c9800b960 comp rx=0 tx=0).stop 2024-04-26T18:45:31.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.398+0000 7f4ca37f0640 1 -- 172.21.15.84:0/4107597282 shutdown_connections 2024-04-26T18:45:31.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.398+0000 7f4ca37f0640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c880a3e40 0x7f4c880a6300 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.398+0000 7f4ca37f0640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c9c106800 0x7f4c9c19d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.398+0000 7f4ca37f0640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c9c1024d0 0x7f4c9c1995a0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.398 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.398+0000 7f4ca37f0640 1 --2- 172.21.15.84:0/4107597282 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c9c101b10 0x7f4c9c199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.398+0000 7f4ca37f0640 1 -- 172.21.15.84:0/4107597282 >> 172.21.15.84:0/4107597282 conn(0x7f4c9c078610 msgr2=0x7f4c9c06b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:31.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.398+0000 7f4ca37f0640 1 -- 172.21.15.84:0/4107597282 shutdown_connections 2024-04-26T18:45:31.399 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.398+0000 7f4ca37f0640 1 -- 172.21.15.84:0/4107597282 wait complete. 2024-04-26T18:45:31.409 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-26T18:45:31.420 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T18:45:31.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.519+0000 7f14c4da4640 1 -- 172.21.15.84:0/3623915569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14c0107810 msgr2=0x7f14c0107bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.519+0000 7f14c4da4640 1 --2- 172.21.15.84:0/3623915569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14c0107810 0x7f14c0107bf0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f14b4009930 tx=0x7f14b402a420 comp rx=0 tx=0).stop 2024-04-26T18:45:31.521 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.519+0000 7f14c4da4640 1 -- 172.21.15.84:0/3623915569 shutdown_connections 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.519+0000 7f14c4da4640 1 --2- 172.21.15.84:0/3623915569 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c01021c0 0x7f14c010e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.519+0000 7f14c4da4640 1 --2- 172.21.15.84:0/3623915569 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14c0101800 0x7f14c0101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.519+0000 7f14c4da4640 1 --2- 172.21.15.84:0/3623915569 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14c0107810 0x7f14c0107bf0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.519+0000 7f14c4da4640 1 -- 172.21.15.84:0/3623915569 >> 172.21.15.84:0/3623915569 conn(0x7f14c0077bb0 msgr2=0x7f14c0077fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.519+0000 7f14c4da4640 1 -- 172.21.15.84:0/3623915569 shutdown_connections 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.519+0000 7f14c4da4640 1 -- 172.21.15.84:0/3623915569 wait complete. 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.519+0000 7f14c4da4640 1 Processor -- start 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14c4da4640 1 -- start start 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14c4da4640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14c0101800 0x7f14c0197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14c4da4640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c01021c0 0x7f14c0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14c4da4640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14c0107810 0x7f14c019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14c4da4640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f14c0111ad0 con 0x7f14c0101800 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14c4da4640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f14c0111950 con 0x7f14c0107810 2024-04-26T18:45:31.522 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14c4da4640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f14c0111c50 con 0x7f14c01021c0 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14be59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14c0101800 0x7f14c0197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14be59c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14c0101800 0x7f14c0197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45528/0 (socket says 172.21.15.84:45528) 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14be59c640 1 -- 172.21.15.84:0/3983638441 learned_addr learned my addr 172.21.15.84:0/3983638441 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14bdd9b640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c01021c0 0x7f14c0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14bed9d640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14c0107810 0x7f14c019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14be59c640 1 -- 172.21.15.84:0/3983638441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c01021c0 msgr2=0x7f14c0198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14be59c640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c01021c0 0x7f14c0198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14be59c640 1 -- 172.21.15.84:0/3983638441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14c0107810 msgr2=0x7f14c019c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14be59c640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14c0107810 0x7f14c019c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14be59c640 1 -- 172.21.15.84:0/3983638441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c019cfc0 con 0x7f14c0101800 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14be59c640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14c0101800 0x7f14c0197fd0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f14b4004770 tx=0x7f14b4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14af7fe640 1 -- 172.21.15.84:0/3983638441 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4035050 con 0x7f14c0101800 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.520+0000 7f14c4da4640 1 -- 172.21.15.84:0/3983638441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c019d250 con 0x7f14c0101800 2024-04-26T18:45:31.523 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.521+0000 7f14af7fe640 1 -- 172.21.15.84:0/3983638441 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b4003aa0 con 0x7f14c0101800 2024-04-26T18:45:31.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.521+0000 7f14af7fe640 1 -- 172.21.15.84:0/3983638441 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b4038a70 con 0x7f14c0101800 2024-04-26T18:45:31.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.521+0000 7f14c4da4640 1 -- 172.21.15.84:0/3983638441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c006af90 con 0x7f14c0101800 2024-04-26T18:45:31.524 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.521+0000 7f14c4da4640 1 -- 172.21.15.84:0/3983638441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14c0102f40 con 0x7f14c0101800 2024-04-26T18:45:31.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.523+0000 7f14af7fe640 1 -- 172.21.15.84:0/3983638441 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f14b40035f0 con 0x7f14c0101800 2024-04-26T18:45:31.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.523+0000 7f14af7fe640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f14a80a3d20 0x7f14a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:31.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.523+0000 7f14bdd9b640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f14a80a3d20 0x7f14a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:31.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.523+0000 7f14af7fe640 1 -- 172.21.15.84:0/3983638441 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7f14b40f2ca0 con 0x7f14c0101800 2024-04-26T18:45:31.525 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.524+0000 7f14bdd9b640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f14a80a3d20 0x7f14a80a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f14c01010c0 tx=0x7f14b000a2d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:31.527 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.527+0000 7f14af7fe640 1 -- 172.21.15.84:0/3983638441 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f14b4038e20 con 0x7f14c0101800 2024-04-26T18:45:31.712 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.713+0000 7f14c4da4640 1 -- 172.21.15.84:0/3983638441 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f14c006b9b0 con 0x7f14c0101800 2024-04-26T18:45:31.714 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.714+0000 7f14af7fe640 1 -- 172.21.15.84:0/3983638441 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v118) ==== 74+0+23197 (secure 0 0 0) 0x7f14b40bf300 con 0x7f14c0101800 2024-04-26T18:45:31.714 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:45:31.715 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":118,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:16:06.267694+0000","modified":"2024-04-26T18:45:12.516121+0000","last_up_change":"2024-04-26T18:23:19.906511+0000","last_in_change":"2024-04-26T18:22:59.541869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:19:31.174024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:24:16.533128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:24:19.742656+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":31,"snap_epoch":117,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1076912-986e-4825-b1d1-5fa9d69bc07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6803","nonce":140050011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6805","nonce":140050011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6809","nonce":140050011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6807","nonce":140050011}]},"public_addr":"172.21.15.84:6803/140050011","cluster_addr":"172.21.15.84:6805/140050011","heartbeat_back_addr":"172.21.15.84:6809/140050011","heartbeat_front_addr":"172.21.15.84:6807/140050011","state":["exists","up"]},{"osd":1,"uuid":"fa93293d-9058-4804-ba4b-7e1e674b0fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6801","nonce":2512857657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6803","nonce":2512857657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6807","nonce":2512857657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6805","nonce":2512857657}]},"public_addr":"172.21.15.103:6801/2512857657","cluster_addr":"172.21.15.103:6803/2512857657","heartbeat_back_addr":"172.21.15.103:6807/2512857657","heartbeat_front_addr":"172.21.15.103:6805/2512857657","state":["exists","up"]},{"osd":2,"uuid":"bf78a44f-80a2-4cd7-974f-5abdf771e650","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6801","nonce":76340924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6803","nonce":76340924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6807","nonce":76340924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6805","nonce":76340924}]},"public_addr":"172.21.15.174:6801/76340924","cluster_addr":"172.21.15.174:6803/76340924","heartbeat_back_addr":"172.21.15.174:6807/76340924","heartbeat_front_addr":"172.21.15.174:6805/76340924","state":["exists","up"]},{"osd":3,"uuid":"55fd0035-bc76-4235-b8b5-5cf5ef770139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6811","nonce":2352789256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6813","nonce":2352789256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6817","nonce":2352789256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6815","nonce":2352789256}]},"public_addr":"172.21.15.84:6811/2352789256","cluster_addr":"172.21.15.84:6813/2352789256","heartbeat_back_addr":"172.21.15.84:6817/2352789256","heartbeat_front_addr":"172.21.15.84:6815/2352789256","state":["exists","up"]},{"osd":4,"uuid":"c278a0f3-e436-4d1c-a009-f9165ebf3c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6809","nonce":1619258252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6811","nonce":1619258252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6815","nonce":1619258252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6813","nonce":1619258252}]},"public_addr":"172.21.15.103:6809/1619258252","cluster_addr":"172.21.15.103:6811/1619258252","heartbeat_back_addr":"172.21.15.103:6815/1619258252","heartbeat_front_addr":"172.21.15.103:6813/1619258252","state":["exists","up"]},{"osd":5,"uuid":"6505be9b-c1b0-4d8b-944e-9920f90cb88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6809","nonce":3168291768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6811","nonce":3168291768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6815","nonce":3168291768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6813","nonce":3168291768}]},"public_addr":"172.21.15.174:6809/3168291768","cluster_addr":"172.21.15.174:6811/3168291768","heartbeat_back_addr":"172.21.15.174:6815/3168291768","heartbeat_front_addr":"172.21.15.174:6813/3168291768","state":["exists","up"]},{"osd":6,"uuid":"a037b66a-7488-469d-b73a-14cc0d35cb5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6819","nonce":886635297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6821","nonce":886635297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6825","nonce":886635297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6823","nonce":886635297}]},"public_addr":"172.21.15.84:6819/886635297","cluster_addr":"172.21.15.84:6821/886635297","heartbeat_back_addr":"172.21.15.84:6825/886635297","heartbeat_front_addr":"172.21.15.84:6823/886635297","state":["exists","up"]},{"osd":7,"uuid":"c425a13f-8019-4484-86a0-a34fa4d8de81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6817","nonce":1738278079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6819","nonce":1738278079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6823","nonce":1738278079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6821","nonce":1738278079}]},"public_addr":"172.21.15.103:6817/1738278079","cluster_addr":"172.21.15.103:6819/1738278079","heartbeat_back_addr":"172.21.15.103:6823/1738278079","heartbeat_front_addr":"172.21.15.103:6821/1738278079","state":["exists","up"]},{"osd":8,"uuid":"687e6c08-bfa4-440c-b3bc-90a5748989d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6817","nonce":38410548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6819","nonce":38410548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6823","nonce":38410548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6821","nonce":38410548}]},"public_addr":"172.21.15.174:6817/38410548","cluster_addr":"172.21.15.174:6819/38410548","heartbeat_back_addr":"172.21.15.174:6823/38410548","heartbeat_front_addr":"172.21.15.174:6821/38410548","state":["exists","up"]},{"osd":9,"uuid":"059eb22d-6dfb-4dd0-b192-84ed42ee1fbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6827","nonce":2926393927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6829","nonce":2926393927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6833","nonce":2926393927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6831","nonce":2926393927}]},"public_addr":"172.21.15.84:6827/2926393927","cluster_addr":"172.21.15.84:6829/2926393927","heartbeat_back_addr":"172.21.15.84:6833/2926393927","heartbeat_front_addr":"172.21.15.84:6831/2926393927","state":["exists","up"]},{"osd":10,"uuid":"da31e305-5220-4a87-a131-005b7c7f76b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6825","nonce":2705726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6827","nonce":2705726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6831","nonce":2705726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6829","nonce":2705726915}]},"public_addr":"172.21.15.103:6825/2705726915","cluster_addr":"172.21.15.103:6827/2705726915","heartbeat_back_addr":"172.21.15.103:6831/2705726915","heartbeat_front_addr":"172.21.15.103:6829/2705726915","state":["exists","up"]},{"osd":11,"uuid":"3fde6c8f-ee02-41ee-a34f-73072b4761e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6825","nonce":1668672128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6827","nonce":1668672128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6831","nonce":1668672128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6829","nonce":1668672128}]},"public_addr":"172.21.15.174:6825/1668672128","cluster_addr":"172.21.15.174:6827/1668672128","heartbeat_back_addr":"172.21.15.174:6831/1668672128","heartbeat_front_addr":"172.21.15.174:6829/1668672128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:39.099188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:03.327608+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:27.714108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:52.506015+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:17.631670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:42.702489+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:07.606697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:34.123214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:59.534658+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:25.357720+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:51.543106+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:23:18.208230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/277477867":"2024-04-27T18:27:57.444610+0000","172.21.15.84:0/559313476":"2024-04-27T18:27:57.444610+0000","172.21.15.84:6800/2059208901":"2024-04-27T18:27:57.444610+0000","172.21.15.84:0/3834640430":"2024-04-27T18:27:57.444610+0000","172.21.15.84:0/2142936585":"2024-04-27T18:27:57.444610+0000","172.21.15.84:0/1604680842":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6800/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/2059208901":"2024-04-27T18:27:57.444610+0000","172.21.15.84:6801/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/538363975":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/952052990":"2024-04-27T18:17:05.083192+0000","172.21.15.84:0/1266942705":"2024-04-27T18:16:37.859669+0000","172.21.15.84:6800/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3970542149":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3607125049":"2024-04-27T18:17:05.083192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":18,"length":1},{"begin":31,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:45:31.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.716+0000 7f14c4da4640 1 -- 172.21.15.84:0/3983638441 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f14a80a3d20 msgr2=0x7f14a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.716 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.716+0000 7f14c4da4640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f14a80a3d20 0x7f14a80a61e0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f14c01010c0 tx=0x7f14b000a2d0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.716+0000 7f14c4da4640 1 -- 172.21.15.84:0/3983638441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14c0101800 msgr2=0x7f14c0197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.716+0000 7f14c4da4640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14c0101800 0x7f14c0197fd0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f14b4004770 tx=0x7f14b4004270 comp rx=0 tx=0).stop 2024-04-26T18:45:31.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.716+0000 7f14c4da4640 1 -- 172.21.15.84:0/3983638441 shutdown_connections 2024-04-26T18:45:31.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.716+0000 7f14c4da4640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f14a80a3d20 0x7f14a80a61e0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.716+0000 7f14c4da4640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f14c0107810 0x7f14c019c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.716+0000 7f14c4da4640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f14c01021c0 0x7f14c0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.716+0000 7f14c4da4640 1 --2- 172.21.15.84:0/3983638441 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f14c0101800 0x7f14c0197fd0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.716+0000 7f14c4da4640 1 -- 172.21.15.84:0/3983638441 >> 172.21.15.84:0/3983638441 conn(0x7f14c0077bb0 msgr2=0x7f14c00ff250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:31.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.716+0000 7f14c4da4640 1 -- 172.21.15.84:0/3983638441 shutdown_connections 2024-04-26T18:45:31.717 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.716+0000 7f14c4da4640 1 -- 172.21.15.84:0/3983638441 wait complete. 2024-04-26T18:45:31.726 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:31 smithi084 ceph-mon[29141]: from='client.17505 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:45:31.726 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:31 smithi084 ceph-mon[29141]: from='client.17511 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:45:31.728 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T18:19:31.174024+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-26T18:24:16.533128+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-26T18:24:19.742656+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 31, 'snap_epoch': 117, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T18:45:31.728 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T18:45:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.829+0000 7f7759a12640 1 -- 172.21.15.84:0/3329193833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77541021c0 msgr2=0x7f775410e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.829+0000 7f7759a12640 1 --2- 172.21.15.84:0/3329193833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77541021c0 0x7f775410e6d0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f774c009cd0 tx=0x7f774c02a8e0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.829+0000 7f7759a12640 1 -- 172.21.15.84:0/3329193833 shutdown_connections 2024-04-26T18:45:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.829+0000 7f7759a12640 1 --2- 172.21.15.84:0/3329193833 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f77541021c0 0x7f775410e6d0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.832 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.829+0000 7f7759a12640 1 --2- 172.21.15.84:0/3329193833 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7754101800 0x7f7754101c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.829+0000 7f7759a12640 1 --2- 172.21.15.84:0/3329193833 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7754107810 0x7f7754107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.829+0000 7f7759a12640 1 -- 172.21.15.84:0/3329193833 >> 172.21.15.84:0/3329193833 conn(0x7f7754078930 msgr2=0x7f7754078d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.830+0000 7f7759a12640 1 -- 172.21.15.84:0/3329193833 shutdown_connections 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.830+0000 7f7759a12640 1 -- 172.21.15.84:0/3329193833 wait complete. 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.830+0000 7f7759a12640 1 Processor -- start 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.830+0000 7f7759a12640 1 -- start start 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.830+0000 7f7759a12640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7754101800 0x7f775419b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.830+0000 7f7759a12640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77541021c0 0x7f775419b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.830+0000 7f7759a12640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7754107810 0x7f775419fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.830+0000 7f7759a12640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7754111b50 con 0x7f7754107810 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.830+0000 7f7759a12640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f77541119d0 con 0x7f77541021c0 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.830+0000 7f7759a12640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7754111cd0 con 0x7f7754101800 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.830+0000 7f77537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7754107810 0x7f775419fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:31.833 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.830+0000 7f77537fe640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7754107810 0x7f775419fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45558/0 (socket says 172.21.15.84:45558) 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.830+0000 7f77537fe640 1 -- 172.21.15.84:0/3841504791 learned_addr learned my addr 172.21.15.84:0/3841504791 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f774bfff640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77541021c0 0x7f775419b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f77537fe640 1 -- 172.21.15.84:0/3841504791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7754101800 msgr2=0x7f775419b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f7752ffd640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7754101800 0x7f775419b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f77537fe640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7754101800 0x7f775419b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f77537fe640 1 -- 172.21.15.84:0/3841504791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77541021c0 msgr2=0x7f775419b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f77537fe640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77541021c0 0x7f775419b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f77537fe640 1 -- 172.21.15.84:0/3841504791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77541a0440 con 0x7f7754107810 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f774bfff640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77541021c0 0x7f775419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f77537fe640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7754107810 0x7f775419fd10 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f774c005cc0 tx=0x7f774c003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f7758a10640 1 -- 172.21.15.84:0/3841504791 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c004240 con 0x7f7754107810 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f7759a12640 1 -- 172.21.15.84:0/3841504791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77541a06d0 con 0x7f7754107810 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f7758a10640 1 -- 172.21.15.84:0/3841504791 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f774c0305a0 con 0x7f7754107810 2024-04-26T18:45:31.834 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f7758a10640 1 -- 172.21.15.84:0/3841504791 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f774c0466c0 con 0x7f7754107810 2024-04-26T18:45:31.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.831+0000 7f7759a12640 1 -- 172.21.15.84:0/3841504791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77541a7fc0 con 0x7f7754107810 2024-04-26T18:45:31.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.832+0000 7f7759a12640 1 -- 172.21.15.84:0/3841504791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7754069750 con 0x7f7754107810 2024-04-26T18:45:31.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.834+0000 7f7758a10640 1 -- 172.21.15.84:0/3841504791 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f774c030a50 con 0x7f7754107810 2024-04-26T18:45:31.835 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.834+0000 7f7758a10640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f77400a3d20 0x7f77400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:31.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.834+0000 7f7752ffd640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f77400a3d20 0x7f77400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:31.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.834+0000 7f7758a10640 1 -- 172.21.15.84:0/3841504791 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7f774c02e030 con 0x7f7754107810 2024-04-26T18:45:31.836 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.835+0000 7f7752ffd640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f77400a3d20 0x7f77400a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f774400a810 tx=0x7f7744008040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:31.838 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:31.838+0000 7f7758a10640 1 -- 172.21.15.84:0/3841504791 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f774c04b060 con 0x7f7754107810 2024-04-26T18:45:31.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:31 smithi103 ceph-mon[32727]: from='client.17505 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:45:31.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:31 smithi103 ceph-mon[32727]: from='client.17511 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:45:31.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:31 smithi174 ceph-mon[32705]: from='client.17505 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:45:31.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:31 smithi174 ceph-mon[32705]: from='client.17511 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T18:45:32.015 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.015+0000 7f7759a12640 1 -- 172.21.15.84:0/3841504791 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f77541a81f0 con 0x7f7754107810 2024-04-26T18:45:32.016 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.015+0000 7f7758a10640 1 -- 172.21.15.84:0/3841504791 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v118) ==== 93+0+10 (secure 0 0 0) 0x7f774c0be5f0 con 0x7f7754107810 2024-04-26T18:45:32.016 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 1 2024-04-26T18:45:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.017+0000 7f7759a12640 1 -- 172.21.15.84:0/3841504791 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f77400a3d20 msgr2=0x7f77400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.017+0000 7f7759a12640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f77400a3d20 0x7f77400a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f774400a810 tx=0x7f7744008040 comp rx=0 tx=0).stop 2024-04-26T18:45:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.017+0000 7f7759a12640 1 -- 172.21.15.84:0/3841504791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7754107810 msgr2=0x7f775419fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.017+0000 7f7759a12640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7754107810 0x7f775419fd10 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f774c005cc0 tx=0x7f774c003ba0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.017+0000 7f7759a12640 1 -- 172.21.15.84:0/3841504791 shutdown_connections 2024-04-26T18:45:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.017+0000 7f7759a12640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f77400a3d20 0x7f77400a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.017+0000 7f7759a12640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7754107810 0x7f775419fd10 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.017+0000 7f7759a12640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f77541021c0 0x7f775419b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.017+0000 7f7759a12640 1 --2- 172.21.15.84:0/3841504791 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7754101800 0x7f775419b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.018+0000 7f7759a12640 1 -- 172.21.15.84:0/3841504791 >> 172.21.15.84:0/3841504791 conn(0x7f7754078930 msgr2=0x7f7754069b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:32.018 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.018+0000 7f7759a12640 1 -- 172.21.15.84:0/3841504791 shutdown_connections 2024-04-26T18:45:32.019 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.018+0000 7f7759a12640 1 -- 172.21.15.84:0/3841504791 wait complete. 2024-04-26T18:45:32.029 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-26T18:45:32.132 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.130+0000 7f95c7493640 1 -- 172.21.15.84:0/3806597963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c0102520 msgr2=0x7f95c010e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.130+0000 7f95c7493640 1 --2- 172.21.15.84:0/3806597963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c0102520 0x7f95c010e650 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f95b8009ca0 tx=0x7f95b802a8c0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.130+0000 7f95c7493640 1 -- 172.21.15.84:0/3806597963 shutdown_connections 2024-04-26T18:45:32.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.130+0000 7f95c7493640 1 --2- 172.21.15.84:0/3806597963 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c0102520 0x7f95c010e650 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.130+0000 7f95c7493640 1 --2- 172.21.15.84:0/3806597963 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95c0101b80 0x7f95c0101fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.130+0000 7f95c7493640 1 --2- 172.21.15.84:0/3806597963 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95c0107880 0x7f95c0107c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.130+0000 7f95c7493640 1 -- 172.21.15.84:0/3806597963 >> 172.21.15.84:0/3806597963 conn(0x7f95c0077630 msgr2=0x7f95c0077a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:32.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.130+0000 7f95c7493640 1 -- 172.21.15.84:0/3806597963 shutdown_connections 2024-04-26T18:45:32.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.130+0000 7f95c7493640 1 -- 172.21.15.84:0/3806597963 wait complete. 2024-04-26T18:45:32.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.130+0000 7f95c7493640 1 Processor -- start 2024-04-26T18:45:32.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c7493640 1 -- start start 2024-04-26T18:45:32.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c7493640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95c0101b80 0x7f95c0197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:32.133 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c7493640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95c0102520 0x7f95c01984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:32.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c7493640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c0107880 0x7f95c019c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:32.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c7493640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f95c0111a80 con 0x7f95c0107880 2024-04-26T18:45:32.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c7493640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f95c0111900 con 0x7f95c0101b80 2024-04-26T18:45:32.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c7493640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f95c0111c00 con 0x7f95c0102520 2024-04-26T18:45:32.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c5a30640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c0107880 0x7f95c019c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:32.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c5a30640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c0107880 0x7f95c019c7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45584/0 (socket says 172.21.15.84:45584) 2024-04-26T18:45:32.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c5a30640 1 -- 172.21.15.84:0/547027639 learned_addr learned my addr 172.21.15.84:0/547027639 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:32.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c522f640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95c0101b80 0x7f95c0197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:32.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c4a2e640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95c0102520 0x7f95c01984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:32.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c5a30640 1 -- 172.21.15.84:0/547027639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95c0102520 msgr2=0x7f95c01984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c5a30640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95c0102520 0x7f95c01984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.134 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c5a30640 1 -- 172.21.15.84:0/547027639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95c0101b80 msgr2=0x7f95c0197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c5a30640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95c0101b80 0x7f95c0197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c5a30640 1 -- 172.21.15.84:0/547027639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95c019cf20 con 0x7f95c0107880 2024-04-26T18:45:32.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95c5a30640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c0107880 0x7f95c019c7f0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f95b8005cc0 tx=0x7f95b802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:32.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.131+0000 7f95b67fc640 1 -- 172.21.15.84:0/547027639 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b8003bf0 con 0x7f95c0107880 2024-04-26T18:45:32.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.132+0000 7f95b67fc640 1 -- 172.21.15.84:0/547027639 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95b8003d90 con 0x7f95c0107880 2024-04-26T18:45:32.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.132+0000 7f95c7493640 1 -- 172.21.15.84:0/547027639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95c019d1b0 con 0x7f95c0107880 2024-04-26T18:45:32.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.132+0000 7f95b67fc640 1 -- 172.21.15.84:0/547027639 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95b802fbc0 con 0x7f95c0107880 2024-04-26T18:45:32.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.132+0000 7f95c7493640 1 -- 172.21.15.84:0/547027639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95c01a4aa0 con 0x7f95c0107880 2024-04-26T18:45:32.135 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.133+0000 7f95c7493640 1 -- 172.21.15.84:0/547027639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95c0102f70 con 0x7f95c0107880 2024-04-26T18:45:32.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.133+0000 7f95b67fc640 1 -- 172.21.15.84:0/547027639 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f95b802fd60 con 0x7f95c0107880 2024-04-26T18:45:32.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.134+0000 7f95b67fc640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f95ac0a3d20 0x7f95ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:32.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.134+0000 7f95c522f640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f95ac0a3d20 0x7f95ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:32.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.134+0000 7f95b67fc640 1 -- 172.21.15.84:0/547027639 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7f95b80f1e50 con 0x7f95c0107880 2024-04-26T18:45:32.136 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.135+0000 7f95c522f640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f95ac0a3d20 0x7f95ac0a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0058c0 tx=0x7f95bc00a2d0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:32.138 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.139+0000 7f95b67fc640 1 -- 172.21.15.84:0/547027639 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95b80be5c0 con 0x7f95c0107880 2024-04-26T18:45:32.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.317+0000 7f95c7493640 1 -- 172.21.15.84:0/547027639 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f95c0109d10 con 0x7f95c0107880 2024-04-26T18:45:32.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.317+0000 7f95b67fc640 1 -- 172.21.15.84:0/547027639 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v118) ==== 104+0+11 (secure 0 0 0) 0x7f95b80098b0 con 0x7f95c0107880 2024-04-26T18:45:32.317 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 32 2024-04-26T18:45:32.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.319+0000 7f95c7493640 1 -- 172.21.15.84:0/547027639 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f95ac0a3d20 msgr2=0x7f95ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.319+0000 7f95c7493640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f95ac0a3d20 0x7f95ac0a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0058c0 tx=0x7f95bc00a2d0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.319+0000 7f95c7493640 1 -- 172.21.15.84:0/547027639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c0107880 msgr2=0x7f95c019c7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.319+0000 7f95c7493640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c0107880 0x7f95c019c7f0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f95b8005cc0 tx=0x7f95b802f380 comp rx=0 tx=0).stop 2024-04-26T18:45:32.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.319+0000 7f95c7493640 1 -- 172.21.15.84:0/547027639 shutdown_connections 2024-04-26T18:45:32.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.319+0000 7f95c7493640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f95ac0a3d20 0x7f95ac0a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.319+0000 7f95c7493640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f95c0107880 0x7f95c019c7f0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.319+0000 7f95c7493640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f95c0102520 0x7f95c01984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.319+0000 7f95c7493640 1 --2- 172.21.15.84:0/547027639 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f95c0101b80 0x7f95c0197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.319+0000 7f95c7493640 1 -- 172.21.15.84:0/547027639 >> 172.21.15.84:0/547027639 conn(0x7f95c0077630 msgr2=0x7f95c0100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:32.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.319+0000 7f95c7493640 1 -- 172.21.15.84:0/547027639 shutdown_connections 2024-04-26T18:45:32.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.319+0000 7f95c7493640 1 -- 172.21.15.84:0/547027639 wait complete. 2024-04-26T18:45:32.330 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-26T18:45:32.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.431+0000 7f5788cbd640 1 -- 172.21.15.84:0/1521439715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f578406afb0 msgr2=0x7f578406b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.431 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.431+0000 7f5788cbd640 1 --2- 172.21.15.84:0/1521439715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f578406afb0 0x7f578406b390 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f5774009900 tx=0x7f577402a410 comp rx=0 tx=0).stop 2024-04-26T18:45:32.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.431+0000 7f5788cbd640 1 -- 172.21.15.84:0/1521439715 shutdown_connections 2024-04-26T18:45:32.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.431+0000 7f5788cbd640 1 --2- 172.21.15.84:0/1521439715 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f578410f340 0x7f5784111730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.431+0000 7f5788cbd640 1 --2- 172.21.15.84:0/1521439715 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f578406b8d0 0x7f578410ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.431+0000 7f5788cbd640 1 --2- 172.21.15.84:0/1521439715 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f578406afb0 0x7f578406b390 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.431+0000 7f5788cbd640 1 -- 172.21.15.84:0/1521439715 >> 172.21.15.84:0/1521439715 conn(0x7f5784078890 msgr2=0x7f5784078ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:32.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.431+0000 7f5788cbd640 1 -- 172.21.15.84:0/1521439715 shutdown_connections 2024-04-26T18:45:32.432 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.431+0000 7f5788cbd640 1 -- 172.21.15.84:0/1521439715 wait complete. 2024-04-26T18:45:32.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f5788cbd640 1 Processor -- start 2024-04-26T18:45:32.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f5788cbd640 1 -- start start 2024-04-26T18:45:32.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f5788cbd640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f578406afb0 0x7f578419b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:32.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f5788cbd640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f578406b8d0 0x7f578419b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:32.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f5788cbd640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f578410f340 0x7f578419fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:32.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f5788cbd640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f5784114e00 con 0x7f578406afb0 2024-04-26T18:45:32.434 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f5788cbd640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5784114c80 con 0x7f578410f340 2024-04-26T18:45:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f5788cbd640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5784114f80 con 0x7f578406b8d0 2024-04-26T18:45:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f578259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f578406afb0 0x7f578419b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f578259c640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f578406afb0 0x7f578419b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45612/0 (socket says 172.21.15.84:45612) 2024-04-26T18:45:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f578259c640 1 -- 172.21.15.84:0/1018809234 learned_addr learned my addr 172.21.15.84:0/1018809234 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f5781d9b640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f578406b8d0 0x7f578419b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f5782d9d640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f578410f340 0x7f578419fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f578259c640 1 -- 172.21.15.84:0/1018809234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f578406b8d0 msgr2=0x7f578419b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f578259c640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f578406b8d0 0x7f578419b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f578259c640 1 -- 172.21.15.84:0/1018809234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f578410f340 msgr2=0x7f578419fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f578259c640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f578410f340 0x7f578419fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.432+0000 7f578259c640 1 -- 172.21.15.84:0/1018809234 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57841a0310 con 0x7f578406afb0 2024-04-26T18:45:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.433+0000 7f578259c640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f578406afb0 0x7f578419b320 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f5774005ab0 tx=0x7f5774003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:32.435 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.433+0000 7f57737fe640 1 -- 172.21.15.84:0/1018809234 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f577403f070 con 0x7f578406afb0 2024-04-26T18:45:32.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.433+0000 7f5788cbd640 1 -- 172.21.15.84:0/1018809234 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57841a05a0 con 0x7f578406afb0 2024-04-26T18:45:32.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.433+0000 7f57737fe640 1 -- 172.21.15.84:0/1018809234 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5774031d60 con 0x7f578406afb0 2024-04-26T18:45:32.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.433+0000 7f57737fe640 1 -- 172.21.15.84:0/1018809234 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5774035030 con 0x7f578406afb0 2024-04-26T18:45:32.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.433+0000 7f5788cbd640 1 -- 172.21.15.84:0/1018809234 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57841a7e90 con 0x7f578406afb0 2024-04-26T18:45:32.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.434+0000 7f5788cbd640 1 -- 172.21.15.84:0/1018809234 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5784069ba0 con 0x7f578406afb0 2024-04-26T18:45:32.436 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.435+0000 7f57737fe640 1 -- 172.21.15.84:0/1018809234 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f5774042070 con 0x7f578406afb0 2024-04-26T18:45:32.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.436+0000 7f57737fe640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f576c0a3d20 0x7f576c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:32.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.436+0000 7f5781d9b640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f576c0a3d20 0x7f576c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:32.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.436+0000 7f57737fe640 1 -- 172.21.15.84:0/1018809234 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7f57740f1140 con 0x7f578406afb0 2024-04-26T18:45:32.437 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.436+0000 7f5781d9b640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f576c0a3d20 0x7f576c0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5778005fd0 tx=0x7f5778005e20 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:32.440 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.440+0000 7f57737fe640 1 -- 172.21.15.84:0/1018809234 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f57740bd840 con 0x7f578406afb0 2024-04-26T18:45:32.619 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.619+0000 7f5788cbd640 1 -- 172.21.15.84:0/1018809234 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f578410a4d0 con 0x7f578406afb0 2024-04-26T18:45:32.620 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.620+0000 7f57737fe640 1 -- 172.21.15.84:0/1018809234 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v118) ==== 100+0+11 (secure 0 0 0) 0x7f57740bd140 con 0x7f578406afb0 2024-04-26T18:45:32.620 INFO:teuthology.orchestra.run.smithi084.stdout:pg_num: 64 2024-04-26T18:45:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.621+0000 7f5788cbd640 1 -- 172.21.15.84:0/1018809234 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f576c0a3d20 msgr2=0x7f576c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.621+0000 7f5788cbd640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f576c0a3d20 0x7f576c0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5778005fd0 tx=0x7f5778005e20 comp rx=0 tx=0).stop 2024-04-26T18:45:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.621+0000 7f5788cbd640 1 -- 172.21.15.84:0/1018809234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f578406afb0 msgr2=0x7f578419b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.621+0000 7f5788cbd640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f578406afb0 0x7f578419b320 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f5774005ab0 tx=0x7f5774003cd0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.622+0000 7f5788cbd640 1 -- 172.21.15.84:0/1018809234 shutdown_connections 2024-04-26T18:45:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.622+0000 7f5788cbd640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f576c0a3d20 0x7f576c0a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.622+0000 7f5788cbd640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f578410f340 0x7f578419fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.622+0000 7f5788cbd640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f578406b8d0 0x7f578419b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.622+0000 7f5788cbd640 1 --2- 172.21.15.84:0/1018809234 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f578406afb0 0x7f578419b320 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.622+0000 7f5788cbd640 1 -- 172.21.15.84:0/1018809234 >> 172.21.15.84:0/1018809234 conn(0x7f5784078890 msgr2=0x7f57841107f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:32.622 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.622+0000 7f5788cbd640 1 -- 172.21.15.84:0/1018809234 shutdown_connections 2024-04-26T18:45:32.623 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.622+0000 7f5788cbd640 1 -- 172.21.15.84:0/1018809234 wait complete. 2024-04-26T18:45:32.633 DEBUG:teuthology.orchestra.run.smithi084:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-26T18:45:32.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.730+0000 7f28b076a640 1 -- 172.21.15.84:0/868595484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a81028f0 msgr2=0x7f28a8102d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.730+0000 7f28b076a640 1 --2- 172.21.15.84:0/868595484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a81028f0 0x7f28a8102d70 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2898009930 tx=0x7f289802a420 comp rx=0 tx=0).stop 2024-04-26T18:45:32.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.730+0000 7f28b076a640 1 -- 172.21.15.84:0/868595484 shutdown_connections 2024-04-26T18:45:32.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.730+0000 7f28b076a640 1 --2- 172.21.15.84:0/868595484 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28a81032b0 0x7f28a810f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.730+0000 7f28b076a640 1 --2- 172.21.15.84:0/868595484 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a81028f0 0x7f28a8102d70 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.733 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.730+0000 7f28b076a640 1 --2- 172.21.15.84:0/868595484 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28a8108900 0x7f28a8108ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.730+0000 7f28b076a640 1 -- 172.21.15.84:0/868595484 >> 172.21.15.84:0/868595484 conn(0x7f28a8079d30 msgr2=0x7f28a807a140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.731+0000 7f28b076a640 1 -- 172.21.15.84:0/868595484 shutdown_connections 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.731+0000 7f28b076a640 1 -- 172.21.15.84:0/868595484 wait complete. 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.731+0000 7f28b076a640 1 Processor -- start 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.731+0000 7f28b076a640 1 -- start start 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.731+0000 7f28b076a640 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28a81028f0 0x7f28a8199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.731+0000 7f28b076a640 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28a81032b0 0x7f28a81996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.731+0000 7f28b076a640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a8108900 0x7f28a819da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.731+0000 7f28b076a640 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f28a8112bc0 con 0x7f28a8108900 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.731+0000 7f28b076a640 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f28a8112a40 con 0x7f28a81028f0 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.731+0000 7f28b076a640 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f28a8112d40 con 0x7f28a81032b0 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.731+0000 7f28aed07640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a8108900 0x7f28a819da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.731+0000 7f28aed07640 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a8108900 0x7f28a819da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:45634/0 (socket says 172.21.15.84:45634) 2024-04-26T18:45:32.734 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.731+0000 7f28aed07640 1 -- 172.21.15.84:0/24350102 learned_addr learned my addr 172.21.15.84:0/24350102 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f28add05640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28a81032b0 0x7f28a81996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f28ae506640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28a81028f0 0x7f28a8199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f28aed07640 1 -- 172.21.15.84:0/24350102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28a81032b0 msgr2=0x7f28a81996a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f28aed07640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28a81032b0 0x7f28a81996a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f28aed07640 1 -- 172.21.15.84:0/24350102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28a81028f0 msgr2=0x7f28a8199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f28aed07640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28a81028f0 0x7f28a8199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f28aed07640 1 -- 172.21.15.84:0/24350102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28a819e150 con 0x7f28a8108900 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f28add05640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28a81032b0 0x7f28a81996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f28aed07640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a8108900 0x7f28a819da20 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f28a000eaa0 tx=0x7f28a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f289f7fe640 1 -- 172.21.15.84:0/24350102 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a000cdb0 con 0x7f28a8108900 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f289f7fe640 1 -- 172.21.15.84:0/24350102 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28a0005020 con 0x7f28a8108900 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f28b076a640 1 -- 172.21.15.84:0/24350102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28a819e440 con 0x7f28a8108900 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f289f7fe640 1 -- 172.21.15.84:0/24350102 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28a00116d0 con 0x7f28a8108900 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.732+0000 7f28b076a640 1 -- 172.21.15.84:0/24350102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28a81a5cd0 con 0x7f28a8108900 2024-04-26T18:45:32.735 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.733+0000 7f28b076a640 1 -- 172.21.15.84:0/24350102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28a8069ba0 con 0x7f28a8108900 2024-04-26T18:45:32.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.734+0000 7f289f7fe640 1 -- 172.21.15.84:0/24350102 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f28a00051d0 con 0x7f28a8108900 2024-04-26T18:45:32.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.734+0000 7f289f7fe640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f28940a3d20 0x7f28940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:45:32.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.735+0000 7f28ae506640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f28940a3d20 0x7f28940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:45:32.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.735+0000 7f289f7fe640 1 -- 172.21.15.84:0/24350102 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 8579+0+0 (secure 0 0 0) 0x7f28a00d0b00 con 0x7f28a8108900 2024-04-26T18:45:32.736 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.735+0000 7f28ae506640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f28940a3d20 0x7f28940a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f28a4005fd0 tx=0x7f28a400b510 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:45:32.739 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.739+0000 7f289f7fe640 1 -- 172.21.15.84:0/24350102 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28a0016030 con 0x7f28a8108900 2024-04-26T18:45:32.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:32 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3983638441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:45:32.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:32 smithi103 ceph-mon[32727]: pgmap v554: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-26T18:45:32.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:32 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3841504791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:45:32.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:32 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/547027639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T18:45:32.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:32 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3983638441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:45:32.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:32 smithi174 ceph-mon[32705]: pgmap v554: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-26T18:45:32.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:32 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3841504791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:45:32.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:32 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/547027639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T18:45:32.964 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.964+0000 7f28b076a640 1 -- 172.21.15.84:0/24350102 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f28a810a8d0 con 0x7f28a8108900 2024-04-26T18:45:32.972 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.973+0000 7f289f7fe640 1 -- 172.21.15.84:0/24350102 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) ==== 81+0+746063 (secure 0 0 0) 0x7f28a009d160 con 0x7f28a8108900 2024-04-26T18:45:32.983 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:45:32.983 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "epoch": 25, 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "active_gid": 15240, 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "active_name": "x", 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "active_addrs": { 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:6800", 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3453983695 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v1", 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:6801", 2024-04-26T18:45:32.984 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3453983695 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: "active_addr": "172.21.15.84:6801/3453983695", 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: "active_change": "2024-04-26T18:27:57.444827+0000", 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: "active_mgr_features": 4540701547738038271, 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: "available": true, 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: "standbys": [ 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: "gid": 15237, 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "y", 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr_features": 4540701547738038271, 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: "available_modules": [ 2024-04-26T18:45:32.985 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "alerts", 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:32.986 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:32.987 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:32.988 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:32.989 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:32.990 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:32.991 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_destination": { 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_destination", 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:32.992 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Email address to send alerts to", 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_from_name": { 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_from_name", 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "Ceph", 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:32.993 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Email From: name", 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_host": { 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_host", 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:32.994 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP server", 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_password": { 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_password", 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Password to authenticate with", 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:32.995 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_port": { 2024-04-26T18:45:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_port", 2024-04-26T18:45:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "465", 2024-04-26T18:45:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:32.996 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP port", 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_sender": { 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_sender", 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP envelope sender", 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:32.997 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_ssl": { 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_ssl", 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:32.998 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_user": { 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_user", 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "User to authenticate as", 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:32.999 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "balancer", 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "active": { 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "active", 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.000 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "begin_time": { 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "begin_time", 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0000", 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.001 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "begin_weekday": { 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "begin_weekday", 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0", 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "6", 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T18:45:33.002 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_max_iterations": { 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_max_iterations", 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "25", 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "250", 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.003 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_metrics": { 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_metrics", 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_step": { 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_step", 2024-04-26T18:45:33.004 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0.5", 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0.001", 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "0.999", 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "aggressiveness of optimization", 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "end_time": { 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "end_time", 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.005 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "2359", 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "end_weekday": { 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "end_weekday", 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.006 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0", 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "6", 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.007 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.008 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.009 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.010 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "min_score": { 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "min_score", 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.011 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": { 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mode", 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "upmap", 2024-04-26T18:45:33.012 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "crush-compat", 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "none", 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "read", 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap", 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap-read" 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Balancer mode", 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "pool_ids": { 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pool_ids", 2024-04-26T18:45:33.013 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.014 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap_max_deviation": { 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "upmap_max_deviation", 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.015 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap_max_optimizations": { 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "upmap_max_optimizations", 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.016 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephadm", 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.017 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_down_multiplier": { 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_down_multiplier", 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3.0", 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.018 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_refresh_rate": { 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_refresh_rate", 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "20", 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_starting_port": { 2024-04-26T18:45:33.019 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_starting_port", 2024-04-26T18:45:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "4721", 2024-04-26T18:45:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T18:45:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.020 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_ptrace": { 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_ptrace", 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "autotune_interval": { 2024-04-26T18:45:33.021 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "autotune_interval", 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: "autotune_memory_target_ratio": { 2024-04-26T18:45:33.022 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0.7", 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm_log_destination": { 2024-04-26T18:45:33.023 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephadm_log_destination", 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "file", 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "file,syslog", 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "syslog" 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.024 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.025 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.025 INFO:teuthology.orchestra.run.smithi084.stdout: "cgroups_split": { 2024-04-26T18:45:33.025 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cgroups_split", 2024-04-26T18:45:33.025 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.025 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.025 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.025 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.025 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.025 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.025 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.025 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T18:45:33.025 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.025 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.026 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.026 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.026 INFO:teuthology.orchestra.run.smithi084.stdout: "config_checks_enabled": { 2024-04-26T18:45:33.026 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_checks_enabled", 2024-04-26T18:45:33.026 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.026 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.026 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.026 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.026 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.026 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.026 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.026 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T18:45:33.026 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "config_dashboard": { 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_dashboard", 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.027 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_alertmanager": { 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_alertmanager", 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_base": { 2024-04-26T18:45:33.028 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_base", 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Container image name, without the tag", 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_elasticsearch": { 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_elasticsearch", 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.029 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "elasticsearch container image", 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_grafana": { 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_grafana", 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.030 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_haproxy": { 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_haproxy", 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T18:45:33.031 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "HAproxy container image", 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_agent": { 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_agent", 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.032 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger agent container image", 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_collector": { 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_collector", 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger collector container image", 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.033 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_query": { 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_query", 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger query container image", 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.034 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_keepalived": { 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_keepalived", 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Keepalived container image", 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.035 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_loki": { 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_loki", 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Loki container image", 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_node_exporter": { 2024-04-26T18:45:33.036 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_node_exporter", 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_nvmeof": { 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_nvmeof", 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.037 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Nvme-of container image", 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_prometheus": { 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_prometheus", 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.038 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_promtail": { 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_promtail", 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.039 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Promtail container image", 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_samba": { 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_samba", 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.040 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Samba/SMB container image", 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_snmp_gateway": { 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_snmp_gateway", 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.041 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SNMP Gateway container image", 2024-04-26T18:45:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: "container_init": { 2024-04-26T18:45:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_init", 2024-04-26T18:45:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.042 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: "daemon_cache_timeout": { 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "daemon_cache_timeout", 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.043 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "default_cephadm_command_timeout": { 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "900", 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.044 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "default_registry": { 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "default_registry", 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io", 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.045 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: "device_cache_timeout": { 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_cache_timeout", 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1800", 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache device inventory", 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.046 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "device_enhanced_scan": { 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_enhanced_scan", 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.047 INFO:teuthology.orchestra.run.smithi084.stdout: "facts_cache_timeout": { 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "facts_cache_timeout", 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache host facts data", 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "grafana_dashboards_path": { 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "grafana_dashboards_path", 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.048 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: "host_check_interval": { 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "host_check_interval", 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.049 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to perform a host check", 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "hw_monitoring": { 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "hw_monitoring", 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.050 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.051 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.051 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.051 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.051 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.051 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T18:45:33.051 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.051 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.051 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.051 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.051 INFO:teuthology.orchestra.run.smithi084.stdout: "inventory_list_all": { 2024-04-26T18:45:33.051 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "inventory_list_all", 2024-04-26T18:45:33.051 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.051 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.052 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.052 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.052 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.052 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.052 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.052 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T18:45:33.052 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.052 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.052 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.052 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.052 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.052 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.052 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.053 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.054 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.054 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.054 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.054 INFO:teuthology.orchestra.run.smithi084.stdout: "log_refresh_metadata": { 2024-04-26T18:45:33.054 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_refresh_metadata", 2024-04-26T18:45:33.054 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.054 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.054 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.054 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.054 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.054 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.055 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.056 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.057 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.058 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "*", 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.059 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "max_count_per_host": { 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_count_per_host", 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.060 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "max number of daemons per service per host", 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "max_osd_draining_count": { 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_osd_draining_count", 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.061 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "migration_current": { 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "migration_current", 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "internal - do not modify", 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.062 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": { 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mode", 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "root", 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm-package", 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "root" 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.063 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "oob_default_addr": { 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "oob_default_addr", 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "169.254.1.1", 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.064 INFO:teuthology.orchestra.run.smithi084.stdout: "prometheus_alerts_path": { 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prometheus_alerts_path", 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_insecure": { 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_insecure", 2024-04-26T18:45:33.065 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_password": { 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_password", 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.066 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_url": { 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_url", 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.067 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_username": { 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_username", 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.068 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "secure_monitoring_stack": { 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "secure_monitoring_stack", 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T18:45:33.069 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "service_discovery_port": { 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "service_discovery_port", 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8765", 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "cephadm service discovery port", 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.070 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "ssh_config_file": { 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssh_config_file", 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "use_agent": { 2024-04-26T18:45:33.071 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "use_agent", 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "use_repo_digest": { 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "use_repo_digest", 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.072 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_failed_host_check": { 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_failed_host_check", 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.073 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_stray_daemons": { 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_stray_daemons", 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.074 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_stray_hosts": { 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_stray_hosts", 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.075 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crash", 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.076 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.077 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.078 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.079 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "retain_interval": { 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "retain_interval", 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "31536000", 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_recent_interval": { 2024-04-26T18:45:33.081 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_recent_interval", 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1209600", 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.082 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "dashboard", 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.083 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.084 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.085 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: "AUDIT_API_ENABLED": { 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.086 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.087 INFO:teuthology.orchestra.run.smithi084.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.088 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.089 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.090 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.091 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.092 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.093 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T18:45:33.094 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.095 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.096 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_URL": { 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_URL", 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.097 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.098 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.099 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T18:45:33.100 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.101 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.102 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.102 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.102 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.102 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.102 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.102 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.102 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.102 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.102 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.102 INFO:teuthology.orchestra.run.smithi084.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T18:45:33.102 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T18:45:33.102 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.102 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "[]", 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.103 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "{}", 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.104 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.105 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.106 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.107 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.108 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.109 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.109 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.109 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.109 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.109 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T18:45:33.109 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T18:45:33.109 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.109 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.109 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.109 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.109 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.109 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.109 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.110 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.111 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.111 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.111 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.111 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T18:45:33.111 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T18:45:33.111 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.111 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.111 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.111 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.111 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.111 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.111 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.112 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.112 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.112 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.112 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.112 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.112 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T18:45:33.112 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T18:45:33.112 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.112 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.112 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.112 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.112 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.113 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.114 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.115 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8", 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.116 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "45", 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.117 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.118 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T18:45:33.118 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T18:45:33.118 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.118 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.118 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.118 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.118 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.118 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.118 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.118 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.118 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.118 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.118 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.119 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.120 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.121 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.122 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.123 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.125 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.125 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.125 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.125 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.125 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.125 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.125 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T18:45:33.125 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T18:45:33.125 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "cross_origin_url": { 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cross_origin_url", 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.126 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "crt_file": { 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crt_file", 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.127 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "debug": { 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "debug", 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.128 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable/disable debug options", 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "jwt_token_ttl": { 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "jwt_token_ttl", 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "28800", 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.129 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.130 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.130 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.130 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.130 INFO:teuthology.orchestra.run.smithi084.stdout: "key_file": { 2024-04-26T18:45:33.130 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "key_file", 2024-04-26T18:45:33.130 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.130 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.130 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.130 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.130 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.130 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.130 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.131 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.131 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.131 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.131 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.131 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.131 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.131 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.131 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.131 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.131 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.131 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.131 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.132 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.133 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.134 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.135 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "motd": { 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "motd", 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "The message of the day", 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.137 INFO:teuthology.orchestra.run.smithi084.stdout: "redirect_resolve_ip_addr": { 2024-04-26T18:45:33.137 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T18:45:33.137 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.137 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.137 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.137 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.137 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.137 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.137 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.137 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.137 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.137 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "::", 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.138 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8080", 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl": { 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl", 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl_server_port": { 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl_server_port", 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8443", 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_behaviour": { 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_behaviour", 2024-04-26T18:45:33.141 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "redirect", 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: "redirect" 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.142 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_error_status_code": { 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_error_status_code", 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "500", 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "400", 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "599", 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "url_prefix": { 2024-04-26T18:45:33.143 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "url_prefix", 2024-04-26T18:45:33.144 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.144 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.144 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.144 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.144 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.144 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.144 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.144 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.144 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.144 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.144 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.144 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.144 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "devicehealth", 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: "enable_monitoring": { 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enable_monitoring", 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.145 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "monitor device health metrics", 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.146 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.147 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.148 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.149 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "mark_out_threshold": { 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mark_out_threshold", 2024-04-26T18:45:33.150 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "2419200", 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: "pool_name": { 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pool_name", 2024-04-26T18:45:33.151 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.152 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.152 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.152 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "device_health_metrics", 2024-04-26T18:45:33.152 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.152 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.152 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.152 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T18:45:33.152 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.152 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.152 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.152 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.152 INFO:teuthology.orchestra.run.smithi084.stdout: "retention_period": { 2024-04-26T18:45:33.152 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "retention_period", 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15552000", 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long to retain device health metrics", 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "scrape_frequency": { 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "scrape_frequency", 2024-04-26T18:45:33.153 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "86400", 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "self_heal": { 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "self_heal", 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.154 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.155 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_threshold": { 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_threshold", 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "7257600", 2024-04-26T18:45:33.156 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "diskprediction_local", 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.157 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.158 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.159 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.160 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.161 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "predict_interval": { 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "predict_interval", 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "86400", 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.162 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "predictor_model": { 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "predictor_model", 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "prophetstor", 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.163 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.164 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "influx", 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": false, 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "influxdb python module not found", 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: "batch_size": { 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "batch_size", 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5000", 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.165 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: "database": { 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "database", 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "ceph", 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.166 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": { 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "hostname", 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.167 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB server hostname", 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "30", 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "5", 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.168 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.169 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.170 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.171 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.172 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "password": { 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "password", 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.173 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.174 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.174 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.174 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.174 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "password of InfluxDB server user", 2024-04-26T18:45:33.174 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.174 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.174 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.174 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.174 INFO:teuthology.orchestra.run.smithi084.stdout: "port": { 2024-04-26T18:45:33.174 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "port", 2024-04-26T18:45:33.174 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.174 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.174 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8086", 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB server port", 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl": { 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl", 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.175 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "false", 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "threads": { 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "threads", 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.176 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "32", 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "username": { 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "username", 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.177 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "username of InfluxDB server user", 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "verify_ssl": { 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "verify_ssl", 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "true", 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.178 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "insights", 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.179 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.180 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.182 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.183 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "iostat", 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.184 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.185 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.187 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.188 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "k8sevents", 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_event_retention_days": { 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ceph_event_retention_days", 2024-04-26T18:45:33.189 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "7", 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "config_check_secs": { 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_check_secs", 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.190 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "10", 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.191 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.192 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.193 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.194 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.195 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "localpool", 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: "failure_domain": { 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "failure_domain", 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.196 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "host", 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "failure domain for any created local pool", 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.197 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.198 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.199 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.200 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.201 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: "min_size": { 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "min_size", 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.202 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default min_size for any created local pool", 2024-04-26T18:45:33.203 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T18:45:33.203 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.203 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.203 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.203 INFO:teuthology.orchestra.run.smithi084.stdout: "num_rep": { 2024-04-26T18:45:33.203 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "num_rep", 2024-04-26T18:45:33.203 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.203 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.203 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.203 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3", 2024-04-26T18:45:33.203 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.203 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.203 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default replica count for any created local pool", 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_num": { 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pg_num", 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "128", 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.204 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "prefix": { 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prefix", 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.205 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "name prefix for any created local pool", 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: "subtree": { 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "subtree", 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "rack", 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.206 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mds_autoscaler", 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.207 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.208 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.210 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.211 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mirroring", 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.212 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.213 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.214 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.214 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.214 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.214 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.214 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.214 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.214 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.214 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.214 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.214 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.214 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.214 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.215 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.215 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.215 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.215 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.215 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.215 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.215 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.215 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.215 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.215 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.215 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.215 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.216 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.217 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.217 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.217 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.217 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.217 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.217 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.217 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.217 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.217 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.217 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.217 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.217 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.217 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "nfs", 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.218 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.219 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.220 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.221 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.222 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "orchestrator", 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: "fail_fs": { 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "fail_fs", 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.223 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.224 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.225 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.226 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.227 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.228 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator": { 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "orchestrator", 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm", 2024-04-26T18:45:33.229 INFO:teuthology.orchestra.run.smithi084.stdout: "rook", 2024-04-26T18:45:33.230 INFO:teuthology.orchestra.run.smithi084.stdout: "test_orchestrator" 2024-04-26T18:45:33.230 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.230 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Orchestrator backend", 2024-04-26T18:45:33.230 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.230 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.230 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.230 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.230 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.230 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.230 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "osd_perf_query", 2024-04-26T18:45:33.230 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.231 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.232 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.233 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.234 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "osd_support", 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.236 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.237 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.238 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.239 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.240 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.241 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.241 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.241 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.241 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pg_autoscaler", 2024-04-26T18:45:33.241 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.241 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.241 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.241 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.241 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.241 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.241 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.241 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.242 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.242 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.242 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.242 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.242 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.242 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.242 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.242 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.242 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.242 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.242 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.242 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.242 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.243 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.244 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.245 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.246 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.246 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.246 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.246 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.246 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.246 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.246 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.246 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.246 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:33.246 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:33.246 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.246 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: "threshold": { 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "threshold", 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.247 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3.0", 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1.0", 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "scaling threshold", 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "progress", 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.248 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_pg_recovery_event": { 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_pg_recovery_event", 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "enabled": { 2024-04-26T18:45:33.249 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enabled", 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.250 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.251 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.252 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.253 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.254 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.254 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.254 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.254 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.254 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.254 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.254 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.254 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.254 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.254 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.254 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.254 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.254 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.255 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.255 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.255 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.255 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.255 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.255 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.255 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.255 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.255 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.255 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.255 INFO:teuthology.orchestra.run.smithi084.stdout: "max_completed_events": { 2024-04-26T18:45:33.255 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_completed_events", 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "50", 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "number of past completed events to remember", 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:33.256 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.257 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.257 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.257 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-04-26T18:45:33.257 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.257 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.257 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.257 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long the module is going to sleep", 2024-04-26T18:45:33.257 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.257 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.257 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.257 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.257 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.257 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prometheus", 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: "cache": { 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cache", 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.258 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: "exclude_perf_counters": { 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "exclude_perf_counters", 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.259 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.260 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.261 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.262 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.263 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.264 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_stats_pools": { 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_stats_pools", 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.265 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "300", 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.266 INFO:teuthology.orchestra.run.smithi084.stdout: "scrape_interval": { 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "scrape_interval", 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15.0", 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-04-26T18:45:33.267 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "::", 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-04-26T18:45:33.268 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "9283", 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "stale_cache_strategy": { 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "stale_cache_strategy", 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.269 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "log", 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_behaviour": { 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_behaviour", 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.270 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "default", 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: "default", 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: "error" 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.271 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_error_status_code": { 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_error_status_code", 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "500", 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "400", 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "599", 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.272 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_support", 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.273 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.274 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.275 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.276 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.277 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.277 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.277 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.277 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.277 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.277 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.277 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.277 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.277 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.277 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.277 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.277 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.277 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: "max_concurrent_snap_create": { 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_concurrent_snap_create", 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.278 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "mirror_snapshot_schedule": { 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.279 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.280 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.280 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.280 INFO:teuthology.orchestra.run.smithi084.stdout: "trash_purge_schedule": { 2024-04-26T18:45:33.280 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "trash_purge_schedule", 2024-04-26T18:45:33.280 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.280 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.280 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.280 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.280 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.280 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "restful", 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.281 INFO:teuthology.orchestra.run.smithi084.stdout: "enable_auth": { 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enable_auth", 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.282 INFO:teuthology.orchestra.run.smithi084.stdout: "key_file": { 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "key_file", 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.283 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.284 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.285 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.286 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.287 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.288 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.289 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rgw", 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.290 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.291 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.292 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.293 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.294 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.295 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rook", 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.296 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.297 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.298 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.299 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.300 INFO:teuthology.orchestra.run.smithi084.stdout: "storage_class": { 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "storage_class", 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "local", 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.301 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "selftest", 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.302 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.303 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.304 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.305 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.306 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: "roption1": { 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "roption1", 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.307 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "roption2": { 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "roption2", 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "xyz", 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.308 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption1": { 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption1", 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.309 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption2": { 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption2", 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.310 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption3": { 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption3", 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.311 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption4": { 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption4", 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.312 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption5": { 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption5", 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption6": { 2024-04-26T18:45:33.313 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption6", 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption7": { 2024-04-26T18:45:33.314 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption7", 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "42", 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "testkey": { 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testkey", 2024-04-26T18:45:33.315 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "testlkey": { 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testlkey", 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.316 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "testnewline": { 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testnewline", 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.317 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snap_schedule", 2024-04-26T18:45:33.318 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_m_granularity": { 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_m_granularity", 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.319 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "dump_on_update": { 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "dump_on_update", 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "dump database to debug log on update", 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.320 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.321 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.322 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.323 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.324 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.325 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "stats", 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.326 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.327 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.327 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.327 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.327 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.327 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.327 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.327 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.327 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.327 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.327 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.327 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.327 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.329 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.329 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.329 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.329 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.329 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.329 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.329 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.329 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.330 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.330 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.330 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.330 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.330 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.330 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.330 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.330 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.330 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.330 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.330 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.330 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.331 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.332 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "status", 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.333 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.334 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.335 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.336 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.336 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.336 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.336 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.336 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.336 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.336 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.336 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.336 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.336 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.336 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.336 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.336 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.337 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "telegraf", 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: "address": { 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "address", 2024-04-26T18:45:33.338 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:33.339 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15", 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.340 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.341 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.342 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.342 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.342 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.342 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.342 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.342 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.342 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.342 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.342 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.342 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.342 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.342 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.342 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.343 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.343 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.343 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.343 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.343 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.343 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.343 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.343 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.343 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.343 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.343 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.343 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.343 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.344 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "telemetry", 2024-04-26T18:45:33.345 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_basic": { 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_basic", 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.346 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_crash": { 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_crash", 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.347 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_device": { 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_device", 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.348 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.349 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_ident": { 2024-04-26T18:45:33.349 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_ident", 2024-04-26T18:45:33.349 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.349 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.349 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.349 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.349 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.349 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.349 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.349 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T18:45:33.349 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.349 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.349 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_perf": { 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_perf", 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.350 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: "contact": { 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "contact", 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.351 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: "description": { 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "description", 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.352 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: "device_url": { 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_url", 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.353 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.354 INFO:teuthology.orchestra.run.smithi084.stdout: "enabled": { 2024-04-26T18:45:33.354 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enabled", 2024-04-26T18:45:33.354 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.354 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.354 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.354 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.354 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.354 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.354 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.354 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.354 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.354 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.354 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "24", 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "8", 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.355 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "last_opt_revision": { 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "last_opt_revision", 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1", 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "leaderboard": { 2024-04-26T18:45:33.356 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "leaderboard", 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: "leaderboard_description": { 2024-04-26T18:45:33.357 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "leaderboard_description", 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.358 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.359 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.360 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.361 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.362 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: "organization": { 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "organization", 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.363 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: "proxy": { 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "proxy", 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.364 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "url": { 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "url", 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.365 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "test_orchestrator", 2024-04-26T18:45:33.366 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.367 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.368 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.369 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.369 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.369 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.369 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.369 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.369 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.369 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.369 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.369 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.369 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.369 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.369 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.370 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.371 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "volumes", 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.372 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.373 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.374 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.374 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.374 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.374 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.374 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.374 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.374 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.374 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.374 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.374 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.374 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.374 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.374 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.375 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.376 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.376 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.376 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.376 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.376 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.376 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.376 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.376 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.376 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.376 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.376 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.376 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.376 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "max_concurrent_clones": { 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_concurrent_clones", 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "4", 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.377 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "periodic_async_work": { 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "periodic_async_work", 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.378 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Periodically check for async work", 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "snapshot_clone_delay": { 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snapshot_clone_delay", 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.379 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "snapshot_clone_no_wait": { 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T18:45:33.380 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix", 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: "discovery_interval": { 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "discovery_interval", 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.381 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "100", 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "identifier": { 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "identifier", 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.382 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.383 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.384 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.385 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.386 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.387 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.388 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_host": { 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_host", 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_port": { 2024-04-26T18:45:33.389 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_port", 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10051", 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_sender": { 2024-04-26T18:45:33.390 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_sender", 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.391 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: "gid": 24791, 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "z", 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: "mgr_features": 4540701547738038271, 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: "available_modules": [ 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "alerts", 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.392 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.393 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.394 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.395 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.396 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.397 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_destination": { 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_destination", 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Email address to send alerts to", 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.398 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_from_name": { 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_from_name", 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "Ceph", 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Email From: name", 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.399 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_host": { 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_host", 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP server", 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.400 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.401 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.401 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_password": { 2024-04-26T18:45:33.401 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_password", 2024-04-26T18:45:33.401 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.401 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.401 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.401 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.401 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.401 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.401 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.401 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Password to authenticate with", 2024-04-26T18:45:33.401 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.401 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_port": { 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_port", 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "465", 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP port", 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.402 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_sender": { 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_sender", 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP envelope sender", 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.403 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_ssl": { 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_ssl", 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.404 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_user": { 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_user", 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "User to authenticate as", 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.405 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "balancer", 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: "active": { 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "active", 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.406 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: "begin_time": { 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "begin_time", 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0000", 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.407 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "begin_weekday": { 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "begin_weekday", 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0", 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "6", 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.408 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_max_iterations": { 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_max_iterations", 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "25", 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "250", 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.409 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_metrics": { 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_metrics", 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.410 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_step": { 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_step", 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0.5", 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0.001", 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "0.999", 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "aggressiveness of optimization", 2024-04-26T18:45:33.411 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T18:45:33.412 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.412 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.412 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.412 INFO:teuthology.orchestra.run.smithi084.stdout: "end_time": { 2024-04-26T18:45:33.412 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "end_time", 2024-04-26T18:45:33.412 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.412 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.412 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.412 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "2359", 2024-04-26T18:45:33.412 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.412 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.412 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.412 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: "end_weekday": { 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "end_weekday", 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0", 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "6", 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.413 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T18:45:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T18:45:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.414 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.415 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.416 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.417 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.418 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: "min_score": { 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "min_score", 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T18:45:33.419 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": { 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mode", 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "upmap", 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "crush-compat", 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "none", 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "read", 2024-04-26T18:45:33.420 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap", 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap-read" 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Balancer mode", 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: "pool_ids": { 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pool_ids", 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.421 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.422 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap_max_deviation": { 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "upmap_max_deviation", 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.423 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap_max_optimizations": { 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "upmap_max_optimizations", 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.424 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephadm", 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_down_multiplier": { 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_down_multiplier", 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.425 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3.0", 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_refresh_rate": { 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_refresh_rate", 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.426 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "20", 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_starting_port": { 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_starting_port", 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.427 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "4721", 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_ptrace": { 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_ptrace", 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.428 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "autotune_interval": { 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "autotune_interval", 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.429 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: "autotune_memory_target_ratio": { 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0.7", 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.430 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm_log_destination": { 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephadm_log_destination", 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.431 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: "file", 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: "file,syslog", 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: "syslog" 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: "cgroups_split": { 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cgroups_split", 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.432 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "config_checks_enabled": { 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_checks_enabled", 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.433 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "config_dashboard": { 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_dashboard", 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.434 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_alertmanager": { 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_alertmanager", 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.435 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_base": { 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_base", 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.436 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Container image name, without the tag", 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_elasticsearch": { 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_elasticsearch", 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.437 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "elasticsearch container image", 2024-04-26T18:45:33.438 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.438 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.438 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.438 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.438 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_grafana": { 2024-04-26T18:45:33.438 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_grafana", 2024-04-26T18:45:33.438 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.438 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.438 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.438 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T18:45:33.438 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.438 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.438 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_haproxy": { 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_haproxy", 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.439 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "HAproxy container image", 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_agent": { 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_agent", 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger agent container image", 2024-04-26T18:45:33.440 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_collector": { 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_collector", 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger collector container image", 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.441 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.442 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.442 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_query": { 2024-04-26T18:45:33.442 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_query", 2024-04-26T18:45:33.442 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.442 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.442 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.442 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T18:45:33.442 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.442 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.442 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.442 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger query container image", 2024-04-26T18:45:33.442 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.442 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_keepalived": { 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_keepalived", 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Keepalived container image", 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.443 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_loki": { 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_loki", 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Loki container image", 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_node_exporter": { 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_node_exporter", 2024-04-26T18:45:33.444 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_nvmeof": { 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_nvmeof", 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.445 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Nvme-of container image", 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_prometheus": { 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_prometheus", 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.446 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_promtail": { 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_promtail", 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.447 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Promtail container image", 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_samba": { 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_samba", 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.448 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Samba/SMB container image", 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_snmp_gateway": { 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_snmp_gateway", 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SNMP Gateway container image", 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.449 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "container_init": { 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_init", 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.450 INFO:teuthology.orchestra.run.smithi084.stdout: "daemon_cache_timeout": { 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "daemon_cache_timeout", 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "default_cephadm_command_timeout": { 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T18:45:33.451 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "900", 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "default_registry": { 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "default_registry", 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.452 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io", 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: "device_cache_timeout": { 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_cache_timeout", 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.453 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1800", 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache device inventory", 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "device_enhanced_scan": { 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_enhanced_scan", 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.454 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "facts_cache_timeout": { 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "facts_cache_timeout", 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.455 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache host facts data", 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "grafana_dashboards_path": { 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "grafana_dashboards_path", 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.456 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "host_check_interval": { 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "host_check_interval", 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.457 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to perform a host check", 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "hw_monitoring": { 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "hw_monitoring", 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T18:45:33.458 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: "inventory_list_all": { 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "inventory_list_all", 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T18:45:33.459 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.460 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "log_refresh_metadata": { 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_refresh_metadata", 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.461 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.462 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.463 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.464 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.465 INFO:teuthology.orchestra.run.smithi084.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T18:45:33.466 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "*", 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "max_count_per_host": { 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_count_per_host", 2024-04-26T18:45:33.467 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "max number of daemons per service per host", 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "max_osd_draining_count": { 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_osd_draining_count", 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.468 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "migration_current": { 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "migration_current", 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.469 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "internal - do not modify", 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": { 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mode", 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.470 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "root", 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm-package", 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: "root" 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: "oob_default_addr": { 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "oob_default_addr", 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.471 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "169.254.1.1", 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "prometheus_alerts_path": { 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prometheus_alerts_path", 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.472 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_insecure": { 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_insecure", 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.473 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_password": { 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_password", 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.474 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_url": { 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_url", 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.475 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_username": { 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_username", 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.476 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "secure_monitoring_stack": { 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "secure_monitoring_stack", 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.477 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "service_discovery_port": { 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "service_discovery_port", 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8765", 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.478 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "cephadm service discovery port", 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "ssh_config_file": { 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssh_config_file", 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.479 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "use_agent": { 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "use_agent", 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.480 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: "use_repo_digest": { 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "use_repo_digest", 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.481 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_failed_host_check": { 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_failed_host_check", 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_stray_daemons": { 2024-04-26T18:45:33.482 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_stray_daemons", 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_stray_hosts": { 2024-04-26T18:45:33.483 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_stray_hosts", 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.484 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crash", 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.485 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.486 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.487 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.488 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "retain_interval": { 2024-04-26T18:45:33.489 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "retain_interval", 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "31536000", 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_recent_interval": { 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_recent_interval", 2024-04-26T18:45:33.490 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1209600", 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.491 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "dashboard", 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.492 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.493 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.494 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: "AUDIT_API_ENABLED": { 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.495 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.496 INFO:teuthology.orchestra.run.smithi084.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T18:45:33.497 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T18:45:33.498 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.499 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.500 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.501 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.502 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.502 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.502 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.502 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.502 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.502 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.502 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.502 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.502 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T18:45:33.502 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T18:45:33.502 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.502 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.503 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.504 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.505 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.506 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_URL": { 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_URL", 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.507 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.508 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.509 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.510 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.511 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.511 INFO:teuthology.orchestra.run.smithi084.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T18:45:33.511 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T18:45:33.511 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.511 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.511 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.511 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.511 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.511 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.511 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.511 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.511 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.511 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.512 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "[]", 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.513 INFO:teuthology.orchestra.run.smithi084.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "{}", 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T18:45:33.514 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T18:45:33.515 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T18:45:33.516 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T18:45:33.517 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.518 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.518 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.518 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.518 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.518 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.518 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.518 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.518 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.518 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.518 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.518 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.518 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T18:45:33.518 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T18:45:33.519 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.520 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.521 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.522 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.523 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.524 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.525 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8", 2024-04-26T18:45:33.526 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "45", 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.527 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.528 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.529 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.530 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.531 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.533 INFO:teuthology.orchestra.run.smithi084.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T18:45:33.533 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T18:45:33.533 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.533 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.533 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.533 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.533 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.533 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.533 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.533 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.534 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.535 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.536 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: "cross_origin_url": { 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cross_origin_url", 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.537 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: "crt_file": { 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crt_file", 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.538 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "debug": { 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "debug", 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable/disable debug options", 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.539 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "jwt_token_ttl": { 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "jwt_token_ttl", 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "28800", 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.540 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "key_file": { 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "key_file", 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.541 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.542 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.543 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.544 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.545 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "motd": { 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "motd", 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.546 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "The message of the day", 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "redirect_resolve_ip_addr": { 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.547 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "::", 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.548 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8080", 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.549 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl": { 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl", 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.550 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl_server_port": { 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl_server_port", 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8443", 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.551 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_behaviour": { 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_behaviour", 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "redirect", 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: "redirect" 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.552 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_error_status_code": { 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_error_status_code", 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "500", 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "400", 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "599", 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.553 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: "url_prefix": { 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "url_prefix", 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.554 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "devicehealth", 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "enable_monitoring": { 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enable_monitoring", 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.555 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "monitor device health metrics", 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.556 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.557 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.558 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.559 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "mark_out_threshold": { 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mark_out_threshold", 2024-04-26T18:45:33.560 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "2419200", 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "pool_name": { 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pool_name", 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.561 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "device_health_metrics", 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "retention_period": { 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "retention_period", 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.562 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.563 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15552000", 2024-04-26T18:45:33.563 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.563 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.563 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.563 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long to retain device health metrics", 2024-04-26T18:45:33.563 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.563 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.563 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.563 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.563 INFO:teuthology.orchestra.run.smithi084.stdout: "scrape_frequency": { 2024-04-26T18:45:33.563 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "scrape_frequency", 2024-04-26T18:45:33.563 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.563 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "86400", 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "self_heal": { 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "self_heal", 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.564 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.565 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.566 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.566 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.566 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.566 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T18:45:33.566 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.566 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.566 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.566 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.566 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_threshold": { 2024-04-26T18:45:33.566 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_threshold", 2024-04-26T18:45:33.566 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.566 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.566 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "7257600", 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "diskprediction_local", 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.567 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.568 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.568 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.568 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.568 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.568 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.568 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.568 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.568 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.568 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.568 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.568 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.568 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.568 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.569 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.570 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.571 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "predict_interval": { 2024-04-26T18:45:33.572 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "predict_interval", 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "86400", 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: "predictor_model": { 2024-04-26T18:45:33.573 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "predictor_model", 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "prophetstor", 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:33.574 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.575 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "influx", 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": false, 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "influxdb python module not found", 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "batch_size": { 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "batch_size", 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5000", 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.576 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: "database": { 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "database", 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "ceph", 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.577 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": { 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "hostname", 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB server hostname", 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.578 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "30", 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "5", 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.579 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.580 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.581 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.582 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.583 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "password": { 2024-04-26T18:45:33.584 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "password", 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "password of InfluxDB server user", 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "port": { 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "port", 2024-04-26T18:45:33.585 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8086", 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB server port", 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl": { 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl", 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.586 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "false", 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "threads": { 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "threads", 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.587 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "32", 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "username": { 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "username", 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.588 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "username of InfluxDB server user", 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "verify_ssl": { 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "verify_ssl", 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "true", 2024-04-26T18:45:33.589 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "insights", 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.590 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.591 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.592 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.593 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.594 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.594 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.594 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.594 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.594 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.594 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.594 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.594 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.594 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.594 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.594 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.594 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.595 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.595 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.595 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.595 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.595 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.595 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.595 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.595 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.595 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.595 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.595 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.595 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.595 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.596 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.596 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.596 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.596 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "iostat", 2024-04-26T18:45:33.596 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.596 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.596 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.596 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.596 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.596 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.596 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.596 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.596 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.597 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.598 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.598 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.598 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.598 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.598 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.598 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.598 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.598 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.598 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.598 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.598 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.598 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.599 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.600 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.601 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.601 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.601 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.601 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.601 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.601 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.601 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.601 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.601 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.601 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.601 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.601 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.602 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "k8sevents", 2024-04-26T18:45:33.602 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.602 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.602 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.602 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_event_retention_days": { 2024-04-26T18:45:33.602 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ceph_event_retention_days", 2024-04-26T18:45:33.602 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.602 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.602 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.602 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "7", 2024-04-26T18:45:33.602 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.602 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi084.stdout: "config_check_secs": { 2024-04-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_check_secs", 2024-04-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.603 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "10", 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.604 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.605 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.606 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.606 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.606 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.606 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.606 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.606 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.606 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.606 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.606 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.606 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.606 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.606 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.606 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.607 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.607 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.607 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.607 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.607 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.607 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.607 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.607 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.607 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.607 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.607 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.607 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.608 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.608 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.608 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.608 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.608 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.608 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.608 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.608 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.608 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.608 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.608 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.608 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.608 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.609 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.610 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "localpool", 2024-04-26T18:45:33.610 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.610 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.610 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.610 INFO:teuthology.orchestra.run.smithi084.stdout: "failure_domain": { 2024-04-26T18:45:33.610 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "failure_domain", 2024-04-26T18:45:33.610 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.610 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.610 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.610 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "host", 2024-04-26T18:45:33.610 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.610 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.610 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "failure domain for any created local pool", 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.611 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.612 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.612 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.612 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.612 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.612 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.612 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.612 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.612 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.612 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.612 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.612 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.613 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.614 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.615 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.616 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.616 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.616 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.616 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.616 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.616 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.616 INFO:teuthology.orchestra.run.smithi084.stdout: "min_size": { 2024-04-26T18:45:33.616 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "min_size", 2024-04-26T18:45:33.616 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.616 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.616 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.616 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.616 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.617 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.617 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.617 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default min_size for any created local pool", 2024-04-26T18:45:33.617 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T18:45:33.617 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.617 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.617 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.617 INFO:teuthology.orchestra.run.smithi084.stdout: "num_rep": { 2024-04-26T18:45:33.617 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "num_rep", 2024-04-26T18:45:33.617 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.617 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.617 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.617 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3", 2024-04-26T18:45:33.618 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.618 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.618 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.618 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default replica count for any created local pool", 2024-04-26T18:45:33.618 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.618 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.618 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.618 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.618 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_num": { 2024-04-26T18:45:33.618 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pg_num", 2024-04-26T18:45:33.618 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.618 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.618 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "128", 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "prefix": { 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prefix", 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.619 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "name prefix for any created local pool", 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "subtree": { 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "subtree", 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "rack", 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.620 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mds_autoscaler", 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.621 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.622 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.622 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.622 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.622 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.622 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.622 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.622 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.622 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.622 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.622 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.622 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.622 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.622 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.623 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.624 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.625 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.626 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.626 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.626 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.626 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.626 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.626 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.626 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.626 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.626 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.626 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.626 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.626 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.627 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mirroring", 2024-04-26T18:45:33.627 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.627 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.627 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.627 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.627 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.627 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.627 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.627 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.627 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.627 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.627 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.628 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.629 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.629 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.629 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.629 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.629 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.629 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.629 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.629 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.629 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.629 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.629 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.629 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.630 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.630 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.630 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.630 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.630 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.630 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.630 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.630 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.630 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.630 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.630 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.630 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.631 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.631 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.631 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.631 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.631 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.631 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.631 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.631 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.631 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.631 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.631 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.631 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.631 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.632 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.632 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.632 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.632 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.632 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.632 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.632 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.632 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.632 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.632 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "nfs", 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.633 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.634 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.634 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.634 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.634 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.634 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.634 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.634 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.634 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.634 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.634 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.634 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.634 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.635 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.636 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.637 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.638 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.638 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.638 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.638 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.638 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.638 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.638 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.638 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.638 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "orchestrator", 2024-04-26T18:45:33.638 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.638 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.638 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.638 INFO:teuthology.orchestra.run.smithi084.stdout: "fail_fs": { 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "fail_fs", 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.639 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.640 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.640 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.640 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.640 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.640 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.640 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.640 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.640 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.640 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.640 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.640 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.640 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.641 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.642 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.643 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator": { 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "orchestrator", 2024-04-26T18:45:33.644 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm", 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: "rook", 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: "test_orchestrator" 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Orchestrator backend", 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.645 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "osd_perf_query", 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.646 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.647 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.648 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.649 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.650 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.650 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.650 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.650 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.650 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.650 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.650 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.650 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.650 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.650 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.650 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.650 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.650 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "osd_support", 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.651 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.652 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.653 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.654 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.654 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.654 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.654 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.654 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.654 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.654 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.654 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.654 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.654 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.654 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.654 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.654 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.655 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.656 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.656 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.656 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.656 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.656 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.656 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.656 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.656 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.656 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.656 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.656 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pg_autoscaler", 2024-04-26T18:45:33.656 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.656 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.657 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.658 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.659 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.659 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.659 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.659 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.659 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.659 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.659 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.659 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.659 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.659 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.659 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.659 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.659 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.660 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.661 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.662 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: "threshold": { 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "threshold", 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3.0", 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1.0", 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "scaling threshold", 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.663 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "progress", 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_pg_recovery_event": { 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_pg_recovery_event", 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.664 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: "enabled": { 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enabled", 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.665 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.666 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.667 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.668 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.669 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.670 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "max_completed_events": { 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_completed_events", 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "50", 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "number of past completed events to remember", 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.671 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long the module is going to sleep", 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.672 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prometheus", 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: "cache": { 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cache", 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.673 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "exclude_perf_counters": { 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "exclude_perf_counters", 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.674 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.675 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.676 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.677 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.678 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.679 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_stats_pools": { 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_stats_pools", 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.680 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "300", 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "scrape_interval": { 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "scrape_interval", 2024-04-26T18:45:33.681 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15.0", 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-04-26T18:45:33.682 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "::", 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.683 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "9283", 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "stale_cache_strategy": { 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "stale_cache_strategy", 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.684 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "log", 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_behaviour": { 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_behaviour", 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.685 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "default", 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: "default", 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: "error" 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_error_status_code": { 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_error_status_code", 2024-04-26T18:45:33.686 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "500", 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "400", 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "599", 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.687 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_support", 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.688 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.689 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.690 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.691 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.692 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.692 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.692 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.692 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.692 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.692 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.692 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.692 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.692 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.692 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.692 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.692 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.692 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.693 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.693 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.693 INFO:teuthology.orchestra.run.smithi084.stdout: "max_concurrent_snap_create": { 2024-04-26T18:45:33.693 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_concurrent_snap_create", 2024-04-26T18:45:33.693 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.693 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.693 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.693 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.694 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.694 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.694 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.694 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.694 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.694 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.694 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.695 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.695 INFO:teuthology.orchestra.run.smithi084.stdout: "mirror_snapshot_schedule": { 2024-04-26T18:45:33.695 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T18:45:33.695 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.695 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.695 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.695 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.695 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.695 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.695 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.695 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "trash_purge_schedule": { 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "trash_purge_schedule", 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.696 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "restful", 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: "enable_auth": { 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enable_auth", 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.697 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: "key_file": { 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "key_file", 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.698 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.699 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.700 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.701 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.702 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.703 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.704 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.705 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rgw", 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.706 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.707 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.708 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.709 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.710 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.710 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.710 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.710 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.710 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.710 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.710 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.710 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.710 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.710 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.710 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.710 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.710 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rook", 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.711 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.712 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.713 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.714 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.715 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "storage_class": { 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "storage_class", 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "local", 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.716 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "selftest", 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.717 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.718 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.719 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.720 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.721 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "roption1": { 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "roption1", 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.722 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "roption2": { 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "roption2", 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "xyz", 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.723 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption1": { 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption1", 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.724 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption2": { 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption2", 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.725 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption3": { 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption3", 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.726 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption4": { 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption4", 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.727 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption5": { 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption5", 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption6": { 2024-04-26T18:45:33.728 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption6", 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption7": { 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption7", 2024-04-26T18:45:33.729 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "42", 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "testkey": { 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testkey", 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.730 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "testlkey": { 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testlkey", 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.731 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "testnewline": { 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testnewline", 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.732 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snap_schedule", 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.733 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_m_granularity": { 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_m_granularity", 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.734 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "dump_on_update": { 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "dump_on_update", 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "dump database to debug log on update", 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.735 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.736 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.737 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.738 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.739 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.741 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "stats", 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.742 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.743 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.744 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.745 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.746 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "status", 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.747 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.748 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.749 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.750 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.751 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "telegraf", 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.752 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "address": { 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "address", 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.753 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15", 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.754 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.755 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.756 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.757 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.758 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "telemetry", 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.759 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_basic": { 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_basic", 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.760 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_crash": { 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_crash", 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_device": { 2024-04-26T18:45:33.761 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_device", 2024-04-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.762 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_ident": { 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_ident", 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.763 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_perf": { 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_perf", 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "contact": { 2024-04-26T18:45:33.764 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "contact", 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: "description": { 2024-04-26T18:45:33.765 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "description", 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "device_url": { 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_url", 2024-04-26T18:45:33.766 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: "enabled": { 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enabled", 2024-04-26T18:45:33.767 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.768 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "24", 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "8", 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "last_opt_revision": { 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "last_opt_revision", 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.769 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1", 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: "leaderboard": { 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "leaderboard", 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.770 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: "leaderboard_description": { 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "leaderboard_description", 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.771 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.772 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.773 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.774 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.775 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.776 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: "organization": { 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "organization", 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.777 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: "proxy": { 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "proxy", 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.778 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: "url": { 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "url", 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.779 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "test_orchestrator", 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.780 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.781 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.782 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.783 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.784 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.784 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.784 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.784 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.784 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.784 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.784 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.784 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.784 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.784 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.784 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.784 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.784 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.785 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "volumes", 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.786 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.787 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.788 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.788 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.788 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.788 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.788 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.788 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.788 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.788 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.788 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.788 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.788 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.788 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.788 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.789 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.790 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "max_concurrent_clones": { 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_concurrent_clones", 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "4", 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.791 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "periodic_async_work": { 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "periodic_async_work", 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Periodically check for async work", 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.792 INFO:teuthology.orchestra.run.smithi084.stdout: "snapshot_clone_delay": { 2024-04-26T18:45:33.793 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snapshot_clone_delay", 2024-04-26T18:45:33.793 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.793 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.793 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.793 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.793 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.793 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.793 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.793 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T18:45:33.793 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.793 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.793 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.793 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: "snapshot_clone_no_wait": { 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.794 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix", 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: "discovery_interval": { 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "discovery_interval", 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "100", 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.795 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "identifier": { 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "identifier", 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.796 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.797 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.798 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.799 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.800 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.801 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.802 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_host": { 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_host", 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_port": { 2024-04-26T18:45:33.803 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_port", 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10051", 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_sender": { 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_sender", 2024-04-26T18:45:33.804 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.805 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: "modules": [ 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm", 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: "dashboard", 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: "iostat", 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: "nfs", 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: "restful", 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: "snap_schedule" 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: "available_modules": [ 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "alerts", 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.806 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.807 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:33.807 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:33.807 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.807 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.807 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.807 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:33.807 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.807 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.807 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.807 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How frequently to reexamine health status", 2024-04-26T18:45:33.807 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.807 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.807 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.808 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.809 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.810 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.811 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_destination": { 2024-04-26T18:45:33.812 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_destination", 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Email address to send alerts to", 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_from_name": { 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_from_name", 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.813 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "Ceph", 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Email From: name", 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_host": { 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_host", 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.814 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP server", 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_password": { 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_password", 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.815 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Password to authenticate with", 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_port": { 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_port", 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "465", 2024-04-26T18:45:33.816 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP port", 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_sender": { 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_sender", 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.817 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SMTP envelope sender", 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_ssl": { 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_ssl", 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.818 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "smtp_user": { 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "smtp_user", 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.819 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "User to authenticate as", 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "balancer", 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: "active": { 2024-04-26T18:45:33.820 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "active", 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "automatically balance PGs across cluster", 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "begin_time": { 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "begin_time", 2024-04-26T18:45:33.821 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0000", 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "beginning time of day to automatically balance", 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: "begin_weekday": { 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "begin_weekday", 2024-04-26T18:45:33.822 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0", 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "6", 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_max_iterations": { 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_max_iterations", 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.823 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "25", 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "250", 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_metrics": { 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_metrics", 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.824 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "pgs,objects,bytes", 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "crush_compat_step": { 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crush_compat_step", 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0.5", 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0.001", 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "0.999", 2024-04-26T18:45:33.825 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.826 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "aggressiveness of optimization", 2024-04-26T18:45:33.826 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-26T18:45:33.826 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.826 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.826 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.826 INFO:teuthology.orchestra.run.smithi084.stdout: "end_time": { 2024-04-26T18:45:33.826 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "end_time", 2024-04-26T18:45:33.826 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.826 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.826 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.826 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "2359", 2024-04-26T18:45:33.826 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.826 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "ending time of day to automatically balance", 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "end_weekday": { 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "end_weekday", 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "0", 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "6", 2024-04-26T18:45:33.827 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.828 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.829 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.830 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.830 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.830 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.830 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.830 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.830 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.830 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.830 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.830 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.830 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.830 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.830 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.830 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.831 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.832 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.832 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.832 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.832 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.832 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.832 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.832 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.832 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.832 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.832 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.832 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.832 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.832 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: "min_score": { 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "min_score", 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.833 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": { 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mode", 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "upmap", 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: "crush-compat", 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: "none", 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: "read", 2024-04-26T18:45:33.834 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap", 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap-read" 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Balancer mode", 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: "pool_ids": { 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pool_ids", 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.835 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.836 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap_max_deviation": { 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "upmap_max_deviation", 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.837 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.838 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-26T18:45:33.838 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-26T18:45:33.838 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.838 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.838 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.838 INFO:teuthology.orchestra.run.smithi084.stdout: "upmap_max_optimizations": { 2024-04-26T18:45:33.838 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "upmap_max_optimizations", 2024-04-26T18:45:33.838 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:33.838 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.838 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.838 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.838 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.838 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephadm", 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_down_multiplier": { 2024-04-26T18:45:33.839 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_down_multiplier", 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3.0", 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_refresh_rate": { 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_refresh_rate", 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.840 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "20", 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "agent_starting_port": { 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "agent_starting_port", 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.841 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "4721", 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_ptrace": { 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_ptrace", 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.842 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "autotune_interval": { 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "autotune_interval", 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.843 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "autotune_memory_target_ratio": { 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "autotune_memory_target_ratio", 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0.7", 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.844 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm_log_destination": { 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cephadm_log_destination", 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.845 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: "file", 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: "file,syslog", 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: "syslog" 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: "cgroups_split": { 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cgroups_split", 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.846 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "config_checks_enabled": { 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_checks_enabled", 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.847 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: "config_dashboard": { 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_dashboard", 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.848 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_alertmanager": { 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_alertmanager", 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-26T18:45:33.849 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_base": { 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_base", 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.850 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Container image name, without the tag", 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_elasticsearch": { 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_elasticsearch", 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.851 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "elasticsearch container image", 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_grafana": { 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_grafana", 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.852 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_haproxy": { 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_haproxy", 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "HAproxy container image", 2024-04-26T18:45:33.853 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_agent": { 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_agent", 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger agent container image", 2024-04-26T18:45:33.854 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_collector": { 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_collector", 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger collector container image", 2024-04-26T18:45:33.855 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_jaeger_query": { 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_jaeger_query", 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Jaeger query container image", 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.856 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_keepalived": { 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_keepalived", 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Keepalived container image", 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.857 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.858 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.858 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_loki": { 2024-04-26T18:45:33.858 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_loki", 2024-04-26T18:45:33.858 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.858 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.858 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.858 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-26T18:45:33.858 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.858 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.858 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.858 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Loki container image", 2024-04-26T18:45:33.858 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.858 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_node_exporter": { 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_node_exporter", 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.859 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_nvmeof": { 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_nvmeof", 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Nvme-of container image", 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.860 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_prometheus": { 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_prometheus", 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Prometheus container image", 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.861 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_promtail": { 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_promtail", 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Promtail container image", 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_samba": { 2024-04-26T18:45:33.862 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_samba", 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Samba/SMB container image", 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "container_image_snmp_gateway": { 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_image_snmp_gateway", 2024-04-26T18:45:33.863 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "SNMP Gateway container image", 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "container_init": { 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "container_init", 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.864 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Run podman/docker with `--init`", 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "daemon_cache_timeout": { 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "daemon_cache_timeout", 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.865 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: "default_cephadm_command_timeout": { 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "default_cephadm_command_timeout", 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.866 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "900", 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "default_registry": { 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "default_registry", 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "docker.io", 2024-04-26T18:45:33.867 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "device_cache_timeout": { 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_cache_timeout", 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1800", 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.868 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache device inventory", 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "device_enhanced_scan": { 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_enhanced_scan", 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.869 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: "facts_cache_timeout": { 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "facts_cache_timeout", 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "seconds to cache host facts data", 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.870 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: "grafana_dashboards_path": { 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "grafana_dashboards_path", 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.871 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: "host_check_interval": { 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "host_check_interval", 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to perform a host check", 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.872 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "hw_monitoring": { 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "hw_monitoring", 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.873 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: "inventory_list_all": { 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "inventory_list_all", 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.874 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.875 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: "log_refresh_metadata": { 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_refresh_metadata", 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.876 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.877 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.878 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-26T18:45:33.878 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.878 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.878 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.878 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.878 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.878 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.878 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.878 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.879 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.880 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.881 INFO:teuthology.orchestra.run.smithi084.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "*", 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.882 INFO:teuthology.orchestra.run.smithi084.stdout: "max_count_per_host": { 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_count_per_host", 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "max number of daemons per service per host", 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "max_osd_draining_count": { 2024-04-26T18:45:33.883 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_osd_draining_count", 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "migration_current": { 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "migration_current", 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.884 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "internal - do not modify", 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "mode": { 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mode", 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.885 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "root", 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm-package", 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: "root" 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "mode for remote execution of cephadm", 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: "oob_default_addr": { 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "oob_default_addr", 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.886 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "169.254.1.1", 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "prometheus_alerts_path": { 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prometheus_alerts_path", 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.887 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_insecure": { 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_insecure", 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.888 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_password": { 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_password", 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.889 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_url": { 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_url", 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.890 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.891 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.891 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-26T18:45:33.891 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.891 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.891 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.891 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.891 INFO:teuthology.orchestra.run.smithi084.stdout: "registry_username": { 2024-04-26T18:45:33.891 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "registry_username", 2024-04-26T18:45:33.891 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.891 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.891 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.891 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.891 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: "secure_monitoring_stack": { 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "secure_monitoring_stack", 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.892 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: "service_discovery_port": { 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "service_discovery_port", 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8765", 2024-04-26T18:45:33.893 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "cephadm service discovery port", 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: "ssh_config_file": { 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssh_config_file", 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.894 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "use_agent": { 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "use_agent", 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.895 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "use_repo_digest": { 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "use_repo_digest", 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.896 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_failed_host_check": { 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_failed_host_check", 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if the host check fails", 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.897 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_stray_daemons": { 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_stray_daemons", 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.898 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_on_stray_hosts": { 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_on_stray_hosts", 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.899 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.900 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.900 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.900 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crash", 2024-04-26T18:45:33.900 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.900 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.900 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.900 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.900 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.900 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.900 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.900 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.900 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.900 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.901 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.902 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.903 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.904 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "retain_interval": { 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "retain_interval", 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "31536000", 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.905 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_recent_interval": { 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_recent_interval", 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1209600", 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.906 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "dashboard", 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-26T18:45:33.907 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: "ALERTMANAGER_API_HOST": { 2024-04-26T18:45:33.908 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-26T18:45:33.909 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "AUDIT_API_ENABLED": { 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "AUDIT_API_ENABLED", 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.910 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.911 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "ENABLE_BROWSABLE_API": { 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.912 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.913 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.914 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.915 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.916 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.917 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.918 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.919 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.919 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.919 INFO:teuthology.orchestra.run.smithi084.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-26T18:45:33.919 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-26T18:45:33.919 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.919 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.919 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.919 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.919 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.919 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.919 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.919 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.919 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.920 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_PASSWORD": { 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.921 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.922 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_URL": { 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_URL", 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.923 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_API_USERNAME": { 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.924 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-26T18:45:33.925 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-26T18:45:33.926 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.927 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.928 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "[]", 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "MULTICLUSTER_CONFIG": { 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.929 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "{}", 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "PROMETHEUS_API_HOST": { 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.930 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.931 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.932 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.933 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.934 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.935 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.936 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.937 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_ENABLED": { 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.938 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-26T18:45:33.939 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.940 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.941 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.941 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.941 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.941 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.941 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.941 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.941 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-26T18:45:33.941 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-26T18:45:33.942 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.942 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.942 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.942 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.942 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.942 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.942 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.942 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.942 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.942 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.942 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.942 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.942 INFO:teuthology.orchestra.run.smithi084.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8", 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-26T18:45:33.943 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "45", 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_ACCESS_KEY": { 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.944 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.945 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "admin", 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_SECRET_KEY": { 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.946 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "RGW_API_SSL_VERIFY": { 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.947 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "UNSAFE_TLS_v1_2": { 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.948 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.949 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.950 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.951 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: "cross_origin_url": { 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cross_origin_url", 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.952 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: "crt_file": { 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "crt_file", 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.953 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "debug": { 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "debug", 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Enable/disable debug options", 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.954 INFO:teuthology.orchestra.run.smithi084.stdout: "jwt_token_ttl": { 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "jwt_token_ttl", 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "28800", 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "key_file": { 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "key_file", 2024-04-26T18:45:33.955 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.956 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.957 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.958 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.959 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.960 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: "motd": { 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "motd", 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.961 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "The message of the day", 2024-04-26T18:45:33.962 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.962 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.962 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.962 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.962 INFO:teuthology.orchestra.run.smithi084.stdout: "redirect_resolve_ip_addr": { 2024-04-26T18:45:33.962 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "redirect_resolve_ip_addr", 2024-04-26T18:45:33.962 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.962 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.962 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.962 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.963 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.963 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.963 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.963 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.963 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.963 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.963 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.963 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.963 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-04-26T18:45:33.963 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-04-26T18:45:33.963 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.963 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.963 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "::", 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.964 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8080", 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl": { 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl", 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.965 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl_server_port": { 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl_server_port", 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.966 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8443", 2024-04-26T18:45:33.967 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.967 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.967 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.967 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.967 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.967 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.967 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.967 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.967 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_behaviour": { 2024-04-26T18:45:33.967 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_behaviour", 2024-04-26T18:45:33.967 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.967 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.967 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "redirect", 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: "redirect" 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_error_status_code": { 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_error_status_code", 2024-04-26T18:45:33.968 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "500", 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "400", 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "599", 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "url_prefix": { 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "url_prefix", 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.969 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.970 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "devicehealth", 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "enable_monitoring": { 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enable_monitoring", 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "monitor device health metrics", 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.971 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.972 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.973 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.974 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.975 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "mark_out_threshold": { 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mark_out_threshold", 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "2419200", 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.976 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: "pool_name": { 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pool_name", 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "device_health_metrics", 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.977 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: "retention_period": { 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "retention_period", 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15552000", 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.978 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.979 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.979 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long to retain device health metrics", 2024-04-26T18:45:33.979 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.979 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.979 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.979 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.979 INFO:teuthology.orchestra.run.smithi084.stdout: "scrape_frequency": { 2024-04-26T18:45:33.979 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "scrape_frequency", 2024-04-26T18:45:33.979 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.979 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.979 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.979 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "86400", 2024-04-26T18:45:33.979 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "self_heal": { 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "self_heal", 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.980 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.981 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how frequently to wake up and check device health", 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "warn_threshold": { 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "warn_threshold", 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "7257600", 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-26T18:45:33.982 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "diskprediction_local", 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.983 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.984 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.985 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.986 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.987 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "predict_interval": { 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "predict_interval", 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "86400", 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.988 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "predictor_model": { 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "predictor_model", 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "prophetstor", 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.989 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "600", 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.990 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.991 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.991 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:33.991 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.991 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:33.991 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "influx", 2024-04-26T18:45:33.991 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": false, 2024-04-26T18:45:33.991 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "influxdb python module not found", 2024-04-26T18:45:33.991 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:33.991 INFO:teuthology.orchestra.run.smithi084.stdout: "batch_size": { 2024-04-26T18:45:33.991 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "batch_size", 2024-04-26T18:45:33.992 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:33.992 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.992 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.992 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5000", 2024-04-26T18:45:33.992 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.992 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.992 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.992 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-26T18:45:33.992 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.992 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.992 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.992 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.992 INFO:teuthology.orchestra.run.smithi084.stdout: "database": { 2024-04-26T18:45:33.993 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "database", 2024-04-26T18:45:33.993 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.993 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.993 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.993 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "ceph", 2024-04-26T18:45:33.993 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.993 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.993 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.993 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-26T18:45:33.993 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.993 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.993 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.994 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.994 INFO:teuthology.orchestra.run.smithi084.stdout: "hostname": { 2024-04-26T18:45:33.994 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "hostname", 2024-04-26T18:45:33.994 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.994 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.994 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.994 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.994 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.994 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.994 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.994 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB server hostname", 2024-04-26T18:45:33.995 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.995 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.995 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.995 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.995 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:33.995 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:33.995 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:33.995 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.995 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:33.995 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "30", 2024-04-26T18:45:33.995 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "5", 2024-04-26T18:45:33.996 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.996 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.996 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-26T18:45:33.996 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.996 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.996 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.996 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.996 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:33.996 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:33.996 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.996 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.996 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.997 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:33.997 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.997 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.997 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:33.997 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:33.997 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:33.997 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:33.997 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:33.997 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:33.998 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:33.998 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:33.998 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.998 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.998 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.998 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.998 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.998 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:33.998 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:33.998 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:33.998 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.998 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.998 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:33.999 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.000 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.001 INFO:teuthology.orchestra.run.smithi084.stdout: "password": { 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "password", 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "password of InfluxDB server user", 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.002 INFO:teuthology.orchestra.run.smithi084.stdout: "port": { 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "port", 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "8086", 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "InfluxDB server port", 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "ssl": { 2024-04-26T18:45:34.003 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ssl", 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "false", 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "threads": { 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "threads", 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.004 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "32", 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "username": { 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "username", 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.005 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "username of InfluxDB server user", 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "verify_ssl": { 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "verify_ssl", 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "true", 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.006 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "insights", 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.007 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.008 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.009 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.010 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.011 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.012 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.012 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.012 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.012 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.012 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.012 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.012 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.012 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "iostat", 2024-04-26T18:45:34.012 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.012 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.012 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.012 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.012 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.013 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.014 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.014 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.014 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.014 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.014 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.014 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.014 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.014 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.014 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.014 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.014 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.015 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.016 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.017 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "k8sevents", 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_event_retention_days": { 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "ceph_event_retention_days", 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "7", 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.018 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.019 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-26T18:45:34.019 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.019 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.019 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.019 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.019 INFO:teuthology.orchestra.run.smithi084.stdout: "config_check_secs": { 2024-04-26T18:45:34.019 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "config_check_secs", 2024-04-26T18:45:34.019 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.019 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.019 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.019 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:34.019 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "10", 2024-04-26T18:45:34.019 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.020 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.020 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-26T18:45:34.020 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.020 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.020 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.020 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.020 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.020 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.020 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.020 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.020 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.020 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.021 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.022 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.023 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.024 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "localpool", 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: "failure_domain": { 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "failure_domain", 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.025 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "host", 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "failure domain for any created local pool", 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.026 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.027 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.028 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.029 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.030 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: "min_size": { 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "min_size", 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.031 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default min_size for any created local pool", 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: "num_rep": { 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "num_rep", 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3", 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.032 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default replica count for any created local pool", 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_num": { 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pg_num", 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "128", 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "default pg_num for any created local pool", 2024-04-26T18:45:34.033 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "prefix": { 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prefix", 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "name prefix for any created local pool", 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.034 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "subtree": { 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "subtree", 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "rack", 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.035 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mds_autoscaler", 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.036 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.037 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.038 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.039 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.040 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mirroring", 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.041 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.042 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.043 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.044 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.045 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "nfs", 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.046 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.047 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.048 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.049 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.050 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "orchestrator", 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.051 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "fail_fs": { 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "fail_fs", 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.052 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.053 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.054 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.055 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.056 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.057 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator": { 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "orchestrator", 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: "cephadm", 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: "rook", 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: "test_orchestrator" 2024-04-26T18:45:34.058 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Orchestrator backend", 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "osd_perf_query", 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.059 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.060 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.061 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.062 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.063 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "osd_support", 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.064 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.065 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.066 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.067 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.068 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "pg_autoscaler", 2024-04-26T18:45:34.069 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.070 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.070 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.070 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.070 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.070 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.070 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.070 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.070 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.070 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.070 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.070 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.071 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.071 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.071 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.071 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.071 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.071 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.071 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.072 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.072 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.072 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.072 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.072 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.072 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.072 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.073 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.073 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.073 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.073 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.073 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.073 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.073 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.074 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.074 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.074 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.074 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.074 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.074 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.074 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.075 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.075 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.075 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.075 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.075 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.075 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.075 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.075 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.075 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.076 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.077 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "threshold": { 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "threshold", 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "3.0", 2024-04-26T18:45:34.078 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1.0", 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "scaling threshold", 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "progress", 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.079 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_pg_recovery_event": { 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_pg_recovery_event", 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "enabled": { 2024-04-26T18:45:34.080 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enabled", 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.081 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.082 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.083 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.084 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.085 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.086 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.086 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.086 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.086 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.086 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.086 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.086 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.086 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.086 INFO:teuthology.orchestra.run.smithi084.stdout: "max_completed_events": { 2024-04-26T18:45:34.086 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_completed_events", 2024-04-26T18:45:34.086 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.086 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "50", 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "number of past completed events to remember", 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: "sleep_interval": { 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "sleep_interval", 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:34.087 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "5", 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "how long the module is going to sleep", 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "prometheus", 2024-04-26T18:45:34.088 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "cache": { 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "cache", 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.089 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "exclude_perf_counters": { 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "exclude_perf_counters", 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.090 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.091 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.092 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.092 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.092 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.092 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.092 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.092 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.092 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.092 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.092 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.092 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.092 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.092 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.092 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.093 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.093 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.093 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.093 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.093 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.093 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.093 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.093 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.093 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.093 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.093 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.093 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.093 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.094 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.095 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.095 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.095 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.095 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.095 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.095 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.095 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.095 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.095 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.095 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_stats_pools": { 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_stats_pools", 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.096 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "300", 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.097 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "scrape_interval": { 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "scrape_interval", 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15.0", 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.098 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "::", 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-26T18:45:34.099 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "9283", 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.100 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "stale_cache_strategy": { 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "stale_cache_strategy", 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "log", 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.101 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_behaviour": { 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_behaviour", 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "default", 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: "default", 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: "error" 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.102 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "standby_error_status_code": { 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "standby_error_status_code", 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "500", 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "400", 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "599", 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.103 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_support", 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.104 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.105 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.106 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.107 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.108 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "max_concurrent_snap_create": { 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_concurrent_snap_create", 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10", 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.109 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: "mirror_snapshot_schedule": { 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "mirror_snapshot_schedule", 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.110 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "trash_purge_schedule": { 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "trash_purge_schedule", 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.111 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "restful", 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: "enable_auth": { 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enable_auth", 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.112 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "key_file": { 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "key_file", 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.113 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.114 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.115 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.116 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.117 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.117 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.117 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.117 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.117 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.117 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.117 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.117 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.117 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.117 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.117 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.117 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.117 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.118 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: "server_addr": { 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_addr", 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.119 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "server_port": { 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "server_port", 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.120 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rgw", 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.121 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.122 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.123 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.124 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.125 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rook", 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.126 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.127 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.128 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.129 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.130 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "storage_class": { 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "storage_class", 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.131 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "local", 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "selftest", 2024-04-26T18:45:34.132 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.133 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.134 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.135 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.136 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "roption1": { 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "roption1", 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.137 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: "roption2": { 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "roption2", 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.138 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "xyz", 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption1": { 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption1", 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.139 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption2": { 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption2", 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.140 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption3": { 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption3", 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "float", 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.141 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption4": { 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption4", 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.142 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.143 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.143 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.143 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.143 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.143 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption5": { 2024-04-26T18:45:34.143 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption5", 2024-04-26T18:45:34.143 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.143 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.143 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.143 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.143 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.143 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.145 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.145 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.145 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.145 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.145 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.145 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.145 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption6": { 2024-04-26T18:45:34.145 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption6", 2024-04-26T18:45:34.145 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.145 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.145 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.146 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:34.146 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.146 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.146 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.146 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.146 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.146 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.146 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.146 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.146 INFO:teuthology.orchestra.run.smithi084.stdout: "rwoption7": { 2024-04-26T18:45:34.146 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rwoption7", 2024-04-26T18:45:34.146 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.146 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "1", 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "42", 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "testkey": { 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testkey", 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.147 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.148 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.148 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.148 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.148 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.148 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.148 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.148 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.148 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.148 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.148 INFO:teuthology.orchestra.run.smithi084.stdout: "testlkey": { 2024-04-26T18:45:34.148 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testlkey", 2024-04-26T18:45:34.148 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.148 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: "testnewline": { 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "testnewline", 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.149 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snap_schedule", 2024-04-26T18:45:34.150 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "allow_m_granularity": { 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "allow_m_granularity", 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "allow minute scheduled snapshots", 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.151 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: "dump_on_update": { 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "dump_on_update", 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "dump database to debug log on update", 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.152 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.153 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.154 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.155 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.156 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.157 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "stats", 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.158 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.159 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.159 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.159 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.159 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.159 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.159 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.159 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.159 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.159 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.159 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.160 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.161 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.162 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "status", 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.163 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.164 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.165 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.166 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.167 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.168 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.168 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.168 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.168 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.168 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.168 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.168 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.168 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.168 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.168 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.168 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.168 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "telegraf", 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "address": { 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "address", 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.169 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.170 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.170 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.170 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.170 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.170 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:34.170 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:34.170 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:34.170 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.170 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.170 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "15", 2024-04-26T18:45:34.170 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.170 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.170 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.171 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.171 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.171 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.171 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.171 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.171 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.171 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.171 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.171 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.171 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.171 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.171 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.171 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.172 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.173 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.174 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.175 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "telemetry", 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_basic": { 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_basic", 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:34.176 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_crash": { 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_crash", 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.177 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.178 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.178 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-26T18:45:34.178 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.178 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.178 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.178 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.178 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_device": { 2024-04-26T18:45:34.178 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_device", 2024-04-26T18:45:34.178 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.178 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.178 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.178 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:34.178 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_ident": { 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_ident", 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.179 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: "channel_perf": { 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "channel_perf", 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.180 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "contact": { 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "contact", 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.181 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: "description": { 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "description", 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.182 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "device_url": { 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "device_url", 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.183 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.184 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.184 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.184 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.184 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.184 INFO:teuthology.orchestra.run.smithi084.stdout: "enabled": { 2024-04-26T18:45:34.184 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "enabled", 2024-04-26T18:45:34.184 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.184 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.184 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.184 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.184 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.184 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.184 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.185 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.185 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.185 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.185 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.185 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.185 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:34.185 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:34.185 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.185 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.185 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.185 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "24", 2024-04-26T18:45:34.185 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "8", 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "last_opt_revision": { 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "last_opt_revision", 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "1", 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.186 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: "leaderboard": { 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "leaderboard", 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.187 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "leaderboard_description": { 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "leaderboard_description", 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.188 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.189 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.190 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.191 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.192 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "organization": { 2024-04-26T18:45:34.193 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "organization", 2024-04-26T18:45:34.194 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.194 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.194 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.194 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.194 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.194 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.194 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.194 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.194 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.194 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.194 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.194 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.194 INFO:teuthology.orchestra.run.smithi084.stdout: "proxy": { 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "proxy", 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "url": { 2024-04-26T18:45:34.195 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "url", 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.196 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.197 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "test_orchestrator", 2024-04-26T18:45:34.197 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.197 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.197 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.197 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.197 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.197 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.197 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.197 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.197 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.197 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.198 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.199 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.200 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.201 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "volumes", 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.202 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.203 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.204 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.205 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.206 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "max_concurrent_clones": { 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "max_concurrent_clones", 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "4", 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.207 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: "periodic_async_work": { 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "periodic_async_work", 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.208 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Periodically check for async work", 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "snapshot_clone_delay": { 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snapshot_clone_delay", 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "0", 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.209 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: "snapshot_clone_no_wait": { 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snapshot_clone_no_wait", 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "True", 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.210 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix", 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: "can_run": true, 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: "error_string": "", 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: "module_options": { 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: "discovery_interval": { 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "discovery_interval", 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "uint", 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.211 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "100", 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: "identifier": { 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "identifier", 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.212 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "interval": { 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "interval", 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "secs", 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "60", 2024-04-26T18:45:34.213 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: "log_level": { 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_level", 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.214 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.215 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster": { 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster", 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_cluster_level": { 2024-04-26T18:45:34.216 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_cluster_level", 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "info", 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [ 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "", 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "critical", 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "debug", 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "error", 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "info", 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "warning" 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.217 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "log_to_file": { 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "log_to_file", 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "bool", 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 1, 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "False", 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.218 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_host": { 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_host", 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "", 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.219 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_port": { 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_port", 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "int", 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "10051", 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.220 INFO:teuthology.orchestra.run.smithi084.stdout: "zabbix_sender": { 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "zabbix_sender", 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "str", 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: "level": "advanced", 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: "flags": 0, 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: "min": "", 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: "max": "", 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: "enum_allowed": [], 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: "desc": "", 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: "long_desc": "", 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: "tags": [], 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: "see_also": [] 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.221 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: "services": { 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: "dashboard": "https://172.21.15.84:8443/" 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: "always_on_modules": { 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: "octopus": [ 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: "balancer", 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: "crash", 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: "devicehealth", 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator", 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_autoscaler", 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: "progress", 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_support", 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: "status", 2024-04-26T18:45:34.222 INFO:teuthology.orchestra.run.smithi084.stdout: "telemetry", 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: "volumes" 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: "pacific": [ 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: "balancer", 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: "crash", 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: "devicehealth", 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator", 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_autoscaler", 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: "progress", 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_support", 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: "status", 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: "telemetry", 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: "volumes" 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.223 INFO:teuthology.orchestra.run.smithi084.stdout: "quincy": [ 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "balancer", 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "crash", 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "devicehealth", 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator", 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_autoscaler", 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "progress", 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_support", 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "status", 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "telemetry", 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "volumes" 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "reef": [ 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "balancer", 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "crash", 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "devicehealth", 2024-04-26T18:45:34.224 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator", 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_autoscaler", 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "progress", 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_support", 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "status", 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "telemetry", 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "volumes" 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: ], 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "squid": [ 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "balancer", 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "crash", 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "devicehealth", 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "orchestrator", 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "pg_autoscaler", 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "progress", 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "rbd_support", 2024-04-26T18:45:34.225 INFO:teuthology.orchestra.run.smithi084.stdout: "status", 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: "telemetry", 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: "volumes" 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: "last_failure_osd_epoch": 94, 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: "active_clients": [ 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "devicehealth", 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 2519367026 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.226 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "libcephsqlite", 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 1698258904 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "rbd_support", 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2024-04-26T18:45:34.227 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 4208795984 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snap_schedule", 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 1296565006 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.228 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "snap_schedule", 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 2490171649 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: "name": "volumes", 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: "addrvec": [ 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: { 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: "type": "v2", 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: "addr": "172.21.15.84:0", 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: "nonce": 3585883176 2024-04-26T18:45:34.229 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.230 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-04-26T18:45:34.230 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:34.230 INFO:teuthology.orchestra.run.smithi084.stdout: ] 2024-04-26T18:45:34.230 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:45:34.231 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:33 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1018809234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T18:45:34.231 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:33 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/24350102' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T18:45:34.231 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:33 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1018809234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T18:45:34.231 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:33 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/24350102' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T18:45:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.978+0000 7f28b076a640 1 -- 172.21.15.84:0/24350102 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f28940a3d20 msgr2=0x7f28940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.978+0000 7f28b076a640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f28940a3d20 0x7f28940a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f28a4005fd0 tx=0x7f28a400b510 comp rx=0 tx=0).stop 2024-04-26T18:45:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.978+0000 7f28b076a640 1 -- 172.21.15.84:0/24350102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a8108900 msgr2=0x7f28a819da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:45:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.978+0000 7f28b076a640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a8108900 0x7f28a819da20 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f28a000eaa0 tx=0x7f28a000ef60 comp rx=0 tx=0).stop 2024-04-26T18:45:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.978+0000 7f28b076a640 1 -- 172.21.15.84:0/24350102 shutdown_connections 2024-04-26T18:45:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.978+0000 7f28b076a640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f28940a3d20 0x7f28940a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.978+0000 7f28b076a640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f28a8108900 0x7f28a819da20 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.978+0000 7f28b076a640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f28a81032b0 0x7f28a81996a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.978+0000 7f28b076a640 1 --2- 172.21.15.84:0/24350102 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f28a81028f0 0x7f28a8199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:45:34.232 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.978+0000 7f28b076a640 1 -- 172.21.15.84:0/24350102 >> 172.21.15.84:0/24350102 conn(0x7f28a8079d30 msgr2=0x7f28a8105530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:45:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.978+0000 7f28b076a640 1 -- 172.21.15.84:0/24350102 shutdown_connections 2024-04-26T18:45:34.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:45:32.978+0000 7f28b076a640 1 -- 172.21.15.84:0/24350102 wait complete. 2024-04-26T18:45:34.233 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:32 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3983638441' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:45:34.233 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:32 smithi084 ceph-mon[29141]: pgmap v554: 97 pgs: 97 active+clean; 4.7 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-04-26T18:45:34.233 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:32 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3841504791' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T18:45:34.233 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:32 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/547027639' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-26T18:45:34.240 DEBUG:tasks.check_counter:Getting stats from a 2024-04-26T18:45:34.241 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-26T18:45:34.241 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:33 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1018809234' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-26T18:45:34.241 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:33 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/24350102' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-26T18:45:34.490 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:45:34.861 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:34 smithi084 ceph-mon[29141]: pgmap v555: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-04-26T18:45:34.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:34 smithi103 ceph-mon[32727]: pgmap v555: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-04-26T18:45:34.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:34 smithi174 ceph-mon[32705]: pgmap v555: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 25 KiB/s wr, 19 op/s 2024-04-26T18:45:36.219 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:45:36.219 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:45:36.219 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 1648, 2024-04-26T18:45:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 1634, 2024-04-26T18:45:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1961104, 2024-04-26T18:45:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 617412, 2024-04-26T18:45:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 5, 2024-04-26T18:45:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 1, 2024-04-26T18:45:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.199859577, 2024-04-26T18:45:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.070804752, 2024-04-26T18:45:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.070710215, 2024-04-26T18:45:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.051280549, 2024-04-26T18:45:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1634, 2024-04-26T18:45:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.080949220, 2024-04-26T18:45:36.220 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000049540 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1539760, 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 218640 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 1174, 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 1434, 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 289284, 2024-04-26T18:45:36.221 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 998636, 2024-04-26T18:45:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 8, 2024-04-26T18:45:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 5, 2024-04-26T18:45:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.128518275, 2024-04-26T18:45:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.044168174, 2024-04-26T18:45:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.028316057, 2024-04-26T18:45:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.044776567, 2024-04-26T18:45:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1431, 2024-04-26T18:45:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.051012238, 2024-04-26T18:45:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000035647 2024-04-26T18:45:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:36.222 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T18:45:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 625872 2024-04-26T18:45:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:45:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 1262, 2024-04-26T18:45:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 1263, 2024-04-26T18:45:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 332413, 2024-04-26T18:45:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 400999, 2024-04-26T18:45:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 6, 2024-04-26T18:45:36.223 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 4, 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.113295369, 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.036031236, 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.030744296, 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.035961878, 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1261, 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.041547296, 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000032947 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.224 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 1, 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-MDSRank": { 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 6043, 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.166685654, 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000027583 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.225 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-PurgeQueue": { 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": { 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: "request": 0, 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: "reply": 0, 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: "reply_latency": { 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.226 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_reply": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "forward": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_complete": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_keys": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_commit": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_split": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_merge": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_top": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_bottom": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pin_tail": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pinned": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_expired": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_with_caps": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": 0, 2024-04-26T18:45:36.227 INFO:teuthology.orchestra.run.smithi084.stdout: "subtrees": 1, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse": 0, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_hit": 0, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_forward": 0, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_discover": 0, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_remote_ino": 0, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_lock": 0, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "load_cent": 0, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "q": 0, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "exported": 0, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "exported_inodes": 0, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "imported": 0, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "imported_inodes": 0, 2024-04-26T18:45:36.228 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_dir_fetch": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_peer_discover": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rfiles": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rbytes": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rsnaps": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_set_tag": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_file_inodes": 0, 2024-04-26T18:45:36.229 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_grant": 0, 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps": 0, 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps_dirty": 0, 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_cap_release": 0, 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "process_request_cap_release": 0 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_cache": { 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update": 0, 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update_receipt": 0, 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_try_discover": 0, 2024-04-26T18:45:36.230 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_send_discover": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_handle_discover": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_delayed": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_created": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_enqueued": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_reintegrated": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_migrated": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_processing": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_started": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_completed": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_path": 0, 2024-04-26T18:45:36.231 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_inode": 0, 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_exportdir": 0, 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_flush": 0, 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragstats": 0, 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_inodestats": 0 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_log": { 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "evlrg": 0, 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "evadd": 0, 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "evex": 0, 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "evtrm": 0, 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "ev": 6, 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "evexg": 0, 2024-04-26T18:45:36.232 INFO:teuthology.orchestra.run.smithi084.stdout: "evexd": 0, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "segadd": 0, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "segex": 0, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "segtrm": 0, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "seg": 2, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "segmjr": 2, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "segexg": 0, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "segexd": 0, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "expos": 4194304, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "wrpos": 4195956, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "rdpos": 4195815, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "jlat": { 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.233 INFO:teuthology.orchestra.run.smithi084.stdout: "replayed": 6 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_mem": { 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "ino": 3, 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "ino+": 3, 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "ino-": 0, 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "dir": 1, 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "dir+": 1, 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "dir-": 0, 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "dn": 0, 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "dn+": 0, 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "dn-": 0, 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "cap": 0, 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "cap+": 0, 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "cap-": 0, 2024-04-26T18:45:36.234 INFO:teuthology.orchestra.run.smithi084.stdout: "rss": 49440, 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "heap": 215416 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_server": { 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_client_request": 0, 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_server_request": 0, 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_request": 0, 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_session": 0, 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_peer_request": 0, 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "req_create_latency": { 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getattr_latency": { 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.235 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getfilelock_latency": { 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: "req_link_latency": { 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookup_latency": { 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.236 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookuphash_latency": { 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupino_latency": { 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupname_latency": { 2024-04-26T18:45:36.237 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupparent_latency": { 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupsnap_latency": { 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lssnap_latency": { 2024-04-26T18:45:36.238 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.239 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.239 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.239 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mkdir_latency": { 2024-04-26T18:45:36.239 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.239 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.239 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.239 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mknod_latency": { 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mksnap_latency": { 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: "req_open_latency": { 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.240 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: "req_readdir_latency": { 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rename_latency": { 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: "req_renamesnap_latency": { 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.241 INFO:teuthology.orchestra.run.smithi084.stdout: "req_snapdiff_latency": { 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmdir_latency": { 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmsnap_latency": { 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.242 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmxattr_latency": { 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setattr_latency": { 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setdirlayout_latency": { 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.243 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setfilelock_latency": { 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setlayout_latency": { 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setxattr_latency": { 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: "req_symlink_latency": { 2024-04-26T18:45:36.244 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: "req_unlink_latency": { 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getvxattr_latency": { 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.245 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_sessions": { 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: "session_count": 0, 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: "session_add": 2, 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: "session_remove": 2, 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_open": 1, 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_stale": 0, 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: "total_load": 0, 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: "average_load": 0, 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: "avg_session_uptime": 1331, 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: "mdthresh_evicted": 0 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:45:36.246 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:45:36.247 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:45:36.248 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 199082, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 60, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 8232, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 172, 2024-04-26T18:45:36.249 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 8512, 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 13, 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: "objecter": { 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: "op_active": 0, 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: "op_laggy": 0, 2024-04-26T18:45:36.250 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send": 3687, 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send_bytes": 21, 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "op_resend": 3, 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "op_reply": 3687, 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "op_latency": { 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3687, 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 3.027947630, 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000821249 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "op_inflight": 0, 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "oplen_avg": { 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3687, 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 3688 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "op": 3687, 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "op_r": 3687, 2024-04-26T18:45:36.251 INFO:teuthology.orchestra.run.smithi084.stdout: "op_w": 0, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "op_rmw": 0, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "op_pg": 0, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_stat": 2423, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_create": 0, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_read": 1262, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_write": 0, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writefull": 0, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writesame": 0, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_append": 0, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_zero": 0, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_truncate": 0, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_delete": 0, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_mapext": 0, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_sparse_read": 0, 2024-04-26T18:45:36.252 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_clonerange": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_getxattr": 1, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_setxattr": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_rmxattr": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_call": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_watch": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_notify": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_other": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_active": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_send": 0, 2024-04-26T18:45:36.253 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_resend": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_ping": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_active": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_send": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_resend": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_active": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_send": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_resend": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_active": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_send": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_resend": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "command_active": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "command_send": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "command_resend": 0, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "map_epoch": 118, 2024-04-26T18:45:36.254 INFO:teuthology.orchestra.run.smithi084.stdout: "map_full": 0, 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "map_inc": 29, 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_sessions": 6, 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_open": 8, 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_close": 2, 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_laggy": 0, 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_wr": 0, 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_rd": 2, 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_del": 0 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "oft": { 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_objs": 0, 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_updates": 0, 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_removes": 0 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.255 INFO:teuthology.orchestra.run.smithi084.stdout: "osd-slow-ops": { 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_ops_count": 0 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "purge_queue": { 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops": 0, 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing": 0, 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_high_water": 0, 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed_ops": 0, 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed": 0, 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_item_in_journal": 0 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 4080, 2024-04-26T18:45:36.256 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 2192589, 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 4080, 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 4080, 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 2192589, 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_bytes": { 2024-04-26T18:45:36.257 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 3687, 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 1657, 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 3687, 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 1657, 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.258 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_ops": { 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 1024, 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 3687, 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 3687, 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 3687, 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 3687, 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-04-26T18:45:36.259 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-04-26T18:45:36.260 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 0, 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 0, 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle-0x55afb92506e0": { 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:36.261 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-04-26T18:45:36.262 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-04-26T18:45:36.262 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 0, 2024-04-26T18:45:36.262 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 0, 2024-04-26T18:45:36.262 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-04-26T18:45:36.262 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:36.262 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:36.262 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:36.262 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:36.262 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:36.262 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:45:36.982 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:36 smithi084 ceph-mon[29141]: pgmap v556: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2024-04-26T18:45:37.013 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-26T18:45:37.014 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-26T18:45:37.014 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-26T18:45:37.014 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-04-26T18:45:37.014 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-26T18:45:37.014 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:45:37.014 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:45:37.014 DEBUG:tasks.check_counter:Getting stats from d 2024-04-26T18:45:37.014 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-26T18:45:37.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:36 smithi103 ceph-mon[32727]: pgmap v556: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2024-04-26T18:45:37.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:36 smithi174 ceph-mon[32705]: pgmap v556: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2024-04-26T18:45:37.258 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:45:38.964 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:45:38.964 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:45:38.964 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 4, 2024-04-26T18:45:38.964 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 4, 2024-04-26T18:45:38.964 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 3040, 2024-04-26T18:45:38.964 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 448, 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 4, 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 0, 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.001413391, 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.000027438, 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.000518896, 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3, 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000636588, 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000212196 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:38.965 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 3, 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 269, 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 480, 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 283408, 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 6, 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 3, 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.021147848, 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.015761045, 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.002093958, 2024-04-26T18:45:38.966 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 266, 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.023406604, 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000087994 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 283408 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:45:38.967 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 360, 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 350, 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1490880, 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 216096, 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 2, 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 1, 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.056966464, 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.022040495, 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.033019923, 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.007094605, 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 349, 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.016727052, 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000047928 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:38.968 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1490880, 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 216096 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 0, 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:45:38.969 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:45:38.970 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:45:38.971 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 148754, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 31, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 0, 2024-04-26T18:45:38.972 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 0, 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 928, 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 2, 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-04-26T18:45:38.973 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 363, 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 1440033, 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 363, 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 363, 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 1440033, 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:38.974 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:38.975 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:45:38.975 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:38 smithi084 ceph-mon[29141]: pgmap v557: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2024-04-26T18:45:39.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:38 smithi103 ceph-mon[32727]: pgmap v557: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2024-04-26T18:45:39.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:38 smithi174 ceph-mon[32705]: pgmap v557: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 7.6 KiB/s wr, 12 op/s 2024-04-26T18:45:39.602 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-26T18:45:39.602 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-26T18:45:39.602 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-26T18:45:39.602 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-26T18:45:39.603 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-26T18:45:39.603 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-26T18:45:39.603 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-26T18:45:39.603 DEBUG:tasks.check_counter:Getting stats from g 2024-04-26T18:45:39.603 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-26T18:45:39.844 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:45:40.984 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:40 smithi084 ceph-mon[29141]: pgmap v558: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-26T18:45:41.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:40 smithi103 ceph-mon[32727]: pgmap v558: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-26T18:45:41.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:40 smithi174 ceph-mon[32705]: pgmap v558: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-04-26T18:45:41.506 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:45:41.506 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:45:41.507 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 1266, 2024-04-26T18:45:41.507 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 1266, 2024-04-26T18:45:41.507 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 427017, 2024-04-26T18:45:41.507 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 401765, 2024-04-26T18:45:41.507 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 7, 2024-04-26T18:45:41.507 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 2, 2024-04-26T18:45:41.507 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.135974088, 2024-04-26T18:45:41.507 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.060509101, 2024-04-26T18:45:41.507 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.027405382, 2024-04-26T18:45:41.507 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.033951861, 2024-04-26T18:45:41.507 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:41.507 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 1265, 2024-04-26T18:45:41.508 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.065475719, 2024-04-26T18:45:41.508 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000051759 2024-04-26T18:45:41.508 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.508 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:41.508 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.508 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.508 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.508 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.508 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T18:45:41.508 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T18:45:41.508 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.508 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:45:41.508 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 2330, 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 2592, 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 594676, 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 1368201, 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 7, 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 5, 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.158392206, 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.052915305, 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.038699364, 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.054541878, 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 2589, 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.081885329, 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000031628 2024-04-26T18:45:41.509 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.510 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:41.510 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.510 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.510 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.510 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.510 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T18:45:41.510 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 627840 2024-04-26T18:45:41.510 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.510 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:45:41.510 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 395, 2024-04-26T18:45:41.510 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 383, 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1538612, 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 220421, 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 5, 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 3, 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.062065512, 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.024028569, 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.036931530, 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.007193678, 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 382, 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.019186349, 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000050226 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.511 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:41.512 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.512 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.512 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.512 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.512 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1537264, 2024-04-26T18:45:41.512 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 219456 2024-04-26T18:45:41.512 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.512 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-04-26T18:45:41.512 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 1, 2024-04-26T18:45:41.512 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-04-26T18:45:41.512 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.512 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-MDSRank": { 2024-04-26T18:45:41.512 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-04-26T18:45:41.513 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-04-26T18:45:41.513 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 6082, 2024-04-26T18:45:41.513 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.139938103, 2024-04-26T18:45:41.513 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000023008 2024-04-26T18:45:41.513 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:41.513 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.513 INFO:teuthology.orchestra.run.smithi084.stdout: "finisher-PurgeQueue": { 2024-04-26T18:45:41.513 INFO:teuthology.orchestra.run.smithi084.stdout: "queue_len": 0, 2024-04-26T18:45:41.513 INFO:teuthology.orchestra.run.smithi084.stdout: "complete_latency": { 2024-04-26T18:45:41.513 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.513 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.514 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.514 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:41.514 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.514 INFO:teuthology.orchestra.run.smithi084.stdout: "mds": { 2024-04-26T18:45:41.514 INFO:teuthology.orchestra.run.smithi084.stdout: "request": 0, 2024-04-26T18:45:41.514 INFO:teuthology.orchestra.run.smithi084.stdout: "reply": 0, 2024-04-26T18:45:41.514 INFO:teuthology.orchestra.run.smithi084.stdout: "reply_latency": { 2024-04-26T18:45:41.514 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.514 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.514 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.514 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.515 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_reply": 0, 2024-04-26T18:45:41.515 INFO:teuthology.orchestra.run.smithi084.stdout: "forward": 0, 2024-04-26T18:45:41.515 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_complete": 0, 2024-04-26T18:45:41.515 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_fetch_keys": 0, 2024-04-26T18:45:41.515 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_commit": 0, 2024-04-26T18:45:41.515 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_split": 0, 2024-04-26T18:45:41.515 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_merge": 0, 2024-04-26T18:45:41.515 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes": 0, 2024-04-26T18:45:41.515 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_top": 0, 2024-04-26T18:45:41.515 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_bottom": 0, 2024-04-26T18:45:41.515 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pin_tail": 0, 2024-04-26T18:45:41.515 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_pinned": 0, 2024-04-26T18:45:41.517 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_expired": 0, 2024-04-26T18:45:41.517 INFO:teuthology.orchestra.run.smithi084.stdout: "inodes_with_caps": 0, 2024-04-26T18:45:41.518 INFO:teuthology.orchestra.run.smithi084.stdout: "caps": 0, 2024-04-26T18:45:41.518 INFO:teuthology.orchestra.run.smithi084.stdout: "subtrees": 1, 2024-04-26T18:45:41.518 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse": 0, 2024-04-26T18:45:41.518 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_hit": 0, 2024-04-26T18:45:41.518 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_forward": 0, 2024-04-26T18:45:41.518 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_discover": 0, 2024-04-26T18:45:41.518 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:45:41.518 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_remote_ino": 0, 2024-04-26T18:45:41.518 INFO:teuthology.orchestra.run.smithi084.stdout: "traverse_lock": 0, 2024-04-26T18:45:41.518 INFO:teuthology.orchestra.run.smithi084.stdout: "load_cent": 0, 2024-04-26T18:45:41.518 INFO:teuthology.orchestra.run.smithi084.stdout: "q": 0, 2024-04-26T18:45:41.518 INFO:teuthology.orchestra.run.smithi084.stdout: "exported": 0, 2024-04-26T18:45:41.519 INFO:teuthology.orchestra.run.smithi084.stdout: "exported_inodes": 0, 2024-04-26T18:45:41.519 INFO:teuthology.orchestra.run.smithi084.stdout: "imported": 0, 2024-04-26T18:45:41.519 INFO:teuthology.orchestra.run.smithi084.stdout: "imported_inodes": 0, 2024-04-26T18:45:41.519 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_dir_fetch": 0, 2024-04-26T18:45:41.519 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:45:41.519 INFO:teuthology.orchestra.run.smithi084.stdout: "openino_peer_discover": 0, 2024-04-26T18:45:41.519 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rfiles": 0, 2024-04-26T18:45:41.519 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rbytes": 0, 2024-04-26T18:45:41.519 INFO:teuthology.orchestra.run.smithi084.stdout: "root_rsnaps": 0, 2024-04-26T18:45:41.519 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:45:41.519 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_set_tag": 0, 2024-04-26T18:45:41.519 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:45:41.520 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:45:41.520 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:45:41.520 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:45:41.520 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:45:41.520 INFO:teuthology.orchestra.run.smithi084.stdout: "scrub_file_inodes": 0, 2024-04-26T18:45:41.520 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:45:41.520 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T18:45:41.520 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_grant": 0, 2024-04-26T18:45:41.520 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T18:45:41.520 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:45:41.520 INFO:teuthology.orchestra.run.smithi084.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:45:41.520 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps": 0, 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_caps_dirty": 0, 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_cap_release": 0, 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: "process_request_cap_release": 0 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_cache": { 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update": 0, 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_update_receipt": 0, 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_try_discover": 0, 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_send_discover": 0, 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: "dir_handle_discover": 0, 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays": 0, 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_delayed": 0, 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:45:41.521 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_created": 0, 2024-04-26T18:45:41.522 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_enqueued": 0, 2024-04-26T18:45:41.522 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_reintegrated": 0, 2024-04-26T18:45:41.522 INFO:teuthology.orchestra.run.smithi084.stdout: "strays_migrated": 0, 2024-04-26T18:45:41.522 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_processing": 0, 2024-04-26T18:45:41.522 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:45:41.522 INFO:teuthology.orchestra.run.smithi084.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:45:41.522 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_started": 0, 2024-04-26T18:45:41.522 INFO:teuthology.orchestra.run.smithi084.stdout: "recovery_completed": 0, 2024-04-26T18:45:41.522 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_path": 0, 2024-04-26T18:45:41.522 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_quiesce_inode": 0, 2024-04-26T18:45:41.522 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:45:41.522 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_exportdir": 0, 2024-04-26T18:45:41.523 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_flush": 0, 2024-04-26T18:45:41.523 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:45:41.523 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_fragstats": 0, 2024-04-26T18:45:41.523 INFO:teuthology.orchestra.run.smithi084.stdout: "ireq_inodestats": 0 2024-04-26T18:45:41.523 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.523 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_log": { 2024-04-26T18:45:41.523 INFO:teuthology.orchestra.run.smithi084.stdout: "evlrg": 0, 2024-04-26T18:45:41.523 INFO:teuthology.orchestra.run.smithi084.stdout: "evadd": 0, 2024-04-26T18:45:41.523 INFO:teuthology.orchestra.run.smithi084.stdout: "evex": 0, 2024-04-26T18:45:41.523 INFO:teuthology.orchestra.run.smithi084.stdout: "evtrm": 0, 2024-04-26T18:45:41.523 INFO:teuthology.orchestra.run.smithi084.stdout: "ev": 5, 2024-04-26T18:45:41.523 INFO:teuthology.orchestra.run.smithi084.stdout: "evexg": 0, 2024-04-26T18:45:41.523 INFO:teuthology.orchestra.run.smithi084.stdout: "evexd": 0, 2024-04-26T18:45:41.524 INFO:teuthology.orchestra.run.smithi084.stdout: "segadd": 0, 2024-04-26T18:45:41.524 INFO:teuthology.orchestra.run.smithi084.stdout: "segex": 0, 2024-04-26T18:45:41.524 INFO:teuthology.orchestra.run.smithi084.stdout: "segtrm": 0, 2024-04-26T18:45:41.524 INFO:teuthology.orchestra.run.smithi084.stdout: "seg": 2, 2024-04-26T18:45:41.524 INFO:teuthology.orchestra.run.smithi084.stdout: "segmjr": 2, 2024-04-26T18:45:41.524 INFO:teuthology.orchestra.run.smithi084.stdout: "segexg": 0, 2024-04-26T18:45:41.524 INFO:teuthology.orchestra.run.smithi084.stdout: "segexd": 0, 2024-04-26T18:45:41.524 INFO:teuthology.orchestra.run.smithi084.stdout: "expos": 4194304, 2024-04-26T18:45:41.524 INFO:teuthology.orchestra.run.smithi084.stdout: "wrpos": 4195732, 2024-04-26T18:45:41.524 INFO:teuthology.orchestra.run.smithi084.stdout: "rdpos": 4195591, 2024-04-26T18:45:41.524 INFO:teuthology.orchestra.run.smithi084.stdout: "jlat": { 2024-04-26T18:45:41.524 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.524 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: "replayed": 5 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_mem": { 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: "ino": 3, 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: "ino+": 3, 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: "ino-": 0, 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: "dir": 1, 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: "dir+": 1, 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: "dir-": 0, 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: "dn": 0, 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: "dn+": 0, 2024-04-26T18:45:41.525 INFO:teuthology.orchestra.run.smithi084.stdout: "dn-": 0, 2024-04-26T18:45:41.526 INFO:teuthology.orchestra.run.smithi084.stdout: "cap": 0, 2024-04-26T18:45:41.526 INFO:teuthology.orchestra.run.smithi084.stdout: "cap+": 0, 2024-04-26T18:45:41.526 INFO:teuthology.orchestra.run.smithi084.stdout: "cap-": 0, 2024-04-26T18:45:41.526 INFO:teuthology.orchestra.run.smithi084.stdout: "rss": 51160, 2024-04-26T18:45:41.526 INFO:teuthology.orchestra.run.smithi084.stdout: "heap": 215416 2024-04-26T18:45:41.526 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.526 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_server": { 2024-04-26T18:45:41.526 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_client_request": 0, 2024-04-26T18:45:41.526 INFO:teuthology.orchestra.run.smithi084.stdout: "dispatch_server_request": 0, 2024-04-26T18:45:41.526 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_request": 0, 2024-04-26T18:45:41.526 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_client_session": 0, 2024-04-26T18:45:41.526 INFO:teuthology.orchestra.run.smithi084.stdout: "handle_peer_request": 0, 2024-04-26T18:45:41.526 INFO:teuthology.orchestra.run.smithi084.stdout: "req_create_latency": { 2024-04-26T18:45:41.527 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.527 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.527 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.527 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.527 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getattr_latency": { 2024-04-26T18:45:41.527 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.527 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.527 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.527 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.527 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getfilelock_latency": { 2024-04-26T18:45:41.528 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.528 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.528 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.528 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.528 INFO:teuthology.orchestra.run.smithi084.stdout: "req_link_latency": { 2024-04-26T18:45:41.528 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.528 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.528 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.528 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.528 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookup_latency": { 2024-04-26T18:45:41.528 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.528 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.529 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.529 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.529 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookuphash_latency": { 2024-04-26T18:45:41.529 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.529 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.529 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.529 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.529 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupino_latency": { 2024-04-26T18:45:41.529 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.529 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.529 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.529 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.529 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupname_latency": { 2024-04-26T18:45:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupparent_latency": { 2024-04-26T18:45:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lookupsnap_latency": { 2024-04-26T18:45:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.530 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.531 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.531 INFO:teuthology.orchestra.run.smithi084.stdout: "req_lssnap_latency": { 2024-04-26T18:45:41.531 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.531 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.531 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.531 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.531 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mkdir_latency": { 2024-04-26T18:45:41.531 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.532 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.532 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.532 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mknod_latency": { 2024-04-26T18:45:41.532 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.532 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.532 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.532 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.532 INFO:teuthology.orchestra.run.smithi084.stdout: "req_mksnap_latency": { 2024-04-26T18:45:41.532 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.532 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.533 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.533 INFO:teuthology.orchestra.run.smithi084.stdout: "req_open_latency": { 2024-04-26T18:45:41.533 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.533 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.533 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.533 INFO:teuthology.orchestra.run.smithi084.stdout: "req_readdir_latency": { 2024-04-26T18:45:41.533 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.533 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.533 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.533 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.533 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rename_latency": { 2024-04-26T18:45:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "req_renamesnap_latency": { 2024-04-26T18:45:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "req_snapdiff_latency": { 2024-04-26T18:45:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.534 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmdir_latency": { 2024-04-26T18:45:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmsnap_latency": { 2024-04-26T18:45:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.535 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: "req_rmxattr_latency": { 2024-04-26T18:45:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setattr_latency": { 2024-04-26T18:45:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setdirlayout_latency": { 2024-04-26T18:45:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.536 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.537 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.537 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setfilelock_latency": { 2024-04-26T18:45:41.537 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.537 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.537 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.537 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.537 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setlayout_latency": { 2024-04-26T18:45:41.537 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.537 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.537 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.538 INFO:teuthology.orchestra.run.smithi084.stdout: "req_setxattr_latency": { 2024-04-26T18:45:41.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.538 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.538 INFO:teuthology.orchestra.run.smithi084.stdout: "req_symlink_latency": { 2024-04-26T18:45:41.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.538 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.538 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.538 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.538 INFO:teuthology.orchestra.run.smithi084.stdout: "req_unlink_latency": { 2024-04-26T18:45:41.539 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.539 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.539 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.539 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:45:41.539 INFO:teuthology.orchestra.run.smithi084.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:45:41.539 INFO:teuthology.orchestra.run.smithi084.stdout: "req_getvxattr_latency": { 2024-04-26T18:45:41.539 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.539 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.539 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.539 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:41.539 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.539 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_sessions": { 2024-04-26T18:45:41.540 INFO:teuthology.orchestra.run.smithi084.stdout: "session_count": 1, 2024-04-26T18:45:41.540 INFO:teuthology.orchestra.run.smithi084.stdout: "session_add": 2, 2024-04-26T18:45:41.540 INFO:teuthology.orchestra.run.smithi084.stdout: "session_remove": 1, 2024-04-26T18:45:41.540 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_open": 2, 2024-04-26T18:45:41.540 INFO:teuthology.orchestra.run.smithi084.stdout: "sessions_stale": 0, 2024-04-26T18:45:41.540 INFO:teuthology.orchestra.run.smithi084.stdout: "total_load": 0, 2024-04-26T18:45:41.540 INFO:teuthology.orchestra.run.smithi084.stdout: "average_load": 0, 2024-04-26T18:45:41.540 INFO:teuthology.orchestra.run.smithi084.stdout: "avg_session_uptime": 3547, 2024-04-26T18:45:41.540 INFO:teuthology.orchestra.run.smithi084.stdout: "mdthresh_evicted": 0 2024-04-26T18:45:41.540 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.540 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-04-26T18:45:41.540 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:45:41.540 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-04-26T18:45:41.541 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:45:41.541 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:45:41.541 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:45:41.541 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:45:41.541 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:45:41.541 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:45:41.541 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:45:41.541 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:45:41.541 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:45:41.541 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:45:41.541 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:45:41.541 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:45:41.541 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:45:41.542 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-04-26T18:45:41.542 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:45:41.542 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-04-26T18:45:41.542 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:45:41.542 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:45:41.542 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:45:41.542 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:45:41.542 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:45:41.542 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:45:41.542 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:45:41.542 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-04-26T18:45:41.542 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:45:41.543 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:45:41.543 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:45:41.543 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-04-26T18:45:41.543 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-04-26T18:45:41.543 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-04-26T18:45:41.543 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:45:41.543 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:45:41.543 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:45:41.543 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:45:41.543 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 199025, 2024-04-26T18:45:41.543 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 58, 2024-04-26T18:45:41.543 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:45:41.544 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-04-26T18:45:41.544 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-04-26T18:45:41.544 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-04-26T18:45:41.544 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:45:41.544 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-04-26T18:45:41.544 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:45:41.544 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-04-26T18:45:41.544 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 8232, 2024-04-26T18:45:41.544 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 172, 2024-04-26T18:45:41.544 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:45:41.544 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:45:41.544 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-04-26T18:45:41.544 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-04-26T18:45:41.545 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 8512, 2024-04-26T18:45:41.545 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 13, 2024-04-26T18:45:41.545 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-04-26T18:45:41.545 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-04-26T18:45:41.545 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-04-26T18:45:41.545 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-04-26T18:45:41.545 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.545 INFO:teuthology.orchestra.run.smithi084.stdout: "objecter": { 2024-04-26T18:45:41.545 INFO:teuthology.orchestra.run.smithi084.stdout: "op_active": 0, 2024-04-26T18:45:41.545 INFO:teuthology.orchestra.run.smithi084.stdout: "op_laggy": 0, 2024-04-26T18:45:41.545 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send": 3593, 2024-04-26T18:45:41.545 INFO:teuthology.orchestra.run.smithi084.stdout: "op_send_bytes": 21, 2024-04-26T18:45:41.545 INFO:teuthology.orchestra.run.smithi084.stdout: "op_resend": 3, 2024-04-26T18:45:41.546 INFO:teuthology.orchestra.run.smithi084.stdout: "op_reply": 3593, 2024-04-26T18:45:41.546 INFO:teuthology.orchestra.run.smithi084.stdout: "op_latency": { 2024-04-26T18:45:41.546 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3593, 2024-04-26T18:45:41.546 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 2.067964085, 2024-04-26T18:45:41.546 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000575553 2024-04-26T18:45:41.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.546 INFO:teuthology.orchestra.run.smithi084.stdout: "op_inflight": 0, 2024-04-26T18:45:41.546 INFO:teuthology.orchestra.run.smithi084.stdout: "oplen_avg": { 2024-04-26T18:45:41.546 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 3593, 2024-04-26T18:45:41.546 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 3594 2024-04-26T18:45:41.546 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.546 INFO:teuthology.orchestra.run.smithi084.stdout: "op": 3593, 2024-04-26T18:45:41.546 INFO:teuthology.orchestra.run.smithi084.stdout: "op_r": 3593, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "op_w": 0, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "op_rmw": 0, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "op_pg": 0, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_stat": 2323, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_create": 0, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_read": 1268, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_write": 0, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writefull": 0, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_writesame": 0, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_append": 0, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_zero": 0, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_truncate": 0, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_delete": 0, 2024-04-26T18:45:41.547 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_mapext": 0, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_sparse_read": 0, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_clonerange": 0, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_getxattr": 1, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_setxattr": 0, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_rmxattr": 0, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_call": 0, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_watch": 0, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_notify": 0, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls": 0, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:45:41.548 INFO:teuthology.orchestra.run.smithi084.stdout: "osdop_other": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_active": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_send": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_resend": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "linger_ping": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_active": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_send": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "poolop_resend": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_active": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_send": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "poolstat_resend": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_active": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_send": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "statfs_resend": 0, 2024-04-26T18:45:41.549 INFO:teuthology.orchestra.run.smithi084.stdout: "command_active": 0, 2024-04-26T18:45:41.550 INFO:teuthology.orchestra.run.smithi084.stdout: "command_send": 0, 2024-04-26T18:45:41.550 INFO:teuthology.orchestra.run.smithi084.stdout: "command_resend": 0, 2024-04-26T18:45:41.550 INFO:teuthology.orchestra.run.smithi084.stdout: "map_epoch": 118, 2024-04-26T18:45:41.550 INFO:teuthology.orchestra.run.smithi084.stdout: "map_full": 0, 2024-04-26T18:45:41.550 INFO:teuthology.orchestra.run.smithi084.stdout: "map_inc": 29, 2024-04-26T18:45:41.550 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_sessions": 6, 2024-04-26T18:45:41.550 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_open": 8, 2024-04-26T18:45:41.550 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_session_close": 2, 2024-04-26T18:45:41.550 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_laggy": 0, 2024-04-26T18:45:41.550 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_wr": 0, 2024-04-26T18:45:41.550 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_rd": 2, 2024-04-26T18:45:41.550 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_del": 0 2024-04-26T18:45:41.550 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.551 INFO:teuthology.orchestra.run.smithi084.stdout: "oft": { 2024-04-26T18:45:41.551 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_objs": 0, 2024-04-26T18:45:41.551 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:45:41.551 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_updates": 0, 2024-04-26T18:45:41.551 INFO:teuthology.orchestra.run.smithi084.stdout: "omap_total_removes": 0 2024-04-26T18:45:41.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.551 INFO:teuthology.orchestra.run.smithi084.stdout: "osd-slow-ops": { 2024-04-26T18:45:41.551 INFO:teuthology.orchestra.run.smithi084.stdout: "slow_ops_count": 0 2024-04-26T18:45:41.551 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.551 INFO:teuthology.orchestra.run.smithi084.stdout: "purge_queue": { 2024-04-26T18:45:41.551 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops": 0, 2024-04-26T18:45:41.551 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T18:45:41.551 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing": 0, 2024-04-26T18:45:41.552 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executing_high_water": 0, 2024-04-26T18:45:41.552 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed_ops": 0, 2024-04-26T18:45:41.552 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_executed": 0, 2024-04-26T18:45:41.552 INFO:teuthology.orchestra.run.smithi084.stdout: "pq_item_in_journal": 0 2024-04-26T18:45:41.552 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.552 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:45:41.552 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-04-26T18:45:41.552 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-04-26T18:45:41.552 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-04-26T18:45:41.552 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 3987, 2024-04-26T18:45:41.552 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 2178359, 2024-04-26T18:45:41.552 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:41.552 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 3987, 2024-04-26T18:45:41.553 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-04-26T18:45:41.553 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-04-26T18:45:41.553 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 3987, 2024-04-26T18:45:41.553 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 2178359, 2024-04-26T18:45:41.553 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-04-26T18:45:41.553 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.553 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.553 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.553 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:41.553 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.553 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_bytes": { 2024-04-26T18:45:41.553 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-04-26T18:45:41.553 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-04-26T18:45:41.554 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-04-26T18:45:41.554 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-04-26T18:45:41.554 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-04-26T18:45:41.554 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:41.554 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:41.554 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 3593, 2024-04-26T18:45:41.554 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 1433, 2024-04-26T18:45:41.554 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 3593, 2024-04-26T18:45:41.554 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 1433, 2024-04-26T18:45:41.554 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-04-26T18:45:41.554 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.554 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.554 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.555 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:41.555 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.555 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-objecter_ops": { 2024-04-26T18:45:41.555 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-04-26T18:45:41.555 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 1024, 2024-04-26T18:45:41.555 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-04-26T18:45:41.555 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-04-26T18:45:41.555 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-04-26T18:45:41.555 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:41.555 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:41.555 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 3593, 2024-04-26T18:45:41.555 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 3593, 2024-04-26T18:45:41.555 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 3593, 2024-04-26T18:45:41.556 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 3593, 2024-04-26T18:45:41.556 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-04-26T18:45:41.556 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.556 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.556 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.556 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:41.556 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.556 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:45:41.556 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-04-26T18:45:41.556 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-04-26T18:45:41.556 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-04-26T18:45:41.556 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-04-26T18:45:41.556 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-04-26T18:45:41.557 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:41.557 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:41.557 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-04-26T18:45:41.557 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-04-26T18:45:41.557 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 0, 2024-04-26T18:45:41.557 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 0, 2024-04-26T18:45:41.557 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-04-26T18:45:41.557 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.557 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.557 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.557 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:41.557 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:41.557 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-write_buf_throttle-0x557ec3eb06e0": { 2024-04-26T18:45:41.558 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-04-26T18:45:41.558 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 3758096384, 2024-04-26T18:45:41.558 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-04-26T18:45:41.558 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 0, 2024-04-26T18:45:41.558 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 0, 2024-04-26T18:45:41.558 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:41.558 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:41.558 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-04-26T18:45:41.558 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-04-26T18:45:41.558 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 0, 2024-04-26T18:45:41.558 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 0, 2024-04-26T18:45:41.558 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-04-26T18:45:41.558 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:41.559 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:41.559 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:41.559 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:41.559 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:41.559 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:45:42.339 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-26T18:45:42.340 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-26T18:45:42.340 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-26T18:45:42.340 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-26T18:45:42.340 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-26T18:45:42.340 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:45:42.340 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:45:42.340 DEBUG:tasks.check_counter:Getting stats from j 2024-04-26T18:45:42.340 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-26T18:45:42.598 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:45:43.009 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:42 smithi084 ceph-mon[29141]: pgmap v559: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-26T18:45:43.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:42 smithi103 ceph-mon[32727]: pgmap v559: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-26T18:45:43.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:42 smithi174 ceph-mon[32705]: pgmap v559: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.2 KiB/s wr, 12 op/s 2024-04-26T18:45:44.451 INFO:teuthology.orchestra.run.smithi084.stdout:{ 2024-04-26T18:45:44.451 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 0, 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 2, 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 0, 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 0, 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 4, 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 0, 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.001079995, 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.000000000, 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.000236095, 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:44.452 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 0 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 3, 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 271, 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 480, 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 283760, 2024-04-26T18:45:44.453 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 6, 2024-04-26T18:45:44.454 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 3, 2024-04-26T18:45:44.454 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.022343286, 2024-04-26T18:45:44.454 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.017123920, 2024-04-26T18:45:44.454 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.001698407, 2024-04-26T18:45:44.454 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-26T18:45:44.454 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:44.454 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 268, 2024-04-26T18:45:44.454 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.022204387, 2024-04-26T18:45:44.454 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000082852 2024-04-26T18:45:44.454 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:44.454 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:44.454 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 283760 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_messages": 367, 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages": 355, 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_bytes": 1494736, 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_bytes": 218464, 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_created_connections": 2, 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_active_connections": 1, 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_total_time": 0.062281661, 2024-04-26T18:45:44.455 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_send_time": 0.025555426, 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_recv_time": 0.035237900, 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_running_fast_dispatch_time": 0.007328263, 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 355, 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.020449248, 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000057603 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_recv_encrypted_bytes": 1494736, 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: "msgr_send_encrypted_bytes": 218464 2024-04-26T18:45:44.456 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:44.457 INFO:teuthology.orchestra.run.smithi084.stdout: "cct": { 2024-04-26T18:45:44.457 INFO:teuthology.orchestra.run.smithi084.stdout: "total_workers": 0, 2024-04-26T18:45:44.457 INFO:teuthology.orchestra.run.smithi084.stdout: "unhealthy_workers": 0 2024-04-26T18:45:44.457 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:44.457 INFO:teuthology.orchestra.run.smithi084.stdout: "mempool": { 2024-04-26T18:45:44.457 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:45:44.457 INFO:teuthology.orchestra.run.smithi084.stdout: "bloom_filter_items": 0, 2024-04-26T18:45:44.457 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:45:44.457 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:45:44.457 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:45:44.457 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:45:44.457 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_extent_items": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_blob_items": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:45:44.458 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_txc_items": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluestore_writing_items": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_bytes": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_items": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:45:44.459 INFO:teuthology.orchestra.run.smithi084.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_bytes": 148771, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_anon_items": 32, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "buffer_meta_items": 0, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_bytes": 0, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_items": 0, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_mapbl_items": 0, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osd_pglog_items": 0, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_bytes": 0, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_items": 0, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:45:44.460 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_bytes": 0, 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "pgmap_items": 0, 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_bytes": 928, 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "mds_co_items": 2, 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_bytes": 0, 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_1_items": 0, 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_bytes": 0, 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "unittest_2_items": 0 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: }, 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "val": 0, 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "max": 104857600, 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "get_started": 0, 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "get": 366, 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "get_sum": 1440423, 2024-04-26T18:45:44.461 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:44.462 INFO:teuthology.orchestra.run.smithi084.stdout: "get_or_fail_success": 366, 2024-04-26T18:45:44.462 INFO:teuthology.orchestra.run.smithi084.stdout: "take": 0, 2024-04-26T18:45:44.462 INFO:teuthology.orchestra.run.smithi084.stdout: "take_sum": 0, 2024-04-26T18:45:44.462 INFO:teuthology.orchestra.run.smithi084.stdout: "put": 366, 2024-04-26T18:45:44.462 INFO:teuthology.orchestra.run.smithi084.stdout: "put_sum": 1440423, 2024-04-26T18:45:44.462 INFO:teuthology.orchestra.run.smithi084.stdout: "wait": { 2024-04-26T18:45:44.462 INFO:teuthology.orchestra.run.smithi084.stdout: "avgcount": 0, 2024-04-26T18:45:44.462 INFO:teuthology.orchestra.run.smithi084.stdout: "sum": 0.000000000, 2024-04-26T18:45:44.462 INFO:teuthology.orchestra.run.smithi084.stdout: "avgtime": 0.000000000 2024-04-26T18:45:44.462 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:44.462 INFO:teuthology.orchestra.run.smithi084.stdout: } 2024-04-26T18:45:44.462 INFO:teuthology.orchestra.run.smithi084.stdout:} 2024-04-26T18:45:45.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:44 smithi084 ceph-mon[29141]: pgmap v560: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 KiB/s wr, 16 op/s 2024-04-26T18:45:45.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:44 smithi103 ceph-mon[32727]: pgmap v560: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 KiB/s wr, 16 op/s 2024-04-26T18:45:45.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:44 smithi174 ceph-mon[32705]: pgmap v560: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.2 KiB/s wr, 16 op/s 2024-04-26T18:45:45.226 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-26T18:45:45.226 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-26T18:45:45.226 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-26T18:45:45.226 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-26T18:45:45.226 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-26T18:45:45.226 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-26T18:45:45.226 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-26T18:45:45.226 DEBUG:tasks.check_counter:Getting stats from b 2024-04-26T18:45:45.227 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-26T18:45:45.475 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:45:46.977 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:46 smithi103 ceph-mon[32727]: pgmap v561: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-04-26T18:45:47.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:46 smithi084 ceph-mon[29141]: pgmap v561: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-04-26T18:45:47.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:46 smithi174 ceph-mon[32705]: pgmap v561: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-04-26T18:45:47.395 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-26T18:45:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:45:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 3694, 2024-04-26T18:45:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 4399, 2024-04-26T18:45:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2515109, 2024-04-26T18:45:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 8712869, 2024-04-26T18:45:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 228, 2024-04-26T18:45:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 7, 2024-04-26T18:45:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.495231531, 2024-04-26T18:45:47.395 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.174555472, 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.227738275, 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.077582534, 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4397, 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.501586411, 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000114074 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4078, 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001048112, 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000257 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1668272, 2024-04-26T18:45:47.396 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 735136 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 5342, 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 7435, 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1671414, 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 11132824, 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 13, 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 6, 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.634439117, 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.239747932, 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.275307867, 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.127562263, 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 7433, 2024-04-26T18:45:47.397 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.353626912, 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000047575 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 8878, 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.002162498, 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000243 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1041856, 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 1729696 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 6640, 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 6861, 2024-04-26T18:45:47.398 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2130356, 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 15614039, 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 10, 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 7, 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.712571182, 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.259369808, 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.255762049, 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.188429436, 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 6860, 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.279061398, 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000040679 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 8853, 2024-04-26T18:45:47.399 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.002239087, 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000252 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 617568, 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 186688 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 6636, 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 2.021216804, 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000304583 2024-04-26T18:45:47.400 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 15, 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000319242, 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000021282 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 880, 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 880, 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-04-26T18:45:47.401 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 880, 2024-04-26T18:45:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 38.581514888, 2024-04-26T18:45:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.043842630 2024-04-26T18:45:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-04-26T18:45:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-04-26T18:45:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-04-26T18:45:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-04-26T18:45:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 486, 2024-04-26T18:45:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-04-26T18:45:47.402 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 47, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 17, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 30, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 22, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 3, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 3, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 2, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 963, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 950, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-04-26T18:45:47.403 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 4, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 234, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 6, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:45:47.404 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 661, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 649, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 599, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 779, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 718, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 2, 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 629 2024-04-26T18:45:47.405 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 23, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 49, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 2, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 3, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 1, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:45:47.406 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 12, 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 164, 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-04-26T18:45:47.407 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 2704, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 2664, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 2664, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 40, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 51, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 86, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 85, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 85, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 1, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 1, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 1, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 28461248, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 28868856, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 0, 2024-04-26T18:45:47.408 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2605, 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 5.610236906, 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.002153641 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 0 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 40, 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 41, 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 1, 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 20, 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 20, 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 0, 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 47, 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 48, 2024-04-26T18:45:47.409 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 1, 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 3, 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 19, 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 16, 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 60176, 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 231800 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 2164, 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 285, 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 880, 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 45, 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 570, 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-04-26T18:45:47.410 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 15, 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.015071631, 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001004775 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 188, 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.037835091, 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000201250 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.411 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4, 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001581884, 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000395471 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-04-26T18:45:47.412 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000263856, 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000263856 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.413 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-04-26T18:45:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 18, 2024-04-26T18:45:47.414 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.003009503, 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000167194 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 8, 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 34.966974057, 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 4.370871757 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 18, 2024-04-26T18:45:47.415 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.090312571, 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.005017365 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000225472, 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000225472 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.005336148, 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.002668074 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2024-04-26T18:45:47.416 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.002312204, 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001156102 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.015774070, 2024-04-26T18:45:47.417 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.007887035 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 12, 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.054715207, 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.004559600 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 600, 2024-04-26T18:45:47.418 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 3.365952637, 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.005609921 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.419 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 8, 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.019848392, 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.002481049 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.002302165, 2024-04-26T18:45:47.420 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.002302165 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 2, 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 6, 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 4, 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 2, 2024-04-26T18:45:47.421 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 84, 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 42, 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 1065, 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:45:47.422 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:45:47.423 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 345030, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 129, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:45:47.424 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-04-26T18:45:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-04-26T18:45:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-04-26T18:45:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:45:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-04-26T18:45:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:45:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-04-26T18:45:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8232, 2024-04-26T18:45:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 172, 2024-04-26T18:45:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:45:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:45:47.425 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 189867, 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 1450, 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 4684, 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 26390751, 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 31, 2024-04-26T18:45:47.426 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 4684, 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4684, 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 28.659451001, 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.006118584 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4684, 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 5691 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 4684, 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 1, 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 4683, 2024-04-26T18:45:47.427 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 466, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 25, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 1, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 2606, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 273, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 523, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-04-26T18:45:47.428 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 50, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 599, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-04-26T18:45:47.429 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-04-26T18:45:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-04-26T18:45:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-04-26T18:45:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-04-26T18:45:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-04-26T18:45:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-04-26T18:45:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-04-26T18:45:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-04-26T18:45:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-04-26T18:45:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-04-26T18:45:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-04-26T18:45:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 118, 2024-04-26T18:45:47.430 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-04-26T18:45:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 29, 2024-04-26T18:45:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 12, 2024-04-26T18:45:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 13, 2024-04-26T18:45:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 1, 2024-04-26T18:45:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-04-26T18:45:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 1134, 2024-04-26T18:45:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 0, 2024-04-26T18:45:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 14 2024-04-26T18:45:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-04-26T18:45:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 1, 2024-04-26T18:45:47.431 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 461 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 1, 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 1, 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 1, 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 1, 2024-04-26T18:45:47.432 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 15672, 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 4688857, 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 15672, 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 15672, 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 4688857, 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.433 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-04-26T18:45:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-26T18:45:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-26T18:45:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-26T18:45:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 4684, 2024-04-26T18:45:47.434 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 26390900, 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4684, 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 26390900, 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:47.435 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 4684, 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 4684, 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4684, 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 4684, 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:47.436 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 1, 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 149, 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 1, 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 1, 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 149, 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x56005c138500": { 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:47.437 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-26T18:45:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 2704, 2024-04-26T18:45:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 24674552, 2024-04-26T18:45:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 2704, 2024-04-26T18:45:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-26T18:45:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-26T18:45:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2605, 2024-04-26T18:45:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 24674552, 2024-04-26T18:45:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:47.438 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:47.439 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:47.439 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-26T18:45:48.111 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-26T18:45:48.111 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-26T18:45:48.111 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=23 2024-04-26T18:45:48.111 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-26T18:45:48.111 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-26T18:45:48.111 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=18 2024-04-26T18:45:48.111 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=12 2024-04-26T18:45:48.111 DEBUG:tasks.check_counter:Getting stats from e 2024-04-26T18:45:48.112 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-26T18:45:48.358 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:45:49.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:48 smithi084 ceph-mon[29141]: pgmap v562: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-04-26T18:45:49.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:48 smithi103 ceph-mon[32727]: pgmap v562: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-04-26T18:45:49.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:48 smithi174 ceph-mon[32705]: pgmap v562: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-04-26T18:45:50.307 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-26T18:45:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:45:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 29, 2024-04-26T18:45:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 248, 2024-04-26T18:45:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 8652, 2024-04-26T18:45:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 642236, 2024-04-26T18:45:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 17, 2024-04-26T18:45:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-04-26T18:45:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.018244926, 2024-04-26T18:45:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.008928839, 2024-04-26T18:45:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.004656254, 2024-04-26T18:45:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.001146963, 2024-04-26T18:45:50.308 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 243, 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.025657581, 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000105586 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 6, 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000647, 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000107 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 562368 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2601, 2024-04-26T18:45:50.309 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1948, 2024-04-26T18:45:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 2126861, 2024-04-26T18:45:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 710969, 2024-04-26T18:45:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 9, 2024-04-26T18:45:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-04-26T18:45:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.260178415, 2024-04-26T18:45:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.115837379, 2024-04-26T18:45:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.151971045, 2024-04-26T18:45:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.017702551, 2024-04-26T18:45:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1951, 2024-04-26T18:45:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.101326553, 2024-04-26T18:45:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000051935 2024-04-26T18:45:50.310 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3455, 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001074300, 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000310 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1551344, 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 228112 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 36, 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 148, 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 8241, 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 184885, 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 10, 2024-04-26T18:45:50.311 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.022005220, 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.007961014, 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.011726903, 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000303869, 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 145, 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.020866064, 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000143903 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 94, 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000025462, 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000270 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.312 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 4160, 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 177520 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 43, 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001432799, 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000033320 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.313 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000004941, 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000002470 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 1, 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 1, 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000116117, 2024-04-26T18:45:50.314 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000116117 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 11, 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 16, 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 16, 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 13, 2024-04-26T18:45:50.315 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 4, 2024-04-26T18:45:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 1, 2024-04-26T18:45:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 1, 2024-04-26T18:45:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 3, 2024-04-26T18:45:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 36, 2024-04-26T18:45:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 30, 2024-04-26T18:45:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-04-26T18:45:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 6, 2024-04-26T18:45:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:45:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-04-26T18:45:50.317 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 4, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 234, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 6, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:45:50.318 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 1, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 1, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-04-26T18:45:50.319 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 18, 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 12, 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 12, 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-04-26T18:45:50.320 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 12, 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 71, 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-04-26T18:45:50.321 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 5, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 5, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 2, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-04-26T18:45:50.322 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 4195732, 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 0, 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 4, 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.006995230, 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001748807 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 0 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 20, 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 22, 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 2, 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 19, 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 23, 2024-04-26T18:45:50.323 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 4, 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 16, 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 20, 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 4, 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 1, 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 1, 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 53644, 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 223608 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 1, 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 1, 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 18, 2024-04-26T18:45:50.324 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 71, 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.325 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.326 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1, 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000116117, 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000116117 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.327 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-04-26T18:45:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-04-26T18:45:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.328 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-04-26T18:45:50.329 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.329 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.329 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.329 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.329 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-04-26T18:45:50.329 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.329 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.330 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-04-26T18:45:50.331 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.332 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.333 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-04-26T18:45:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-04-26T18:45:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-04-26T18:45:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.334 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-04-26T18:45:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-04-26T18:45:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:45:50.335 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 2, 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 1, 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 1, 2024-04-26T18:45:50.336 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 1046, 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:45:50.337 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:45:50.338 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 247703, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 79, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:45:50.339 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8232, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 172, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 83053, 2024-04-26T18:45:50.340 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 338, 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 41, 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 11820, 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 33, 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 41, 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 41, 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.215995928, 2024-04-26T18:45:50.341 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.005268193 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 41, 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 63 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 41, 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 0, 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 41, 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 0, 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 10, 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 0, 2024-04-26T18:45:50.342 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 4, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 8, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 6, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 0, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 20, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:45:50.343 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-04-26T18:45:50.344 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 118, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 29, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 9, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 14, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 5, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 14, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 0, 2024-04-26T18:45:50.345 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 1 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 1, 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 83 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T18:45:50.346 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 2662, 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1878384, 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 2662, 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-26T18:45:50.347 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2662, 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1878384, 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-26T18:45:50.348 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 41, 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 11820, 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 41, 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 11820, 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-04-26T18:45:50.349 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 41, 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 41, 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 41, 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 41, 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:50.350 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:50.351 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x5648856c8500": { 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 5, 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1428, 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 5, 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4, 2024-04-26T18:45:50.352 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1428, 2024-04-26T18:45:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:50.353 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:50.353 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-26T18:45:51.092 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:50 smithi103 ceph-mon[32727]: pgmap v563: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-04-26T18:45:51.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:50 smithi084 ceph-mon[29141]: pgmap v563: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-04-26T18:45:51.118 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-26T18:45:51.118 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-26T18:45:51.118 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-26T18:45:51.118 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=18 2024-04-26T18:45:51.118 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-04-26T18:45:51.118 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:45:51.118 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:45:51.118 DEBUG:tasks.check_counter:Getting stats from h 2024-04-26T18:45:51.118 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-26T18:45:51.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:50 smithi174 ceph-mon[32705]: pgmap v563: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-04-26T18:45:51.355 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:45:53.078 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:52 smithi084 ceph-mon[29141]: pgmap v564: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:45:53.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:52 smithi103 ceph-mon[32727]: pgmap v564: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:45:53.154 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-26T18:45:53.155 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:45:53.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 2236, 2024-04-26T18:45:53.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 1609, 2024-04-26T18:45:53.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 588746, 2024-04-26T18:45:53.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 512204, 2024-04-26T18:45:53.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 9, 2024-04-26T18:45:53.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-04-26T18:45:53.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.203290753, 2024-04-26T18:45:53.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.099321923, 2024-04-26T18:45:53.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.113089332, 2024-04-26T18:45:53.155 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.007486099, 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 1611, 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.088787917, 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000055113 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3593, 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001056102, 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000293 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 2368, 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 10432 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:45:53.156 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 456, 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 436, 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1570660, 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 265298, 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 9, 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 5, 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.072710179, 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.025348567, 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.043128527, 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.010146541, 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 436, 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.040124707, 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000092029 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.157 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000161, 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000080 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1566000, 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 228560 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 30, 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 307, 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 7890, 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 690084, 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 17, 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 7, 2024-04-26T18:45:53.158 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.021937455, 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.011688311, 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.006788033, 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000165124, 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 300, 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.029387877, 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000097959 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 46, 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000006390, 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000138 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 5920, 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 640048 2024-04-26T18:45:53.159 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 42, 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.001783990, 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000042475 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2024-04-26T18:45:53.160 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000007053, 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000003526 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 7, 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 5, 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 5, 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.003152221, 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000630444 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 2, 2024-04-26T18:45:53.161 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 11, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 16, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 16, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 13, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 5, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 3, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 48, 2024-04-26T18:45:53.162 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 37, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 2, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 7, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 4, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 234, 2024-04-26T18:45:53.163 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 6, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 4, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 4, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-04-26T18:45:53.164 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 18, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 12, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 13, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-04-26T18:45:53.165 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 12, 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 71, 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-04-26T18:45:53.166 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 7, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 7, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 2, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 4196347, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 0, 2024-04-26T18:45:53.167 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 6, 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.011473052, 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001912175 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 0 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 20, 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 22, 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 2, 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 19, 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 23, 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 4, 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 16, 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 21, 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 5, 2024-04-26T18:45:53.168 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 4, 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 4, 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 52756, 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 223608 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 11, 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 7, 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 21, 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 72, 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.169 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 2, 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.002600410, 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.001300205 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.170 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3, 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000551811, 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000183937 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.171 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.172 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.173 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-04-26T18:45:53.173 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.173 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.173 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.173 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.173 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-04-26T18:45:53.173 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.173 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.173 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.174 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.175 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.175 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-04-26T18:45:53.175 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.175 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.175 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.175 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-04-26T18:45:53.175 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.175 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.175 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.175 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.175 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-04-26T18:45:53.176 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.177 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-04-26T18:45:53.178 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:53.179 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.180 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-04-26T18:45:53.180 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2024-04-26T18:45:53.180 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 3, 2024-04-26T18:45:53.180 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 2, 2024-04-26T18:45:53.180 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 1, 2024-04-26T18:45:53.180 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-04-26T18:45:53.180 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 4, 2024-04-26T18:45:53.180 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 2, 2024-04-26T18:45:53.180 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 1070, 2024-04-26T18:45:53.180 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-04-26T18:45:53.180 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.180 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-04-26T18:45:53.180 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:45:53.181 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-04-26T18:45:53.182 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 247927, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 82, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-04-26T18:45:53.183 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8232, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 172, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 82765, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 346, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-04-26T18:45:53.184 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 44, 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 12490, 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 33, 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 44, 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 44, 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.211996004, 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.004818091 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 44, 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 68 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 44, 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 0, 2024-04-26T18:45:53.185 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 44, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 0, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 10, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 0, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 6, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 7, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 6, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-04-26T18:45:53.186 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 20, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-04-26T18:45:53.187 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 118, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 29, 2024-04-26T18:45:53.188 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 9, 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 13, 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 4, 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 17, 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 0, 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 2 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 1, 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 83 2024-04-26T18:45:53.189 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:53.190 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 2718, 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 1896197, 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 2718, 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 2718, 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 1896197, 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-04-26T18:45:53.191 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:53.192 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-26T18:45:53.192 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:53.192 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-26T18:45:53.192 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-26T18:45:53.192 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:53.192 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:53.192 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 44, 2024-04-26T18:45:53.192 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 12490, 2024-04-26T18:45:53.192 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 44, 2024-04-26T18:45:53.192 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 12490, 2024-04-26T18:45:53.192 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:53.192 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.192 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.193 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.193 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:53.193 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.193 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-04-26T18:45:53.193 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:53.193 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-04-26T18:45:53.193 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:53.193 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-26T18:45:53.194 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-26T18:45:53.194 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:53.194 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:53.194 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 44, 2024-04-26T18:45:53.194 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 44, 2024-04-26T18:45:53.194 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 44, 2024-04-26T18:45:53.195 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 44, 2024-04-26T18:45:53.195 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:53.195 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.195 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.195 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.195 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:53.195 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.196 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:45:53.196 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:53.196 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-26T18:45:53.196 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:53.196 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-26T18:45:53.196 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-26T18:45:53.196 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:53.197 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:53.197 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-26T18:45:53.197 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-26T18:45:53.197 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-26T18:45:53.197 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-26T18:45:53.197 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:53.197 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.198 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.198 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.198 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:53.198 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:53.198 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x564e9ea5a500": { 2024-04-26T18:45:53.198 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:53.199 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-26T18:45:53.199 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:53.199 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 7, 2024-04-26T18:45:53.199 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 2043, 2024-04-26T18:45:53.199 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:53.199 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 7, 2024-04-26T18:45:53.199 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-26T18:45:53.200 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-26T18:45:53.200 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 6, 2024-04-26T18:45:53.200 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2043, 2024-04-26T18:45:53.200 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:53.200 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:53.200 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:53.200 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:53.201 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:53.201 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:53.201 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-26T18:45:53.202 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:52 smithi174 ceph-mon[32705]: pgmap v564: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:45:54.051 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-26T18:45:54.051 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-26T18:45:54.051 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-26T18:45:54.052 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=18 2024-04-26T18:45:54.052 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-26T18:45:54.052 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:45:54.052 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:45:54.052 DEBUG:tasks.check_counter:Getting stats from k 2024-04-26T18:45:54.052 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-26T18:45:54.290 INFO:teuthology.orchestra.run.smithi103.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.b/config 2024-04-26T18:45:55.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:54 smithi084 ceph-mon[29141]: pgmap v565: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T18:45:55.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:54 smithi103 ceph-mon[32727]: pgmap v565: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T18:45:55.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:54 smithi174 ceph-mon[32705]: pgmap v565: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T18:45:56.058 INFO:teuthology.orchestra.run.smithi103.stdout:{ 2024-04-26T18:45:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:45:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 3751, 2024-04-26T18:45:56.058 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 3751, 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1062618, 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 1192001, 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 6, 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.298167988, 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.099243845, 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.070492054, 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.105769090, 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3750, 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.117313966, 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000031283 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.059 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 5, 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 276, 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1036, 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 724906, 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 7, 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 4, 2024-04-26T18:45:56.060 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.015751128, 2024-04-26T18:45:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.010696272, 2024-04-26T18:45:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.002199292, 2024-04-26T18:45:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.000047314, 2024-04-26T18:45:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 273, 2024-04-26T18:45:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.018310226, 2024-04-26T18:45:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000067070 2024-04-26T18:45:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.061 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 480, 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 724208 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_messages": 404, 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages": 392, 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_bytes": 1540524, 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_bytes": 226830, 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_created_connections": 4, 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_active_connections": 2, 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_total_time": 0.069728409, 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_send_time": 0.025053961, 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_recv_time": 0.040662208, 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_running_fast_dispatch_time": 0.009572391, 2024-04-26T18:45:56.062 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 391, 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.021207432, 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000054238 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_recv_encrypted_bytes": 1540256, 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "msgr_send_encrypted_bytes": 226512 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "cct": { 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "total_workers": 1, 2024-04-26T18:45:56.063 INFO:teuthology.orchestra.run.smithi103.stdout: "unhealthy_workers": 0 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-MDSRank": { 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 6294, 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.159031048, 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000025267 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "finisher-PurgeQueue": { 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "queue_len": 0, 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "complete_latency": { 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:56.064 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "mds": { 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "request": 0, 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "reply": 0, 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "reply_latency": { 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_reply": 0, 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "forward": 0, 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_complete": 0, 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_fetch_keys": 0, 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_commit": 0, 2024-04-26T18:45:56.065 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_split": 0, 2024-04-26T18:45:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_merge": 0, 2024-04-26T18:45:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes": 0, 2024-04-26T18:45:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_top": 0, 2024-04-26T18:45:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_bottom": 0, 2024-04-26T18:45:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pin_tail": 0, 2024-04-26T18:45:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_pinned": 0, 2024-04-26T18:45:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_expired": 0, 2024-04-26T18:45:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "inodes_with_caps": 0, 2024-04-26T18:45:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "caps": 0, 2024-04-26T18:45:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "subtrees": 1, 2024-04-26T18:45:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse": 0, 2024-04-26T18:45:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_hit": 0, 2024-04-26T18:45:56.066 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_forward": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_discover": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_remote_ino": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "traverse_lock": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "load_cent": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "q": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "exported": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "exported_inodes": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "imported": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "imported_inodes": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_dir_fetch": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "openino_peer_discover": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rfiles": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rbytes": 0, 2024-04-26T18:45:56.067 INFO:teuthology.orchestra.run.smithi103.stdout: "root_rsnaps": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_set_tag": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "scrub_file_inodes": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_grant": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:45:56.068 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps": 0, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_caps_dirty": 0, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_cap_release": 0, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "process_request_cap_release": 0 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_cache": { 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update": 0, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_update_receipt": 0, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_try_discover": 0, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_send_discover": 0, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "dir_handle_discover": 0, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays": 0, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_delayed": 0, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_created": 0, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_enqueued": 0, 2024-04-26T18:45:56.069 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_reintegrated": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "strays_migrated": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_processing": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_started": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "recovery_completed": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_path": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_quiesce_inode": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_exportdir": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_flush": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_fragstats": 0, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "ireq_inodestats": 0 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.070 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_log": { 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "evlrg": 0, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "evadd": 0, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "evex": 0, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "evtrm": 0, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "ev": 7, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "evexg": 0, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "evexd": 0, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "segadd": 0, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "segex": 0, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "segtrm": 0, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "seg": 2, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "segmjr": 2, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "segexg": 0, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "segexd": 0, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "expos": 4194304, 2024-04-26T18:45:56.071 INFO:teuthology.orchestra.run.smithi103.stdout: "wrpos": 4196347, 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "rdpos": 4196206, 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "jlat": { 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "replayed": 7 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_mem": { 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "ino": 3, 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "ino+": 3, 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "ino-": 0, 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "dir": 1, 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "dir+": 1, 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "dir-": 0, 2024-04-26T18:45:56.072 INFO:teuthology.orchestra.run.smithi103.stdout: "dn": 0, 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "dn+": 0, 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "dn-": 0, 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "cap": 0, 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "cap+": 0, 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "cap-": 0, 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "rss": 50384, 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "heap": 215416 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_server": { 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_client_request": 0, 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "dispatch_server_request": 0, 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_request": 0, 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_client_session": 0, 2024-04-26T18:45:56.073 INFO:teuthology.orchestra.run.smithi103.stdout: "handle_peer_request": 0, 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "req_create_latency": { 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getattr_latency": { 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getfilelock_latency": { 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.074 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "req_link_latency": { 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookup_latency": { 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookuphash_latency": { 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.075 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupino_latency": { 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupname_latency": { 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupparent_latency": { 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lookupsnap_latency": { 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.076 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "req_lssnap_latency": { 2024-04-26T18:45:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mkdir_latency": { 2024-04-26T18:45:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.077 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mknod_latency": { 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "req_mksnap_latency": { 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "req_open_latency": { 2024-04-26T18:45:56.078 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "req_readdir_latency": { 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rename_latency": { 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "req_renamesnap_latency": { 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.079 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "req_snapdiff_latency": { 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmdir_latency": { 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmsnap_latency": { 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.080 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "req_rmxattr_latency": { 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setattr_latency": { 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setdirlayout_latency": { 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setfilelock_latency": { 2024-04-26T18:45:56.081 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setlayout_latency": { 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "req_setxattr_latency": { 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "req_symlink_latency": { 2024-04-26T18:45:56.082 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "req_unlink_latency": { 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "req_getvxattr_latency": { 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.083 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_sessions": { 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "session_count": 1, 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "session_add": 3, 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "session_remove": 2, 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_open": 2, 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "sessions_stale": 0, 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "total_load": 0, 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "average_load": 0, 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "avg_session_uptime": 3515, 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "mdthresh_evicted": 0 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "mempool": { 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:45:56.084 INFO:teuthology.orchestra.run.smithi103.stdout: "bloom_filter_items": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_extent_items": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_blob_items": 0, 2024-04-26T18:45:56.085 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_txc_items": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluestore_writing_items": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_bytes": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_items": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:45:56.086 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_bytes": 182883, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_anon_items": 51, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "buffer_meta_items": 0, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_bytes": 0, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_items": 0, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_mapbl_items": 0, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_pglog_items": 0, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_bytes": 8232, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_items": 172, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_bytes": 0, 2024-04-26T18:45:56.087 INFO:teuthology.orchestra.run.smithi103.stdout: "pgmap_items": 0, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_bytes": 8512, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "mds_co_items": 13, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_bytes": 0, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_1_items": 0, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_bytes": 0, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "unittest_2_items": 0 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "objecter": { 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "op_active": 0, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "op_laggy": 0, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send": 3750, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "op_send_bytes": 21, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "op_resend": 3, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "op_reply": 3750, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "op_latency": { 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3750, 2024-04-26T18:45:56.088 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 1.974961927, 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000526656 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "op_inflight": 0, 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "oplen_avg": { 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 3750, 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 3751 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "op": 3750, 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "op_r": 3750, 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "op_w": 0, 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "op_rmw": 0, 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "op_pg": 0, 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_stat": 2464, 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_create": 0, 2024-04-26T18:45:56.089 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_read": 1284, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_write": 0, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writefull": 0, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_writesame": 0, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_append": 0, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_zero": 0, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_truncate": 0, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_delete": 0, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_mapext": 0, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_sparse_read": 0, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_clonerange": 0, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_getxattr": 1, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_setxattr": 0, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:45:56.090 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_rmxattr": 0, 2024-04-26T18:45:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:45:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_call": 0, 2024-04-26T18:45:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_watch": 0, 2024-04-26T18:45:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_notify": 0, 2024-04-26T18:45:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:45:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls": 0, 2024-04-26T18:45:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:45:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "osdop_other": 0, 2024-04-26T18:45:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_active": 0, 2024-04-26T18:45:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_send": 0, 2024-04-26T18:45:56.091 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_resend": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "linger_ping": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_active": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_send": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "poolop_resend": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_active": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_send": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "poolstat_resend": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_active": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_send": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "statfs_resend": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "command_active": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "command_send": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "command_resend": 0, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "map_epoch": 118, 2024-04-26T18:45:56.092 INFO:teuthology.orchestra.run.smithi103.stdout: "map_full": 0, 2024-04-26T18:45:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "map_inc": 29, 2024-04-26T18:45:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_sessions": 4, 2024-04-26T18:45:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_open": 5, 2024-04-26T18:45:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_session_close": 1, 2024-04-26T18:45:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "osd_laggy": 0, 2024-04-26T18:45:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_wr": 0, 2024-04-26T18:45:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_rd": 2, 2024-04-26T18:45:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_del": 0 2024-04-26T18:45:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "oft": { 2024-04-26T18:45:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_objs": 0, 2024-04-26T18:45:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:45:56.093 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_updates": 0, 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "omap_total_removes": 0 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "osd-slow-ops": { 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "slow_ops_count": 0 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "purge_queue": { 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops": 0, 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing": 0, 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executing_high_water": 0, 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed_ops": 0, 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_executed": 0, 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: "pq_item_in_journal": 0 2024-04-26T18:45:56.094 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 4156, 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 2206407, 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 4156, 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 4156, 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2206407, 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.095 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_bytes": { 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 104857600, 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3750, 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 2048, 2024-04-26T18:45:56.096 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3750, 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 2048, 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-objecter_ops": { 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 1024, 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-26T18:45:56.097 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 3750, 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 3750, 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 3750, 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 3750, 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-26T18:45:56.098 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: }, 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "throttle-write_buf_throttle-0x55ea226146e0": { 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "val": 0, 2024-04-26T18:45:56.099 INFO:teuthology.orchestra.run.smithi103.stdout: "max": 3758096384, 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "get_started": 0, 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "get": 0, 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "get_sum": 0, 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "take": 0, 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "take_sum": 0, 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "put": 0, 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "put_sum": 0, 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "wait": { 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "avgcount": 0, 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "sum": 0.000000000, 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: "avgtime": 0.000000000 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:56.100 INFO:teuthology.orchestra.run.smithi103.stdout: } 2024-04-26T18:45:56.101 INFO:teuthology.orchestra.run.smithi103.stdout:} 2024-04-26T18:45:56.841 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-26T18:45:56.841 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-26T18:45:56.841 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-26T18:45:56.841 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-04-26T18:45:56.841 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-04-26T18:45:56.841 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:45:56.841 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:45:56.841 DEBUG:tasks.check_counter:Getting stats from c 2024-04-26T18:45:56.841 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-26T18:45:56.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:56 smithi103 ceph-mon[32727]: pgmap v566: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:45:57.092 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:45:57.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:56 smithi084 ceph-mon[29141]: pgmap v566: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:45:57.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:56 smithi174 ceph-mon[32705]: pgmap v566: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:45:58.930 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-04-26T18:45:58.930 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:45:58.930 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 35, 2024-04-26T18:45:58.930 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 39, 2024-04-26T18:45:58.930 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 8717, 2024-04-26T18:45:58.930 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 60599, 2024-04-26T18:45:58.931 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 14, 2024-04-26T18:45:58.931 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-04-26T18:45:58.931 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.007029270, 2024-04-26T18:45:58.931 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.000811834, 2024-04-26T18:45:58.931 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.004090836, 2024-04-26T18:45:58.931 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000592389, 2024-04-26T18:45:58.931 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:58.931 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 37, 2024-04-26T18:45:58.931 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.013501832, 2024-04-26T18:45:58.931 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000364914 2024-04-26T18:45:58.931 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.931 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:58.931 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 24, 2024-04-26T18:45:58.932 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000003101, 2024-04-26T18:45:58.932 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000129 2024-04-26T18:45:58.932 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.932 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 2512, 2024-04-26T18:45:58.932 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 2656 2024-04-26T18:45:58.932 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.932 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:45:58.932 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 444, 2024-04-26T18:45:58.932 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 428, 2024-04-26T18:45:58.932 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1561533, 2024-04-26T18:45:58.932 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 235890, 2024-04-26T18:45:58.932 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2024-04-26T18:45:58.932 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-04-26T18:45:58.933 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.072176783, 2024-04-26T18:45:58.933 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.025476802, 2024-04-26T18:45:58.933 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.041849999, 2024-04-26T18:45:58.933 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.009808112, 2024-04-26T18:45:58.933 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:58.933 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 427, 2024-04-26T18:45:58.933 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.044605562, 2024-04-26T18:45:58.933 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000104462 2024-04-26T18:45:58.933 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.933 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:58.933 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.933 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.933 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.934 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.934 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1557504, 2024-04-26T18:45:58.934 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 228592 2024-04-26T18:45:58.934 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.934 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:45:58.934 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 2236, 2024-04-26T18:45:58.934 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1815, 2024-04-26T18:45:58.934 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 590966, 2024-04-26T18:45:58.934 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 973065, 2024-04-26T18:45:58.934 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 12, 2024-04-26T18:45:58.934 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-04-26T18:45:58.934 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.167206890, 2024-04-26T18:45:58.934 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.071745939, 2024-04-26T18:45:58.935 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.097987501, 2024-04-26T18:45:58.935 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.008012850, 2024-04-26T18:45:58.935 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:45:58.935 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1813, 2024-04-26T18:45:58.935 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.084724490, 2024-04-26T18:45:58.935 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000046731 2024-04-26T18:45:58.935 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.935 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:45:58.935 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3542, 2024-04-26T18:45:58.935 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000730323, 2024-04-26T18:45:58.935 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000206 2024-04-26T18:45:58.935 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.935 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3360, 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 637424 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 51, 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.002168970, 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000042528 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:45:58.936 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000007164, 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000003582 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 2, 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 1, 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1, 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000184786, 2024-04-26T18:45:58.937 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000184786 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 1, 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 11, 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 16, 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 16, 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 13, 2024-04-26T18:45:58.938 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 4, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 3, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 39, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 31, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 7, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-04-26T18:45:58.939 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 4, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 234, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 6, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:45:58.940 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 1 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 16, 2024-04-26T18:45:58.941 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 11, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 12, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-04-26T18:45:58.942 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 12, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 71, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 6, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 6, 2024-04-26T18:45:58.943 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 2, 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 4195956, 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 5, 2024-04-26T18:45:58.944 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.011311026, 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.002262205 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 20, 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 22, 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 2, 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 19, 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 22, 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 3, 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 16, 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 20, 2024-04-26T18:45:58.945 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 4, 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 1, 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 1, 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 52724, 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223608 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 3, 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 2, 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 8, 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 71, 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-04-26T18:45:58.946 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1, 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000184786, 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000184786 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.947 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.948 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.949 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-04-26T18:45:58.950 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.950 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.950 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.950 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.950 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-04-26T18:45:58.950 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.950 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.950 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.950 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.950 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-04-26T18:45:58.950 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.951 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.951 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.951 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.951 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-04-26T18:45:58.951 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.951 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.951 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.951 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.951 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.952 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.953 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.954 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.955 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.956 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-04-26T18:45:58.956 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.956 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.956 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.956 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.956 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-04-26T18:45:58.956 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.956 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.956 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.956 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.956 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-04-26T18:45:58.956 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.956 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:45:58.957 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.958 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-04-26T18:45:58.958 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 0, 2024-04-26T18:45:58.958 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 2, 2024-04-26T18:45:58.958 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 2, 2024-04-26T18:45:58.958 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 0, 2024-04-26T18:45:58.958 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-04-26T18:45:58.958 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 1, 2024-04-26T18:45:58.958 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 1, 2024-04-26T18:45:58.958 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 84, 2024-04-26T18:45:58.958 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-04-26T18:45:58.958 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.958 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-04-26T18:45:58.958 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-04-26T18:45:58.959 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-04-26T18:45:58.960 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 247607, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 78, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8232, 2024-04-26T18:45:58.961 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 172, 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 82717, 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 340, 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-04-26T18:45:58.962 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-04-26T18:45:58.963 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 42, 2024-04-26T18:45:58.963 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 12044, 2024-04-26T18:45:58.963 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 28, 2024-04-26T18:45:58.963 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 42, 2024-04-26T18:45:58.963 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-04-26T18:45:58.963 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 42, 2024-04-26T18:45:58.963 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.210997511, 2024-04-26T18:45:58.963 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.005023750 2024-04-26T18:45:58.963 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 42, 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 64 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 42, 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 0, 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 42, 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 0, 2024-04-26T18:45:58.964 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 5, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 8, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 6, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 0, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:45:58.965 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-04-26T18:45:58.966 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 118, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 29, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 9, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 12, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 3, 2024-04-26T18:45:58.967 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 14, 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 1 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 83 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-04-26T18:45:58.968 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 2711, 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1891933, 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:58.969 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 2711, 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2711, 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1891933, 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-04-26T18:45:58.970 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 42, 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 12044, 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 42, 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 12044, 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:45:58.971 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 42, 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 42, 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 42, 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 42, 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:45:58.972 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-04-26T18:45:58.973 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55c2f104c500": { 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 6, 2024-04-26T18:45:58.974 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1652, 2024-04-26T18:45:58.975 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:45:58.975 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 6, 2024-04-26T18:45:58.975 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-04-26T18:45:58.975 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-04-26T18:45:58.975 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 5, 2024-04-26T18:45:58.975 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1652, 2024-04-26T18:45:58.975 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:45:58.975 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:45:58.975 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:45:58.975 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:45:58.975 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:45:58.975 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:45:58.975 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-04-26T18:45:59.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:45:58 smithi084 ceph-mon[29141]: pgmap v567: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:45:59.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:45:58 smithi103 ceph-mon[32727]: pgmap v567: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:45:59.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:45:58 smithi174 ceph-mon[32705]: pgmap v567: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-26T18:45:59.776 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-26T18:45:59.777 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-26T18:45:59.777 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-26T18:45:59.777 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=16 2024-04-26T18:45:59.777 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-26T18:45:59.777 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:45:59.777 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:45:59.777 DEBUG:tasks.check_counter:Getting stats from f 2024-04-26T18:45:59.777 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-26T18:46:00.021 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:46:01.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:00 smithi084 ceph-mon[29141]: pgmap v568: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T18:46:01.124 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:00 smithi174 ceph-mon[32705]: pgmap v568: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T18:46:01.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:00 smithi103 ceph-mon[32727]: pgmap v568: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-26T18:46:01.730 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-04-26T18:46:01.730 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:46:01.730 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 2765, 2024-04-26T18:46:01.730 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 2765, 2024-04-26T18:46:01.730 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 12912169, 2024-04-26T18:46:01.730 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 878456, 2024-04-26T18:46:01.731 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 7, 2024-04-26T18:46:01.731 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-04-26T18:46:01.731 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.289726118, 2024-04-26T18:46:01.731 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.083073671, 2024-04-26T18:46:01.731 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.088628394, 2024-04-26T18:46:01.731 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.093251444, 2024-04-26T18:46:01.731 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:46:01.731 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2764, 2024-04-26T18:46:01.731 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.108366305, 2024-04-26T18:46:01.731 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000039206 2024-04-26T18:46:01.731 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.731 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:46:01.731 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 966, 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 954, 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 5875614, 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 404488, 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 5, 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.129531140, 2024-04-26T18:46:01.732 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.042600197, 2024-04-26T18:46:01.733 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.061291071, 2024-04-26T18:46:01.733 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.027287238, 2024-04-26T18:46:01.733 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:46:01.733 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 953, 2024-04-26T18:46:01.733 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.043652657, 2024-04-26T18:46:01.733 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000045805 2024-04-26T18:46:01.733 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.733 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:46:01.733 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.733 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.733 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.733 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1539600, 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 224752 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1213, 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1481, 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 8692121, 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1020459, 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 7, 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.125664660, 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.049011100, 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.030990252, 2024-04-26T18:46:01.734 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.033456594, 2024-04-26T18:46:01.735 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:46:01.735 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1478, 2024-04-26T18:46:01.735 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.060878109, 2024-04-26T18:46:01.735 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000041189 2024-04-26T18:46:01.735 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.735 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:46:01.735 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.735 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.735 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.735 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.735 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T18:46:01.735 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 635360 2024-04-26T18:46:01.735 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 7728, 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.215860795, 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000027932 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-04-26T18:46:01.736 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.737 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-04-26T18:46:01.738 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-04-26T18:46:01.738 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-04-26T18:46:01.738 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-04-26T18:46:01.738 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-04-26T18:46:01.738 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-04-26T18:46:01.738 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-04-26T18:46:01.738 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 36, 2024-04-26T18:46:01.738 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 16, 2024-04-26T18:46:01.738 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 20, 2024-04-26T18:46:01.738 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-04-26T18:46:01.738 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 14, 2024-04-26T18:46:01.738 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-04-26T18:46:01.738 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 2, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-04-26T18:46:01.739 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 4, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 234, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 6, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-04-26T18:46:01.740 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-04-26T18:46:01.741 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-04-26T18:46:01.742 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 2710, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-04-26T18:46:01.743 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 81, 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 28461248, 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 28877654, 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 28873491, 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 2710 2024-04-26T18:46:01.744 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 31, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 32, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 1, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 11, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 11, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 0, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 36, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 36, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 0, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 51692, 2024-04-26T18:46:01.745 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 215416 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.746 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.747 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.748 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.749 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.750 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.750 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-04-26T18:46:01.750 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.750 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.750 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.751 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.752 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.753 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.754 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.755 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.756 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 2, 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 6, 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 4, 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 5, 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 4219, 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-04-26T18:46:01.757 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:46:01.758 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-04-26T18:46:01.759 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 208930, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 92, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-04-26T18:46:01.760 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8232, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 172, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 145417, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 1129, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-04-26T18:46:01.761 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 4537, 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 26, 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 4, 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 4537, 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4537, 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.264972368, 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000499222 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 4537, 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 4538 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 4537, 2024-04-26T18:46:01.762 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 4537, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 2553, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 1981, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-04-26T18:46:01.763 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 2, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-04-26T18:46:01.764 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 118, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-04-26T18:46:01.765 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 29, 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 7, 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 8, 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 1, 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-04-26T18:46:01.766 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-04-26T18:46:01.767 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 4940, 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 27011726, 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 4940, 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 4940, 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 27011726, 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.768 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 4537, 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 24683360, 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 4537, 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 24683360, 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.769 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 4537, 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 4537, 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 4537, 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 4537, 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:01.770 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-04-26T18:46:01.771 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-04-26T18:46:01.772 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:01.772 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.772 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.772 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.772 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:01.772 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:01.772 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x5615b2d8e6e0": { 2024-04-26T18:46:01.772 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:01.772 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-04-26T18:46:01.772 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:01.772 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:46:01.772 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:46:01.773 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:01.773 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:46:01.773 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-04-26T18:46:01.773 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-04-26T18:46:01.773 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-04-26T18:46:01.773 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-04-26T18:46:01.773 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:01.773 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:01.773 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:01.773 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:01.773 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:01.773 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:01.773 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-04-26T18:46:02.497 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-26T18:46:02.497 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-26T18:46:02.497 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-26T18:46:02.497 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-26T18:46:02.497 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-04-26T18:46:02.497 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:46:02.497 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:46:02.497 DEBUG:tasks.check_counter:Getting stats from i 2024-04-26T18:46:02.498 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-26T18:46:02.744 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:46:03.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:02 smithi084 ceph-mon[29141]: pgmap v569: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:46:03.122 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:02 smithi174 ceph-mon[32705]: pgmap v569: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:46:03.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:02 smithi103 ceph-mon[32727]: pgmap v569: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-26T18:46:04.315 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-04-26T18:46:04.316 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:46:04.316 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 37, 2024-04-26T18:46:04.316 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 262, 2024-04-26T18:46:04.316 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 8599, 2024-04-26T18:46:04.316 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 652226, 2024-04-26T18:46:04.316 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 18, 2024-04-26T18:46:04.316 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-04-26T18:46:04.316 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.022008207, 2024-04-26T18:46:04.316 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.009418714, 2024-04-26T18:46:04.316 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.007376560, 2024-04-26T18:46:04.316 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.000724645, 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 255, 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.074635998, 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000292690 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 13, 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000001449, 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000111 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 568032 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.317 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:46:04.318 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 2255, 2024-04-26T18:46:04.318 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1626, 2024-04-26T18:46:04.318 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 596378, 2024-04-26T18:46:04.318 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 659324, 2024-04-26T18:46:04.318 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 10, 2024-04-26T18:46:04.318 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 5, 2024-04-26T18:46:04.318 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.161092136, 2024-04-26T18:46:04.318 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.066368773, 2024-04-26T18:46:04.318 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.100007524, 2024-04-26T18:46:04.318 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.008255330, 2024-04-26T18:46:04.318 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:46:04.318 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1628, 2024-04-26T18:46:04.318 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.070087014, 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000043050 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3601, 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000747409, 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000207 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3872, 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 165248 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 464, 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 474, 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1575531, 2024-04-26T18:46:04.319 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 250676, 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.072829100, 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.026976890, 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.042703351, 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.009310770, 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 474, 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.031044930, 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000065495 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 58, 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000011510, 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000198 2024-04-26T18:46:04.320 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1572720, 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 240672 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 47, 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.002518575, 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000053586 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:04.321 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.322 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-04-26T18:46:04.322 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-04-26T18:46:04.322 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-04-26T18:46:04.322 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-04-26T18:46:04.322 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000007491, 2024-04-26T18:46:04.322 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000003745 2024-04-26T18:46:04.322 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:04.322 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.322 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-04-26T18:46:04.322 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 5, 2024-04-26T18:46:04.322 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 3, 2024-04-26T18:46:04.322 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3, 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000934831, 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000311610 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 2, 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 11, 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 16, 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 16, 2024-04-26T18:46:04.323 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 13, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 4, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 3, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 42, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 36, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 6, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-04-26T18:46:04.324 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 4, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 234, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 6, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:46:04.325 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 3, 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 3, 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-04-26T18:46:04.326 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-04-26T18:46:04.327 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 18, 2024-04-26T18:46:04.327 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 12, 2024-04-26T18:46:04.327 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 12, 2024-04-26T18:46:04.327 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-04-26T18:46:04.327 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-04-26T18:46:04.327 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-04-26T18:46:04.327 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:46:04.327 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-04-26T18:46:04.327 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-04-26T18:46:04.327 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-04-26T18:46:04.327 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-04-26T18:46:04.327 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-04-26T18:46:04.327 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 12, 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 71, 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-04-26T18:46:04.328 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 7, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 7, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 2, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-04-26T18:46:04.329 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 4196347, 2024-04-26T18:46:04.330 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 0, 2024-04-26T18:46:04.330 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-04-26T18:46:04.330 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6, 2024-04-26T18:46:04.330 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.010204106, 2024-04-26T18:46:04.330 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.001700684 2024-04-26T18:46:04.330 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.330 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 0 2024-04-26T18:46:04.330 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.330 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-04-26T18:46:04.330 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 20, 2024-04-26T18:46:04.330 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 22, 2024-04-26T18:46:04.330 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 2, 2024-04-26T18:46:04.330 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 19, 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 23, 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 4, 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 16, 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 20, 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 4, 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 3, 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 3, 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 53596, 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 223608 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 5, 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-04-26T18:46:04.331 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 5, 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 21, 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 71, 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1, 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000322162, 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000322162 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-04-26T18:46:04.332 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.333 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.333 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.333 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.333 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-04-26T18:46:04.333 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.333 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.333 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.333 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.333 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-04-26T18:46:04.333 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 2, 2024-04-26T18:46:04.333 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000612669, 2024-04-26T18:46:04.333 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000306334 2024-04-26T18:46:04.333 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.334 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.335 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.336 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.336 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-04-26T18:46:04.336 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.336 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.336 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.336 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.336 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-04-26T18:46:04.337 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.337 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.337 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.337 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.337 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-04-26T18:46:04.337 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.337 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.337 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.337 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.337 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-04-26T18:46:04.337 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.337 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.337 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.338 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.338 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-04-26T18:46:04.338 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.338 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.338 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.338 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.338 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-04-26T18:46:04.338 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.338 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.338 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.338 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.338 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.339 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.340 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-04-26T18:46:04.340 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.340 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.340 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.340 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.340 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-04-26T18:46:04.340 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.340 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.340 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.340 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.340 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-04-26T18:46:04.340 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.340 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.341 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.341 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.341 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-04-26T18:46:04.341 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.341 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.341 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.341 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.341 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-04-26T18:46:04.341 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.341 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.341 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.341 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.341 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.342 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.343 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-04-26T18:46:04.343 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.343 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.343 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.343 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.343 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:46:04.343 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:46:04.343 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-04-26T18:46:04.343 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.343 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.343 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.343 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:04.343 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 3, 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 2, 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 1, 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 2, 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 2, 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 1083, 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-04-26T18:46:04.344 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-04-26T18:46:04.345 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-04-26T18:46:04.346 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 247847, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 81, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8232, 2024-04-26T18:46:04.347 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 172, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 82749, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 344, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 45, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 12660, 2024-04-26T18:46:04.348 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 32, 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 45, 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 45, 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.263996873, 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.005866597 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 45, 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 69 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 45, 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 0, 2024-04-26T18:46:04.349 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 45, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 0, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 10, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 0, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 6, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 8, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 6, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-04-26T18:46:04.350 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 20, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-04-26T18:46:04.351 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 118, 2024-04-26T18:46:04.352 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-04-26T18:46:04.353 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 29, 2024-04-26T18:46:04.353 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 9, 2024-04-26T18:46:04.353 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 11, 2024-04-26T18:46:04.353 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 2, 2024-04-26T18:46:04.353 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-04-26T18:46:04.353 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 17, 2024-04-26T18:46:04.353 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 0, 2024-04-26T18:46:04.353 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 2 2024-04-26T18:46:04.353 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.353 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-04-26T18:46:04.353 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 1, 2024-04-26T18:46:04.353 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:46:04.353 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 83 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-04-26T18:46:04.354 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 2752, 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 1906270, 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 2752, 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 2752, 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 1906270, 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.355 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.356 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.356 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:04.356 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.356 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-04-26T18:46:04.356 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:04.356 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-04-26T18:46:04.356 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:04.356 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:46:04.356 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:46:04.356 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:04.356 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:46:04.356 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 45, 2024-04-26T18:46:04.356 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 12660, 2024-04-26T18:46:04.357 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 45, 2024-04-26T18:46:04.357 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 12660, 2024-04-26T18:46:04.357 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:04.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.357 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.357 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.357 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:04.357 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.357 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-04-26T18:46:04.357 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:04.357 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-04-26T18:46:04.357 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:04.357 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:46:04.358 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:46:04.358 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:04.358 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:46:04.358 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 45, 2024-04-26T18:46:04.358 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 45, 2024-04-26T18:46:04.358 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 45, 2024-04-26T18:46:04.358 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 45, 2024-04-26T18:46:04.358 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:04.358 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.358 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.358 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.358 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:04.358 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.359 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:46:04.359 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:04.359 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-04-26T18:46:04.359 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:04.359 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:46:04.359 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:46:04.359 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:04.359 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:46:04.359 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-04-26T18:46:04.359 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-04-26T18:46:04.359 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-04-26T18:46:04.359 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-04-26T18:46:04.359 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:04.360 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.360 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.360 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.360 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:04.360 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:04.360 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x55e9ca0c4500": { 2024-04-26T18:46:04.360 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:04.360 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-04-26T18:46:04.360 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:04.360 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 7, 2024-04-26T18:46:04.360 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 2043, 2024-04-26T18:46:04.360 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:04.361 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 7, 2024-04-26T18:46:04.361 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-04-26T18:46:04.361 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-04-26T18:46:04.361 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 6, 2024-04-26T18:46:04.361 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2043, 2024-04-26T18:46:04.361 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:04.361 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:04.361 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:04.361 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:04.361 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:04.361 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:04.361 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-04-26T18:46:04.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:04 smithi174 ceph-mon[32705]: pgmap v570: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T18:46:04.998 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-26T18:46:04.998 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-26T18:46:04.998 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-26T18:46:04.998 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=18 2024-04-26T18:46:04.999 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-26T18:46:04.999 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:46:04.999 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:46:04.999 DEBUG:tasks.check_counter:Getting stats from l 2024-04-26T18:46:04.999 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-26T18:46:05.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:04 smithi084 ceph-mon[29141]: pgmap v570: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T18:46:05.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:04 smithi103 ceph-mon[32727]: pgmap v570: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 45 KiB/s wr, 29 op/s 2024-04-26T18:46:05.234 INFO:teuthology.orchestra.run.smithi174.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.c/config 2024-04-26T18:46:06.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:06 smithi174 ceph-mon[32705]: pgmap v571: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T18:46:06.946 INFO:teuthology.orchestra.run.smithi174.stdout:{ 2024-04-26T18:46:06.946 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-0": { 2024-04-26T18:46:06.946 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1297, 2024-04-26T18:46:06.946 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1297, 2024-04-26T18:46:06.946 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 344424, 2024-04-26T18:46:06.946 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 411686, 2024-04-26T18:46:06.947 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 7, 2024-04-26T18:46:06.947 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 2, 2024-04-26T18:46:06.947 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.126305683, 2024-04-26T18:46:06.947 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.039317674, 2024-04-26T18:46:06.947 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.034111286, 2024-04-26T18:46:06.947 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.039918124, 2024-04-26T18:46:06.947 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:46:06.947 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1296, 2024-04-26T18:46:06.947 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.056873301, 2024-04-26T18:46:06.947 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000043883 2024-04-26T18:46:06.947 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.947 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:46:06.947 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 3040, 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 448 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-1": { 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1495, 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1483, 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 1808988, 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 574739, 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 4, 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 3, 2024-04-26T18:46:06.948 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.185692208, 2024-04-26T18:46:06.949 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.055504788, 2024-04-26T18:46:06.949 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.070935372, 2024-04-26T18:46:06.949 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.055712021, 2024-04-26T18:46:06.949 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:46:06.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1482, 2024-04-26T18:46:06.949 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.059080756, 2024-04-26T18:46:06.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000039865 2024-04-26T18:46:06.949 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.949 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:46:06.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.949 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.949 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.949 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.950 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 1541072, 2024-04-26T18:46:06.950 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 228432 2024-04-26T18:46:06.950 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.950 INFO:teuthology.orchestra.run.smithi174.stdout: "AsyncMessenger::Worker-2": { 2024-04-26T18:46:06.950 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_messages": 1288, 2024-04-26T18:46:06.950 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages": 1561, 2024-04-26T18:46:06.950 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_bytes": 432416, 2024-04-26T18:46:06.950 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_bytes": 1046804, 2024-04-26T18:46:06.950 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_created_connections": 6, 2024-04-26T18:46:06.950 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_active_connections": 4, 2024-04-26T18:46:06.950 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_total_time": 0.172750097, 2024-04-26T18:46:06.950 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_send_time": 0.058248377, 2024-04-26T18:46:06.951 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_recv_time": 0.042682480, 2024-04-26T18:46:06.951 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_running_fast_dispatch_time": 0.053681739, 2024-04-26T18:46:06.951 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_messages_queue_lat": { 2024-04-26T18:46:06.951 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 1558, 2024-04-26T18:46:06.951 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.095292142, 2024-04-26T18:46:06.951 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000061163 2024-04-26T18:46:06.951 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.951 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_handle_ack_lat": { 2024-04-26T18:46:06.951 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.951 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.951 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.951 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.951 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: "msgr_send_encrypted_bytes": 637856 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: "cct": { 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: "total_workers": 1, 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: "unhealthy_workers": 0 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-MDSRank": { 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 6131, 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.190152324, 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000031014 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.952 INFO:teuthology.orchestra.run.smithi174.stdout: "finisher-PurgeQueue": { 2024-04-26T18:46:06.953 INFO:teuthology.orchestra.run.smithi174.stdout: "queue_len": 0, 2024-04-26T18:46:06.953 INFO:teuthology.orchestra.run.smithi174.stdout: "complete_latency": { 2024-04-26T18:46:06.953 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.953 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.953 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.953 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:06.953 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.953 INFO:teuthology.orchestra.run.smithi174.stdout: "mds": { 2024-04-26T18:46:06.953 INFO:teuthology.orchestra.run.smithi174.stdout: "request": 0, 2024-04-26T18:46:06.953 INFO:teuthology.orchestra.run.smithi174.stdout: "reply": 0, 2024-04-26T18:46:06.953 INFO:teuthology.orchestra.run.smithi174.stdout: "reply_latency": { 2024-04-26T18:46:06.953 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.953 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.954 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.954 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.954 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_reply": 0, 2024-04-26T18:46:06.954 INFO:teuthology.orchestra.run.smithi174.stdout: "forward": 0, 2024-04-26T18:46:06.954 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_complete": 0, 2024-04-26T18:46:06.954 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_fetch_keys": 0, 2024-04-26T18:46:06.954 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_commit": 0, 2024-04-26T18:46:06.954 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_split": 0, 2024-04-26T18:46:06.954 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_merge": 0, 2024-04-26T18:46:06.954 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes": 0, 2024-04-26T18:46:06.954 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_top": 0, 2024-04-26T18:46:06.954 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_bottom": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pin_tail": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_pinned": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_expired": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "inodes_with_caps": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "caps": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "subtrees": 1, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_hit": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_forward": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_discover": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_dir_fetch": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_remote_ino": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "traverse_lock": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "load_cent": 0, 2024-04-26T18:46:06.955 INFO:teuthology.orchestra.run.smithi174.stdout: "q": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "exported": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "exported_inodes": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "imported": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "imported_inodes": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_dir_fetch": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_backtrace_fetch": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "openino_peer_discover": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rfiles": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rbytes": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "root_rsnaps": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_fetch": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_set_tag": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_backtrace_repaired": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_inotable_repaired": 0, 2024-04-26T18:46:06.956 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_inodes": 0, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dir_base_inodes": 0, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_dirfrag_rstats": 0, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "scrub_file_inodes": 0, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_inode_file_caps": 0, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_revoke": 0, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_grant": 0, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_trunc": 0, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps": 0, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_caps_dirty": 0, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_cap_release": 0, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "process_request_cap_release": 0 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_cache": { 2024-04-26T18:46:06.957 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_update_receipt": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_try_discover": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_send_discover": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "dir_handle_discover": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_delayed": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "num_strays_enqueuing": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_created": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_enqueued": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_reintegrated": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "strays_migrated": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_processing": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_enqueued": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "num_recovering_prioritized": 0, 2024-04-26T18:46:06.958 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_started": 0, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "recovery_completed": 0, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_path": 0, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_quiesce_inode": 0, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_enqueue_scrub": 0, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_exportdir": 0, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_flush": 0, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragmentdir": 0, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_fragstats": 0, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "ireq_inodestats": 0 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_log": { 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "evlrg": 0, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "evadd": 0, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "evex": 0, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "evtrm": 0, 2024-04-26T18:46:06.959 INFO:teuthology.orchestra.run.smithi174.stdout: "ev": 7, 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "evexg": 0, 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "evexd": 0, 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "segadd": 0, 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "segex": 0, 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "segtrm": 0, 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "seg": 2, 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "segmjr": 2, 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "segexg": 0, 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "segexd": 0, 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "expos": 4194304, 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "wrpos": 4196347, 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "rdpos": 4196206, 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "jlat": { 2024-04-26T18:46:06.960 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "replayed": 7 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_mem": { 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "ino": 3, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "ino+": 3, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "ino-": 0, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "dir": 1, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "dir+": 1, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "dir-": 0, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "dn": 0, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "dn+": 0, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "dn-": 0, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "cap": 0, 2024-04-26T18:46:06.961 INFO:teuthology.orchestra.run.smithi174.stdout: "cap+": 0, 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: "cap-": 0, 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: "rss": 51064, 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: "heap": 215416 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_server": { 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_client_request": 0, 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: "dispatch_server_request": 0, 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_request": 0, 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_client_session": 0, 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: "handle_peer_request": 0, 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: "req_create_latency": { 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.962 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.963 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getattr_latency": { 2024-04-26T18:46:06.963 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.963 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.963 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.963 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.963 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getfilelock_latency": { 2024-04-26T18:46:06.963 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.963 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.963 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.963 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.963 INFO:teuthology.orchestra.run.smithi174.stdout: "req_link_latency": { 2024-04-26T18:46:06.963 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.963 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookup_latency": { 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookuphash_latency": { 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupino_latency": { 2024-04-26T18:46:06.964 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupname_latency": { 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupparent_latency": { 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lookupsnap_latency": { 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.965 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.966 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.966 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.966 INFO:teuthology.orchestra.run.smithi174.stdout: "req_lssnap_latency": { 2024-04-26T18:46:06.966 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.966 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.966 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.966 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.966 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mkdir_latency": { 2024-04-26T18:46:06.966 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.967 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.967 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.967 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.967 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mknod_latency": { 2024-04-26T18:46:06.967 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.967 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.967 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.967 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.967 INFO:teuthology.orchestra.run.smithi174.stdout: "req_mksnap_latency": { 2024-04-26T18:46:06.967 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.967 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: "req_open_latency": { 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: "req_readdir_latency": { 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rename_latency": { 2024-04-26T18:46:06.968 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: "req_renamesnap_latency": { 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: "req_snapdiff_latency": { 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.969 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmdir_latency": { 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmsnap_latency": { 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: "req_rmxattr_latency": { 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.970 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setattr_latency": { 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setdirlayout_latency": { 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setfilelock_latency": { 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.971 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setlayout_latency": { 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: "req_setxattr_latency": { 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: "req_symlink_latency": { 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.972 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: "req_unlink_latency": { 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_revoke_eviction": 0, 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: "cap_acquisition_throttle": 0, 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: "req_getvxattr_latency": { 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:06.973 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_sessions": { 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: "session_count": 1, 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: "session_add": 3, 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: "session_remove": 2, 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_open": 3, 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: "sessions_stale": 0, 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: "total_load": 0, 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: "average_load": 0, 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: "avg_session_uptime": 4848, 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: "mdthresh_evicted": 0 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: "mempool": { 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_bytes": 0, 2024-04-26T18:46:06.974 INFO:teuthology.orchestra.run.smithi174.stdout: "bloom_filter_items": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_bytes": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_alloc_items": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_bytes": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_data_items": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_onode_items": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_meta_items": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_bytes": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_other_items": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_cache_buffer_items": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_bytes": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_extent_items": 0, 2024-04-26T18:46:06.975 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_bytes": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_blob_items": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_shared_blob_items": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_inline_bl_items": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_bytes": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_fsck_items": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_bytes": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_txc_items": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_deferred_items": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_bytes": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluestore_writing_items": 0, 2024-04-26T18:46:06.976 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_bytes": 0, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_items": 0, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_bytes": 0, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_reader_items": 0, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_bytes": 0, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "bluefs_file_writer_items": 0, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_bytes": 190961, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_anon_items": 55, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_bytes": 0, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "buffer_meta_items": 0, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_bytes": 0, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_items": 0, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_bytes": 0, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_mapbl_items": 0, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_bytes": 0, 2024-04-26T18:46:06.977 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_pglog_items": 0, 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_bytes": 8232, 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_items": 172, 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_bytes": 0, 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "osdmap_mapping_items": 0, 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_bytes": 0, 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "pgmap_items": 0, 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_bytes": 8512, 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "mds_co_items": 13, 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_bytes": 0, 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_1_items": 0, 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_bytes": 0, 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "unittest_2_items": 0 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "objecter": { 2024-04-26T18:46:06.978 INFO:teuthology.orchestra.run.smithi174.stdout: "op_active": 0, 2024-04-26T18:46:06.979 INFO:teuthology.orchestra.run.smithi174.stdout: "op_laggy": 0, 2024-04-26T18:46:06.979 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send": 3668, 2024-04-26T18:46:06.979 INFO:teuthology.orchestra.run.smithi174.stdout: "op_send_bytes": 21, 2024-04-26T18:46:06.979 INFO:teuthology.orchestra.run.smithi174.stdout: "op_resend": 3, 2024-04-26T18:46:06.979 INFO:teuthology.orchestra.run.smithi174.stdout: "op_reply": 3668, 2024-04-26T18:46:06.979 INFO:teuthology.orchestra.run.smithi174.stdout: "op_latency": { 2024-04-26T18:46:06.979 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3668, 2024-04-26T18:46:06.979 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 2.156973693, 2024-04-26T18:46:06.979 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000588051 2024-04-26T18:46:06.979 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.979 INFO:teuthology.orchestra.run.smithi174.stdout: "op_inflight": 0, 2024-04-26T18:46:06.979 INFO:teuthology.orchestra.run.smithi174.stdout: "oplen_avg": { 2024-04-26T18:46:06.979 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 3668, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 3669 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "op": 3668, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "op_r": 3668, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "op_w": 0, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "op_rmw": 0, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "op_pg": 0, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_stat": 2373, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_create": 0, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_read": 1293, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_write": 0, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writefull": 0, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_writesame": 0, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_append": 0, 2024-04-26T18:46:06.980 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_zero": 0, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_truncate": 0, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_delete": 0, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_mapext": 0, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_sparse_read": 0, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_clonerange": 0, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_getxattr": 1, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_setxattr": 0, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_cmpxattr": 0, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_rmxattr": 0, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_resetxattrs": 0, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_call": 0, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_watch": 0, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_notify": 0, 2024-04-26T18:46:06.981 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_src_cmpxattr": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_pgls_filter": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "osdop_other": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_active": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_send": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_resend": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "linger_ping": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_active": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_send": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "poolop_resend": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_active": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_send": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "poolstat_resend": 0, 2024-04-26T18:46:06.982 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_active": 0, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_send": 0, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "statfs_resend": 0, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "command_active": 0, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "command_send": 0, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "command_resend": 0, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "map_epoch": 118, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "map_full": 0, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "map_inc": 29, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_sessions": 5, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_open": 6, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_session_close": 1, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "osd_laggy": 0, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_wr": 0, 2024-04-26T18:46:06.983 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_rd": 2, 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_del": 0 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: "oft": { 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_objs": 0, 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_kv_pairs": 0, 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_updates": 0, 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: "omap_total_removes": 0 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: "osd-slow-ops": { 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: "slow_ops_count": 0 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: "purge_queue": { 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops": 0, 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_ops_high_water": 0, 2024-04-26T18:46:06.984 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing": 0, 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executing_high_water": 0, 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed_ops": 0, 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_executed": 0, 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "pq_item_in_journal": 0 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 4076, 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 2195144, 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 4076, 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-04-26T18:46:06.985 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 4076, 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2195144, 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_bytes": { 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 104857600, 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:46:06.986 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 3668, 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 2048, 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 3668, 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 2048, 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-objecter_ops": { 2024-04-26T18:46:06.987 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:06.988 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 1024, 2024-04-26T18:46:06.988 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:06.988 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:46:06.988 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:46:06.988 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:06.988 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:46:06.988 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 3668, 2024-04-26T18:46:06.988 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 3668, 2024-04-26T18:46:06.988 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 3668, 2024-04-26T18:46:06.988 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 3668, 2024-04-26T18:46:06.988 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:06.988 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.988 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle": { 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-04-26T18:46:06.989 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: }, 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: "throttle-write_buf_throttle-0x5563752e46e0": { 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: "val": 0, 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: "max": 3758096384, 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: "get_started": 0, 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: "get": 0, 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: "get_sum": 0, 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_fail": 0, 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: "get_or_fail_success": 0, 2024-04-26T18:46:06.990 INFO:teuthology.orchestra.run.smithi174.stdout: "take": 0, 2024-04-26T18:46:06.991 INFO:teuthology.orchestra.run.smithi174.stdout: "take_sum": 0, 2024-04-26T18:46:06.991 INFO:teuthology.orchestra.run.smithi174.stdout: "put": 0, 2024-04-26T18:46:06.991 INFO:teuthology.orchestra.run.smithi174.stdout: "put_sum": 0, 2024-04-26T18:46:06.991 INFO:teuthology.orchestra.run.smithi174.stdout: "wait": { 2024-04-26T18:46:06.991 INFO:teuthology.orchestra.run.smithi174.stdout: "avgcount": 0, 2024-04-26T18:46:06.991 INFO:teuthology.orchestra.run.smithi174.stdout: "sum": 0.000000000, 2024-04-26T18:46:06.991 INFO:teuthology.orchestra.run.smithi174.stdout: "avgtime": 0.000000000 2024-04-26T18:46:06.991 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:06.991 INFO:teuthology.orchestra.run.smithi174.stdout: } 2024-04-26T18:46:06.991 INFO:teuthology.orchestra.run.smithi174.stdout:} 2024-04-26T18:46:07.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:06 smithi084 ceph-mon[29141]: pgmap v571: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T18:46:07.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:06 smithi103 ceph-mon[32727]: pgmap v571: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T18:46:07.801 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-26T18:46:07.801 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-26T18:46:07.801 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-26T18:46:07.801 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=0 2024-04-26T18:46:07.801 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-04-26T18:46:07.801 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-26T18:46:07.801 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-26T18:46:07.801 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T18:46:07.812 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T18:46:07.812 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:46:07.812 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T18:46:07.833 DEBUG:teuthology.orchestra.run.smithi084:> set -ex 2024-04-26T18:46:07.833 DEBUG:teuthology.orchestra.run.smithi084:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T18:46:07.892 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd blocklist ls 2024-04-26T18:46:08.184 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:46:09.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:08 smithi103 ceph-mon[32727]: pgmap v572: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T18:46:09.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:08 smithi174 ceph-mon[32705]: pgmap v572: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T18:46:09.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:08 smithi084 ceph-mon[29141]: pgmap v572: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 45 KiB/s wr, 25 op/s 2024-04-26T18:46:09.941 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:09 smithi084 ceph-mon[29141]: osdmap e119: 12 total, 12 up, 12 in 2024-04-26T18:46:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.045+0000 7f7cc607f700 1 -- 172.21.15.84:0/2580000757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cc0102810 msgr2=0x7f7cc0110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.045+0000 7f7cc607f700 1 --2- 172.21.15.84:0/2580000757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cc0102810 0x7f7cc0110310 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800b0a0 tx=0x7f7cb800b3b0 comp rx=0 tx=0).stop 2024-04-26T18:46:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.046+0000 7f7cc607f700 1 -- 172.21.15.84:0/2580000757 shutdown_connections 2024-04-26T18:46:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.046+0000 7f7cc607f700 1 --2- 172.21.15.84:0/2580000757 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cc0102810 0x7f7cc0110310 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.046+0000 7f7cc607f700 1 --2- 172.21.15.84:0/2580000757 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cc0109600 0x7f7cc01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.046+0000 7f7cc607f700 1 --2- 172.21.15.84:0/2580000757 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cc0101e50 0x7f7cc01022d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.046+0000 7f7cc607f700 1 -- 172.21.15.84:0/2580000757 >> 172.21.15.84:0/2580000757 conn(0x7f7cc0077590 msgr2=0x7f7cc00779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.046+0000 7f7cc607f700 1 -- 172.21.15.84:0/2580000757 shutdown_connections 2024-04-26T18:46:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.046+0000 7f7cc607f700 1 -- 172.21.15.84:0/2580000757 wait complete. 2024-04-26T18:46:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.046+0000 7f7cc607f700 1 Processor -- start 2024-04-26T18:46:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.046+0000 7f7cc607f700 1 -- start start 2024-04-26T18:46:10.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cc607f700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cc0101e50 0x7f7cc019f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cc607f700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cc0102810 0x7f7cc019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cc607f700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cc0109600 0x7f7cc0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cc607f700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc01136b0 con 0x7f7cc0101e50 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cc607f700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc0113530 con 0x7f7cc0109600 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cc607f700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc0113830 con 0x7f7cc0102810 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbf7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cc0101e50 0x7f7cc019f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbf7fe700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cc0101e50 0x7f7cc019f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55400/0 (socket says 172.21.15.84:55400) 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbf7fe700 1 -- 172.21.15.84:0/2076997593 learned_addr learned my addr 172.21.15.84:0/2076997593 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbeffd700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cc0102810 0x7f7cc019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbffff700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cc0109600 0x7f7cc0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbf7fe700 1 -- 172.21.15.84:0/2076997593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cc0102810 msgr2=0x7f7cc019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbf7fe700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cc0102810 0x7f7cc019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbf7fe700 1 -- 172.21.15.84:0/2076997593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cc0109600 msgr2=0x7f7cc0199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbf7fe700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cc0109600 0x7f7cc0199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:10.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbf7fe700 1 -- 172.21.15.84:0/2076997593 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc019a0f0 con 0x7f7cc0101e50 2024-04-26T18:46:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbf7fe700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cc0101e50 0x7f7cc019f7d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400b530 tx=0x7f7cb400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbcff9700 1 -- 172.21.15.84:0/2076997593 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4015020 con 0x7f7cc0101e50 2024-04-26T18:46:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cc607f700 1 -- 172.21.15.84:0/2076997593 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc019a3e0 con 0x7f7cc0101e50 2024-04-26T18:46:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbcff9700 1 -- 172.21.15.84:0/2076997593 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb4011ce0 con 0x7f7cc0101e50 2024-04-26T18:46:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cbcff9700 1 -- 172.21.15.84:0/2076997593 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4010e40 con 0x7f7cc0101e50 2024-04-26T18:46:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.047+0000 7f7cc607f700 1 -- 172.21.15.84:0/2076997593 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc01a66a0 con 0x7f7cc0101e50 2024-04-26T18:46:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.048+0000 7f7cc607f700 1 -- 172.21.15.84:0/2076997593 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc010ba40 con 0x7f7cc0101e50 2024-04-26T18:46:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.050+0000 7f7cbcff9700 1 -- 172.21.15.84:0/2076997593 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f7cb4018020 con 0x7f7cc0101e50 2024-04-26T18:46:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.050+0000 7f7cbcff9700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7ca80a3db0 0x7f7ca80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.050+0000 7f7cbeffd700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7ca80a3db0 0x7f7ca80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.050+0000 7f7cbcff9700 1 -- 172.21.15.84:0/2076997593 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8623+0+0 (secure 0 0 0) 0x7f7cb40d0580 con 0x7f7cc0101e50 2024-04-26T18:46:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.051+0000 7f7cbeffd700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7ca80a3db0 0x7f7ca80a6270 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0077b90 tx=0x7f7cb0009190 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:10.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.054+0000 7f7cbcff9700 1 -- 172.21.15.84:0/2076997593 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7cb401f020 con 0x7f7cc0101e50 2024-04-26T18:46:10.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:09 smithi103 ceph-mon[32727]: osdmap e119: 12 total, 12 up, 12 in 2024-04-26T18:46:10.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:09 smithi174 ceph-mon[32705]: osdmap e119: 12 total, 12 up, 12 in 2024-04-26T18:46:10.266 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.266+0000 7f7cc607f700 1 -- 172.21.15.84:0/2076997593 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7cc006a110 con 0x7f7cc0101e50 2024-04-26T18:46:10.267 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.267+0000 7f7cbcff9700 1 -- 172.21.15.84:0/2076997593 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v119) ==== 81+0+942 (secure 0 0 0) 0x7f7cb409ccd0 con 0x7f7cc0101e50 2024-04-26T18:46:10.267 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/277477867 2024-04-27T18:27:57.444610+0000 2024-04-26T18:46:10.267 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/559313476 2024-04-27T18:27:57.444610+0000 2024-04-26T18:46:10.267 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6800/2059208901 2024-04-27T18:27:57.444610+0000 2024-04-26T18:46:10.267 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3834640430 2024-04-27T18:27:57.444610+0000 2024-04-26T18:46:10.267 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/2142936585 2024-04-27T18:27:57.444610+0000 2024-04-26T18:46:10.267 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1604680842 2024-04-27T18:17:05.083192+0000 2024-04-26T18:46:10.267 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6801/3919933286 2024-04-27T18:17:05.083192+0000 2024-04-26T18:46:10.267 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6800/3919933286 2024-04-27T18:17:05.083192+0000 2024-04-26T18:46:10.267 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6801/2059208901 2024-04-27T18:27:57.444610+0000 2024-04-26T18:46:10.268 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6801/1448727861 2024-04-27T18:16:37.859669+0000 2024-04-26T18:46:10.268 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/538363975 2024-04-27T18:16:37.859669+0000 2024-04-26T18:46:10.268 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/952052990 2024-04-27T18:17:05.083192+0000 2024-04-26T18:46:10.268 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/1266942705 2024-04-27T18:16:37.859669+0000 2024-04-26T18:46:10.268 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:6800/1448727861 2024-04-27T18:16:37.859669+0000 2024-04-26T18:46:10.268 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3970542149 2024-04-27T18:16:37.859669+0000 2024-04-26T18:46:10.268 INFO:teuthology.orchestra.run.smithi084.stdout:172.21.15.84:0/3607125049 2024-04-27T18:17:05.083192+0000 2024-04-26T18:46:10.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.269+0000 7f7cc607f700 1 -- 172.21.15.84:0/2076997593 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7ca80a3db0 msgr2=0x7f7ca80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:10.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.269+0000 7f7cc607f700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7ca80a3db0 0x7f7ca80a6270 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7cc0077b90 tx=0x7f7cb0009190 comp rx=0 tx=0).stop 2024-04-26T18:46:10.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.269+0000 7f7cc607f700 1 -- 172.21.15.84:0/2076997593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cc0101e50 msgr2=0x7f7cc019f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:10.269 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.269+0000 7f7cc607f700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cc0101e50 0x7f7cc019f7d0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400b530 tx=0x7f7cb400b840 comp rx=0 tx=0).stop 2024-04-26T18:46:10.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.269+0000 7f7cc607f700 1 -- 172.21.15.84:0/2076997593 shutdown_connections 2024-04-26T18:46:10.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.269+0000 7f7cc607f700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f7ca80a3db0 0x7f7ca80a6270 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:10.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.269+0000 7f7cc607f700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f7cc0101e50 0x7f7cc019f7d0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:10.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.269+0000 7f7cc607f700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f7cc0109600 0x7f7cc0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:10.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.269+0000 7f7cc607f700 1 --2- 172.21.15.84:0/2076997593 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f7cc0102810 0x7f7cc019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:10.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.269+0000 7f7cc607f700 1 -- 172.21.15.84:0/2076997593 >> 172.21.15.84:0/2076997593 conn(0x7f7cc0077590 msgr2=0x7f7cc0100ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:10.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.269+0000 7f7cc607f700 1 -- 172.21.15.84:0/2076997593 shutdown_connections 2024-04-26T18:46:10.270 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:10.269+0000 7f7cc607f700 1 -- 172.21.15.84:0/2076997593 wait complete. 2024-04-26T18:46:10.270 INFO:teuthology.orchestra.run.smithi084.stderr:listed 16 entries 2024-04-26T18:46:10.909 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T18:46:10.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:46:10.909 DEBUG:teuthology.orchestra.run.smithi084:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T18:46:10.951 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi084.front.sepia.ceph.com 2024-04-26T18:46:10.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:46:10.952 DEBUG:teuthology.orchestra.run.smithi084:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T18:46:11.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:46:11.013 DEBUG:teuthology.orchestra.run.smithi084:> ip netns list 2024-04-26T18:46:11.069 INFO:teuthology.orchestra.run.smithi084.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T18:46:11.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:46:11.069 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T18:46:11.126 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:10 smithi103 ceph-mon[32727]: pgmap v574: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T18:46:11.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:10 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/2076997593' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T18:46:11.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:10 smithi084 ceph-mon[29141]: pgmap v574: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T18:46:11.129 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:10 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/2076997593' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T18:46:11.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T18:46:11.133 DEBUG:teuthology.orchestra.run.smithi084:> sudo ip link delete ceph-brx 2024-04-26T18:46:11.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:10 smithi174 ceph-mon[32705]: pgmap v574: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T18:46:11.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:10 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/2076997593' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T18:46:11.234 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T18:46:11.247 INFO:tasks.cephadm:Teardown begin 2024-04-26T18:46:11.248 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-26T18:46:11.248 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:46:11.499 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:46:13.001 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:12 smithi084 ceph-mon[29141]: pgmap v575: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T18:46:13.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:12 smithi103 ceph-mon[32727]: pgmap v575: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T18:46:13.188 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:12 smithi174 ceph-mon[32705]: pgmap v575: 97 pgs: 97 active+clean; 4.8 MiB data, 456 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 54 KiB/s wr, 30 op/s 2024-04-26T18:46:13.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.236+0000 7ff3c299d700 1 -- 172.21.15.84:0/210373398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc06c580 msgr2=0x7ff3bc1102b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:13.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.236+0000 7ff3c299d700 1 --2- 172.21.15.84:0/210373398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc06c580 0x7ff3bc1102b0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7ff3b400b0d0 tx=0x7ff3b400b3e0 comp rx=0 tx=0).stop 2024-04-26T18:46:13.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.236+0000 7ff3c299d700 1 -- 172.21.15.84:0/210373398 shutdown_connections 2024-04-26T18:46:13.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.236+0000 7ff3c299d700 1 --2- 172.21.15.84:0/210373398 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc06c580 0x7ff3bc1102b0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:13.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.236+0000 7ff3c299d700 1 --2- 172.21.15.84:0/210373398 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3bc109490 0x7ff3bc109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:13.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.236+0000 7ff3c299d700 1 --2- 172.21.15.84:0/210373398 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc06bbc0 0x7ff3bc06c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:13.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.236+0000 7ff3c299d700 1 -- 172.21.15.84:0/210373398 >> 172.21.15.84:0/210373398 conn(0x7ff3bc078390 msgr2=0x7ff3bc0787a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:13.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.236+0000 7ff3c299d700 1 -- 172.21.15.84:0/210373398 shutdown_connections 2024-04-26T18:46:13.238 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.236+0000 7ff3c299d700 1 -- 172.21.15.84:0/210373398 wait complete. 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.236+0000 7ff3c299d700 1 Processor -- start 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.236+0000 7ff3c299d700 1 -- start start 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c299d700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc06bbc0 0x7ff3bc19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c299d700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3bc06c580 0x7ff3bc19fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c299d700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc109490 0x7ff3bc1998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c299d700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc1135f0 con 0x7ff3bc109490 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c299d700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc113470 con 0x7ff3bc06c580 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c299d700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7ff3bc113770 con 0x7ff3bc06bbc0 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c0f3a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc109490 0x7ff3bc1998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c0f3a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc109490 0x7ff3bc1998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55426/0 (socket says 172.21.15.84:55426) 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c0f3a700 1 -- 172.21.15.84:0/3432923235 learned_addr learned my addr 172.21.15.84:0/3432923235 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3bbfff700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc06bbc0 0x7ff3bc19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3bb7fe700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3bc06c580 0x7ff3bc19fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c0f3a700 1 -- 172.21.15.84:0/3432923235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc06bbc0 msgr2=0x7ff3bc19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:13.239 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c0f3a700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc06bbc0 0x7ff3bc19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c0f3a700 1 -- 172.21.15.84:0/3432923235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3bc06c580 msgr2=0x7ff3bc19fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c0f3a700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3bc06c580 0x7ff3bc19fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c0f3a700 1 -- 172.21.15.84:0/3432923235 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3bc19a030 con 0x7ff3bc109490 2024-04-26T18:46:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.237+0000 7ff3c0f3a700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc109490 0x7ff3bc1998d0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009fd0 tx=0x7ff3b4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.238+0000 7ff3b97fa700 1 -- 172.21.15.84:0/3432923235 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b401f070 con 0x7ff3bc109490 2024-04-26T18:46:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.238+0000 7ff3b97fa700 1 -- 172.21.15.84:0/3432923235 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3b4007e50 con 0x7ff3bc109490 2024-04-26T18:46:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.238+0000 7ff3b97fa700 1 -- 172.21.15.84:0/3432923235 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3b401b6a0 con 0x7ff3bc109490 2024-04-26T18:46:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.238+0000 7ff3c299d700 1 -- 172.21.15.84:0/3432923235 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3bc19a2c0 con 0x7ff3bc109490 2024-04-26T18:46:13.240 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.238+0000 7ff3c299d700 1 -- 172.21.15.84:0/3432923235 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3bc1a6590 con 0x7ff3bc109490 2024-04-26T18:46:13.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.239+0000 7ff3c299d700 1 -- 172.21.15.84:0/3432923235 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3bc06a110 con 0x7ff3bc109490 2024-04-26T18:46:13.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.240+0000 7ff3b97fa700 1 -- 172.21.15.84:0/3432923235 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7ff3b4022070 con 0x7ff3bc109490 2024-04-26T18:46:13.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.240+0000 7ff3b97fa700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff3a40a3e30 0x7ff3a40a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:13.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.240+0000 7ff3bbfff700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff3a40a3e30 0x7ff3a40a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:13.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.240+0000 7ff3b97fa700 1 -- 172.21.15.84:0/3432923235 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8623+0+0 (secure 0 0 0) 0x7ff3b400f020 con 0x7ff3bc109490 2024-04-26T18:46:13.241 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.241+0000 7ff3bbfff700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff3a40a3e30 0x7ff3a40a62f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac005950 tx=0x7ff3ac00a300 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:13.245 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.245+0000 7ff3b97fa700 1 -- 172.21.15.84:0/3432923235 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff3b4029020 con 0x7ff3bc109490 2024-04-26T18:46:13.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.488+0000 7ff3c299d700 1 -- 172.21.15.84:0/3432923235 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3bc19adb0 con 0x7ff3bc109490 2024-04-26T18:46:13.489 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.489+0000 7ff3b97fa700 1 -- 172.21.15.84:0/3432923235 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7ff3b409e070 con 0x7ff3bc109490 2024-04-26T18:46:13.489 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:46:13.490 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:46:13.491 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.491+0000 7ff3c299d700 1 -- 172.21.15.84:0/3432923235 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff3a40a3e30 msgr2=0x7ff3a40a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.491+0000 7ff3c299d700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff3a40a3e30 0x7ff3a40a62f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac005950 tx=0x7ff3ac00a300 comp rx=0 tx=0).stop 2024-04-26T18:46:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.491+0000 7ff3c299d700 1 -- 172.21.15.84:0/3432923235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc109490 msgr2=0x7ff3bc1998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.491+0000 7ff3c299d700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc109490 0x7ff3bc1998d0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff3b4009fd0 tx=0x7ff3b4007a20 comp rx=0 tx=0).stop 2024-04-26T18:46:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.491+0000 7ff3c299d700 1 -- 172.21.15.84:0/3432923235 shutdown_connections 2024-04-26T18:46:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.491+0000 7ff3c299d700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7ff3a40a3e30 0x7ff3a40a62f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.491+0000 7ff3c299d700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7ff3bc109490 0x7ff3bc1998d0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.491+0000 7ff3c299d700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7ff3bc06bbc0 0x7ff3bc19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.491+0000 7ff3c299d700 1 --2- 172.21.15.84:0/3432923235 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7ff3bc06c580 0x7ff3bc19fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.491+0000 7ff3c299d700 1 -- 172.21.15.84:0/3432923235 >> 172.21.15.84:0/3432923235 conn(0x7ff3bc078390 msgr2=0x7ff3bc101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.491+0000 7ff3c299d700 1 -- 172.21.15.84:0/3432923235 shutdown_connections 2024-04-26T18:46:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:13.491+0000 7ff3c299d700 1 -- 172.21.15.84:0/3432923235 wait complete. 2024-04-26T18:46:13.492 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:46:14.094 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:13 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3432923235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:46:14.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:13 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3432923235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:46:14.161 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs dump --format=json 2024-04-26T18:46:14.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:13 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3432923235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:46:14.397 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:46:15.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:14 smithi103 ceph-mon[32727]: pgmap v576: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-04-26T18:46:15.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:14 smithi174 ceph-mon[32705]: pgmap v576: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-04-26T18:46:15.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:14 smithi084 ceph-mon[29141]: pgmap v576: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-04-26T18:46:16.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.064+0000 7f0e57ad9700 1 -- 172.21.15.84:0/738210882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e50103470 msgr2=0x7f0e501038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:16.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.064+0000 7f0e57ad9700 1 --2- 172.21.15.84:0/738210882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e50103470 0x7f0e501038f0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f0e40009990 tx=0x7f0e40009ca0 comp rx=0 tx=0).stop 2024-04-26T18:46:16.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.064+0000 7f0e57ad9700 1 -- 172.21.15.84:0/738210882 shutdown_connections 2024-04-26T18:46:16.067 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.064+0000 7f0e57ad9700 1 --2- 172.21.15.84:0/738210882 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e50103e30 0x7f0e50110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.064+0000 7f0e57ad9700 1 --2- 172.21.15.84:0/738210882 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e501094c0 0x7f0e501098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.064+0000 7f0e57ad9700 1 --2- 172.21.15.84:0/738210882 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e50103470 0x7f0e501038f0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.064+0000 7f0e57ad9700 1 -- 172.21.15.84:0/738210882 >> 172.21.15.84:0/738210882 conn(0x7f0e5007a580 msgr2=0x7f0e5007a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.064+0000 7f0e57ad9700 1 -- 172.21.15.84:0/738210882 shutdown_connections 2024-04-26T18:46:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.064+0000 7f0e57ad9700 1 -- 172.21.15.84:0/738210882 wait complete. 2024-04-26T18:46:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.064+0000 7f0e57ad9700 1 Processor -- start 2024-04-26T18:46:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.064+0000 7f0e57ad9700 1 -- start start 2024-04-26T18:46:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e57ad9700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e50103470 0x7f0e501a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e57ad9700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e50103e30 0x7f0e501a5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e57ad9700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e501094c0 0x7f0e5010f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e57ad9700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f0e501135e0 con 0x7f0e501094c0 2024-04-26T18:46:16.068 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e57ad9700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f0e50113460 con 0x7f0e50103470 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e57ad9700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f0e50113760 con 0x7f0e50103e30 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e56076700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e501094c0 0x7f0e5010f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e56076700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e501094c0 0x7f0e5010f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:55448/0 (socket says 172.21.15.84:55448) 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e56076700 1 -- 172.21.15.84:0/1161878326 learned_addr learned my addr 172.21.15.84:0/1161878326 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e55074700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e50103e30 0x7f0e501a5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e55875700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e50103470 0x7f0e501a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e56076700 1 -- 172.21.15.84:0/1161878326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e50103e30 msgr2=0x7f0e501a5280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e56076700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e50103e30 0x7f0e501a5280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e56076700 1 -- 172.21.15.84:0/1161878326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e50103470 msgr2=0x7f0e501a4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e56076700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e50103470 0x7f0e501a4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e56076700 1 -- 172.21.15.84:0/1161878326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e5010fd70 con 0x7f0e501094c0 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e55875700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e50103470 0x7f0e501a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.065+0000 7f0e56076700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e501094c0 0x7f0e5010f4a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0e48007d70 tx=0x7f0e4800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:16.069 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.066+0000 7f0e46ffd700 1 -- 172.21.15.84:0/1161878326 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e48016020 con 0x7f0e501094c0 2024-04-26T18:46:16.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.066+0000 7f0e57ad9700 1 -- 172.21.15.84:0/1161878326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e50110060 con 0x7f0e501094c0 2024-04-26T18:46:16.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.066+0000 7f0e46ffd700 1 -- 172.21.15.84:0/1161878326 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e4800ad40 con 0x7f0e501094c0 2024-04-26T18:46:16.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.066+0000 7f0e46ffd700 1 -- 172.21.15.84:0/1161878326 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e480115e0 con 0x7f0e501094c0 2024-04-26T18:46:16.070 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.066+0000 7f0e57ad9700 1 -- 172.21.15.84:0/1161878326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e501ace40 con 0x7f0e501094c0 2024-04-26T18:46:16.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.067+0000 7f0e57ad9700 1 -- 172.21.15.84:0/1161878326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e50104b60 con 0x7f0e501094c0 2024-04-26T18:46:16.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.068+0000 7f0e46ffd700 1 -- 172.21.15.84:0/1161878326 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f0e48019020 con 0x7f0e501094c0 2024-04-26T18:46:16.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.068+0000 7f0e46ffd700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f0e3c0a3e30 0x7f0e3c0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:16.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.068+0000 7f0e55875700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f0e3c0a3e30 0x7f0e3c0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:16.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.068+0000 7f0e46ffd700 1 -- 172.21.15.84:0/1161878326 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 8623+0+0 (secure 0 0 0) 0x7f0e480d06b0 con 0x7f0e501094c0 2024-04-26T18:46:16.071 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.069+0000 7f0e55875700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f0e3c0a3e30 0x7f0e3c0a62f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c005e10 tx=0x7f0e4c005da0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:16.074 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.073+0000 7f0e46ffd700 1 -- 172.21.15.84:0/1161878326 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0e48020020 con 0x7f0e501094c0 2024-04-26T18:46:16.316 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.315+0000 7f0e57ad9700 1 -- 172.21.15.84:0/1161878326 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0e501a14c0 con 0x7f0e501094c0 2024-04-26T18:46:16.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.315+0000 7f0e46ffd700 1 -- 172.21.15.84:0/1161878326 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11936 (secure 0 0 0) 0x7f0e4809ce00 con 0x7f0e501094c0 2024-04-26T18:46:16.317 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:46:16.317 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6835/1853556606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6834","nonce":1853556606},{"type":"v1","addr":"172.21.15.84:6835","nonce":1853556606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.84:6837/3254529801","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6836","nonce":3254529801},{"type":"v1","addr":"172.21.15.84:6837","nonce":3254529801}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-26T18:24:22.810992+0000","modified":"2024-04-26T18:27:56.147537+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24434,"mds_1":24430,"mds_2":24422,"mds_3":24418,"mds_4":24407},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14598":{"gid":14598,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6839/96876655","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6838","nonce":96876655},{"type":"v1","addr":"172.21.15.84:6839","nonce":96876655}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14610":{"gid":14610,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.84:6841/2504248516","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6840","nonce":2504248516},{"type":"v1","addr":"172.21.15.84:6841","nonce":2504248516}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6833/280377862","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6832","nonce":280377862},{"type":"v1","addr":"172.21.15.174:6833","nonce":280377862}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24389":{"gid":24389,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.103:6833/3711970972","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6832","nonce":3711970972},{"type":"v1","addr":"172.21.15.103:6833","nonce":3711970972}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24406":{"gid":24406,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.174:6835/1310802824","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6834","nonce":1310802824},{"type":"v1","addr":"172.21.15.174:6835","nonce":1310802824}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24407":{"gid":24407,"name":"e","rank":4,"incarnation":25,"state":"up:active","state_seq":14,"addr":"172.21.15.103:6835/2529642104","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6834","nonce":2529642104},{"type":"v1","addr":"172.21.15.103:6835","nonce":2529642104}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24418":{"gid":24418,"name":"c","rank":3,"incarnation":23,"state":"up:active","state_seq":13,"addr":"172.21.15.174:6837/1532309720","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6836","nonce":1532309720},{"type":"v1","addr":"172.21.15.174:6837","nonce":1532309720}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24422":{"gid":24422,"name":"h","rank":2,"incarnation":21,"state":"up:active","state_seq":12,"addr":"172.21.15.103:6837/2271231481","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6836","nonce":2271231481},{"type":"v1","addr":"172.21.15.103:6837","nonce":2271231481}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24430":{"gid":24430,"name":"i","rank":1,"incarnation":19,"state":"up:active","state_seq":11,"addr":"172.21.15.174:6839/518809558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6838","nonce":518809558},{"type":"v1","addr":"172.21.15.174:6839","nonce":518809558}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24434":{"gid":24434,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.103:6839/2465442393","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6838","nonce":2465442393},{"type":"v1","addr":"172.21.15.103:6839","nonce":2465442393}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24434,"qdb_cluster":[24434,24430,24422,24407,24418]},"id":1}]} 2024-04-26T18:46:16.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.318+0000 7f0e57ad9700 1 -- 172.21.15.84:0/1161878326 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f0e3c0a3e30 msgr2=0x7f0e3c0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:16.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.318+0000 7f0e57ad9700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f0e3c0a3e30 0x7f0e3c0a62f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f0e4c005e10 tx=0x7f0e4c005da0 comp rx=0 tx=0).stop 2024-04-26T18:46:16.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.318+0000 7f0e57ad9700 1 -- 172.21.15.84:0/1161878326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e501094c0 msgr2=0x7f0e5010f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:16.319 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.318+0000 7f0e57ad9700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e501094c0 0x7f0e5010f4a0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f0e48007d70 tx=0x7f0e4800a340 comp rx=0 tx=0).stop 2024-04-26T18:46:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.318+0000 7f0e57ad9700 1 -- 172.21.15.84:0/1161878326 shutdown_connections 2024-04-26T18:46:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.318+0000 7f0e57ad9700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f0e3c0a3e30 0x7f0e3c0a62f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.318+0000 7f0e57ad9700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f0e501094c0 0x7f0e5010f4a0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.318+0000 7f0e57ad9700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f0e50103470 0x7f0e501a4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.318+0000 7f0e57ad9700 1 --2- 172.21.15.84:0/1161878326 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f0e50103e30 0x7f0e501a5280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.318+0000 7f0e57ad9700 1 -- 172.21.15.84:0/1161878326 >> 172.21.15.84:0/1161878326 conn(0x7f0e5007a580 msgr2=0x7f0e50102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.318+0000 7f0e57ad9700 1 -- 172.21.15.84:0/1161878326 shutdown_connections 2024-04-26T18:46:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:16.318+0000 7f0e57ad9700 1 -- 172.21.15.84:0/1161878326 wait complete. 2024-04-26T18:46:16.320 INFO:teuthology.orchestra.run.smithi084.stderr:dumped fsmap epoch 29 2024-04-26T18:46:17.089 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T18:46:17.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:16 smithi103 ceph-mon[32727]: pgmap v577: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-04-26T18:46:17.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:16 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1161878326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:46:17.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:16 smithi174 ceph-mon[32705]: pgmap v577: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-04-26T18:46:17.189 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:16 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1161878326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:46:17.328 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:46:17.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:16 smithi084 ceph-mon[29141]: pgmap v577: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-04-26T18:46:17.344 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:16 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1161878326' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T18:46:18.805 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:18 smithi084 ceph-mon[29141]: osdmap e120: 12 total, 12 up, 12 in 2024-04-26T18:46:18.805 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:18 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:46:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:18 smithi103 ceph-mon[32727]: osdmap e120: 12 total, 12 up, 12 in 2024-04-26T18:46:18.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:18 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:46:18.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:18 smithi174 ceph-mon[32705]: osdmap e120: 12 total, 12 up, 12 in 2024-04-26T18:46:18.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:18 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T18:46:19.026 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.023+0000 7f8269dd4700 1 -- 172.21.15.84:0/2963198496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82641094e0 msgr2=0x7f82641098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:19.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.023+0000 7f8269dd4700 1 --2- 172.21.15.84:0/2963198496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82641094e0 0x7f82641098c0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f8254009990 tx=0x7f8254009ca0 comp rx=0 tx=0).stop 2024-04-26T18:46:19.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.023+0000 7f8269dd4700 1 -- 172.21.15.84:0/2963198496 shutdown_connections 2024-04-26T18:46:19.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.023+0000 7f8269dd4700 1 --2- 172.21.15.84:0/2963198496 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f82641094e0 0x7f82641098c0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:19.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.023+0000 7f8269dd4700 1 --2- 172.21.15.84:0/2963198496 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f8264103e60 0x7f82641102b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:19.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.023+0000 7f8269dd4700 1 --2- 172.21.15.84:0/2963198496 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82641034a0 0x7f8264103920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:19.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.023+0000 7f8269dd4700 1 -- 172.21.15.84:0/2963198496 >> 172.21.15.84:0/2963198496 conn(0x7f826407a590 msgr2=0x7f826407a9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:19.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.023+0000 7f8269dd4700 1 -- 172.21.15.84:0/2963198496 shutdown_connections 2024-04-26T18:46:19.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.023+0000 7f8269dd4700 1 -- 172.21.15.84:0/2963198496 wait complete. 2024-04-26T18:46:19.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8269dd4700 1 Processor -- start 2024-04-26T18:46:19.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8269dd4700 1 -- start start 2024-04-26T18:46:19.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8269dd4700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82641034a0 0x7f826419f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:19.027 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8269dd4700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8264103e60 0x7f826419fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:19.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8269dd4700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82641094e0 0x7f8264199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:19.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8269dd4700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f82641135e0 con 0x7f8264103e60 2024-04-26T18:46:19.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8269dd4700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f8264113460 con 0x7f82641034a0 2024-04-26T18:46:19.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8269dd4700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f8264113760 con 0x7f82641094e0 2024-04-26T18:46:19.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8262ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8264103e60 0x7f826419fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:19.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8262ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8264103e60 0x7f826419fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39408/0 (socket says 172.21.15.84:39408) 2024-04-26T18:46:19.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8262ffd700 1 -- 172.21.15.84:0/3788775230 learned_addr learned my addr 172.21.15.84:0/3788775230 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:46:19.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f82637fe700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82641034a0 0x7f826419f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:19.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8263fff700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82641094e0 0x7f8264199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:19.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8262ffd700 1 -- 172.21.15.84:0/3788775230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82641094e0 msgr2=0x7f8264199a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:19.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8262ffd700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82641094e0 0x7f8264199a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:19.028 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8262ffd700 1 -- 172.21.15.84:0/3788775230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82641034a0 msgr2=0x7f826419f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:19.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8262ffd700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82641034a0 0x7f826419f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:19.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8262ffd700 1 -- 172.21.15.84:0/3788775230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f826419a1a0 con 0x7f8264103e60 2024-04-26T18:46:19.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.024+0000 7f8263fff700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82641094e0 0x7f8264199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:46:19.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.025+0000 7f8262ffd700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8264103e60 0x7f826419fdc0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f825800e9a0 tx=0x7f825800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:19.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.025+0000 7f8260ff9700 1 -- 172.21.15.84:0/3788775230 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f825800cbd0 con 0x7f8264103e60 2024-04-26T18:46:19.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.025+0000 7f8260ff9700 1 -- 172.21.15.84:0/3788775230 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f825800cd70 con 0x7f8264103e60 2024-04-26T18:46:19.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.025+0000 7f8269dd4700 1 -- 172.21.15.84:0/3788775230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f826419a490 con 0x7f8264103e60 2024-04-26T18:46:19.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.025+0000 7f8260ff9700 1 -- 172.21.15.84:0/3788775230 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8258018680 con 0x7f8264103e60 2024-04-26T18:46:19.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.025+0000 7f8269dd4700 1 -- 172.21.15.84:0/3788775230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82641a6750 con 0x7f8264103e60 2024-04-26T18:46:19.029 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.026+0000 7f8269dd4700 1 -- 172.21.15.84:0/3788775230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f826406a110 con 0x7f8264103e60 2024-04-26T18:46:19.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.027+0000 7f8260ff9700 1 -- 172.21.15.84:0/3788775230 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f8258010430 con 0x7f8264103e60 2024-04-26T18:46:19.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.027+0000 7f8260ff9700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f824c0a3db0 0x7f824c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:19.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.027+0000 7f82637fe700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f824c0a3db0 0x7f824c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:19.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.027+0000 7f8260ff9700 1 -- 172.21.15.84:0/3788775230 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8579+0+0 (secure 0 0 0) 0x7f82580d0f40 con 0x7f8264103e60 2024-04-26T18:46:19.030 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.028+0000 7f82637fe700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f824c0a3db0 0x7f824c0a6270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f825400b2e0 tx=0x7f825401a040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:19.032 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.031+0000 7f8260ff9700 1 -- 172.21.15.84:0/3788775230 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f825809d5a0 con 0x7f8264103e60 2024-04-26T18:46:19.231 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.230+0000 7f8269dd4700 1 -- 172.21.15.84:0/3788775230 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f826402e490 con 0x7f8264103e60 2024-04-26T18:46:19.233 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.232+0000 7f8260ff9700 1 -- 172.21.15.84:0/3788775230 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+23197 (secure 0 0 0) 0x7f825809cea0 con 0x7f8264103e60 2024-04-26T18:46:19.233 INFO:teuthology.orchestra.run.smithi084.stdout: 2024-04-26T18:46:19.234 INFO:teuthology.orchestra.run.smithi084.stdout:{"epoch":120,"fsid":"df3f3816-03f8-11ef-bc93-c7b262605968","created":"2024-04-26T18:16:06.267694+0000","modified":"2024-04-26T18:46:17.530159+0000","last_up_change":"2024-04-26T18:23:19.906511+0000","last_in_change":"2024-04-26T18:22:59.541869+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T18:19:31.174024+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-26T18:24:16.533128+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-26T18:24:19.742656+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":119,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b1076912-986e-4825-b1d1-5fa9d69bc07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6802","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6803","nonce":140050011}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6804","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6805","nonce":140050011}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6808","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6809","nonce":140050011}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6806","nonce":140050011},{"type":"v1","addr":"172.21.15.84:6807","nonce":140050011}]},"public_addr":"172.21.15.84:6803/140050011","cluster_addr":"172.21.15.84:6805/140050011","heartbeat_back_addr":"172.21.15.84:6809/140050011","heartbeat_front_addr":"172.21.15.84:6807/140050011","state":["exists","up"]},{"osd":1,"uuid":"fa93293d-9058-4804-ba4b-7e1e674b0fb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6800","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6801","nonce":2512857657}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6802","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6803","nonce":2512857657}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6806","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6807","nonce":2512857657}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6804","nonce":2512857657},{"type":"v1","addr":"172.21.15.103:6805","nonce":2512857657}]},"public_addr":"172.21.15.103:6801/2512857657","cluster_addr":"172.21.15.103:6803/2512857657","heartbeat_back_addr":"172.21.15.103:6807/2512857657","heartbeat_front_addr":"172.21.15.103:6805/2512857657","state":["exists","up"]},{"osd":2,"uuid":"bf78a44f-80a2-4cd7-974f-5abdf771e650","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6800","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6801","nonce":76340924}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6802","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6803","nonce":76340924}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6806","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6807","nonce":76340924}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6804","nonce":76340924},{"type":"v1","addr":"172.21.15.174:6805","nonce":76340924}]},"public_addr":"172.21.15.174:6801/76340924","cluster_addr":"172.21.15.174:6803/76340924","heartbeat_back_addr":"172.21.15.174:6807/76340924","heartbeat_front_addr":"172.21.15.174:6805/76340924","state":["exists","up"]},{"osd":3,"uuid":"55fd0035-bc76-4235-b8b5-5cf5ef770139","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6810","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6811","nonce":2352789256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6812","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6813","nonce":2352789256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6816","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6817","nonce":2352789256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6814","nonce":2352789256},{"type":"v1","addr":"172.21.15.84:6815","nonce":2352789256}]},"public_addr":"172.21.15.84:6811/2352789256","cluster_addr":"172.21.15.84:6813/2352789256","heartbeat_back_addr":"172.21.15.84:6817/2352789256","heartbeat_front_addr":"172.21.15.84:6815/2352789256","state":["exists","up"]},{"osd":4,"uuid":"c278a0f3-e436-4d1c-a009-f9165ebf3c60","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6808","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6809","nonce":1619258252}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6810","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6811","nonce":1619258252}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6814","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6815","nonce":1619258252}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6812","nonce":1619258252},{"type":"v1","addr":"172.21.15.103:6813","nonce":1619258252}]},"public_addr":"172.21.15.103:6809/1619258252","cluster_addr":"172.21.15.103:6811/1619258252","heartbeat_back_addr":"172.21.15.103:6815/1619258252","heartbeat_front_addr":"172.21.15.103:6813/1619258252","state":["exists","up"]},{"osd":5,"uuid":"6505be9b-c1b0-4d8b-944e-9920f90cb88c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6808","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6809","nonce":3168291768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6810","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6811","nonce":3168291768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6814","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6815","nonce":3168291768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6812","nonce":3168291768},{"type":"v1","addr":"172.21.15.174:6813","nonce":3168291768}]},"public_addr":"172.21.15.174:6809/3168291768","cluster_addr":"172.21.15.174:6811/3168291768","heartbeat_back_addr":"172.21.15.174:6815/3168291768","heartbeat_front_addr":"172.21.15.174:6813/3168291768","state":["exists","up"]},{"osd":6,"uuid":"a037b66a-7488-469d-b73a-14cc0d35cb5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6818","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6819","nonce":886635297}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6820","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6821","nonce":886635297}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6824","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6825","nonce":886635297}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6822","nonce":886635297},{"type":"v1","addr":"172.21.15.84:6823","nonce":886635297}]},"public_addr":"172.21.15.84:6819/886635297","cluster_addr":"172.21.15.84:6821/886635297","heartbeat_back_addr":"172.21.15.84:6825/886635297","heartbeat_front_addr":"172.21.15.84:6823/886635297","state":["exists","up"]},{"osd":7,"uuid":"c425a13f-8019-4484-86a0-a34fa4d8de81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6816","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6817","nonce":1738278079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6818","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6819","nonce":1738278079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6822","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6823","nonce":1738278079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6820","nonce":1738278079},{"type":"v1","addr":"172.21.15.103:6821","nonce":1738278079}]},"public_addr":"172.21.15.103:6817/1738278079","cluster_addr":"172.21.15.103:6819/1738278079","heartbeat_back_addr":"172.21.15.103:6823/1738278079","heartbeat_front_addr":"172.21.15.103:6821/1738278079","state":["exists","up"]},{"osd":8,"uuid":"687e6c08-bfa4-440c-b3bc-90a5748989d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6816","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6817","nonce":38410548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6818","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6819","nonce":38410548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6822","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6823","nonce":38410548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6820","nonce":38410548},{"type":"v1","addr":"172.21.15.174:6821","nonce":38410548}]},"public_addr":"172.21.15.174:6817/38410548","cluster_addr":"172.21.15.174:6819/38410548","heartbeat_back_addr":"172.21.15.174:6823/38410548","heartbeat_front_addr":"172.21.15.174:6821/38410548","state":["exists","up"]},{"osd":9,"uuid":"059eb22d-6dfb-4dd0-b192-84ed42ee1fbc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6826","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6827","nonce":2926393927}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6828","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6829","nonce":2926393927}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6832","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6833","nonce":2926393927}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.84:6830","nonce":2926393927},{"type":"v1","addr":"172.21.15.84:6831","nonce":2926393927}]},"public_addr":"172.21.15.84:6827/2926393927","cluster_addr":"172.21.15.84:6829/2926393927","heartbeat_back_addr":"172.21.15.84:6833/2926393927","heartbeat_front_addr":"172.21.15.84:6831/2926393927","state":["exists","up"]},{"osd":10,"uuid":"da31e305-5220-4a87-a131-005b7c7f76b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6824","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6825","nonce":2705726915}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6826","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6827","nonce":2705726915}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6830","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6831","nonce":2705726915}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.103:6828","nonce":2705726915},{"type":"v1","addr":"172.21.15.103:6829","nonce":2705726915}]},"public_addr":"172.21.15.103:6825/2705726915","cluster_addr":"172.21.15.103:6827/2705726915","heartbeat_back_addr":"172.21.15.103:6831/2705726915","heartbeat_front_addr":"172.21.15.103:6829/2705726915","state":["exists","up"]},{"osd":11,"uuid":"3fde6c8f-ee02-41ee-a34f-73072b4761e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6824","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6825","nonce":1668672128}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6826","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6827","nonce":1668672128}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6830","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6831","nonce":1668672128}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.174:6828","nonce":1668672128},{"type":"v1","addr":"172.21.15.174:6829","nonce":1668672128}]},"public_addr":"172.21.15.174:6825/1668672128","cluster_addr":"172.21.15.174:6827/1668672128","heartbeat_back_addr":"172.21.15.174:6831/1668672128","heartbeat_front_addr":"172.21.15.174:6829/1668672128","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:18:39.099188+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:03.327608+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:27.714108+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:19:52.506015+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:17.631670+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:20:42.702489+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:07.606697+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:34.123214+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:21:59.534658+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:25.357720+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:22:51.543106+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T18:23:18.208230+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":8,"to":11}]},{"pgid":"2.c","mappings":[{"from":2,"to":1}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4}]},{"pgid":"3.0","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":6}]},{"pgid":"3.31","mappings":[{"from":4,"to":7}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"3.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.84:0/277477867":"2024-04-27T18:27:57.444610+0000","172.21.15.84:0/559313476":"2024-04-27T18:27:57.444610+0000","172.21.15.84:6800/2059208901":"2024-04-27T18:27:57.444610+0000","172.21.15.84:0/3834640430":"2024-04-27T18:27:57.444610+0000","172.21.15.84:0/2142936585":"2024-04-27T18:27:57.444610+0000","172.21.15.84:0/1604680842":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6800/3919933286":"2024-04-27T18:17:05.083192+0000","172.21.15.84:6801/2059208901":"2024-04-27T18:27:57.444610+0000","172.21.15.84:6801/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/538363975":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/952052990":"2024-04-27T18:17:05.083192+0000","172.21.15.84:0/1266942705":"2024-04-27T18:16:37.859669+0000","172.21.15.84:6800/1448727861":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3970542149":"2024-04-27T18:16:37.859669+0000","172.21.15.84:0/3607125049":"2024-04-27T18:17:05.083192+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":20,"length":1},{"begin":33,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T18:46:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.234+0000 7f8269dd4700 1 -- 172.21.15.84:0/3788775230 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f824c0a3db0 msgr2=0x7f824c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.234+0000 7f8269dd4700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f824c0a3db0 0x7f824c0a6270 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f825400b2e0 tx=0x7f825401a040 comp rx=0 tx=0).stop 2024-04-26T18:46:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.234+0000 7f8269dd4700 1 -- 172.21.15.84:0/3788775230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8264103e60 msgr2=0x7f826419fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.234+0000 7f8269dd4700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8264103e60 0x7f826419fdc0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f825800e9a0 tx=0x7f825800ecb0 comp rx=0 tx=0).stop 2024-04-26T18:46:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.234+0000 7f8269dd4700 1 -- 172.21.15.84:0/3788775230 shutdown_connections 2024-04-26T18:46:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.234+0000 7f8269dd4700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f824c0a3db0 0x7f824c0a6270 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.234+0000 7f8269dd4700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f82641094e0 0x7f8264199a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.234+0000 7f8269dd4700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f82641034a0 0x7f826419f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.234+0000 7f8269dd4700 1 --2- 172.21.15.84:0/3788775230 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f8264103e60 0x7f826419fdc0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.234+0000 7f8269dd4700 1 -- 172.21.15.84:0/3788775230 >> 172.21.15.84:0/3788775230 conn(0x7f826407a590 msgr2=0x7f8264102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:19.235 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.234+0000 7f8269dd4700 1 -- 172.21.15.84:0/3788775230 shutdown_connections 2024-04-26T18:46:19.236 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:19.234+0000 7f8269dd4700 1 -- 172.21.15.84:0/3788775230 wait complete. 2024-04-26T18:46:19.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:19 smithi084 ceph-mon[29141]: pgmap v579: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 689 B/s wr, 6 op/s 2024-04-26T18:46:19.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:19 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:46:19.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:19 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:46:19.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:19 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:46:19.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:19 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3788775230' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:46:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:19 smithi103 ceph-mon[32727]: pgmap v579: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 689 B/s wr, 6 op/s 2024-04-26T18:46:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:19 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:46:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:19 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:46:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:19 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:46:19.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:19 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3788775230' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:46:19.899 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs fail cephfs 2024-04-26T18:46:19.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:19 smithi174 ceph-mon[32705]: pgmap v579: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 689 B/s wr, 6 op/s 2024-04-26T18:46:19.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:19 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T18:46:19.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:19 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T18:46:19.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:19 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' 2024-04-26T18:46:19.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:19 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3788775230' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T18:46:20.135 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:46:21.793 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.792+0000 7fcb30886700 1 -- 172.21.15.84:0/1657986669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c103e30 msgr2=0x7fcb2c110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:21.794 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.792+0000 7fcb30886700 1 --2- 172.21.15.84:0/1657986669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c103e30 0x7fcb2c110290 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400b0a0 tx=0x7fcb2400b3b0 comp rx=0 tx=0).stop 2024-04-26T18:46:21.794 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:21 smithi084 ceph-mon[29141]: pgmap v580: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-04-26T18:46:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.793+0000 7fcb30886700 1 -- 172.21.15.84:0/1657986669 shutdown_connections 2024-04-26T18:46:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.793+0000 7fcb30886700 1 --2- 172.21.15.84:0/1657986669 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c103e30 0x7fcb2c110290 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.793+0000 7fcb30886700 1 --2- 172.21.15.84:0/1657986669 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb2c1094c0 0x7fcb2c1098a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.793+0000 7fcb30886700 1 --2- 172.21.15.84:0/1657986669 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb2c103470 0x7fcb2c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.793+0000 7fcb30886700 1 -- 172.21.15.84:0/1657986669 >> 172.21.15.84:0/1657986669 conn(0x7fcb2c07a580 msgr2=0x7fcb2c07a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.793+0000 7fcb30886700 1 -- 172.21.15.84:0/1657986669 shutdown_connections 2024-04-26T18:46:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.793+0000 7fcb30886700 1 -- 172.21.15.84:0/1657986669 wait complete. 2024-04-26T18:46:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.793+0000 7fcb30886700 1 Processor -- start 2024-04-26T18:46:21.796 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb30886700 1 -- start start 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb30886700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb2c103470 0x7fcb2c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb30886700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c103e30 0x7fcb2c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb30886700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb2c1094c0 0x7fcb2c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb30886700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c1135e0 con 0x7fcb2c103e30 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb30886700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c113460 con 0x7fcb2c1094c0 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb30886700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7fcb2c113760 con 0x7fcb2c103470 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb21d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c103e30 0x7fcb2c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb21d9b700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c103e30 0x7fcb2c19fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39432/0 (socket says 172.21.15.84:39432) 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb21d9b700 1 -- 172.21.15.84:0/1960988362 learned_addr learned my addr 172.21.15.84:0/1960988362 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb2a59c700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb2c103470 0x7fcb2c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb2ad9d700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb2c1094c0 0x7fcb2c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb21d9b700 1 -- 172.21.15.84:0/1960988362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb2c103470 msgr2=0x7fcb2c19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb21d9b700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb2c103470 0x7fcb2c19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:21.797 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb21d9b700 1 -- 172.21.15.84:0/1960988362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb2c1094c0 msgr2=0x7fcb2c1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:21.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb21d9b700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb2c1094c0 0x7fcb2c1998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:21.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.794+0000 7fcb21d9b700 1 -- 172.21.15.84:0/1960988362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb2c19a000 con 0x7fcb2c103e30 2024-04-26T18:46:21.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.795+0000 7fcb2ad9d700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb2c1094c0 0x7fcb2c1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:46:21.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.795+0000 7fcb21d9b700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c103e30 0x7fcb2c19fc20 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400c9b0 tx=0x7fcb1400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:21.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.795+0000 7fcb23fff700 1 -- 172.21.15.84:0/1960988362 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14007b00 con 0x7fcb2c103e30 2024-04-26T18:46:21.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.795+0000 7fcb23fff700 1 -- 172.21.15.84:0/1960988362 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb14007ca0 con 0x7fcb2c103e30 2024-04-26T18:46:21.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.795+0000 7fcb23fff700 1 -- 172.21.15.84:0/1960988362 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb14018660 con 0x7fcb2c103e30 2024-04-26T18:46:21.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.795+0000 7fcb30886700 1 -- 172.21.15.84:0/1960988362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb2c19a2f0 con 0x7fcb2c103e30 2024-04-26T18:46:21.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.795+0000 7fcb30886700 1 -- 172.21.15.84:0/1960988362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb2c1a65b0 con 0x7fcb2c103e30 2024-04-26T18:46:21.798 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.796+0000 7fcb30886700 1 -- 172.21.15.84:0/1960988362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb2c06a110 con 0x7fcb2c103e30 2024-04-26T18:46:21.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.797+0000 7fcb23fff700 1 -- 172.21.15.84:0/1960988362 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7fcb1400f450 con 0x7fcb2c103e30 2024-04-26T18:46:21.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.798+0000 7fcb23fff700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcb180a3db0 0x7fcb180a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:21.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.798+0000 7fcb2a59c700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcb180a3db0 0x7fcb180a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:21.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.798+0000 7fcb23fff700 1 -- 172.21.15.84:0/1960988362 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 8579+0+0 (secure 0 0 0) 0x7fcb140d0ab0 con 0x7fcb2c103e30 2024-04-26T18:46:21.799 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.799+0000 7fcb2a59c700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcb180a3db0 0x7fcb180a6270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c19a9e0 tx=0x7fcb1c00b3c0 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:21.802 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:21.802+0000 7fcb23fff700 1 -- 172.21.15.84:0/1960988362 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcb14014030 con 0x7fcb2c103e30 2024-04-26T18:46:21.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:21 smithi103 ceph-mon[32727]: pgmap v580: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-04-26T18:46:21.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:21 smithi174 ceph-mon[32705]: pgmap v580: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-04-26T18:46:22.045 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.045+0000 7fcb30886700 1 -- 172.21.15.84:0/1960988362 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fcb2c19ae00 con 0x7fcb2c103e30 2024-04-26T18:46:22.047 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.047+0000 7fcb23fff700 1 -- 172.21.15.84:0/1960988362 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7fcb2c19ae00 con 0x7fcb2c103e30 2024-04-26T18:46:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.051+0000 7fcb30886700 1 -- 172.21.15.84:0/1960988362 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcb180a3db0 msgr2=0x7fcb180a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:22.051 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.051+0000 7fcb30886700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcb180a3db0 0x7fcb180a6270 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c19a9e0 tx=0x7fcb1c00b3c0 comp rx=0 tx=0).stop 2024-04-26T18:46:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.051+0000 7fcb30886700 1 -- 172.21.15.84:0/1960988362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c103e30 msgr2=0x7fcb2c19fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.051+0000 7fcb30886700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c103e30 0x7fcb2c19fc20 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400c9b0 tx=0x7fcb1400ccc0 comp rx=0 tx=0).stop 2024-04-26T18:46:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.051+0000 7fcb30886700 1 -- 172.21.15.84:0/1960988362 shutdown_connections 2024-04-26T18:46:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.051+0000 7fcb30886700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7fcb180a3db0 0x7fcb180a6270 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.051+0000 7fcb30886700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7fcb2c1094c0 0x7fcb2c1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.051+0000 7fcb30886700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7fcb2c103470 0x7fcb2c19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.051+0000 7fcb30886700 1 --2- 172.21.15.84:0/1960988362 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7fcb2c103e30 0x7fcb2c19fc20 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.051+0000 7fcb30886700 1 -- 172.21.15.84:0/1960988362 >> 172.21.15.84:0/1960988362 conn(0x7fcb2c07a580 msgr2=0x7fcb2c1021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.051+0000 7fcb30886700 1 -- 172.21.15.84:0/1960988362 shutdown_connections 2024-04-26T18:46:22.052 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:22.051+0000 7fcb30886700 1 -- 172.21.15.84:0/1960988362 wait complete. 2024-04-26T18:46:22.053 INFO:teuthology.orchestra.run.smithi084.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-26T18:46:22.345 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a[29137]: 2024-04-26T18:46:22.046+0000 7f83e2f28700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T18:46:22.345 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-a[71360]: ignoring --setuser ceph since I am not root 2024-04-26T18:46:22.345 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-a[71360]: ignoring --setgroup ceph since I am not root 2024-04-26T18:46:22.345 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-mds[71364]: -- 172.21.15.84:0/1106455058 <== mon.0 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c6b2607a40 con 0x55c6b25dac00 2024-04-26T18:46:22.345 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-a[71360]: starting mds.a at 2024-04-26T18:46:22.346 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-g[70923]: ignoring --setuser ceph since I am not root 2024-04-26T18:46:22.346 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-g[70923]: ignoring --setgroup ceph since I am not root 2024-04-26T18:46:22.346 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-mds[70940]: -- 172.21.15.84:0/721111171 <== mon.0 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5648de025a40 con 0x5648ddff8c00 2024-04-26T18:46:22.346 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-g[70923]: starting mds.g at 2024-04-26T18:46:22.377 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-h[62637]: ignoring --setuser ceph since I am not root 2024-04-26T18:46:22.377 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-h[62637]: ignoring --setgroup ceph since I am not root 2024-04-26T18:46:22.377 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-mds[62641]: -- 172.21.15.103:0/3462697520 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55624a6c5a40 con 0x55624a698800 2024-04-26T18:46:22.378 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-h[62637]: starting mds.h at 2024-04-26T18:46:22.378 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-k[62003]: ignoring --setuser ceph since I am not root 2024-04-26T18:46:22.378 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-k[62003]: ignoring --setgroup ceph since I am not root 2024-04-26T18:46:22.378 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-k[62003]: starting mds.k at 2024-04-26T18:46:22.378 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b[62948]: ignoring --setuser ceph since I am not root 2024-04-26T18:46:22.379 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b[62948]: ignoring --setgroup ceph since I am not root 2024-04-26T18:46:22.379 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-mds[62952]: -- 172.21.15.103:0/2951941425 <== mon.1 v2:172.21.15.174:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555f2ef6fa40 con 0x555f2ef42800 2024-04-26T18:46:22.379 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b[62948]: starting mds.b at 2024-04-26T18:46:22.379 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-e[62324]: ignoring --setuser ceph since I am not root 2024-04-26T18:46:22.379 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-e[62324]: ignoring --setgroup ceph since I am not root 2024-04-26T18:46:22.379 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-mds[62328]: -- 172.21.15.103:0/3972284350 <== mon.2 v2:172.21.15.103:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558bb8057a40 con 0x558bb802ac00 2024-04-26T18:46:22.379 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-e[62324]: starting mds.e at 2024-04-26T18:46:22.442 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-l[62537]: ignoring --setuser ceph since I am not root 2024-04-26T18:46:22.442 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-l[62537]: ignoring --setgroup ceph since I am not root 2024-04-26T18:46:22.442 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-mds[62541]: -- 172.21.15.174:0/3700868554 <== mon.0 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d60547ba40 con 0x55d60544ec00 2024-04-26T18:46:22.443 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-l[62537]: starting mds.l at 2024-04-26T18:46:22.443 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-c[63169]: ignoring --setuser ceph since I am not root 2024-04-26T18:46:22.443 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-c[63169]: ignoring --setgroup ceph since I am not root 2024-04-26T18:46:22.443 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-mds[63173]: -- 172.21.15.174:0/3512439289 <== mon.0 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d15e889a40 con 0x55d15e85d000 2024-04-26T18:46:22.443 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-c[63169]: starting mds.c at 2024-04-26T18:46:22.444 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-i[63475]: ignoring --setuser ceph since I am not root 2024-04-26T18:46:22.444 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-i[63475]: ignoring --setgroup ceph since I am not root 2024-04-26T18:46:22.444 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-mds[63479]: -- 172.21.15.174:0/3185175899 <== mon.0 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5618ea0a3a40 con 0x5618ea076800 2024-04-26T18:46:22.444 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-i[63475]: starting mds.i at 2024-04-26T18:46:22.444 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-f[62856]: ignoring --setuser ceph since I am not root 2024-04-26T18:46:22.444 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-f[62856]: ignoring --setgroup ceph since I am not root 2024-04-26T18:46:22.444 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-mds[62860]: -- 172.21.15.174:0/499807240 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x5650d783e4e0 con 0x5650d7814c00 2024-04-26T18:46:22.444 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-mds[62860]: -- 172.21.15.174:0/499807240 <== mon.0 v2:172.21.15.84:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5650d7841a40 con 0x5650d7814c00 2024-04-26T18:46:22.444 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-f[62856]: starting mds.f at 2024-04-26T18:46:22.730 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1960988362' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T18:46:22.731 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-mon[29141]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:46:22.731 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-mon[29141]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T18:46:22.731 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1960988362' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T18:46:22.731 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-mon[29141]: osdmap e121: 12 total, 12 up, 12 in 2024-04-26T18:46:22.731 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:22 smithi084 ceph-mon[29141]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-26T18:46:22.758 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-04-26T18:46:22.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1960988362' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T18:46:22.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-mon[32727]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:46:22.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-mon[32727]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T18:46:22.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1960988362' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T18:46:22.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-mon[32727]: osdmap e121: 12 total, 12 up, 12 in 2024-04-26T18:46:22.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:22 smithi103 ceph-mon[32727]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-26T18:46:22.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1960988362' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T18:46:22.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-mon[32705]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T18:46:22.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-mon[32705]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T18:46:22.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1960988362' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T18:46:22.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-mon[32705]: osdmap e121: 12 total, 12 up, 12 in 2024-04-26T18:46:22.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:22 smithi174 ceph-mon[32705]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-26T18:46:23.000 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:46:23.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: pgmap v581: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-04-26T18:46:23.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.103:6832/3326692813,v1:172.21.15.103:6834/3326692813] up:boot 2024-04-26T18:46:23.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.103:6833/1215348153,v1:172.21.15.103:6835/1215348153] up:boot 2024-04-26T18:46:23.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.84:6838/1935837979,v1:172.21.15.84:6840/1935837979] up:boot 2024-04-26T18:46:23.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.84:6839/3900662610,v1:172.21.15.84:6841/3900662610] up:boot 2024-04-26T18:46:23.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.174:6833/2371680209,v1:172.21.15.174:6837/2371680209] up:boot 2024-04-26T18:46:23.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.174:6835/3249055547,v1:172.21.15.174:6839/3249055547] up:boot 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.174:6832/2983559863,v1:172.21.15.174:6836/2983559863] up:boot 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.174:6834/2414234273,v1:172.21.15.174:6838/2414234273] up:boot 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.103:6836/1285067540,v1:172.21.15.103:6837/1285067540] up:boot 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: mds.? [v2:172.21.15.103:6838/2585378499,v1:172.21.15.103:6839/2585378499] up:boot 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:46:23.846 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:23 smithi084 ceph-mon[29141]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:46:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: pgmap v581: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-04-26T18:46:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.103:6832/3326692813,v1:172.21.15.103:6834/3326692813] up:boot 2024-04-26T18:46:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.103:6833/1215348153,v1:172.21.15.103:6835/1215348153] up:boot 2024-04-26T18:46:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.84:6838/1935837979,v1:172.21.15.84:6840/1935837979] up:boot 2024-04-26T18:46:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.84:6839/3900662610,v1:172.21.15.84:6841/3900662610] up:boot 2024-04-26T18:46:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.174:6833/2371680209,v1:172.21.15.174:6837/2371680209] up:boot 2024-04-26T18:46:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.174:6835/3249055547,v1:172.21.15.174:6839/3249055547] up:boot 2024-04-26T18:46:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.174:6832/2983559863,v1:172.21.15.174:6836/2983559863] up:boot 2024-04-26T18:46:23.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.174:6834/2414234273,v1:172.21.15.174:6838/2414234273] up:boot 2024-04-26T18:46:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.103:6836/1285067540,v1:172.21.15.103:6837/1285067540] up:boot 2024-04-26T18:46:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: mds.? [v2:172.21.15.103:6838/2585378499,v1:172.21.15.103:6839/2585378499] up:boot 2024-04-26T18:46:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-04-26T18:46:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:46:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:46:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:46:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:46:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:46:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:46:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:46:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:46:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:46:23.878 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:23 smithi103 ceph-mon[32727]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:46:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: pgmap v581: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 614 B/s wr, 11 op/s 2024-04-26T18:46:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.103:6832/3326692813,v1:172.21.15.103:6834/3326692813] up:boot 2024-04-26T18:46:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.103:6833/1215348153,v1:172.21.15.103:6835/1215348153] up:boot 2024-04-26T18:46:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.84:6838/1935837979,v1:172.21.15.84:6840/1935837979] up:boot 2024-04-26T18:46:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.84:6839/3900662610,v1:172.21.15.84:6841/3900662610] up:boot 2024-04-26T18:46:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.174:6833/2371680209,v1:172.21.15.174:6837/2371680209] up:boot 2024-04-26T18:46:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.174:6835/3249055547,v1:172.21.15.174:6839/3249055547] up:boot 2024-04-26T18:46:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.174:6832/2983559863,v1:172.21.15.174:6836/2983559863] up:boot 2024-04-26T18:46:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.174:6834/2414234273,v1:172.21.15.174:6838/2414234273] up:boot 2024-04-26T18:46:23.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.103:6836/1285067540,v1:172.21.15.103:6837/1285067540] up:boot 2024-04-26T18:46:23.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: mds.? [v2:172.21.15.103:6838/2585378499,v1:172.21.15.103:6839/2585378499] up:boot 2024-04-26T18:46:23.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-04-26T18:46:23.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-26T18:46:23.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-26T18:46:23.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-26T18:46:23.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-26T18:46:23.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-26T18:46:23.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-26T18:46:23.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-26T18:46:23.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-26T18:46:23.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-26T18:46:23.940 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:23 smithi174 ceph-mon[32705]: from='mgr.15240 172.21.15.84:0/1846275647' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-26T18:46:24.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.696+0000 7f5259a9a700 1 -- 172.21.15.84:0/1232594961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254108380 msgr2=0x7f5254108760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:24.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.696+0000 7f5259a9a700 1 --2- 172.21.15.84:0/1232594961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254108380 0x7f5254108760 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f524c00b0a0 tx=0x7f524c00b3b0 comp rx=0 tx=0).stop 2024-04-26T18:46:24.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.696+0000 7f5259a9a700 1 -- 172.21.15.84:0/1232594961 shutdown_connections 2024-04-26T18:46:24.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.696+0000 7f5259a9a700 1 --2- 172.21.15.84:0/1232594961 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254108380 0x7f5254108760 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:24.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.696+0000 7f5259a9a700 1 --2- 172.21.15.84:0/1232594961 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5254102cd0 0x7f525410f1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:24.698 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.696+0000 7f5259a9a700 1 --2- 172.21.15.84:0/1232594961 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254102310 0x7f5254102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:24.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.696+0000 7f5259a9a700 1 -- 172.21.15.84:0/1232594961 >> 172.21.15.84:0/1232594961 conn(0x7f5254079290 msgr2=0x7f52540796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:24.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.696+0000 7f5259a9a700 1 -- 172.21.15.84:0/1232594961 shutdown_connections 2024-04-26T18:46:24.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.696+0000 7f5259a9a700 1 -- 172.21.15.84:0/1232594961 wait complete. 2024-04-26T18:46:24.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.696+0000 7f5259a9a700 1 Processor -- start 2024-04-26T18:46:24.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5259a9a700 1 -- start start 2024-04-26T18:46:24.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5259a9a700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254102310 0x7f5254103ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:24.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5259a9a700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254102cd0 0x7f5254104530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:24.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5259a9a700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5254108380 0x7f5254104a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:24.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5259a9a700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f52541124d0 con 0x7f5254102cd0 2024-04-26T18:46:24.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5259a9a700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f5254112350 con 0x7f5254108380 2024-04-26T18:46:24.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5259a9a700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f5254112650 con 0x7f5254102310 2024-04-26T18:46:24.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5252ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254102cd0 0x7f5254104530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:24.699 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5252ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254102cd0 0x7f5254104530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:39480/0 (socket says 172.21.15.84:39480) 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5252ffd700 1 -- 172.21.15.84:0/662068312 learned_addr learned my addr 172.21.15.84:0/662068312 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f52537fe700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254102310 0x7f5254103ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5253fff700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5254108380 0x7f5254104a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5252ffd700 1 -- 172.21.15.84:0/662068312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254102310 msgr2=0x7f5254103ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5252ffd700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254102310 0x7f5254103ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5252ffd700 1 -- 172.21.15.84:0/662068312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5254108380 msgr2=0x7f5254104a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5252ffd700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5254108380 0x7f5254104a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f5252ffd700 1 -- 172.21.15.84:0/662068312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52541a7f70 con 0x7f5254102cd0 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.697+0000 7f52537fe700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254102310 0x7f5254103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.698+0000 7f5252ffd700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254102cd0 0x7f5254104530 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f524800d980 tx=0x7f524800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.698+0000 7f5250ff9700 1 -- 172.21.15.84:0/662068312 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5248017070 con 0x7f5254102cd0 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.698+0000 7f5250ff9700 1 -- 172.21.15.84:0/662068312 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5248005020 con 0x7f5254102cd0 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.698+0000 7f5259a9a700 1 -- 172.21.15.84:0/662068312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52541a8260 con 0x7f5254102cd0 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.698+0000 7f5250ff9700 1 -- 172.21.15.84:0/662068312 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5248009b50 con 0x7f5254102cd0 2024-04-26T18:46:24.700 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.698+0000 7f5259a9a700 1 -- 172.21.15.84:0/662068312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52541a8750 con 0x7f5254102cd0 2024-04-26T18:46:24.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.699+0000 7f5259a9a700 1 -- 172.21.15.84:0/662068312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f525406a110 con 0x7f5254102cd0 2024-04-26T18:46:24.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.700+0000 7f5250ff9700 1 -- 172.21.15.84:0/662068312 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f52480051d0 con 0x7f5254102cd0 2024-04-26T18:46:24.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.700+0000 7f5250ff9700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f523c0a3db0 0x7f523c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:24.701 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.700+0000 7f52537fe700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f523c0a3db0 0x7f523c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:24.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.700+0000 7f5250ff9700 1 -- 172.21.15.84:0/662068312 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8965+0+0 (secure 0 0 0) 0x7f52480d0910 con 0x7f5254102cd0 2024-04-26T18:46:24.702 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.701+0000 7f52537fe700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f523c0a3db0 0x7f523c0a6270 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f524c009e60 tx=0x7f524c006230 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:24.705 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.705+0000 7f5250ff9700 1 -- 172.21.15.84:0/662068312 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5248020020 con 0x7f5254102cd0 2024-04-26T18:46:24.947 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:24.947+0000 7f5259a9a700 1 -- 172.21.15.84:0/662068312 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f52541a89b0 con 0x7f5254102cd0 2024-04-26T18:46:25.556 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:25.556+0000 7f5250ff9700 1 -- 172.21.15.84:0/662068312 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) ==== 104+0+0 (secure 0 0 0) 0x7f524809cdf0 con 0x7f5254102cd0 2024-04-26T18:46:25.560 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:25.560+0000 7f5259a9a700 1 -- 172.21.15.84:0/662068312 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f523c0a3db0 msgr2=0x7f523c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:25.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:25.560+0000 7f5259a9a700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f523c0a3db0 0x7f523c0a6270 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f524c009e60 tx=0x7f524c006230 comp rx=0 tx=0).stop 2024-04-26T18:46:25.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:25.560+0000 7f5259a9a700 1 -- 172.21.15.84:0/662068312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254102cd0 msgr2=0x7f5254104530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:25.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:25.560+0000 7f5259a9a700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254102cd0 0x7f5254104530 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f524800d980 tx=0x7f524800dc90 comp rx=0 tx=0).stop 2024-04-26T18:46:25.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:25.560+0000 7f5259a9a700 1 -- 172.21.15.84:0/662068312 shutdown_connections 2024-04-26T18:46:25.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:25.560+0000 7f5259a9a700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f523c0a3db0 0x7f523c0a6270 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:25.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:25.560+0000 7f5259a9a700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f5254108380 0x7f5254104a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:25.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:25.560+0000 7f5259a9a700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f5254102310 0x7f5254103ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:25.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:25.560+0000 7f5259a9a700 1 --2- 172.21.15.84:0/662068312 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f5254102cd0 0x7f5254104530 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:25.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:25.560+0000 7f5259a9a700 1 -- 172.21.15.84:0/662068312 >> 172.21.15.84:0/662068312 conn(0x7f5254079290 msgr2=0x7f52541016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:25.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:25.561+0000 7f5259a9a700 1 -- 172.21.15.84:0/662068312 shutdown_connections 2024-04-26T18:46:25.561 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:25.561+0000 7f5259a9a700 1 -- 172.21.15.84:0/662068312 wait complete. 2024-04-26T18:46:25.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:25 smithi084 ceph-mon[29141]: pgmap v583: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-26T18:46:25.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:25 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/662068312' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:46:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:25 smithi103 ceph-mon[32727]: pgmap v583: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-26T18:46:25.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:25 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/662068312' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:46:25.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:25 smithi174 ceph-mon[32705]: pgmap v583: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-26T18:46:25.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:25 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/662068312' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T18:46:26.274 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-26T18:46:26.519 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:46:26.806 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:26 smithi084 ceph-mon[29141]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:46:26.806 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:26 smithi084 ceph-mon[29141]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T18:46:26.806 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:26 smithi084 ceph-mon[29141]: Cluster is now healthy 2024-04-26T18:46:26.806 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:26 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/662068312' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:46:26.806 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:26 smithi084 ceph-mon[29141]: fsmap 12 up:standby 2024-04-26T18:46:26.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:26 smithi103 ceph-mon[32727]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:46:26.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:26 smithi103 ceph-mon[32727]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T18:46:26.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:26 smithi103 ceph-mon[32727]: Cluster is now healthy 2024-04-26T18:46:26.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:26 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/662068312' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:46:26.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:26 smithi103 ceph-mon[32727]: fsmap 12 up:standby 2024-04-26T18:46:26.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:26 smithi174 ceph-mon[32705]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T18:46:26.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:26 smithi174 ceph-mon[32705]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T18:46:26.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:26 smithi174 ceph-mon[32705]: Cluster is now healthy 2024-04-26T18:46:26.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:26 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/662068312' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-26T18:46:26.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:26 smithi174 ceph-mon[32705]: fsmap 12 up:standby 2024-04-26T18:46:27.732 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:27 smithi084 ceph-mon[29141]: pgmap v584: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T18:46:27.840 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:27 smithi174 ceph-mon[32705]: pgmap v584: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T18:46:27.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:27 smithi103 ceph-mon[32727]: pgmap v584: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-26T18:46:28.310 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.308+0000 7f4c2aae6700 1 -- 172.21.15.84:0/3282693556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c2406c360 msgr2=0x7f4c241102d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.308+0000 7f4c2aae6700 1 --2- 172.21.15.84:0/3282693556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c2406c360 0x7f4c241102d0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c009990 tx=0x7f4c1c009ca0 comp rx=0 tx=0).stop 2024-04-26T18:46:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.308+0000 7f4c2aae6700 1 -- 172.21.15.84:0/3282693556 shutdown_connections 2024-04-26T18:46:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.308+0000 7f4c2aae6700 1 --2- 172.21.15.84:0/3282693556 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c2406c360 0x7f4c241102d0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.308+0000 7f4c2aae6700 1 --2- 172.21.15.84:0/3282693556 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c24109490 0x7f4c24109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.308+0000 7f4c2aae6700 1 --2- 172.21.15.84:0/3282693556 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c2406b9c0 0x7f4c2406be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.308+0000 7f4c2aae6700 1 -- 172.21.15.84:0/3282693556 >> 172.21.15.84:0/3282693556 conn(0x7f4c24079290 msgr2=0x7f4c240796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.308+0000 7f4c2aae6700 1 -- 172.21.15.84:0/3282693556 shutdown_connections 2024-04-26T18:46:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.308+0000 7f4c2aae6700 1 -- 172.21.15.84:0/3282693556 wait complete. 2024-04-26T18:46:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.309+0000 7f4c2aae6700 1 Processor -- start 2024-04-26T18:46:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.309+0000 7f4c2aae6700 1 -- start start 2024-04-26T18:46:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.309+0000 7f4c2aae6700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c2406b9c0 0x7f4c241023e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.309+0000 7f4c2aae6700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c2406c360 0x7f4c24102920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:28.311 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.309+0000 7f4c2aae6700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c24109490 0x7f4c24102e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.309+0000 7f4c2aae6700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f4c241135b0 con 0x7f4c2406c360 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.309+0000 7f4c2aae6700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24113430 con 0x7f4c2406b9c0 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.309+0000 7f4c2aae6700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24113730 con 0x7f4c24109490 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.309+0000 7f4c1bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c2406c360 0x7f4c24102920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.309+0000 7f4c1bfff700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c2406c360 0x7f4c24102920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60824/0 (socket says 172.21.15.84:60824) 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.309+0000 7f4c1bfff700 1 -- 172.21.15.84:0/1010315555 learned_addr learned my addr 172.21.15.84:0/1010315555 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.309+0000 7f4c28882700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c2406b9c0 0x7f4c241023e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.310+0000 7f4c29083700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c24109490 0x7f4c24102e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.310+0000 7f4c1bfff700 1 -- 172.21.15.84:0/1010315555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c24109490 msgr2=0x7f4c24102e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.310+0000 7f4c1bfff700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c24109490 0x7f4c24102e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.310+0000 7f4c1bfff700 1 -- 172.21.15.84:0/1010315555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c2406b9c0 msgr2=0x7f4c241023e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.310+0000 7f4c1bfff700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c2406b9c0 0x7f4c241023e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.310+0000 7f4c1bfff700 1 -- 172.21.15.84:0/1010315555 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c241a7f90 con 0x7f4c2406c360 2024-04-26T18:46:28.312 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.310+0000 7f4c29083700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c24109490 0x7f4c24102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T18:46:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.310+0000 7f4c1bfff700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c2406c360 0x7f4c24102920 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400c9b0 tx=0x7f4c1400cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.310+0000 7f4c19ffb700 1 -- 172.21.15.84:0/1010315555 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14007b00 con 0x7f4c2406c360 2024-04-26T18:46:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.310+0000 7f4c2aae6700 1 -- 172.21.15.84:0/1010315555 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c241a8280 con 0x7f4c2406c360 2024-04-26T18:46:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.310+0000 7f4c19ffb700 1 -- 172.21.15.84:0/1010315555 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c14007ca0 con 0x7f4c2406c360 2024-04-26T18:46:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.310+0000 7f4c19ffb700 1 -- 172.21.15.84:0/1010315555 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c14018730 con 0x7f4c2406c360 2024-04-26T18:46:28.313 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.310+0000 7f4c2aae6700 1 -- 172.21.15.84:0/1010315555 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c241a87c0 con 0x7f4c2406c360 2024-04-26T18:46:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.311+0000 7f4c2aae6700 1 -- 172.21.15.84:0/1010315555 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c2406a110 con 0x7f4c2406c360 2024-04-26T18:46:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.312+0000 7f4c19ffb700 1 -- 172.21.15.84:0/1010315555 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f4c1400f450 con 0x7f4c2406c360 2024-04-26T18:46:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.313+0000 7f4c19ffb700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c0c0a3db0 0x7f4c0c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.313+0000 7f4c19ffb700 1 -- 172.21.15.84:0/1010315555 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 8965+0+0 (secure 0 0 0) 0x7f4c140d0a60 con 0x7f4c2406c360 2024-04-26T18:46:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.313+0000 7f4c28882700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c0c0a3db0 0x7f4c0c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:28.314 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.313+0000 7f4c28882700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c0c0a3db0 0x7f4c0c0a6270 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009c90 tx=0x7f4c20009040 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:28.317 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.317+0000 7f4c19ffb700 1 -- 172.21.15.84:0/1010315555 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c14014030 con 0x7f4c2406c360 2024-04-26T18:46:28.514 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.514+0000 7f4c2aae6700 1 -- 172.21.15.84:0/1010315555 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f4c24103910 con 0x7f4c2406c360 2024-04-26T18:46:28.559 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.559+0000 7f4c19ffb700 1 -- 172.21.15.84:0/1010315555 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v122) ==== 181+0+0 (secure 0 0 0) 0x7f4c1409cf40 con 0x7f4c2406c360 2024-04-26T18:46:28.559 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:28 smithi084 ceph-mon[29141]: pgmap v585: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T18:46:28.560 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:28 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1010315555' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T18:46:28.563 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.563+0000 7f4c2aae6700 1 -- 172.21.15.84:0/1010315555 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c0c0a3db0 msgr2=0x7f4c0c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.563+0000 7f4c2aae6700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c0c0a3db0 0x7f4c0c0a6270 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009c90 tx=0x7f4c20009040 comp rx=0 tx=0).stop 2024-04-26T18:46:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.563+0000 7f4c2aae6700 1 -- 172.21.15.84:0/1010315555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c2406c360 msgr2=0x7f4c24102920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.563+0000 7f4c2aae6700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c2406c360 0x7f4c24102920 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400c9b0 tx=0x7f4c1400cd70 comp rx=0 tx=0).stop 2024-04-26T18:46:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.563+0000 7f4c2aae6700 1 -- 172.21.15.84:0/1010315555 shutdown_connections 2024-04-26T18:46:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.563+0000 7f4c2aae6700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f4c0c0a3db0 0x7f4c0c0a6270 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.563+0000 7f4c2aae6700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f4c24109490 0x7f4c24102e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.563+0000 7f4c2aae6700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f4c2406b9c0 0x7f4c241023e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.563+0000 7f4c2aae6700 1 --2- 172.21.15.84:0/1010315555 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f4c2406c360 0x7f4c24102920 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.563+0000 7f4c2aae6700 1 -- 172.21.15.84:0/1010315555 >> 172.21.15.84:0/1010315555 conn(0x7f4c24079290 msgr2=0x7f4c241003c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.563+0000 7f4c2aae6700 1 -- 172.21.15.84:0/1010315555 shutdown_connections 2024-04-26T18:46:28.564 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:28.564+0000 7f4c2aae6700 1 -- 172.21.15.84:0/1010315555 wait complete. 2024-04-26T18:46:28.565 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_metadata' removed 2024-04-26T18:46:28.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:28 smithi174 ceph-mon[32705]: pgmap v585: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T18:46:28.855 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:28 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1010315555' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T18:46:28.876 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:28 smithi103 ceph-mon[32727]: pgmap v585: 97 pgs: 97 active+clean; 4.8 MiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-26T18:46:28.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:28 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1010315555' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T18:46:29.263 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1 shell --fsid df3f3816-03f8-11ef-bc93-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-26T18:46:29.504 INFO:teuthology.orchestra.run.smithi084.stderr:Inferring config /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/mon.a/config 2024-04-26T18:46:29.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:29 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/1010315555' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T18:46:29.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:29 smithi084 ceph-mon[29141]: osdmap e122: 12 total, 12 up, 12 in 2024-04-26T18:46:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:29 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/1010315555' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T18:46:29.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:29 smithi103 ceph-mon[32727]: osdmap e122: 12 total, 12 up, 12 in 2024-04-26T18:46:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:29 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/1010315555' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T18:46:29.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:29 smithi174 ceph-mon[32705]: osdmap e122: 12 total, 12 up, 12 in 2024-04-26T18:46:30.845 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:30 smithi084 ceph-mon[29141]: pgmap v587: 65 pgs: 65 active+clean; 577 KiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:46:30.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:30 smithi103 ceph-mon[32727]: pgmap v587: 65 pgs: 65 active+clean; 577 KiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:46:30.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:30 smithi174 ceph-mon[32705]: pgmap v587: 65 pgs: 65 active+clean; 577 KiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:46:31.367 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.367+0000 7f06c9d66700 1 -- 172.21.15.84:0/4261541562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c41025b0 msgr2=0x7f06c4102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:31.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.367+0000 7f06c9d66700 1 --2- 172.21.15.84:0/4261541562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c41025b0 0x7f06c4102990 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f06b80099c0 tx=0x7f06b8009cd0 comp rx=0 tx=0).stop 2024-04-26T18:46:31.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.367+0000 7f06c9d66700 1 -- 172.21.15.84:0/4261541562 shutdown_connections 2024-04-26T18:46:31.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.367+0000 7f06c9d66700 1 --2- 172.21.15.84:0/4261541562 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c41025b0 0x7f06c4102990 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:31.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.367+0000 7f06c9d66700 1 --2- 172.21.15.84:0/4261541562 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06c4107640 0x7f06c410f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:31.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.367+0000 7f06c9d66700 1 --2- 172.21.15.84:0/4261541562 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c4102f60 0x7f06c4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:31.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.367+0000 7f06c9d66700 1 -- 172.21.15.84:0/4261541562 >> 172.21.15.84:0/4261541562 conn(0x7f06c40792d0 msgr2=0x7f06c40796e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:31.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.367+0000 7f06c9d66700 1 -- 172.21.15.84:0/4261541562 shutdown_connections 2024-04-26T18:46:31.368 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.367+0000 7f06c9d66700 1 -- 172.21.15.84:0/4261541562 wait complete. 2024-04-26T18:46:31.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.368+0000 7f06c9d66700 1 Processor -- start 2024-04-26T18:46:31.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.368+0000 7f06c9d66700 1 -- start start 2024-04-26T18:46:31.370 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.368+0000 7f06c9d66700 1 --2- >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c41025b0 0x7f06c4104000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.368+0000 7f06c9d66700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c4102f60 0x7f06c4104540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.368+0000 7f06c9d66700 1 --2- >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06c4107640 0x7f06c4104a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.368+0000 7f06c9d66700 1 -- --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_getmap magic: 0 -- 0x7f06c4112550 con 0x7f06c4102f60 2024-04-26T18:46:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.368+0000 7f06c9d66700 1 -- --> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41123d0 con 0x7f06c41025b0 2024-04-26T18:46:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.368+0000 7f06c9d66700 1 -- --> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] -- mon_getmap magic: 0 -- 0x7f06c41126d0 con 0x7f06c4107640 2024-04-26T18:46:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.368+0000 7f06c2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c4102f60 0x7f06c4104540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.368+0000 7f06c2ffd700 1 --2- >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c4102f60 0x7f06c4104540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.84:3300/0 says I am v2:172.21.15.84:60834/0 (socket says 172.21.15.84:60834) 2024-04-26T18:46:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.368+0000 7f06c2ffd700 1 -- 172.21.15.84:0/3582980558 learned_addr learned my addr 172.21.15.84:0/3582980558 (peer_addr_for_me v2:172.21.15.84:0/0) 2024-04-26T18:46:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.368+0000 7f06c37fe700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c41025b0 0x7f06c4104000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.368+0000 7f06c3fff700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06c4107640 0x7f06c4104a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.369+0000 7f06c2ffd700 1 -- 172.21.15.84:0/3582980558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06c4107640 msgr2=0x7f06c4104a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.369+0000 7f06c2ffd700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06c4107640 0x7f06c4104a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:31.371 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.369+0000 7f06c2ffd700 1 -- 172.21.15.84:0/3582980558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c41025b0 msgr2=0x7f06c4104000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:31.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.369+0000 7f06c2ffd700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c41025b0 0x7f06c4104000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:31.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.369+0000 7f06c2ffd700 1 -- 172.21.15.84:0/3582980558 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06c41a8070 con 0x7f06c4102f60 2024-04-26T18:46:31.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.369+0000 7f06c2ffd700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c4102f60 0x7f06c4104540 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f06b400c9e0 tx=0x7f06b400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:31.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.369+0000 7f06c0ff9700 1 -- 172.21.15.84:0/3582980558 <== mon.0 v2:172.21.15.84:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b4007aa0 con 0x7f06c4102f60 2024-04-26T18:46:31.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.369+0000 7f06c9d66700 1 -- 172.21.15.84:0/3582980558 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06c41a8360 con 0x7f06c4102f60 2024-04-26T18:46:31.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.369+0000 7f06c0ff9700 1 -- 172.21.15.84:0/3582980558 <== mon.0 v2:172.21.15.84:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06b4007c40 con 0x7f06c4102f60 2024-04-26T18:46:31.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.369+0000 7f06c0ff9700 1 -- 172.21.15.84:0/3582980558 <== mon.0 v2:172.21.15.84:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06b400fe60 con 0x7f06c4102f60 2024-04-26T18:46:31.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.369+0000 7f06c9d66700 1 -- 172.21.15.84:0/3582980558 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06c41a88f0 con 0x7f06c4102f60 2024-04-26T18:46:31.372 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.370+0000 7f06c9d66700 1 -- 172.21.15.84:0/3582980558 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06c406a110 con 0x7f06c4102f60 2024-04-26T18:46:31.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.371+0000 7f06c0ff9700 1 -- 172.21.15.84:0/3582980558 <== mon.0 v2:172.21.15.84:3300/0 4 ==== mgrmap(e 25) ==== 138777+0+0 (secure 0 0 0) 0x7f06b400f450 con 0x7f06c4102f60 2024-04-26T18:46:31.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.371+0000 7f06c0ff9700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f06ac0a3db0 0x7f06ac0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T18:46:31.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.372+0000 7f06c37fe700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f06ac0a3db0 0x7f06ac0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T18:46:31.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.372+0000 7f06c0ff9700 1 -- 172.21.15.84:0/3582980558 <== mon.0 v2:172.21.15.84:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 8411+0+0 (secure 0 0 0) 0x7f06b40d06a0 con 0x7f06c4102f60 2024-04-26T18:46:31.373 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.372+0000 7f06c37fe700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f06ac0a3db0 0x7f06ac0a6270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009990 tx=0x7f06b8009650 comp rx=0 tx=0).ready entity=mgr.15240 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T18:46:31.376 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.376+0000 7f06c0ff9700 1 -- 172.21.15.84:0/3582980558 <== mon.0 v2:172.21.15.84:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06b4014030 con 0x7f06c4102f60 2024-04-26T18:46:31.572 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.572+0000 7f06c9d66700 1 -- 172.21.15.84:0/3582980558 --> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f06c402e490 con 0x7f06c4102f60 2024-04-26T18:46:31.624 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.625+0000 7f06c0ff9700 1 -- 172.21.15.84:0/3582980558 <== mon.0 v2:172.21.15.84:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v123) ==== 169+0+0 (secure 0 0 0) 0x7f06b409cdb0 con 0x7f06c4102f60 2024-04-26T18:46:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.628+0000 7f06c9d66700 1 -- 172.21.15.84:0/3582980558 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f06ac0a3db0 msgr2=0x7f06ac0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.628+0000 7f06c9d66700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f06ac0a3db0 0x7f06ac0a6270 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f06b8009990 tx=0x7f06b8009650 comp rx=0 tx=0).stop 2024-04-26T18:46:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.628+0000 7f06c9d66700 1 -- 172.21.15.84:0/3582980558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c4102f60 msgr2=0x7f06c4104540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T18:46:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.628+0000 7f06c9d66700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c4102f60 0x7f06c4104540 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f06b400c9e0 tx=0x7f06b400ccf0 comp rx=0 tx=0).stop 2024-04-26T18:46:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.628+0000 7f06c9d66700 1 -- 172.21.15.84:0/3582980558 shutdown_connections 2024-04-26T18:46:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.628+0000 7f06c9d66700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.84:6800/3453983695,v1:172.21.15.84:6801/3453983695] conn(0x7f06ac0a3db0 0x7f06ac0a6270 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.628+0000 7f06c9d66700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.174:3300/0,v1:172.21.15.174:6789/0] conn(0x7f06c4107640 0x7f06c4104a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.628+0000 7f06c9d66700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.103:3300/0,v1:172.21.15.103:6789/0] conn(0x7f06c41025b0 0x7f06c4104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.628+0000 7f06c9d66700 1 --2- 172.21.15.84:0/3582980558 >> [v2:172.21.15.84:3300/0,v1:172.21.15.84:6789/0] conn(0x7f06c4102f60 0x7f06c4104540 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T18:46:31.628 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.628+0000 7f06c9d66700 1 -- 172.21.15.84:0/3582980558 >> 172.21.15.84:0/3582980558 conn(0x7f06c40792d0 msgr2=0x7f06c406bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T18:46:31.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.629+0000 7f06c9d66700 1 -- 172.21.15.84:0/3582980558 shutdown_connections 2024-04-26T18:46:31.629 INFO:teuthology.orchestra.run.smithi084.stderr:2024-04-26T18:46:31.629+0000 7f06c9d66700 1 -- 172.21.15.84:0/3582980558 wait complete. 2024-04-26T18:46:31.630 INFO:teuthology.orchestra.run.smithi084.stderr:pool 'cephfs_data' removed 2024-04-26T18:46:31.844 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:31 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3582980558' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T18:46:31.938 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:31 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3582980558' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T18:46:32.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:31 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3582980558' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-26T18:46:32.270 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:46:32.295 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:46:32.328 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:46:32.360 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T18:46:32.360 DEBUG:teuthology.orchestra.run.smithi084:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T18:46:32.376 DEBUG:teuthology.orchestra.run.smithi103:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T18:46:32.392 DEBUG:teuthology.orchestra.run.smithi174:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T18:46:32.418 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T18:46:32.419 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-26T18:46:32.419 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.a 2024-04-26T18:46:32.594 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:32 smithi084 systemd[1]: Stopping Ceph mon.a for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:46:32.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:32 smithi174 ceph-mon[32705]: from='client.? 172.21.15.84:0/3582980558' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T18:46:32.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:32 smithi174 ceph-mon[32705]: osdmap e123: 12 total, 12 up, 12 in 2024-04-26T18:46:32.939 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:32 smithi174 ceph-mon[32705]: pgmap v589: 1 pgs: 1 active+clean; 577 KiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:46:33.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:32 smithi084 ceph-mon[29141]: from='client.? 172.21.15.84:0/3582980558' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T18:46:33.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:32 smithi084 ceph-mon[29141]: osdmap e123: 12 total, 12 up, 12 in 2024-04-26T18:46:33.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:32 smithi084 ceph-mon[29141]: pgmap v589: 1 pgs: 1 active+clean; 577 KiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:46:33.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:32 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a[29137]: 2024-04-26T18:46:32.720+0000 7f83e8733700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:46:33.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:32 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a[29137]: 2024-04-26T18:46:32.720+0000 7f83e8733700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T18:46:33.095 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:32 smithi084 podman[109208]: 2024-04-26 18:46:32.864616759 +0000 UTC m=+0.264974448 container died b25e6427f5bac1cabae0495c3c24e5ae335036bcf695e962e0ab151853ef5bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T18:46:33.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:32 smithi103 ceph-mon[32727]: from='client.? 172.21.15.84:0/3582980558' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-26T18:46:33.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:32 smithi103 ceph-mon[32727]: osdmap e123: 12 total, 12 up, 12 in 2024-04-26T18:46:33.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:32 smithi103 ceph-mon[32727]: pgmap v589: 1 pgs: 1 active+clean; 577 KiB data, 461 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-26T18:46:33.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:33 smithi084 podman[109208]: 2024-04-26 18:46:33.338937284 +0000 UTC m=+0.739294970 container cleanup b25e6427f5bac1cabae0495c3c24e5ae335036bcf695e962e0ab151853ef5bfd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-04-26T18:46:33.595 INFO:journalctl@ceph.mon.a.smithi084.stdout:Apr 26 18:46:33 smithi084 bash[109208]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-a 2024-04-26T18:46:33.763 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.a.service' 2024-04-26T18:46:33.812 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:46:33.813 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-26T18:46:33.813 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-26T18:46:33.813 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.b 2024-04-26T18:46:34.127 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:33 smithi103 systemd[1]: Stopping Ceph mon.b for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:46:34.619 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:34 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-b[32722]: 2024-04-26T18:46:34.133+0000 7f980b16c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:46:34.619 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:34 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-b[32722]: 2024-04-26T18:46:34.133+0000 7f980b16c700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-26T18:46:34.619 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:34 smithi103 podman[70090]: 2024-04-26 18:46:34.32870496 +0000 UTC m=+0.323392074 container died d58834cea0b8df53b21e3d54573aeaeaf7c9dc54d31be2862479b32ab69bdee8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-b, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS) 2024-04-26T18:46:34.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:34 smithi103 podman[70090]: 2024-04-26 18:46:34.743914145 +0000 UTC m=+0.738601266 container cleanup d58834cea0b8df53b21e3d54573aeaeaf7c9dc54d31be2862479b32ab69bdee8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-b, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T18:46:34.877 INFO:journalctl@ceph.mon.b.smithi103.stdout:Apr 26 18:46:34 smithi103 bash[70090]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-b 2024-04-26T18:46:35.081 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.b.service' 2024-04-26T18:46:35.127 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:46:35.127 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-26T18:46:35.127 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-26T18:46:35.127 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.c 2024-04-26T18:46:35.435 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:35 smithi174 systemd[1]: Stopping Ceph mon.c for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:46:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:35 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-c[32688]: 2024-04-26T18:46:35.434+0000 7f2926d64700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:46:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:35 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-c[32688]: 2024-04-26T18:46:35.434+0000 7f2926d64700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T18:46:35.689 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:35 smithi174 podman[71376]: 2024-04-26 18:46:35.4560574 +0000 UTC m=+0.137572717 container died 8f4977f04f29ed9fb432fbeeb2c0f68b1682083a2d6635d736687467f13d5e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-c, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:46:36.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:35 smithi174 podman[71376]: 2024-04-26 18:46:35.877923575 +0000 UTC m=+0.559438882 container cleanup 8f4977f04f29ed9fb432fbeeb2c0f68b1682083a2d6635d736687467f13d5e32 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T18:46:36.087 INFO:journalctl@ceph.mon.c.smithi174.stdout:Apr 26 18:46:35 smithi174 bash[71376]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mon-c 2024-04-26T18:46:36.295 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mon.c.service' 2024-04-26T18:46:36.336 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:46:36.336 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-26T18:46:36.337 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-26T18:46:36.337 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mgr.x 2024-04-26T18:46:36.643 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:46:36 smithi084 systemd[1]: Stopping Ceph mgr.x for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:46:36.980 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:46:36 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x[29375]: 2024-04-26T18:46:36.643+0000 7f73c3bc2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:46:36.980 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:46:36 smithi084 podman[109332]: 2024-04-26 18:46:36.674683545 +0000 UTC m=+0.144946915 container died aab0ee9dc54e9fc68e5eb4bee5f1271004d2fa1106bb2e0f5205a3157505e9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T18:46:37.344 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:46:37 smithi084 podman[109332]: 2024-04-26 18:46:37.071860186 +0000 UTC m=+0.542123554 container cleanup aab0ee9dc54e9fc68e5eb4bee5f1271004d2fa1106bb2e0f5205a3157505e9da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bc94a2a924f4a25e7c0317e85c91b85bf7cac0b1, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD) 2024-04-26T18:46:37.345 INFO:journalctl@ceph.mgr.x.smithi084.stdout:Apr 26 18:46:37 smithi084 bash[109332]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-x 2024-04-26T18:46:37.552 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mgr.x.service' 2024-04-26T18:46:37.585 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:46:37.585 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-26T18:46:37.585 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-26T18:46:37.586 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mgr.y 2024-04-26T18:46:37.877 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:46:37 smithi103 systemd[1]: Stopping Ceph mgr.y for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:46:38.250 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:46:37 smithi103 podman[70212]: 2024-04-26 18:46:37.938631331 +0000 UTC m=+0.158718532 container died e973bf7c756de0be52e200c13ddc2e9305b1ae7ddcd210a157f828eb6a57f656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:46:38.576 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:46:38 smithi103 podman[70212]: 2024-04-26 18:46:38.342448892 +0000 UTC m=+0.562536100 container cleanup e973bf7c756de0be52e200c13ddc2e9305b1ae7ddcd210a157f828eb6a57f656 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T18:46:38.577 INFO:journalctl@ceph.mgr.y.smithi103.stdout:Apr 26 18:46:38 smithi103 bash[70212]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-y 2024-04-26T18:46:38.706 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mgr.y.service' 2024-04-26T18:46:38.739 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:46:38.740 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-26T18:46:38.740 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-26T18:46:38.740 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mgr.z 2024-04-26T18:46:39.094 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:46:38 smithi174 systemd[1]: Stopping Ceph mgr.z for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:46:39.359 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:46:39 smithi174 podman[71496]: 2024-04-26 18:46:39.094811955 +0000 UTC m=+0.160427436 container died 04b25399f61fe7ca2dfb6b58cb0f5e509c066aee0e91f75c1f63a87ea424b3f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T18:46:39.689 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:46:39 smithi174 podman[71496]: 2024-04-26 18:46:39.458040503 +0000 UTC m=+0.523655980 container cleanup 04b25399f61fe7ca2dfb6b58cb0f5e509c066aee0e91f75c1f63a87ea424b3f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T18:46:39.689 INFO:journalctl@ceph.mgr.z.smithi174.stdout:Apr 26 18:46:39 smithi174 bash[71496]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mgr-z 2024-04-26T18:46:39.936 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mgr.z.service' 2024-04-26T18:46:39.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:46:39.969 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-26T18:46:39.970 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T18:46:39.970 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.0 2024-04-26T18:46:40.276 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:40 smithi084 systemd[1]: Stopping Ceph osd.0 for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:46:40.542 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:40 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0[43734]: 2024-04-26T18:46:40.275+0000 7feeed05d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:46:40.542 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:40 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0[43734]: 2024-04-26T18:46:40.275+0000 7feeed05d700 -1 osd.0 123 *** Got signal Terminated *** 2024-04-26T18:46:40.542 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:40 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0[43734]: 2024-04-26T18:46:40.275+0000 7feeed05d700 -1 osd.0 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:46:45.844 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:45 smithi084 podman[109452]: 2024-04-26 18:46:45.590721779 +0000 UTC m=+5.436565041 container died e7a8a1969e648c0dedc31100df4e8d65a61980ea9c84ad3a37990167c53c7df6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:46:46.345 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:46 smithi084 podman[109452]: 2024-04-26 18:46:46.118362143 +0000 UTC m=+5.964205388 container cleanup e7a8a1969e648c0dedc31100df4e8d65a61980ea9c84ad3a37990167c53c7df6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T18:46:46.345 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:46 smithi084 bash[109452]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0 2024-04-26T18:46:46.786 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:46 smithi084 podman[109481]: 2024-04-26 18:46:46.353463521 +0000 UTC m=+0.760875233 container remove e7a8a1969e648c0dedc31100df4e8d65a61980ea9c84ad3a37990167c53c7df6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:46:47.095 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:46 smithi084 podman[109575]: 2024-04-26 18:46:46.689609202 +0000 UTC m=+0.017672230 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:46:47.095 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:46 smithi084 podman[109575]: 2024-04-26 18:46:46.821293054 +0000 UTC m=+0.149356128 container create a423381583a08484ee6d703b3a3a86e4971e2e8dff4887fe7d237f83ef69f4ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-04-26T18:46:47.487 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:47 smithi084 podman[109575]: 2024-04-26 18:46:47.19127577 +0000 UTC m=+0.519338798 container init a423381583a08484ee6d703b3a3a86e4971e2e8dff4887fe7d237f83ef69f4ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T18:46:47.487 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:47 smithi084 podman[109575]: 2024-04-26 18:46:47.194636153 +0000 UTC m=+0.522699187 container start a423381583a08484ee6d703b3a3a86e4971e2e8dff4887fe7d237f83ef69f4ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T18:46:47.487 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:47 smithi084 podman[109575]: 2024-04-26 18:46:47.291029555 +0000 UTC m=+0.619092593 container attach a423381583a08484ee6d703b3a3a86e4971e2e8dff4887fe7d237f83ef69f4ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:46:47.844 INFO:journalctl@ceph.osd.0.smithi084.stdout:Apr 26 18:46:47 smithi084 podman[109575]: 2024-04-26 18:46:47.487935921 +0000 UTC m=+0.815998961 container died a423381583a08484ee6d703b3a3a86e4971e2e8dff4887fe7d237f83ef69f4ef (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:46:48.188 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T18:46:48.229 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:46:48.230 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T18:46:48.230 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T18:46:48.230 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.1 2024-04-26T18:46:48.547 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:48 smithi103 systemd[1]: Stopping Ceph osd.1 for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:46:48.877 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:48 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1[37335]: 2024-04-26T18:46:48.546+0000 7fc3e29a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:46:48.877 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:48 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1[37335]: 2024-04-26T18:46:48.546+0000 7fc3e29a8700 -1 osd.1 123 *** Got signal Terminated *** 2024-04-26T18:46:48.877 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:48 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1[37335]: 2024-04-26T18:46:48.546+0000 7fc3e29a8700 -1 osd.1 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:46:54.356 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:54 smithi103 podman[70340]: 2024-04-26 18:46:54.066151992 +0000 UTC m=+5.645957074 container died d09f6fd496d194087b1bf222c0ce24f4c7ac5b96c9aade17832749727e83fcad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T18:46:54.627 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:54 smithi103 podman[70340]: 2024-04-26 18:46:54.450301467 +0000 UTC m=+6.030106547 container cleanup d09f6fd496d194087b1bf222c0ce24f4c7ac5b96c9aade17832749727e83fcad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T18:46:54.627 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:54 smithi103 bash[70340]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1 2024-04-26T18:46:55.074 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:54 smithi103 podman[70359]: 2024-04-26 18:46:54.727036296 +0000 UTC m=+0.652465136 container remove d09f6fd496d194087b1bf222c0ce24f4c7ac5b96c9aade17832749727e83fcad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-04-26T18:46:55.372 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:55 smithi103 podman[70451]: 2024-04-26 18:46:54.97793988 +0000 UTC m=+0.018408954 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:46:55.372 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:55 smithi103 podman[70451]: 2024-04-26 18:46:55.103142161 +0000 UTC m=+0.143611209 container create a7c8c9d0ff3e81aae14791a2cdec6286244c5790eff7d1d7fb13fe6711ddb3c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424) 2024-04-26T18:46:55.627 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:55 smithi103 podman[70451]: 2024-04-26 18:46:55.37308702 +0000 UTC m=+0.413556069 container init a7c8c9d0ff3e81aae14791a2cdec6286244c5790eff7d1d7fb13fe6711ddb3c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T18:46:55.627 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:55 smithi103 podman[70451]: 2024-04-26 18:46:55.376463667 +0000 UTC m=+0.416932717 container start a7c8c9d0ff3e81aae14791a2cdec6286244c5790eff7d1d7fb13fe6711ddb3c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-04-26T18:46:55.627 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:55 smithi103 podman[70451]: 2024-04-26 18:46:55.489830213 +0000 UTC m=+0.530299269 container attach a7c8c9d0ff3e81aae14791a2cdec6286244c5790eff7d1d7fb13fe6711ddb3c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T18:46:55.985 INFO:journalctl@ceph.osd.1.smithi103.stdout:Apr 26 18:46:55 smithi103 podman[70451]: 2024-04-26 18:46:55.672118511 +0000 UTC m=+0.712587574 container died a7c8c9d0ff3e81aae14791a2cdec6286244c5790eff7d1d7fb13fe6711ddb3c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T18:46:56.320 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T18:46:56.359 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:46:56.359 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T18:46:56.361 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T18:46:56.361 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.2 2024-04-26T18:46:56.654 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:46:56 smithi174 systemd[1]: Stopping Ceph osd.2 for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:46:56.939 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:46:56 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2[37862]: 2024-04-26T18:46:56.653+0000 7f936ea0a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:46:56.939 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:46:56 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2[37862]: 2024-04-26T18:46:56.653+0000 7f936ea0a700 -1 osd.2 123 *** Got signal Terminated *** 2024-04-26T18:46:56.939 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:46:56 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2[37862]: 2024-04-26T18:46:56.653+0000 7f936ea0a700 -1 osd.2 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:47:02.421 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:47:02 smithi174 podman[71621]: 2024-04-26 18:47:02.13670953 +0000 UTC m=+5.583976150 container died a5b9bff37f57753bd8be9e6fcaabd08d10d0d18c9ace32d9aaef506e6c3740b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T18:47:02.689 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:47:02 smithi174 podman[71621]: 2024-04-26 18:47:02.529088222 +0000 UTC m=+5.976354843 container cleanup a5b9bff37f57753bd8be9e6fcaabd08d10d0d18c9ace32d9aaef506e6c3740b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-26T18:47:02.689 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:47:02 smithi174 bash[71621]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2 2024-04-26T18:47:03.155 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:47:02 smithi174 podman[71640]: 2024-04-26 18:47:02.797056739 +0000 UTC m=+0.657205289 container remove a5b9bff37f57753bd8be9e6fcaabd08d10d0d18c9ace32d9aaef506e6c3740b4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:47:03.439 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:47:03 smithi174 podman[71729]: 2024-04-26 18:47:03.058477639 +0000 UTC m=+0.017890250 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:47:03.439 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:47:03 smithi174 podman[71729]: 2024-04-26 18:47:03.173405461 +0000 UTC m=+0.132818111 container create 2eef426bd8030c416bc6b236a6fcb9bc2bb81181f8ae4039f6b38b86a90e7307 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:47:03.439 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:47:03 smithi174 podman[71729]: 2024-04-26 18:47:03.39235002 +0000 UTC m=+0.351762628 container init 2eef426bd8030c416bc6b236a6fcb9bc2bb81181f8ae4039f6b38b86a90e7307 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T18:47:03.439 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:47:03 smithi174 podman[71729]: 2024-04-26 18:47:03.395509798 +0000 UTC m=+0.354922414 container start 2eef426bd8030c416bc6b236a6fcb9bc2bb81181f8ae4039f6b38b86a90e7307 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T18:47:03.690 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:47:03 smithi174 podman[71729]: 2024-04-26 18:47:03.492686866 +0000 UTC m=+0.452099513 container attach 2eef426bd8030c416bc6b236a6fcb9bc2bb81181f8ae4039f6b38b86a90e7307 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:47:03.986 INFO:journalctl@ceph.osd.2.smithi174.stdout:Apr 26 18:47:03 smithi174 podman[71729]: 2024-04-26 18:47:03.690889817 +0000 UTC m=+0.650302427 container died 2eef426bd8030c416bc6b236a6fcb9bc2bb81181f8ae4039f6b38b86a90e7307 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:47:04.287 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T18:47:04.324 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:47:04.324 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T18:47:04.324 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T18:47:04.324 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.3 2024-04-26T18:47:04.638 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:04 smithi084 systemd[1]: Stopping Ceph osd.3 for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:47:05.094 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:04 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3[50414]: 2024-04-26T18:47:04.637+0000 7f4a37663700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:47:05.094 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:04 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3[50414]: 2024-04-26T18:47:04.637+0000 7f4a37663700 -1 osd.3 123 *** Got signal Terminated *** 2024-04-26T18:47:05.095 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:04 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3[50414]: 2024-04-26T18:47:04.637+0000 7f4a37663700 -1 osd.3 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:47:06.689 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:06 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:06.403+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:07.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:06 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:06.709+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:07.689 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:07 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:07.403+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:07.689 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:07 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:07.229+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:08.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:07 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:07.681+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:08.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:07 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:07.742+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:08.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:08 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:08.430+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:08.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:08 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:08.185+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:09.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:08 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:08.778+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:09.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:08 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:08.639+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:09.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:08 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:08.933+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:09.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:09 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:09.382+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:09.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:09 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:09.145+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:09.778 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:09 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3[50414]: 2024-04-26T18:47:09.403+0000 7f4a33c57700 -1 osd.3 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.117510+0000 front 2024-04-26T18:46:45.117545+0000 (oldest deadline 2024-04-26T18:47:08.617797+0000) 2024-04-26T18:47:09.779 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:09 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:09.603+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:10.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:09 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:09.779+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:10.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:09 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:09.890+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:10.380 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:10 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:10.125+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:10.429 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:10 smithi084 podman[109682]: 2024-04-26 18:47:10.152323392 +0000 UTC m=+5.638380051 container died f3c1755e99950fabadff5e2d054cc7f7b6287c98c151ebdf734ee46f76972f19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-26T18:47:10.688 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:10 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:10.381+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:10.786 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:10 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:10.607+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:10.786 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:10 smithi084 podman[109682]: 2024-04-26 18:47:10.512773263 +0000 UTC m=+5.998829921 container cleanup f3c1755e99950fabadff5e2d054cc7f7b6287c98c151ebdf734ee46f76972f19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:47:10.786 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:10 smithi084 bash[109682]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3 2024-04-26T18:47:11.094 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:10 smithi084 podman[109699]: 2024-04-26 18:47:10.839496648 +0000 UTC m=+0.682241577 container remove f3c1755e99950fabadff5e2d054cc7f7b6287c98c151ebdf734ee46f76972f19 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T18:47:11.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:10 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:10.786+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:11.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:10 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:10.882+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:11.402 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:11 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:11.113+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:11.459 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:11 smithi084 podman[109789]: 2024-04-26 18:47:11.034958457 +0000 UTC m=+0.017701072 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:47:11.459 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:11 smithi084 podman[109789]: 2024-04-26 18:47:11.148706576 +0000 UTC m=+0.131449166 container create b7368e3040503c7f9a3491eca7d428b137144a2256f5d069b1309dd95d34cb7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:47:11.459 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:11 smithi084 podman[109789]: 2024-04-26 18:47:11.360153473 +0000 UTC m=+0.342896070 container init b7368e3040503c7f9a3491eca7d428b137144a2256f5d069b1309dd95d34cb7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:47:11.459 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:11 smithi084 podman[109789]: 2024-04-26 18:47:11.363383029 +0000 UTC m=+0.346125621 container start b7368e3040503c7f9a3491eca7d428b137144a2256f5d069b1309dd95d34cb7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5) 2024-04-26T18:47:11.459 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:11 smithi084 podman[109789]: 2024-04-26 18:47:11.459550868 +0000 UTC m=+0.442293475 container attach b7368e3040503c7f9a3491eca7d428b137144a2256f5d069b1309dd95d34cb7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-26T18:47:11.689 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:11 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:11.402+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:11.831 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:11 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:11.648+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:11.831 INFO:journalctl@ceph.osd.3.smithi084.stdout:Apr 26 18:47:11 smithi084 podman[109789]: 2024-04-26 18:47:11.64802173 +0000 UTC m=+0.630764340 container died b7368e3040503c7f9a3491eca7d428b137144a2256f5d069b1309dd95d34cb7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-04-26T18:47:12.094 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:11 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:11.831+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:12.272 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T18:47:12.309 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:47:12.310 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T18:47:12.310 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T18:47:12.310 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.4 2024-04-26T18:47:12.337 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:11 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:11.880+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:12.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:12 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:12.377+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:12.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:12 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:12.135+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:12.627 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:12 smithi103 systemd[1]: Stopping Ceph osd.4 for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:47:12.844 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:12 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:12.679+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:13.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:12 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:12.837+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:13.127 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:12 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4[44113]: 2024-04-26T18:47:12.640+0000 7f4fe19d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:47:13.127 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:12 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4[44113]: 2024-04-26T18:47:12.640+0000 7f4fe19d8700 -1 osd.4 123 *** Got signal Terminated *** 2024-04-26T18:47:13.127 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:12 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4[44113]: 2024-04-26T18:47:12.640+0000 7f4fe19d8700 -1 osd.4 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:47:13.344 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:12 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:12.876+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:13.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:13 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:13.328+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:13.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:13 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:13.185+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:13.844 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:13 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:13.660+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:14.344 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:13 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:13.851+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:14.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:13 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:13.880+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:14.689 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:14 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:14.301+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:14.689 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:14 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:14.204+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:15.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:14 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:14.658+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:15.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:14 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:14.897+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:15.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:14 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:14.897+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:15.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:14 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:14.843+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:15.339 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:15 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:15.163+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:15.339 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:15 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:15.163+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:15.689 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:15 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:15.339+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:15.689 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:15 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:15.339+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:48.509328+0000 front 2024-04-26T18:46:48.509400+0000 (oldest deadline 2024-04-26T18:47:14.309900+0000) 2024-04-26T18:47:16.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:15 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:15.682+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:16.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:15 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:15.901+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:16.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:15 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:15.901+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:16.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:15 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:15.876+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:16.127 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:15 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:15.900+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:16.141 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:15 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:15.698+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:16.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:16 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:16.325+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:16.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:16 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:16.325+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:48.509328+0000 front 2024-04-26T18:46:48.509400+0000 (oldest deadline 2024-04-26T18:47:14.309900+0000) 2024-04-26T18:47:16.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:16 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:16.141+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:16.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:16 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:16.141+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:16.938 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:16 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:16.672+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:17.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:16 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:16.692+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:17.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:16 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:16.883+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:17.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:16 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:16.883+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:17.127 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:16 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:16.940+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:17.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:16 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:16.853+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:17.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:16 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:16.853+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:17.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:17 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:17.359+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:17.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:17 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:17.359+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:48.509328+0000 front 2024-04-26T18:46:48.509400+0000 (oldest deadline 2024-04-26T18:47:14.309900+0000) 2024-04-26T18:47:17.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:17 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:17.167+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:17.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:17 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:17.167+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:17.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:17 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:17.656+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:17.961 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:17 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:17.829+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:17.961 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:17 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:17.829+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:18.094 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:17 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:17.920+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:18.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:17 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:17.920+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:18.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:17 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:17.672+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:18.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:17 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:17.672+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:18.230 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:18 smithi103 podman[70573]: 2024-04-26 18:47:18.114456314 +0000 UTC m=+5.611135877 container died bee3b91c288bb75206a4392e224c1fa7845c6a72da12f481f46b1d0c740b8aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:47:18.230 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:17 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:17.959+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:18.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:18 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:18.366+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:18.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:18 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:18.366+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:48.509328+0000 front 2024-04-26T18:46:48.509400+0000 (oldest deadline 2024-04-26T18:47:14.309900+0000) 2024-04-26T18:47:18.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:18 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:18.155+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:18.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:18 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:18.155+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:18.860 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:18 smithi103 podman[70573]: 2024-04-26 18:47:18.515956726 +0000 UTC m=+6.012636281 container cleanup bee3b91c288bb75206a4392e224c1fa7845c6a72da12f481f46b1d0c740b8aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T18:47:18.860 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:18 smithi103 bash[70573]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4 2024-04-26T18:47:18.860 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:18 smithi103 podman[70590]: 2024-04-26 18:47:18.725675711 +0000 UTC m=+0.605153625 container remove bee3b91c288bb75206a4392e224c1fa7845c6a72da12f481f46b1d0c740b8aff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:47:18.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:18 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:18.673+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:19.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:18 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:18.946+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:19.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:18 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:18.946+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:19.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:18 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:18.701+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:19.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:18 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:18.701+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:19.127 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:19 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:19.000+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:19.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:18 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:18.858+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:19.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:18 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:18.858+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:19.412 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:19 smithi103 podman[70683]: 2024-04-26 18:47:19.035167755 +0000 UTC m=+0.018168466 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:47:19.412 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:19 smithi103 podman[70683]: 2024-04-26 18:47:19.135110244 +0000 UTC m=+0.118110998 container create 9b3d0fc4bf6124a5f2c8c410747da7d7c096aaf71596aaf1c40cf4c00589f141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:47:19.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:19 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:19.318+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:19.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:19 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:19.318+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:48.509328+0000 front 2024-04-26T18:46:48.509400+0000 (oldest deadline 2024-04-26T18:47:14.309900+0000) 2024-04-26T18:47:19.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:19 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:19.127+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:19.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:19 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:19.127+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:19.706 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:19 smithi103 podman[70683]: 2024-04-26 18:47:19.413160058 +0000 UTC m=+0.396160787 container init 9b3d0fc4bf6124a5f2c8c410747da7d7c096aaf71596aaf1c40cf4c00589f141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:47:19.706 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:19 smithi103 podman[70683]: 2024-04-26 18:47:19.416718364 +0000 UTC m=+0.399719083 container start 9b3d0fc4bf6124a5f2c8c410747da7d7c096aaf71596aaf1c40cf4c00589f141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:47:19.706 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:19 smithi103 podman[70683]: 2024-04-26 18:47:19.512951348 +0000 UTC m=+0.495952064 container attach 9b3d0fc4bf6124a5f2c8c410747da7d7c096aaf71596aaf1c40cf4c00589f141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T18:47:19.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:19 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:19.642+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:19.983 INFO:journalctl@ceph.osd.4.smithi103.stdout:Apr 26 18:47:19 smithi103 podman[70683]: 2024-04-26 18:47:19.706775979 +0000 UTC m=+0.689776690 container died 9b3d0fc4bf6124a5f2c8c410747da7d7c096aaf71596aaf1c40cf4c00589f141 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T18:47:19.984 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:19 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:19.983+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:19.984 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:19 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:19.899+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:19.984 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:19 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:19.899+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:20.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:19 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:19.975+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:20.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:19 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:19.975+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:20.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:19 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:19.729+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:20.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:19 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:19.729+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:20.437 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T18:47:20.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:20.307+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:20.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:20.307+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:48.509328+0000 front 2024-04-26T18:46:48.509400+0000 (oldest deadline 2024-04-26T18:47:14.309900+0000) 2024-04-26T18:47:20.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:20.106+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:20.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:20.106+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:20.514 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:47:20.514 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T18:47:20.514 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T18:47:20.514 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.5 2024-04-26T18:47:20.832 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:20 smithi174 systemd[1]: Stopping Ceph osd.5 for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:47:20.833 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:20.626+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:21.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:20 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:20.992+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:21.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:20 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:20.992+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:21.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:20 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:20.766+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:21.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:20 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:20.766+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:21.108 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:20.832+0000 7f6196021700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:47:21.109 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:20.832+0000 7f6196021700 -1 osd.5 123 *** Got signal Terminated *** 2024-04-26T18:47:21.109 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:20 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:20.832+0000 7f6196021700 -1 osd.5 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:47:21.126 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:21 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:21.017+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:21.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:20 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:20.874+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:21.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:20 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:20.874+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:21.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:21 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:21.108+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:21.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:21 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:21.108+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:21.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:21 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:21.284+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:21.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:21 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:21.284+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:48.509328+0000 front 2024-04-26T18:46:48.509400+0000 (oldest deadline 2024-04-26T18:47:14.309900+0000) 2024-04-26T18:47:21.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:21 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:21.579+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:22.003 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:21 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:21.728+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:22.003 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:21 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:21.728+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:22.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:22.003+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:22.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:22.003+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:22.377 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:21 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:21.997+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:22.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:21 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:21.923+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:22.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:21 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:21.923+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:22.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:22.081+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:22.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:22.081+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:22.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:22.304+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:22.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:22.304+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:48.509328+0000 front 2024-04-26T18:46:48.509400+0000 (oldest deadline 2024-04-26T18:47:14.309900+0000) 2024-04-26T18:47:22.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:22 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:22.559+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:23.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:23 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:23.003+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:23.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:23 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:23.003+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:23.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:22.757+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:23.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:22 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:22.757+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:23.377 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:22.971+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:23.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:22.897+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:23.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:22 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:22.897+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:23.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:23 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:23.053+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:23.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:23 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:23.053+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:23.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:23 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:23.263+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:23.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:23 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:23.263+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:48.509328+0000 front 2024-04-26T18:46:48.509400+0000 (oldest deadline 2024-04-26T18:47:14.309900+0000) 2024-04-26T18:47:23.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:23 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:23.576+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:23.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:23 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:23.576+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:24.094 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:24.005+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:24.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:24.005+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:24.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:24.005+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:57.669855+0000 front 2024-04-26T18:46:57.669772+0000 (oldest deadline 2024-04-26T18:47:23.569933+0000) 2024-04-26T18:47:24.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:23 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:23.778+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:24.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:23 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:23.778+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:24.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:23 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:23.778+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:24.272 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:24 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:24.012+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:24.272 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:24 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:24.012+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:24.377 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:24 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:24.003+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:24.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:23 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:23.921+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:24.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:23 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:23.921+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:24.551 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:24 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:24.272+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:24.551 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:24 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:24.272+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:48.509328+0000 front 2024-04-26T18:46:48.509400+0000 (oldest deadline 2024-04-26T18:47:14.309900+0000) 2024-04-26T18:47:24.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:24 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:24.551+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:24.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:24 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:24.551+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:25.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:24.995+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:25.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:24.995+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:25.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:24.995+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:57.669855+0000 front 2024-04-26T18:46:57.669772+0000 (oldest deadline 2024-04-26T18:47:23.569933+0000) 2024-04-26T18:47:25.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:24.769+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:25.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:24.769+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:25.096 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:24 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:24.769+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:25.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:24 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:24.916+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:25.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:24 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:24.916+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:25.377 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:25 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:25.041+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:25.378 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:25 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:25.041+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:25.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:25 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:25.307+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.108418+0000 front 2024-04-26T18:46:45.108341+0000 (oldest deadline 2024-04-26T18:47:06.208313+0000) 2024-04-26T18:47:25.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:25 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:25.307+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:48.509328+0000 front 2024-04-26T18:46:48.509400+0000 (oldest deadline 2024-04-26T18:47:14.309900+0000) 2024-04-26T18:47:25.439 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:25 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5[44817]: 2024-04-26T18:47:25.307+0000 7f618e60d700 -1 osd.5 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.111325+0000 front 2024-04-26T18:47:00.111352+0000 (oldest deadline 2024-04-26T18:47:24.811463+0000) 2024-04-26T18:47:25.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:25 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:25.061+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:25.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:25 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:25.061+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:25.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:25 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:25.061+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:25.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:25 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:25.575+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:25.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:25 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:25.575+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:26.043 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:25 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:25.723+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:26.043 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:25 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:25.723+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:26.043 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:25 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:25.723+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:26.127 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:26 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:26.082+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:26.127 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:26 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:26.082+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:26.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:25 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:25.866+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:26.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:25 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:25.866+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:26.128 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:25 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:25.866+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:26.344 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:26 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:26.043+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:26.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:26 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:26.043+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:26.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:26 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:26.043+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:57.669855+0000 front 2024-04-26T18:46:57.669772+0000 (oldest deadline 2024-04-26T18:47:23.569933+0000) 2024-04-26T18:47:26.425 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:26 smithi174 podman[71848]: 2024-04-26 18:47:26.153253196 +0000 UTC m=+5.445329563 container died 261f011c9b7501ba4b49c8bb403076126ae6107c6dce4dd63d9bb40a3b4b2539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T18:47:26.426 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:26 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:26.040+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:26.426 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:26 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:26.040+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:26.426 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:26 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:26.040+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:26.689 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:26 smithi174 podman[71848]: 2024-04-26 18:47:26.517322513 +0000 UTC m=+5.809398891 container cleanup 261f011c9b7501ba4b49c8bb403076126ae6107c6dce4dd63d9bb40a3b4b2539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:47:26.689 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:26 smithi174 bash[71848]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5 2024-04-26T18:47:26.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:26 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:26.545+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:26.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:26 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:26.545+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:27.059 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:26 smithi174 podman[71867]: 2024-04-26 18:47:26.802071215 +0000 UTC m=+0.649554992 container remove 261f011c9b7501ba4b49c8bb403076126ae6107c6dce4dd63d9bb40a3b4b2539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T18:47:27.089 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:26 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:26.741+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:27.089 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:26 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:26.741+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:27.089 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:26 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:26.741+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:27.344 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:27 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:27.089+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:27.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:27 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:27.089+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:27.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:27 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:27.089+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:57.669855+0000 front 2024-04-26T18:46:57.669772+0000 (oldest deadline 2024-04-26T18:47:23.569933+0000) 2024-04-26T18:47:27.377 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:27 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:27.107+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:27.377 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:27 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:27.107+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:27.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:26 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:26.894+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:27.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:26 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:26.894+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:27.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:26 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:26.894+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:27.389 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:27 smithi174 podman[71957]: 2024-04-26 18:47:27.033299253 +0000 UTC m=+0.017895024 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:47:27.389 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:27 smithi174 podman[71957]: 2024-04-26 18:47:27.145563186 +0000 UTC m=+0.130158976 container create d3c8c054d595351936cfe397e891d8f96b55ca1a835c77e78aff86282a3bb7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-26T18:47:27.389 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:27 smithi174 podman[71957]: 2024-04-26 18:47:27.39012982 +0000 UTC m=+0.374725566 container init d3c8c054d595351936cfe397e891d8f96b55ca1a835c77e78aff86282a3bb7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T18:47:27.390 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:27 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:27.058+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:27.390 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:27 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:27.058+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:27.390 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:27 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:27.058+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:27.686 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:27 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:27.521+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:27.686 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:27 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:27.521+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:27.686 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:27 smithi174 podman[71957]: 2024-04-26 18:47:27.393225185 +0000 UTC m=+0.377820939 container start d3c8c054d595351936cfe397e891d8f96b55ca1a835c77e78aff86282a3bb7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS) 2024-04-26T18:47:27.690 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:27 smithi174 podman[71957]: 2024-04-26 18:47:27.498548805 +0000 UTC m=+0.483144552 container attach d3c8c054d595351936cfe397e891d8f96b55ca1a835c77e78aff86282a3bb7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T18:47:27.939 INFO:journalctl@ceph.osd.5.smithi174.stdout:Apr 26 18:47:27 smithi174 podman[71957]: 2024-04-26 18:47:27.687041939 +0000 UTC m=+0.671637745 container died d3c8c054d595351936cfe397e891d8f96b55ca1a835c77e78aff86282a3bb7a4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T18:47:28.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:27 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:27.713+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:28.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:27 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:27.713+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:28.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:27 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:27.713+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:28.127 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:28 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:28.098+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:28.127 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:28 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:28.098+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:28.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:27 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:27.868+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:28.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:27 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:27.868+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:28.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:27 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:27.868+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:28.260 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:28 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:28.023+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:28.260 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:28 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:28.023+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:28.260 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:28 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:28.023+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:28.284 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T18:47:28.325 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:47:28.326 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T18:47:28.326 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-26T18:47:28.326 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.6 2024-04-26T18:47:28.355 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:28 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:28.104+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:28.356 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:28 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:28.104+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:28.356 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:28 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:28.104+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:57.669855+0000 front 2024-04-26T18:46:57.669772+0000 (oldest deadline 2024-04-26T18:47:23.569933+0000) 2024-04-26T18:47:28.622 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:28 smithi084 systemd[1]: Stopping Ceph osd.6 for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:47:28.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:28 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:28.500+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:28.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:28 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:28.500+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:29.087 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:28 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:28.704+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:29.087 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:28 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:28.704+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:29.088 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:28 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:28.704+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:29.088 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:28 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:28.622+0000 7f0504c82700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:47:29.088 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:28 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:28.622+0000 7f0504c82700 -1 osd.6 123 *** Got signal Terminated *** 2024-04-26T18:47:29.088 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:28 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:28.622+0000 7f0504c82700 -1 osd.6 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:47:29.088 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:29 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:29.087+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:29.088 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:29 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:29.087+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:29.088 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:29 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:29.087+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:57.669855+0000 front 2024-04-26T18:46:57.669772+0000 (oldest deadline 2024-04-26T18:47:23.569933+0000) 2024-04-26T18:47:29.377 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:29 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:29.069+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:29.377 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:29 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:29.069+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:29.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:28 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:28.912+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:29.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:28 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:28.912+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:29.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:28 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:28.912+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:29.438 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:29 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:29.067+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:29.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:29 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:29.067+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:29.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:29 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:29.067+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:29.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:29 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:29.491+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:29.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:29 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:29.491+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:30.070 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:29 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:29.678+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:30.070 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:29 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:29.678+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:30.070 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:29 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:29.678+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:30.127 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:30 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:30.102+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:30.127 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:30 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:30.102+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:30.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:29 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:29.868+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:30.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:29 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:29.868+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:30.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:29 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:29.868+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:30.344 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:30 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:30.070+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:30.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:30 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:30.070+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:30.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:30 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:30.070+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:57.669855+0000 front 2024-04-26T18:46:57.669772+0000 (oldest deadline 2024-04-26T18:47:23.569933+0000) 2024-04-26T18:47:30.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:30 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:30.045+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:30.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:30 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:30.045+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:30.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:30 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:30.045+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:30.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:30 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:30.520+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:30.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:30 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:30.520+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:31.072 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:30 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:30.687+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:31.072 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:30 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:30.687+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:31.072 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:30 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:30.687+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:31.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:30 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:30.833+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:31.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:30 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:30.833+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:31.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:30 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:30.833+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:31.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:30 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:30.833+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:31.344 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:31 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:31.071+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:31.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:31 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:31.071+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:31.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:31 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:31.071+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:57.669855+0000 front 2024-04-26T18:46:57.669772+0000 (oldest deadline 2024-04-26T18:47:23.569933+0000) 2024-04-26T18:47:31.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:31 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:31.082+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:31.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:31 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:31.082+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:31.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:31 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:31.082+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:31.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:31 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:31.140+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:31.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:31 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:31.140+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:31.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:31 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:31.477+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:31.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:31 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:31.477+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:32.041 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:31 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:31.714+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:32.041 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:31 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:31.714+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:32.041 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:31 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:31.714+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:32.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:31 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:31.849+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:32.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:31 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:31.849+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:32.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:31 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:31.849+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:32.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:31 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:31.849+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:32.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:32 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:32.040+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:32.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:32 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:32.040+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:32.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:32 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:32.040+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:57.669855+0000 front 2024-04-26T18:46:57.669772+0000 (oldest deadline 2024-04-26T18:47:23.569933+0000) 2024-04-26T18:47:32.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:32 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:32.087+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:32.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:32 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:32.087+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:32.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:32 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:32.087+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:32.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:32 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:32.183+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:32.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:32 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:32.183+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:32.628 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:32 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:32.183+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:07.722562+0000 front 2024-04-26T18:47:07.722641+0000 (oldest deadline 2024-04-26T18:47:31.222245+0000) 2024-04-26T18:47:32.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:32 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:32.478+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:32.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:32 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:32.478+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:33.011 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:32 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:32.744+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:33.011 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:32 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:32.744+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:33.011 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:32 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:32.744+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:33.011 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:32 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:32.744+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:33.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:32 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:32.814+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:33.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:32 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:32.814+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:33.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:32 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:32.814+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:33.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:32 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:32.814+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:33.344 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:33 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:33.011+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.867648+0000 front 2024-04-26T18:46:44.867616+0000 (oldest deadline 2024-04-26T18:47:07.167664+0000) 2024-04-26T18:47:33.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:33 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:33.011+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.468752+0000 front 2024-04-26T18:46:52.468764+0000 (oldest deadline 2024-04-26T18:47:14.768708+0000) 2024-04-26T18:47:33.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:33 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:33.011+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:57.669855+0000 front 2024-04-26T18:46:57.669772+0000 (oldest deadline 2024-04-26T18:47:23.569933+0000) 2024-04-26T18:47:33.345 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:33 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6[57291]: 2024-04-26T18:47:33.011+0000 7f0501276700 -1 osd.6 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.172696+0000 front 2024-04-26T18:47:09.172737+0000 (oldest deadline 2024-04-26T18:47:32.072843+0000) 2024-04-26T18:47:33.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:33 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:33.091+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:33.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:33 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:33.091+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:33.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:33 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:33.091+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:33.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:33 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:33.181+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:33.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:33 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:33.181+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:33.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:33 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:33.181+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:07.722562+0000 front 2024-04-26T18:47:07.722641+0000 (oldest deadline 2024-04-26T18:47:31.222245+0000) 2024-04-26T18:47:33.716 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:33 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:33.460+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:33.716 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:33 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:33.460+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:34.067 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:33 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:33.732+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:34.067 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:33 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:33.732+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:34.067 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:33 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:33.732+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:34.067 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:33 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:33.732+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:34.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:33 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:33.833+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:34.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:33 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:33.833+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:34.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:33 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:33.833+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:34.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:33 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:33.833+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:34.344 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:34 smithi084 podman[109906]: 2024-04-26 18:47:34.067506053 +0000 UTC m=+5.545048243 container died 1e7566819412fda6150629a30bfae9677bf64d559a46285ec293d8f38a502ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:47:34.432 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:34 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:34.102+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:34.432 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:34 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:34.103+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:34.432 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:34 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:34.103+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:34.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:34 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:34.138+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:34.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:34 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:34.138+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:34.628 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:34 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:34.138+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:07.722562+0000 front 2024-04-26T18:47:07.722641+0000 (oldest deadline 2024-04-26T18:47:31.222245+0000) 2024-04-26T18:47:34.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:34 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:34.431+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:34.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:34 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:34.431+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:34.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:34 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:34.431+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:34.765 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:34 smithi084 podman[109906]: 2024-04-26 18:47:34.547743415 +0000 UTC m=+6.025285593 container cleanup 1e7566819412fda6150629a30bfae9677bf64d559a46285ec293d8f38a502ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:47:34.765 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:34 smithi084 bash[109906]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6 2024-04-26T18:47:35.095 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:34 smithi084 podman[109924]: 2024-04-26 18:47:34.84911952 +0000 UTC m=+0.777746639 container remove 1e7566819412fda6150629a30bfae9677bf64d559a46285ec293d8f38a502ae5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T18:47:35.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:34 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:34.764+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:35.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:34 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:34.764+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:35.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:34 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:34.764+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:35.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:34 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:34.764+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:35.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:34 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:34.867+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:35.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:34 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:34.867+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:35.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:34 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:34.867+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:35.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:34 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:34.867+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:35.386 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:35 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:35.096+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:35.386 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:35 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:35.096+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:35.386 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:35 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:35.096+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:35.436 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:35 smithi084 podman[110011]: 2024-04-26 18:47:35.068980025 +0000 UTC m=+0.018191006 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:47:35.436 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:35 smithi084 podman[110011]: 2024-04-26 18:47:35.191691479 +0000 UTC m=+0.140902456 container create 6020276617a5e96ec97ade3f8d5605267461d6f2a40e94a7605bc7285ce398aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6-deactivate, ceph=True, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:47:35.436 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:35 smithi084 podman[110011]: 2024-04-26 18:47:35.436834131 +0000 UTC m=+0.386045110 container init 6020276617a5e96ec97ade3f8d5605267461d6f2a40e94a7605bc7285ce398aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T18:47:35.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:35 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:35.142+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:35.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:35 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:35.142+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:35.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:35 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:35.142+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:07.722562+0000 front 2024-04-26T18:47:07.722641+0000 (oldest deadline 2024-04-26T18:47:31.222245+0000) 2024-04-26T18:47:35.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:35 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:35.386+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:35.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:35 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:35.386+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:35.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:35 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:35.386+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:35.727 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:35 smithi084 podman[110011]: 2024-04-26 18:47:35.440465882 +0000 UTC m=+0.389676861 container start 6020276617a5e96ec97ade3f8d5605267461d6f2a40e94a7605bc7285ce398aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=) 2024-04-26T18:47:35.727 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:35 smithi084 podman[110011]: 2024-04-26 18:47:35.536107464 +0000 UTC m=+0.485318466 container attach 6020276617a5e96ec97ade3f8d5605267461d6f2a40e94a7605bc7285ce398aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:47:36.021 INFO:journalctl@ceph.osd.6.smithi084.stdout:Apr 26 18:47:35 smithi084 podman[110011]: 2024-04-26 18:47:35.728018771 +0000 UTC m=+0.677229753 container died 6020276617a5e96ec97ade3f8d5605267461d6f2a40e94a7605bc7285ce398aa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:47:36.021 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:35 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:35.804+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:36.022 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:35 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:35.804+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:36.022 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:35 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:35.804+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:36.022 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:35 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:35.804+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:36.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:35 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:35.831+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:36.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:35 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:35.831+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:36.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:35 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:35.831+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:36.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:35 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:35.831+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:36.316 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.6.service' 2024-04-26T18:47:36.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:47:36.393 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-26T18:47:36.393 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-26T18:47:36.393 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.7 2024-04-26T18:47:36.394 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:36 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:36.101+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:36.394 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:36 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:36.101+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:36.394 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:36 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:36.101+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:36.444 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:36 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:36.185+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:36.444 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:36 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:36.185+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:36.444 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:36 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:36.185+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:07.722562+0000 front 2024-04-26T18:47:07.722641+0000 (oldest deadline 2024-04-26T18:47:31.222245+0000) 2024-04-26T18:47:36.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:36 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:36.392+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:36.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:36 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:36.392+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:36.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:36 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:36.392+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:36.711 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:36 smithi103 systemd[1]: Stopping Ceph osd.7 for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:47:37.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:36 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:36.840+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:37.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:36 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:36.840+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:37.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:36 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:36.840+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:37.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:36 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:36.840+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:37.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:36 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:36.843+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:37.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:36 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:36.843+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:37.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:36 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:36.843+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:37.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:36 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:36.843+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:37.127 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:36 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:36.711+0000 7f09a5ddd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:47:37.127 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:36 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:36.711+0000 7f09a5ddd700 -1 osd.7 123 *** Got signal Terminated *** 2024-04-26T18:47:37.127 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:36 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:36.711+0000 7f09a5ddd700 -1 osd.7 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:47:37.397 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:37 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:37.123+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:37.397 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:37 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:37.123+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:37.397 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:37 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:37.123+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:37.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:37 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:37.155+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:37.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:37 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:37.155+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:37.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:37 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:37.155+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:07.722562+0000 front 2024-04-26T18:47:07.722641+0000 (oldest deadline 2024-04-26T18:47:31.222245+0000) 2024-04-26T18:47:37.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:37 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:37.396+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:37.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:37 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:37.396+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:37.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:37 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:37.396+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:38.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:37 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:37.824+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:38.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:37 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:37.824+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:38.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:37 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:37.824+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:38.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:37 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:37.824+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:38.344 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:37 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:37.853+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:38.345 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:37 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:37.853+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:38.345 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:37 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:37.853+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:38.345 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:37 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:37.853+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:38.400 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:38 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:38.178+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:38.400 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:38 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:38.178+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:38.400 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:38 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:38.178+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:07.722562+0000 front 2024-04-26T18:47:07.722641+0000 (oldest deadline 2024-04-26T18:47:31.222245+0000) 2024-04-26T18:47:38.425 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:38 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:38.122+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:38.425 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:38 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:38.122+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:38.425 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:38 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:38.122+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:38.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:38 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:38.425+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:38.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:38 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:38.425+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:38.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:38 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:38.425+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:39.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:38 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:38.832+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:39.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:38 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:38.832+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:39.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:38 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:38.832+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:39.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:38 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:38.832+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:39.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:38 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:38.803+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:39.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:38 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:38.803+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:39.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:38 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:38.803+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:39.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:38 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:38.803+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:39.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:38 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:38.803+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:39.391 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:39 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:39.105+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:39.391 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:39 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:39.105+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:39.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:39 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:39.105+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:39.392 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:39 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:39.105+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:39.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:39 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:39.133+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:39.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:39 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:39.133+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:39.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:39 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:39.133+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:07.722562+0000 front 2024-04-26T18:47:07.722641+0000 (oldest deadline 2024-04-26T18:47:31.222245+0000) 2024-04-26T18:47:39.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:39 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:39.392+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:39.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:39 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:39.392+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:39.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:39 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:39.392+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:40.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:39 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:39.820+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:40.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:39 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:39.820+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:40.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:39 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:39.820+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:40.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:39 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:39.820+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:40.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:39 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:39.827+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:40.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:39 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:39.827+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:40.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:39 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:39.827+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:40.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:39 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:39.827+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:40.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:39 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:39.827+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:40.410 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:40 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:40.059+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:40.411 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:40 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:40.059+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:40.411 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:40 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:40.059+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:40.411 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:40 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:40.059+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:40.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:40 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:40.163+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:40.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:40 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:40.163+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:40.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:40 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:40.163+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:07.722562+0000 front 2024-04-26T18:47:07.722641+0000 (oldest deadline 2024-04-26T18:47:31.222245+0000) 2024-04-26T18:47:40.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:40 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:40.163+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.023516+0000 front 2024-04-26T18:47:17.023435+0000 (oldest deadline 2024-04-26T18:47:39.323535+0000) 2024-04-26T18:47:40.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:40 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:40.411+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:40.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:40 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:40.411+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:40.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:40 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:40.411+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:40.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:40 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:40.411+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:14.394765+0000 front 2024-04-26T18:47:14.394804+0000 (oldest deadline 2024-04-26T18:47:40.294843+0000) 2024-04-26T18:47:41.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:40 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:40.805+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:41.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:40 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:40.805+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:41.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:40 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:40.805+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:41.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:40 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:40.805+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:41.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:40 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:40.801+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:41.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:40 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:40.801+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:41.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:40 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:40.801+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:41.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:40 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:40.801+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:41.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:40 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:40.801+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:41.426 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:41 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:41.031+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:41.426 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:41 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:41.031+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:41.426 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:41 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:41.031+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:41.426 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:41 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:41.031+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:41.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:41 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:41.204+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.619605+0000 front 2024-04-26T18:46:49.619920+0000 (oldest deadline 2024-04-26T18:47:14.919744+0000) 2024-04-26T18:47:41.627 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:41 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:41.204+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.721448+0000 front 2024-04-26T18:47:00.721339+0000 (oldest deadline 2024-04-26T18:47:24.221203+0000) 2024-04-26T18:47:41.628 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:41 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:41.204+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:07.722562+0000 front 2024-04-26T18:47:07.722641+0000 (oldest deadline 2024-04-26T18:47:31.222245+0000) 2024-04-26T18:47:41.628 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:41 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7[50801]: 2024-04-26T18:47:41.204+0000 7f099e3c9700 -1 osd.7 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.023516+0000 front 2024-04-26T18:47:17.023435+0000 (oldest deadline 2024-04-26T18:47:39.323535+0000) 2024-04-26T18:47:41.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:41 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:41.426+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:41.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:41 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:41.426+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:41.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:41 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:41.426+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:41.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:41 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:41.426+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:14.394765+0000 front 2024-04-26T18:47:14.394804+0000 (oldest deadline 2024-04-26T18:47:40.294843+0000) 2024-04-26T18:47:42.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:41 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:41.812+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:42.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:41 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:41.812+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:42.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:41 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:41.812+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:42.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:41 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:41.812+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:42.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:41 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:41.812+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:42.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:41 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:41.810+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:42.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:41 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:41.810+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:42.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:41 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:41.810+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:42.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:41 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:41.810+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:42.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:41 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:41.810+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:42.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:42 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:42.022+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:42.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:42 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:42.022+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:42.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:42 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:42.022+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:42.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:42 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:42.022+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:42.546 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:42 smithi103 podman[70788]: 2024-04-26 18:47:42.257759891 +0000 UTC m=+5.667408151 container died d6bde4067942929bff98d3f2c2c81ea0cd4723e075c53c02f97e4a66ffd20fb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-26T18:47:42.851 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:42 smithi103 podman[70788]: 2024-04-26 18:47:42.646027182 +0000 UTC m=+6.055675420 container cleanup d6bde4067942929bff98d3f2c2c81ea0cd4723e075c53c02f97e4a66ffd20fb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True) 2024-04-26T18:47:42.851 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:42 smithi103 bash[70788]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7 2024-04-26T18:47:42.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:42 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:42.469+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:42.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:42 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:42.469+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:42.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:42 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:42.469+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:42.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:42 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:42.469+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:14.394765+0000 front 2024-04-26T18:47:14.394804+0000 (oldest deadline 2024-04-26T18:47:40.294843+0000) 2024-04-26T18:47:43.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:42.801+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:43.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:42.801+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:43.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:42.801+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:43.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:42.801+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:43.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:42 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:42.801+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:43.127 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:42 smithi103 podman[70818]: 2024-04-26 18:47:42.897549816 +0000 UTC m=+0.639010259 container remove d6bde4067942929bff98d3f2c2c81ea0cd4723e075c53c02f97e4a66ffd20fb3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T18:47:43.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:42 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:42.851+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:43.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:42 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:42.851+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:43.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:42 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:42.851+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:43.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:42 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:42.851+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:43.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:42 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:42.851+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:43.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:43 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:43.037+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:43.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:43 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:43.037+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:43.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:43 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:43.037+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:43.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:43 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:43.037+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:43.508 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:43 smithi103 podman[70904]: 2024-04-26 18:47:43.114931397 +0000 UTC m=+0.018769671 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:47:43.508 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:43 smithi103 podman[70904]: 2024-04-26 18:47:43.240130435 +0000 UTC m=+0.143968713 container create 1f5ae4b683584471e4ffc934f7f5d8e8f8fb58f4538703fc712e90d356bac5d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-04-26T18:47:43.509 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:43 smithi103 podman[70904]: 2024-04-26 18:47:43.509241608 +0000 UTC m=+0.413079881 container init 1f5ae4b683584471e4ffc934f7f5d8e8f8fb58f4538703fc712e90d356bac5d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:47:43.807 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:43 smithi103 podman[70904]: 2024-04-26 18:47:43.512541916 +0000 UTC m=+0.416380196 container start 1f5ae4b683584471e4ffc934f7f5d8e8f8fb58f4538703fc712e90d356bac5d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-26T18:47:43.807 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:43 smithi103 podman[70904]: 2024-04-26 18:47:43.609568881 +0000 UTC m=+0.513407162 container attach 1f5ae4b683584471e4ffc934f7f5d8e8f8fb58f4538703fc712e90d356bac5d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:47:43.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:43 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:43.440+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:43.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:43 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:43.440+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:43.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:43 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:43.440+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:43.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:43 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:43.440+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:14.394765+0000 front 2024-04-26T18:47:14.394804+0000 (oldest deadline 2024-04-26T18:47:40.294843+0000) 2024-04-26T18:47:44.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:43 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:43.808+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:44.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:43 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:43.808+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:44.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:43 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:43.808+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:44.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:43 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:43.808+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:44.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:43 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:43.808+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:44.104 INFO:journalctl@ceph.osd.7.smithi103.stdout:Apr 26 18:47:43 smithi103 podman[70904]: 2024-04-26 18:47:43.808181441 +0000 UTC m=+0.712019714 container died 1f5ae4b683584471e4ffc934f7f5d8e8f8fb58f4538703fc712e90d356bac5d5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T18:47:44.104 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:43 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:43.828+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:44.104 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:43 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:43.828+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:44.104 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:43 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:43.828+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:44.104 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:43 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:43.828+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:44.104 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:43 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:43.828+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:44.425 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.7.service' 2024-04-26T18:47:44.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:44 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:44.050+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:44.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:44 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:44.050+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:44.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:44 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:44.050+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:44.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:44 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:44.050+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:44.463 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:47:44.463 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-26T18:47:44.463 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-26T18:47:44.463 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.8 2024-04-26T18:47:44.780 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:44 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:44.457+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:44.780 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:44 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:44.457+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:44.780 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:44 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:44.457+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:44.781 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:44 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:44.457+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:14.394765+0000 front 2024-04-26T18:47:14.394804+0000 (oldest deadline 2024-04-26T18:47:40.294843+0000) 2024-04-26T18:47:44.781 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:44 smithi174 systemd[1]: Stopping Ceph osd.8 for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:47:44.781 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:44 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:44.779+0000 7f2292217700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:47:44.781 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:44 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:44.779+0000 7f2292217700 -1 osd.8 123 *** Got signal Terminated *** 2024-04-26T18:47:44.781 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:44 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:44.779+0000 7f2292217700 -1 osd.8 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:47:45.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:44 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:44.787+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:45.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:44 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:44.787+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:45.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:44 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:44.787+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:45.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:44 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:44.787+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:45.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:44 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:44.787+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:45.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:44 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:44.866+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:45.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:44 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:44.866+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:45.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:44 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:44.866+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:45.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:44 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:44.866+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:45.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:44 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:44.866+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:45.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:45 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:45.086+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:45.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:45 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:45.086+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:45.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:45 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:45.086+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:45.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:45 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:45.086+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:45.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:45 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:45.447+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:45.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:45 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:45.447+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:45.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:45 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:45.447+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:45.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:45 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:45.447+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:14.394765+0000 front 2024-04-26T18:47:14.394804+0000 (oldest deadline 2024-04-26T18:47:40.294843+0000) 2024-04-26T18:47:46.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:45 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:45.812+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:46.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:45 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:45.812+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:46.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:45 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:45.812+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:46.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:45 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:45.812+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:46.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:45 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:45.812+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:46.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:45 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:45.873+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:46.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:45 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:45.873+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:46.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:45 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:45.873+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:46.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:45 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:45.873+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:46.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:45 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:45.873+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:46.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:46 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:46.095+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:46.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:46 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:46.095+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:46.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:46 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:46.095+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:46.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:46 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:46.095+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:46.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:46 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:46.449+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:46.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:46 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:46.449+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:46.940 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:46 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:46.449+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:46.940 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:46 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:46.449+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:14.394765+0000 front 2024-04-26T18:47:14.394804+0000 (oldest deadline 2024-04-26T18:47:40.294843+0000) 2024-04-26T18:47:47.344 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:46 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:46.846+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:47.345 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:46 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:46.846+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:47.345 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:46 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:46.846+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:47.345 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:46 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:46.846+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:47.345 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:46 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:46.846+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:47.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:46 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:46.912+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:47.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:46 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:46.912+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:47.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:46 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:46.912+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:47.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:46 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:46.912+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:47.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:46 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:46.912+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:47.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:47 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:47.077+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:47.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:47 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:47.077+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:47.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:47 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:47.077+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:47.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:47 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:47.077+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:47.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:47 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:47.439+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:47.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:47 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:47.439+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:47.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:47 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:47.439+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:47.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:47 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:47.439+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:14.394765+0000 front 2024-04-26T18:47:14.394804+0000 (oldest deadline 2024-04-26T18:47:40.294843+0000) 2024-04-26T18:47:47.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:47 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:47.439+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:24.196548+0000 front 2024-04-26T18:47:24.196584+0000 (oldest deadline 2024-04-26T18:47:47.096607+0000) 2024-04-26T18:47:48.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:47 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:47.819+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:48.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:47 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:47.819+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:48.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:47 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:47.819+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:48.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:47 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:47.819+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:48.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:47 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:47.819+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:48.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:47 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:47.885+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:48.378 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:47 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:47.885+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:48.378 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:47 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:47.885+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:48.378 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:47 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:47.885+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:48.378 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:47 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:47.885+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:48.378 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:47 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:47.885+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:47:48.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:48 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:48.043+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:48.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:48 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:48.043+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:48.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:48 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:48.043+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:48.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:48 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:48.043+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:48.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:48 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:48.469+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:48.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:48 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:48.469+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:48.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:48 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:48.469+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:48.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:48 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:48.469+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:14.394765+0000 front 2024-04-26T18:47:14.394804+0000 (oldest deadline 2024-04-26T18:47:40.294843+0000) 2024-04-26T18:47:48.939 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:48 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:48.469+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:24.196548+0000 front 2024-04-26T18:47:24.196584+0000 (oldest deadline 2024-04-26T18:47:47.096607+0000) 2024-04-26T18:47:49.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:48 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:48.829+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:49.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:48 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:48.829+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:49.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:48 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:48.829+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:49.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:48 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:48.829+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:49.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:48 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:48.829+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:49.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:48 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:48.926+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:49.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:48 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:48.926+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:49.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:48 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:48.926+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:49.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:48 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:48.926+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:49.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:48 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:48.926+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:49.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:48 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:48.926+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:47:49.425 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:49 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:49.071+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:49.425 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:49 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:49.071+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:49.425 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:49 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:49.071+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:49.425 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:49 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:49.071+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:49.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:49 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:49.425+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.091555+0000 front 2024-04-26T18:46:51.091748+0000 (oldest deadline 2024-04-26T18:47:15.191502+0000) 2024-04-26T18:47:49.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:49 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:49.425+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:58.592872+0000 front 2024-04-26T18:46:58.592895+0000 (oldest deadline 2024-04-26T18:47:23.293068+0000) 2024-04-26T18:47:49.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:49 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:49.425+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:08.494539+0000 front 2024-04-26T18:47:08.494467+0000 (oldest deadline 2024-04-26T18:47:34.394388+0000) 2024-04-26T18:47:49.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:49 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:49.425+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:14.394765+0000 front 2024-04-26T18:47:14.394804+0000 (oldest deadline 2024-04-26T18:47:40.294843+0000) 2024-04-26T18:47:49.689 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:49 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8[51517]: 2024-04-26T18:47:49.425+0000 7f228a803700 -1 osd.8 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:24.196548+0000 front 2024-04-26T18:47:24.196584+0000 (oldest deadline 2024-04-26T18:47:47.096607+0000) 2024-04-26T18:47:50.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:49 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:49.802+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:50.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:49 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:49.802+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:50.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:49 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:49.802+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:50.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:49 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:49.802+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:50.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:49 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:49.802+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:50.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:49 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:49.899+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:50.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:49 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:49.899+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:50.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:49 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:49.899+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:50.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:49 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:49.899+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:50.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:49 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:49.899+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:50.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:49 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:49.899+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:47:50.439 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:50 smithi174 podman[72075]: 2024-04-26 18:47:50.25870106 +0000 UTC m=+5.605605812 container died ae9212cbe92e6cadb02d583633f4b5df6528a30759d5ce7925bb506e2bf7c1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-26T18:47:50.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:50 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:50.096+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:50.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:50 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:50.096+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:50.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:50 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:50.096+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:50.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:50 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:50.096+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:50.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:50 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:50.096+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:47:50.921 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:50 smithi174 podman[72075]: 2024-04-26 18:47:50.646271959 +0000 UTC m=+5.993176693 container cleanup ae9212cbe92e6cadb02d583633f4b5df6528a30759d5ce7925bb506e2bf7c1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:47:50.921 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:50 smithi174 bash[72075]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8 2024-04-26T18:47:51.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:50 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:50.768+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:51.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:50 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:50.768+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:51.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:50 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:50.768+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:51.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:50 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:50.768+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:51.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:50 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:50.768+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:51.189 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:50 smithi174 podman[72092]: 2024-04-26 18:47:50.922259637 +0000 UTC m=+0.666192200 container remove ae9212cbe92e6cadb02d583633f4b5df6528a30759d5ce7925bb506e2bf7c1a7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:47:51.189 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:51 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:51.092+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:51.189 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:51 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:51.092+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:51.189 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:51 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:51.092+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:51.189 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:51 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:51.092+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:51.189 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:51 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:51.092+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:47:51.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:50 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:50.883+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:51.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:50 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:50.883+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:51.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:50 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:50.883+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:51.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:50 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:50.883+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:51.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:50 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:50.883+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:51.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:50 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:50.883+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:47:51.525 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:51 smithi174 podman[72178]: 2024-04-26 18:47:51.16548553 +0000 UTC m=+0.017690274 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:47:51.525 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:51 smithi174 podman[72178]: 2024-04-26 18:47:51.289818978 +0000 UTC m=+0.142023712 container create 3b2619549339241f0aba3cdca0f8bf350e785b08134cf47901cc74715813db58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T18:47:51.525 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:51 smithi174 podman[72178]: 2024-04-26 18:47:51.525695725 +0000 UTC m=+0.377900469 container init 3b2619549339241f0aba3cdca0f8bf350e785b08134cf47901cc74715813db58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8-deactivate, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:47:51.826 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:51 smithi174 podman[72178]: 2024-04-26 18:47:51.529218316 +0000 UTC m=+0.381423053 container start 3b2619549339241f0aba3cdca0f8bf350e785b08134cf47901cc74715813db58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:47:51.826 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:51 smithi174 podman[72178]: 2024-04-26 18:47:51.701154883 +0000 UTC m=+0.553359626 container attach 3b2619549339241f0aba3cdca0f8bf350e785b08134cf47901cc74715813db58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True) 2024-04-26T18:47:52.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:51 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:51.726+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:52.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:51 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:51.727+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:52.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:51 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:51.727+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:52.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:51 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:51.727+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:52.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:51 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:51.727+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:52.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:51 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:51.727+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:25.304775+0000 front 2024-04-26T18:47:25.304680+0000 (oldest deadline 2024-04-26T18:47:51.204678+0000) 2024-04-26T18:47:52.107 INFO:journalctl@ceph.osd.8.smithi174.stdout:Apr 26 18:47:51 smithi174 podman[72178]: 2024-04-26 18:47:51.827604213 +0000 UTC m=+0.679808954 container died 3b2619549339241f0aba3cdca0f8bf350e785b08134cf47901cc74715813db58 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T18:47:52.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:51 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:51.910+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:52.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:51 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:51.910+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:52.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:51 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:51.910+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:52.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:51 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:51.910+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:52.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:51 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:51.910+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:52.377 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:51 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:51.910+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:47:52.378 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:52 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:52.107+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:52.378 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:52 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:52.107+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:52.379 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:52 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:52.107+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:52.379 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:52 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:52.107+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:52.379 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:52 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:52.107+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:47:52.398 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.8.service' 2024-04-26T18:47:52.436 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:47:52.436 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-26T18:47:52.436 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-26T18:47:52.436 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.9 2024-04-26T18:47:52.594 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:52 smithi084 systemd[1]: Stopping Ceph osd.9 for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:47:53.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:52 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:52.679+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:53.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:52 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:52.679+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:53.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:52 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:52.679+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:53.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:52 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:52.679+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:53.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:52 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:52.679+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:53.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:52 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:52.679+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:25.304775+0000 front 2024-04-26T18:47:25.304680+0000 (oldest deadline 2024-04-26T18:47:51.204678+0000) 2024-04-26T18:47:53.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:52 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:52.740+0000 7f356a1fd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:47:53.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:52 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:52.740+0000 7f356a1fd700 -1 osd.9 123 *** Got signal Terminated *** 2024-04-26T18:47:53.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:52 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:52.740+0000 7f356a1fd700 -1 osd.9 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:47:53.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:52 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:52.863+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:53.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:52 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:52.863+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:53.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:52 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:52.863+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:53.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:52 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:52.863+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:53.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:52 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:52.863+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:53.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:52 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:52.863+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:47:53.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:53 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:53.095+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:53.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:53 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:53.095+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:53.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:53 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:53.095+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:53.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:53 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:53.095+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:53.440 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:53 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:53.095+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:47:54.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:53 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:53.666+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:54.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:53 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:53.666+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:54.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:53 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:53.666+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:54.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:53 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:53.666+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:54.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:53 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:53.666+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:54.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:53 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:53.666+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:25.304775+0000 front 2024-04-26T18:47:25.304680+0000 (oldest deadline 2024-04-26T18:47:51.204678+0000) 2024-04-26T18:47:54.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:53 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:53.845+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:54.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:53 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:53.845+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:54.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:53 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:53.845+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:54.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:53 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:53.845+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:54.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:53 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:53.845+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:54.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:53 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:53.845+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:47:54.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:54 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:54.107+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:54.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:54 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:54.107+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:54.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:54 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:54.107+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:54.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:54 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:54.107+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:54.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:54 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:54.107+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:47:55.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:54 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:54.632+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:55.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:54 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:54.632+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:55.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:54 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:54.632+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:55.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:54 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:54.632+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:55.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:54 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:54.632+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:55.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:54 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:54.632+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:25.304775+0000 front 2024-04-26T18:47:25.304680+0000 (oldest deadline 2024-04-26T18:47:51.204678+0000) 2024-04-26T18:47:55.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:54 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:54.856+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:55.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:54 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:54.856+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:55.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:54 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:54.856+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:55.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:54 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:54.856+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:55.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:54 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:54.856+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:55.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:54 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:54.856+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:47:55.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:55 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:55.072+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:55.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:55 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:55.072+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:55.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:55 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:55.072+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:55.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:55 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:55.072+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:55.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:55 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:55.072+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:47:55.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:55 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:55.072+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:47:56.094 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:55 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:55.639+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:56.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:55 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:55.639+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:56.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:55 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:55.639+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:56.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:55 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:55.639+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:56.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:55 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:55.639+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:56.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:55 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:55.639+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:25.304775+0000 front 2024-04-26T18:47:25.304680+0000 (oldest deadline 2024-04-26T18:47:51.204678+0000) 2024-04-26T18:47:56.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:55 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:55.639+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:31.205182+0000 front 2024-04-26T18:47:31.205223+0000 (oldest deadline 2024-04-26T18:47:54.705260+0000) 2024-04-26T18:47:56.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:55 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:55.836+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:56.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:55 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:55.836+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:56.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:55 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:55.836+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:56.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:55 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:55.836+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:56.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:55 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:55.836+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:56.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:55 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:55.836+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:47:56.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:56 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:56.109+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:56.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:56 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:56.109+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:56.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:56 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:56.109+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:56.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:56 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:56.109+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:56.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:56 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:56.109+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:47:56.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:56 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:56.109+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:47:56.845 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:56 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:56.592+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:56.845 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:56 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:56.592+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:56.845 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:56 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:56.592+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:56.845 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:56 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:56.592+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:56.845 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:56 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:56.592+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:56.845 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:56 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:56.592+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:25.304775+0000 front 2024-04-26T18:47:25.304680+0000 (oldest deadline 2024-04-26T18:47:51.204678+0000) 2024-04-26T18:47:56.845 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:56 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:56.592+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:31.205182+0000 front 2024-04-26T18:47:31.205223+0000 (oldest deadline 2024-04-26T18:47:54.705260+0000) 2024-04-26T18:47:57.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:56 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:56.801+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:57.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:56 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:56.801+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:57.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:56 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:56.801+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:57.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:56 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:56.801+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:57.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:56 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:56.801+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:57.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:56 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:56.801+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:47:57.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:57 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:57.148+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:57.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:57 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:57.148+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:57.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:57 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:57.148+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:57.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:57 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:57.148+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:57.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:57 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:57.148+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:47:57.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:57 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:57.148+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:47:58.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:57.633+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:42.696861+0000 front 2024-04-26T18:46:42.696880+0000 (oldest deadline 2024-04-26T18:47:06.197260+0000) 2024-04-26T18:47:58.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:57.633+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:52.599167+0000 front 2024-04-26T18:46:52.599196+0000 (oldest deadline 2024-04-26T18:47:16.699313+0000) 2024-04-26T18:47:58.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:57.633+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.800592+0000 front 2024-04-26T18:47:00.800536+0000 (oldest deadline 2024-04-26T18:47:23.100705+0000) 2024-04-26T18:47:58.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:57.633+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:06.601772+0000 front 2024-04-26T18:47:06.601927+0000 (oldest deadline 2024-04-26T18:47:31.802586+0000) 2024-04-26T18:47:58.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:57.633+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.903714+0000 front 2024-04-26T18:47:15.903613+0000 (oldest deadline 2024-04-26T18:47:41.203627+0000) 2024-04-26T18:47:58.095 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:57.633+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:25.304775+0000 front 2024-04-26T18:47:25.304680+0000 (oldest deadline 2024-04-26T18:47:51.204678+0000) 2024-04-26T18:47:58.096 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:57 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9[64304]: 2024-04-26T18:47:57.633+0000 7f35627e9700 -1 osd.9 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:31.205182+0000 front 2024-04-26T18:47:31.205223+0000 (oldest deadline 2024-04-26T18:47:54.705260+0000) 2024-04-26T18:47:58.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:57 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:57.807+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:58.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:57 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:57.807+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:58.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:57 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:57.807+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:58.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:57 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:57.807+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:58.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:57 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:57.807+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:58.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:57 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:57.807+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:47:58.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:57 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:57.807+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:31.633472+0000 front 2024-04-26T18:47:31.633436+0000 (oldest deadline 2024-04-26T18:47:56.933233+0000) 2024-04-26T18:47:58.399 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:58 smithi084 podman[110128]: 2024-04-26 18:47:58.135999977 +0000 UTC m=+5.512040568 container died 1a7fd3385d64a0e79854435a9ac6c6d0902401ac70b4b30b0da588d50e72ce57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T18:47:58.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:58 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:58.155+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:58.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:58 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:58.155+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:58.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:58 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:58.155+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:58.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:58 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:58.155+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:58.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:58 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:58.155+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:47:58.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:58 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:58.155+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:47:58.748 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:58 smithi084 podman[110128]: 2024-04-26 18:47:58.498634342 +0000 UTC m=+5.874674932 container cleanup 1a7fd3385d64a0e79854435a9ac6c6d0902401ac70b4b30b0da588d50e72ce57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-26T18:47:58.749 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:58 smithi084 bash[110128]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9 2024-04-26T18:47:59.074 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:58 smithi084 podman[110145]: 2024-04-26 18:47:58.749586713 +0000 UTC m=+0.610252879 container remove 1a7fd3385d64a0e79854435a9ac6c6d0902401ac70b4b30b0da588d50e72ce57 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T18:47:59.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:58 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:58.784+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:47:59.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:58 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:58.784+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:47:59.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:58 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:58.784+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:47:59.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:58 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:58.784+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:47:59.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:58 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:58.784+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:47:59.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:58 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:58.784+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:47:59.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:58 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:58.784+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:31.633472+0000 front 2024-04-26T18:47:31.633436+0000 (oldest deadline 2024-04-26T18:47:56.933233+0000) 2024-04-26T18:47:59.345 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:59 smithi084 podman[110238]: 2024-04-26 18:47:58.978356401 +0000 UTC m=+0.018164055 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:47:59.345 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:59 smithi084 podman[110238]: 2024-04-26 18:47:59.083734754 +0000 UTC m=+0.123542397 container create 0211e2f04a5875db6eacea5abcbf5498492d72b7d9c5afb17609ed84b85698ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux ) 2024-04-26T18:47:59.345 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:59 smithi084 podman[110238]: 2024-04-26 18:47:59.302780652 +0000 UTC m=+0.342588306 container init 0211e2f04a5875db6eacea5abcbf5498492d72b7d9c5afb17609ed84b85698ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2) 2024-04-26T18:47:59.345 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:59 smithi084 podman[110238]: 2024-04-26 18:47:59.305861481 +0000 UTC m=+0.345669131 container start 0211e2f04a5875db6eacea5abcbf5498492d72b7d9c5afb17609ed84b85698ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9-deactivate, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:47:59.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:59 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:59.163+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:47:59.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:59 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:59.163+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:47:59.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:59 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:59.163+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:47:59.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:59 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:59.163+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:47:59.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:59 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:59.163+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:47:59.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:47:59 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:47:59.163+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:47:59.845 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:59 smithi084 podman[110238]: 2024-04-26 18:47:59.386101795 +0000 UTC m=+0.425909449 container attach 0211e2f04a5875db6eacea5abcbf5498492d72b7d9c5afb17609ed84b85698ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T18:47:59.845 INFO:journalctl@ceph.osd.9.smithi084.stdout:Apr 26 18:47:59 smithi084 podman[110238]: 2024-04-26 18:47:59.596728804 +0000 UTC m=+0.636536447 container died 0211e2f04a5875db6eacea5abcbf5498492d72b7d9c5afb17609ed84b85698ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T18:48:00.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:59 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:59.771+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:48:00.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:59 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:59.771+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:48:00.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:59 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:59.771+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:48:00.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:59 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:59.771+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:48:00.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:59 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:59.771+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:48:00.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:59 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:59.771+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:48:00.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:47:59 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:47:59.771+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:31.633472+0000 front 2024-04-26T18:47:31.633436+0000 (oldest deadline 2024-04-26T18:47:56.933233+0000) 2024-04-26T18:48:00.159 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.9.service' 2024-04-26T18:48:00.196 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:00.197 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-26T18:48:00.197 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-26T18:48:00.197 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.10 2024-04-26T18:48:00.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:00 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:00.185+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:00.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:00 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:00.185+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:00.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:00 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:00.185+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:00.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:00 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:00.185+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:00.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:00 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:00.185+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:00.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:00 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:00.185+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:00.499 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:00 smithi103 systemd[1]: Stopping Ceph osd.10 for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:00.499 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:00 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:00.499+0000 7faa8f8d4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:00.499 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:00 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:00.499+0000 7faa8f8d4700 -1 osd.10 123 *** Got signal Terminated *** 2024-04-26T18:48:00.499 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:00 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:00.499+0000 7faa8f8d4700 -1 osd.10 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:48:01.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:00 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:00.794+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:48:01.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:00 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:00.794+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:48:01.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:00 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:00.794+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:48:01.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:00 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:00.794+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:48:01.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:00 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:00.794+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:48:01.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:00 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:00.794+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:48:01.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:00 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:00.794+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:31.633472+0000 front 2024-04-26T18:47:31.633436+0000 (oldest deadline 2024-04-26T18:47:56.933233+0000) 2024-04-26T18:48:01.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:01 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:01.153+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:01.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:01 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:01.153+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:01.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:01 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:01.153+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:01.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:01 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:01.153+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:01.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:01 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:01.153+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:01.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:01 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:01.153+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:02.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:01 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:01.826+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:48:02.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:01 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:01.826+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:48:02.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:01 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:01.826+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:48:02.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:01 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:01.826+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:48:02.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:01 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:01.826+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:48:02.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:01 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:01.826+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:48:02.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:01 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:01.826+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:31.633472+0000 front 2024-04-26T18:47:31.633436+0000 (oldest deadline 2024-04-26T18:47:56.933233+0000) 2024-04-26T18:48:02.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:02 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:02.154+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:02.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:02 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:02.154+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:02.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:02 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:02.154+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:02.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:02 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:02.154+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:02.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:02 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:02.154+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:02.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:02 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:02.154+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:03.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:02 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:02.789+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:48:03.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:02 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:02.789+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:48:03.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:02 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:02.789+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:48:03.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:02 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:02.789+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:48:03.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:02 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:02.789+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:48:03.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:02 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:02.789+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:48:03.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:02 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:02.789+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:31.633472+0000 front 2024-04-26T18:47:31.633436+0000 (oldest deadline 2024-04-26T18:47:56.933233+0000) 2024-04-26T18:48:03.689 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:03 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:03.189+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:03.775 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:03 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:03.189+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:03.775 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:03 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:03.189+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:03.775 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:03 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:03.189+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:03.775 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:03 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:03.189+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:03.775 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:03 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:03.189+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:04.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:03 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:03.808+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:48:04.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:03 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:03.808+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:48:04.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:03 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:03.808+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:48:04.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:03 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:03.808+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:48:04.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:03 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:03.808+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:48:04.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:03 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:03.808+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:48:04.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:03 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:03.808+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:31.633472+0000 front 2024-04-26T18:47:31.633436+0000 (oldest deadline 2024-04-26T18:47:56.933233+0000) 2024-04-26T18:48:04.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:04 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:04.178+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:04.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:04 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:04.178+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:04.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:04 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:04.178+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:04.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:04 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:04.178+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:04.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:04 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:04.178+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:04.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:04 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:04.178+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:05.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:04 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:04.771+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:45.125372+0000 front 2024-04-26T18:46:45.125360+0000 (oldest deadline 2024-04-26T18:47:08.025256+0000) 2024-04-26T18:48:05.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:04 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:04.771+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:51.427116+0000 front 2024-04-26T18:46:51.427097+0000 (oldest deadline 2024-04-26T18:47:16.727246+0000) 2024-04-26T18:48:05.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:04 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:04.771+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:47:00.228167+0000 front 2024-04-26T18:47:00.228297+0000 (oldest deadline 2024-04-26T18:47:25.528174+0000) 2024-04-26T18:48:05.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:04 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:04.771+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6814 osd.3 since back 2024-04-26T18:47:09.629308+0000 front 2024-04-26T18:47:09.629383+0000 (oldest deadline 2024-04-26T18:47:30.729273+0000) 2024-04-26T18:48:05.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:04 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:04.771+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:15.231098+0000 front 2024-04-26T18:47:15.231145+0000 (oldest deadline 2024-04-26T18:47:38.731272+0000) 2024-04-26T18:48:05.127 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:04 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:04.772+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:22.832254+0000 front 2024-04-26T18:47:22.832182+0000 (oldest deadline 2024-04-26T18:47:46.932140+0000) 2024-04-26T18:48:05.128 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:04 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10[57710]: 2024-04-26T18:48:04.772+0000 7faa87ec0700 -1 osd.10 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:31.633472+0000 front 2024-04-26T18:47:31.633436+0000 (oldest deadline 2024-04-26T18:47:56.933233+0000) 2024-04-26T18:48:05.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:05 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:05.142+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:05.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:05 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:05.142+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:05.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:05 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:05.142+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:05.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:05 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:05.142+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:05.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:05 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:05.142+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:05.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:05 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:05.142+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:06.233 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:05 smithi103 podman[71010]: 2024-04-26 18:48:05.969927109 +0000 UTC m=+5.582865373 container died e17472705489e3ba6b7709c1f101868ac33ef8f445e82194e2e94102ec59df61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T18:48:06.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:06 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:06.101+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:06.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:06 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:06.101+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:06.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:06 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:06.101+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:06.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:06 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:06.101+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:06.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:06 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:06.101+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:06.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:06 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:06.101+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:06.627 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:06 smithi103 podman[71010]: 2024-04-26 18:48:06.299531112 +0000 UTC m=+5.912469387 container cleanup e17472705489e3ba6b7709c1f101868ac33ef8f445e82194e2e94102ec59df61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0) 2024-04-26T18:48:06.627 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:06 smithi103 bash[71010]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10 2024-04-26T18:48:06.627 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:06 smithi103 podman[71025]: 2024-04-26 18:48:06.47600584 +0000 UTC m=+0.505817390 container remove e17472705489e3ba6b7709c1f101868ac33ef8f445e82194e2e94102ec59df61 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True) 2024-04-26T18:48:07.237 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:06 smithi103 podman[71113]: 2024-04-26 18:48:06.796903815 +0000 UTC m=+0.017925706 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:48:07.237 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:06 smithi103 podman[71113]: 2024-04-26 18:48:06.901625962 +0000 UTC m=+0.122647848 container create f45d91796de4085aa4db918d26914a57505b487443930bfad352310786b9ac22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T18:48:07.237 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:07 smithi103 podman[71113]: 2024-04-26 18:48:07.121930333 +0000 UTC m=+0.342952221 container init f45d91796de4085aa4db918d26914a57505b487443930bfad352310786b9ac22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True) 2024-04-26T18:48:07.237 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:07 smithi103 podman[71113]: 2024-04-26 18:48:07.125461859 +0000 UTC m=+0.346483748 container start f45d91796de4085aa4db918d26914a57505b487443930bfad352310786b9ac22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:48:07.237 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:07 smithi103 podman[71113]: 2024-04-26 18:48:07.237775347 +0000 UTC m=+0.458797235 container attach f45d91796de4085aa4db918d26914a57505b487443930bfad352310786b9ac22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-04-26T18:48:07.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:07 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:07.119+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:07.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:07 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:07.119+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:07.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:07 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:07.119+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:07.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:07 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:07.119+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:07.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:07 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:07.119+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:07.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:07 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:07.119+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:07.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:07 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:07.119+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-04-26T18:47:40.495230+0000 front 2024-04-26T18:47:40.495199+0000 (oldest deadline 2024-04-26T18:48:06.395116+0000) 2024-04-26T18:48:07.627 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:07 smithi103 podman[71113]: 2024-04-26 18:48:07.427857962 +0000 UTC m=+0.648879850 container died f45d91796de4085aa4db918d26914a57505b487443930bfad352310786b9ac22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:48:07.959 INFO:journalctl@ceph.osd.10.smithi103.stdout:Apr 26 18:48:07 smithi103 podman[71138]: 2024-04-26 18:48:07.941518876 +0000 UTC m=+0.508613461 container remove f45d91796de4085aa4db918d26914a57505b487443930bfad352310786b9ac22 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-10-deactivate, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T18:48:07.965 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.10.service' 2024-04-26T18:48:08.002 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:08.002 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-26T18:48:08.002 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-26T18:48:08.002 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.11 2024-04-26T18:48:08.331 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:08 smithi174 systemd[1]: Stopping Ceph osd.11 for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:08.331 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:08 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:08.092+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:08.331 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:08 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:08.092+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:08.331 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:08 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:08.092+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:08.331 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:08 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:08.092+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:08.331 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:08 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:08.092+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:08.331 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:08 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:08.092+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:08.331 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:08 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:08.092+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-04-26T18:47:40.495230+0000 front 2024-04-26T18:47:40.495199+0000 (oldest deadline 2024-04-26T18:48:06.395116+0000) 2024-04-26T18:48:08.331 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:08 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:08.330+0000 7f3306c3a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:08.331 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:08 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:08.330+0000 7f3306c3a700 -1 osd.11 123 *** Got signal Terminated *** 2024-04-26T18:48:08.332 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:08 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:08.330+0000 7f3306c3a700 -1 osd.11 123 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T18:48:09.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:09 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:09.136+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:09.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:09 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:09.136+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:09.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:09 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:09.136+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:09.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:09 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:09.136+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:09.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:09 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:09.136+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:09.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:09 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:09.136+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:09.440 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:09 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:09.136+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-04-26T18:47:40.495230+0000 front 2024-04-26T18:47:40.495199+0000 (oldest deadline 2024-04-26T18:48:06.395116+0000) 2024-04-26T18:48:10.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:10 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:10.156+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:10.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:10 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:10.156+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:10.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:10 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:10.156+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:10.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:10 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:10.156+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:10.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:10 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:10.156+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:10.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:10 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:10.156+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:10.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:10 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:10.156+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-04-26T18:47:40.495230+0000 front 2024-04-26T18:47:40.495199+0000 (oldest deadline 2024-04-26T18:48:06.395116+0000) 2024-04-26T18:48:11.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:11 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:11.147+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:11.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:11 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:11.148+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:11.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:11 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:11.148+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:11.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:11 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:11.148+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:11.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:11 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:11.148+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:11.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:11 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:11.148+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:11.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:11 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:11.148+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-04-26T18:47:40.495230+0000 front 2024-04-26T18:47:40.495199+0000 (oldest deadline 2024-04-26T18:48:06.395116+0000) 2024-04-26T18:48:11.440 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:11 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:11.148+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6820 osd.8 since back 2024-04-26T18:47:46.395637+0000 front 2024-04-26T18:47:46.395673+0000 (oldest deadline 2024-04-26T18:48:10.495615+0000) 2024-04-26T18:48:12.394 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:12 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:12.131+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:12.394 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:12 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:12.131+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:12.394 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:12 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:12.131+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:12.394 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:12 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:12.131+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:12.394 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:12 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:12.131+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:12.394 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:12 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:12.131+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:12.394 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:12 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:12.131+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-04-26T18:47:40.495230+0000 front 2024-04-26T18:47:40.495199+0000 (oldest deadline 2024-04-26T18:48:06.395116+0000) 2024-04-26T18:48:12.394 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:12 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:12.131+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6820 osd.8 since back 2024-04-26T18:47:46.395637+0000 front 2024-04-26T18:47:46.395673+0000 (oldest deadline 2024-04-26T18:48:10.495615+0000) 2024-04-26T18:48:13.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:13 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:13.120+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6806 osd.0 since back 2024-04-26T18:46:44.586474+0000 front 2024-04-26T18:46:44.586438+0000 (oldest deadline 2024-04-26T18:47:06.286443+0000) 2024-04-26T18:48:13.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:13 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:13.120+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6804 osd.1 since back 2024-04-26T18:46:49.787764+0000 front 2024-04-26T18:46:49.787837+0000 (oldest deadline 2024-04-26T18:47:15.087669+0000) 2024-04-26T18:48:13.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:13 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:13.120+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6804 osd.2 since back 2024-04-26T18:46:59.089403+0000 front 2024-04-26T18:46:59.089368+0000 (oldest deadline 2024-04-26T18:47:24.989464+0000) 2024-04-26T18:48:13.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:13 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:13.120+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6812 osd.4 since back 2024-04-26T18:47:17.291341+0000 front 2024-04-26T18:47:17.291465+0000 (oldest deadline 2024-04-26T18:47:38.991327+0000) 2024-04-26T18:48:13.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:13 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:13.120+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6812 osd.5 since back 2024-04-26T18:47:23.692163+0000 front 2024-04-26T18:47:23.692200+0000 (oldest deadline 2024-04-26T18:47:49.592360+0000) 2024-04-26T18:48:13.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:13 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:13.120+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.84:6822 osd.6 since back 2024-04-26T18:47:29.593027+0000 front 2024-04-26T18:47:29.592884+0000 (oldest deadline 2024-04-26T18:47:54.892782+0000) 2024-04-26T18:48:13.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:13 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:13.120+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.103:6820 osd.7 since back 2024-04-26T18:47:40.495230+0000 front 2024-04-26T18:47:40.495199+0000 (oldest deadline 2024-04-26T18:48:06.395116+0000) 2024-04-26T18:48:13.439 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:13 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11[58112]: 2024-04-26T18:48:13.120+0000 7f32ff226700 -1 osd.11 123 heartbeat_check: no reply from 172.21.15.174:6820 osd.8 since back 2024-04-26T18:47:46.395637+0000 front 2024-04-26T18:47:46.395673+0000 (oldest deadline 2024-04-26T18:48:10.495615+0000) 2024-04-26T18:48:14.015 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:13 smithi174 podman[72294]: 2024-04-26 18:48:13.757134575 +0000 UTC m=+5.557451115 container died 24f8d78d28c6867e19a82bf0b0e26f36baffd66c624a83cab1bfc95e6ad10d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:48:14.306 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:14 smithi174 podman[72294]: 2024-04-26 18:48:14.105825846 +0000 UTC m=+5.906142380 container cleanup 24f8d78d28c6867e19a82bf0b0e26f36baffd66c624a83cab1bfc95e6ad10d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:48:14.306 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:14 smithi174 bash[72294]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11 2024-04-26T18:48:14.689 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:14 smithi174 podman[72311]: 2024-04-26 18:48:14.306821193 +0000 UTC m=+0.545330194 container remove 24f8d78d28c6867e19a82bf0b0e26f36baffd66c624a83cab1bfc95e6ad10d0e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T18:48:14.992 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:14 smithi174 podman[72400]: 2024-04-26 18:48:14.619817059 +0000 UTC m=+0.017845247 image pull 3a6bde214c5b0b19fc991f4bf5457f79505becbb4a5f8542663f01cefd0dd555 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc 2024-04-26T18:48:14.992 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:14 smithi174 podman[72400]: 2024-04-26 18:48:14.740965646 +0000 UTC m=+0.138993896 container create 4093f20fc6c2c660d9faf775dd660d05cc19c98c132c978de51459651d4879b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T18:48:14.992 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:14 smithi174 podman[72400]: 2024-04-26 18:48:14.993208222 +0000 UTC m=+0.391236410 container init 4093f20fc6c2c660d9faf775dd660d05cc19c98c132c978de51459651d4879b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:48:15.283 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:14 smithi174 podman[72400]: 2024-04-26 18:48:14.996341392 +0000 UTC m=+0.394369584 container start 4093f20fc6c2c660d9faf775dd660d05cc19c98c132c978de51459651d4879b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T18:48:15.283 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:15 smithi174 podman[72400]: 2024-04-26 18:48:15.059699423 +0000 UTC m=+0.457727612 container attach 4093f20fc6c2c660d9faf775dd660d05cc19c98c132c978de51459651d4879b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T18:48:15.577 INFO:journalctl@ceph.osd.11.smithi174.stdout:Apr 26 18:48:15 smithi174 podman[72400]: 2024-04-26 18:48:15.284300378 +0000 UTC m=+0.682328570 container died 4093f20fc6c2c660d9faf775dd660d05cc19c98c132c978de51459651d4879b5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240424, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T18:48:15.873 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@osd.11.service' 2024-04-26T18:48:15.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:15.909 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-26T18:48:15.909 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-26T18:48:15.909 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.a 2024-04-26T18:48:16.206 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:48:15 smithi084 systemd[1]: Stopping Ceph mds.a for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:16.594 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:48:16 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-a[71360]: 2024-04-26T18:48:16.205+0000 7f9f3ea4d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:16.595 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:48:16 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-a[71360]: 2024-04-26T18:48:16.205+0000 7f9f3ea4d700 -1 mds.a *** got signal Terminated *** 2024-04-26T18:48:18.381 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:48:18 smithi084 podman[110342]: 2024-04-26 18:48:18.123557974 +0000 UTC m=+2.018825110 container died c63b0eed8f432deedcba9d732b8030e62123b46738f6e8be1052a0a5d75a6b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True) 2024-04-26T18:48:18.773 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:48:18 smithi084 podman[110342]: 2024-04-26 18:48:18.480742078 +0000 UTC m=+2.376009202 container cleanup c63b0eed8f432deedcba9d732b8030e62123b46738f6e8be1052a0a5d75a6b9f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T18:48:18.773 INFO:journalctl@ceph.mds.a.smithi084.stdout:Apr 26 18:48:18 smithi084 bash[110342]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-a 2024-04-26T18:48:18.896 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.a.service' 2024-04-26T18:48:18.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:18.932 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-26T18:48:18.932 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-26T18:48:18.932 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.d 2024-04-26T18:48:19.095 INFO:journalctl@ceph.mds.d.smithi084.stdout:Apr 26 18:48:19 smithi084 systemd[1]: Stopping Ceph mds.d for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:19.594 INFO:journalctl@ceph.mds.d.smithi084.stdout:Apr 26 18:48:19 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-d[70095]: 2024-04-26T18:48:19.259+0000 7f74f05ac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:19.595 INFO:journalctl@ceph.mds.d.smithi084.stdout:Apr 26 18:48:19 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-d[70095]: 2024-04-26T18:48:19.259+0000 7f74f05ac700 -1 mds.d *** got signal Terminated *** 2024-04-26T18:48:20.951 INFO:journalctl@ceph.mds.d.smithi084.stdout:Apr 26 18:48:20 smithi084 podman[110478]: 2024-04-26 18:48:20.693749618 +0000 UTC m=+1.533470483 container died 8acd3260a4977d4f41a00ec9fc5c022e521d10220fc603ae69f7d2c76221b9fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-d, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:48:21.318 INFO:journalctl@ceph.mds.d.smithi084.stdout:Apr 26 18:48:21 smithi084 podman[110478]: 2024-04-26 18:48:21.042792206 +0000 UTC m=+1.882513081 container cleanup 8acd3260a4977d4f41a00ec9fc5c022e521d10220fc603ae69f7d2c76221b9fd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T18:48:21.318 INFO:journalctl@ceph.mds.d.smithi084.stdout:Apr 26 18:48:21 smithi084 bash[110478]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-d 2024-04-26T18:48:21.467 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.d.service' 2024-04-26T18:48:21.499 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:21.499 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-26T18:48:21.499 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-26T18:48:21.499 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.g 2024-04-26T18:48:21.844 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:48:21 smithi084 systemd[1]: Stopping Ceph mds.g for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:21.845 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:48:21 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-g[70923]: 2024-04-26T18:48:21.812+0000 7f5600cdb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:21.845 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:48:21 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-g[70923]: 2024-04-26T18:48:21.812+0000 7f5600cdb700 -1 mds.g *** got signal Terminated *** 2024-04-26T18:48:26.095 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:48:25 smithi084 podman[110597]: 2024-04-26 18:48:25.826843402 +0000 UTC m=+4.112955299 container died da32549d9cfa71f1624dc86f11ae8f8e15c83faba0e682a8d32010ef3c491c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-g, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True) 2024-04-26T18:48:26.479 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:48:26 smithi084 podman[110597]: 2024-04-26 18:48:26.245906081 +0000 UTC m=+4.532017978 container cleanup da32549d9cfa71f1624dc86f11ae8f8e15c83faba0e682a8d32010ef3c491c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-04-26T18:48:26.480 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:48:26 smithi084 bash[110597]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-g 2024-04-26T18:48:26.715 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.g.service' 2024-04-26T18:48:26.735 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:48:26 smithi084 podman[110611]: 2024-04-26 18:48:26.480322166 +0000 UTC m=+0.654742865 container remove da32549d9cfa71f1624dc86f11ae8f8e15c83faba0e682a8d32010ef3c491c53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-g, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T18:48:26.735 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:48:26 smithi084 systemd[1]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.g.service: Deactivated successfully. 2024-04-26T18:48:26.735 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:48:26 smithi084 systemd[1]: Stopped Ceph mds.g for df3f3816-03f8-11ef-bc93-c7b262605968. 2024-04-26T18:48:26.735 INFO:journalctl@ceph.mds.g.smithi084.stdout:Apr 26 18:48:26 smithi084 systemd[1]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.g.service: Consumed 2.798s CPU time. 2024-04-26T18:48:26.745 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:26.746 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-26T18:48:26.746 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-26T18:48:26.746 DEBUG:teuthology.orchestra.run.smithi084:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.j 2024-04-26T18:48:27.057 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:48:26 smithi084 systemd[1]: Stopping Ceph mds.j for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:27.344 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:48:27 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-j[69631]: 2024-04-26T18:48:27.057+0000 7f31bc41c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:27.344 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:48:27 smithi084 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-j[69631]: 2024-04-26T18:48:27.057+0000 7f31bc41c700 -1 mds.j *** got signal Terminated *** 2024-04-26T18:48:28.241 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:48:28 smithi084 podman[110719]: 2024-04-26 18:48:28.030433169 +0000 UTC m=+1.066316495 container died b83ce86a90444b2d6fbd498754d36f8333d83294b82e17add50cbef7accae60d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-j, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T18:48:28.595 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:48:28 smithi084 podman[110719]: 2024-04-26 18:48:28.441670063 +0000 UTC m=+1.477553387 container cleanup b83ce86a90444b2d6fbd498754d36f8333d83294b82e17add50cbef7accae60d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-j, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:48:28.595 INFO:journalctl@ceph.mds.j.smithi084.stdout:Apr 26 18:48:28 smithi084 bash[110719]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-j 2024-04-26T18:48:28.766 DEBUG:teuthology.orchestra.run.smithi084:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.j.service' 2024-04-26T18:48:28.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:28.797 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-26T18:48:28.797 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-26T18:48:28.797 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.b 2024-04-26T18:48:29.101 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:48:28 smithi103 systemd[1]: Stopping Ceph mds.b for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:29.377 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:48:29 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b[62948]: 2024-04-26T18:48:29.100+0000 7fa73dc6f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:29.377 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:48:29 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b[62948]: 2024-04-26T18:48:29.100+0000 7fa73dc6f700 -1 mds.b *** got signal Terminated *** 2024-04-26T18:48:30.377 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:48:30 smithi103 podman[71218]: 2024-04-26 18:48:30.12550882 +0000 UTC m=+1.133395676 container stop 9008ada52e454062cd95881beb9020cb7f4d83344d5808c58d540a7ce938c682 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T18:48:30.377 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:48:30 smithi103 podman[71218]: 2024-04-26 18:48:30.151113254 +0000 UTC m=+1.159000109 container died 9008ada52e454062cd95881beb9020cb7f4d83344d5808c58d540a7ce938c682 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-26T18:48:30.877 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:48:30 smithi103 podman[71218]: 2024-04-26 18:48:30.509692175 +0000 UTC m=+1.517579025 container cleanup 9008ada52e454062cd95881beb9020cb7f4d83344d5808c58d540a7ce938c682 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-04-26T18:48:30.877 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:48:30 smithi103 bash[71218]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b 2024-04-26T18:48:30.877 INFO:journalctl@ceph.mds.b.smithi103.stdout:Apr 26 18:48:30 smithi103 podman[71232]: 2024-04-26 18:48:30.677585762 +0000 UTC m=+0.545757280 container remove 9008ada52e454062cd95881beb9020cb7f4d83344d5808c58d540a7ce938c682 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-26T18:48:30.936 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.b.service' 2024-04-26T18:48:30.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:30.969 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-26T18:48:30.969 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-26T18:48:30.969 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.e 2024-04-26T18:48:31.280 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:48:31 smithi103 systemd[1]: Stopping Ceph mds.e for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:31.627 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:48:31 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-e[62324]: 2024-04-26T18:48:31.279+0000 7fb2cd143700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:31.627 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:48:31 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-e[62324]: 2024-04-26T18:48:31.279+0000 7fb2cd143700 -1 mds.e *** got signal Terminated *** 2024-04-26T18:48:34.396 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:48:34 smithi103 podman[71341]: 2024-04-26 18:48:34.130101969 +0000 UTC m=+2.946920944 container died 3c7b2ab98ba066d990a2bdbadbc134c3c99c06012ed494397a37547652d3cadf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T18:48:34.396 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:48:34 smithi103 podman[71341]: 2024-04-26 18:48:34.396997898 +0000 UTC m=+3.213816858 container cleanup 3c7b2ab98ba066d990a2bdbadbc134c3c99c06012ed494397a37547652d3cadf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-04-26T18:48:34.739 INFO:journalctl@ceph.mds.e.smithi103.stdout:Apr 26 18:48:34 smithi103 bash[71341]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-e 2024-04-26T18:48:34.873 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.e.service' 2024-04-26T18:48:34.904 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:34.904 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-26T18:48:34.904 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-26T18:48:34.904 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.h 2024-04-26T18:48:35.377 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:48:34 smithi103 systemd[1]: Stopping Ceph mds.h for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:35.377 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:48:35 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-h[62637]: 2024-04-26T18:48:35.234+0000 7f384b5bd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:35.377 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:48:35 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-h[62637]: 2024-04-26T18:48:35.234+0000 7f384b5bd700 -1 mds.h *** got signal Terminated *** 2024-04-26T18:48:38.377 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:48:38 smithi103 podman[71464]: 2024-04-26 18:48:38.122832879 +0000 UTC m=+2.996615228 container died 90fc70c84d828cce78dc134db9ed647d6e1aba71ff25f01282908fa3064cbefc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-h, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-fixes-bc94a2a) 2024-04-26T18:48:38.635 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:48:38 smithi103 podman[71464]: 2024-04-26 18:48:38.468002288 +0000 UTC m=+3.341784649 container cleanup 90fc70c84d828cce78dc134db9ed647d6e1aba71ff25f01282908fa3064cbefc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-h, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:48:38.635 INFO:journalctl@ceph.mds.h.smithi103.stdout:Apr 26 18:48:38 smithi103 bash[71464]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-h 2024-04-26T18:48:38.841 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.h.service' 2024-04-26T18:48:38.871 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:38.871 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-26T18:48:38.871 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-26T18:48:38.871 DEBUG:teuthology.orchestra.run.smithi103:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.k 2024-04-26T18:48:39.196 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:48:38 smithi103 systemd[1]: Stopping Ceph mds.k for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:39.627 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:48:39 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-k[62003]: 2024-04-26T18:48:39.197+0000 7fa0fc94c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:39.627 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:48:39 smithi103 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-k[62003]: 2024-04-26T18:48:39.197+0000 7fa0fc94c700 -1 mds.k *** got signal Terminated *** 2024-04-26T18:48:42.378 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:48:42 smithi103 podman[71584]: 2024-04-26 18:48:42.122973296 +0000 UTC m=+3.030147692 container died 4956ca44d39c545c1c1b0f98bc070f025a279995979044e537c849b641055143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-k, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:48:42.647 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:48:42 smithi103 podman[71584]: 2024-04-26 18:48:42.455956719 +0000 UTC m=+3.363131185 container cleanup 4956ca44d39c545c1c1b0f98bc070f025a279995979044e537c849b641055143 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-k, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-26T18:48:42.648 INFO:journalctl@ceph.mds.k.smithi103.stdout:Apr 26 18:48:42 smithi103 bash[71584]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-k 2024-04-26T18:48:42.855 DEBUG:teuthology.orchestra.run.smithi103:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.k.service' 2024-04-26T18:48:42.889 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:42.889 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-26T18:48:42.889 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-26T18:48:42.889 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.c 2024-04-26T18:48:43.179 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:48:42 smithi174 systemd[1]: Stopping Ceph mds.c for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:43.439 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:48:43 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-c[63169]: 2024-04-26T18:48:43.178+0000 7fcea03a4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:43.439 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:48:43 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-c[63169]: 2024-04-26T18:48:43.178+0000 7fcea03a4700 -1 mds.c *** got signal Terminated *** 2024-04-26T18:48:46.379 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:48:46 smithi174 podman[72506]: 2024-04-26 18:48:46.124064743 +0000 UTC m=+3.044085902 container died 8cc2cd012148bb407de99061ab9e65f4c5dbab60803d776a5331f38123f5ba10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-c, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-04-26T18:48:46.689 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:48:46 smithi174 podman[72506]: 2024-04-26 18:48:46.478418449 +0000 UTC m=+3.398439614 container cleanup 8cc2cd012148bb407de99061ab9e65f4c5dbab60803d776a5331f38123f5ba10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-c, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T18:48:46.689 INFO:journalctl@ceph.mds.c.smithi174.stdout:Apr 26 18:48:46 smithi174 bash[72506]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-c 2024-04-26T18:48:46.947 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.c.service' 2024-04-26T18:48:46.982 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:46.982 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-26T18:48:46.982 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-26T18:48:46.982 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.f 2024-04-26T18:48:47.298 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:48:47 smithi174 systemd[1]: Stopping Ceph mds.f for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:47.688 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:48:47 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-f[62856]: 2024-04-26T18:48:47.298+0000 7f9fc3e9d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:47.689 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:48:47 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-f[62856]: 2024-04-26T18:48:47.298+0000 7f9fc3e9d700 -1 mds.f *** got signal Terminated *** 2024-04-26T18:48:50.406 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:48:50 smithi174 podman[72640]: 2024-04-26 18:48:50.126648115 +0000 UTC m=+2.922264233 container died 7321510a55a7f5378f681906036bbb4f64f1a0ba0d304054e023699c86626ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-f, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T18:48:50.689 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:48:50 smithi174 podman[72640]: 2024-04-26 18:48:50.489772006 +0000 UTC m=+3.285388126 container cleanup 7321510a55a7f5378f681906036bbb4f64f1a0ba0d304054e023699c86626ccf (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T18:48:50.689 INFO:journalctl@ceph.mds.f.smithi174.stdout:Apr 26 18:48:50 smithi174 bash[72640]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-f 2024-04-26T18:48:50.919 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.f.service' 2024-04-26T18:48:50.950 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:50.951 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-26T18:48:50.951 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-26T18:48:50.951 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.i 2024-04-26T18:48:51.268 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:48:51 smithi174 systemd[1]: Stopping Ceph mds.i for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:51.689 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:48:51 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-i[63475]: 2024-04-26T18:48:51.268+0000 7f4bb467c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:51.689 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:48:51 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-i[63475]: 2024-04-26T18:48:51.268+0000 7f4bb467c700 -1 mds.i *** got signal Terminated *** 2024-04-26T18:48:54.189 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:48:54 smithi174 podman[72760]: 2024-04-26 18:48:54.1265848 +0000 UTC m=+2.966266977 container died d8edfcd179c3056e081287296792787080c7e3796bff22d99b284be159eb0ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T18:48:54.689 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:48:54 smithi174 podman[72760]: 2024-04-26 18:48:54.476419874 +0000 UTC m=+3.316102049 container cleanup d8edfcd179c3056e081287296792787080c7e3796bff22d99b284be159eb0ea9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-i, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-04-26T18:48:54.689 INFO:journalctl@ceph.mds.i.smithi174.stdout:Apr 26 18:48:54 smithi174 bash[72760]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-i 2024-04-26T18:48:54.947 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.i.service' 2024-04-26T18:48:54.977 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:54.977 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-26T18:48:54.978 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-26T18:48:54.978 DEBUG:teuthology.orchestra.run.smithi174:> sudo systemctl stop ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.l 2024-04-26T18:48:55.439 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:48:55 smithi174 systemd[1]: Stopping Ceph mds.l for df3f3816-03f8-11ef-bc93-c7b262605968... 2024-04-26T18:48:55.439 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:48:55 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:48:55.296+0000 7f2dd9eac700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T18:48:55.439 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:48:55 smithi174 ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-l[62537]: 2024-04-26T18:48:55.296+0000 7f2dd9eac700 -1 mds.l *** got signal Terminated *** 2024-04-26T18:48:58.421 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:48:58 smithi174 podman[72879]: 2024-04-26 18:48:58.130962824 +0000 UTC m=+2.932983123 container died e004f7cad9de9d5f9353b50f5c20d286b70783aec9123ba63d1cd0c7d622b743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-l, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True) 2024-04-26T18:48:58.689 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:48:58 smithi174 podman[72879]: 2024-04-26 18:48:58.504705214 +0000 UTC m=+3.306725515 container cleanup e004f7cad9de9d5f9353b50f5c20d286b70783aec9123ba63d1cd0c7d622b743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-26T18:48:58.689 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:48:58 smithi174 bash[72879]: ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-l 2024-04-26T18:48:58.689 INFO:journalctl@ceph.mds.l.smithi174.stdout:Apr 26 18:48:58 smithi174 podman[72893]: 2024-04-26 18:48:58.647214089 +0000 UTC m=+0.517702057 container remove e004f7cad9de9d5f9353b50f5c20d286b70783aec9123ba63d1cd0c7d622b743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:2aadaf96fcd676c1b4c3580ed8dd1b2a67fa40f49767a9535ff3eae823efe0dc, name=ceph-df3f3816-03f8-11ef-bc93-c7b262605968-mds-l, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, ceph=True, RELEASE=wip-lusov-quiescer-fixes-bc94a2a, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T18:48:58.885 DEBUG:teuthology.orchestra.run.smithi174:> sudo pkill -f 'journalctl -f -n 0 -u ceph-df3f3816-03f8-11ef-bc93-c7b262605968@mds.l.service' 2024-04-26T18:48:58.915 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T18:48:58.915 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-26T18:48:58.915 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid df3f3816-03f8-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T18:48:59.106 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:49:01.574 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid df3f3816-03f8-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T18:49:01.766 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:49:04.060 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid df3f3816-03f8-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T18:49:04.247 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:49:06.597 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:49:06.628 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:49:06.658 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T18:49:06.683 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T18:49:06.685 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917/remote/smithi084/crash 2024-04-26T18:49:06.686 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/crash -- . 2024-04-26T18:49:06.725 INFO:teuthology.orchestra.run.smithi084.stderr:tar: /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T18:49:06.725 INFO:teuthology.orchestra.run.smithi084.stderr:tar: Error is not recoverable: exiting now 2024-04-26T18:49:06.727 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917/remote/smithi103/crash 2024-04-26T18:49:06.728 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/crash -- . 2024-04-26T18:49:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:tar: /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T18:49:06.807 INFO:teuthology.orchestra.run.smithi103.stderr:tar: Error is not recoverable: exiting now 2024-04-26T18:49:06.810 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917/remote/smithi174/crash 2024-04-26T18:49:06.810 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/crash -- . 2024-04-26T18:49:06.846 INFO:teuthology.orchestra.run.smithi174.stderr:tar: /var/lib/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T18:49:06.847 INFO:teuthology.orchestra.run.smithi174.stderr:tar: Error is not recoverable: exiting now 2024-04-26T18:49:06.848 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T18:49:06.848 DEBUG:teuthology.orchestra.run.smithi084:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-04-26T18:49:06.889 INFO:tasks.cephadm:Compressing logs... 2024-04-26T18:49:06.889 DEBUG:teuthology.orchestra.run.smithi084:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T18:49:06.931 DEBUG:teuthology.orchestra.run.smithi103:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T18:49:06.934 DEBUG:teuthology.orchestra.run.smithi174:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T18:49:06.954 INFO:teuthology.orchestra.run.smithi084.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T18:49:06.954 INFO:teuthology.orchestra.run.smithi084.stderr:: No such file or directory 2024-04-26T18:49:06.954 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.log 2024-04-26T18:49:06.956 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T18:49:06.956 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.g.log 2024-04-26T18:49:06.956 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.j.log 2024-04-26T18:49:06.956 INFO:teuthology.orchestra.run.smithi084.stderr:gzip/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.g.log: -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T18:49:06.956 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T18:49:06.960 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mgr.x.log 2024-04-26T18:49:06.960 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mon.a.log 2024-04-26T18:49:06.960 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mgr.x.log: 90.4% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T18:49:06.960 INFO:teuthology.orchestra.run.smithi084.stderr:gzip 87.8% -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.6.log 2024-04-26T18:49:06.961 INFO:teuthology.orchestra.run.smithi084.stderr: -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T18:49:06.964 INFO:teuthology.orchestra.run.smithi174.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T18:49:06.965 INFO:teuthology.orchestra.run.smithi174.stderr:: No such file or directory 2024-04-26T18:49:06.965 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.log 2024-04-26T18:49:06.965 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T18:49:06.965 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.f.log 2024-04-26T18:49:06.965 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T18:49:06.965 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T18:49:06.965 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.f.log: /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.c.log 2024-04-26T18:49:06.965 INFO:teuthology.orchestra.run.smithi174.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T18:49:06.965 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mon.c.log 2024-04-26T18:49:06.965 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mgr.z.log 2024-04-26T18:49:06.965 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.i.log 2024-04-26T18:49:06.965 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mgr.z.log: 87.8% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T18:49:06.966 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T18:49:06.966 INFO:teuthology.orchestra.run.smithi174.stderr: 90.5%/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.i.log: -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T18:49:06.971 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mon.a.log: 94.6% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-26T18:49:06.971 INFO:teuthology.orchestra.run.smithi084.stderr: 94.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T18:49:06.971 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T18:49:06.971 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.a.log 2024-04-26T18:49:06.972 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T18:49:06.972 INFO:teuthology.orchestra.run.smithi084.stderr: 84.2% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T18:49:06.972 INFO:teuthology.orchestra.run.smithi103.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T18:49:06.972 INFO:teuthology.orchestra.run.smithi103.stderr:: No such file or directory 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.log 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.7.log 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.b.log 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- 92.1% /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.4.log -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mgr.y.log 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.h.log 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.10.log 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr: 90.5% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr: 87.8% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.e.log 2024-04-26T18:49:06.973 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T18:49:06.974 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T18:49:06.976 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.e.log: 92.3% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-26T18:49:06.976 INFO:teuthology.orchestra.run.smithi103.stderr:gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mon.b.log 2024-04-26T18:49:06.977 INFO:teuthology.orchestra.run.smithi174.stderr: 92.4% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-26T18:49:06.977 INFO:teuthology.orchestra.run.smithi174.stderr:gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.11.log 2024-04-26T18:49:06.977 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.8.log 2024-04-26T18:49:06.977 INFO:teuthology.orchestra.run.smithi174.stderr: 83.9% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T18:49:06.977 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T18:49:06.982 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.l.log 2024-04-26T18:49:06.986 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.d.log 2024-04-26T18:49:06.987 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.k.log 2024-04-26T18:49:06.989 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T18:49:07.001 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T18:49:07.002 INFO:teuthology.orchestra.run.smithi084.stderr:gzip -5 --verbose -- /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.9.log 2024-04-26T18:49:07.008 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-26T18:49:07.017 INFO:teuthology.orchestra.run.smithi174.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-volume.log: /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T18:49:07.036 INFO:teuthology.orchestra.run.smithi084.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.9.log: 94.7% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-26T18:49:07.064 INFO:teuthology.orchestra.run.smithi084.stderr: 94.7% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-26T18:49:07.094 INFO:teuthology.orchestra.run.smithi174.stderr: 93.3% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-26T18:49:07.103 INFO:teuthology.orchestra.run.smithi103.stderr:/var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.k.log: 93.2% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-26T18:49:07.137 INFO:teuthology.orchestra.run.smithi174.stderr: 94.7% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-26T18:49:07.143 INFO:teuthology.orchestra.run.smithi103.stderr: 94.7% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-26T18:49:07.146 INFO:teuthology.orchestra.run.smithi174.stderr: 93.2% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-26T18:49:07.153 INFO:teuthology.orchestra.run.smithi103.stderr: 93.3% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-26T18:49:07.292 INFO:teuthology.orchestra.run.smithi174.stderr: 93.8% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-26T18:49:07.447 INFO:teuthology.orchestra.run.smithi103.stderr: 92.1% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-26T18:49:07.451 INFO:teuthology.orchestra.run.smithi174.stderr: 92.3% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-26T18:49:07.451 INFO:teuthology.orchestra.run.smithi084.stderr: 90.3% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-26T18:49:08.224 INFO:teuthology.orchestra.run.smithi084.stderr: 90.3% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-26T18:49:08.378 INFO:teuthology.orchestra.run.smithi084.stderr: 92.2% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T18:49:08.467 INFO:teuthology.orchestra.run.smithi174.stderr: 92.8% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-26T18:49:08.493 INFO:teuthology.orchestra.run.smithi084.stderr: 92.0% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-26T18:49:08.501 INFO:teuthology.orchestra.run.smithi174.stderr: 91.9% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T18:49:08.664 INFO:teuthology.orchestra.run.smithi103.stderr: 91.5% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T18:49:08.726 INFO:teuthology.orchestra.run.smithi174.stderr: 92.6% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-26T18:49:08.818 INFO:teuthology.orchestra.run.smithi084.stderr: 92.9% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T18:49:08.971 INFO:teuthology.orchestra.run.smithi084.stderr: 92.3% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-26T18:49:08.973 INFO:teuthology.orchestra.run.smithi084.stderr: 2024-04-26T18:49:08.973 INFO:teuthology.orchestra.run.smithi084.stderr:real 0m2.029s 2024-04-26T18:49:08.973 INFO:teuthology.orchestra.run.smithi084.stderr:user 0m8.445s 2024-04-26T18:49:08.973 INFO:teuthology.orchestra.run.smithi084.stderr:sys 0m0.324s 2024-04-26T18:49:09.022 INFO:teuthology.orchestra.run.smithi103.stderr: 92.2% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T18:49:09.085 INFO:teuthology.orchestra.run.smithi103.stderr: 93.0% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-26T18:49:09.345 INFO:teuthology.orchestra.run.smithi103.stderr: 92.5% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-26T18:49:09.460 INFO:teuthology.orchestra.run.smithi103.stderr: 92.3% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-26T18:49:09.462 INFO:teuthology.orchestra.run.smithi103.stderr: 2024-04-26T18:49:09.462 INFO:teuthology.orchestra.run.smithi103.stderr:real 0m2.513s 2024-04-26T18:49:09.462 INFO:teuthology.orchestra.run.smithi103.stderr:user 0m11.241s 2024-04-26T18:49:09.462 INFO:teuthology.orchestra.run.smithi103.stderr:sys 0m0.394s 2024-04-26T18:49:09.540 INFO:teuthology.orchestra.run.smithi174.stderr: 92.1% -- replaced with /var/log/ceph/df3f3816-03f8-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T18:49:09.543 INFO:teuthology.orchestra.run.smithi174.stderr: 2024-04-26T18:49:09.543 INFO:teuthology.orchestra.run.smithi174.stderr:real 0m2.595s 2024-04-26T18:49:09.543 INFO:teuthology.orchestra.run.smithi174.stderr:user 0m8.286s 2024-04-26T18:49:09.543 INFO:teuthology.orchestra.run.smithi174.stderr:sys 0m0.304s 2024-04-26T18:49:09.543 INFO:tasks.cephadm:Archiving logs... 2024-04-26T18:49:09.543 DEBUG:teuthology.misc:Transferring archived files from smithi084:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917/remote/smithi084/log 2024-04-26T18:49:09.544 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T18:49:10.719 DEBUG:teuthology.misc:Transferring archived files from smithi103:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917/remote/smithi103/log 2024-04-26T18:49:10.719 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T18:49:12.290 DEBUG:teuthology.misc:Transferring archived files from smithi174:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917/remote/smithi174/log 2024-04-26T18:49:12.291 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T18:49:13.510 INFO:tasks.cephadm:Removing cluster... 2024-04-26T18:49:13.510 DEBUG:teuthology.orchestra.run.smithi084:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid df3f3816-03f8-11ef-bc93-c7b262605968 --force 2024-04-26T18:49:13.698 INFO:teuthology.orchestra.run.smithi084.stdout:Deleting cluster with fsid: df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:49:13.967 DEBUG:teuthology.orchestra.run.smithi103:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid df3f3816-03f8-11ef-bc93-c7b262605968 --force 2024-04-26T18:49:14.159 INFO:teuthology.orchestra.run.smithi103.stdout:Deleting cluster with fsid: df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:49:14.475 DEBUG:teuthology.orchestra.run.smithi174:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid df3f3816-03f8-11ef-bc93-c7b262605968 --force 2024-04-26T18:49:14.674 INFO:teuthology.orchestra.run.smithi174.stdout:Deleting cluster with fsid: df3f3816-03f8-11ef-bc93-c7b262605968 2024-04-26T18:49:14.960 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T18:49:14.961 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T18:49:14.980 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T18:49:14.996 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T18:49:15.013 INFO:tasks.cephadm:Teardown complete 2024-04-26T18:49:15.013 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T18:49:15.027 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T18:49:15.028 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T18:49:15.030 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T18:49:15.039 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T18:49:15.139 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T18:49:15.150 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T18:49:15.150 DEBUG:teuthology.orchestra.run.smithi084:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T18:49:15.153 DEBUG:teuthology.orchestra.run.smithi103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T18:49:15.155 DEBUG:teuthology.orchestra.run.smithi174:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T18:49:15.167 INFO:teuthology.orchestra.run.smithi084.stderr:bash: line 1: ntpq: command not found 2024-04-26T18:49:15.167 INFO:teuthology.orchestra.run.smithi103.stderr:bash: line 1: ntpq: command not found 2024-04-26T18:49:15.169 INFO:teuthology.orchestra.run.smithi174.stderr:bash: line 1: ntpq: command not found 2024-04-26T18:49:15.172 INFO:teuthology.orchestra.run.smithi103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T18:49:15.172 INFO:teuthology.orchestra.run.smithi103.stdout:=============================================================================== 2024-04-26T18:49:15.172 INFO:teuthology.orchestra.run.smithi103.stdout:^* hv01.front.sepia.ceph.com 3 7 377 95 +768us[ +771us] +/- 44ms 2024-04-26T18:49:15.172 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 30 -604us[ -604us] +/- 62ms 2024-04-26T18:49:15.172 INFO:teuthology.orchestra.run.smithi103.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 -673us[ -673us] +/- 90ms 2024-04-26T18:49:15.172 INFO:teuthology.orchestra.run.smithi103.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:49:15.174 INFO:teuthology.orchestra.run.smithi174.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T18:49:15.174 INFO:teuthology.orchestra.run.smithi174.stdout:=============================================================================== 2024-04-26T18:49:15.174 INFO:teuthology.orchestra.run.smithi174.stdout:^* hv01.front.sepia.ceph.com 3 6 377 30 +756us[ +753us] +/- 44ms 2024-04-26T18:49:15.174 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 32 -603us[ -606us] +/- 62ms 2024-04-26T18:49:15.174 INFO:teuthology.orchestra.run.smithi174.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 31 -679us[ -682us] +/- 90ms 2024-04-26T18:49:15.174 INFO:teuthology.orchestra.run.smithi174.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:49:15.174 INFO:teuthology.orchestra.run.smithi084.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T18:49:15.174 INFO:teuthology.orchestra.run.smithi084.stdout:=============================================================================== 2024-04-26T18:49:15.174 INFO:teuthology.orchestra.run.smithi084.stdout:^* hv01.front.sepia.ceph.com 3 6 377 34 +760us[ +756us] +/- 44ms 2024-04-26T18:49:15.175 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 30 -608us[ -608us] +/- 62ms 2024-04-26T18:49:15.175 INFO:teuthology.orchestra.run.smithi084.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 33 -673us[ -673us] +/- 90ms 2024-04-26T18:49:15.175 INFO:teuthology.orchestra.run.smithi084.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T18:49:15.175 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T18:49:15.193 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T18:49:15.193 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T18:49:15.271 DEBUG:teuthology.orchestra.run.smithi084:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T18:49:15.353 DEBUG:teuthology.orchestra.run.smithi103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T18:49:15.420 DEBUG:teuthology.orchestra.run.smithi174:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T18:49:15.485 DEBUG:teuthology.orchestra.run.smithi084:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T18:49:15.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:49:15.515 DEBUG:teuthology.orchestra.run.smithi103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T18:49:15.541 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:49:15.542 DEBUG:teuthology.orchestra.run.smithi174:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T18:49:15.567 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:49:15.568 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T18:49:15.580 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T18:49:15.618 INFO:teuthology.task.internal:Duration was 2420.196303 seconds 2024-04-26T18:49:15.618 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T18:49:15.680 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T18:49:15.680 DEBUG:teuthology.orchestra.run.smithi084:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T18:49:15.683 DEBUG:teuthology.orchestra.run.smithi103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T18:49:15.685 DEBUG:teuthology.orchestra.run.smithi174:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T18:49:15.720 INFO:teuthology.orchestra.run.smithi103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T18:49:15.722 INFO:teuthology.orchestra.run.smithi174.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T18:49:15.722 INFO:teuthology.orchestra.run.smithi084.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T18:49:16.229 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T18:49:16.229 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi084.front.sepia.ceph.com 2024-04-26T18:49:16.229 DEBUG:teuthology.orchestra.run.smithi084:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T18:49:16.248 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi103.front.sepia.ceph.com 2024-04-26T18:49:16.249 DEBUG:teuthology.orchestra.run.smithi103:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T18:49:16.275 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi174.front.sepia.ceph.com 2024-04-26T18:49:16.275 DEBUG:teuthology.orchestra.run.smithi174:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T18:49:16.297 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T18:49:16.297 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T18:49:16.300 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T18:49:16.319 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T18:49:16.362 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T18:49:16.362 DEBUG:teuthology.orchestra.run.smithi084:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T18:49:16.364 DEBUG:teuthology.orchestra.run.smithi103:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T18:49:16.384 DEBUG:teuthology.orchestra.run.smithi174:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T18:49:17.177 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T18:49:17.190 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T18:49:17.191 DEBUG:teuthology.orchestra.run.smithi084:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T18:49:17.214 DEBUG:teuthology.orchestra.run.smithi103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T18:49:17.239 DEBUG:teuthology.orchestra.run.smithi174:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T18:49:17.263 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T18:49:17.274 DEBUG:teuthology.orchestra.run.smithi084:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T18:49:17.276 DEBUG:teuthology.orchestra.run.smithi103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T18:49:17.282 DEBUG:teuthology.orchestra.run.smithi174:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T18:49:17.296 INFO:teuthology.orchestra.run.smithi084.stdout:kernel.core_pattern = core 2024-04-26T18:49:17.301 INFO:teuthology.orchestra.run.smithi103.stdout:kernel.core_pattern = core 2024-04-26T18:49:17.324 INFO:teuthology.orchestra.run.smithi174.stdout:kernel.core_pattern = core 2024-04-26T18:49:17.351 DEBUG:teuthology.orchestra.run.smithi084:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T18:49:17.367 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:49:17.368 DEBUG:teuthology.orchestra.run.smithi103:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T18:49:17.398 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:49:17.399 DEBUG:teuthology.orchestra.run.smithi174:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T18:49:17.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T18:49:17.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T18:49:17.425 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T18:49:17.425 DEBUG:teuthology.misc:Transferring archived files from smithi084:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917/remote/smithi084 2024-04-26T18:49:17.426 DEBUG:teuthology.orchestra.run.smithi084:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T18:49:17.478 DEBUG:teuthology.misc:Transferring archived files from smithi103:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917/remote/smithi103 2024-04-26T18:49:17.478 DEBUG:teuthology.orchestra.run.smithi103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T18:49:17.513 DEBUG:teuthology.misc:Transferring archived files from smithi174:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-26_17:24:23-fs-wip-lusov-quiescer-fixes-distro-default-smithi/7674917/remote/smithi174 2024-04-26T18:49:17.513 DEBUG:teuthology.orchestra.run.smithi174:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T18:49:17.549 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T18:49:17.550 DEBUG:teuthology.orchestra.run.smithi084:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T18:49:17.552 DEBUG:teuthology.orchestra.run.smithi103:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T18:49:17.554 DEBUG:teuthology.orchestra.run.smithi174:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T18:49:17.596 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T18:49:17.610 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T18:49:17.610 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T18:49:17.631 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T18:49:17.631 DEBUG:teuthology.orchestra.run.smithi084:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T18:49:17.634 DEBUG:teuthology.orchestra.run.smithi103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T18:49:17.636 DEBUG:teuthology.orchestra.run.smithi174:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T18:49:17.645 INFO:teuthology.orchestra.run.smithi084.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 18:49 /home/ubuntu/cephtest 2024-04-26T18:49:17.647 INFO:teuthology.orchestra.run.smithi103.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 18:49 /home/ubuntu/cephtest 2024-04-26T18:49:17.650 INFO:teuthology.orchestra.run.smithi174.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 18:49 /home/ubuntu/cephtest 2024-04-26T18:49:17.652 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T18:49:17.691 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T18:49:17.847 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/no} objectstore-ec/bluestore-comp omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2420.1963033676147 flavor: default owner: scheduled_leonidus@teuthology success: true 2024-04-26T18:49:17.847 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T18:49:17.916 INFO:teuthology.run:pass